@fluidinference , AI Researcher https://xitter.cf/t.co/8ko2FKWp4A

Joined July 2025
a 1,000 followers , time to take a break
8
15
828
pruned 35B to 11.5B supporting english and mandarin on a M5 Pro with MLX. 64 of 256 experts per layer, 22 GB → 8.9 GB. ~0.3 s to first audio, ~2× real time. huggingface.co/alexwengg/dra…
$52M seed round. 15M+ users. But this is what we're most proud of yet. Introducing Drama 3: the most emotive and controllable TTS model in the world And yes, it's open-weight. You can now control how even a single word should sound, change its tone, and shift your voice mid-sentence. Uncle Roger was impressed too. Try it here: s.fish.audio/rqh0l8 Comment “DRAMA” within the next 48 hours for 1 month free. If you're an enterprise your first month is on us: s.fish.audio/qz9r0t
3
25
1,523
What’s funny about LLMs and this took me a while to realize is . They operate similar to computers . Kv caches are like ram or memory , starting off as a way to save recompute cost now as storage. Attention layer is like a cpu instruction loader and the forward feedback layer is like the executioner .
1
121
saw that Underdog-Bark-3.0 already has a coreml encoder. got curious how the full pipeline would look in coreml and some of my own encoder touches. resulting in 1 h of audio in 13 s (276× RTFx) and LibriSpeech test-other 4.96 % WER @ 140× on M5 Pro . hf repo coming soon.
Today we’re releasing Underdog Bark 3.0, the #1 frontier speech model It's a tiny 163 MB download as accurate as NVIDIA’s Parakeet (15x bigger). It beats OpenAI's Whisper large-v3 (57x bigger) It's fast- transcribes 1 min of audio in <0.25s on a Macbook. On recorded meetings (AMI), Bark 3.0 gets 7.82% of words wrong. Parakeet gets 9.42%. OpenAI's Whisper large-v3 gets 13.63% error rate Bark runs on both your Laptop & Phone. It’s free and open source under CC-BY 4.0 Try private on-device meeting notes & dictation, powered by Bark, in Underdog today: underdog.ai/bark
4
4
40
1,632
Phonon 31m now runs on ANE ~20 ms to first audio, ~20x real time on an M5 Pro, 103 voices, and Gradium's text normalizer ported to Swift. code: github.com/FluidInference/Fl… Model: huggingface.co/FluidInferenc… Thanks @GradiumAI and @kyutai_labs for open-sourcing this 🙏
A new frontier for on-device speech. We're open-sourcing Phonon, a suite of TTS models under 50m parameters, setting a new standard for naturalness and accuracy across languages, with the broadest inference support to date. Phonon-31m is available under Apache 2.0. gradium.ai/phonon🧵
1
4
109
4,435
The new 18M / 7M / 2M Pocket TTS models now run on coreml in FluidAudio, open source, with the flow model on ANE. M5 Pro: 18M at 24 rtfx, 2M at 51 rtfx, TTFT ~25 ms after, under 110 MB of memory. macOS 15 / iOS 18. Model: huggingface.co/FluidInferenc… Code: github.com/FluidInference/Fl… Great work @kyutai_labs 🙏
Pocket TTS has been getting updates those last few days. While it was ~100M parameters, we release smaller versions, down to 1.9M: an 18.2M model with the quality of the original, a 6.9M model, and a 1.9M model that runs 103x faster than real time on one CPU core. All with voice cloning.
6
6
1
74
2,650
All in one month
4
256
D3 27B driving a first-person lane runner: every row is one hosted call, the model reads the next three rows as text, each move labelled with its consequence, and picks. ~220 ms per decision, 200+ rows.
Introduce Decision 3.0 Free API 🚀 for developers to learn, research and explore what decision models can do #1 on both Jev Decision Index. Six models live, 0.6B to 27B. Text, images or video in, a probability for every answer out One click, no account. SystemOne API compatible
1
7
375
d3-edge on a Mac, from a parkour head cam 🏃 Every 2 s of footage, 8 plain yes/no questions up high? long drop below? holding on? on steel? answered in ~110 ms on MLX, labels drawn live on the video with some music. Core ML + MLX ports 🤗 huggingface.co/FluidInferenc…
Turn any camera into a stream of decisions. Every second, d3-edge (0.6B) watched this rescue and answered 8 plain-English questions in ~0.1 s. New questions, new scene: pet cam, bird feeder, doorstep. What would you ask? huggingface.co/vllm-sr/d3-ed… #Jev #Decision
3
4
1
16
1,076
looping back to this post, reading it had inspired me to start finetuning and training my models, resulting in over a 1000 followers in less than a month. heres a model that can play Space Invaders. 635K params, 6 typed actions (left / stay / right × fire), 0.23 ms per decision, 100% on the Neural Engine.
5
12
658
A few days ago @XunzhuoLiu reached out with early access to D3, I didn't expect to end it with: - a blog post vllm-sr.ai/blog/decision-3-0… 🥳 - joining 2 Semantic Router workgroups (Router Models & Inference Runtime, DX & Ecosystem) vllm-sr.ai/community/work-gr… 🎉 - @ClementDelangue to repost my d3 coreml post 🤯 Thank you everyone for the trust! To celebrate here's d3, distilled and on Core ML, playing Super Mario on a M5 Pro 🍄
Congratulations to the vllm-sr team on releasing Decision 3.0, a SOTA multimodal Jev model with weights now open-sourced on HF. 🎉🎉🎉
1
4
14
1,082
reading this article has pushed me to actually try training from scratch instead of just calling bigger models. 294K params, plays Pac-Man, 0.15 ms per move, 100% on the Neural Engine. Early concept, but feels like the right shape.
7
2
51
1,716
Update: it's faster now. 4-bit experts plus a pipelined decode loop. The same 14-page paper in 53 s instead of 70 s, at 263 tok/s instead of ~200. Equations and tables now render as it reads. huggingface.co/FluidInferenc…
🚨 CHINA JUST OPEN SOURCED AN OCR THAT EATS 100 PAGE PDFS IN ONE PASS It’s called Unlimited OCR. Only 3B params. Runs locally. Every other OCR tool chops your doc into pages and loses the thread. This one reads the whole thing in one continuous pass. → One shot full context parsing (32K context window) → Multilingual out of the box → 93% on the standard parsing benchmark (6 points over baseline) → Under 0.11 error rate past 40 pages → Runs 100% locally on your own hardware → Works with Transformers, vLLM, SGLang, Docker, Ollama, llama.cpp Traditional cloud OCR (Textract, Google Vision, Azure Doc Intelligence) costs $1.50 to $15 per 1,000 pages. This runs on your machine. For free. Forever. Baidu built it explicitly to push DeepSeek OCR one step further. Already at 1.9M downloads on Hugging Face and most people have no idea it exists yet. 100% open source.
2
10
107
4,910
Inspired by Shahir's demo, FluidAudio Phonon 2 coreml transcribing an hour of audio in 5.6 s and a full 24 hours in 2 min 19 s on a M5 Pro. Phonon-2 Core ML 1 hour rtfx went from 480× → 644×. test-clean wer the same while 150 -> 195 rtfx github.com/FluidInference/Fl… huggingface.co/FluidInferenc…
I turned @googlegemma EmbeddingGemma 2, an embedding model, into a speech-to-text model, and it turned out to be one of the fastest STT models you can run on device. On a base M3 MacBook Air it transcribes an hour of audio in about 13 seconds: 288× real time. The idea came in March with Gemini Embedding 2. It puts text, images, audio and video into one shared space, so a spoken word lands right next to the written word. If that's true, you should be able to read a transcript straight out of the embedding model. I couldnt tried it then, but last week EmbeddingGemma 2 came out as an open model, so I tried it.
6
21
3
288
14,322