
Solving conversational intelligence @duplexio
Copenhagen, Denmark
Joined January 2011
- Tweets288
- Following980
- Followers239
- Likes570
Pinned Tweet
After a year of building, I'm finally ready to share what I've been working on.
Duo is a full-duplex speech-to-speech model that you can run on your laptop.
Stay tuned for more!
Anders Thuesen retweeted
Of course that's your contention.
You're a first-year RL engineer.
You just read Prime Intellect's prime-rl writeup and saw rollout workers streaming trajectories into a trainer that never waits on them, so naturally you believe decoupling generation from training is free throughput and staleness is a rounding error.
Next month, when you get your hands on an RL cookbook, the Unsloth GRPO handbook, probably, you'll be singing from the treetops about colocating vLLM inside the training process, and the 400 milliseconds you save on every weight sync by waking and sleeping the optimizer like you have not just moved the idle GPU somewhere else.
That'll last until sometime next year, when you watch a livestream of Will Brown talking about verifiers and become convinced that none of the systems work matters, because an environment is just a Python function and the reward figures itself out.
"Well, as a matter of fact, I won't, because a verifier..."
"A verifier is exactly the thing your policy learns to game first. It finds the one branch you forgot to test and farms it for maximum reward in fifty steps, and your async sampler is already three policy versions stale while it does it, so the importance ratio is fiction by the time the update lands."
Yeah, I read that too. Every training-inference mismatch thread since the sampler's logprobs and the trainer's logprobs first disagreed in BF16 and nobody noticed for 200 steps.
Do you have any thoughts of your own on the subject, or should I just point my agent at the prime-rl repo, set group size to 64, and let it pay an H100 to argue with itself for me?
Anders Thuesen retweeted
A speech-to-speech model need not be full duplex. But full duplex is about more than just generating speech.
Speech-to-speech describes the modality. Full duplex describes the interaction: listening and speaking simultaneously.
Related, but not interchangeable.
The distinction matters more than people think.
Getting good conversational speech data ain't easy.
But the guys from @agenticdataco rose to the challenge and helped gather the early datasets for Duo.
We believe most interaction with AI will happen through speech. But models still talk like machines.
This is a data problem.
Today we're launching @agenticdataco (YC F26): real-world speech data + audio-native RL environments so speech models sound and act human.
21
967
Anders Thuesen retweeted
Today’s ticketing systems weren’t built for agents.
Over the past 3 months, we’ve been building a new product at Parahelp to solve this: Infrastructure purpose-built for your agents (like Claude, Grok Bot, or one built in-house) and team to resolve support tickets together.
Anders Thuesen retweeted
Jensen Huang when asked whether he would rather relive his 20s or be 20 years old today.
Anders Thuesen retweeted
today, The Agentic Data Company is releasing the world’s largest free conversational dataset.
- 1,000 hours English dual-channel
- natural conversations between friends/family
free for commercial and research use. theagenticdatacompany.com/op…
Anders Thuesen retweeted
Treat your body like a data center
Consume as much water and literature as possible
Upset local city council with your existence
Woz and Jobs would definitely have been running qwen 3.8 27b had it dropped back in their days
1
228
Anders Thuesen retweeted
Introducing 💾moi: a generative workspace where agents builds their own UI.
✴ Skill for Claude Code, Codex, Hermes & OpenClaw
✴ Your agent extends the UI with full-stack widgets and pages
✴ Continues to live as copilot in these apps
✴ Build live dashboards, personal software, company OS
✴ Work with the agent together on a shared canvas, apply themes, modify the UI on the fly
The future of software is agentic, personalised and adaptive
Anders Thuesen retweeted
The newest model in the Mamba series is finally here 🐍
Hybrid models have become increasingly popular, raising the importance of designing the next generation of linear models.
We've introduced several SSM-centric ideas to significantly increase Mamba-2's modeling capabilities without compromising on speed. The resulting Mamba-3 model has noticeable performance gains over the most popular previous linear models (such as Mamba-2 and Gated DeltaNet) at all sizes.
This is the first Mamba that was student led: all credit to @aakash_lahoti @kevinyli_ @_berlinchen @caitWW9, and of course @tri_dao!
Anders Thuesen retweeted
Introducing 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔: Rethinking depth-wise aggregation.
Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with learned, input-dependent attention over preceding layers.
🔹 Enables networks to selectively retrieve past representations, naturally mitigating dilution and hidden-state growth.
🔹 Introduces Block AttnRes, partitioning layers into compressed blocks to make cross-layer attention practical at scale.
🔹 Serves as an efficient drop-in replacement, demonstrating a 1.25x compute advantage with negligible (<2%) inference latency overhead.
🔹 Validated on the Kimi Linear architecture (48B total, 3B activated parameters), delivering consistent downstream performance gains.
🔗Full report:
github.com/MoonshotAI/Attent…
Anders Thuesen retweeted
community-contributed Mamba-3-minimal code!
this will be a great supplement to the official code, which should be released very soon 👀
The Mamba-3 architecture is incredible, but the math actively fights against standard PyTorch.
I just open-sourced mamba3-minimal: a pure PyTorch, single-file implementation of the Mamba-3 architecture.👇
github.com/VikramKarLex/mamb…
Anders Thuesen retweeted
soooo... how many papers do we think are invalidated by this? And now think about how many other bugs there must be in any re-implementations of... basically anything.
We identified an issue with the Mamba-2 🐍 initialization in HuggingFace and FlashLinearAttention repository (dt_bias being incorrectly initialized).
This bug is related to 2 main issues:
1. init being incorrect (torch.ones) if Mamba-2 layers are used in isolation without the Mamba2ForCausalLM model class (this has been already fixed: github.com/fla-org/flash-lin…).
2. Skipping initialization due to meta device init for DTensors with FSDP-2 (github.com/fla-org/flash-lin… will fix this issue upon merging).
The difference is substantial. Mamba-2 seems to be quite sensitive to the initialization.
Check out our experiments at the 7B MoE scale: wandb.ai/mayank31398/mamba-t…
Special thanks to @kevinyli_, @bharatrunwal2, @HanGuo97, @tri_dao and @_albertgu 🙏
Also thanks to @SonglinYang4 for quickly helping in merging the PR.





















