Principal Applied Scientist @ Microsoft 🤖 ex-AMZN Alexa 🔊 📘 Author, Building Speech AI | Agents/NLP/IR/Speech | 🔁 Building agents that build agents
Seattle
Joined May 2009
- Tweets2.4K
- Following381
- Followers5.6K
- Likes394K
Pinned Tweet
The Kindle edition of Building Speech AI book is live.
It runs from spectrograms and MFCCs through wav2vec and Whisper to neural TTS, speaker diarization, audio language models, and the latency engineering behind real-time voice agents.
Runnable Python in every chapter.
#voice agents #speech #audio #audioLM #voice #book
amazon maintains job guidelines and roles explicit in clear documentation. nice to see them refresh these time to time and keep it relevant.
Interesting Amazon putting this out aboutamazon.com/news/company…
RLHF, RLFT, RLVR and recently RLCD (jev or decision model), RLCR (collective rewards) ....
we need more RL paradigms, great to see these swarm or collective behavior studied scientifically.
AI has changed so much around us, but education hasn't absorbed it evenly. Many classrooms still give every student the same linear, standardized path.
At the IEEE STEM Summit 2026 (Oct 22–23), I'll be discussing this through one question: how do voice assistants hear us? 1/5
The result is a spectrogram, the picture many recognizers start from. A whistle is one bright line, a clap a vertical stripe across many frequencies, a hiss high-frequency noise, and vowels show vocal-fold harmonics plus formants that tell "ee" from "ah." 4/5
From those patterns, an acoustic model estimates the speech sounds and a language model picks the likely words.
See your own voice live, in the browser, no login, no audio leaves the page: prdeepakbabu.github.io/stem_…
#IEEESTEMSummit #AIinEducation 5/5
A very interesting book on designing and training neural networks beyond the familiar gradient-descent family of techniques. #book #AI
Our Neuroevolution textbook is finally in print!
Free online edition: neuroevolutionbook.com/
Pre-order: a.co/d/06DpkH9h
I am incredibly grateful to my co-authors Sebastian Risi, Yujin Tang, and Risto Miikkulainen for making this happen. Neuroevolution is a subject very dear to my heart. It is the field that convinced me that nature has already figured out how to build intelligence: through evolution, collective behavior, and adaptation under constraints.
This idea, that intelligence emerges from evolution operating under constraints rather than unlimited resources, is what eventually led me to founding Sakana AI here in Japan.
The concepts in this book about open-ended creativity and self-organizing systems are exactly what we build at @SakanaAILabs. Our name and logo are inspired by schools of fish moving together, adapting as one. It is the core philosophy of what we build. This book captures the theoretical foundations of that belief.
$12.99 on Kindle, free to read on Kindle Unlimited. Paperback is on the same page.
amazon.com/dp/B0HKHV3BCN
Companion code is open either way:
github.com/prdeepakbabu/buil…
Good points. Effectively long tail problems that were once not economically feasible can be tamed now
刚看完阿里巴巴CEO吴泳铭云栖大会这 18 分钟，有一段话我印象特别深，我觉得不管大家怎么看阿里，这 18 分钟值得找个时间安静的看一遍，我做了一版中英文字幕的分享给大家。
他说今天机器思考总量还不到人类的 3%，但终局是 1000 倍，
然后举了个例子：新生儿早衰症，全球可能就几百个患者，按现在的商业逻辑没有公司愿意花几十年去做这个药。
但如果思考的成本趋近于零，上百万个 Agent 可以不眠不休地跑实验——那些因为不赚钱就没人管的病，第一次有了被认真对待的可能。
这段话让我回味了很久，不是因为讲了多牛逼的技术，关键是指向一个我以前没想过的问题，人类社会到底有多少重要的事，仅仅因为投入不划算就被放弃了？
另一个让我觉得很深刻的判断，他说今天的 Vibe Coding 就是 1882 年的电灯，替代的是已有的工作，
原话是：仅仅替代现有工作，创造不了真正的新时代，代表性产品还没出现。
这句话放在整个行业天天喊颠覆的氛围里，我觉得反而是最清醒的。
我们现在做的所有事，可能都只是在用新工具干旧活，
真正属于这个时代的东西，也许还在某个人的脑子里，还没被想出来。
他还有一个类比我很喜欢：1900 年电器已经进了家庭，但当时全球一年的发电量，放到今天只够全世界用两小时。
我们就站在这个位置上，基础设施远远没有铺够，需求远远没有被释放。
so adding to RLHF, RLAIF, RLFT we now have RLCD (RL for calibrated decisions)
A new system one model by Jev uses this architecture treating generate as extreme classification done in parallel threads for ultra fast responses. #Jev #RLCD
The logits from LLMs are unlikely to be calibrated but Jev specifically solves the calibration problem for decision making problems.
closest paper I could find (not official)
arxiv.org/html/2503.02623v6
I think there is enough talk about solving naiver strokes, but little about so what - real world implications.
If we take an extremely favorable case, we should be having more efficient aircrafts, understanding of turbulance, accurate weather predictions, blood-flow modeling etc.. #GPTAstra
recent observations likely linked to agentic coding and poor quality gates ?
@Apple iPhone randomly offloads app that are frequently and recently used ones.
@Google @YouTube when you focus on video tile on your TV, usually previews the video content for a few seconds and if you click the video it usually continues instead of starting from 0:00 like it is doing now.
OpenAI’s upcoming Astra model reportedly uses recurrent-depth reasoning
arxiv.org/abs/2502.05171
Deepak Babu Piskala @ICML 2026 retweeted
To welcome you back to school, it's time for a new release of Speech and Language Processing! Our August 2026 edition finally has a chapter 1! Check it out here: web.stanford.edu/~jurafsky/s…
this is a neat article. it talks about
- treating everything as code helps iterate on agents and decisions as trackable entities.
- turn feedback into learning loop - likes, replies on slack for the agent actions.
- turn physical processes into config files so it can be seamlessly automated.
- managed agents help solve the infra problem while emp can focus on the automation itself.
Anthropic 昨天分享了一个很值得看的企业 Agent 案例。
ABC Legal 有 1100 名员工，他们先让财务、营销、运营等岗位的 15 个非工程人员，用 Claude Code 自己搭 Managed Agent。
一周后，15 个人全部做出了能工作的 Agent。一个月左右，公司已经跑起了 50+ 个生产 Agent。
他们能把这件事扩起来，关键还是在 Harness。
每个 Agent 都用统一模板管理，Prompt、tools、memory、schedule 全部放进 Git。任何修改都走 PR，所以版本、Review、Rollback 和审计都有了。
更有意思的是他们还做了一套自改进循环。
Agent 先干活，人直接在 Slack 里回复或者点 emoji。另一个 Agent 收集这些反馈变成数据，再由 Tuner 定期提出 Prompt 和配置修改，自动开 PR，最后交给人审核。
模型没有重新训练，Agent 也可以靠真实使用中的反馈持续改自己的 Harness。
我觉得这可能确实是企业 Agent 很实际的一条路线。
以后「谁能做 Agent」的门槛会继续降低，难的还是怎么把权限、反馈、评测和版本管理这些基础设施先搭好。
文章
claude.com/blog/how-abc-lega…
Deepak Babu Piskala @ICML 2026 retweeted
Latest Deep RL class lectures are now online!
youtube.com/playlist?list=PL…
Thanks to @seohong_park, we now have CS185/285 for spring 2026 available to everyone to watch.
Course website here: rail.eecs.berkeley.edu/deepr…
Apologies for a few recording glitches (it's not a perfect system).
a nice intro to watermarking tech. basically model providers like Ant can use the AR decode parameters to do tie breaks using a random number generator = f(secret key). simple yet effective. anthropic.com/news/claude-te…
