Developer Advocate @NebiusAI. Building with open models | coding agents | LLM inference
San Jose, CA, USA
Joined February 2025
- Tweets303
- Following541
- Followers292
- Likes503
going live in 90 mins (10 am PT / 1pm ET / 7pm CET)
dev.to/events/hacktoberfest-…
and twitch.tv/mlh
See you there!
What actually happens when you send a request to an LLM?
Gateway → tokenizer → router → queue → prefill → decode → streamed back to you, one token at a time.
I'm tracing the whole trip tomorrow at Hacktoberfest 2026. ⚡
Thu, Oct 22 online @ 10am PT / 1pm ET
link below 👇
What Happens When You Call a Model API"
👉 dev.to/events/hacktoberfest-…
More Hacktoberfest talks: hacktoberfest.com/schedule/
Friday fun 🎉
The Nemotron family (Super, Lightning, Ultra) went to the carnival… and raced horses 🎪🏇
One of them is a total sprinter. Guess who? 👀
Link below 👇
taking muse-wonder, gpt-astra and claude-opus for a (horse) ride :-)
it was a close race - latency was a big decider
see below 👇
(just a fun test, not a benchmark)
Our talk at @aiDotEngineer World's Fair SF is now live! 🎥
@dylan and I explored "Optimizing Open Models for Production" - the techniques that make LLM inference faster, more efficient, and cheaper at @nebiustf.
We covered a few key optimizations:
- KV caching - Reuse previously computed key-value states instead of recomputing them. One of the biggest performance wins, with 5-10x speedups in some scenarios.
- Cache-aware routing - Route requests to workers that already have the relevant KV cache, improving cache hit rates and reducing redundant computation.
- Speculative decoding - A smaller model drafts tokens while a larger model verifies them, accelerating generation without sacrificing output quality.
- Disaggregated prefill and decoding - Prefill is compute-heavy, while decoding is memory-bandwidth-heavy. Running them on separate GPU pools lets us optimize each independently.
- Quantization - Smaller models, less memory, faster inference. But aggressive quantization can hurt quality, so finding the right balance is key.
Lots of practical techniques for getting more performance out of open models!
🎥 Watch the talk:
youtube.com/watch?v=TRe1u7dH…
📊 Slides:
sujee.dev/events/2026-06-29_…
Friday fun! I built a horse race for LLMs. 🏇
LLM Derby puts models head-to-head on the same questions. Every correct answer moves a horse forward. First to 20 wins.
This race:
🏇 Qwen3.8-27B
🏇 gpt-6-astra
🏇 claude-sonnet-5-5
Who won? Watch the video 👇
Try the game (in your browser): sujee.github.io/llm-derby/
Just a fun game, not a benchmark 😄
Sujee Maniyam retweeted
🎂 I’m turning my dog into a video game for his second birthday!!
Tomorrow on Build With @nebiusai :
@sujee_dev tests AI models then we’ve got 30 minutes to build a game. You vote on the levels!
9 a.m. Pacific · X + YouTube. Let’s go!! 🔥
35 years ago today, Linux 0.02 was released 🐧
I've been using Linux for ~31 of those years.
1995: recompiling kernels to get new hardware working.
2026: Linux Mint + NVIDIA GPU, running LLMs locally.
A lot has changed. Linux is still there.
I made this little visual history of Linux 👇
What’s your favorite Linux memory?
how i made this video.. prompts + stack
xitter.cf/sujee_dev/status/21075…
A few of you asked how I made this Linux history video:
youtube.com/watch?v=lkOxEeqw…
It started with one prompt in Claude Code (Opus 5.5):
"Create visualization tracing Linux history. A 2 min video, with cool music."
That's the whole prompt. No 3-page storyboard. No extra skills, MCP servers or API keys.
~7 minutes later I had a 2-minute video with an original soundtrack. The music is generated in Python, not by a music service.
That first pass got me ~80% there. The rest was notes in the same session, like:
"0:30: keep this sequence longer"
Takeaway: you don't need a complicated setup or a long prompt to make good motion graphics. Start with one sentence and direct from there.
The prompt, the code, and how to remix it yourself: github.com/sujee/visual-stor…
Meet Mistral Large 4, aka Le Chonk.
• 1T parameters, natively multimodal. 49B active.
It is the best open weights model from US or Europe on aggregated benchmarks.
• State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding.
• Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure.
• Available to all via API today. Working with cybersecurity partners privately.
Open weights release end of October.
My Le Chonk 🐈😂
Meet Mistral Large 4, aka Le Chonk.
• 1T parameters, natively multimodal. 49B active.
It is the best open weights model from US or Europe on aggregated benchmarks.
• State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding.
• Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure.
• Available to all via API today. Working with cybersecurity partners privately.
Open weights release end of October.
A few of you asked how I made this Linux history video:
youtube.com/watch?v=lkOxEeqw…
It started with one prompt in Claude Code (Opus 5.5):
"Create visualization tracing Linux history. A 2 min video, with cool music."
That's the whole prompt. No 3-page storyboard. No extra skills, MCP servers or API keys.
~7 minutes later I had a 2-minute video with an original soundtrack. The music is generated in Python, not by a music service.
That first pass got me ~80% there. The rest was notes in the same session, like:
"0:30: keep this sequence longer"
Takeaway: you don't need a complicated setup or a long prompt to make good motion graphics. Start with one sentence and direct from there.
The prompt, the code, and how to remix it yourself: github.com/sujee/visual-stor…