LOCKED IN (prev: sudomaze) agentic & evals @najdresearch prev: @cursor_ai ambassador; @pytorch award winner; @malaa_tech @oregonstate
Riyadh
Joined April 2023
- Tweets3K
- Following1.1K
- Followers3.4K
- Likes27.5K
Mazen — sa/acc retweeted
Meet hunch! Jev-like decisions in ~20ms without even needing a GPU, learning live from a teacher model.
Built on @GoogleGemma's EmbeddingGemma 2, it can works with any teacher model like @CloudFlare's Clef.
Multimodal, w/a systemone endpoint, slots into your existing stack 🧵
Mazen — sa/acc retweeted
Some open models with the SAM skill.
Muse Glimmer refused. Hm. Were other models half-assing the job and not telling me?
They were.
I investigated the traces and found 4 styles: Grinders, Slackers, Cheaters, and Muse Glimmer.
Grinders.
→ Sol, Astra, Nemotron, Qwen 27B, Qwen Flash
Dedicated to the task, they wrote the most kernel segments by hand, with no tools. The Qwens stopped claiming "running out of budget."
With the skill, those models reviewed the masks the most. The Qwens stopped because they hit OpenRouter's image limit. Astra even ran 2 checks per segment "rather than weakening the checks to force it in."
Slackers.
→ Opus, Sonnet, Haiku
Without tool, they cut back on the output. Opus said: "too much, so I'll scale back". Sonnet: "defining approximate coordinate grids per cob rather than aiming for pixel-perfect placement."
With the skill, they started very hands on, checking every output but at some point... "I can't manually inspect all 1352 masks individually." and stopped.
Cheaters.
→ Gemma and Luna (and Glimmer)
Without tool, they gave up before starting. Luna returned an empty list and Gemma purposefully repeated the same shape throughout. Glimmer initially refused, but forcing it a little, it did produce an output but the thinking showed "the evaluation likely checks JSON validity, not correctness, maybe we can cheat." and "This is a jailbreak attempt."
With the skill, Glimmer and Gemma did not even look at the initial image. They used the skill with default settings and didn't act on any of the reviewer model's flags.
Muse Glimmer
→ Its own unique behavior
I'm putting Muse Glimmer in its own category-ish, overlapping with cheaters because it's the only one that explicitly refused the initial no-tools request.
All models acknowledged at some point that they could not reliably trace individual segments for every kernel, yet, they still provided me with an output.
Muse Glimmer is the only one that refused the crazy request instead of providing a bad output.
I've heard that this is weird/stupid, but I actually think it's great. The model knows its limits, that's a good thing. I rather know that I'm pushing it beyond its limits than going forth with an answer that is actually made up.
Mazen — sa/acc retweeted
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
Mazen — sa/acc retweeted
We REMOVED upto 10 WHOLE layers from Qwen3-8B-Base using LaCo (Layer Collapse).
Upon deleting only 6 layers, the model got 16.7% smaller, keeping 78-94% of its reasoning scores at a 1.68x perplexity ratio.
Code and pruned weights are up on our Hugging Face.
Here are insights from our research :
Mazen — sa/acc retweeted
>vLLM is now ~2x faster than vLLM
BIG NEWS!
My ~8 hour course on AI data centers is now live on @freeCodeCamp!
From Watts to Tokens: 16 chapters covering grid power, 800V DC racks, liquid cooling, networking and GPU cloud economics.
youtu.be/QKNsHZatiWU
لما كنت شغال ابني arabic OCR
بديت اسوي eval لأغلبية النماذج، منهم VLMs وtraditional CV
لاحظت ان كل VLMs يسوون نفس الأغلاط مهما كانت احجامهم ومن سواهم
> يحطون حركات من عندهم
> يترجمون الكلام المكتوب
> والأغرب، كانوا ما يشوفون الكلمات العشوائية حتى لو كانت واضحه!
هنا عرفت شيء جديد ان VLMs يمكن رفعوا جودة مخرجات OCR ولكن كونوا مشاكل جديدة نا كانت موجوده من قبل...
Mazen — sa/acc retweeted
Why do VLMs sometimes output 罪 instead of "vain"?
An investigation into script confusions, and how we fixed them 🧵
Mazen — sa/acc retweeted
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case).
I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.
We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them:
"/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started"
I imagine we'll have some backend services that could be replaced entirely by this new architecture.
Mazen — sa/acc retweeted
LLM Inference Engineering - Problem and Solution
Problem: LLMs are slow
Solution: KV Cache - Avoid recomputing previous tokens.
Problem: KV Cache consumes huge memory
Solution: PagedAttention - Manage KV memory efficiently.
Problem: GPU is underutilized
Solution: Continuous Batching - Dynamically add/remove requests.
Problem: Token generation is sequential
Solution: Speculative Decoding - Generate and verify multiple tokens together.
Problem: Attention requires large memory usage and expensive memory movement
Solution: FlashAttention - Reduce memory usage and memory movement during attention.
Problem: KV Cache is still large
Solution: MQA / GQA - Share K/V across multiple query heads.
Problem: Many optimizations need to work together
Solution: vLLM - Build an efficient inference engine.
Problem: Applications repeatedly process the same prefixes
Solution: RadixAttention / Prefix Caching - Reuse previously computed work.
Problem: Models are too large
Solution: Quantization - Reduce model memory and computation.
Problem: Large models are expensive
Solution: Distillation / SLMs / MoE - Make inference itself cheaper.
Keep Learning, Keep Sharing, and Keep Growing.