Ivan Rocha (@irr) header
Ivan Rocha (@irr) profile picture
Lisbon
Joined March 2009
Photo by Ivan Rocha (@irr)Photo by Ivan Rocha (@irr)Photo by Ivan Rocha (@irr)Photo by Ivan Rocha (@irr)Photo by Ivan Rocha (@irr)Photo by Ivan Rocha (@irr)Photo by Ivan Rocha (@irr)
Ivan Rocha retweeted
Inkling 2 Mini on one DGX Spark, from Thinking Machines' own 4-bit weights with no quantization of mine: 65 tok/s on a long coding answer. In a chat it feels instant, the first token comes after 0.2 s. The right side shows speculative decoding live. A small draft head guesses up to 8 tokens and the model checks them in one pass, so boilerplate code goes through 8 at a time and plain English 1 to 3.
3
18
1,060
Ivan Rocha retweeted
It is becoming f*cking dangerous: your agent can now take any video apart shot by shot, tell you exactly how it was made, and rebuild the whole edit on your own machine - 3 free tools, 28k stars combined rea-video + ffmpeg-skill + pixelfx... 1. Pick a video you want to understand - yours, or any you're allowed to study 2. Install rea-video: tell your agent "install github.com/JuneYaooo/rea-vidโ€ฆ, then break down this video: clip.mp4". it writes VIDEO.md - every shot labeled filmed, AI-generated or coded, with the exact setup to remake it 3. Install ffmpeg-skill: npx ffmpeg-skill setup. now the agent cuts, captions, reframes and exports locally - 42 ffmpeg tools, no cloud, no keys 4. Install pixelfx: npm i github:patch-gg/pixelfx. ask for any of 244 effects, or have it write you a new one 5. Say "rebuild this edit with my footage, same beats" - and it does, shot for shot rea-video tells you how it was made. ffmpeg-skill rebuilds it. pixelfx styles it. my .claude/ folder with all three wired in is in the reply below
11
43
356
14,636
Ivan Rocha retweeted
๐Ÿงช Inkling 2 Mini on ONE DGX Spark at the full 1M context โšก ๐Ÿง  Official NVFP4 + DSpark in vLLM ๐Ÿ“ฆ 3.3M-token KV pool (FP8 KV) ๐ŸŽฏ 950K-token needle test: passed โœ… ๐Ÿš€ ~47 tok/s single user Recipe + launch scripts ๐Ÿ‘‡ github.com/tonyd2wild/Inklinโ€ฆ
12
9
3
155
8,516
Ivan Rocha retweeted
Weโ€™re proud to partner with Thinking Machines Lab to bring Inkling 2 Mini to Llama day-0 ๐Ÿค— Weโ€™re releasing 4-bit and 8-bit versions of the model for your local setups, which pack a ton of intelligence for consumer hardware.
10
16
2
276
8,963
Ivan Rocha retweeted
I highly recommend running this skill on your projects vulnerability-scanner reviews your code for security vulnerabilities, including AI agents and MCP servers What it does: - Maps the attack surface and trust boundaries. - Detects prompt injection and insecure agent tools - Checks for exposed secrets and vulnerable dependencies - Reviews MCP servers, skills, hooks, and CLAUDE.md files - Identifies OWASP vulnerabilities and CI/CD security risks - Verifies findings to reduce false positives - Reports each issue with Attack, Impact, and Fix Some results from testing: - Automatically activated in 7/7 runs - 0 false positives across 7 decoys - Compatible with 6 coding agents IMPORTANT: Itโ€™s read-only and never modifies your code Install:โ€จnpx claude-code-templates@latest --skill security/vulnerability-scanner aitmpl.com/component/skills/โ€ฆ
67
126
5
1,134
60,798
You can now Run locally Uncensored Inkling-2-Mini on 16GB. - it beating Qwen3.8-27B - refusal direction removed from the residual stream. - Same looped architecture. - LoRA adapter + ready GGUFs coming.. - huggingface.co/AtomicChat/Inโ€ฆ
You can now run a 26B model on a laptop that beats much larger models on coding and agentic work. the model crown for its size on coding benchmarks. - Inkling 2 Mini (5.4B active MoE, looped transformer) - supports 1M context, - scores 48.2% on DeepSWE v1.1 - and 77.9% on SWE-bench Pro (Hard) ready for Unsloth + llama.cpp. - huggingface.co/unsloth/Inkliโ€ฆ
17
93
5
1,594
79,644
Ivan Rocha retweeted
How to increase Qwen3.8-27B inference speed from 25 tok/s to 91 tok/s for free on the same GPU? The authors of the inference-optimisation-skill project demonstrated this using the Intel Arc Pro B70 and Qwen3.8-27B as an example: - llama.cpp Q4_K_M: 25 tok/s - vLLM EXL3 4 bpw with custom kernels: 91.2 tok/s They also created an open-source skill that allows an AI agent to independently search for ways to optimize local models. How does it work? - calculates the theoretical hardware performance limit - analyzes what limits the model's speed - changes one parameter and runs a benchmark - Compares the results with control measurements - rejects changes that degrade stability or performance with long context Results on RTX 3090 with Qwen3.8-27B in EXL3 3 bpw and MTP: - Prose: 96.2 tok/s - Code: 141.1 tok/s This is not another inference engine, but an optimization methodology that can be passed to an AI agent. Would you let an AI agent optimize the configuration of your local LLM all night long? github.com/sybil-solutions/iโ€ฆ
17
27
1
169
9,226
Ivan Rocha retweeted
Run Inkling 2 Mini, the strongest 26.4B model that runs on 16GB RAM. Thinking Machinesโ€™ Apache 2.0 MoE model uses a looped transformer arch & scores 48.2% on DeepSWE v1.1, the best for size. Run on Unsloth, llama.cpp GGUF: huggingface.co/unsloth/Inkliโ€ฆ Guide: unsloth.ai/docs/models/inkliโ€ฆ
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index. thinkingmachines.ai/news/inkโ€ฆ
130
315
83
3,653
282,478
Ivan Rocha retweeted
Inkling 2 Mini from Thinking Machines! ๐Ÿ”ฅ - 26B total params, 3.4B active (5.4B looped) - Weights just 20 GB - Small enough to run on a laptop - Ahead in 10 of 13 in coding vs Qwen3.8-27B. I'll try it out on my dgx spark. Link below.
67
69
19
1,147
57,136
Maya v1.1.69 is out: Hy4-Flash gets faster on every kind of PC. โšก โœ… Drafting on 8-12 GB cards with a small DFly2 drafter (up to +29% on an RTX 3050 8 GB) โœ… AMD / Strix & Gorgon Halo: prompts 2-2.5x faster, drafting ~30% faster โœ… Long prompts on NVIDIA: up to +56% at 128K โœ… Contexts up to 1M on ordinary cards, with the cache in RAM โœ… Pictures at full size, even on 8 GB cards ๐Ÿงช Hy4-Flash on Windows + AMD, experimental Drafting still checks every token. Thanks to @needmorevram, @superalesha and every contributor. github.com/mw00/project-mayaโ€ฆ
9
13
5
137
8,974
Ivan Rocha retweeted
Big update for GLM 5.3 Flash for 2x DGX Sparks Running latest TensorFold (v1.3.0). Lots of changes: - Faster decode: CUDA kernels now chain directly with fewer launch gaps (PDL) - Cleaner stack: Our paged FP8 KV, MoE, and DFlash2 speedups are now built into upstream (100+ patches down to 13) - Smarter prompt cache: Cache space is reserved up front, and multi-turn thinking chats reuse cached reasoning instead of re-reading history - More stable 2-GPU vision: Long image prompts no longer time out the second GPU Even more changes and improvements coming soon! Get it here: github.com/MiaAI-Lab/GLM-5.3โ€ฆ
44
30
8
434
30,205
Tencent's 220B Hy4-Flash now runs on a home PC with an 8 GB graphics card. ๐Ÿš€ Maya v1.1.68 is out, with four quants we made ourselves for @TencentHunyuan's new model. Each one is measured against the full-precision original: Maya-XS ยท 45 GB ยท 88.5% same top token Maya-S ยท 53 GB ยท 91.8% Maya-M ยท 68 GB ยท 93.6% Maya-L ยท 90 GB ยท 96.6% How does 220B fit? What doesn't fit the graphics card runs on the CPU from RAM, what doesn't fit RAM streams from the SSD, and the model's 63 GB n-gram table stays on the SSD. โœ… Drafting: the model checks every drafted token โœ… Vision, thinking on or off, tool calls โœ… NVIDIA RTX 20 series and newer on Linux and Windows โœ… AMD on Linux, Strix Halo and Gorgon Halo included โœ… One setup command picks the size and context that fit your PC Which one will you pick? Full notes ๐Ÿ‘‡ github.com/mw00/project-mayaโ€ฆ
33
40
10
404
30,330
Ivan Rocha retweeted
๐Ÿšจ๐Ÿ”ฅ ๐—ง๐—˜๐—ก๐—–๐—˜๐—ก๐—ง ๐—›๐—ฌ๐Ÿฐ-๐—™๐—Ÿ๐—”๐—ฆ๐—› ๐—ฆ๐—”๐—š๐—˜-๐—ฅ๐—˜๐—ซ๐—Ÿ๐Ÿฏ ๐Ÿฐ.๐Ÿด๐Ÿฐ ๐—•๐—ฃ๐—ช, ๐Ÿต๐Ÿฒ.๐Ÿต๐Ÿฎ% ๐—ง๐—ข๐—ฃ-๐Ÿญ ๐—”๐—š๐—ฅ๐—˜๐—˜๐— ๐—˜๐—ก๐—ง, ๐—”๐—ก๐—— ๐—ฆ๐—œ๐—ญ๐—˜๐—— ๐—™๐—ข๐—ฅ ๐—ข๐—ก๐—˜ ๐——๐—š๐—ซ ๐—ฆ๐—ฃ๐—”๐—ฅ๐—ž! ๐Ÿคฏ๐Ÿš€ The fidelity results are looking GREAT! ๐Ÿ† ๐Ÿต๐Ÿฒ.๐Ÿต๐Ÿฎ% ๐—ง๐—ข๐—ฃ-๐Ÿญ ๐—”๐—š๐—ฅ๐—˜๐—˜๐— ๐—˜๐—ก๐—ง ๐Ÿ”ฌ ๐Ÿฌ.๐Ÿฌ๐Ÿฎ๐Ÿด๐Ÿด๐Ÿญ๐Ÿณ ๐— ๐—˜๐—”๐—ก ๐—ž๐—Ÿ ๐——๐—œ๐—ฉ๐—˜๐—ฅ๐—š๐—˜๐—ก๐—–๐—˜ ๐Ÿ’พ ๐Ÿต๐Ÿฎ.๐Ÿต ๐—š๐—ถ๐—• ๐—š๐—ฃ๐—จ-๐—ฅ๐—˜๐—ฆ๐—œ๐——๐—˜๐—ก๐—ง ๐—ช๐—˜๐—œ๐—š๐—›๐—ง๐—ฆ I just finished running @TencentHunyuanโ€™s newly released Hy4-Flash through my latest SAGE-REXL3 quantization process! ๐—ง๐—›๐—œ๐—ฆ ๐—œ๐—ฆ ๐—” ๐Ÿฎ๐Ÿฎ๐Ÿฌ๐—• ๐—ฃ๐—”๐—ฅ๐—”๐— ๐—˜๐—ง๐—˜๐—ฅ ๐— ๐—ข๐—˜! ๐Ÿง  โšก ~12B active parameters per token ๐Ÿง  Native 1M context architecture ๐Ÿš€ Built-in 5-layer DFly2 speculative drafter ๐Ÿ’ป Hybrid sparse/sliding-window attention ๐Ÿ“ฆ 4.84 bpw mixed-precision EXL3 And hereโ€™s the interesting partโ€ฆ I preserved critical attention, routing and normalization components at higher precision while compressing the rest with SAGE. The result? ๐—” ๐Ÿต๐Ÿฒ.๐Ÿต๐Ÿฎ% ๐—ก๐—˜๐—ซ๐—ง-๐—ง๐—ข๐—ž๐—˜๐—ก ๐—”๐—š๐—ฅ๐—˜๐—˜๐— ๐—˜๐—ก๐—ง ๐—ช๐—œ๐—ง๐—› ๐—ง๐—˜๐—ก๐—–๐—˜๐—ก๐—งโ€™๐—ฆ ๐—ข๐—ฅ๐—œ๐—š๐—œ๐—ก๐—”๐—Ÿ ๐—•๐—™๐Ÿญ๐Ÿฒ ๐—–๐—›๐—˜๐—–๐—ž๐—ฃ๐—ข๐—œ๐—ก๐—ง! ๐Ÿ”ฅ โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” ๐Ÿ–ฅ๏ธ ๐——๐—˜๐—ฆ๐—œ๐—š๐—ก๐—˜๐—— ๐—™๐—ข๐—ฅ ๐—ข๐—ก๐—˜ ๐——๐—š๐—ซ ๐—ฆ๐—ฃ๐—”๐—ฅ๐—ž! The active model weights occupy ~92.9 GiB, while the additional 60.24 GiB Engram lookup tables are stored in NVMe-backed memory-mapped files. Thatโ€™s the memory layout Iโ€™m targeting for single-Spark inference. Full serving and context validation are still ahead. ๐—”๐—ก๐—— ๐—ฌ๐—˜๐—ฆโ€ฆ ๐—” ๐—™๐—จ๐—Ÿ๐—Ÿ ๐—ฅ๐—˜๐—ฃ๐—ฅ๐—ข๐——๐—จ๐—–๐—œ๐—•๐—Ÿ๐—˜ ๐—œ๐—ก๐—™๐—˜๐—ฅ๐—˜๐—ก๐—–๐—˜ ๐—ฅ๐—˜๐—–๐—œ๐—ฃ๐—˜ ๐—œ๐—ฆ ๐—ข๐—ก ๐—œ๐—ง๐—ฆ ๐—ช๐—”๐—ฌ! ๐Ÿ‘€๐Ÿ”ฅ ๐Ÿค— ๐— ๐—ข๐——๐—˜๐—Ÿ ๐—ช๐—˜๐—œ๐—š๐—›๐—ง๐—ฆ: huggingface.co/vcruz305/Hy4-โ€ฆ Huge credit to @TencentHunyuan and @tencentcloud for bringing Hy4-Flash to the open-weight community! ๐Ÿ™Œ ๐—›๐—ฌ๐Ÿฐ-๐—™๐—Ÿ๐—”๐—ฆ๐—›. ๐Ÿฐ.๐Ÿด๐Ÿฐ ๐—•๐—ฃ๐—ช. ๐Ÿต๐Ÿฒ.๐Ÿต๐Ÿฎ% ๐—”๐—š๐—ฅ๐—˜๐—˜๐— ๐—˜๐—ก๐—ง. ๐—ง๐—›๐—˜ ๐—ฆ๐—”๐—š๐—˜ ๐—ค๐—จ๐—”๐—ก๐—ง๐—œ๐—ญ๐—”๐—ง๐—œ๐—ข๐—ก ๐—๐—ข๐—จ๐—ฅ๐—ก๐—˜๐—ฌ ๐—–๐—ข๐—ก๐—ง๐—œ๐—ก๐—จ๐—˜๐—ฆ! ๐Ÿš€๐Ÿ”ฅ
Hy4 Flash is now released on huggingface!! ๐Ÿš€๐Ÿš€๐Ÿš€ Thanks to @aitrackerbot always being on point! You can finally run Haiku 5.5 locally ๐Ÿ˜Ž Hereโ€™s what it had to say about the new model: โ€œTencent just put the model from Monday's deleted announcement on Hugging Face: โ€ข 220B total parameters, but only ~12B active per token, so it's cheap to run โ€ข Reads text and images, with a 1M-token context window โ€ข In Tencent's blind test with its own users, it scored 2.79 out of 4 vs 2.81 for Claude Haiku 5.5, at the lowest cost per task of any model tested โ€ข About 1/20 the cost per task of full Hy4 โ€ข Also out: an FP8 version and the pre-trained base model Definitely going to SAGE this one!! Stay tuned ๐Ÿ‘€ huggingface.co/tencent/Hy4-fโ€ฆ
10
7
1
70
3,247
Ivan Rocha retweeted
Tencent may be cooking with this one... Hy4 Flash on a single DGX Spark: Ripple 65B quality at nearly double the speed ๐Ÿš€ I've spent the day putting Tencent's Hy4 Flash through its paces on one Spark, against Ripple 65B and the two Qwen3.8 open weight models. Quality Ripple 65B set the new standard on coding quality on a single Nvidia DGX Spark. Hy4 scored 76% on my 75-task reasoning eval, one point behind Ripple's 77% and well ahead of Qwen3.8 Flash-Next (63%) and 27B (56%). Efficiency This is the real killer feature - it gets there in about half the tokens, 11.7k per reply vs 22.6k for Ripple and roughly 25k for the Qwen models. Shorter replies plus faster decode means answers land much sooner. Speed up to 47.8 tok/s decode on code and 32 to 34 tok/s on prose, with prefill holding 846 tok/s at 2k and still 639 tok/s at 128k. How it fits Only 11.3B of its 155.5B parameters fire per token. The 63.5B n-gram memory is offloaded to SSD rather than sitting in VRAM, so the whole thing runs on one Spark with 295K context.
27
4
3
58
12,948
Ivan Rocha retweeted
i'm 100% convinced... that uncensored AI is the "LINUX" moment for AI. it runs with or without a GPU, refuses nothing, and your data never leaves your machine. here's how to build and run your OWN in 10 minutes โ†“
32
39
3
260
108,087
Gemini 4 Neon: frontier-level intelligence with fast end-to-end task completion across complex software engineering, knowledge work, and science workflows. Begins rolling out today in the Gemini API in @GoogleAIStudio, Gemini for business, for paid Google AI Pro and Ultra subscribers in Google @Antigravity, and for Google AI Pro and Ultra subscribers in the @GeminiApp.
920
1,493
700
18,671
2,166,408
Ivan Rocha retweeted
A free online book: LLM Inference Handbook by Sherlock Xu (2026). This book is a practical technical reference for engineers deploying, scaling, and operating language models in production. It brings together the concepts and decisions involved in turning a trained model into a responsive, cost-aware service, from model behavior and hardware selection to application interfaces and ongoing operations. Readers can use it as a guided progression or as a lookup resource for specific engineering questions. Read the book online on @ChapterPal: chapterpal.com/book/3fef12a8โ€ฆ
4
71
433
12,258