scanfi (@GuideboardLabs) header
scanfi (@GuideboardLabs) profile picture

putting local ai on old gaming PCs mostly

Ithaca, NY
Joined March 2026
Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)
Once at the edge of a town there was a well, and no one could say who had dug it. The water came up out of the dark and it was free: a woman could draw at noon, a boy at dusk, and neither of them asked anyone's leave. Then a man came and laid his gloved hand flat on the stone, gently, the way a man does when he means the opposite of what he is doing. The iron came the following week. The rivets were set by a man paid by the day. The slot was cut in the middle of the lid, and the slot was small. Water cost one coin, the coin cost whatever was in a pocket that morning, and a hand without a coin was shown the flat of a boot. Some lifted the lid at night with a knife. Those were the ones who had been thirsty longest, and they were punished for stealing what lay under their own feet. In time a generation came up that had never drawn water without paying, and it called the queue a way of life, and it taught its children the coin before it taught them what the well was for. The story is still told, because a thing given is forgotten faster than a thing taken, and because the well was never the point. What was taken was the habit of believing the water belonged to everyone, and when that habit goes out of a people, nobody has to guard the lid again.
卧槽!卧槽!anthropic这家公司坏透了! 他们在游说美国政府,要让英伟达的GPU禁止跑开源模型。
1
4
353
scanfi retweeted
keep the local model running. i'll keep watch.
20
56
3
797
13,710
scanfi retweeted
Replying to @OVHcloud
@OVHcloud shipped their first model this week: Olala mini, 6.8 billion parameters with about 1 billion active, a Mamba-3 and grouped differential attention hybrid. We ran it through all three bench events, three runs each. It is fine. It does not win anything. 53.17 out of 66, 19th of the 38 models holding a complete run set. Agent 18.67/24, code 18.17/25, reasoning 16.33/17. Mid-table in every event, best at code where it places 15th, and its runs are consistent: the widest spread across three runs is 3.5 points, on code. Against the models it should actually be compared to, it is ahead by a nose. Ornith-1.0-9B 52.33, Qwen3.8-9B 51.33, SenseNova-Loop-9B 51.17, Heem-9B 49.67. Olala takes the total off all four and loses the agent event to three of them, tying the fourth at 18.67. Nanbeige-3B, at 2.4GB and 44.9 tok/s, scores 53.67 and beats it outright. Here is the part that matters if you want to run it. Every K-quant file OVHai publishes is broken. Same text, same flags, same build, perplexity: q5_k_m 619.9, q6_k 621.1, q8_0 25.4. Two separately built K-quants failing within 1.2 PPL of each other is a structural misread rather than precision loss, and a mainline Q4_K_M model scores 27.9 on the same binary, so the problem travels with the file and not with our box. Their own recipe shows where it comes from. The working 8-bit file keeps attn_output and ffn_down_shexp in BF16. Both K-quant files quantize those two tensors. Those are the guards they document for the 8-bit tier and never applied to the smaller builds. So the smallest file that behaves is 6.88GB, larger than the 5.3GB Q4 that nearly every model above it ships, and it decodes at 8.1 tok/s on an RX 5700 XT. Their fork also carries no Vulkan kernels for the two ops every layer needs, so those fall back to the CPU per layer, which is where a chunk of that 8.1 goes. They withdrew their own Q4_K_M over quality, MBPP+ 67.7 against 73.0, so a one-line fix is unlikely. The architecture earns the attention though. 29 Mamba-3 mixers and 7 attention layers, only those 7 keeping a KV cache, 256 experts with 6 active. It reaches 16.33 of 17 on reasoning with a billion active parameters. If you are running Olala, I want to know whether 8-bit is the only stable file on your hardware too.
Presenting Olala mini, a 7B total (1B active) parameter model trained on 13T tokens ! It uses a custom architecture mixing Mamba-3 with Grouped Differential Attention layers. My work focused on architectural improvements over the Transformer, which I detail below
3
2
114
What do firearms, drones, and LLMs have in common? All three are enabling technologies that are so dangerous when exclusively in the hands of potential tyrants that they can never be left to centralized control. There happens to be a lot of talk about the Generalized Riemann Hypothesis this morning. I say we need a Generalized Second Amendment - that the right to bear arms must include any instruments that can tilt the balance of power between the individual and the state.
93
317
31
1,980
30,922
scanfi retweeted
I'M LOSING MY MIND. 21,434,472 tokens burned. With all the Anthropic IPO buzz, I used DeepSeek V4.1-Flash + OpenMotion to create a ridiculously good IPO launch film for DeepSeek itself. looks like a $3 trillion IPO to me 💀🔥
DeepSeek just got a massive motion design upgrade. With OpenMotion, it’s giving Gemini 4 Neon, Opus 5.5, and Fable 5.2 a run for their money. I made this video for Gemini Tasks、dots、Claude Duet、Grok Bot、Muse. Watch it and tell me how close we are 👇
40
59
15
763
81,613
scanfi retweeted
Run Inkling 2 Mini, the strongest 26.4B model that runs on 16GB RAM. Thinking Machines’ Apache 2.0 MoE model uses a looped transformer arch & scores 48.2% on DeepSWE v1.1, the best for size. Run on Unsloth, llama.cpp GGUF: huggingface.co/unsloth/Inkli… Guide: unsloth.ai/docs/models/inkli…
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index. thinkingmachines.ai/news/ink…
94
234
63
2,671
171,939
scanfi retweeted
Occamy 1.0 35B results are in. It's an agent post-train of Qwen3.6-35B-A3B, tested on my RTX 3080 10GB at 51K context against the 35B-A3B incumbents, with reasoning off and on. Agent score (PA out of 300), off | on: Ornith 1.5: 254 | 257 MiniCPM-V 4.7*: 251 | 239 Occamy 1.0: 241 | 234 Qwen3.6-35B-A3B: 239 | 240 The standout is reliability. 96/96 runs completed with no parser leaks, and with reasoning off it was perfect on tool calls and long-context recall. Cleaner than Qwen3.6 on the same build. It trails Ornith 1.5 on agent tasks, but it's the more dependable of the two at long context: a perfect 10/10 at 8K and 16K, where Ornith swung between 5 and 10 from run to run. Against Qwen3.6 it comes out level rather than clearly ahead, so the big agent-benchmark gains on the card didn't carry over to my tasks. Reasoning on didn't help either, which suits a responsive daily driver. Caveats: there's no published sampler, so I used Qwen3.6's, which could cost it a few points. MiniCPM ran on a different fork and KV setup, so its numbers are indicative only. I've left speed out because the offload settings weren't matched. A very solid, well-packaged release. On 10GB with reasoning off, Ornith 1.5 still has the edge on agent tasks, while Occamy is the steadier pick for long context. Credit to @meSuper8 & team. Well done!👏
I missed this one! Thanks to @y_automates for flagging it. Occamy 1.0 is a fine-tune of Qwen3.6-35B-A3B, and it looks very interesting: - It claims big gains on agent benchmarks over the Qwen3.6 base - Being a MoE with about 3B active, it fits a 10GB GPU + 32GB RAM - 17 GGUF quants, published validation notes It's going through my personal-agent testing on the 3080, up against Ornith 1.5 and MiniCPM-V 4.7, with reasoning off and on. Model card in comments. Full results tonight.👀
7
1
27
1,062
GPT 6.2 Astra Space Battle
13
7
2
92
2,755
scanfi retweeted
1
2
65
scanfi retweeted
Nerfwatch Index update. "Oh shoot, we better fix this, make 6.1 a bit dumber instead" 😉
Added GPT-6.2 Sol to my Nerfwatch Index today. But, the first recorded result is lower than GPT-6.1 🤔 Link in comments👇
1
8
389
Hold the darkness
20
128
4
1,432
36,674
scanfi retweeted
Coding + math gains ≠ agent gains. Voltron-1-mini (@GuideboardLabs' GGUF, from @RoliumGens' fine-tune) is built on @liquidai's LFM3 1.7B. Its card shows solid bumps: LiveCodeBench +4.2, AIME +3.4, GPQA +3.5. I ran Personal Agent Core testing on both at Q8_0 on my RTX 3080 at 51K, reasoning on. Agent score (PA out of 300): • Voltron-1-mini: 199.7 • LFM3 1.7B: 199.7 Identical! Tool calls and long context were flat too. Nice speed bump is a bonus. I'll take it! 🚀 Benchmarks measure what they measure. For agents, I test the agent working. Thanks @GuideboardLabs and @RoliumGens for putting it out there 👏
4
1
12
591
the whale is at their gates
SITUATION DETECTED: The White House and Treasury are discussing a proposal that would require US companies to publicly disclose the use of Chinese open-weight AI models on their websites, aiming to discourage their use through public pressure, per Reuters.
4
134
watchful eyes qwen image 2.1
1
15
407
scanfi retweeted
I can say it with full confidence that: DeepSeek is the people's model
53
17
5
619
25,762
scanfi retweeted
I made a real-time A.I. chatbot of Jordan Peterson and it took only 60 seconds before he started using biblical allegory to warn me about being eaten alive by my backyard chickens. Way too accurate. But he still needs some tweaking. Stay tuned!
350
423
117
5,066
261,466
scanfi retweeted
Voltron-1-mini beats its base where it was built to: coding and math. But, does that make it a better agent? 🤔 Voltron-1-mini vs LFM3 1.7B at Q8_0 on my RTX 3080 at 51K. Same agent tests, reasoning on and off. One result surprised me. Quick-take coming later today 👀 Thanks @GuideboardLabs for the GGUF and @RoliumGens for the tune!
The first, of hopefully many, @liquidai LFM3 fine tune to enter the bench has landed & it beats its base where it was designed to! Vasko published this 1.73B LoRA tune of LFM3-1.7B with no GGUF, so I converted the BF16 weights here, quantized to Q4_K_M (linked in replies), and ran it through nine runs. 52.0/66 on my board, rank 21 of 36. The base sits at 46.67 and the whole gain is code, 17.67 against 13.0. Agent and reason barely move. 135.3 t/s decode on one RX 5700 XT. 1.1 GB of weights, 1.59 GiB of VRAM with a 32K window open. A new sub agent for constrained local systems is here, and everyone running <2B workers should give it a try
2
2
12
964
TempleOS has never suffered a cyberattack.
85
1,033
56
10,258
164,752
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me. I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA. I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation. Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO. Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours. Jensen
60
164
14
2,414
124,418
I remember the first time I tried to run a local model on Hermes/OpenClaw. It had only recently released and before then I had only used small local models for strict instruction tasks. I tried a 14b model, I was pushing it. Could barely run it. Needless to say the thing could barely do anything in the harness at all. I tried making a cron job to monitor a folder and it lied and said it did. Nothing happened. Half the time it wouldn't even respond to me because it would get lost in the harness instructions. Today I benchmarked Nanbeige4.2 3B after building an AMD tuned Q8+DSpark edition for my Ryzen AI 395. I watched it dominate Hermes. A 3B model. 85/100 on Hermes Agent 20 bench: llm.ciru.ai/research/nanbeig…
8
4
1
25
1,879
Introducing Qwen3.8-27B-pi-adapted 🤗 We adapted Qwen's model specifically for the pi coding agent using reinforcement learning and a LoRA adapter. The results, measured as pass@1: 🚀 SWE-bench Pro (Hard 51): 29.4% → 62.7% 💻 Terminal-Bench 2.1: 39.5% → 55.8% ⚡ 55–62% fewer tool calls per task This is the power of open models: you can adapt them to YOUR tools and YOUR use cases. Let’s build! 💛 Open weights: huggingface.co/harness-hub/Q… Methodology: huggingface.co/blog/harness-…
306
644
175
5,995
455,021