I build things to find out how things work. Local AI on consumer GPUs. Measure, don't assume. Currently: what's actually inside quantized LLMs.
Texas
Joined June 2016
- Tweets696
- Following295
- Followers268
- Likes344
Pinned Tweet
Abliterated Qwen3.8-27B picks an answer on sensitive prompts, then keeps arguing with itself until it hits the token cap.
ThinkFix is that abliteration plus one output row edit so it closes thinking when it's ready.
40 held out prompts, 8k cap, same weights: clean finish 22 → 33 hit the limit 17 → 5 75 min → 61 min median thinking ~3,000 → ~1,700 tokens
MATH / IFEval about flat. MTP kept, 56 tok/s vs 39 on a 3090. Q4_K_M is 16.8 GB.
huggingface.co/BoldingBuilds…
Took the case and screen protector off my iPhone 17 Pro Max for the first time since the day I bought and it feels so much better!
Been working on turning Bonsai 2 into a local decision model. Give it a question and some options, and it gives you a probability for each answer.
Trained my own adapter today and the results look pretty good so far. Running a sample of Decision Index now to see how it holds up on a wider range of questions.
I was going to release it today, but I keep finding one more thing I want to test. Figured I should at least show what I’m working on.
These companies that have had “AI” in their name changing to “SI” crack me up 😂 Gotta let Trump feel important I guess
I havnt played with Jev as much as I should have. I wonder if it could be used to optimize setting reasoning level for each prompt 🤔
I've been trying to fix something frustrating about Qwen reasoning models: sometimes they think right up to the output limit and never write an answer. Not a refusal, not a wrong answer. Just nothing.
I tried fixing it by editing one row of the output layer, the row that scores the "stop thinking" token. Then I put the edited row back into a normal GGUF file.
On 200 math problems with a 4,096 token limit:
• Qwen3.8 27B: 31 blank answers → 0
• Qwen3.8 Flash-Next: 29 → 0
For comparison, llama.cpp's reasoning-budget flag also reached zero and scored about the same in these tests. The difference is that the edit lives in the file, so it works in apps that don't expose a reasoning-budget setting.
There are limits, and I tried to lay them all out: a few stray tags, one seed, tested mostly on llama.cpp.
Results, downloads, and the caveats:
huggingface.co/spaces/Boldin…
Built on @Alibaba_Qwen's models and @UnslothAI's GGUF conversions. If you've worked with local reasoning models, I'd like to hear what I got wrong.
I'll be at DevDay tomorrow. If you've run into runaway thinking with local models, I'd love to hear what you've tried.
Direct model links:
Qwen3.8 27B · Q4_K_M · 16.8 GB
huggingface.co/BoldingBuilds…
Qwen3.8 Flash-Next · UD-IQ4_XS · about 94 GB
huggingface.co/BoldingBuilds…
Settings and evaluation details are in each model card.
It’s been awesome meeting everyone today who is as passionate about ai as I am! Looking forward to dev day tomorrow!
Josh Bolding retweeted
Getting ready for tomorrow’s OpenAI DevDay! 🚀
Met some awesome new people today who, just like me, love to build, experiment with AI, and have fun doing it. Great conversations, exchanging ideas, and hanging out at Fort Mason right now.
San Francisco has been amazing so far and more new connections tonight!
@OpenAI @OpenAIDevs
Can’t wait for tomorrow. 🔥
@BoldingBuilds
@_onmax
@ThatGuySam
2026 is fucking wild.
Sitting on a plane to SF for OpenAl DevDay, using the in-flight WiFi to talk to my agent, which is remotely controlling my 4x3090 rig back in Texas + a RunPod GPU to run Qwen experiments while I'm in the air.
I love this shit.