AI engineering, cybersecurity, local LLMs, ai tools, open source
Joined June 2013
- Tweets1.3K
- Following612
- Followers694
- Likes2.6K
Pinned Tweet
I have created a fully interactive 3D medieval library based on three.js.
Everything can be moved or taken, and every book can be browsed.
GPT-6 Astra + DeepSeek V4.1 Flash 1017 + Blender
Music generated locally with ACE-Step 1.5 using my own custom tool.
Open source code on my GitHub.
Someone just built a 32-bit RISC-V CPU INSIDE Qwen3.5-4B. And arithmetic accuracy jumped from 48.6% to 100%.
No external Python interpreter. No tool execution. The CPU runs directly inside the model's forward passes.
The project is called Qwenputer:
- A real 32-bit RISC-V CPU compiled into the model's weights
- Gated DeltaNet state used as RAM
- Runs C and MicroPython programs internally
- Qwen's original weights remain unchanged
Instead of predicting the result of a calculation, the model can actually execute instructions to compute it.
nitter.gq/divagr1925/status/2115…
GitHub: github.com/divagr18/qwenpute…
Model: huggingface.co/divagr1925/Qw…
Technical write-up: divagr.com/blog/qwenputer
Which AI agent would you trust to run your business while you sleep?
Grok Bot, OpenBot, Hermes Agent, or something you built yourself?
Local Qwen generates 234 tok/s in JSON format. In plain text, only 70
The same local AI model: 70 tok/s in prose and 234 tok/s when generating JSON.
More than 3 times faster. Without changing hardware.
In Qwen3.6-35B-A3B-Splash tests on Mac mini M6:
- EN Prose: 69.6 tok/s
- Python Code: 135.9 tok/s
- JSON Extraction: 234.1 tok/s
Why? Speculative decoding works better when subsequent tokens are easier to predict.
Repetitive JSON and code structures can provide much greater speedup than ordinary conversation.
nitter.gq/swarogan/status/210392…
Benchmark of 16 local LLMs - results and script: github.com/HDZucht/Mac-Mini-…
1,035 tokens/sec on a single RTX 3090. Running Qwen3.8-27B.
Someone just released HyperQwen, an open-source project pushing Qwen3.8-27B to some ridiculous inference speeds using an optimized version of vLLM.
The benchmark results:
1,035 tok/s across 64 concurrent requests
381 tok/s on prompt-reproduction tasks
127 tok/s for a single user
Up to 262K context
All on one RTX 3090 with 24GB VRAM.
No new hardware. Just aggressive inference optimization.
HyperQwen is on GitHub
HyperQwen:
github.com/syv-ai/HyperQwen
NVIDIA limits the performance of its graphics cards. Someone has found a way to remove this limit.
The gpu-unlock project unlocks the full BF16 performance of Tensor Cores on RTX 20, 30, and 40 series GPUs.
BF16 results:
RTX 3060: 27 → 51 TFLOPS (+87%)
RTX 3090: 75 → 125 TFLOPS (+65%)
RTX 4080: 104 → 193 TFLOPS (+87%)
The author also reports 25–55% faster LLM prefill, 25–50% faster fine-tuning, and 30–55% faster image generation.
No BIOS flashing or NVIDIA driver modifications required.
The project is free and available on GitHub 👇
GitHub: github.com/Zagitta/gpu-unloc…
It is worth checking the requirements and limitations before use. Not all AI applications accelerate to the same extent.
Gemini 4 Argon finally came in handy for something.
It improved the vllm code, speeding up Gemma4 E2B performance by almost 3x, without losing token consistency or quality.
On A100, it's a jump of up to 108k t/s
And I'm still waiting for Gemma5
nitter.gq/googlegemma/status/211…
Speeding up Gemma!
Gemini 4 Argon enabled a 2.85x speedup on an A100 via vLLM, jumping from ~37,900 to 108,000+ tokens/sec on gemma-4-E2B-it.
✅ 100% token parity
✅ Zero quality drop
✅ Done in days, not months
See the exact changes across 8 PRs: goo.gle/4zNtANl
Small local models are useless for coding.
Or are they?
I use small local models daily for various tasks. Sometimes I run OpenCodeReview, but mostly I use them for automation: reviewing new tickets for my SaaS products, checking emails, etc.
But can they actually code?
To find out, I built a testing harness to measure their coding performance, see whether MTP helps with code generation, and more.
I'll release the whole thing as open source on GitHub. Feedback is welcome. nitter.gq/swarogan/status/210392…
A Breakthrough in Local Model KV Cache
Researchers have developed SemKV, a technique for compressing KV caches without context loss.
Results:
6x smaller KV cache
No statistically detectable quality loss in tests
Up to 7.9x compression with TurboQuant-MSE
The trick? Store tokens with different precision levels instead of deleting them.
We're focusing on reducing model weights, while the next big breakthrough for local AI could be reducing its memory.
Paper: arxiv.org/abs/2608.28911
Running AI models locally? These are the tools you need to know.
The essential toolkit for local AI in 2026.
1. Chat & UI
Easy-to-use interfaces for chatting with locally hosted AI models.
LM Studio, Ollama + Open WebUI
2. Local Inference
High-performance runtimes for running AI models on your own hardware.
llama.cpp, ExLlamaV3, MLX-LM, vLLM, SGLang, TensorRT-LLM, LiteRT-LM
3. Model Serving & APIs
Exposing locally hosted models through APIs and serving multiple users.
Ollama, LocalAI, llama.cpp Server, vLLM, SGLang
4. Mobile & Edge
Running AI directly on smartphones, embedded systems, and low-power devices.
Google AI Edge Gallery, LiteRT-LM, Cactus
5. Fine-tuning & Training
Customizing and training models with your own datasets.
Unsloth, Axolotl, LLaMA Factory, ms-swift, Transformers + PEFT, Keras, Tunix
6. Coding Agents & Harnesses
Connecting local models to coding tools, agent frameworks, and autonomous workflows.
OpenCode, Cline, Kilo Code, DeepSeek Harness, Pi, Hermes Agent, Aider, goose, OpenHands
7. Code Review & Debugging
Reviewing code changes, identifying bugs, and analyzing repositories.
DSH Code Review, DeepSeek Harness PR Review
Which tools am I missing?
nitter.gq/swarogan/status/210392…
Qwen 4 is coming out on October 29-31th
I hope it makes such a qualitative leap that the current benchmarks will have to be rewritten.
🚨Attention!
X introduces a feature that lets you send $1 by long-pressing the Like button via X Money.
How often will this happen by accident?
Is this the end of SaaS?
According to SaaSHub, the number of abandoned SaaS products is rising, and 2026 is already the worst year on record.
Wow! DeepSeek showed what a real model update should look like.
It took second place in cybersecurity.
Anthropic's results in this benchmark show that it is useless in cyber.
DeepSeek V4.1 Flash 1017 is the new #1 openly-available model in the Artificial Analysis Cyber Index. Other proprietary models may offer better cyber defense performance but are restricted access
@deepseek_ai recently released DeepSeek V4.1 Flash 1017, an update to their flagship open-weights model DeepSeek V4.1 Flash. With an Artificial Analysis Cyber Index score of 57%, DeepSeek V4.1 Flash 1017 is the #1 openly-available model for cyber defense capabilities.
The Artificial Analysis Cyber Index measures capability on agentic cyber defense tasks and is composed of three benchmarks: CWE-Bench-AA, DeepsecBench-AA and CyberGym-E2E-AA. The Cyber Index primarily evaluates models through their publicly accessible APIs, reflecting the cyber capabilities available to developers.
Many models, particularly proprietary models from OpenAI, Anthropic and Google, may have stronger defensive cyber capabilities but encounter safety refusals that reduce their overall scores. GPT-6 Sol (Daybreak Blue, max), available through OpenAI’s trusted-access program, leads the Index, scoring 32 points above its public counterpart driven by fewer refusals. DeepSeek V4.1 Flash 1017 encounters no safety refusals.
Key results:
➤ DeepSeek V4.1 Flash 1017 has the highest score among publicly-available models on the Artificial Analysis Cyber Index and #2 position overall. DeepSeek V4.1 Flash 1017 (max) scores 57% ahead of the previously leading publicly-available models Grok 4.7 (xhigh) and MiMo-V2.6-Pro by 1 point. It places #2 overall, with the trusted-access model GPT-6 Sol (Daybreak Blue, max) leading by 12 points with a score of 69%
➤ DeepSeek V4.1 Flash 1017 improves on its predecessor DeepSeek V4.1 Flash by 16 points. DeepSeek V4.1 Flash 1017 (max)’s 16-point improvement from 41% to 57% is driven by a substantial 53-point improvement on CyberGym-E2E-AA from 23% to 76%. On CWE-Bench-AA and DeepsecBench-AA it scores 69% and 25%, compared to 68% and 31% for DeepSeek V4.1 Flash (max)
➤ With a Cost per Task of $0.42, DeepSeek V4.1 Flash 1017 sits on the Capability vs. Cost Pareto frontier. DeepSeek V4.1 Flash 1017 costs $0.42 per task, more than 2x MiMo-V2.6-Pro (56%) at $0.18 per task but 4x cheaper than GPT-6 Sol (Daybreak Blue, max, 69%) at $1.77 per task
