create some libraries or reusable assets along with the skills and you can even use qwen 27b do wonders with it..
Unsloth is back with a banger
We’re releasing new DeepSeek-V4.1-Flash-1017 GGUFs, ranking highest across benchmarks.
Dynamic V3.1 leads across nearly all benchmarks: GPQA Diamond, AIME25, LiveCodeBench v6, MMLU-Pro, Aider & more.
Run on 64GB+ RAM.
GGUF: huggingface.co/unsloth/DeepS…
Blog: unsloth.ai/docs/models/deeps…
Thanks for making the DRAM on my rig so valuable.. few months back i didn't even know how valuable and scarce it was.
Yes.. I will donate all my resources for the greater good, along with my assets and properties so those can be utilized most efficiently.
sam: local inference is a terrible allocation of scarce resources
“a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre
but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time
you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment
somehow many people's intuition is the opposite of that”
Finally adding single pass of layers through multiple chunks in the prompt, made my llamacpp fork prefill speed reach faster than strata.
prefill: 350t/s
decode: 21t/s
on 16GB 5060ti
64GB ram
Quant : GSQ RCO Q2_xxs
I feel like at this point there is not much point on releasing dense models for edge AI.
any lab is building for Consumer hardware should make bigger MOE with high sparsity, and add engram as an additional knowledge table.
Thats it!!
What???? these are insane benchmarks. matching glm 5.3??
How is it even possible ??
🚀Excited to announce Nanbeige4.3-33B!
To push the performance ceiling of small agents, our strategy in Nanbeige4.3 starts with building a powerful teacher first.
Nanbeige4.3-33B delivers SOTA agentic performance among open models at ~30B parameters:
* Strong results on agentic coding and cowork benchmarks.
* When integrated into mainstream agent harnesses, it tackles real-world tasks with cost-efficient inference.
* It will serve as a teacher for upcoming smaller Nanbeige models, starting with Nanbeige4.3-3B.
huggingface.co/Nanbeige/Nanb…
huggingface.co/Nanbeige/Nanb…
#LLM #Agent
looks like there are now many candidates that can replace the OG Qwen 35b moe now.. really glad to see some competition on this segment
a 40b model with 2b active 😍
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
Codacus retweeted
Our largest model yet.
763B parameters, compressed with GSQ-RCO and co-optimized with Strata (by @coldniko) for local inference.
On Hugging Face this Friday.
Much more is coming. Follow us for future releases.
Wow the other side is now, entering the arena,
77 on DeepSWE and Peaked in Automations!!
waiting for others to show their cards
🚀 DeepSeek-V4.1-Flash-1017 is here!
Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our website, app, and API.
🛠️ Optimized for @DeepSeekHarness, Tencent CodeBuddy, WorkBuddy, and more. Upgrade to DeepSeek Harness v0.2.1-rc.1 or later for the best experience with the new model.
1/4
Hot Take: JEV is just an LLM with one single pass parallel call for all the field..
So finally we have a sub 40b moe model other than the qwen
🚀 Introducing MiniCPM-V 4.7, our new open-source MLLM series!
Advancing the Performance–Efficiency Pareto Frontier.
It brings strong multimodal capabilities to iOS and Android devices with extreme efficiency.
Two new models:
🔹 MiniCPM-V 4.7 35B-A3B — Bring SOTA performance to local devices with newly introduced LES (Lookahead Expert Scheduling) technique, a new milestone of on-device multimodal AI.
🔹 MiniCPM-V 4.7 1B — Lightweight, efficient multimodal AI. Easier to deploy.
⚡ What's new?
SOTA performance: Competitive results across 34 benchmarks spanning multimodal understanding, STEM reasoning, OCR and long-video comprehension.
Fast on-device inference: LES predicts and preloads expert weights to reduce inference stalls, boosting 5× speedup on iPad Pro M4 (16GB) with byte-for-byte identical outputs. It enables MiniCPM-V 4.7 35B-A3B to achieve 17+ tokens/s on iPad Pro M4 (16GB) and 13+ tokens/s on iPhone 17 Pro Max, while costing only 3GB memory 🤯
Efficient visual processing: 4× and 16× hybrid visual token compression. Even at 16× compression, 35B-A3B scores 90.4 on OCRBench, 5.2 points ahead of Qwen3.6 35B-A3B.
Easier customization: We introduce Self-Replay Fine-Tuning, a simple way to customize new target capabilities at low cost while maintaining general performance.
Broad hardware support: Day-0 adaptations for Apple, Intel, AMD, Arm and Rockchip, with many more platforms supported through FlagOS!
Download, deploy, and build something we haven't imagined yet. 👀👇
🔗 GitHub: github.com/OpenBMB/MiniCPM-V
🤗 Hugging Face: huggingface.co/collections/o…
