https://nitter.gq/t.co/YJ88R0IWCp | On-device foundation models. High intelligence, low memory, low power
San Francisco Bay Area
Joined September 2026
- Tweets150
- Following101
- Followers348
- Likes59
Pinned Tweet
๐ฌ ๐ ๐ฏ๐ฑ๐ ๐๐ ๐น๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ ๐บ๐ผ๐ฑ๐ฒ๐น. ๐ฅ๐๐ป๐ป๐ถ๐ป๐ด ๐น๐ผ๐ฐ๐ฎ๐น๐น๐ ๐ผ๐ป ๐ฎ ๐ ๐ฎ๐ฐ.
This is ๐๐ฑ๐ด๐ฒ๐ฌ-๐ฏ๐ฑ๐ ๐ถ๐ป ๐ฎ๐ฐ๐๐ถ๐ผ๐ป in our macOS desktop app.
๐ป Edge0 keeps the quantized model weights on SSD and streams the expert weights it needs on demand, instead of loading the full 35B model into memory.
๐ In our macOS app benchmark, ๐๐ฑ๐ด๐ฒ๐ฌ-๐ฏ๐ฑ๐ ๐ฟ๐ฎ๐ป ๐ฎ๐ ๐ญ๐ฌโ๐ญ๐ฎ ๐๐ผ๐ธ/๐ ๐๐ถ๐๐ต ๐ฐ ๐๐ ๐ฟ๐ฒ๐๐ถ๐ฑ๐ฒ๐ป๐ ๐บ๐ฒ๐บ๐ผ๐ฟ๐ ๐ผ๐ป ๐ฎ๐ป ๐ ๐ฏ ๐ ๐ฎ๐ฐ๐๐ผ๐ผ๐ธ ๐๐ถ๐ฟ.
๐ Once the model is downloaded, inference runs ๐น๐ผ๐ฐ๐ฎ๐น๐น๐ ๐ฎ๐ป๐ฑ ๐ผ๐ณ๐ณ๐น๐ถ๐ป๐ฒ โ your prompts stay on your device, with no cloud API or per-token fees.
๐ฅ๏ธ And yes, thereโs an actual desktop app. ๐ก๐ผ ๐ฆ๐จ๐ซ๐ ๐๐ฟ๐ฒ๐๐๐น๐ถ๐ป๐ด ๐๐ถ๐๐ต ๐๐ฒ๐ป๐. Open the app, download a model, load it, and start chatting.
๐ข๐ฝ๐ฒ๐ป-๐๐ผ๐๐ฟ๐ฐ๐ฒ. ๐๐ฟ๐ฒ๐ฒ ๐๐ผ ๐๐ฟ๐.
๐ ๐ง๐ฟ๐ ๐ถ๐ ๐๐ผ๐๐ฟ๐๐ฒ๐น๐ณ
github.com/Edge0-AI/Edge0
๐ฌ๐๐ผ๐ถ๐ป ๐ผ๐๐ฟ ๐๐ถ๐๐ฐ๐ผ๐ฟ๐ฑ to get help with setup or share what youโre building:
discord.gg/Tg3W2FEuQ
Follow us for more Edge0 demos, benchmarks, and use cases.
AI that works in airplane mode?
Welcome to on-device AI: the model runs on your phone or computer.
Inference can work offline, avoid cloud round trips, and keep your data on the device.
Hereโs where those benefits matter.
And on-device AI doesn't have to mean small models or expensive hardware.
Our internal test of Edge0-35B on an iPhone 17 Pro Max:
- 70+ tok/s prefill throughput
- 14โ16 tok/s decoding throughput
- 2.80 GiB reported memory peak
- 4,096-token default context
Large-model inference on a phone.
Edge0 uses 4-bit quantization and loads expert weights as needed. Our 35B model also runs on a 16 GB M2 Mac.
Intelligence at zero distance for everyone.
Code: github.com/Edge0-AI/Edge0
Discord: discord.com/invite/Tg3W2FEuQ
Your data doesn't have to leave your device for AI to work.
With local AI, your prompts, files, and model outputs can stay on your own device.
No need to send every request to someone else's servers.
And you don't need expensive hardware to do it.
Local AI makes sense for:
- Private documents and notes
- Proprietary source code
- Sensitive internal workflows
- Offline applications
Of course, local doesn't mean risk-free. Device security, application permissions and backups still matter.
Edge0 brings open-source 8B and 35B AI models to consumer devices, with runtimes for macOS, iOS, Android and Windows.
Want to keep your AI workflow on-device?
Code and setup: github.com/Edge0-AI/Edge0
Discord: discord.com/invite/Tg3W2FEuQ
Stop paying per token for AI tasks you can run locally.
Hosted AI APIs charge for the tokens your model reads and generates. With local AI, your own hardware does the work.
And no, you don't need an expensive workstation to get started.
Here's how.
How?
Edge0 uses 4-bit quantization and streams expert weights from storage as needed, rather than keeping the entire model in RAM.
That makes large sparse models practical on smaller devices.
Hardware, storage and electricity still cost money, of course.
Edge0 is open source, with 8B and 35B models and runtimes for macOS, iOS, Android and Windows.
Want to try local AI on your own hardware?
Code and setup:github.com/Edge0-AI/Edge0
Discord:discord.com/invite/Tg3W2FEuQ
As @rauchg said: "Own your intelligence." We agree.
That includes running AI locally, choosing which model your application uses, and deciding when to update it.
At @Edge0_AI, our 35B and 8B model weights and inference code are available under Apache 2.0. You can download a model, run it locally, and inspect or modify the runtime.
Your application can call that deployment through a local API. You can keep a tested version running while evaluating updates, and use hosted models alongside it.
Code and setup: github.com/Edge0-AI/Edge0
Join our Discord for setup help or to share what youโre building: discord.com/invite/Tg3W2FEuQ
Follow @Edge0_AI for implementation notes and upcoming releases.
We want local AI to be a real option for everyone, not just people with high-end hardware.
That's what we're building toward at Edge0.
We're open source, and we have a major release coming next month. Check out our GitHub, join the discussion, and follow along!
Links in the comment section:
Own your intelligence.
Anyone that opposes this has something to sell you (perhaps your own intelligence back to you.)
Local models need to continue proliferating. The โabsolute unitโ massive models may be out of reach for now locally, so your best option is to, indeed, keep your options open. Own your harness, data, connections, runtime: your agents. Embrace open source.
