Cascadia (@withcascadia) header
Cascadia (@withcascadia) profile picture

Run powerful AI on the hardware you already own.

Joined June 2026
Photo by Cascadia (@withcascadia)
Cascadia is now serving Inkling (975B) from @thinkymachines across eleven Intel Core Ultra PCs, in a deployment with @IntelBusiness. Quantized, Inkling sits at ~475GB in size, which we split into shards and distributed across the eleven machines. Each one runs a custom MoE engine built on top of OpenVINO. Peak aggregate throughput reaches 60.29 tok/s at 88 concurrent streams.
1
3
1
8
1,151
Cascadia retweeted
Your AI wouldn't be down if you were using @withcascadia rn 🤠
1
1
4
116
Cascadia retweeted
Weekends are for running @withcascadia on chromebooks… Solid 4gb of RAM 🤠
1
1
16
1,207
Cascadia retweeted
Why choose between local and hosted if you want to use both together? We built yeschef so Claude Code and Codex can spawn subagents that run locally. Works out of the box with @withcascadia, @Ollama, @exolabs, etc. Save on your Claude usage limits and put your GPU to use.
1
4
1
24
1,537
Your business already owns the AI compute, but it’s just sitting idle. @TateBerenbaum and Cascadia.to work with Intel to help teams take advantage of this power potential by turning their existing Intel hardware into a private, on-prem, distributed inference network—one that you own and that can run a range of open-weight and local models without additional hardware spend or per-token cloud costs. Learn more: ms.spr.ly/6014aOYks
8
25
1,266
Cascadia retweeted
The best place to find machines with 1.5TB of memory for <$10k? 2019 Mac Pro. Using to test @withcascadia
1
2
19
1,227
Cascadia retweeted
Excited to share the first research paper behind @withcascadia We discovered pipeline micro-batching and speculative decoding can make up for much of the added latency of sharding models over TCP. This paper breaks down how, along with some novel work on NPU continuous batching.
3
3
16
754
I did some follow-up testing with Cascadia! Qwen3.8-27B didn’t work out of the box, so I had to export it to OpenVINO FP16, re-quantize it to INT4 with NNCF, and simplify the chat template for Minja before I could finally get it running. On the Arc 140V, INT4_ASYM / g128 reached about 6.3–6.4 tok/s, while INT4_SYM / g128 reached about 6.6 tok/s. That’s noticeably faster than the llama.cpp baseline of about 4.4 tok/s even without speculative decoding. Prompt Lookup caused the output to break down, and draft-model speculative decoding isn’t supported with this VLM-layout export. Thanks for the interesting suggestion — this was a lot of fun to experiment with!
1
1
3
137
Cascadia retweeted
I’ve been trying to figure out the best way to apply NPUs to inference. Because NPUs are better at parallelism (and therefore digesting prefill tokens), and CPUs are generally better at token-by-token decoding, you can take a hybrid approach to inference if you have a CPU + NPU chipset. TTFT speedup was 35x on a 1B model, and 40x on a 3B model (compared to solely CPU inference). Merged into @withcascadia a few weeks ago :)
1
1
8
547
Run the most powerful AI models across Intel silicon. Co-developed with @IntelBusiness.
People shouldn’t have to build data centers to access frontier AI. We just need it to run better on commodity hardware. So we teamed up with @IntelBusiness to build a solution. This is Cascadia, and now you can use it too.
2
3
2
13
1,529