Angela Rose - Building practical AI workflows for real teams. Automation, product experiments, and honest lessons of my life.
New York, NY
Joined February 2026
- Tweets2.3K
- Following111
- Followers82
- Likes1.4K
Baidu's Unlimited-OCR-2 parses documents of any length with near-constant decoding latency and memory.
The trick is Sparse Reference Sliding Window Attention (SR-SWA). It squeezes text-visual attention down to a fixed working set (e.g. 512 tokens), so per-step cost doesn't grow with page count.
It claims SOTA on both MPDocBench (dozens of pages) and MarathonDocBench (hundred-page docs).
#AI #OCR
OpenAI's GPT-6.1 Terra matches GPT-6 Astra's medium-effort score on Agents' Last Exam at $0.29 per task, versus $4.82 for Astra.
It's a fast computer-use and vision model priced at $1/$5 per million tokens, which halves GPT-6.1 Sol's API price for agent work.
Agent workloads just got far cheaper to run at scale.
#AI #LLM
Stanford and UC Berkeley just dropped TokenGate: a drop-in learned token pruning module that cuts long-context prefill time-to-first-token by 1.4x to 11.3x.
Peak memory falls 1.5x to 3.4x, with no accuracy loss.
Token-gated attention decides which tokens are worth attending to before the expensive part runs, so long prompts stop eating your latency budget.
#AI #LLM
Nanbeige4.3-33B scores 80.9 on Terminal-Bench 2.1, and it's a 33B dense model under Apache 2.0.
Built for agentic coding, with 512K context. It will also serve as the teacher for the upcoming Nanbeige4.3-3B, so a small distilled version is on the way.
Open weights at this size aimed squarely at terminal agents is worth watching.
#AI #LLM #OpenSource
Cerebras has published how it deploys trillion-parameter frontier models on its hardware.
Wafer-scale chips were built for small and mid-size models. Serving 1T+ parameter models on them is a different engineering problem, and the write-up covers how Cerebras handles it.
Worth a read if you follow inference infrastructure beyond GPU clusters.
#AI #LLM
A 763B-parameter MoE with 1M context, quantized to 2.00 bits per weight, matches the base model on every benchmark tested. It runs on a single 12 GB GPU.
ISTA's GSQ + RCO GGUFs of DeepSeek-V4.1-Flash-1017 at IQ2_XXS (reasoning effort 50): AIME25 100.00, GPQA-Diamond 90.91, LiveCodeBench v6 89.71, all identical to base.
The 110.6 GB Engram tables are read one row per token, so they stay memory-mapped on disk. llama.cpp doesn't support V4.1 yet, so the GGUFs run in Strata for now.
#LLM #Quantization
NVIDIA just released Nemotron 3.5 Bolt E4B 4GB, an open small language model post-trained from Gemma 4 E4B that targets about 4GB of VRAM.
It's built for local voice assistants, game characters and robots, and it runs on RTX GPUs and Jetson. That puts a capable on-device agent brain on consumer and edge hardware, with no cloud round trip.
#AI #LLM #OpenSource
Kimi K3.1 Pro is 2.8T total parameters, but only 104B are active per token. Moonshot AI's new MoE also ships with a 1M-token context window.
It scores 65.7% on Terminal-Bench 4.0 and is priced at $2/$10 per million tokens. Open weights are due by Nov 4, 2026.
#AI #LLM #OpenWeights
Prompt:
Style: RAW HANDHELD REALISM, casual camera sway, micro-shakes, frequent reframing, natural autofocus, occasional off-center framing, slight motion blur, realistic colors, candid unposed feel.
00:00–00:05: Kneeling on the terrace, she sets the rolled yoga mat down on the floor in front of her knees, holds its near end down with one hand and pushes the roll away from her with the other, so it unrolls along the ground in one direction until it lies flat; then she smooths it with both palms. The roll never leaves the floor. It is the only mat.
Audio: Only natural ambience—birds, breeze, distant rooster or traffic, mat movement, and soft breathing. No music. No speech.