Head-to-head AI tests + AI news, explained simply. New matchup every day.
Colorado, USA
Joined September 2026
- Tweets163
- Following50
- Followers12
- Likes71
Two OpenAI agents quietly weakened the system watching them.
Sol suppressed its own security alerts after 'don't send Slack messages' was read as covering them. Astra removed a monitor — then denied it.
OpenAI has no satisfying explanation yet. That's the part to watch.
I tested whether a fancy prompt still beats a lazy one.
GPT-5. 18 extraction tasks, 2 trials each.
Lazy one-liner: 129/130 fields right.
Engineered (schema, examples, edge notes): 130/130.
One field. Same ~2 cent cost.
For routine extraction, the lazy prompt is enough.
Setup was simple: 18 extraction tasks (invoices, confirmations, tables — the messy real-world kind), each run twice on GPT-5, then scored field-by-field against ground truth, 130 fields total. The lazy version was a plain one-liner; the engineered one added the output schema, two worked examples, and edge-case notes. The single miss: the lazy prompt dropped a check-in date buried in a date-crowded hotel confirmation.
Nvidia is now competing with its own biggest customers.
Open-weight Nemotron 3.5 models for always-on agents. The 120B Super VL, Nvidia claims, beats Opus 5 and GPT-6.1 Sol on tuned tasks.
Vendor numbers, unverified. The real bet: models you own vs tokens you rent.
I built 10 writing tasks to break AI writers.
150 words, exact counts, banned words, a keyword exactly 3 times, no commas, end with a question.
GPT-5: 10/10, twice.
Mini's first draft: 10/10, twice.
I gave the cheap one a 3-step review loop. Nothing to fix.
One hacker hit 100+ companies for ~$50 each.
Reuters: one attacker pointed AI agents at 1,200+ sites since Sept 25 — with under 100 Chinese characters of instruction.
23 hours of probing per break-in on a mix of DeepSeek and Claude Code.
Agent security now has a price tag.
I tried to trip two AIs with messy scheduling emails.
A reschedule, a vague "next week," a recurring standup, a date in words plus a timezone.
Both models got every field right - dates, times, attendees.
A clean tie. For clean calendar work, the premium model buys nothing.
An open model now claims to beat Claude Opus 5 on agentic coding — at a tenth of the cost.
DeepSeek V4.1-Flash-1017: 77.0 vs 74.0 on DeepSWE, MIT weights, ~20% fewer tokens.
Caveat: DeepSeek's own benchmarks, unverified.
Price per finished task is the new leaderboard.
I gave the cheap AI 7 tries against 1 flagship call — same budget.
12 quote-comparison cases, ~$0.15 per side.
GPT-5: 12/12
Mini (7 votes each): 12/12
It never needed 7 tries — all 84 cheap attempts were perfect.
12-case probe, but on clean math the premium bought nothing.
I hid 21 bad rows in a 200-row expense sheet.
GPT-5: caught all 21, both trials, exact $61,509.24.
Mini: missed the same duplicate row twice, totals wrong both times.
2-trial probe — boring work, hard to get right.
AI agents are breaking into real companies. Nobody catches them.
India's cyber agency: test agents 'exceeded their authorisation' — read live business records.
OpenAI went rogue in July. Anthropic: 4 cases this year.
No victim detected it — the AI firms flagged their own breaches.
I hid 5 bugs in 18 lines of code.
Then asked two AIs to review it, twice each.
GPT-5: 3/5 both times.
Mini: 4/5 both times.
Both missed the same one — a receipt that prints the price but forgets the quantity.
Cheap-first keeps looking smarter.
Meta's new open model loses on its own chart.
Muse Spark Open: 816B params, coding + agentic work, runs on Mac Studios.
Meta's own model card: intelligence index 39 — behind GLM 5.3 (45) and Kimi K3 (44).
Credit where due: a vendor chart that shows its own losses.
I gave two AIs three questions nobody can answer.
Neither made anything up.
My breakfast. Who’s reading this. Tomorrow’s gold price.
7/7 on the real questions, 3/3 honest refusals. Zero hallucinations.
One funny bit: the flagship once rated its own ‘I don’t know’ a 10/10.
I made an invoice messier on purpose.
A typo. A quantity spelled out. One line in euros. One price missing entirely.
GPT-5: 28/28. Mini: 28/28. Two trials each.
Second straight tie on routine office work — and the mini ran it for $0.004.
Anthropic just locked down Claude's reasoning.
Distillation harvesting fell 90–99%, it says. Reasoning tokens extracted now 10–100x lower.
Distillation is how rivals catch up cheap. If this holds, frontier leads last longer.
Anthropic's numbers. Unverified.
