Leon Ho

Pick one task you can finish. Give it your attention before deciding what comes next.

Introducing e2e The agentic testing framework for any app ~/ npx e2e init → Fully Open Source → Mix deterministic and agentic APIs → Supports web, mobile, and more → Bring your own agent and infrastructure → Run locally or in CI tester.army/e2e

GPT-6 Sol and Luna push the cost efficiency frontier by halving cost relative to GPT-5.6 Sol and Luna. Intelligence Index and Coding Agent Index scores remain level with GPT-5.6, with progress in some evaluations and regressions in others Pricing is approximately half that of GPT-5.6: Sol drops from $4/$20 to $2/$10 per million input/output tokens, and Luna from $0.20/$1.20 to $0.10/$0.50, with the same 90% discount for cache reads and 25% premium for cache writes. Key takeaways: ➤ Halves Cost per Task: GPT-6 Sol (max) costs $1.06 per task to run the Artificial Analysis Intelligence Index, ~50% less than GPT-5.6 Sol (max) at $1.99. GPT-6 Luna (max) costs $0.07 per task, ~60% less than GPT-5.6 Luna (max) at $0.18. This is driven by the price cut, as both models use slightly more output tokens per task (31k vs 29k for Sol, and 51k vs 41k for Luna). These two releases allow OpenAI to capture a significant portion of the cost efficiency Pareto frontier. ➤ In the Coding Agent Index, Sol improves but Luna regresses: In OpenAI's Codex harness, GPT-6 Sol (max) scores 57 in the Artificial Analysis Coding Agent Index, up 2 points from GPT-5.6 Sol (max), with gains in Terminal-Bench 4.0 (43% vs 37%) and SWE-Atlas-QnA (58% vs 54%). At $2.99 per task it costs ~50% less than GPT-5.6 Sol (max) and sits on the Pareto frontier of Coding Agent Index vs Cost per Task. GPT-6 Luna (max) scores 41, down 2 points from GPT-5.6 Luna (max), with lower scores in SWE-Atlas-QnA (44% vs 49%) and DeepSWE v1.1 (64% vs 66%), at ~60% lower cost per task. ➤ Significant reduction in hallucination: Both models hallucinate less in AA-Omniscience, our knowledge and hallucination benchmark. GPT-6 Sol (max) cuts its hallucination rate from 92% to 60% and GPT-6 Luna (max) from 93% to 77%. Sol achieves this by declining to answer more often: it attempts 83% of questions vs 99% for GPT-5.6 Sol (max), which cuts wrong answers by about a quarter but also lowers accuracy 5 points from 59% to 54%. Luna's accuracy is broadly unchanged at 44% vs 43% while it answers fewer questions. On the AA-Omniscience Index, Sol improves from 22 to 27 and Luna from -10 to 1. ➤ Mix of improvement and regression across evals: Beyond AA-Omniscience, both models improve in AutomationBench-AA (Sol 62% vs 60%, Luna 53% vs 50%) and Terminal-Bench 4.0 (Sol 44% vs 40%, Luna 13% vs 12%). However, we observe regressions in two key knowledge work evaluations. In GDPval-AA v2.1, our benchmark adapted from OpenAI's dataset of economically valuable tasks across 44 occupations, Sol drops ~100 Elo points and Luna ~75. Luna also drops ~45 Elo points in AA-Briefcase v1.1, while Sol is level. AA-Briefcase v1.1 is a private evaluation across multi-week knowledge work projects, with thousands of input files. Our team has manually inspected hundreds of model outputs: the regressions tend to be driven by reduced presentation quality and deliverables that omit rubric elements. Congratulations @OpenAI and @sama on the launch!

Higher usage limits and lower cost give you more flexibility and room to iterate.

Opus 5.5 performs at the level of Fable 5.1. It's ~30% faster and ~40% cheaper than Opus 5 per task. In Claude Code: - 5-hour session limits increase 20% today - Opus 5.5 is priced lower, so it goes 25% further within limits - Pro, Max, and Team users get a reset to use anytime

We open sourced BrowserSkill, a bridge between your agent and your actual browser. most tools give the agent a blank browser. We let it borrow a tab from yours, then hand it back. > login state is already there, it just works where you're signed in > captchas and confirmation dialogs come back to you, then it continues > it's a CLI, not an MCP server => any agent that can run a shell can use it, and you see every call it makes one thing that's easy to miss: the agent asks before borrowing a tab, and that switch lives in your browser settings, not in a flag, so it can't be talked around. one line to install, works with Cursor, Claude Code, Codex, Hermes, Openclaw, CodeBuddy, WorkBuddy. Everything runs locally, MIT. github.com/Tencent/BrowserSk…

Astra's new compaction is experimental and is off by default. You must enable it. Type this prompt into Codex. Allows Astra to: 1. Save notes across context windows 2. Search messages and tool calls in earlier context windows

Today we’re open-sourcing Lily, the local inference engine we built for hybrid compute in Perplexity Computer. Lily is specialized for Qwen3.6-35B-A3B on Apple silicon, built so on-device compute doesn’t bottleneck Computer tasks. Read more: perplexity.ai/hub/blog/optim…

Update on @harvey’s model training effort. We post-trained a model we are calling Tenet: - Achieves SOTA on LAB - Generalizes 3rd party legal benchmarks - Uses sub-agents for domain specific capabilities Tenet uses Kimi K3 as base and was post-trained in collaboration with @FireworksAI_HQ: - Rank-64 LoRA over the full network - GSPO with importance-ratio masking - 134 B300 GPUs for 2 months Despite not being trained on 3rd party legal datasets we found improvements on: - @mercor’s Apex Agents - Corporate Law - @crosbylegal’s Redline Bench - LegalBench Tenet also learned how to use domain-specific subagents (separate post-trained models) for complex tasks: - M&A Diligence: training in an RLM harness for long-horizon tasks (with @baseten) - Review Table: specialist models for high-volume structured data extraction (with @appliedcompute) - Firm Knowledge: parametric memory and structured notes for more efficient enterprise search (with @engram) These results suggest we can significantly scale training and we plan to: - Scale both human and synthetic data significantly and scale training to 1K and then 10K GPUs - This scale will let us move to full parameter fine-tuning and larger models - We are now starting to post train models in our production harnesses - Post-train other open-source base models to provide customers with model choice If these problems sound interesting we are hiring for our post-training team

BREAKING: Moderna stock, $MRNA, extends gains to +212% on the day in after hours trading following positive phase 3 results of first the mRNA treatment to prevent cancer. The stock has now added over +$50 billion in market cap today.

GPT-5.6 Sol 1M in Codex. This used to only work for API keys, but we just flipped the switch and works for usage through ChatGPT accounts now too. The same warning applies, there is a reason the current context length is the default, we have tuned it to ~perfection. But you do you!

alright - Grok Bot i just spent quite some time with it since its introduction yesterday, did a bunch of work, and here's my real experience and initial thoughts 1. it's a very capable product packaged behind an extremely simple user interface remember Buzz? that app gave me the opposite impression, which was a few simple pieces of technology put together into a complex and somewhat disorienting experience if I am to reverse-PM Grok Bot, i would say it's targeting an average knowledge worker who doesn't have technical background and couldn't care less about clis, skills, context window, tokens, mcp, memory, cloud sandboxes, secrets, cron jobs etc etc, and just want a powerful agent that can actually get some work done and i'd say it's done a fantastic job at designing for that audience. all the complexity is well hidden - the user only sees the bots, and get most things done by talking to them as i revealed in my previous tweet, grok bot was clearly built by cursor and rebranded to grok - it speaks to the strength of cursor's product team there 2. next i'll reverse-engineer it a bit as well Grok Bot gives every user one virtual machine in the cloud. both the mac app and the ios app are just a thin client that communicate with bots in the cloud VM all the same users' bots share this same VM resource, but each bot has its own virtual desktop so they won't compete on the GUI interactions the VM has 8 vCPU cores, 16GB of RAM and 128 GB of disk running a debian system, which is plenty for knowledge work. if you rent an equivalent of this machine from most cloud providers it can cost ~$200/month give or take for each full month, which is too high of an operational cost, so there must have be some optimizations done to hibernate/resume the VM automatically. this probably has some abuse vectors - some people will absolutely run the machine as a free VPS to do other things even then, this machine is not sufficient if you want to do a lot of coding work which needs to run resource intensive tasks like build, test etc. but.. it seems they have already thought of that - coding work can be delegated to cursor cloud agents, not in the Grok Bot VM itself each bot is an instance of an agent harness running in the VM. this harness doesn't look like the same as Cursor. it's something called "Sand". if you ask it, it'll tell you and even show you its current JS code because the harness runs in a real VM environment, it's able to work with anything - CLIs, MCP, code execution, skills etc - which provides the foundation for an agent that's pretty much as powerful as the ones we run locally 3. many people are complaining about the pricing don't worry about that - and i'll tell you why by reverse-GTM this as well it's clear that the product is designed for all the average knowledge workers whose usage wouldn't be as heavy as developers who run many agent loops all day. people who just need to summarize some emails once in a while will never pay for $200+ a month, and there's no way the team doesn't already know this the reason they are only providing access to ultra and supergrok heavy subscribers is that they want the first wave of adopters to be the power users and tinkerers (like me) who will play with the product meaningfully, can give them feedback, and will not churn even if the product has rough edges. through this phase they can also test the reliability, scalability and cost efficiency of their system, to develop more confidence in a broader rollout i'm pretty sure in a few weeks they will relax the gating and give access to subscribers at lower pricing points, and open up to teams and enterprises as well 4. where it can improve to be honest - i'm an engineer, so i'm not exactly representative of the target audience, but i'm going to share my reactions anyway: - i wish i can see what model is being used. i don't want to have a bot touch a critical task unless i know it's using a big model :) given how fast the agent responds (which is a good thing), i often have the worry it's running a cheap open source model for efficiency purpose that might screw things up if the goal is to hide complexity, consider having an intelligence slider or something for power users to have some visibility and agency would go a long way - the bots currently are designed to be "persistent". for example, it looks like a bot can create other bots, but can't delete them this limitation made it hard for me to replicate @myfirstmate experience in there i think it would be better to think of bots as a primitive in the system and allow full agentic control over them. at the very least, if bot A created bot B, then give A permission to delete B. this can open up a whole new door for power users to create any freeform multi-agent system without constraints - i started moving a bunch of my regular tasks into Grok Bot, but one thing did give me a pause - it's vendor lock-in unlike hermes and openclaw where i own all my stuff, once i moved my workloads into Grok Bot, i'm tied to their subscription and if one day i want to use another LLM subscription instead, it would be harder to make the switch to ease this concern, an "export" flow would be helpful. it would be slightly against their business to do so, but it's a tradeoff between retaining users who actually want to leave vs allowing users who want to join to have one less thing to worry about before they do - the choice is actually clear alright - that's most of my thoughts for now! overall, i'm really impressed. this is a well-executed agent product that has a lot of depth deliberately and carefully curated behind a simple interface. i look forward to what's coming next!