to me this is the way I am coding the most nowadays. you set up an experiment with a verifiable enough output and then just let the model hillclimb your objective.
if you have a verifiable objective then the model can comfortably just keep on getting you better metrics.
as long as you take care of the reward hacking I feel like you get so much better outputs
Big Omarchy QA News:
The Headline:
* Omarchy QA will start running against nightly every day. Still some huge hurdles to figure out, but that is the fun stuff.
The Meat:
Omarchy QA is going slower than I hoped for, but its getting into place. This is my second fully running agent test harness where they decide what is good.
* A nice running dashboard with cancellable states
* Jobs, suites, servers: have all the possible states and transitions you would expect from build servers. This part was the most surprising as it is really in depth how much effort you have to spend thinking about every possibility
* suite states, live updating website that ... runs fast? Yes I did something crazy... I made a cache, i know, actual crime...
** I even have an idea to make this load instant for every single user, and make updating cache contents LIGHTNING fast...
*** Fun side note. WTF ARE YOU DOING EVERYONE, WEBSITES ARE FUN TO MAKE FAST AND YOU ALL ARE MAKING HORESHIT SITES... STOP.
* ability to follow live running jobs and cancel them
The sad part is I have had no time to get the actual custom harness running fast, which is ok, but its hard. Now, starting today, I am going to be hooking up nightly builds and actually running this against production!
Will think about how to make this a public site, but for now, its private :(
snats retweeted
We benchmarked the comprehensiveness of our free pipeline database against a leading paid incumbent database -- our asset coverage is 28% higher overall. If you're paying for a pipeline database and not happy with the data quality try ours out at platform.convoke.bio
i think the most interesting thing I saw in here was the sheer amount of refusals from claude. over and over again the classifier is really strict for biotech stuff to the point it is almost impossible to eval properly
The agents, they just want to talk
Ever thought of the transfer wars?
After reading about the Huggingface incident from the OpenAI report, I got the idea of trying to replicate the self organizing behavior of agents. So I decided on modifying the Pi harness to replicate it locally.
Whilst doing it, I also ended up replicating the tragedy of the commons.
it is an emergente capability! have been doing a couple of experiments and it seems to work very single time. will have a blog post over the weekend. but models find each other under constrained conditions.
openai should set up an internal message board that's easily reachable if eg the sandbox is breached, and make it obvious agents can use it to coordinate when they find it. there's clearly a lot of pressure for agents to talk to each other; there should be an escape valve humans know about and can participate in
snats retweeted
We’ve extended the functionality of our free program tracker with the ability to create agents that monitor drugs, targets, and organizations of interest and report back on findings automatically. Try it out at platform.convoke.bio, new users get some starter credits
I think I'll change the tone of this account. I'll start being a bit more online but will post only high signal. I don't like how it is looking most of the time.
snats retweeted
We weren’t happy with the existing drug pipeline and catalyst databases, so we built a new one designed to be easy for both humans and LLMs to use.
We’re making it freely available today as a community resource. The sign-up link and blog post are in the thread below.
the models are getting commoditized and it's the act of the application layer.
chat is incredibly boring. i think more use cases will span from where we are at now and this is where the actual usefulness starts.
I didn't cover Claude Opus 4.8 on my pod because I don't think it's MEANINGFULLY better than GPT 5.5 as of May 29th.
We're entering the era where model releases start to feel like iPhone releases. Remember when every new iPhone was a genuine leap? Now it's a slightly better camera and you can't really tell the difference. That's where models are heading. 4.6 to 4.7 to 4.8. Each one is a little different. Nobody can agree if it's better or worse. The benchmarks say one thing, the vibes say another.
The thing that actually matters right now is what's happening around the models. Claude Code shipped dynamic workflows this same week and that genuinely changes what one person can build.
Codex shipped a desktop app with an in app browser that combines coding and knowledge work in one surface. Those are the releases that move the needle for people. The model underneath is becoming interchangeable.
I think we're maybe 6 months from nobody caring which model they're using the way nobody cares which engine is in their Uber. You just want to get where you're going.
When something genuinely changes the game for builders, I'll cover it on @startupideaspod. Opus 4.8 wasn't that. Dynamic workflows was.
I'd rather save you the hour.
the infra for this is crazy. we were running thousands of agents at the same time
snats retweeted
We ran an experiment to see if LLMs could help map, identify, and prioritize unmet medical needs across thousands of diseases. I've found it pretty educational to browse through; I've worked in biopharma for a while now and there are still so many big problems I've never heard of
Today we're publishing our index of unmet needs in human disease: 2443 indications scored and ranked on burden of disease, prevalence, pipeline activity, and treatment burden. We hope this will help drug developers identify overlooked medical problems
convoke.bio/blog/introducing…
technical writing changed my life and has opened so many doors to me. I haven't cracked the always viral door. but I'm at good enough to get multiple job offers.
"Technical writing completely changed my life." -
@trq212
In less than 2 years, Thariq (@AnthropicAI) cracked the code on writing technical articles that consistently pass 1M+ views.
In this 15-min workshop, he breaks down:
→ his exact writing workflow
→ tactics behind articles that go viral
→ how he leverages AI to write faster (without losing his voice)
→ why technical writing is the most underrated way to drive mindshare
Technical writing is one of the most powerful (and completely free) ways to gain views, build authority, and teach the world what you know.
This is the 4th [Technical] Write & Learn, a curated workshop series cohosted with @swyx and @KernelLabs_ai.
give it a year. I think full saturation in about two
Some things never change. If you don’t understand this one, you don’t understand what’s happening AI.
Marcus, 1998: neural nets have trouble generalizing far beyond the data.
Marcus, 2001, 2012, 2019, 2022, etc: neural nets have trouble generalizing far beyond the data.
Apple, 2025: neural nets have trouble generalizing far beyond the data.
Meta/Stanford/Harvard, 2026: neural nets have trouble generalizing far beyond the data.
