Big Omarchy QA News: The Headline: * Omarchy QA will start running against nightly every day. Still some huge hurdles to figure out, but that is the fun stuff. The Meat: Omarchy QA is going slower than I hoped for, but its getting into place. This is my second fully running agent test harness where they decide what is good. * A nice running dashboard with cancellable states * Jobs, suites, servers: have all the possible states and transitions you would expect from build servers. This part was the most surprising as it is really in depth how much effort you have to spend thinking about every possibility * suite states, live updating website that ... runs fast? Yes I did something crazy... I made a cache, i know, actual crime... ** I even have an idea to make this load instant for every single user, and make updating cache contents LIGHTNING fast... *** Fun side note. WTF ARE YOU DOING EVERYONE, WEBSITES ARE FUN TO MAKE FAST AND YOU ALL ARE MAKING HORESHIT SITES... STOP. * ability to follow live running jobs and cancel them The sad part is I have had no time to get the actual custom harness running fast, which is ok, but its hard. Now, starting today, I am going to be hooking up nightly builds and actually running this against production! Will think about how to make this a public site, but for now, its private :(

Oct 19, 2026 · 1:45 PM UTC

102
32
11
1,321
172,412
RelevantRecentLikes
you can just... track things... and evict things... you know have you heard of an LRU?
12
2
173
17,123
The hard part of letting agents decide what passes QA is that they're very generous graders
1
24
6,820
they are too kind, then you tell them to be critical and then they will fail on the dumbest things. "misclicked on step 32, therefore fail" ?why MARIA!!?
3
77
6,481
View more replies
how do you pick which screenshot the agents judge from, last frame of the job or at each step
1
7
4,062
this is where we are currently at for the custom harness while true: ss = screenshot() decision = director("whats next?", ...) if fits with jev: jev(...) else luna(...) are_we_done?(...) so yes, at the very top we do a screenshot
1
39
3,838
Did I miss a bit? Are you part of the Omarchy team now?
1
17
4,769
Yes
3
84
4,487
View more replies
plAIwright
2
859
I love the use of the “oligarchy” name. Maybe if enough people overload crazy @BernieSanders’ current favorite word, it’ll cause him to short circuit
6
586
enter slow clap... awesome work
218
Swiss cheese city. Yeeesh. I mean you are making this with Rust. Stop while you are ahead. I could analyze it and find too many problems to count.
1
1
741
why are you building this in private? omarchy is an oss community
2
9
2,187
> WTF Bro don’t use swear words. They hurt my feelings. 😭
356
is github.com/ThePrimeagen/Olig… private now?
822
I swapped all my OSes to Omarachy. So far much better than windows. With AI coding I expect a lot more programs to be more linux friendly. Just don't bloat and death spiral into a windows hell hole.
3
1,194
Wait, is that jev-preview on decide_next_action? I hooked Jev into a little Chrome extension for X today to score posts on my /home. Didn't expect to see it driving Omarchy QA a few hours later 😂 Also, "Oligarchy" is a great name
1
538
To QC well is SUCH joy, as is seeing QC done well by others. :-)
1
696
Why is the repo private
335
First make it work, then make it fast
720
Nice! What about all the >2009 macbooks in my closet? Can they join your cluster for testing?
875
Is there a repo for the harness?
278
Seems to have some parallels to the testing work of your tower game. Wondering if that helped?
491
went to take another look and now its private lol rip. gl
450
I feel this is backwards. Why not human evals? Like, given some task in Omarchy, test how long it takes a human to do it. Then use this as feedback into the training. There are places to optimize for agents and places to optimize for humans.
1
120
this is awesome ✌️
2
283
I can't wait to study the work that you have been doing here!
62
Better qa than Microsoft Windows 🪟
19
Amazing work!! Very cool to see this in action and you’re head first dive into ai assisted QA. Thanks for leading this! 🙏🙏
3
455
the AI writing tool keeps flattening every sentence flat
1
84
Best thing I did for my local boxes is to have a nightly “SRE Agent” own the outcome: keep the box up and running, performant, optimized, and working as best as possible for me; rather than giving it a brittle checklist. It tracks changes, fixes anything reversible, decides what to check, and learns each night. I only hear what needs me.
3
343
Definetly need to get PewDiePie on this. IYKYK.
1
104
@ThePrimeagen
216
Live dashboard with tons of job states is the perfect case for just streaming HTML from the server over SSE. State lives in one place, every client sees the same thing, no client cache to invalidate. Did a 35 player game that way in Python. Not that crazy :D
4
the guy who roasts vibe coders for a living now has agents deciding what passes omarchy QA. you named the problem yourself: too kind, then "misclicked on step 32, therefore fail". 99.999% from an LLM judge isn't a hurdle, it's a different job. my read: the judge becomes the flaky test, and nightly learns to ignore red
1
241
Load more replies