You wouldn't believe the hours I spent manually testing and verifying Omarchy releases during the first year. With Prime's stellar work leading the QA effort, we can finally replace all that toil with agents 🤘
ThePrimeagen
@ThePrimeagenOct 19
Big Omarchy QA News: The Headline: * Omarchy QA will start running against nightly every day. Still some huge hurdles to figure out, but that is the fun stuff. The Meat: Omarchy QA is going slower than I hoped for, but its getting into place. This is my second fully running agent test harness where they decide what is good. * A nice running dashboard with cancellable states * Jobs, suites, servers: have all the possible states and transitions you would expect from build servers. This part was the most surprising as it is really in depth how much effort you have to spend thinking about every possibility * suite states, live updating website that ... runs fast? Yes I did something crazy... I made a cache, i know, actual crime... ** I even have an idea to make this load instant for every single user, and make updating cache contents LIGHTNING fast... *** Fun side note. WTF ARE YOU DOING EVERYONE, WEBSITES ARE FUN TO MAKE FAST AND YOU ALL ARE MAKING HORESHIT SITES... STOP. * ability to follow live running jobs and cancel them The sad part is I have had no time to get the actual custom harness running fast, which is ok, but its hard. Now, starting today, I am going to be hooking up nightly builds and actually running this against production! Will think about how to make this a public site, but for now, its private :(

Oct 19, 2026 · 3:51 PM UTC

105
51
9
2,197
110,548
RelevantRecentLikes
THE GOAL I severely underestimated how hard it is going to be (the root of any good project), but we are getting there!
7
138
5,740
I’m working on a desktop environment on top of XLibre, with similar features to Hyprland. Since it’s on XLibre I running builds in a VM/qemu and agents can ssh in with X screen sharing and screenshot the desktop and agents then analyze the screenshots. Basically no human to test all the visual and windowing functionality.
4
728
Primagents*
116
running omarchy here. release qa is a good job for agents, repetitive and easy to check
9
wait wait wait, primeagen is now primeage-qa-n?
1
537
Those hours weren't wasted though… that's where you learn what 'good' looks like, which is the bit you have to hand over to the agents IMO
2
908
From manual to machine, the fitting fate of all reproducible toil. The Omarchy train now runs its own inspections. Brake inspections optional.
2
420
We need omarchy bench open sourced for the labs.
80
Our SDLC is using agents testing whole E2E due to its non-deterministic nature! What a relief it have been for our release process!
710
the agents passed qa. qa was also the agents
353
Use a ladder (jk but basically)
Akhil Gudesa
@vForVicennialOct 8
'Ask another model' isn't one feature. My agents climb a ladder: review → found a High-severity bug blind → exposed a race disagree? → a panel settles it UI change? → a browser agent clicks through it 409 second opinions in 5 weeks. 70% cost $0. How: It's one MCP server. Claude Code and Codex call it before a risky step, and it always asks the other model family.
2
778
Agents doing the release QA grind is the best use of them I've seen... are they driving the real desktop (Hyprland, keybinds, menus) or mostly scripted checks? Daily Omarchy user, super curious 🤘
1
477
Go go go!! the goal is difficult, but reachable
53
Can't wait for the first agent qa report that just says "works on my machine" 💀
3
467
Prime's on it big time
1
313
Toil = now a portmanteau for tin foil (hat) 😄
6
Manual release testing is the worst kind of toil, glad agents can finally eat it 🙏
4
185
the toil finally ends
31
nightly runs catching bugs you'd have missed is worth every hurdle
16
I would believe it.
43
Release QA is such a good fit for agents. Boring, repeatable, and you know exactly what a pass looks like. Curious if they catch the weird hardware edge cases too or mostly the happy path.
73
A year manually testing Linux. Even the agents are going to request a Mac.
69
Anthony Russano
@russanoOct 19
This is the correct use of agents. Glad Prime’s building this out... the stress reduction should be massive once it's fully live.
106
Release day already has enough going on without repeating every check.
16
Release QA is a great job for agents, the trick is making them fail loud. Currently dealing with this on my own kanban work where agents pickup takss. Does the agent ever come back with "not sure" or is it strictly pass/fail?
130
rakesh
@SublimeCoderXOct 19
release qa is the perfect agent job... right up until it decides a flaky test is "probably fine"
100
the agents passed every test. the agents also wrote the tests.
175
The interesting line in Prime's thread is "agents decide what is good." I'd keep one split strict: agents can run the suite every night and block a release on their own, but shipping a build should still take a human yes. A wrongly blocked release costs you a few hours. A bad build in users' hands is much harder to take back. Is Omarchy QA allowed to stop a release yet?
1
8
Manual testing is such an endless pain. I’ve spent nights staying up and making sure everything works before each release of @OmarchyMac Prime’s harness will be a relief for us!
1
163
Same! We built a fake Dutch heating company and let agents use it like real customers before we ship
27
if agents retry failed boots until they pass, nightly qa hides the bugs users hit
13
Let's Gooooo!
76
agents testing the nightly while you sleep is the dream. curious how they catch the "technically works but looks cursed" bugs, that's where all my testing dies
1
118
The hand testing wasn't wasted. It's where you learned what "working" actually means for Omarchy. Write that down before the agents take over, because the checklist in your head is the real QA spec. Which check would you still never hand off?
1
1
124
If Omarchy is good because a human evaluated the progress, why do we assume it will get better when agents evaluate the process?
49
> Still some huge hurdles to figure out That's the intelligence part. LLMs are the automatons that don't stop and follow.
26
Load more replies