I help maintain tree-sitter / To whom much is given, much will be required
/home/your_mom
Joined March 2024
- Tweets542
- Following122
- Followers58
- Likes39.9K
Lillis retweeted
I've published a new, general-purpose terminal specification that gives any program a way to tell the terminal what it's doing: idle, working, waiting, finished, or failed, and why. It is easy to implement on both sides. I've written more about why: mitchellh.com/writing/progra…
The problem this solves is generic and applies in compelling ways to all sorts of use cases (e.g. package managers, build tools), but it really rears a hideous head when it comes to AI and the "agentic inbox" problem.
I found over 250 different agent orchestrators (with various goals) that all individually implement a heuristic based approach to detecting whether things like Claude Code are working, blocked, done, etc.
If you look at @herdrdev's commit history, you can see around 10 compatibility changes to detect Claude Code within just the past 3 months. (This isn't a dig at Herdr! They're doing great work with what they have, and everyone else is doing the same thing).
Heuristics are not the way. And over-indexing on agentic use cases isn't the way, because this problem is really interesting for other tools too: Homebrew, Terraform, Cargo, etc. etc.
Proprietary protocols and out of band APIs are not the way. They have an O(N) integration problem and require extra work to work over SSH or within local VMs/containers.
The program status specification (OSC 7501) solves this in a reliable, deterministic way that is general-purpose and can be easily adopted by the entire industry.
Read more: mitchellh.com/writing/progra…
Lillis retweeted
Interestingly, Astra is wrong about the UB because it is missing historical context on when this legendary fast inverse square root code was written:
1) This code was written in 1999 for 32 bit computers when practically `sizeof(long) == sizeof(float) == 4` so the "OOB on 64-bit longs" and "possible misalignment" does not apply here
2) In 1999, aliasing UB (aka type punning) was technically UB but it was literally everywhere and even the Linux kernel did it for its memcpy implementation. When GCC 2.95 came out it added optimizations that optimized against type punning and it broke so much software they had to default disable it a few months later!
The harshest realisation for me after we’ve got LLMs is that there is no such thing as “good code”. Everybody’s code sucks. You can always take seemly good code and find bugs and improve the design, and this process doesn’t seem to converge. The goal I was chasing my whole career turned out to be a marriage: the closer you get, the further it appears. But LLMs allowed me to prove this empirically, so now I’m more at peace with my code. If it works after sufficient testing, I leave it alone; not because it’s “good” but because it’s good enough.
P.S. I ran Astra on this famous John Carmack’s code and it found five verifiable bugs.
More performance improvements to tree-sitter generate. The master branch is now ~3-5x faster and uses 3-5x less memory than 0.26. More improvements to come!
2
42
If you’re unfortunate enough to work on a submodule-heavy project and are tired of waiting for git status, checkout subspy:
-identical output to git status
-100-1000x faster on Windows
-67-360x faster on Linux
-experimental shell prompt integration
github.com/WillLillis/subspy
Lillis retweeted
5% improvement on peak opus 4.6, sold as a 40% improvement by preemptively nerfing opus
and everyone will just eat it up
Lillis retweeted
LLMs are great tech, and I've learnt a lot about technology that I never would've or had time to thanks to them. But I feel like people are vastly overestimating how good they are for programming, *especially* in contexts where 90% of the time isn't spent accomodating other people's mistakes.
What I mean by that is solving real problems instead of undoing bad transformations of data a previous part of the system did for the 100th time, or inventing something new *and* not total garbage and not restricted by bad interfaces that already exist.
Some people in my audience already know what I'm talking about by all this vagueposting (2-3, maybe 6-7 at most), and it requires questioning everything about computing which LLMs are absolutely horrible at.
And unfortunately, 95+% of the code most people (including me) have written is within the "useless busywork" definition. We might eventually get a world where that isn't true, but for the time being LLMs will be useful for aiding the production of the counter-boilerplate and speeding up well articulated research queries.
Man, computers suck
interprocess uses named pipes on windows which means you can’t set timeouts for reads. How am I supposed to thrive in these conditions
1
98
Lillis retweeted
I think about this Dostoevsky excerpt a lot (Notes from Underground):
Now I ask you: what can be expected of man since he is a being endowed with strange qualities? Shower upon him every earthly blessing, drown him in a sea of happiness, so that nothing but bubbles of bliss can be seen on the surface; give him economic prosperity, such that he should have nothing else to do but sleep, eat cakes and busy himself with the continuation of his species, and even then out of sheer ingratitude, sheer spite, man would play you some nasty trick. He would even risk his cakes and would deliberately desire the most fatal rubbish, the most uneconomical absurdity, simply to introduce into all this positive good sense his fatal fantastic element. It is just his fantastic dreams, his vulgar folly that he will desire to retain, simply in order to prove to himself--as though that were so necessary-- that men still are men and not the keys of a piano, which the laws of nature threaten to control so completely that soon one will be able to desire nothing but by the calendar. And that is not all: even if man really were nothing but a piano-key, even if this were proved to him by natural science and mathematics, even then he would not become reasonable, but would purposely do something perverse out of simple ingratitude, simply to gain his point.
[…]
for the whole work of man really seems to consist in nothing but proving to himself every minute that he is a man and not a piano-key!
[…]
Good heavens, gentlemen, what sort of free will is left when we come to tabulation and arithmetic, when it will all be a case of twice two make four? Twice two makes four without my will. As if free will meant that!
This quoted post is unavailable.
I need to stop trying to work on top of existing mechanisms that doesn’t have a well established protocol. The way git handles lock file acquisition is utterly insane and invites race conditions from external tooling.
> acquire index.lock
> gather information about the state of the repo
> write updated index to index.lock
> delete index
> rename index.lock to index (!!!!)
> do some other operation (create rebase-merge directory if a rebase command was issued)
> hope nobody else touches index during this time (?????)
> try to reacquire index.lock
Why wouldn’t you just hold the lock file for the entire operation? I just want ‘git status’ to not take 30 seconds dawg
44
