hacker, engineer, musician. building @codewiresh local and vm orchestration system.
Joined August 2006
- Tweets278
- Following2.9K
- Followers5.5K
- Likes12.3K
If you leave California for Florida, it is quite possible the only reason is you love Disney World more than Disney Land, because there is no other reason.
Noel retweeted
It is a weird weird year to be a formal methods expert.
Some unorganized thoughts on the new @AnthropicAI and @theoremlabs post on formal verification (anthropic.com/research/forma…):
- This is very cool.
- The main innovation is verifying a later stage of the program. Normally provers take the source code, LLM it into a theorem language, and then prove that, which means you have to make assumptions about source semantics and compilation properties. It's very easy to make a mistake in translating and prove the wrong thing!
What they instead do is compile the code into assembly, and then (if I understand this correctly), compile that assembly into a theorem language (which they call LPS). So as long as the compiler is faithful you can skip the whole divergence problem.
- I'm glad they clarify that you still have to trust the translation process, the "hardware, OS, OCaml" compiler, the LPS model, and also assume that the program is single threaded with fixed memory. The single biggest source of errors in formally verified code is when the proof assumes something that isn't true in practice. It's really, really important to state your assumptions!
- I'm not clear why they need to compile the fuzzer into the theorem. Is it to act as a description of valid inputs and workload? Maybe to embed assertions in as theorems?
- Holy crap, 180 million lines of Rocq. Is that more Rocq than exists in the world already? I notice they say a 600,000 line proof corresponds to roughly 20k CPU instructions. That would be like, 1-2k lines of source code at most? Really goes to show how inefficient this approach is and how it only works because you can throw LLMs at the problem.
- This, probably deliberately, doesn't address the single biggest gap in AI-based verification: AIs cannot come up with good properties. Doesn't matter if you're doing unit testing, property testing, formal verification. I always assumed this was a "frontier labs haven't trained for it yet" thing, but if Anthropic hasn't talked about that here, maybe they've tried and haven't been successful yet. That's interesting to me.
Instead, they take a known universal property (this code doesn't crash or get flagged by UBsan) and prove that instead. Still very useful, but makes it harder to generalize to proving arbitrary things about arbitrary programs.
- Anybody reading the official post should also read their expert review appendix (www-cdn.anthropic.com/files/…). It points out some interesting things, like the fact that most of the code the LLM wrote wasn't used in the proof (over one billion lines of Rocq), that they had to prove the equivalence between fuzzed and unfuzzed programs, and (this is most interesting), some experiments find that the LLM-method of proofwriting may not scale to more complex things.
- Also read the description of LPS (github.com/theorem-labs/oss-…) to understand the limitations of the assembly -> Rocq compilation. In particular, it does not handle concurrency at all (which is one of the main use cases for formal methods!) or OS signals. Also it can't prove things about program arguments with slash characters, meaning they can prove stuff about `rm foo` but not `rm foo/bar`.
- The official skill (github.com/theorem-labs/oss-…) doesn't tell us much about the nature of FM and AI. Probably too much for me to hope of any skill.
- Some of the article's specific caveats and throat clearings make me think someone at Anthropic reads my stuff ( ͡° ͜ʖ ͡°)
Noel retweeted
what a weird week
Several Linux Kernel Vulnerabilities Accidentally Posted To Public Kernel Mailing List
As high as CVSS 9.0. Whoops.
phoronix.com/news/LKML-New-T…
Noel retweeted
In 2025, we published a bold proposal to use fusion reactors for gold production.
And now we’ve done it: Mercury transmuted into stable gold, using fusion neutrons, at 10x the record set by CERN in 2025.
We’ve reduced alchemy to a series of difficult but tractable engineering problems.
Noel retweeted
A San Francisco startup says it just turned mercury into real gold, and their own math says gold is one of the cheapest things their machine can make. Aim it at a different metal and it makes a cancer drug ingredient worth tens of billions of dollars a gram.
Here's how it works. When you fuse two kinds of hydrogen, deuterium and tritium, every single reaction kicks out a neutron moving at 14.1 MeV, which is absurdly fast. Fire one into mercury-198 and it knocks another neutron loose on the way through, so two come out and you're left with mercury-197. Give that a few days and it decays into gold-197, the only stable kind of gold there is. The whole trick needs neutrons above about 6 MeV, and regular nuclear reactors barely make those, while fusion makes one every time it fires.
Now swap the mercury for radium-226. Same reaction, one neutron in and two out, and you end up with radium-225, which decays into actinium-225. Novartis is building its next prostate cancer drug around actinium-225, and it's so scarce it sells by the millicurie, roughly $1,200 to $2,500 apiece. Price it by the gram and you're into the tens of billions of dollars, for a material so radioactive nobody could safely hold a gram of it anyway.
Marathon's team actually put a dollar value on a single fusion neutron in a paper this year. Use it to make electricity and it's worth about $3 × 10⁻²⁰. Gold comes in at $2.6 × 10⁻²⁰, basically a wash. Molybdenum-99, which goes into about 80% of nuclear medicine scans, is worth $2.6 × 10⁻¹⁷ a neutron. Actinium-225 is worth about $10⁻¹⁰, around 3 billion times what the electricity's worth.
That's why fusion people should care. Every fusion company you've heard of is chasing electricity, and selling electricity only pays once the plasma puts out something like 10 to 22 times the energy you pump in to heat it. The best result so far, at JET in England, was about a third. Add gold on the side and Marathon's math says you break even around 3x. For molybdenum-99, the number is 0.001x.
So a machine that gives back one-thousandth of the energy you feed it can still make money, as long as the neutrons hit the right metal.
By their numbers, a 3 megawatt machine running way below breakeven could cover the whole world's supply of molybdenum-99, which today comes from splitting uranium in a handful of research reactors. Today's experimental fusion machines already beat the gain that job needs.
Their second post basically says this out loud. Marathon now describes itself as an isotope company for radiopharma. Their chart shows roughly 390 targeted radiotherapy trials. Pluvicto, the prostate cancer drug Novartis launched at $42,500 a dose, did $1.9 billion in sales last year.
Gold still matters to them, just later. It's about a $700 billion a year market, and Marathon's paper figures it could soak up around 2 terawatts of fusion capacity, which is roughly the scale where fusion actually moves the grid. So the plan is to sell cancer isotopes out of small machines now and use gold to pay for the giant ones, years before electricity can pay for itself.
Last month, we demonstrated high-rate alchemy: turning mercury into real, stable gold, beating CERN’s record by a factor of 10.
Last week, we enriched isotopes in the world’s most powerful plasma centrifuge, beating the Soviet record that stood for 40 years.
Now, we’re building a fusion reactor in San Francisco to achieve record power density.
Noel retweeted
So basically, today PHP got two critical bugs that let anyone run code on your server without even logging in, which is bad, and then the details leaked before the patch was even out, which is worse, and it works on WordPress and MediaWiki and Magento and Joomla and Laravel, and Censys says like 1.5 million servers are running a version with no fix, so the NSA was like "take your system offline," and the FBI was like "take your system offline," and CISA and the UK and CERT-EU were all like "take your system offline," and then the Prime Minister of Japan, the actual Prime Minister, posted at 5:55 in the morning telling 48.8 million people to take their system offline, so everyone took their system offline, like the Dutch cybersecurity agency took its own website offline, and the Dutch parliament went offline, and the Port of Rotterdam went offline, and Finland and France and Denmark started pulling government sites, and Wikipedia went down, and Archive.org went down, and Asana and GOG and Something Awful went down, and the University of Vienna's webmail went down for some reason, and that's just PHP. Because GitLab also dropped a CVSS 10.0, which is the highest score they give out, for a bug that's been there since 2014, so that's every self-hosted GitLab for twelve years, and then LiteLLM 1.107.0 got hijacked and shipped a credential stealer and a remote access trojan in every single wheel and the Docker images, including latest, and you didn't even have to import it, just installing it was enough, and it's the second time this year, and then Railway paused every deployment on the entire platform because someone got into the environment variables, so now everyone has to rotate every secret they've ever stored there, and TiDB Cloud said someone was walking around inside with stolen credentials for basically a month and an extortion group says they took 10.6 terabytes, and Zeabur got breached for the second time in two months and is just shutting down, like the whole company, and Node.js patched remote code execution and a TLS bypass, and CISA says Plex is being actively exploited, and someone took over 300 Tor exit relays to steal SSH passwords, and SafePay ransomware has hit over 600 critical infrastructure orgs, and India's biggest bank filed a data loss incident, and Japan has been in an official state of emergency in cyberspace since October 9, which honestly explains the 5:55 AM post. Oh, and eMudhra, a certificate authority that's issued over 60 million certificates, whose entire job is proving websites are who they say they are, was supposed to be using private keys that are random numbers over a hundred digits long, and one of them was 4. Just 4. And the others were like 66 and 237. Anyway, it's Wednesday.
How fast does a PR's CI in your project typically take?
What about main branch CI/CD?
What's the speed you're at today, and the speed you want to get to?
Noel retweeted
Everyone talks about what AI will do. But almost nobody asks who will pay for the machines. Taking advantage of AI will require an extraordinary amount of capital, from data centers to new power plants. Will that capital be around?
I try to address that question in the first draft of my new paper “Saving for the Machines.”
To show how important capital is to fuel the transition to an AI-based economy, I document how the same technology path that generates 11.7% annual growth when households choose their saving rate optimally delivers only 2.2% when the saving rate remains fixed. In other words, if we experience slow growth over the next few years despite fast improvements in AI, this slow growth may reflect too little capital accumulation instead of a technological bottleneck.
The key to this result is that AI raises the capital share by moving tasks from labor to machines, which requires that those machines are built. In comparison, ordinary technical progress makes labor and existing machines better at the tasks they already perform.
To inspect the mechanism, I start from an accounting identity: the saving rate is the capital share times the ratio of investment to capital income. If I take the evolution of the capital share as given, I can ask how much the latter ratio moves in response to that evolution. Crucially, with an optimizing household, this ratio barely moves. As machines take over tasks and the return to capital rises, the household responds by letting consumption grow faster, by the intertemporal elasticity of substitution times the rise in the return. To pay for that faster consumption, capital must also grow faster by the same amount. So, the household keeps investing about the same fraction of what the machines earn. And since machines earn a larger share of output, the saving rate also goes up. With a fixed saving rate, the same accounting identity tells me the opposite: the ratio falls in proportion as the capital share rises.
Will the U.S. and other advanced economies then generate enough capital? For many reasons, I doubt it. We might end up in a situation where many of the opportunities offered by AI are left on the table. This is particularly important for all who think that AI might get us out of many problems, such as fiscal troubles (like in France) or ultra-low fertility. We need to save for the machines, and it is not clear we can do that.
Anyway, this paper has been long in the making (many months of hard work), but I have learned much from it. In particular, working through the model changed my mind about many issues related to AI. Again and again, my intuition said X, but the math said Y. And math always won in the end. That is why we write formal models.
A nice example is Proposition 5, whose proof is the one I like the most in the paper. With improving machines and strong task complementarity, there is no fixed balanced growth path because the target itself keeps moving. The nice part of the proof is to realize that, after the right normalization, the moving target becomes a fixed point, and the saddle gets steeper as machines improve. So instead of falling behind, the economy tracks the moving BGP more and more closely. A fair amount of ODE work to get there, but a very simple economic idea underneath it once you get it.
The paper is here:
sas.upenn.edu/~jesusfv/Savin…
and the slides are here:
sas.upenn.edu/~jesusfv/Slide…
Noel retweeted
For your listening pleasure: 17 new tracks of instrumental music pulling from jazz-funk, Italian movie soundtracks, electronica, hip-hop, art rock and beyond.
Now Playing: “Side Two” zeldman.com/2026/10/19/now-p… via @zeldman
Noel retweeted
When I was writing Unreal Engine 1, my time was divided as follows:
- Around 30% went to seriously deep coding on hard problems: assembly language texture mappers, C++ BSP trees, data persistence, object oriented systems design, and networking.
- Around 70% was just typing and refactoring large bodies of code needed for a complete engine feature set. This was not hard, and while I was doing it my mind was mostly on the deeper coding topics, thinking and planning for what's next. It didn't feel much different from vibe coding today, except that it was just me typing code on an IBM Model M keyboard rather than prompting it and reviewing the results.
So, it has always seemed perfectly logical that we would rely on AI for that 70%, once the AI could do it with a similar quality standard as we could, which has never been perfection. It came as no surprise that, after a few years of AI model evolution, it was able to take the reigns on that 70% and free up programmers' time for more valuable tasks.
What's more of a surprise is the emergence of AI as a sort of insta-Carmack able to solve hard technical problems quite cleverly and often cleanly, and that it's able to reason about and work in complex 3D scenes both by operating the tools and by directly understanding the raw data files.
I abandoned assembly language in 1998 when out-of-order CPUs made much simpler C/C++ code faster, and never I missed it. If prompting through architectural documents, concept art, and design plans can transcend typing code and tweaking vertices, that will enable developers to spend their time on higher-level, higher value technical and creative tasks.
What matters is the end product, whether it's a game or an engine. Anything developers can do to try out more ideas and make better software and games faster is a win for everyone.
Noel retweeted
Here's my take on what a software factory really is
This is my talk from @aiDotEngineer NYC, and I decided to record a version for everyone to watch on X.
tl;dr - software engineering is becoming factory engineering, and there's new work (especially factory benchmarking) that engineers should start focusing on.
Ever wonder what a software factory is?
xitter.cf/zachlloydtweets/status…
Here's my take on what a software factory really is
This is my talk from @aiDotEngineer NYC, and I decided to record a version for everyone to watch on X.
tl;dr - software engineering is becoming factory engineering, and there's new work (especially factory benchmarking) that engineers should start focusing on.
The hackers have won again. It's been several days now in a row that I've been to the grocery store in Amsterdam, and shelves have been much more empty than usual. For the past two days it's been hard to find fresh vegetables. And the sign says it all.
Locally for voice transcription, I have switched from Cohere to Phonon.
Transcription is insanely fast. Short clips went from 1s to 16ms, and 128 second clips from 17s to 350ms.
Rumor is Fermion is just 1 dev. Maybe wrong, but seems like it.
fermionresearch.com/research…
"when it is to my advantage, i will converse with the common folk."
1/2 Thanks Gavin for an especially thoughtful exchange. I don't usually spend much time on social media but I wanted to engage here because it really brings out the heart of an important conversation.
First, on regulation, I think that “either concentrate it in the hands of a chosen few companies and politicians via regulation or distribute it widely” is a false choice. I know that there’s a sort of Silicon Valley shorthand where regulation = regulatory capture = concentration of power, but I’ve always found this to be an overly simplified picture of the world. Many people outside this bubble think of regulation as something that constrains corporate power and benefits ordinary people. I don’t necessarily agree with that perspective either, rather I think it’s complicated and really depends on what the “regulation” consists of. But in particular I think that those in the “regulation = regulatory capture = concentration of power” frame often underrate the decentralizing power of objective and fair institutional processes. A crude analogy is that the formal court system can sometimes feel stuffy and elitist, but it does a much better job of defending the rights of vulnerable individuals than the alternative, mob justice. At their best, institutions can vest power in ideas rather than people, and thereby decentralize that power.
This is why Anthropic has always made its policy proposals very carefully. We try very hard to make proposals that disadvantage (slow down) frontier AI companies while *advantaging* smaller competitors. California’s SB53 (which we supported), and even the much-maligned SB 1047 (which we were ambivalent on), completely exempt any company below a certain amount of revenue or model training costs from being covered at all (it was $500M for SB 53, lower for 1047 but we objected to that). More recently the testing process we’ve advocated for at CAISI and the White House involves more rigorous tests for frontier models than off-frontier models — something that differentially advantages challengers. Similarly, the “Pacing the Frontier” letter envisions (or at least Anthropic’s preferred implementation of it envisions) modulating the pace of the very best models while not constraining those who are catching up. This hurts the business interests of the frontier labs and helps challengers, including open-weights!
Overall my view is that AI is *structurally* a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws). Open-weights do help some with this but are nowhere near a sufficient solution because they simply shift the concentration somewhat to those with the most compute and chips (which are roughly the frontier labs plus maybe hardware providers). By contrast I think the right “rules of the road” can simultaneously (a) address AI’s cyber/bio/alignment risks, (b) institutionally constrain the power of the frontier AI companies, and (c) leave room for open-weights models while also addressing the specific risks that they bring.
BTW I do not think that the events of the last few months have “failed to result in [my] preferred regulatory path”. The approach that the Trump administration is reported to be taking — pre-deployment testing for frontier models, and also testing of open-weights models when they get closer to the frontier — is one that I am very supportive of, though of course I have to see the details to be sure. I am also supportive of Demis Hassabis’ ideas around a FINRA-like entity. This contrasts with six months ago when most of the industry was still pushing for preemption of all state regulation and no apparent federal approach either.