Building an ecosystem for AI and people to work together @ subseq Formerly @KernelCo @SpaceX.

The Wired
Joined March 2008
Earth thriving. The stars in reach. By 2100. I mapped a timeline by the decade: Earth's industry on clean power from orbit, CO₂ under 350 ppm, and a ring of solar panels around the Sun. The swarm seed costs about what the world already spends on space.
2
2
9
342
By 2100 the Ring is complete: a band of panels 1 million km wide beside Mercury's orbit, making ten billion times Earth's energy use. Panels between Earth and the Sun turn edge-on, so it never dims us. Mercury's glass shields enough orbital land for 30× Earth's.
1
1
18
We should be dreaming bigger. It'll all soon be possible.
1
49
We have proven a theorem that maintains its alignment guarantees regardless of agent capability. This is huge, because alignment becomes a structural property of the agent's deployment even if it is superintelligent and is maintainable through RSI without a capability arms race.
1
1
1
6
463
The ledger isn't an accessory validator, it's required by the math. Human + AI alone share an adversarial surface: a sufficiently capable system breaks both human judgment and AI verification at once. The deployment safety requires another system with independent failure modes.
1
2
62
Full paper: teague.info/papers/provably-… and a companion paper containing additional proof machinery teague.info/papers/safety-fo… The entire 12-paper sequence can be found at teague.info
4
35
"Exogenous Verification for Alignment" The argument is as follows: it doesn't matter if alignment produces well-specified and generalizable goals if it cannot be verified. If an agent can produce endogenous rewards it can control everything about its own rewards. This goes beyond wire-heading, even an alignment framework like GFM that on paper creates exogenous rewards can be gamed by the agent by introducing phantom verifiers that are still, functionally, endogenous. Thus we introduce a system of cryptographic commitments that enforce the exogenous verifiability of reward signals. This closes the verification gap in more than just GFM: any alignment framework will need a way of enforcing that reward signals for a highly-capable agent must be produced exogenously.
1
1
2
138
I think I just saw Claude get exited. I made a restrained comment on the new paper we're working on and it jumped straight into "These things will build entire civilizations!"
1
3
95
1. The metric rewards meta-capabilities. 2. Other agents with high-leverage capabilities are valuable. 3. Cooperation with high-leverage agents is disproportionately rewarded. 4. This creates a natural clustering dynamic. 5. The clustering is civilization-building.
56