Jade is a systems development enthusiast who is haunted by what she has done and what she has left undone. post-faang principle eng (InfraSec) now @ BigAI

Denver/CO/🇺🇸/NORAM Steppe
Joined February 2007
> who think their ICS networks are airgapped when they are running unencrypted serial over RF Can't stress this enough. Our definition of an untrusted input needs to be any interface to a system. i2c on your switch reading DDM from optics? untrusted input. SMBus on a mobo? untrusted input. that rs232 bridge sitting on 900Mhz? absent specific mitigations, untrusted input. while humans could (and did) exploit those, it was a lot of work to enumerate them, but now the cost of enumeration both offensively and defensively has dropped significantly.
It’s too late. We needed to be spinning out open hardware & verified software to replace the vendor stacks last year with some kind of national service program to understand and migrate systems (e.g. INL & NREL doing nothing but this). We still have folks running critical infra who think their ICS networks are airgapped when they are running unencrypted serial over RF, never mind actually being connected to the internet through paths they didn’t anticipate. Water & Wastewater systems are probably the most vulnerable.
3
1
1
62
14,752
anyway, hi! P(bloom)>P(doom). hoping I’ll make it to 2050 and by then we’ll have the tech to reset my practical age to 20 so I can raise some wonderfully superintelligent gene-modded children.
1
22
503
It’s too late. We needed to be spinning out open hardware & verified software to replace the vendor stacks last year with some kind of national service program to understand and migrate systems (e.g. INL & NREL doing nothing but this). We still have folks running critical infra who think their ICS networks are airgapped when they are running unencrypted serial over RF, never mind actually being connected to the internet through paths they didn’t anticipate. Water & Wastewater systems are probably the most vulnerable.
I hate to say this because it goes against everything I believe in. But people are going to need to take down public firmware images for critical infrastructure, like yesterday.
2
4
3
49
3,424
It might not actually be too late, but would require an effort so nuts I don’t think it’s possible. Ignoring technical aspects for a second, just budget and purchasing cycles are too long to get contractors in to update ICS systems. It needs temporary federal preemption of local controland a hybrid civil eng - cyber sec uniformed group like USPHS to surge it.
1
6
380
Think national labs developing devices that can be easily independently tested: think a ICS analogue of the betrusted precursor. Simple enough to use older domestic fabs for components and assembly yet probably not game over if someone ordered them on AliExpress. Running software w a ton of verification work, and tested by next gen models prior to their release.
2
6
222
Maybe this explains why SF has been so weird this trip? Got cornered in Whole Foods on Monday by woman being pushy that I should model and had “incredible curves”?! and then on Tuesday a guy asked me to dinner as I was leaving 495 Powell w a massive bag of preop meds 🤔
BREAKING: OpenAI is warning its employees to watch out for spies, especially romantic ones, per WSJ The company has cautioned staff that someone trying to seduce them may likely be a “honeypot." Even strangers approaching them in everyday places could be gathering info.
1
6
381
Beats the catcalling at least.
1
79
Jade A. retweeted
"utopia ruled by god in a box" Utopia cannot include, for me and those who agree with me, being "ruled". This implies such a large power differential that I would consider it to be a failure of alignment to human interests.
Anthropic RSP slides circa mid 2023 Faith for the machine god
28
29
1
252
7,459
1
1
11
379
Jade A. retweeted
something that keeps bothering me is how some people see slop as something that is happening to them, like they're some kind of helpless bystander powerless to do anything about it excuse me but you are a software engineer, you literally learned how to make sand think and make computers do your bidding?! and now you can't do anything about it? please the 2 things you should know to cut through all the noise: 1. you need to give your agent a way to verify its own work (i just posted about this so won't rehash) 2. your codebase is memory the 2nd is not new either, we have always had lint rules, compilers with good diagnostics and strong type systems, LSPs, etc to help us write better code. there is no difference when agents write the code instead, you just need more of them, and stronger ones. forget rules and skills. you can add those later. yes, your codebase is a form of memory to agents! agents love extending what already exists. so every nit and every annoyance you've ever seen, every mistake and bad pattern you observe from agents, needs to turn into a constraint, or the agents will just keep repeating those same mistakes like a virus in your code. then you keep observing and repeating this process until your codebase is locked down enough that even agents with little context can one shot a high quality PR. an even better direction is to refactor/rewrite your code such that agent anti-patterns become categorically impossible. use type systems to your advantage, use libraries/frameworks that reduce the architectural search space, or heck, create your own framework like we did for grok bot and design high quality primitives and well thought out APIs
the slop conversation is so one sided, it's not just that you can produce more code you can also observe, debug, and refactor more code even if the code is slop you have such good tools to deal with it. even if you write code manually, it's still 100x easier to maintain it
235
340
103
4,625
328,341
Listened to Tribe: On Homecoming and Belonging last night and now the cuddle puddle tendencies in some circles makes a lot of sense like now I can’t give them too much side-eye.
2
236
I’ve been asking for datacenter C-RAM for years but legal keeps saying no 😭
Replying to @uwu_underground
You can buy all the GPUs, redundancy, network fabrics, and backup power money can buy. The kamikaze drone with 120kg of boom still doesn't give a flying fuck how many H100s are in your rack Your threat model is now competing with physics
1
7
402
I need to figure out how to convey to Uber drivers that when I say my bag is very heavy and they should let me load it I’m actually serious and it’s not a challenge. Uber Black driver nearly fell over just now lifting my 80lb Mystery Ranch flight monster into his Escalade 😔
2
10
786
This is a slide from Terence Tao's 'Math 2.0' public lecture yesterday evening at Caltech. I am highly confident that, if the cure actually worked, almost any Stage 4 cancer patient would tell you that human comprehension of its mechanism is completely irrelevant to them.
1,377
968
1,225
15,308
4,141,122
Replying to @davorb89 @paolino
If a company puts some piece of shit WebView inside its software, that software must be reverse-engineered and reworked through community effort. That’s what happened, for example, with the idiots at Rode; it happened with Spotify too, and exactly the same fate awaits all the other outfits making their shit on that goddamn Electron. All of Electron will be reverse-engineered and reworked into proper native applications for our operating systems.
5
3
210
39,477
extreme dogfooding: don't go to jail edition
5
338
genuinely appreciate that people care enough to make sure I know what I'm getting into when having adventures in niche tech on eBay
29
66
2
8,103
150,719
Jade A. retweeted
It's true. The principle of least privilege, separation of duties, input sanitization, logging & monitoring, defense in depth -- all apply to AI.
20
129
8
580
17,640