bindi_ (@math_on_tap) header
bindi_ (@math_on_tap) profile picture

GPU engineer | math PhD

SF Bay Area
Joined December 2023
Photo by bindi_ (@math_on_tap)Photo by bindi_ (@math_on_tap)Photo by bindi_ (@math_on_tap)Photo by bindi_ (@math_on_tap)Photo by bindi_ (@math_on_tap)Photo by bindi_ (@math_on_tap)Photo by bindi_ (@math_on_tap)Photo by bindi_ (@math_on_tap)Photo by bindi_ (@math_on_tap)Photo by bindi_ (@math_on_tap)
bindi_ retweeted
30 foot waves at Mavericks this weekend (like 15 mins from SF btw)
25
7
5
394
19,520
bindi_ retweeted
Replying to @jbulltard1
The ones who think it’s no big deal generally lack integrity themselves & have no clue what it entails.
12
26
1
1,116
104,258
I have a major result to share. I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China. This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model. Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves. I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems. I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used. It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20. These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs. I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at unemploymentlabs.com/checkpo…. There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.
1,720
6,069
2,079
39,124
6,048,234
tried to schedule an infra happy hour this week but there's nothing that infra people like more than maximizing overlap, even for happy hours.🫪
6
2
81
5,055
たまたまバスで知り合いと一緒になった時、 しばらく話した後にその人が「じゃあイヤホンするね〜」と言って会話をやめたのでこの人めっちゃ良いじゃんと思った。
41
910
113
49,067
1,875,821
I genuinely don’t understand why more people aren’t talking about this. A vineyard in Missouri figured out how to control weeds without spraying herbicides or using lawnmowers. They bought 12 miniature sheep for $900. The sheep were too short to reach the grapes, so they ate the weeds instead. In a USDA-funded trial, the vineyard reported spending just $75 on feed and veterinary care that year, while eliminating herbicide use and mowing in the test area. And the sheep naturally fertilized the soil while they worked. Sometimes I wonder how many problems we’re spending billions trying to solve with technology when nature already has a solution.
153
1,099
48
4,852
61,236
Replying to @LanLan93083241
是的 完全不想把很多心思放在玩脑筋上 有人和我玩脑筋也不会再和那个人接触 不坦诚我就觉得你这人不纯粹不值得深交
10
2
164
6,333
I have stage 4 cancer. I thought I'd fight it alone. I was wrong. Since mid October, ~20 strangers on X have stepped up to help me. All for free so far: - An NYU Langone lab offered to run my tumor data through their system to sharpen my vaccine targets - A U Michigan lab offered to test drugs on my actual tumor cells - A biotech ran AI analysis on my tumor RNA - A vaccine company offered to design a second vaccine to check the first one Scientists. Engineers. People I've never met. Cancer has been picking us off one at a time for all of human history. It counts on us fighting it alone, scared and quiet. Not anymore. We have AI. We have the internet. We have each other. Cancer, we're coming for you, bitch. Buckle up!
279
619
60
8,456
326,678
i miss the time when seeing cool tech stuff made me impressed. knowing that someone put in a huge effort into not only learning the skills but also having the dedication to finish was really admirable. now with llms people are just skipping the process - it all feels hollow to me
38
113
10
1,141
18,082
bindi_ retweeted
*megakernels vibecoded by people who have no clue what tma is are absolutely overrated. On the other hand, people who do know what they’re doing can absolutely (and will likely need it to) reach SOL BW usage
Megakernels are overrated
8
2
1
94
13,956
Happy Birthday Carl Sagan
15
312
33
1,314
24,190
the religion can survive a diwali without crackers, i promise.
188
588
75
5,836
99,256
I hope people know how to correctly dispose of Diwali decorations. Please recycle and reuse. Stop dumping toxic chemicals into water supplies or contributing to landfill.
2
244
Diwali at SFO airport.
14
245
4,185
Happy Diwali to all my desi friends. Today we can take a pause from thinking about RAM and think about Ram 🙏
10
17
4
377
10,697
Every PDEs talk has the same structure. Here's the wave equation, it models waves. Here's Burgers' equation it models burgers
18
70
7
935
29,914
bindi_ retweeted
Herman Millers are about to be dirt cheap
95
144
25
7,841
1,381,194
First restaurant I'm gonna try when I'm in London is Dishoom. I need to see the hype for myself.
207
7
62
392
239,600
bindi_ retweeted
no to be controversial but you shouldn’t have to work between november 1st and like january 5th
44
1,773
77
29,865
306,534
very important task category in Millennials' Last Eval
my colleagues are making fun of me for using Muse so much, but I just got a fuck ton of plants from Facebook marketplace for an absurd price so who’s laughing now
3
3
51
3,483