Information persistence hunter extraordinaire. 2x Dad, AI Guy, FP Enjoyer, Math for Fun, Tech Leadership. My views do not represent my employer.
NYC
Joined September 2008
- Tweets146K
- Following3.2K
- Followers10.1K
- Likes115K
Rick retweeted
站在今天看，俄罗斯这波操作，属于手握着一把天胡的牌打得稀烂。
如果不出兵乌克兰，今天全球AI的牌桌上，除了中美，大概率真得坐着一个俄罗斯。
搞AI、跑大模型，底层逻辑就是“吃电、散热、堆机器”。这两年全世界的数据中心老板都在为电费和机房降温发愁。俄罗斯属于老天爷喂到饭：西伯利亚开窗就是纯天然顶级空调；境内水电、核电、天然气很便宜，当年挖矿的都选西伯利亚。
另外俄罗斯人的数学和算法也强，各大国际编程竞赛，Kaggle机器学习竞赛常被俄国人屠榜。全世界除了中美，俄罗斯也搞出了自己完整的互联网生态：搜东西有Yandex，社交有VK，还有Telegram。现在在美股被炒上天的算力黑马Nebius，核心班底就是原先Yandex那帮俄罗斯人。
如果疫情后俄罗斯出地皮、出冷气、出便宜电，再拉着中国这种基建狂魔帮忙拉高压电网、建厂房、铺光纤，欧美资本开服务器工厂。俄国人完全可以成为欧亚大陆最大的“超级算力工厂”，中美神仙打架，它在中间两头收过路费。
结果呢？普京一拍大腿：不行，我得去抢那几块地。
普京的脑子，彻底停留在了19世纪沙皇俄国那种“开疆拓土”的旧梦里。什么有没有出海口啊，有没有所谓的军事缓冲带啊，就算抢方法也多的是，这可这是21世纪啊！现在的世界霸权争夺，抢的是半导体供应链，抢的是先进算法，抢的是高精尖人才。你去顿巴斯抢了几片烂泥地、几座废弃钢铁厂，代价却是把整个国家的未来都押了上去。
枪声一响，一切全崩了。稍微有点本事的程序员、算法工程师，连夜跑路。Yandex核心技术团队，宁可巨亏，惨烈的断尾求生、割裂母国，也要跑到欧洲重组成Nebius。普京把最懂未来的人，全逼成了难民和海外创业者。
在人工智能技术爆发的黎明前夜，全世界都在疯狂狂飙算力、抢占新工业革命的门票；而俄罗斯的老大，却带着几十万人，在泥潭里打着一百年前的一战式堑壕战。
再说一遍，普京不行，国内崇拜普京的那帮人也不行
X is starting to be flooded with these cut off videos, I'm guilty of watching some not knowing they will suddenly end halfway, but would rather they weren't here.
Rick retweeted
2309.08632 evergreen
The good news: you can mid-train yourself to the frontier!
At a fraction of the cost! Without distillation!
What we did: We took Qwen3.5 14B and mid- and post-trained it to beat every other open model and perform as well as the very best frontier models.
This is no simple toy task or format fixing, we are training the model to understand and reason over DNA sequences. Bio is the next frontier for AI and universal access will be crucial to make progress.
The fun part: because we train the model on both natural language and DNA sequences you can actually 'chat to your DNA': Ask it questions, ponder the impact of mutations, or ask it to generate a small sequence.
We'll soon see the frontier labs move away from math and cyber towards bio. It already started, and we believe researchers should have access to capable models.
We are releasing the full recipe, so you can do it, too!
Model: huggingface.co/HuggingFaceBi…
Blog: huggingface.co/spaces/Huggin…
Rick retweeted
When you are live on an earnings call and the CEO or CFO pauses for five seconds before answering a question about accounts receivables, you feel something. You cannot explain it cleanly. You just know that pause meant something. Read the transcript two hours later and that pause is gone entirely. The transcript says he answered the question. It does not say he needed five seconds to decide how.
When you visit management in person and the receptionist has been there for twenty-two years, that tells you something. When the parking lot is full at 4pm on a Friday, that tells you something. When the CEO walks you through the factory floor and he has unscripted personal dialogue with the line workers with what looks like genuine respect rather than performed deference, that tells you something significant about how this place operates.
None of that is in the 10-K.
microcapclub.com/the-last-mo…
29% of americans have pension plans, over 50% of current retirees have pension plans. Will Total Boomer Luxury Communism win again? Probably safe to bet on.
WSJ: America’s public pensions now depend on the AI industry that the politicians who run them are turning against.
13 large US public pensions booked $138B in paper gains from AI stocks since late 2022.
Those 15 stocks, including hyperscalers and chipmakers, made up 27% of the funds’ domestic equity in June, up from 12% when ChatGPT launched.
That concentration drove roughly 57% of their US stock gains, without counting AI exposure through foreign shares, private equity or private debt.
> New York paused new data centers while its retirement fund’s AI holdings kept climbing.
> Pennsylvania’s treasurer fights data-center growth while overseeing a pension holding billions in AI assets.
> California union leaders want pension managers to weigh AI’s harm to workers while the state’s $650B fund backs data-center providers with $500M.
> New York’s comptroller pressed tech firms over residents’ power, land and water fears while overseeing a fund profiting from those firms.
Strange how every year everyone gets a bit black pilled in the fall, it’s been happening for as long as it’s been here. The twitterverse has its cycles.
Rick retweeted
A big part of why America is so goated is we handed it over to autists in the early days and just let them ball out.
Rick retweeted
Absolutely wild results in the details:
- Claude built and formally verified a Python interpreter
- Claude wrote ~1Bn lines of Rocq
- Claude built an x86 semantics that can boot Ubuntu
New on our Frontier Red Team blog: using Claude to formally verify software at large scale.
Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).
With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.
Read our research in full here: anthropic.com/research/forma…
Rick retweeted
thesis: any sufficiently motivated person who has access to an abliterated GLM-5.3, or DS V4.1 Flash 1013 model and access to enough compute can do whatever they want right now
so, outrun the bear if you're big. or hide from the bear if you're small.
Rick retweeted
HOLY MOLY. Something genuinely strange just happened in AI training.
Q Labs says it can now pretrain a transformer WITHOUT backpropagation, cutting the time to reach a fixed validation-loss target from 1.76 hours to just 1.6 seconds in only 10 days. A nearly 4,000x speedup.
The latest run also beats backprop + Adam on validation loss.
Its Dust algorithm was already estimated to be up to 10,000x more compute-efficient than the previous state-of-the-art zeroth-order method.
The biggest question now is scaling. Some of the tricks won't scale, so the team is moving from 10M to 30M training tokens.
I think the last month might have been the biggest month in (zeroth-order) optimization history.
In the 10 days since we released the Noise speedrun, the time to pretrain a transformer went from 1.76 hours to 1.6 seconds, a 4,000x (!!!) speedup. Dust itself was up to 10,000x more compute-efficient than the previous state-of-the-art.
Not only can we pretrain a transformer without backprop, it's much more efficient than anyone would've expected. Search fking works!
Now, scaling is the biggest open question. Some of the tricks that brought the time down to 1.6 seconds will simply not scale. To discard those, we'll be scaling the dataset every time it gets overfit, starting with 30M tokens instead of 10M.
Huge thanks to @jayden_teoh_, @spicey_lemonade, @RobertTLange, @EmanueleRodola, @ChunyuanDeng, @LuochengHuman, @Alaxarr, @pavleculafic, @IsaacYong13, Euraba and areu01or00 (GitHub), for the contributions. And @cs_serdar for gracefully maintaining the speedrun and @bishmdl76 for helping out.
Expect a lot more from us over the coming weeks.
