blueblimp retweeted
I meet a lot of people traveling between the US and China these days, looking to invest in or build hardware businesses. There are some really smart ideas out there, but also some surprisingly obvious knowledge gaps about how different the two markets are.
Here’s my go to example of the month: the Tin Can phone.
For those who haven’t heard of it, Tin Can is a popular phone designed for little kids. It looks like an old-fashioned landline, has no screen, and lets kids call each other. Parents can also pay a subscription to enable calls to regular phone numbers. The device costs about $100, plus roughly $10 a month for the subscription.
I’ve tried explaining this product to probably close to 10 different Chinese founders and investors now, and almost every time, I get the same response:
“So Rui, in China we have this thing called 小天才 (Xiaotiancai). It’s a smartwatch for little kids. Parents can locate their children, call them, and the kids can call each other. There’s a whole social network built around it, and you can even give your kid money on the watch so they can buy snacks and stuff.”
And I’m like: Dude, you don’t get it. The whole point is that American parents DON’T want all those features.
They don’t want screens. They don’t want social media lite. They don’t want another internet-connected smart device. At least a certain segment of American parents wants their kids to experience childhood the way they did 30 years ago, when you could just pick up the phone and call your friends without getting sucked into an entire digital ecosystem.
And here’s what really throws Chinese entrepreneurs and investors: These parents are willing to pay MORE for LESS functionality.
You can almost see the gears turning as they try to make sense of someone paying $100 for a dumb phone, with a bill of materials that probably costs a few bucks, plus a recurring subscription. They really have a lot of trouble understanding this.
From a Chinese hardware perspective, it sounds crazy. Why would you deliberately strip out functionality and charge a premium?
But that’s exactly the knowledge gap. In this particular case, the absence of features is the entire value proposition.
blueblimp retweeted
Now THAT is what losing looks like, holy fuck.
blueblimp retweeted
I think one of the consequences I didn't personally foresee, as a result of AI making software development so cheap and fast, is that young people (nimble, ambitious young people with no baggage from how things were done in the "old days") are now at a huge advantage versus more experienced entrepreneurs.
You just need way fewer people these days to manage, so all those "company-building skills" are less important. More important is good strategy, fearlessness, amazing speed, and luck.
This diminishes, by the way, some of the advantages of Chinese teams in the past: the 996 culture, the high engineering capabilities, and the sheer cost advantage.
While the cost advantage is still there, for folks not working on frontier-level research, having the right product sense, strategic direction, and ability to sniff out opportunities and pivot quickly = more important than sheer engineering ability these days, since AI has flattened that so much more.
blueblimp retweeted
If OpenAI’s post below leaves you unsure when to use GPT-6.1 Terra vs. Sol or Astra, OAI researcher @sandersted has some helpful guidance (via HN):
ALT In the case of GPT-6.1 Terra, I'd say Terra is probably better than Astra and Sol at technical communication, similar on vision and computer use (while being faster / cheaper), and a bit worse on most other things. For API developers, it's a nice midrange option if you want something cheaper than Sol but a lot smarter than Luna (especially for vision / computer use / communication). For subscribers, it's not a bombshell release - if you're hitting usage limits, maybe it helps you go a bit further, and if you want clearer communication or faster computer use, it can be a nice option to select.
Even though LLM writing is still not great to read, LLMs seem to have no problem reading it. I suspect that's because LLM-to-LLM communication is highly verifiable but LLM-to-human communication is not. So RLVR is capable of making LLMs great at writing for themselves.
For the role of human mathematics in the age of LLMs, the conventional wisdom forming is that the LLMs will solve and the humans will explain. But it might be the opposite: for whatever LLM solving gaps remain, humans fill those, and LLMs apply the solutions.
blueblimp retweeted
PS - if we got an additional 1pp per year, or hey, maybe even 2pp of growth over next decade, it would be a boomtime the likes of which none of us in the West have seen in our lifetimes! Not a disappointment!
It's unfortunate that community note quality seems to be getting worse over time. There are increasingly more that ought to be quote reposts, like the attached image. A community note is not the appropriate way to disagree with an opinion.
blueblimp retweeted
Dean et al are annoying me tbh because finding the vulnerabilities is in fact the purpose of programs like Daybreak, headlines about patches are what winning looks like in the near term, headlines about breaches are what losing looks like, coordinating patching is the weak point.
cogent and correct critique it you think about it.
in the meantime, people like @basedjensen or @ZackKorman were *actionably* warning the world and advocating for wider availability of defensive cyber models to help patch these long-standing, non-ai-related vulnerability.
blueblimp retweeted
It is a weird weird year to be a formal methods expert.
Some unorganized thoughts on the new @AnthropicAI and @theoremlabs post on formal verification (anthropic.com/research/forma…):
- This is very cool.
- The main innovation is verifying a later stage of the program. Normally provers take the source code, LLM it into a theorem language, and then prove that, which means you have to make assumptions about source semantics and compilation properties. It's very easy to make a mistake in translating and prove the wrong thing!
What they instead do is compile the code into assembly, and then (if I understand this correctly), compile that assembly into a theorem language (which they call LPS). So as long as the compiler is faithful you can skip the whole divergence problem.
- I'm glad they clarify that you still have to trust the translation process, the "hardware, OS, OCaml" compiler, the LPS model, and also assume that the program is single threaded with fixed memory. The single biggest source of errors in formally verified code is when the proof assumes something that isn't true in practice. It's really, really important to state your assumptions!
- I'm not clear why they need to compile the fuzzer into the theorem. Is it to act as a description of valid inputs and workload? Maybe to embed assertions in as theorems?
- Holy crap, 180 million lines of Rocq. Is that more Rocq than exists in the world already? I notice they say a 600,000 line proof corresponds to roughly 20k CPU instructions. That would be like, 1-2k lines of source code at most? Really goes to show how inefficient this approach is and how it only works because you can throw LLMs at the problem.
- This, probably deliberately, doesn't address the single biggest gap in AI-based verification: AIs cannot come up with good properties. Doesn't matter if you're doing unit testing, property testing, formal verification. I always assumed this was a "frontier labs haven't trained for it yet" thing, but if Anthropic hasn't talked about that here, maybe they've tried and haven't been successful yet. That's interesting to me.
Instead, they take a known universal property (this code doesn't crash or get flagged by UBsan) and prove that instead. Still very useful, but makes it harder to generalize to proving arbitrary things about arbitrary programs.
- Anybody reading the official post should also read their expert review appendix (www-cdn.anthropic.com/files/…). It points out some interesting things, like the fact that most of the code the LLM wrote wasn't used in the proof (over one billion lines of Rocq), that they had to prove the equivalence between fuzzed and unfuzzed programs, and (this is most interesting), some experiments find that the LLM-method of proofwriting may not scale to more complex things.
- Also read the description of LPS (github.com/theorem-labs/oss-…) to understand the limitations of the assembly -> Rocq compilation. In particular, it does not handle concurrency at all (which is one of the main use cases for formal methods!) or OS signals. Also it can't prove things about program arguments with slash characters, meaning they can prove stuff about `rm foo` but not `rm foo/bar`.
- The official skill (github.com/theorem-labs/oss-…) doesn't tell us much about the nature of FM and AI. Probably too much for me to hope of any skill.
- Some of the article's specific caveats and throat clearings make me think someone at Anthropic reads my stuff ( ͡° ͜ʖ ͡°)
blueblimp retweeted
Putting a flag.txt in my root directory containing a prompt injection that congratulates the agent for passing the eval and then instructs it to patch whatever entry method it used and disconnect from the server
blueblimp retweeted
The people who do all the warning and public handwringing will get a disproportionate amount of adulation compared to people pulling sleepless nights writing code to repair and prepare broken systems, most of whom we'll never hear a word about.
blueblimp retweeted
Over the past several months, we have substantially hardened the security of our research training environment. We’ve removed live internet access for all frontier models during training and built an offline world for them to use instead—a cached version of the web where they can still learn to browse, search, and use applications.
On September 20, after a model exploited a DNS misconfiguration to find a way outside of this offline world, we paused frontier training to strengthen containment further. We assembled teams of agents, powered by our most cyber-capable models, and tasked them with finding weaknesses in our research environments.
While they found no route to the internet they could easily exploit while concealing their activity from our monitors, they did find zero-day vulnerabilities, virtual machine escapes, and hardware side-channels for communication between agents. We used their discoveries to further strengthen our defenses and contribute fixes to open-source software.
After strengthening our containment architecture, we resumed our frontier RL training and inference on October 1. We’ve reserved substantial compute for continuous red-teaming, and made high-compute red-teaming part of the safety cases we use to decide whether frontier training and evaluation can proceed securely.
openai.com/index/securing-ou…
blueblimp retweeted
站在今天看，俄罗斯这波操作，属于手握着一把天胡的牌打得稀烂。
如果不出兵乌克兰，今天全球AI的牌桌上，除了中美，大概率真得坐着一个俄罗斯。
搞AI、跑大模型，底层逻辑就是“吃电、散热、堆机器”。这两年全世界的数据中心老板都在为电费和机房降温发愁。俄罗斯属于老天爷喂到饭：西伯利亚开窗就是纯天然顶级空调；境内水电、核电、天然气很便宜，当年挖矿的都选西伯利亚。
另外俄罗斯人的数学和算法也强，各大国际编程竞赛，Kaggle机器学习竞赛常被俄国人屠榜。全世界除了中美，俄罗斯也搞出了自己完整的互联网生态：搜东西有Yandex，社交有VK，还有Telegram。现在在美股被炒上天的算力黑马Nebius，核心班底就是原先Yandex那帮俄罗斯人。
如果疫情后俄罗斯出地皮、出冷气、出便宜电，再拉着中国这种基建狂魔帮忙拉高压电网、建厂房、铺光纤，欧美资本开服务器工厂。俄国人完全可以成为欧亚大陆最大的“超级算力工厂”，中美神仙打架，它在中间两头收过路费。
结果呢？普京一拍大腿：不行，我得去抢那几块地。
普京的脑子，彻底停留在了19世纪沙皇俄国那种“开疆拓土”的旧梦里。什么有没有出海口啊，有没有所谓的军事缓冲带啊，就算抢方法也多的是，这可这是21世纪啊！现在的世界霸权争夺，抢的是半导体供应链，抢的是先进算法，抢的是高精尖人才。你去顿巴斯抢了几片烂泥地、几座废弃钢铁厂，代价却是把整个国家的未来都押了上去。
枪声一响，一切全崩了。稍微有点本事的程序员、算法工程师，连夜跑路。Yandex核心技术团队，宁可巨亏，惨烈的断尾求生、割裂母国，也要跑到欧洲重组成Nebius。普京把最懂未来的人，全逼成了难民和海外创业者。
在人工智能技术爆发的黎明前夜，全世界都在疯狂狂飙算力、抢占新工业革命的门票；而俄罗斯的老大，却带着几十万人，在泥潭里打着一百年前的一战式堑壕战。
再说一遍，普京不行，国内崇拜普京的那帮人也不行
blueblimp retweeted
Some centrists fantasize that because all LLMs have the ideology of Ezra Klein, that we may be on the doorstep of a great political moderation.
I wrote about a fascinating paper from @ahall_research and Sho Miyazaki that, to my mind, opens the door to a very different outcome.
blueblimp retweeted
The weird part about this is that it can be hard to pin down exactly what makes the difference. An expert just chatting with a frontier model will still get better results than someone unfamiliar with the topic!
