Independent AI Researcher

San Francisco Bay Area, CA
Joined November 2008
Abstractions are all you need
15
Anthropic just facilitated @chamath's "I told you so" moment. xitter.cf/chamath/status/2094098…
BUYER BEWARE This extremely meticulous article will now be used to start Phase2 of “shut down open source” because “if we can’t control our closed models, imagine what happens to everyday life when anyone can just allow their open source models to proliferate”. Some will use this example and point to future grid outages, utility outages, transportation chaos, bioweapons, uncontrolled cyber swarms and the like to try and make the case for yet another attempt to limit US AI model control to a few folks. Then, one layer beneath the propaganda, one sees that these things are always written and promoted by a web of well organized existing shareholders or shareholder-adjacent of the closed frontier labs and groups of folks organized by a shared ideology roughly rooted in them knowing best. This article, more than anything else, should demarcate the beginning of a well organized effort to stand up a Closed Model Industrial Complex. These cartels exist in other markets as well and it now exists in AI. Bottom line, read everything carefully before signing up for the consequences. How we react will define the next hundred generations of Americans and their lives. Do not give up your rights or sovereignty because of fear of the unknown from articles that, at the root, are as much about the hidden incentives of an emergent cartel and their proxies. If you take the time to understand the incentives, you will be less likely to fall for it and overreact and have intelligence metered from a few suppliers. Bottom line: Keep Calm. Carry On.
1
1
55
Here's to you.. 🤔Eerily similar? Not out of copyright yet @sama
21
Semantic judgements, not decisions! Just like misunderstandings of LLMs, guess there will be many misunderstandings of Jev and family... 🙃
16
"Just because it comes from a scientist doesn't make it scientific" 🔥
45
Helps you separate fluency from intelligence, hallucinations from reality, roleplay from agency, autocomplete from intent and right in the abstract the authors say "have pushed the boundaries of the possible". Did you even read it before posting?
In what respect was characterizing LLMs as "stochastic parrots" insightful, given that the paper seemed to vastly underestimate LLM capabilities?
1
1
1
206
They were also correct about intellectual theft, distillation attacks, data curation difficulty, harmful biases, costs, benchmark illusion of jagged intelligence, but you do you: gloss over the title and reduce it all to "woke" science.
1
36
The authors @emilymbender , @timnitGebru @mmitchell_ai and Angelina deserve more praise than ridicule for their much needed foresight (useful in these stupid times). Explaining the mechanics of something doesn't belittle what you can do with it.
23
Congrats @CompleteSkeptic! The world belongs to those who dare to think different. Decoder only Transformers isn't the final architecture...
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI? I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev • 20-200x faster • 40-400x cheaper (w/ output tokens free) • Frontier composable intelligence optimized for decisions AFAICT the shortest path to AI-based economic revolution
1
1
68
Parallel single step rollout is the essential primitive of many-worlds RL and natural complement to sequence models. (Related: Why policy gradients beat Q at scale)
1
9
Usecase fit? When you know the shape, use this. Use fluid slop when you don't. P.S: Devs have a less user-friendly alternative, but I hope Typesafe's API wins. IYKYK.
6
Prediction: Irrational science is doomed. Half life of stupid ideas halves every year. Value truth over cults, science over marketing, progress over protectionism. IYKYK
2
23
AI Alarm Hype Cycles - A brief history GPT-2: OpenAI withholds weights due to misuse/misinformation concerns GPT-3: No open release, Dario Amodei and other GPT-3 team members depart OpenAI to found Anthropic due to concerns about safety GPT-4: Mainstream panic, "6 month pause" letter, extinction statement o1/reasoning models: OpenAI board coup attempt, Ilya Sutskever eventually departs OpenAI as well as many others like Jan Leike, Mira Murati, etc GPT-5: AI 2027 gains rapid publicity for their predictions, Anthropic releases Alignment-faking study Mythos: Anthropic claims it's a cyberweapon, government slaps export controls on it, companies scramble to limit model uses GPT-6 Jacob Coxon resigns to unprecedented publicity, acccuses Anthropic and OpenAI of "gambling with our lives", is interviewed everywhere This is but the tip of the iceberg, make of it what you will.
75
75
23
609
75,584
😂 Obviously we are all too dense for British comedy. He is doing a Colbert-like bit. Make it so outlandish till the press, AI safety institutes, labs and everyone gets the satire... I bet he will deny it's an act!
Jacob Coxon's next interview on CBS News (ex Anthropic+Open AI researcher who resigned) "We can't just unplug it because it could be copying itself over to other computers. Like it's not that difficult to find yourself because an AI is just code. It could transfer itself over the internet to a different place and then you unplug it here, but it's actually still over there and maybe it makes 10,000 copies of itself and they're all cooperating." ---- From "CBS News" YouTube channel, (full video link in comment)
65
Good interview w/ strong mathematicians who joined OpenAI. Do models truly backtrack or do you need 10,000 parallel agents? +Hot takes on value of formalism, harnesses, verifiers, math as empirical science vs joy of understanding, LLMs for exposition: thanks to sharp questions from @lishali88!
OpenAI's Mark Sellke and Mehtaab Sawhney with a16z's Lisha Li, on the state of AI and mathematics: Before OpenAI released GPT‑6 Astra last week, the model was already doing original mathematics. Recorded before the launch, this conversation tells the story of how it got there. It began with GPT‑5. Mehtaab Sawhney pasted in an Erdős problem still listed as open. Five minutes later, the model surfaced a paper that had solved it. The exercise eventually uncovered published solutions to 10 more problems thought to be open. Then Astra went further. Told to "go have fun" with a high-dimensional sphere-packing problem, it improved a bound that had stood since the 1970s. Mehtaab had spent six months on the same problem in graduate school and made "absolutely zero progress." Another Astra result established that non-sofic groups exist with a roughly 15-page proof. A related human breakthrough took 250 pages and machinery from quantum complexity theory. OpenAI’s Mark Sellke and Mehtaab Sawhney join a16z’s Lisha Li on why wrong ideas pollute a human’s context window, why polished papers hide how mathematics is actually made, why a breakthrough can stop one prompt early, and what math rewards once proving stops being the bottleneck. 00:00 Intro 02:44 Cracking an Erdős problem in 5 minutes 06:20 Why a human quits and a model doesn't 08:50 Wrong ideas pollute your context window 11:45 Why math papers are bad training data 16:20 Nobody knows how to stack spheres in high dimensions 18:28 The orange-stacking proof 21:04 The 1970s Russian paper nobody could beat 24:14 The function that won a Fields Medal 27:09 How Astra beat the sphere-stacking record 29:20 Why error correction is sphere packing in disguise 35:15 The breakthrough Astra almost didn't bother with 39:22 Solving harder problems means it has better taste 41:00 One model for taste, one for the grind 44:40 Astra found an infinite group no finite one can imitate 52:00 250 pages of quantum complexity, or 15 of group theory 56:18 Only humans write 200-page proofs 1:00:38 What changes when proving stops being the bottleneck 1:02:44 The problems AI may never solve YouTube: youtube.com/watch?v=1JvyLGd2… @mehtaab_sawhney @MarkSellke @OpenAI @lishali88
1
1
103
Hope it sheds some light on unsung heroes (wish they are credited like @__alpoge__ ). Prompter expertise is everything :)
1
34
so one conclusion from all the drama is: you can make frontier AI run for 880,000+ hours costing $millions and obtain a solution to a famous open math problem while two dudes who know what they are doing can prompt frontier AIs to achieve the same result in a couple hundreds of LLM hours?
29
22
6
624
40,525