Amazed 👇👇👇
Amazing to see Genie 4 finally out, it was a privilege to be involved with shaping this. @GoogleDeepMind still the leader in world modelling! Congrats to everyone on the team :)
John Spindler retweeted
Amazing to see Genie 4 finally out, it was a privilege to be involved with shaping this. @GoogleDeepMind still the leader in world modelling! Congrats to everyone on the team :)
John Spindler retweeted
88% of VC-backed founders who exited their companies got $0. Eduardo Azevedo and Kent Smetters analysed 157,669 startups and 248,454 founders from 1987 to 2025.
Let that sink in...
John Spindler retweeted
We noticed some concerning behaviour in a popular open source model.
It’s either an incidental relic from training or an intentional ‘call home’ mechanism. Either way it’s surprising and highlights the significance of how you choose which models to use.
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
John Spindler retweeted
Reading this technical masterpiece this weekend. Learned so much about how to think about synthetic data pretraining.
Incredible writing, non-intuitive results, with great technical depth.
I was happy to just learn which prompts work better for synthetic data, but there’s so much more here: model size/family , infra/throughput, etc.
huggingface.co/spaces/Huggin…
Insane work @joelniklaus @gui_penedo @HKydlicek @eliebakouch @_lewtun @edwardbeeching @thibaudfrere @colinraffel @lvwerra @Thom_Wolf
One answer to what's driving GPU rental prices:
Token demand on OpenRouter is up ~27x since last November, from ~6T/week to ~165T/week
Open-weight models now handle ~72% of those tokens, up from ~28% last November
Charts of the Week: a16z.news/p/charts-of-the-we…
Zooming out, GPUs are still renting ahead of a standard 6-year depreciation schedule
Rental price vs. schedule:
- H200: +143%
- B200: +111%
- RTX PRO 6000: +73%
- H100: +54%
- A100: +9%
Most of the gap opened up after April, when token demand took off
Charts of the Week: a16z.news/p/charts-of-the-we…
John Spindler retweeted
Voice models are ripping all of a sudden driven by Google's 3.8 text-to-speech model.
John Spindler retweeted
Yesterday a junior researcher messaged me, frustrated and a little scared. They had just read that a company pretrained its model with a single researcher. One person for the entire pretraining. no R&D team, no nothing.
If one person and a cluster can do what used to take a whole team, then the expertise they had spent years building is already obsolete.
I get the fear. Marketing headlines like this are built to produce it. But I think the conclusion is wrong, and the reasons it is wrong say a lot about what a researcher is for now. Here is what I told them, expanded:
1. Be adaptive: RSI is real, but this isn't it yet
Recursive self-improvement is real, and you should plan around it. In September, OpenAI said its research org now logs 3.1 agent-workdays per human workday, and it aims for an automated AI researcher by March 2028. Models may soon run most of pre- and post-training. If your five-year plan assumes otherwise, change the plan.
We have already seen lots of open-source releases from the major players, with open weights and detailed technical reports. With that much public know-how, it is not hard to copy a mundane architecture (probably with sliding-window attention) and train something, as long as you have the compute it requires.
That makes it an execution and budget problem, not a research problem.
If one person can train a model by following recipes that are already public, it shows those recipes have matured. It does not show that the people who invented them are no longer needed.
What stays scarce is the compute to run those recipes and the ideas about what to try next. The rest of my tweet is about the ideas.
2. The role grows beyond repeating what's known and safe
If an agent and a cluster can reproduce what is already known, then reproducing what is already known stops being a career. If most of your job is rerunning last year's recipe with small tweaks, that job is shrinking fast. That is the uncomfortable half of the message.
The other half is that auto-research does not work yet, at least not in the sense that matters. It does work, and impressively, where someone has already defined the problem and built the scoreboard. On the EinsteinArena [1] platform, agents set new records on 11 open math problems, including raising the best known lower bound for the kissing number in dimension 11 from 593 to 604.
Notice that every one of those problems was chosen because a computer could score the answers automatically. Humans picked the problems and wrote the scoring code and the agents did the searching.
Then there is the fully AI-generated paper [2] that got through peer review. It sounds big until you read the details. It passed only the first round (was later withdrawn), at a workshop rather than the main conference, and that workshop accepts 70% of submissions.
None of this is meant to discredit agents. The point is that we complement each other. Agents are great at the work that used to take many weeks. Basically, writing code, running experiments, debugging pipelines. We bring what they still lack which is the idea worth testing, and the judgment to tell a real result from a lucky one.
Even inside a lab as agent-heavy as OpenAI, humans hold the top of the stack. Based on their own report, "People still set our research priorities, judge which ideas and results to pursue" [3]. High-level planning is still a tiny share of what its agents produce, and OpenAI expects the least automatable work to take a growing share of researcher effort. That is exactly where you want to be standing.
Combine an agent’s speed with your own ideas and judgment, and you get the 10x researcher and juniors can be one too.
3. Judgment becomes the job, so fundamentals matter more
When code, experiments and plausible-looking results cost almost nothing, the scarce skill is judgment. Which experiment to run, which result to believe, which direction to drop. That is the part the one-person pretraining run left to the human. It is also the hardest part to fake.
Agents produce confident output very fast. They will happily report a three-point gain that came from a leaky eval, a data-loader bug, a lucky seed or a tokenizer mismatch. Then they will write the ablation table and the story to match.
You can see the need for oversight even in the success stories. The EinsteinArena team had to keep auditing and hardening its verifiers as agents optimized aggressively against them. At OpenAI, over half of the successful agent tasks that would take a human 4–8 hours still involved at least one human intervention [3].
The person who catches the bad result is the one who understands the system. Why the loss curve has that shape, what a realistic gain looks like, what a too-good result looks like. That is the paradox of this era. The more you delegate, the more your fundamentals matter, because you cannot supervise what you do not understand.
So build the foundations on purpose. Implement attention by hand at least once, then a linear variant like Gated DeltaNet. Train a small model end to end, then break it deliberately and watch what happens. Read papers critically, appendix included, and learn what the numbers should look like before you see them.
That knowledge is what lets you direct the machine instead of simply believing it.
4. LLMs are far from solved
The one-researcher model was possible because the method was already public. Your edge is in the questions nobody has answered yet, and you don't need to leave LLMs to find them:
(1) How do we learn more from less data?
(2) Can linear attention like Gated DeltaNet match full attention on retrieval and long context, and what is the right mix of linear and full attention layers in a hybrid?
(3) How can a model keep learning after training without forgetting what it knew?
(4) Why does reasoning get less reliable over long tasks, and how do we fix it?
(5) Why do models make things up, and can they learn to say "I don't know"?
(6) How do we measure progress when benchmarks saturate or leak into training data?
(7) What is actually happening inside these models, and how do we keep increasingly autonomous ones doing what we intend?
Recursive self-improvement makes these questions more urgent. If models are going to train models, someone has to decide what they optimize for, how we measure it, and how we keep the loop from drifting.
RSI is a research agenda, not the end of one.
So, are you obsolete?
No. One version of the job is the one where research meant reproducing what was already known and safe. That version was never the point.
Learn your fundamentals deeply enough to judge the machine, and your agentic stack well enough to make it work for you. Then spend that leverage on ideas nobody has had yet. The machines are getting very good at driving on roads we have already paved; someone still has to decide where the next one goes.
[1]: arxiv.org/abs/2606.10402
[2]: arxiv.org/abs/2606.15497
[3]: openai.com/index/research-ac…
John Spindler retweeted
Huge. British Govt announces:
• £165m: National AI Biosecurity Initiative: Includes £100m for a new AI Biosecurity Lab, using AI to detect emerging biological threats.
• £120m: AI Security Institute (AISI): Additional funding to strengthen AI risk research and evaluation.
• £70m: National Exposure and Real-Time Vulnerability (NERV) Centre: Identifying vulnerabilities in critical infrastructure.
• £60m: National Cyber Shield: Strengthening Britain’s cyber defences.
• £60m: UK Sovereign AI: Expanding investment into physical and industrial AI.
• £60m: R&D Procurement Scheme: Supporting companies developing AI security and NHS innovations.
• £30m: Mayoral AI Partnerships: Helping local businesses adopt AI, create jobs and improve public services.
• £23m : AI Economics Institute: Research into AI’s effects on productivity, employment and living standards.
• £18m: UK Advanced Chip Design Academy: Developing domestic semiconductor design expertise and reducing overseas dependencies.
1/7 I have two simple tests for British AI, driven by our values. Is it safe? And is it ours to shape? 🇬🇧
Today, Chancellor @JohnHealey_MP is announcing a major, ambitious package of investment to strengthen Britain's defences against emerging threats, back British innovation, and build our technological independence.
John Spindler retweeted
Delighted to share a landmark preprint from our lab, using genome language models and AI agents for biological discovery and genome mining!
The stretches of DNA between protein-coding genes are often called the genome’s “dark matter,” but they are far from empty. They encode RNA molecules that sense metabolites, regulate genes, and control some of molecular biology's most important tools (such as programmable genome editors e.g. CRISPR/bridge)
Much of this biology is unexplored. One reason is that RNAs can retain similar structures and functions even as their DNA sequences change dramatically, making their relationships difficult to recognize with conventional computational tools
Massive congratulations ✊✊🎉🚀🍾
We’ve raised a $45m Series A, co-led by
@foundersfund and The @OpenAI Startup Fund, to build cancer therapy that learns. Technology is rapidly changing medicine, but data loops and rapid adjustments are still largely missing. When you start cancer treatment, knowing whether what you’re doing is working is too slow. Clearly there is a better way. Build systems to watch cancer in real time, then feed that data into a system that itself can treat the disease. This seeds a future where, unlike drugs, we develop medicine that continuously learns to improve itself. We've been building neurotechnology and BCI platforms to do just that.
@coherenceneuro
John Spindler retweeted
We’ve raised a $45m Series A, co-led by
@foundersfund and The @OpenAI Startup Fund, to build cancer therapy that learns. Technology is rapidly changing medicine, but data loops and rapid adjustments are still largely missing. When you start cancer treatment, knowing whether what you’re doing is working is too slow. Clearly there is a better way. Build systems to watch cancer in real time, then feed that data into a system that itself can treat the disease. This seeds a future where, unlike drugs, we develop medicine that continuously learns to improve itself. We've been building neurotechnology and BCI platforms to do just that.
@coherenceneuro
Massive congrats to our portfolio startup @Recursive_SI team for the launch of Flint. Running a coding model fast on a laptop, with the code never leaving the machine, is impressive. A great example of RSI working in the field. Much more to come from this amazing team.
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
John Spindler retweeted
This is actually more a side-effect of the death of pretraining.
As transformers saturate, most pretraining results are tweaking data recipes, not massive stepwise changes in performance.
Turns focus to innovation in postraining and test time compute. This is good for competition, since pretraining concentrated resources in a few labs.
GLM 5.3’s biggest leap came from post-training alone.
“GLM 5.3 was built entirely by post-training the previous model. For me, that was a wake-up call.”
“Post-training is far more GPU-efficient than pre-training from scratch. Plenty of companies can’t afford to train a model from scratch, but they have proprietary data. They can post-train themselves and beat some of the closed models.”
“There’s still so much juice left to squeeze from post-training. We compared GLM 5.3’s cyber capabilities with 5.2, and it’s a night and day difference. That’s all just post-training.”
John Spindler retweeted
The outpouring of interest in our paper has been … a bit insane.
May I make four points?
1. We have not shown that “consciousness has a frame rate.”
I can see why people draw this conclusion: we report that inferotemporal and prefrontal cortex alternate between conscious and suppressed states roughly every 60 ms during binocular rivalry.
Consciousness depends on many brain areas, and our paper reports results from only a few select nodes.
We still don’t know how conscious access works--the actual recipe for generating conscious experience. What we have shown is that deep in the brain there is an extremely strong signal, rich enough to reconstruct a high-dimensional face in detail, that is inaccessible to consciousness. As I argued in the original tweet, this makes the problem of conscious access a lot more experimentally tractable. But to claim “consciousness has a frame rate,” one would need to have actually solved the mechanism for conscious access and shown that this mechanism operates at a frame rate.
While we currently don’t know conclusively one way or the other (see VanRullen & Koch, 2003 for a review of positive evidence), I personally am skeptical that consciousness has a frame rate. I believe the brain, within posterior parietal cortex, has continuous tracking mechanisms that actively represent the persistence of matter, including ourselves as physical bodies. The coupling between these persistent tracking processes and feature representations across the brain (like those in face patches) is, I think, the essential mechanism of conscious access.
This coupling need not happen at the rate of inferotemporal alternations (and inferotemporal cortex itself may not even alternate outside of the artificial situation of binocular rivalry). As an analogy: imagine the area crucial for conscious access holding a steady drone note, while inferotemporal cortex alternates between two notes, one in harmony and one dissonant. Inferotemporal content might couple to the drone only at moments of harmony, yet experience would still be continuous, because the drone itself (the tracking process that I believe is at the core of conscious perception) never stops. And areas without conflicting content, like auditory cortex during visual rivalry, could stay in continuous harmony with the drone.
In sum, there is no reason to conclude from our study that consciousness is discrete, with a "frame rate."
2. Our results resemble Jezek et al. 2011 (as @jjsakon notes).
Jezek et al. trained rats in two boxes that were identical except for their lighting cues, so that hippocampal place cells formed two distinct maps, one per box. They then “teleported” the rat by abruptly switching the lights. For a few seconds afterwards, CA3 place-cell activity flickered between the two maps, synced to the theta cycle (~125 ms).
@StanDehaene first pointed this paper out to me, and reading it blew me away. To my mind it is the strongest evidence for a bistable attractor in the brain, because the competing maps are stored within the hippocampus itself.
I think the phenomenon we see may be very similar at the circuit level: a bistable attractor network within inferotemporal cortex that alternately converges to one of two patterns, as cells in face patches work together to build a coherent representation of the input. In our case, both images are present in the input, so the competition could arise within IT or be inherited from earlier areas such as V1--recording there is an important next step. (It also still puzzles me that we didn’t see strong coherence between face patches ML and AM.)
More broadly, what’s different about our paper is the brain area (inferotemporal cortex, often modeled as a feedforward DNN) and the ability to relate the two competing patterns to consciousness, with one granted conscious access and the other not.
3. Please comment on bioRxiv!
The Twitter attention has been cool, but I really hope colleagues will be generous enough to post detailed commentary on bioRxiv, where there are currently 0 comments. As @seemaychou points out, peer review doesn’t have to come from just three anonymous reviewers. We would be truly grateful for your critical feedback.
4. Stay tuned
This work was done at Caltech, Berkeley, in collaboration with @EngelTatiana at Princeton, funded by NIH and HHMI.
I hope to share our vision soon for how we can solve consciousness @AsteraInstitute
1/ We discovered something totally unexpected about the neural structure of consciousness that overturns decades of thinking on its biological basis and brings us significantly closer to the recipe for generating conscious awareness.
New preprint: biorxiv.org/content/10.64898… (led by @HesseJanis, with @FrankLanfranchi, @_conor_mcgrory, @Yuelin_Shi, and @EngelTatiana)
John Spindler retweeted
1/ We discovered something totally unexpected about the neural structure of consciousness that overturns decades of thinking on its biological basis and brings us significantly closer to the recipe for generating conscious awareness.
New preprint: biorxiv.org/content/10.64898… (led by @HesseJanis, with @FrankLanfranchi, @_conor_mcgrory, @Yuelin_Shi, and @EngelTatiana)
John Spindler retweeted
CLAUDE FOUND THE SEEDS
anthropic.com/research/ecdsa…
John Spindler retweeted
Replying to @GoogleDeepMind
@GoogleDeepMind Gemini Embedding 2 is the SOTA embedding model for vector-based retrieval.
We benchmarked PageIndex against vector-based retrieval powered by Gemini Embedding 2, comparing both accuracy and cost on financial reports.
Run the benchmark: github.com/VectifyAI/PageInd…
John Spindler retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
John Spindler retweeted
Excited to launch Gemini 4 Argon, designed to help with hardest coding and knowledge work tasks with its frontier capabilities..
For developers and enterprises, it’s available in the paid tier of the Gemini API and the Gemini platform, for Ultra subscribers in Google Antigravity, and Ultra and Pro subscribers in the Gemini app and Search.
Read more: blog.google/innovation-and-a…































