Building https://nitter.gq/t.co/5tJtX6UbyK, agentic AI for autonomous R&D. Previously https://nitter.gq/t.co/zQdC7zN9Xl (acq. Workday).
Joined December 2008
- Tweets1.9K
- Following410
- Followers949
- Likes1.3K
Filip Dousek retweeted
In our new npj Aging paper, we show that a 30-second voice recording captures a signature of biological aging
Using 7,000 participants from the Human Phenotype Project (humanphenotypeproject.org) we trained voice-based age models to predict chronological age. The resulting Voice Age was among the strongest single-modality aging clocks and added information beyond metabolomics, imaging, physiology and lifestyle
A higher Voice Age gap was associated with adiposity, sleep-disordered breathing, poorer nocturnal oxygenation and liver-related traits, suggesting that voice may provide a simple, scalable biomarker of accelerated biological aging
nature.com/articles/s41514-0…
Filip Dousek retweeted
We need a more nimble regulatory framework for outbreaks like hantavirus.
There is already a compelling therapeutic candidate: a human mAb isolated from a recovered patient, with strong preclinical activity against the Andes variant.
Yet access remains months to years away.
Filip Dousek retweeted
Introducing Code on Canvas in @pendev
AI design tool for Claude and Codex.
Design ❤️ Code are officially in a relationship, opening new ways to create on canvas.
Ask agent to generate custom design tools inside Pencil on the fly, create interactive components, generative art and more, but still keep the full manual design control.
This is just the beginning. Let us know what you think. And let's take it to the next level together.
Download the new Pencil update today.
Filip Dousek retweeted
This is a perfect example of @ScottAdamsSays' two movies on one screen metaphor.
Some of us see a totally disrespectful Zelensky walking away from a verbal agreement and trying to litigate and negotiate in front of a live TV audience.
Others see an eval mafia boss meticulously orchestrating a public smack-down of a heroic leader of a poor victimized nation under attack.
Two movies one screen.
Filip Dousek retweeted
so I’ll be sharing something pretty crazy that I did with o3-mini-high last night (later today)
through one prompt, o3-mini-high simulated building 5 agents, all phd experts in different fields, and all with personas we might resonate with (like one of them is Dr. Turing. lol)
once o3-m-h built” these agents, it simulated responding to every problem by orchestrating a group discussion and debate among the 5 scientists.
I let them run wild for a long time last night.
4o orchestrated the discussion, and the choices made and topics discussed by the ai were chosen entirely independent of me. I had no say in what they discussed.
what did they discuss, you say?
digital identity and synthetic sapience.
the scientists all collaboratively wrote a white paper which outlines a proposed governmental framework to implement things like ai rights, responsibilities and roles in society.
crazy sh*t, tbh…excited to show you what went down. genuinely.
Filip Dousek retweeted
o1/o3/R1 models are the story of LLMs evolving beyond next token prediction.
Most training compute now goes into a far more complex training objective which I think likely allows the model to learn far more complex statistical features.
RLHF couldn’t scale enough. The reward model was exploring too low-dimensional reward space, leading to instability when applied at scale. So we were stuck with models that were trained (mostly) as next token predictors until more scalable training objectives were developed.
The models are no longer primarily rewarded for predicting the next token. I don’t think they are even rewarded for just predicting an idealised Chain of Thought (a granular solution to complex problems). I think o1 is likely instead rewarded for exploring a tree-like structure of potential reasoning paths with many branches of possible CoTs. This is synthetic data with branches presumably pruned (optimised for training efficiency and diversity), scored by verifiers/reward models and potentially also annotated with extra LLM feedback. This trains an instinct for starting on the best reasoning path, feeling when it is on a dead end branch, instinctively navigating to a better route and knowing when it has arrived at the right answer. They learn to one shot a diverse and efficient route through the space of possible Chains of Thought to arrive at a solution.
I think the magic of o1 and o3 models is not that they are learning to “reason” (which they are) but that they are learning an “instinct” for taking optimum reasoning steps.
o1/o3 are just single transformers, however it is possible to run multiple in parallel independently (presumably o1-pro) and in future it would be possible to also create whole search trees at inference time instead of exploring a single or independent parallel paths.
Filip Dousek retweeted
Why did o3 make such rapid gains over o1?
I think it is due to a new recursive feedback loop between test time compute capability and distillation of test time capability gains back into the original base model.
This is a consequence of 3 key takeaways from AI/LLM progress the last 2 years:
A) Model distillation for LLMs works very well (and, relatedly, synthetic data can work very well)
B) We found a new training objective (learning an intuition for navigating full trees of possible Chains of Thought, described below) which for the first time allows RL to scale in training and leads to models with increased output diversity that can 1) scale inference time compute effectively and 2) scale synthetic data further without collapsing
C) Scaling inference time compute leads to predictably higher model capability
This leads to the feedback loop.
Model distillation allows a more capable model to teach a less capable student model and significantly close the capability gap between them.
But now we don’t need to use a more capable base model; we can just use the same model with its test time compute–scaled capability boost to teach its own base model.
This creates a new model v2 that starts from a higher base of capabilities; but again we can predictably increase these via inference scaling. A test time–boosted v2 can then be used to teach itself via model diffusion, leading to v3. And so on.
Separate to this feedback loop, we also have easy wins from scaling RL training data and flops much further. Next token prediction pre-training compute scaling still works as well, but it doesn’t currently look like the highest capability return on compute investment compared to these new methods.
Filip Dousek retweeted
Considering a fresh hypothesis about dreams: “dreaming” is a search for sensory permutations that will trigger release conditions for obsolete latches
There’s no scientific consensus on why we dream, although I really like @erikphoel’s idea that they generate synthetic data so our brains don’t overfit. But following vasocomputation (“latch theory”), I wonder if dreams are (also) a core tool for the body to release unneeded latches
Basic thesis (vasocomputation):
Vascular latches are semi-permanent vascular muscle clenches that act as ‘side channel memory’ on the dynamic range of local neurons (see quoted tweet). They often form when the body decides it really needs to remember something. A latch is essentially a sensory lock: tension that is held until some perception occurs (e.g. “my tension released when I saw my husband come home from the war”)
Many of our latches are important for keeping us functional/contextual, but many of them are also unnecessary, i.e. frozen compute that doesn’t matter anymore (e.g. shards of memory from college, old jobs, old relationships, etc). Memory is valuable and garbage collection is hard so I think it’s very common to have unnecessary latches build up. But the body doesn’t particularly know which are useful — if it did, the unnecessary latches would be open already!
How can the body do ‘garbage collection’ on these latches? I suspect “dreams” are an advanced technology for generating sensory permutations as keys to test against the body’s latches and see what opens. Sometimes as part of a large narrative, but sometimes just sensory permutations that might plausibly hit some completion criterion. Basically, dreams are a hash attack against tension — and if one of the hashes hits and we reach catharsis in a dream, we benefit from it in ‘real life’ — the latch stays open
Predictions:
- If you don’t dream, you’ll accumulate chronic vascular tension (latches) over time; after a cathartic dream this will measurably drop
- Dreams that feel the most cathartic will be those that involve unresolved elements of your past (and this need not be profound; it could be “I never returned a library book when I was 9 and I totally forgot about it until I had a dream I returned it and now I feel weirdly better, like some hidden tension dissolved”)
- The body will stochastically attempt classes of dreams that could release commonly-stuck computations. One core strategy here is ‘restarting and completing scenarios’ — starting a scenario from scratch and running it to its conclusion might trigger a release condition if something similar is latched.
“Running from some mysterious figure” is a super common dream, perhaps because it would be an obvious ancestral situation with frozen computation; there’s an imperative to get away from something, but the chase takes a long time to be ‘over’ and this makes the tension more sticky (increasing the chance of it latching). If you go through the same scenario, but it resolves more quickly, it could open related old latches
I think a dream questionnaire on “what did you dream about? Did this have sensory or abstract relevance to some stressful event in your past? How did the dream end? Did this dream produce catharsis?” would be pretty interesting
Of course, the ancestral distribution of sensory permutations to try as keys while dreaming might be different than the ideal distribution now, given that modern life & modern trauma (‘forced memory’) is significantly different than in our ancestral environment. Maybe there’s room for improvement — thinking about framing what @EricWollberg & @weslouis_ are building at Prophetic as a ‘latch-opening dojo’*
If latches were metaphorical, this theory might be evocative but not particularly diagnostic. But latches are very concrete physical things! Bullish for the possibility of inducing more cathartic dreams
*cf an old NYT piece about dreams as a ‘threat dojo’;
Research context for vasocomputation: nitter.gq/johnsonmxe/status/1863…
Thinking about trauma & latches — I think there are several different kinds of ‘shapes’ that can get stuck (latched) in our nervous systems, but I think most of them at least rhyme with this:
You’re out on a walk in the jungle with your friends, and you step on a snake. It bites you; this really hurts. You roll around in pain, fade to black, and wake up in a hospital. The body vows: “that was really awful. I will never let that happen again.”
Your body’s response is to latch shards of snake-perception into your nervous system. Next time you’re walking in the jungle, you often mistake sticks, roots, & shadows for snakes. But you will be ready for the snake when it comes. By latching shards of snake-perception, you preload the context (set the Bayesian priors) so it loads much more readily
But due to the nature of perception, trauma-latching goes further: not only are you jumpy about *seeing* snakes, but your body will try to latch affordances for *dealing with* the snake. Perception is not impersonal; every perception includes affordances for interaction, i.e. ”what can I do to x” and “what can x do to me”? (Wilson et al; big thanks to my friend Warren for the pointer). So the trauma response latches both fragments of the feeling of the snake biting your leg, and (at least for many men) fragments of the feeling of kicking the snake.
Now if you trip over a snake *or* a stick, you are going to kick the living daylights out of it
Humans are social animals and many of our salient threats and accompanying traumas are social — we can extrapolate this basic “frozen shards of perception+affordances” pattern to e.g. all the conflicts and attachment trauma we had when we were too young to defend ourselves (physically, emotionally, socially). An added complication is that the logic of how to identify these bad situations, and how to deal with them, will be latched shards of how we perceived the world when we were small. Which are probably not ideal. To be human is to navigate the world with this system, and navigate the second-order effects of this system
A compelling aspect of this model (I think @jonnym1ller is calling this “latch theory”, which I really like) is that these latches are concrete physical things. They are discrete, localized, computationally significant, measurable, and openable
(See my “research context” note in this thread for papers/links)
Filip Dousek retweeted
Lovable is the fastest growing startup in Europe ever, $4m ARR in 4 weeks, this is how we did it:
//1
Filip Dousek retweeted
Cracked team in Europe:
Most of us are misfits in some way. But also have many dimensions of genius.
We have a mission that unites us: Build the last piece of software.
And to play this on hard mode, we’re doing it headquartered in Europe. To show it can be done. And to MEGA (make europe great again).
//5
Filip Dousek retweeted
The new paradigm is LLM has officially become a CPU and test time compute resembles time complexity, and context resembles space complexity. Natural language “program” is the universal algorithm guided by thoughts — going through steps of guessing, verifying, searching, branching, learning and self correcting.
