ml • search • abstraction • cog sci
Texas
Joined December 2021
- Tweets714
- Following632
- Followers148
- Likes5.1K
Doing some prototyping with Jev from @typesafeai this week.
These system one models are really neat. And they are very friendly for vibe prototyping and rapid exploration of product ideas.
From a conversation this evening...
I'm not really "writing" code. So it feels weird to say that. I sometimes say "generated" code which feels more accurate but weird.
The conversation wove through TV, movies, and film making. Out came the suggestion from my father in-law to use their terms. Created by. Directed. Produced. This feels like a much better match...
Make your prompts simple. Minimize mcp, minimize local tools, minimize agents md, minimize your input.
Taking a little time to craft well written instructions will foster better results with the current models.
Another super fun idea, where a model just changes another model's weights directly to improve outcomes. A model mechanic giving another model a tune-up.
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
A good thread about the dualities of science
Absolutely in love with his conversation w/ @dr_mcgilchrist .
Science needs both rigor and imagination. Our methods help us test what we think we know, while curiosity helps us see connections and ask questions that don’t fit so perfectly within one field.
We haven’t seen the same kind of major breakthroughs since the 1970s, probably in part because we’ve lost the capacity to follow the intuition and curiosity that help us see how the pieces connect to the whole.
Why have we become so focused on the details, and so concerned with maintaining rigor, that we’ve lost sight of the bigger picture? Is it funding? Is it how we train the next gen of scholars?
With so much instability in science right now, how do we use this high entropic state to carve a new path toward discoveries that last and actually help our lives, our communities and the world flourish?
It's high time we as scientists have an open conversation about this.
Tapping a few fellow scientists , love to hear your thoughts/suggestions to embrace
@LocasaleLab @JasonSynaptic @slavov_n @itchdoctor @drmichaellevin @skdh @leslievosshall @MitoPsychoBio
There are some wild things happening lately, where the weights and related processes of LLMs are being explored as their own substrate. Computational processes, subliminal storage, etc.
divagr.com/blog/qwenputer
This is cool and a creative example of adapting Jev / decision models to various tasks
github.com/jkrup/jeveryword
One big advantage of LLM APIs is simplicity of use. Two big inputs, each free form text. One output of arbitrary text. This simplicity fosters adoption within the software domain.
The system one models require more work to frame problems for effective application. They are super powerful but this will hinder or slow diffusion.
The day is done. Happy birthday bek. I think about you all the time.
A favorite memory:
Early morning, heading to the office / data center. We'd probably just finished breakfast at Denver diner but I don't recall. We're heading up Speer in the Saturn wagon. I'm in the passenger seat as you swing left onto Wazee with Modest Mouse cranked up.
I guess context engineering is the new feature engineering.
It's still kind of wild how powerful end-to-end learning and learned representations can be.
loop(org --> do)
One thing I'm grappling with is how the model of implementation you describe affects the planning/organizing on the other side of the loop of iteration.
Most work abstracts to an organize-then-do loop. What I'm finding is the ease of generating implementation and local tools is driving me to sometimes think through doing which sort of blends the two parts of organize-then-do.
Remember how Dr. Isbell,
in his Machine Learning class,
said he didn't care about where you got the code for your projects?
That comes to mind a lot. It's especially interesting now.