Co-founder OuterProduct. CS PhD @UCSanDiego. https://nitter.gq/t.co/K0XhWFo3GK Deep learning from first principles (+ applications) Topics include xRFM, AGOP, Colonel Blotto
Joined August 2021
- Tweets235
- Following623
- Followers418
- Likes794
Pinned Tweet
What is the nature of feature learning in deep networks? We propose that neural networks recover a statistic known as the average gradient outer product (AGOP).
Github: github.com/aradha/recursive_…
arXiv: arxiv.org/abs/2212.13881
Daniel Beaglehole retweeted
Is there a safe path to releasing open-weight models? We think so.
A few takes on AI safety for your Friday afternoon 🧵
Releasing weights indiscriminately isn't safe. Neither is keeping capable models inside a few labs.
We think there's a path between them. We haven't mapped all of it. Our new post covers the part we can see: how we assessed Inkling, and why access should widen in stages.
thinkingmachines.ai/blog/a-s…
Daniel Beaglehole retweeted
The model factory is open @thinkymachines! Meet our first open-weights model: Inkling.
Today, we are introducing Inkling.
Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available.
thinkingmachines.ai/news/int…
Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵
Daniel Beaglehole retweeted
The @DARPA AIQ program! Mathematical foundations for AI evaluation.
Great work by @dbeagleholeCS, Adityanarayanan Radhakrishnan, Enric Boix-Adserà, and Misha Belkin
science.org/eprint/URRDNX9WE…
Daniel Beaglehole retweeted
The idea of using locality has appeared from time to time in the multi-index literature. Here is a nice operationalization, using a "local expected gradient product (EGOP)"-based continuously varying index subspace. arxiv.org/abs/2601.07061
Daniel Beaglehole retweeted
New paper studies when spectral gradient methods (e.g., Muon) help in deep learning:
1. We identify a pervasive form of ill-conditioning in DL: post-activations matrices are low-stable rank.
2. We then explain why spectral methods can perform well despite this.
Long thread
Really nice to see independent validation of xRFM on tabular data. On a benchmark of 300 datasets, xRFM outperforms all Gradient Boosted Trees - XGBoost, CatBoost, LightGBM, etc. - and is in-line with the top 1-2 neural networks (such as TabPFNv2)
arxiv.org/abs/2407.00956
See the nice benchmark from the TALENT group that produced these results:
github.com/LAMDA-Tabular/TAL…
3
167
Our method combines tree-based spatial partitions with feature learning kernel machines (tabular optimized based Recursive Feature Machine models, arxiv.org/abs/2508.10053).
1
4
171
For the base implementation you can try xRFM at: github.com/dmbeaglehole/xRFM. We also offer xRFM with automatic preprocessing and hyperparameter tuning through pytabkit: github.com/dholzmueller/pyta… (thanks @DHolzmueller)
4
134
Really cool application of steering for music generation!
We found a way to steer AI music gen toward specific notes, chords, and tempos, without retraining the model or significantly sacrificing audio quality!
Introducing MusicRFM 🎵
Paper: arxiv.org/abs/2510.19127
Audio: astradzhao.github.io/MusicRF…
Code: github.com/astradzhao/music-…
(1/5)
3
450
Daniel Beaglehole retweeted
Update:
we were able to close the gap between neural networks and reweighted kernel methods on sparse hierarchical functions with hypercube data. Interestingly the kernel methods outperform carefully tuned networks in our tests.
I highly recommend trying out our new model xRFM, which scales RFM to millions of samples. xRFM is SOTA on regression and among the top few methods for classification on 300 datasets compared to 31 other methods including neural nets / trees like XGBoost, TabPFN-v2, RealMLP, etc
xRFM: Accurate, scalable, and interpretable feature learning models for tabular data ift.tt/P3pDd9f
Our method combines RFMs with tree-based splits to achieve log linear scaling (basically linear but we have use median computation at every tree node) in the number of samples.
1
5
244
This was an amazing collaboration with @DHolzmueller, Adit Radha (aditradha.com), and Misha Belkin (misha.belkin-wang.org)
3
206
Daniel Beaglehole retweeted
xRFM: Accurate, scalable, and interpretable feature learning models for tabular data ift.tt/P3pDd9f
Daniel Beaglehole retweeted
Super excited to share that we have an Oral presentation for this paper next week at ICML!
It will be on Tuesday at 10am (Oral 1E) in West Ballroom D, I'll be presenting 4th at 10:45am :)
Our poster will be on Wednesday at 11am and I encourage you to stop by and chat!
Daniel Beaglehole retweeted
🚨 What if you could hijack any LLM's brain using external expert models? ExpertSteer does exactly that! 🧠⚡
Meet ExpertSteer: a breakthrough that lets you inject expert knowledge into any LLM, guiding its responses without updating model parameters.
Really cool! I have found the multi-index model (especially the sparse-coordinate type) to be a useful abstraction for some practical scenarios, e.g. tabular data as they consider here
Iteratively reweighted kernel machines efficiently learn sparse functions ift.tt/mGXAes3
5
620
Thank you to the Simons Institute for hosting me! It is always a pleasure!
Stay tuned for an updated draft of our paper with diverse steering examples. I also mention adding feature learning to kernel machines on tabular data, which I am excited about! More to say soon...
Can we tell when LLMs hallucinate or produce toxic/harmful content? Fascinating talk by @dbeagleholeCS at the Simons Institute about detecting semantic features learned by LLMs and using them to steer their output, & the link to recursive feature machines.
youtube.com/watch?v=TlLu32yH…
Daniel Beaglehole retweeted
Can we tell when LLMs hallucinate or produce toxic/harmful content? Fascinating talk by @dbeagleholeCS at the Simons Institute about detecting semantic features learned by LLMs and using them to steer their output, & the link to recursive feature machines.
youtube.com/watch?v=TlLu32yH…