Opinions are my own. Ph.D. student at @RiceCompSci in @vislang. Previously @RealityLabs @AdobeResearch, @InariAILab, @AdaVivInc.
Houston, Texas
Joined December 2017
- Tweets612
- Following457
- Followers82
- Likes23K
Jefferson Enrique Hernandez Cevallos retweeted
Was reading the score-centering RL paper to add some notes on it the my book, and they have a great table in the appendix on the different importance sampling based train-inference mismatch methods in RL. These have been the biggest focus of RL algorithm research since the GRPO takeoff, imo.
Jefferson Enrique Hernandez Cevallos retweeted
Foundation models are trained in stages: pretraining, midtraining, and post-training. But is this separation fundamental?
We introduce unified training: a single algorithm for the full pipeline. This gives
⚙️ Simpler infrastructure
🔬 A framework for better science
(1/n)
Jefferson Enrique Hernandez Cevallos retweeted
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
Jefferson Enrique Hernandez Cevallos retweeted
You can efficiently estimate calibration errors without binning. With @BachFrancis @JordanMich45361 @DHolzmueller and @Eugene_Berta . The catch is to write the calibration error as a drop in risk for the right loss: no bins, a lower bound, and over/under-confidence for free.
Jefferson Enrique Hernandez Cevallos retweeted
This paper went completely under the radar!
Meta researchers have developed a novel and simple self-supervised learning framework competitive with DINOv2 and DINOv3!
"Scalable Patch-Level Self-Supervised Learning"
The authors introduce JEM, a principled student-teacher method that learns to align patch representations across different views. Additionally, it incorporates explicit regularization to ensure representations are informative and prevent structural collapse.
The framework outperforms DINOv2 in a controlled setting on dense tasks, and even beats the DINOv3 checkpoint despite it being trained on 12x less data.
Jefferson Enrique Hernandez Cevallos retweeted
Tinker's new StableReinforce objective is Reinforce + Score Centering. Here's how I reverse-engineered it 🧵
Off-policy RL is difficult; runs can suddenly collapse because of async RL staleness or numerical mismatch.
StableReinforce is a new method on Tinker that keeps your RL runs stable, even when learning drifts from sampling.
tinker-docs.thinkingmachines…
Jefferson Enrique Hernandez Cevallos retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
Jefferson Enrique Hernandez Cevallos retweeted
-----BEGIN RSA PRIVATE KEY-----
4
-----END RSA PRIVATE KEY-----
Jefferson Enrique Hernandez Cevallos retweeted
Are ViTs really as explainable as we think?🔍
We show that patch-token explainability (like in Logit Lens) is not inherent to ViTs, as commonly assumed!
Instead, it's a fragile byproduct of their training conditions.
1/7
🔗ronensc.github.io/Explainabi…
w/ Ronen Schaffer,@t_michaeli
ALT While in some ViTs object tokens score higher than background tokens as we might expect, in other ViTs background tokens are hotter. In some ViTs explainability maps can even be unexplainable!
Jefferson Enrique Hernandez Cevallos retweeted
Can you sample a robot policy the way you sample an image? 🎲
No RL run. No reward. No fine-tuning.
We train a diffusion model on the 𝘄𝗲𝗶𝗴𝗵𝘁𝘀 of thousands of RL policy networks, then sample brand-new ones that walk 🚶, dance 🕺, and spin-kick 🦵 on a real Unitree G1 🤖
The best part? The 𝘄𝗲𝗶𝗴𝗵𝘁𝘀 have a secret 👀 Every network, no matter the motion, pays the most attention to the hips and waist — embodiment comes first; motion comes second. 🦾
📄 arXiv: arxiv.org/abs/2610.20606
🌐 Website: humanoid-weight-diffusion.gi…
#Robotics #HumanoidRobots #DiffusionModels #RobotLearning #EmbodiedAI
Jefferson Enrique Hernandez Cevallos retweeted
Humans think visually and abstractly 💭: we built a dataset for learning to solve text-based problems with visual chain-of-thought! This work will also be presented at EMNLP 2026 Findings 🔜
We are presenting COVA-SFT in BayLearn 2026!
A lot of researchers are visiting this problem, and you should definitely check this dataset out!!
Humans think visually, even for text-only problems. We generated a large scale public dataset to use.
Paper: arxiv.org/abs/2608.28958
Jefferson Enrique Hernandez Cevallos retweeted
How should models continually learn from new data?
SFT on heavily post-trained models often forgets and fails to generalize, so many works try to make the data more on-policy, e.g., on-policy self-distillation, Pedagogical RL...
If you've tried, you might find it’s tricky to balance on-policyness and data quality. What if on-policy isn't necessary? We show that off-policy training actually works well for continually learning, if done right!
🧵1/7
Jefferson Enrique Hernandez Cevallos retweeted
Today, most multimodal LLMs are basically a vision encoder stitched to a language model.
The alternative is to drop the vision backbone entirely and project raw image patches straight into the decoder.
But which one scales better: a rich visual prior or a simpler architecture?
1/n
ALT All images from https://arxiv.org/abs/2609.35457v1
Jefferson Enrique Hernandez Cevallos retweeted
Can complex multi-step reasoning emerge purely from cells that only talk to immediate neighbors?
Happy to share our paper “Reasoning with Neural Cellular Automata (NCA)”, from our team at Google, Paradigms of Intelligence 🧵👇
Jefferson Enrique Hernandez Cevallos retweeted
New research (165 pages!)
We thought there were too many benchmarks for video understanding, so we squeezed the benchmarks dry and got video-index
(cover inspiration: gradient canopy 😜
Jefferson Enrique Hernandez Cevallos retweeted
We let an agent self-improve by exploring and internalizing ”on this sort of task, keep this sort of thing in mind“ bits of self-feedback. It learned to solve tasks where the original policy failed 128 attempts in a row and GRPO flatlined.
Paper: arxiv.org/abs/2609.37633
🧵1/8
Jefferson Enrique Hernandez Cevallos retweeted
Most people think JEPAs are inherently non-contrastive. But recent models (LeJEPA, LeWM, etc.) use the SIGReg regularizer, which approximates a sliced MMD whose expansion reveals pairwise repulsion between samples in the batch.
Jefferson Enrique Hernandez Cevallos retweeted
With every new release, frontier models seem to do more in computer vision, including tasks we once thought only specialist models could do well.
🚀 𝗛𝗼𝘄 𝗺𝘂𝗰𝗵 𝗼𝗳 𝗰𝗼𝗺𝗽𝘂𝘁𝗲𝗿 𝘃𝗶𝘀𝗶𝗼𝗻 𝗰𝗮𝗻 𝗳𝗿𝗼𝗻𝘁𝗶𝗲𝗿 𝗺𝗼𝗱𝗲𝗹𝘀 𝗵𝗮𝗻𝗱𝗹𝗲 𝘁𝗼𝗱𝗮𝘆?
1/4
Jefferson Enrique Hernandez Cevallos retweeted
Paper thread! I recently read this paper from some FAIR colleagues and NYU folks in depth, so making a thread while it's fresh in my head.
The main point is seeing how far on the SMALL side we can go in scaling laws without losing fit or predictability, and what it takes?
