Postdoc @Mila_Quebec @UMontreal @IVADO_Qc Like to explain unexpected behavior of neural nets 🤯

Montréal, Québec
Joined December 2018
🚨 New paper alert: Loss Smoothing for Stable Adaptation Under Distribution Shift 💡Don't switch objectives abruptly — anneal from source to targe. This preserves useful features and improves adaptation in fine-tuning, CL, and offline-to-online RL. Thread by @dapatil211 👇
Excited to share our latest work: Loss Smoothing for Stable Adaptation Under Distribution Shift. TL;DR: When adapting a model, don’t abruptly switch to the new objective. Gradually smooth the transition from the old objective to the new one. Preprint: arxiv.org/abs/2607.00634 🧵 1/n
1
3
12
2,336
Ekaterina Lobacheva @ ICML 2026 retweeted
New Paper: Subliminal Learning doesn’t need a shared base model 🤯. We show how a “teacher’s” hidden animal preferences reaches the “student” even when both differ in weight initialization, data order, pretraining corpus or even architectures! 🧵
11
26
10
236
18,840
Ekaterina Lobacheva @ ICML 2026 retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
135
785
145
6,452
781,766
Ekaterina Lobacheva @ ICML 2026 retweeted
Quite happy to say that our paper on representation learning for dynamical systems was accepted to #NeurIPS2026 and I'll be presenting it in Sydney this December! Very succinctly, our work, titled "Sparse Koopman Autoencoders Identify Local Dynamical Regimes in Multibasin Systems", is about learning linear representations of state for dynamical systems with multiple basins of attraction. Inducing sparsity in a latent representation can be a very powerful method to model multiple basins of attraction with linear dynamics. By training a Koopman autoencoder with a bias for producing sparse latent representations, we find that a sparse-latent model both forecasts the dynamics better and has greater ability to identify the current local regime of the current state compared to a model without sparsity-inducing effects. More broadly, I think our work on sparse Koopman autoencoders could be relevant to both the robotics/world modeling and interpretability communities (just brainstorming here). - Robotics: To my knowledge, switching dynamics are common in robotic systems, i.e. trajectories seem to transition between dynamical modes. With a sparse code, you could get a switch between modes for free. - Interpretability: Sparse autoencoders might not be all the rage anymore, but I think Koopman modeling is still very natural for understanding the dynamics of models that apply the same computation at multiple steps to advance the dynamics. This applies to most recurrent models, such as looped/weight-tied transformers (dm me if you want to discuss!) Thanks to my amazing collaborators @ukreddy126 @mahanfathi @apsarathchandar @RGoroshin. I was thinking about what to say about this (clueless grad student tries to figure out social media) and this post ended up being really late lol, my bad. See you in Sydney! Paper: arxiv.org/abs/2608.29057 Code: github.com/aidanmrli/skae
3
23
3
179
6,836
Ekaterina Lobacheva @ ICML 2026 retweeted
New paper! 🧵 In communication-efficient distributed training, workers take several local steps, and an outer optimizer (often with Nesterov momentum) applies their averaged update to accelerate training. We study when this is stable, and derive an exact criterion (in the mean-square sense).
1
6
4
46
6,264
Join Mila 🔥
Mila's annual supervision request process is now open to receive MSc and PhD applications for Fall 2027 admission! For more information, visit mila.quebec/en/prospective-s…
1
187
Ekaterina Lobacheva @ ICML 2026 retweeted
How much representational diversity exists around a trained network? A lot! Moving a ViT’s weights by 1.3% while preserving its predictions, its representations end up less similar to the original than a random-init ViT’s ‼️ New paper: Hessian Null Space Continuation (HNC) 💫🧵 (1/12)
9
55
8
354
30,591
Ekaterina Lobacheva @ ICML 2026 retweeted
Can a 1M-parameter model that has never seen an image help an 85M-parameter ViT learn to see? We train a tiny recurrent model on abstract data, then expand its weights. ImageNet-1k top-1: 77.6% → 79.8% in our main setting. Introducing Procedural Core 🧵
7
17
2
108
9,509
Ekaterina Lobacheva @ ICML 2026 retweeted
Introducing RoboJEPA, the first scaling law for multiembodiment world models. We scaled JEPA predictors up to 8B params, showed scaling laws extrapolate and correlated compute scaling with planning performance. We release all checkpoints and the code. robojepa.github.io
38
137
9
796
52,805
Ekaterina Lobacheva @ ICML 2026 retweeted
1/ At COLM? Check out our poster "(How) Learning Rates Regulate Catastrophic Overtraining" during the second poster session on Tuesday in the Grand Ballroom! Not at COLM? Read the thread below 👇
2
8
1
35
4,167
🎓Applying to AI grad programs? Our lab offers 1:1 application feedback for students from underrepresented communities. Apply by Nov 1 👇
🎓 Graduate Program Application Mentorship Applying to grad school doesn't have to be overwhelming! Get personalized 1:1 support from students who’ve successfully navigated the process. 🔗 Apply now: forms.gle/FyjEuLmjcdREF3Ba6 🗓️ Deadline: Nov 1, 2026 👇 Here is what you get:
3
235
Ekaterina Lobacheva @ ICML 2026 retweeted
🎉 Thrilled to share that 8 papers from our lab (and collaborations with lab members) have been accepted at #NeurIPS2026! The work spans RL, world models, dynamical systems, LLM safety & interpretability, tabular foundation models, and forecasting. Huge congrats to everyone 🧵👇
2
20
1
61
4,643
Ekaterina Lobacheva @ ICML 2026 retweeted
meet @mostik_ai! what happens when you put 12 PhDs in one room for four months? first place on the ARC-AGI leaderboard, which I can't say much about while the competition is still running. and this, which I can. everyone's arguing about whether open models will catch up to frontier models. we think it's the wrong question. here's the one we pose: why does a frontier model have to generate your answer at all, when the only thing you need from it is the reasoning? we do this by enabling models to communicate in latent space. through our protocol, hidden states pass straight from a frontier model into a small one running on your infrastructure -- no text between them, and neither model is fine-tuned. two models from different families, sharing reasoning, both left untouched. how do we know it works? we tested it on a setup where a 753B model reads the problem, and a 4B edge-class model writes the answer. with this approach, we get results 80% as accurate as the frontier model, but at 20x faster performance. we're committed to preventing frontier model lock-in and are already partnering with inference providers to accelerate open-weight adoption. we've done this between 15 of us, in four months, 12 PhDs and a Fields medalist, backed by @generalcatalyst WIRED has the first external account of the company and the work: wired.com/story/russian-star… full writeup, the setup, and all the numbers: mostik.ai/read-more
295
259
255
2,447
1,281,137
Ekaterina Lobacheva @ ICML 2026 retweeted
Simple ideas often generalize best! Loss smoothing is one of those really simple ideas that works surprisingly well for adaptation under distribution shift. Really excited about this line of work!
Our third invited talk by Sarath Chandar (@apsarathchandar) was about: 🎙️ Stable Adaptation Under Distribution Shift 🎥 youtube.com/watch?v=eddBBYye…
5
57
5,371
Ekaterina Lobacheva @ ICML 2026 retweeted
The Bayesian interpretation is insufficient for Tabular Foundation Models. A model trained on MNIST generalizes in-context to California Housing, even though the prior places no support on the downstream task. In this new pre-print, we try to explain why. (arXiv: 2608.17957)
1
2
1
20
2,422
Curious about the connections of neural network training dynamics and statistical physics? Check out our new #IJCAI2026 paper showing that scale-invariant NNs behave like ideal gases at stationarity! Thread by @irsadrtdinov 👇
💨 Did you know neural networks can behave like literal ideal gases? 💨 In our #IJCAI2026 paper, we show that at stationarity, scale-invariant NNs surprisingly obey... the ideal gas law 🤯 Thermodynamics helps us to understand how train hyperparams shape final solutions! 🧵👇1/8
2
5
46
4,354
Ekaterina Lobacheva @ ICML 2026 retweeted
New paper! Synthetic Persona Pretraining (SPP): Alignment from Token Zero Rather than aligning models after pretraining, we do so from the very start. SPP models are more faithful to our constitution, less misaligned, & more robust to jailbreaks. Advantage grows with scale! 🧵
3
24
8
111
15,182
Ekaterina Lobacheva @ ICML 2026 retweeted
Turns out that many contemporary LLMs still struggle to explain probabilities and uncertainty in plain natural language 👀
1
12
2
14
3,453
Ekaterina Lobacheva @ ICML 2026 retweeted
For the third year in a row, we held our annual Highschool Internship for recent graduates from across Montreal! 🎓🎉💻 Ten bright students underwent a month-long summer school and got introduced & coached on the basics of computer science and programming!
1
8
2
26
5,253
Ekaterina Lobacheva @ ICML 2026 retweeted
📣 We are organizing the first InterpScience Workshop @ NeurIPS 2026 in Sydney! The goal of the workshop is to build a more rigorous scientific foundation of LLM interpretability. 📝 Papers due: Aug 28 🌐 interpscience.github.io/ ✉️ [email protected] [1/7]
3
15
3
102
13,647