@MariaLomeli_

Researcher and engineer @AIatMeta, FAIR | PhD from @GatsbyUCL and former postdoc @CambridgeMLG

 London, UK
mlomeli1.github.io
Joined August 2022
  • Tweets149
  • Following562
  • Followers830
15Media
🚨Our new paper is out: Model Casting and Low-Parameter Gating: Towards More Sparsely Activated FFNs We present model casting, a mid-training recipe that drastically sparsifies the activations within the FeedForward Network layer of a transformer. Model casting consists of switching to an activation that outputs zeros for all negative inputs (R-S+) and using an L1 loss to set the required sparsity as a target at training time.
2
12
2
30
5,595
Maria Lomeli retweeted
Arthur Gretton@ArthurGrettonOct 25
PhD applications at @GatsbyUCL are open! Full funding available regardless of nationality. Includes ✨new✨ Jump Trading studentships in AI with potential application to finance, eligible for employment opportunities with Jump Trading during PhD. Deadline: 🚨November 9th🚨
Gatsby Computational Neuroscience Unit@GatsbyUCLSep 15
Interested in a #PhD in machine learning & theoretical neuroscience? Applications for 2027 entry to the Gatsby Unit PhD programme now open! 💰 Fully-funded 4-year programme 🤝 Close links to Sainsbury Wellcome Centre & the ELLIS Unit @ UCL (@uclcsml) ℹ️ ucl.ac.uk/life-sciences/gats…
1
19
162
7,433
📍 Paris · 24-week internship You'll be following in the footsteps of @MekalaDheeraj, @ulyanapiterbarg , @nehzux and part of a vibrant team xitter.cf/ThomasScialom/status/2… Apply here: metacareers.com/profile/job_…
Thomas Scialom@ThomasScialomOct 19
Hiring PhD interns in Paris! 🚀 We helped build Muse & focus on its post-training. Our team spans RL, evals, long-horizon agents, simulation and more Past interns joined full-time. Bonus: Muse access! 😉 Apply here (no DMs): metacareers.com/profile/job_…
3
1
30
4,275
If you are attending Neurips@Sydney, don’t forget to stop by @loiccabannes poster.
Loic cabannes@loiccabannesOct 9
Very happy to announce that Sparse Delta Memory (SDM), has been accepted at #Neurips2026 and I'll be presenting it in Sydney at the main conf in December!
4
243
Maria Lomeli retweeted
Loic cabannes@loiccabannesOct 9
Very happy to announce that Sparse Delta Memory (SDM), has been accepted at #Neurips2026 and I'll be presenting it in Sydney at the main conf in December!
9
12
1
100
3,740
Maria Lomeli retweeted
Yoram Bachrach@yorambacOct 8
Very glad to share the technical talk @maryamfazel and I gave at Meta Connect on how we built Muse Glimmer, our open-weight AI for local agents: goals, pre-training, mid-training, post-training, distillation, quantization, inference and benchmarks: developers.meta.com/resource…

Muse Glimmer: open-weight AI for local agents from Meta

Introducing Muse Glimmer, our open-weight model from Meta that is designed to power local agents. We'll go deep on architecture, benchmarks, and real-world deployments.

developers.meta.com
12
1
51
2,719
Furthermore, our technique works with off-the-shelf models such as Qwen3-1.7B and extends to MoE models. For further details please see our Arxiv paper: arxiv.org/abs/2609.31975

Model Casting and Low-Parameter Gating: Towards More Sparsely...

This paper introduces model casting, a mid-training recipe that drastically sparsifies the activations within the Feed-Forward Network (FFN) layer. With this strategy, at inference time, we first...

arxiv.org
1
2
3
225
This work was done with my amazing collaborators @agroudiev (joint first author), @DouzeMatthijs, @loiccabannes, Pierre Emmanuel Mazaré, @francoisfleuret, @maxmbeck, @algoriddle, @NailaMurray and @hjegou. Enjoy!
3
138
Maria Lomeli retweeted
Charles Arnal
@arnal_charlesSep 15
AI4Maths: we’re hiring! My team at Meta FAIR in Paris has postdoctoral and full-time research positions open. Come join us if you’re excited about: - Making frontier models better at maths - Proof assistants and formalizing mathematics at scale - Working hand in hand with the mathematical community to shape the future of mathematics—beyond “big model solves big problem” Apply here: Postdoc: metacareers.com/profile/job_… Research Scientist: metacareers.com/profile/job_… Research Scientist: metacareers.com/profile/job_… @AIatMeta
9
29
1
212
27,463
Interested in a #PhD in machine learning & theoretical neuroscience? Applications for 2027 entry to the Gatsby Unit PhD programme now open! 💰 Fully-funded 4-year programme 🤝 Close links to Sainsbury Wellcome Centre & the ELLIS Unit @ UCL (@uclcsml) ℹ️ ucl.ac.uk/life-sciences/gats…
31
1
149
16,345
Maria Lomeli retweeted
Manuel Faysse
@ManuelFaysseSep 1
Surfing on this paper buzz, if you really want to maintain performance during post-training while heavily sparsifying, instead of SWA, try "Self-Pruned KV Attention" where the model learns by itself which KVs it needs to remember outside the sliding window while discarding the rest ! StreamingLLM like designs are a subset of the configurations that can emerge through self sparsification: arxiv.org/abs/2605.14037

Self-Pruned Key-Value Attention: Learning When to Write by...

Under modern test-time compute and agentic paradigms, language models process ever-longer sequences. Efficient text generation with transformer architectures is increasingly constrained by the...

arxiv.org
Alexia Jolicoeur-Martineau
@jm_alexiaAug 31
Simple beats complicated: We show that switching to a sliding-window attention mask with attention sinks (at no cost) beats linear attention post-training. Huge thanks to my collaborators @RheaSukthanker, @CameronPashmina, and @Emy_Aze. Paper: arxiv.org/abs/2608.28444

Sliding-window beats linear attention

Due to the nature of quadratic attention, Large Language Models (LLMs) consume a lot of memory and energy: every new token costs more than the previous one, and its keys and values must be stored...

arxiv.org
1
3
35
5,030
Ya se viene! 👇 Estudiantes: ir consiguiendo la carta de recomendación y el certificado de estudiante regular – at Santa Fé, Argentina
khipu.ai
@Khipu_AIJul 15
🚀 ¡Las postulaciones para KHIPU 2027 abren pronto! Del 5 al 9 de abril de 2027, São Paulo será sede del encuentro latinoamericano de inteligencia artificial. 🚨Para participar, es necesario aplicar 🎓 Numerosas becas disponibles para alojamiento y transporte 📅 Apertura de postulaciones: 20 de julio de 2026 ⏳ Cierre de postulaciones: 31 de agosto de 2026 Conoce más sobre las ediciones anteriores en khipu.ai 📣 ¡Corré la voz! #KHIPU2027 #InteligenciaArtificial #IA #Latinoamérica #Becas #SãoPaulo
2
9
642
Must-read papers of the week ▪️ Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling ▪️ Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity ▪️ SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibe ▪️ LLM-as-a-Verifier: A General-Purpose Verification Framework ▪️ Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Process ▪️ OpenCoF: Learning to Reason Through Video Generation ▪️ dOPSD: On-Policy Self-Distillation for Diffusion Language Models ▪️ Flex-Forcing: Towards a Unified Autoregressive and Bidirectional Video Diffusion Model ▪️ TurnOPD: Making On-Policy Distillation Turn-Aware for Efficient Long-Horizon Agent Training ▪️ Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning ▪️ Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models Find the full list and the most impactful AI news here: turingpost.com/p/if-we-must-…
4
5
26
4,282
Maria Lomeli retweeted
Loic cabannes@loiccabannesJul 9
Introducing Sparse Delta Memory (SDM) - The first work of my PhD 🎓. SDM combines the GatedDeltaNet update with Product Key sparsity, enabling a recurrent state 3000x larger at the same FLOPs and significantly improving long-context performance. Let RNNs be sparse!
25
72
5
601
52,435
Load more