PhD Student at @McGillU / @Mila_Quebec; AI Safety
Joined September 2020
- Tweets113
- Following205
- Followers212
- Likes593
Nicholas Meade retweeted
Excited to share that I have joined @MistralAI, where I will be building a team in Montreal and across Canada to push the frontier of safe, capable, and open LLMs.
Mistral has played, and will continue to play, an important role in keeping frontier AI open and advancing it safely. I deeply believe in Mistral's mission, and since joining, that conviction has only grown after seeing firsthand the ambition and commitment of the people here.
A big part of my role will also be building strong bridges between Mistral and @McGillU, @Mila_Quebec, and the broader Montreal and Canadian AI ecosystem.
And we are hiring! Across levels and across the LLM stack. If you are excited about pushing the frontier, reach out.
Can you release a model's weights openly and still control who gets access to its more sensitive capabilities?
Tiered Language Models keep those capabilities in the weights but gate access behind a secret key.
Check out this approach to safe open-weight release in our paper:
Nicholas Meade retweeted
Excited to share our new paper!
“Forecasting Downstream Performance of LLMs With Proxy Metrics”
w/ my amazing advisors @sivareddyg, @mariusmosbach, @DBahdanau
Cross-entropy loss is a poor predictor of how models perform on downstream tasks (esp. reasoning). We propose something better: proxy metrics computed over expert reasoning traces.
🧵 Thread below 👇
Nicholas Meade retweeted
SAEs fail at OOD tasks. Why?
Features in superposition are linearly representable but not linearly accessible. Instead of discarding sparse coding, we embrace the geometry of superposition and use methods equipped to handle the nonlinearity it induces.
Nicholas Meade retweeted
Mechanistic interpretability aims to understand models — and the more superhuman or incoherent they become, the more we need that understanding to be reliable. We propose a framework for this, drawing on established tools from causal reasoning and statistical identifiability:
🧵
Nicholas Meade retweeted
Checkout our latest work on building self-supervised text embeddings without relying on contrastive data. ☝️
The main idea behind LLM2Vec-Gen is trying to encode a model's answer to a query, rather than the query itself.
Your LLM already knows the answer. Why is your embedding model still encoding the question?
🚨Introducing LLM2Vec-Gen: your frozen LLM generates the answer's embedding in a single forward pass — without ever generating the answer. Not only that, the frozen LLM can decode the embedding back into text.
🏆 SOTA self-supervised embeddings
🛡️ Free transfer of instruction-following, safety, and reasoning
Nicholas Meade retweeted
Your LLM already knows the answer. Why is your embedding model still encoding the question?
🚨Introducing LLM2Vec-Gen: your frozen LLM generates the answer's embedding in a single forward pass — without ever generating the answer. Not only that, the frozen LLM can decode the embedding back into text.
🏆 SOTA self-supervised embeddings
🛡️ Free transfer of instruction-following, safety, and reasoning
Nicholas Meade retweeted
LLM2Vec-Gen represents a major paradigm shift for embeddings/retrieval. Why encode the query when the LLM already knows what to look for and can directly produce an embedding for it?
Best part: it’s self-supervised, and it does all of this while the LLM remains completely frozen.
Think about it: "solve x² + 3x − 4 = 0" has zero reasoning in it. But the LLM's response does. By encoding the response, the embedding captures the reasoning --- and the better the LLM reasons, the better the embedding. This is why our results scale with model size. As LLMs get smarter, our embeddings automatically get better.
LLM2Vec-Gen is also the first demonstration of the promise of @ylecun's JEPA for text embeddings. The alignment loss is JEPA — predict in representation space, not token space. The reconstruction loss goes beyond --- it keeps embeddings decodable.
This paradigm shift opens new frontiers:
🔬 Can we build a full JEPA for language where the teacher and student are the same LLM?
⚡ Can LLMs reason in compressed space without ever generating text?
🤖 Can agents reason in compression tokens and carry that directly into retrieval?
💬 Can agents talk to each other in compression tokens instead of text --- dense, fast, and still human-readable?
LLM2Vec-Gen is a first step toward all four.
Your LLM already knows the answer. Why is your embedding model still encoding the question?
🚨Introducing LLM2Vec-Gen: your frozen LLM generates the answer's embedding in a single forward pass — without ever generating the answer. Not only that, the frozen LLM can decode the embedding back into text.
🏆 SOTA self-supervised embeddings
🛡️ Free transfer of instruction-following, safety, and reasoning
Nicholas Meade retweeted
I wrote my first ever blog post! "Agentic Coding: A New Abstraction Layer in the Programming Stack"
I give some thoughts about how my coding changed with agents, where I think it's headed, and how the resistance to adopting them echoes past shifts in CS.
Link below👇
Nicholas Meade retweeted
Honored to be a Sloan Fellow. So grateful to my wonderful students, mentors, colleagues, friends and family, thank you! ❤️
Congrats to the 126 early-career scholars awarded a 2026 Sloan Research Fellowship, whose creativity and innovation set them apart as the next generation of scientific leaders! Our Fellows represent 7 fields and 44 institutions across the US and Canada.
sloan.org/fellowships/2026-F…
ALT 2026 Congratulations Sloan Research Fellows!
Nicholas Meade retweeted
🚨New paper
Are visual tokens going into an LLM interpretable 🤔
Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”...
We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡
Details 🧵
Nicholas Meade retweeted
🚨How do LLMs acquire human values?🤔
We often point to preference optimization. However, in our new work, we trace how and when model values shift during post-training and uncover surprising dynamics.
We ask: How do data, algorithms, and their interaction shape model values?🧵
Nicholas Meade retweeted
It’s clear next-gen reasoning LLMs will run for millions of tokens. RL at 1M needs ~100× compute than 128K. Our Markovian Thinking keeps compute scaling linear instead. Check out Milad’s thread; some of my perspectives below:
Nicholas Meade retweeted
Introducing linear scaling of reasoning:
𝐓𝐡𝐞 𝐌𝐚𝐫𝐤𝐨𝐯𝐢𝐚𝐧 𝐓𝐡𝐢𝐧𝐤𝐞𝐫
Reformulate RL so thinking scales 𝐎(𝐧) 𝐜𝐨𝐦𝐩𝐮𝐭𝐞, not O(n^2), with O(1) 𝐦𝐞𝐦𝐨𝐫𝐲, architecture-agnostic.
Train R1-1.5B into a markovian thinker with 96K thought budget, ~2X accuracy 🧵
Nicholas Meade retweeted
i will be presenting AgentRewardBench at
#COLM2025 next week!
session: #3
date: wednesday 11am to 1pm
poster: #545
come learn more about the paper, my recent works or just chat about anything (montreal, mila, etc.)
here's a teaser of my poster :)
AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories
We are releasing the first benchmark to evaluate how well automatic evaluators, such as LLM judges, can evaluate web agent trajectories.
We find that rule-based evals underreport success rates, and no single LLM judge excels across all benchmarks.
We collect trajectories from web agents built on four LLMs (Claude 3.7, GPT-4o, Llama 3.3, Qwen2.5-VL) across popular web benchmarks (AssistantBench, WebArena, VWA, WorkArena, WorkArena++).
An amazing team effort with: @a_kazemnejad @ncmeade @arkil_patel @dcshin718 @alejaz_a @karstanczak @ptshaw2 @chrisjpal @sivareddyg
Nicholas Meade retweeted
🚨 New paper!
LLMs, when asked harmful questions, sometimes produce outputs that look helpful (and harmful) — but are actually 𝗱𝗲𝗹𝗶𝗯𝗲𝗿𝗮𝘁𝗲𝗹𝘆 𝘄𝗿𝗼𝗻𝗴
What’s bad - current LLM-based jailbreak scorers can’t tell the difference (me neither)
More in 🧵👇
If you're interested in working on agent safety (and are a student in Canada) you should apply to this! @gspandana is extremely smart and one of the kindest people I've gotten to work with
Internship @ServiceNowRSRCH to build the next generation of computer use agents that
are safe and secure from malicious attacks. Focus on intervention strategies, defenses to make agents robust against unsafe behavior..
Apply here: bit.ly/3V3mmTg
Nicholas Meade retweeted
🚨 Incredibly excited to share that I'm starting my research group focusing on AI safety and alignment at the ELLIS Institute Tübingen and Max Planck Institute for Intelligent Systems in September 2025! 🚨
Hiring. I'm looking for multiple PhD students: both those able to start in Fall 2025 (i.e., as soon as possible) and through centralized programs like CLS, IMPRS, and ELLIS (the deadlines are in November) to start in Spring–Fall 2026. I'm also searching for postdocs, master's thesis students, and research interns. Fill the Google form below if you're interested!
Research group. We will focus on developing algorithmic solutions to reduce harms from advanced general-purpose AI models. We're particularly interested in alignment of autonomous LLM agents, which are becoming increasingly capable and pose a variety of emerging risks. We're also interested in rigorous AI evaluations and informing the public about the risks and capabilities of frontier AI models. Additionally, we aim to advance our understanding of how AI models generalize, which is crucial for ensuring their steerability and reducing associated risks. For more information about research topics relevant to our group, please check the following documents:
- International AI Safety Report,
- An Approach to Technical AGI Safety and Security by DeepMind,
- Open Philanthropy’s 2025 RFP for Technical AI Safety Research.
Research style. We are not necessarily interested in getting X papers accepted at NeurIPS/ICML/ICLR. We are interested in making an impact: this can be papers (and NeurIPS/ICML/ICLR are great venues), but also open-source repositories, benchmarks, blog posts, even social media posts—literally anything that can be genuinely useful for other researchers and the general public.
Broader vision. Current machine learning methods are fundamentally different from what they used to be pre-2022. The Bitter Lesson summarized and predicted this shift very well back in 2019: "general methods that leverage computation are ultimately the most effective". Taking this into account, we are only interested in studying methods that are general and scale with intelligence and compute. Everything that helps to advance their safety and alignment with societal values is relevant to us. We believe getting this—some may call it "AGI"—right is one of the most important challenges of our time.
Join us on this journey!
Nicholas Meade retweeted
What's the path to scalable and safe web agents? Is web agents the new semantic parsing? I will be giving a talk at the ACL REALM workshop today at 9:30 am. Come check out if you are interested in the history and contemporary work in this area. Lot of other exciting speakers. #ACL2025 #ACL2025NLP
nitter.gq/nouhadziri/status/1950…
ALT The path to scalable and safe web agents. Speaker: Siva Reddy. Affiliations: Mila, McGill NLP, ServiceNow Research
Come join us tomorrow at the 1st LLM agents workshop in ACL (REALM); amazing talks and oral presentations are ahead, with an incredible lineup of speakers🔥🔥 #ACL2025 #ACL2025NLP 🇦🇹🇦🇹
Where? Room 1.61-62
When? tomorrow Thursday (starting 8:45am)
realm-workshop.github.io/
