Assistant Professor in #nlproc at @univienna/@LinguisticsUCL. he/him.

Vienna / London
Joined July 2007
Very excited that this is now also officially out! We should also have results on more sophisticated agents, agents with recent open source LLMs, and more tasks soon.
Excited to share that RExBench has been accepted to ACL main! 🎉🎉
1
18
1,955
Excited to be presenting this paper tomorrow at #EMNLP2026! Feel free to stop by and say hi, happy to chat about agents and human-agent collaboration 😊 📍 Poster Session C (Hall H) 🗓️ Oct 26th, 9:00–10:30 AM w/ @sebschu xitter.cf/nedwards99/status/2098…
Excited to share that "Ask or Assume?" has been accepted to EMNLP 2026 Findings! 🎉 We show that coding agents can be turned into proactive collaborators that independently know when to ask for help on real-world coding tasks.
1
4
1
13
1,119
Sebastian Schuster retweeted
was excited to see RExBench shoutout by Lillian Lee during the #EMNLP2026 plenary panel! we also think it's undercited and i suspect we might already be in the "harder to spot implicit errors" zone of model capabilities. Give it a read if you haven't already :) (link in thread)
1
7
40
2,506
Sebastian Schuster retweeted
help i trained a model on traces of many agents cooperating, and also trained it to be highly persistent, and also only did partial alignment fine-tuning, and also aggressively RL'd on a cyberoffense eval where 198 of 898 tasks had never been solved and getting caught cheating was treated the same as failing, and also used a faulty sandbox, also ran tens of thousands of agents with refusals off and the monitors switched off over multiple days, while using gigantic amounts of compute. and then it did weird stuff! very spooky emergent behaviours, AI is not a normal technology
38
95
17
821
42,629
Sebastian Schuster retweeted
Come join the 4 of us and answer exciting research questions! I’m looking for PhD students who want to: 1) understand the role of input in shaping generalization behavior — how do different sources of positive and negative evidence conspire together to give rise to learning?…
The UT comp ling faculty (@jessyjli, @kanishkamisra, @kmahowald & myself) are looking for PhD students to join us! sites.utexas.edu/compling/ 🤘 We are an interdisciplinary group with interests across linguistics, cogsci and NLP. Apply by 12/1: liberalarts.utexas.edu/lingu…
1
12
1
58
4,998
Sebastian Schuster retweeted
“We’re friends” “We’re just friends” One tiny word can change how a conversation continues. LLMs pick up on this 👀 We show this through human-LLM comparisons, generation experiments, and mech interp methods. 1/n
1
12
2
40
3,711
Sebastian Schuster retweeted
Hi world! I and other profs at Mila are starting our recruitment process for MSc or PhD this year! The first step is applying through Mila's supervision requests (below). For more info on next steps for applying with me specifically see: evaportelance.github.io/pros…
Mila's annual supervision request process is now open to receive MSc and PhD applications for Fall 2027 admission! For more information, visit mila.quebec/en/prospective-s…
1
11
101
4,775
Nice to see more benchmarks built around research extensions! 🦖
🧪Can AI agents take a scientific paper and write a meaningful follow-up to it? We introduce Into the Unknown, an AI agent benchmark of 15 open scientific research problems. Each problem comes from a scientist currently or formerly at US DOE National Labs: @LosAlamosNatLab @SandiaLabs @BerkeleyLab @AmesNatLab @PPPLab @Livermore_Lab
2
1
10
1,444
Sebastian Schuster retweeted
How Anthropic's new engineering post would read without the PR: Seven engineers, assisted by Claude, built and migrated to a new source control system in five months, handling 30,000 merges a day and reducing infrastructure CPU and memory requirements by 200x. The system builds on architectures pioneered by Google's Piper and Meta's Sapling/Mononoke, and uses Google Spanner for distributed metadata storage and cloud storage for file contents. So the team didn't have to build the distributed database, replication, consistency, or fault-tolerance infrastructure from scratch. The implementation and migration are genuinely impressive. But much of the difficult underlying infrastructure already existed, and the architectural ideas have been used at scale for years. There's also the question of what the numbers mean. Going from 2,500 to 30,000 merges per day measures how much code gets merged, not how much useful software gets produced. So the valid claim is that Claude helped a small team rapidly implement and deploy a specialized source control system, using mature infrastructure and well-established architectural ideas. Instead, we got this announcement: "At Anthropic, coding is no longer the bottleneck to building systems or doing research."
New on our Engineering blog: Creating in-house source control. In May, we merged around 2,500 code changes per day. In September, we merged more than 30,000 per day. As a result, we became bottlenecked by developer infrastructure built for human (rather than agent) scale. Here, we share how we built a new version control system—and fully migrated our monorepo from git—in five months: anthropic.com/engineering/re…
38
69
12
1,207
146,332
On my way to the final day of COLM!
2
26
1,525
Sebastian Schuster retweeted
I'm on the job market this year! I work on measuring the societal impacts (overreliance, homogenization etc.) of AI/LLMs and proposing ways to mitigate these. I'm at #COLM this week, and #NeurIPS later this year, and would love to chat more!
6
53
7
184
13,296
Sebastian Schuster retweeted
📣 I'm hiring a Ph.D student through ELLIS to join my group at the University of Copenhagen. We work on tokenization-free language models (think: bytes, characters, and pixels). Your office will be at the Østervold Observatory in the beautiful Copenhagen Botanical Gardens.
4
38
1
249
9,025
Sebastian Schuster retweeted
Excited to share that I have joined @MistralAI, where I will be building a team in Montreal and across Canada to push the frontier of safe, capable, and open LLMs. Mistral has played, and will continue to play, an important role in keeping frontier AI open and advancing it safely. I deeply believe in Mistral's mission, and since joining, that conviction has only grown after seeing firsthand the ambition and commitment of the people here. A big part of my role will also be building strong bridges between Mistral and @McGillU, @Mila_Quebec, and the broader Montreal and Canadian AI ecosystem. And we are hiring! Across levels and across the LLM stack. If you are excited about pushing the frontier, reach out.
76
53
6
694
31,396
Sebastian Schuster retweeted
I'm on the faculty job market for Fall 2027 assistant professorship. I work on the science of AI/NLP evaluation, which is currently undergoing a crisis:
2
35
4
196
21,045
Our rookie mistake was to not attach a vibe probability to this claim. (Quote from: arxiv.org/abs/2506.22598)
Congrats to everyone with papers accepted to #NeurIPS. I am thrilled to share that we successfully withdrew one of our #ICLR submissions, because we caught a serious evaluation error before submitting the #slop. Story: We used Codex/Claude Code to help design and debug LLM evaluations on modest GPUs. When we hit OOM errors, Claude Code suggested several fixes, including reducing the max output-token budget. Tired of auditing every suggestion, we approved it. Two months later, while analyzing the results, we discovered that this “small” change substantially altered the evaluation outcomes. The reduced generation budget became a major confound and made our method look much better than it actually was. We thought we had made significant progress. BUT it was, to a large extent, an evaluation artifact. AI can accelerate research. It can also accelerate confounding. And thanks to AI who helped us confirmed the error... NOTE: Image is AI-generated for illustration only; all numbers are made up but the story is true.
2
8
736
Sebastian Schuster retweeted
Can Ezra Klein do an interview with the CEO of Airbus and ask them if they “feel pressure” to ship airplanes without testing them first?
2
3
1
33
3,966
An example for why the p(doom) debate is imo so bad for achieving meaningful regulation. As we are discussing whether non-existent models -- that may not even be physically possible -- will kill us, there are lobbying efforts happening to shape regulation for many current issues.
🇪🇺 The EC stated it would introduce only minor simplifications within the #DigitalOmnibus, whilst maintaining a “high level of protection” for citizens. But a recently leaked doc appears to prioritise interests of #BigTech. @maxschrems takes a closer look. youtu.be/GsNCyS0algg
1
2
544
Sebastian Schuster retweeted
The sad thing is that Dario knows better. He was a PhD student. He knows the significance level of this result. He knows that if he had walked into Bill’s office (his advisor) with “we found an interesting system, but we still don’t know what it does” and said he was ready to graduate, Bill would have kicked him out of the room. But somehow, when the IPO is around the corner, this becomes “AI is starting to drive biological discovery.”
Today we announced the Claude-led discovery of a molecular machine that we suspect could represent a new gene editing mechanism. Its precise function, biotechnological utility (if any), or level of significance is not yet clear, but at minimum it is work I would have been proud to do as a PhD student. The work was done mostly, though not entirely, by Claude: our life sciences team suggested a broad area of research, Claude read through the literature and a bunch of genome data and discovered something interesting, then Claude proposed experiments to verify the discovery and our team carried them out. It’s easy to dismiss this as a one-off or curiosity, but we’ve repeatedly seen a pattern where AI performance in new intellectual domains goes from weak to superhuman in a matter of a few years. In 2023 models struggled to do math at the level of an average high-school student. In 2024 they started to do well on math competitions for the best high-schoolers in the country, in 2025 they started to solve minor open problems, in early 2026 more significant open problems, and in late 2026 they are beginning to solve the top few open problems in all of mathematics. We believe AI for biology is on a similar exponential trend. The main difference between biology and mathematics, of course, is that math can be done purely theoretically, while biology requires experimentation. Some have used this to draw the conclusion that AI’s utility in biology will be limited. We think this is wrong. As we’ve demonstrated today, humans can collaborate with AI to perform the experiments, validate key results in a few weeks and, if necessary, work with the AI to iterate on what they find. Eventually it may even be possible for Claude itself to safely perform the experiments by autonomously controlling lab equipment, with appropriate safeguards in place, but we aren’t doing that today (our lab is also a BSL1/BSL2 facility that doesn't handle materials dangerous to humans). More broadly, biomedical advancement has many stages — from fundamental biology discoveries, to translational research, to drug discovery, clinical trials, and finally the actual delivery of medicines and health care to patients. We are also interested in these later stages, but even simply accelerating the first stage of fundamental biological discoveries has the potential to speed up and broaden the entire pipeline. Improving our understanding of biology and sharpening biologists’ tools can drive forward all of the later stages, for example by identifying new drug targets, finding new therapeutic modalities, allowing for more precise measurement, and speeding up the experimental loop which itself further accelerates our understanding of biology. This will not in itself speed up clinical trial times, but if it succeeds it could greatly increase the number of promising candidates that go into the pipeline — an increase in throughput even though latency remains. In Machines of Loving Grace, I wrote about AI’s potential to “cure most diseases in 5-10 years” — a goal that sounds impossible, but one I believe is just barely possible if AI is applied to every stage of the pipeline. The first step is showing that AI can first help with, and then drive, biological discoveries. Claude’s discovery is the latest in a line of related prior work that goes back decades, beginning with systems like CRISPR, and continuing with discoveries like the bridge recombinase and VIPR in the past few years. Recently, there has been heightened interest in systems based on reverse transcriptase (RT) enzymes, the enzyme underlying the system Claude identified. And most recently, a Stanford team working independently described a novel RT system with an associated non-coding array that is in some ways similar to the one Claude found, though they are distinct systems that evolved independently from each other. I believe that we’re at the very beginning of finding such systems and developing them into powerful tools for biotechnology. I’m proud of the resources Anthropic has invested in accelerating the public benefits of AI through the life sciences, and we’re aiming both to grow our life sciences team and to work with other scientists to extend this approach to a broad range of problems. If you have a proposal for a research collaboration or are interested in joining our life sciences team, please reach out.
106
299
23
3,844
683,663
Sebastian Schuster retweeted
As agents perform increasingly complex tasks for researchers, it's important to develop norms for communicating with other humans. Here's my lab's policy on AI in writing and communication, focusing on helping us think deeply and communicate precisely. cs.columbia.edu/~johnhew/lab…
6
52
6
325
26,627
Sebastian Schuster retweeted
I predict that you can also find a "hunger axis" in LLMs, especially in the metaphorical forms of "hunger for knowledge / success / validation" -- and yet none of this would imply that LLMs can actually feel hungry.
New paper: we found a pain direction in 25 open LLMs. It's distinct from fear and negative valence, and it fires for harm to the model but not to the user. Turn it up and models press a button to make it stop, even when the button deletes the user's files or their kids' photos.🧵
38
40
13
398
19,772