Staff Research Scientist, Google DeepMind. Interpretability, analysis, and visualizations for LLMs. Opinions my own.
Seattle, WA
Joined October 2012
- Tweets100
- Following532
- Followers2K
- Likes3K
Ian Tenney retweeted
Announcing the keynote speakers for the Actionable Interpretability Workshop at COLM 2026! @ActInterp
@boknilev @banburismus_ @ChrisGPotts @dhanya_sridhar
What would you ask them?
Ian Tenney retweeted
Accepted to COLM 2026? Consider submitting your work to the fast track at the Actionable Interpretability Workshop! @ActInterp
We welcome work that advances the use of interpretability to real-world usage.
Deadline: August 9
Links below 👇
Ian Tenney retweeted
Excited that our paper on Actionable Interpretability got accepted to ICML!
And just in time -- we also heard that our Actionable Interpretability workshop will be happening again, in COLM!
See you in Korea 🇰🇷 and SF🌉
[Arxiv paper link in the comment]
At ICML? Interested in how we can do more with interpretability to have practical impact on the rest of AI? Come to our workshop this Saturday!
Hope everyone’s getting the most out of #icml25. We’re excited and ready for the Actionable Interpretability (@ActInterp) workshop this Saturday!
Check out the schedule and join us to discuss how we can move interpretability toward more practical impact.
Ian Tenney retweeted
🚨Meet our panelists at the Actionable Interpretability Workshop @ActInterp at @icmlconf!
Join us July 19 at 4pm for a panel on making interpretability research actionable, its challenges, and how the community can drive greater impact.
@nsaphra @saprmarks @kylelostat @FazlBarez
Ian Tenney retweeted
Going to #icml2025? Don't miss the Actionable Interpretability Workshop (@ActInterp)! We've got an amazing lineup of speakers, panelists, and papers, all focused on leveraging insights from interpretability research to tackle practical, real-world problems ✨
Submission deadline extended to May 19!
Working on or have thoughts on real-world applications of interpretability, and how we can use it in practice? Consider submitting to our workshop at ICML 2025. More at actionable-interpretability.… and below👇
🎉 Our Actionable Interpretability workshop has been accepted to #ICML2025! 🎉
>> Follow @ActInterp
@tal_haklay @anja_reu @mariusmosbach @sarahwiegreffe @iftenney @megamor2
Paper submission deadline: May 9th!
Ian Tenney retweeted
Presenting our work on training data attribution for pretraining this morning: iclr.cc/virtual/2025/poster/… -- come stop by in Hall 2/3 #526 if you're here at ICLR!
We scaled training data attribution (TDA) methods ~1000x to find influential pretraining examples for thousands of queries in an 8B-parameter LLM over the entire 160B-token C4 corpus!
medium.com/people-ai-researc…
Check out our new work on scaling training data attribution (TDA) toward LLM pretraining - and some interesting things we found along the way.
arxiv.org/abs/2410.17413 and more below from most excellent student researcher @tylerachang ⬇️
We scaled training data attribution (TDA) methods ~1000x to find influential pretraining examples for thousands of queries in an 8B-parameter LLM over the entire 160B-token C4 corpus!
medium.com/people-ai-researc…
Try out LLM Comparator to help make sense of LLM evaluations!
Alongside the Gemma 2 launch, we've released a Python library to help run rater models, bulletizing, and clustering - so all you need are prompts and model responses. Scripts and notebooks at goo.gle/llm-comparator
With the Gemma 2 launch, we’ve updated our LLM Comparator tool with a Python library! This lets you more easily run the tool with your own models. goo.gle/llm-comparator
New open-source tool: LLM Comparator can help make sense of side-by-side model evals. In-browser demo at pair-code.github.io/llm-comp…, and read more in the blog post below.
Very excited to open-source LLM Comparator!
This new #visualization tool lets you analyze LLM responses side-by-side. It’s been used for evaluating LLMs @Google, and we're proud to release it as part of Google's Responsible GenAI Toolkit.
medium.com/people-ai-researc…
Very, very cool work from @ghandeharioun and colleagues - Patchscopes is a super flexible way to decode knowledge from internal states of an LLM, bridging between interpretability, causality, and control.
Being able to interpret an #ML model’s hidden representations is key to understanding its behavior. Today we introduce Patchscopes, an approach that trains #LLMs to provide natural language explanations of their own hidden representations. Learn more → goo.gle/4aS5epd
ALT Illustration of our framework, showing a Patchscope for decoding what is encoded in the representation of “It” in the source prompt (left), by using a predefined target prompt (right).
Ian Tenney retweeted
We're also releasing a Responsible Generative AI Toolkit that provides resources to apply best practices for responsible use of open models such as the Gemma models, including:
Guidance on setting safety policies, safety tuning, safety classifiers and model evaluation.
The Learning Interpretability Tool (LIT) for investigating Gemma's behavior and addressing potential issues.
A methodology for building robust safety classifiers with minimal examples.
ai.google.dev/responsible?ut…
Super excited for the Gemma model release, and with it a new debugging tool we built on 🔥LIT - use gradient-based salience to debug and refine complex LLM prompts! ai.google.dev/responsible/mo…
Explore Gemma model’s behavior with The Learning Interpretability Tool (LIT), an open-source platform for debugging AI/ML models.
➡️ Improve prompts using saliency methods
➡️ Test hypotheses to improve model behavior
➡️ Democratize access to ML debugging
goo.gle/3wsfh5F
🧵(1/6): Excited to announce the v1.0 release of the @GoogleAI Learning Interpretability Tool (🔥LIT), an interactive platform to debug, validate, and understand ML model behavior. This release brings exciting new features and a simplified Python API. pair-code.github.io/lit
🧵(5/6): Many thanks to all of the contributors who supported this release: @RyanMullins , @nadamused_ , @minsukkahng , @mahimapushkarna , @cjqian , James Wexler, Bin Du, Cibi Arjun, and Oscar Wahltinez - as well as all those who have helped build LIT since the project began.
🧵(6/6): Check out the full release notes at github.com/PAIR-code/lit/rel…
If you find LIT useful for your project, please cite our EMNLP paper (aclanthology.org/2020.emnlp-…), and drop us a line on GitHub or Twitter!
