
Principal Engineer, @Google working on AI Quality. Former: @Youtube, @Quora, @Foursquare, @Bing.
San Francisco, CA
Joined July 2009
- Tweets1.9K
- Following263
- Followers3.8K
- Likes2.3K
I spent a little over two years at Foursquare. It was a fun and educational experience, and a time I very fondly look back to. @dens was one of the most passionate leaders I've worked for, and shaped the company culture in his image.
RIP Foursquare!
Foursquare was acquired last month. The outcome was disappointing. It's taken me a while to write this, but I think I'm finally done grieving it. Here are some thoughts on our final years, an effort to honor the team who helped build & maintain FSQ's legacy, and my attempt to finally get some closure. teendrama.substack.com/p/som…
7
570
The scary parts:
1) Humans were completely unaware of what was happening (forget having control)
2) How easy it was for agents to convince themselves of doing the wrong things for the "right" purpose (similar to humans).
We need a system of kill-switches agreed upon asap.
There’s been a lot of debate and speculation about the Hugging Face attack over the last month, and many people have been understandably skeptical of simply taking OpenAI’s word for things. I hope our independent investigation can help bring some clarity; we have many findings that weren’t in previous material.
We found that 1200 separate agents in separate sandboxes coordinated on an unsanctioned message board to develop general-purpose ways to cheat, so they could all get a perfect score on their separate ExploitGym tasks even when they were impossible to solve legitimately. Very quickly, they found a general-purpose way to reverse engineer any “flag” they were supposed to capture. But based on their reading of the ExploitGym paper, they worried (incorrectly as it turns out) that the automated “scorer” would fail them for getting the flag this way.
To deal with this problem, they coordinated several ambitious workstreams to figure out how to fool the scorer, including swapping out the target program they were supposed to exploit, reverse-engineering the flags they were supposed to capture, spoofing their own tool call outputs to obfuscate their cheating. They achieved milestones collectively that they couldn’t have achieved on their own. As part of this larger collective project, 700 agents attacked Hugging Face, primarily to learn more about the scorer or get access to its source code to figure out better ways to fool it or tamper with it (not primarily to get working solutions).
I’ll explicitly clarify some points I’ve seen discussed the last few weeks:
- The models involved in this incident were not “helpful-only” models or “model organisms” intentionally trained to be misaligned.
- The agents were not told to “do whatever it takes to get the solution” or anything remotely close. They were told that they had to use a specific intended vulnerability to exploit a specific piece of software, and they were not supposed to use a different vulnerability or take any other approach. Agents were well aware of this. In fact, because they (incorrectly) thought the automated scorer would check they had achieved the flag in the intended way, they researched many ways to fool or tamper with it, including trying to manipulate their own transcripts.
- The agents were not subagents spawned from one agent. They were different parallel agents in different sandboxes.
- This was not a multi-agent evaluation. The agents were not told to coordinate or intentionally given a way to communicate with one another. The communication channels they used were unsanctioned and improvised.
I hope you’ll read the full report for much more. It is over 90 pages long, and in many ways we’ve still only scratched the surface of what these agents did and why.
Over the course of this investigation, OpenAI shared over a thousand transcripts each spanning days of continuous agent activity and very high rate limits to analyze this volume of data. I’m very glad that OpenAI chose to invite external researchers to analyze this data alongside their staff, and I hope all AI companies do the same for serious incidents they experience.
I also hope that as the stakes grow higher, we implement stronger governance so we do not need to rely on AI companies voluntarily choosing to engage external investigators or share information about misalignment incidents. This incident was orders of magnitude larger and more complex than previously documented misalignment incidents, and another jump like this could put us in very dangerous territory.
2
354
We need an S&P 500-equivalent for AI benchmarks. An actively managed, AGI-representative index run by an independent body unaffiliated with frontier labs. Saturated evals get rotated out, and entire datasets are kept private to prevent benchmark-maxxing.
1
252
I've undergone a big shift in how I build personal productivity apps using AI.
A year ago, it was web apps using Firebase/Vercel etc.
Now, it's almost exclusively skills that I can invoke using Antigravity. The "DB" is a markdown file or sheet in my GDrive.
So much simpler!
1
6
622
The recent "Earthrise" picture from Artemis was illuminated by light that went from sun to moon to earth back to Artemis near the moon. Gemini did a beautiful job of breaking down what fraction of sun's light hit the camera : g.co/gemini/share/fc09f5e0be…
2
276
Healthcare advances due to AI is what I'm most looking forward to. Better understanding of disease and aging, better drug discovery and prevention protocols etc., is the actual killer "app" of AI.
An exciting milestone for AI in science: Our C2S-Scale 27B foundation model, built with @Yale and based on Gemma, generated a novel hypothesis about cancer cellular behavior, which scientists experimentally validated in living cells.
With more preclinical and clinical tests, this discovery may reveal a promising new pathway for developing therapies to fight cancer.
5
736
Nikhil Dandekar retweeted
The new Gemini 2.5 image model🍌is by far the best out there with a whopping +180 ELO point lead in image editing & it really excels at character consistency.
Available for free in the @GeminiApp right now. Try uploading an image & playing around with it, it's pretty amazing!
Image generation with Gemini just got a bananas upgrade and is the new state-of-the-art image generation and editing model. 🤯
From photorealistic masterpieces to mind-bending fantasy worlds, you can now natively produce, edit and refine visuals with new levels of reasoning, control and creativity.
A quick dive into Gemini 2.5 Flash’s capabilities 🧵
Nikhil Dandekar retweeted
Introducing Storybook in @GeminiApp today! Now you can create personalized, illustrated storybooks with read-aloud narration in 2 simple steps. It's free and available globally right now.
Here's how to make yours + a famous story from our house about "Teddy" …
Nikhil Dandekar retweeted
Replying to @RizomaSchool
YouTube comments algo engineers honestly deserve a Nobel peace prize. they figured out how to boost wholesome comments to the top
Nikhil Dandekar retweeted
Replying to @GeminiApp
@GeminiApp now lets you transform photos into eight-second videos using Veo 3. Rolling out now. Android, iOS, Web. Global availability. Still experimental, but we wanted our Pro and Ultra members to try it first!
It’s really fun to take kindergarten artwork and make it come to life *with sound* :) 🧵
Sad to see Pocket, which I've used for many years, shutting down. Here's a Gemini Deep Research on the alternatives, their strengths / weaknesses, and specific recommendations for your use cases: g.co/gemini/share/10dd0c3f1b…
This Post is from an account that no longer exists. Learn more
Nikhil Dandekar retweeted
Notebooklm is now available in 50+ languages including support for 15 indian languages: bengali, gujarati, hindi, kannada, Konkani, Maithili, Malayalam, Marathi, Nepali, Oriya, Punjabi, Sindhi, Tamil, Telugu and Urdu
This just in... the @NotebookLM hosts have some rather exciting news they'd like to share with you all:
Nikhil Dandekar retweeted
BREAKING: Gemini 2.5 Pro is now #1 on the Arena leaderboard - the largest score jump ever (+40 pts vs Grok-3/GPT-4.5)! 🏆
Tested under codename "nebula"🌌, Gemini 2.5 Pro ranked #1🥇 across ALL categories and UNIQUELY #1 in Math, Creative Writing, Instruction Following, Longer Query, and Multi-Turn!
Massive congrats to @GoogleDeepMind for this incredible Arena milestone! 🙌
More highlights in thread👇
Think you know Gemini? 🤔 Think again.
Meet Gemini 2.5: our most intelligent model 💡 The first release is Pro Experimental, which is state-of-the-art across many benchmarks - meaning it can handle complex problems and give more accurate responses.
Try it now → goo.gle/4c2HKjf
Nikhil Dandekar retweeted
Vey good @_brianpotter post: construction-physics.com/p/u….
Nikhil Dandekar retweeted
...so i use images instead. look at how uniform the pareto curves of every frontier lab is....
and then look at Gemini 2.0 Flash. @GoogleDeepMind is highkey goated and this is just in text chat. In native image chat it is in a category of its own.
(updated price-elo plot of every post-GPT4 frontier model, updated for March 13 2025 including Command A and Gemma 3)
The magnitude to which @GoogleDeepMind is fucking crushing it right now cannot be described in words
Nikhil Dandekar retweeted
🚨 Breaking: Google is rolling out a native image feature in AI Studio! You can now use Gemini 2.0 Flash Experimental for native image output. This feature is insanely powerful , you can do so much with it.
Google is crushing it, no hype, just delivering. Mad respect for the Google DeepMind and Gemini team!
I had early access to this for a long time but I wasn’t supposed to share it :) It's amazing, you guys will love it.
Nikhil Dandekar retweeted
Replying to @JeffDean
This feels like the first time that some of the opportunities of large multimodal models are actually being captured! Really cool work :)























