Concept (@conceptdotdev) header
Concept (@conceptdotdev) profile picture
Joined January 2026
🧑‍🤝‍🧑 Company Agent Bench measures how well AI agents understand a company's people, ownership, and communication patterns from workplace chat history. I've been really, really pleased to see how easily Gemini 4 Neon can navigate through the massive internal system that is @Google - glad to see that there's a benchmark to measure this capability!
Gemini Neon takes a strong lead on Company Agent Bench at 87.4%, ahead of Claude Fable 5.2 (82.2%) and GPT-6.2 Astra (79.7%). Even at low effort, it scores 85.3%, beating Fable 5.2, Opus 5.5, GPT-6.2 Astra, and GPT-6.2 Sol at every reasoning effort we tested. New Google company agent product loading? 👀 Full results below!
12
8
64
2,935
Concept retweeted
Gemini Neon takes a strong lead on Company Agent Bench at 87.4%, ahead of Claude Fable 5.2 (82.2%) and GPT-6.2 Astra (79.7%). Even at low effort, it scores 85.3%, beating Fable 5.2, Opus 5.5, GPT-6.2 Astra, and GPT-6.2 Sol at every reasoning effort we tested. New Google company agent product loading? 👀 Full results below!
Today we’re releasing Company Agent Bench, a benchmark for company assistant products. We compare Dots, Grok Bot, Claude Tag, and Devin on how well an agent understands its company. We also found that API models with a simple harness perform better than these products (1/6)🧵
20
21
2
192
11,510
We launched companyagentbench.com to measure how well agents understand specific companies. Read the results here:
Today we’re releasing Company Agent Bench, a benchmark for company assistant products. We compare Dots, Grok Bot, Claude Tag, and Devin on how well an agent understands its company. We also found that API models with a simple harness perform better than these products (1/6)🧵
8
388
Concept retweeted
Today, we’re launching companyagentbench.com @OpenAI Dots placed #1 across agent products while @AnthropicAI Fable 5.2 placed #1 overall via API. Our goal is to measure how well agents understand a specific company. Check out the results and findings below! Shoutout @joshzqin for spearheading this project
Today we’re releasing Company Agent Bench, a benchmark for company assistant products. We compare Dots, Grok Bot, Claude Tag, and Devin on how well an agent understands its company. We also found that API models with a simple harness perform better than these products (1/6)🧵
23
6
1
150
12,674
Concept retweeted
Today we’re releasing Company Agent Bench, a benchmark for company assistant products. We compare Dots, Grok Bot, Claude Tag, and Devin on how well an agent understands its company. We also found that API models with a simple harness perform better than these products (1/6)🧵
24
24
7
220
34,527
Concept retweeted
I think there's an interesting tension in the market rn. Agents only really started to work about a year ago with Opus 4.5. Many vertical AI companies since have gone exponential in growth (just chart out the valuation and revenue numbers YTD). At the same time, when talking to enterprises, they aren't feeling a truly MAGICAL wow that agents have delivered significant business impact. So what's the actual gap between agents that can solve millennium prizes and delivering real business impact? My cofounder Phil who was a researcher at OAI/GDM has been thinking deeply about this problem and talks through where agents fail today and the next frontier research problems to close this gap.
6
2
2
64
6,653
Concept retweeted
The Frontier or Nowhere We’re hiring engineers. Come work with me & @philhchen
21
2
2
83
11,241
Concept retweeted
Gemini 4 Argon reaches Astra-level performance on multimodal data annotation. Significant improvement compared to Gemini 3.8. Argon also better at identifying correct actions in egocentric videos.
Today, we're sharing results from our partnership with @MeckaAI to see if frontier models can label egocentric videos for robot learning. We had five frontier models caption 200 clips with human grading. GPT-6 Astra which performed the best got 43.5% of clips fully correct.
4
1
1
28
1,974
Incredible model, great to partner
1
143
Great to collab on this!
4
319
Replying to @MeckaAI @tkkong
Let's go! Great to partner with the team
18
Concept retweeted
Frontier models still struggle to accurately caption egocentric videos. Dense video captioning remains a task where a specialized model, our collaboration with @MeckaAI, outperforms general-purpose foundation models. Many interesting problems still to solve in robotics.
Today, we're sharing results from our partnership with @MeckaAI to see if frontier models can label egocentric videos for robot learning. We had five frontier models caption 200 clips with human grading. GPT-6 Astra which performed the best got 43.5% of clips fully correct.
6
3
1
49
3,338
Sharing our first research post with @MeckaAI
Today, we're sharing results from our partnership with @MeckaAI to see if frontier models can label egocentric videos for robot learning. We had five frontier models caption 200 clips with human grading. GPT-6 Astra which performed the best got 43.5% of clips fully correct.
3
1
21
1,487
Concept retweeted
We’re hiring for designers & engineers. Come work with me & @philhchen
86
7
1
439
32,518
Concept retweeted
We're opening up early access. Phil and I have been working with teams to ship internal AI apps and agents. We’re applying our learnings from Ramp Labs, OpenAI, and DeepMind to help operations teams scale faster. DM if you're interested in trying it out!
30
8
5
175
41,841