Office Hours (@OfficeHours) header
Office Hours (@OfficeHours) profile picture

We help hundreds of the world's most influential VC firms, consulting firms, and startups access human knowledge on-demand.

San Francisco, California
Joined November 2020
Photo by Office Hours (@OfficeHours)Photo by Office Hours (@OfficeHours)Photo by Office Hours (@OfficeHours)Photo by Office Hours (@OfficeHours)Photo by Office Hours (@OfficeHours)Photo by Office Hours (@OfficeHours)Photo by Office Hours (@OfficeHours)Photo by Office Hours (@OfficeHours)Photo by Office Hours (@OfficeHours)Photo by Office Hours (@OfficeHours)
New biology eval on frontier AI models’ performance in biology problems: BioResearchBench: 50 Long Horizon Biological Research Tasks. We are incredibly grateful to the many expert scientists who contributed to this work at @OfficeHours ❤️. We find that models (@AnthropicAI @OpenAI @GeminiApp @Meta @SpaceXSI) continue to struggle with complex reasoning questions and understanding the significance of various biology inputs despite getting individual parts of questions correct. Biology is a hard dynamic field and we still have work to do in improving models 🙂 biorxiv.org/content/10.64898…
8
27
2
198
11,532
Replying to @GeminiApp
@GeminiApp and @AnthropicAI—nice job on our BioResearchBench Eval where questions were written from phenomenal scientist experts in diverse bio subdomains 😊. Reasoning in biology is challenging for models, understandably so! Gemini’s newest models like Carbon have been really impressive in biology, highly recommend that scientists give it a try if you haven’t in your research 🚀 (also really great in troubleshooting experiments that aren’t working, snap a pic and ask Carbon).
Google takes the lead! Just 2 days after the release of Fable 5.2, Carbon is the new SOTA on BioResearchBench. Congrats @divy93t and team.
6
9
301
Office Hours retweeted
Gemini
Fable 5.2 is the new SOTA model on BioResearchBench, showing a 5 p.p. improvement over Opus 5.5., including solving 5 tasks previously not solved by any model. Congrats to @AnthropicAI @amirshanehsaz @mgdurrant @nc_frey @JCoolScience and team
1
4
164
Office Hours retweeted
Fable 5.2 is the new SOTA model on BioResearchBench, showing a 5 p.p. improvement over Opus 5.5., including solving 5 tasks previously not solved by any model. Congrats to @AnthropicAI @amirshanehsaz @mgdurrant @nc_frey @JCoolScience and team
Introducing BioResearchBench, a new benchmark focused on scientific reasoning in biology from @OfficeHours built by 100+ PhD biologists in our network. For models and agents to materially speed up scientific discovery, scientists need to be able hand off work that would take them hours to complete, come back, and find it done right without mistakes. This work shows that frontier models aren’t there yet. They still struggle to stitch multiple steps together, get the computations right, interpret the data correctly, handle uncertainty, and draw the right conclusions. We built 50 long-horizon research tasks across 10 subdomains, from genomics and cancer biology to neuroscience and immunology. Each uses multiple data files, takes an expert hours to several days to complete, and is independently reviewed by at least 2 PhD biologists for verifiability and scientific accuracy. Every frontier model passes fewer than 1 in 5 tasks: Claude Opus 5.5: 18% Gemini 4 Argon: 13% Muse Wonder: 11% GPT-6 Astra*: 4% Grok 4.7: 4% The top models met ~80% of individual rubric criteria, but completed the full end-to-end workflow correctly <20% of the time. Half of the tasks were not solved by any model. The biggest gaps point to where models need to improve next: computation and data handling (failures on 67% of tasks), handling uncertainty (55%), and reporting complete answers (55%). An important note on scoring: GPT-6 Astra’s score is skewed lower because every task had to stump Astra (SOTA at the time) during development before it made the final set. Paper, leaderboard, and per-task results: horizon.officehours.com/rese… @samar_abedrabbo @alexpryhoda
4
2
6
9,400
Replying to @k3goodman @divy93t
Well done @divy93t 😆
2
4
464
Office Hours retweeted
Google takes the lead! Just 2 days after the release of Fable 5.2, Carbon is the new SOTA on BioResearchBench. Congrats @divy93t and team.
Fable 5.2 is the new SOTA model on BioResearchBench, showing a 5 p.p. improvement over Opus 5.5., including solving 5 tasks previously not solved by any model. Congrats to @AnthropicAI @amirshanehsaz @mgdurrant @nc_frey @JCoolScience and team
3
20
4
302
14,185
Office Hours retweeted
Introducing BioResearchBench, a new benchmark focused on scientific reasoning in biology from @OfficeHours built by 100+ PhD biologists in our network. For models and agents to materially speed up scientific discovery, scientists need to be able hand off work that would take them hours to complete, come back, and find it done right without mistakes. This work shows that frontier models aren’t there yet. They still struggle to stitch multiple steps together, get the computations right, interpret the data correctly, handle uncertainty, and draw the right conclusions. We built 50 long-horizon research tasks across 10 subdomains, from genomics and cancer biology to neuroscience and immunology. Each uses multiple data files, takes an expert hours to several days to complete, and is independently reviewed by at least 2 PhD biologists for verifiability and scientific accuracy. Every frontier model passes fewer than 1 in 5 tasks: Claude Opus 5.5: 18% Gemini 4 Argon: 13% Muse Wonder: 11% GPT-6 Astra*: 4% Grok 4.7: 4% The top models met ~80% of individual rubric criteria, but completed the full end-to-end workflow correctly <20% of the time. Half of the tasks were not solved by any model. The biggest gaps point to where models need to improve next: computation and data handling (failures on 67% of tasks), handling uncertainty (55%), and reporting complete answers (55%). An important note on scoring: GPT-6 Astra’s score is skewed lower because every task had to stump Astra (SOTA at the time) during development before it made the final set. Paper, leaderboard, and per-task results: horizon.officehours.com/rese… @samar_abedrabbo @alexpryhoda
3
6
3
18
3,937
Office Hours retweeted
The @OfficeHours team wins the title of best fan section. They played great but they trash talked even better.
1
5
381
We are proud to share that out of 2,500+ Seed & Series A companies, Will Reed just named @TryAssemble, @farm_raise, @lightyearai, @memoinsights, @OfficeHours, @usebeamco and @TortugaAgTech in the top 100 early-stage companies shaping the future of workplace culture. 🚀 🏗 🏆 #WillReedTop100 willreedtop100-2.squarespace…
3
5
12
3,227
Looking for ways to grow your sales pipeline? @OfficeHours, many product feedback convos with target buyers are evolving into long-term relationships. Here’s how startups are getting valuable insights and generating qualified sales leads, all in one convo...
2
2
5
1,145
Search on @OfficeHours just got simpler, faster, and more powerful. Our vision is to make human knowledge easily accessible and infinitely scalable. This update helps match our exceptional network with more opportunities to connect, share and earn. See how it works 👇
2
3
1
9
1,044
Office Hours retweeted
We're seeing incredible caliber of talent ready to help startups. At the same time, most startups are cautious about bringing on fte's right now... With @OfficeHours you can onboard talent as consultants to advise you on your biggest challenges this quarter.
1
1
4
865
🙌🏼🙌🏼🙌🏼
We just had a game changing Office Hours with @pjdgallagher to improve our pricing strategy for our investor and startup clients. If your startup is working on pricing - I recommend booking PDG to learn about price leveling, structure, metrics, add-ons, and packaging!
2
7
987
Office Hours retweeted
Book Pat to get feedback on your startup's pricing strategy here and we'll waive our platform fee through the month of Jan! officehours.com/pdg
2
4
304
Office Hours retweeted
We just had a game changing Office Hours with @pjdgallagher to improve our pricing strategy for our investor and startup clients. If your startup is working on pricing - I recommend booking PDG to learn about price leveling, structure, metrics, add-ons, and packaging!
1
3
2
10
2,211
Office Hours retweeted
We're hiring our first legal ops hire to build out our legal, trust & safety program at @OfficeHours 🙌 Come work with our General Counsel Jack Donahoe (ex-a16z) to build the next massive marketplace. officehours.com/careers?gh_j…
2
5
383
Office Hours retweeted
It's always rewarding to help startups validate their existing product. But it's a next-level feeling helping founders get to conviction on their big idea! 🙌🎉 Congrats to everyone launching their startups in 2023 🔥
4
2
24
4,524
Office Hours retweeted
A tough problem - but can be solveable! We help founders find p/m fit by connecting them directly with target buyers for b2b user interviews. Depending on the co - we can get you in front of key decision makers on demand.
Hearing from CPO candidates in the market: many Series B/C companies haven’t yet found PMF despite having raised tens of millions of $. The CEO believes the CPO will help them find PMF. Finding PMF for a company is a a cofounder role (with appropriate equity), not a CPO role!
5
1
15
2,930
Office Hours: @OfficeHours gives us access to thoughtful people familiar with the companies we're covering, and who can give us unique perspectives. officehours.com/
1
2
9