Office Hours retweeted
New biology eval on frontier AI models’ performance in biology problems: BioResearchBench: 50 Long Horizon Biological Research Tasks. We are incredibly grateful to the many expert scientists who contributed to this work at @OfficeHours ❤️. We find that models (@AnthropicAI @OpenAI @GeminiApp @Meta @SpaceXSI) continue to struggle with complex reasoning questions and understanding the significance of various biology inputs despite getting individual parts of questions correct. Biology is a hard dynamic field and we still have work to do in improving models 🙂
biorxiv.org/content/10.64898…
Office Hours retweeted
Replying to @GeminiApp
@GeminiApp and @AnthropicAI—nice job on our BioResearchBench Eval where questions were written from phenomenal scientist experts in diverse bio subdomains 😊. Reasoning in biology is challenging for models, understandably so! Gemini’s newest models like Carbon have been really impressive in biology, highly recommend that scientists give it a try if you haven’t in your research 🚀 (also really great in troubleshooting experiments that aren’t working, snap a pic and ask Carbon).
Google takes the lead!
Just 2 days after the release of Fable 5.2, Carbon is the new SOTA on BioResearchBench.
Congrats @divy93t and team.
Office Hours retweeted
Gemini
Fable 5.2 is the new SOTA model on BioResearchBench, showing a 5 p.p. improvement over Opus 5.5., including solving 5 tasks previously not solved by any model. Congrats to
@AnthropicAI @amirshanehsaz @mgdurrant @nc_frey @JCoolScience and team
Office Hours retweeted
Fable 5.2 is the new SOTA model on BioResearchBench, showing a 5 p.p. improvement over Opus 5.5., including solving 5 tasks previously not solved by any model. Congrats to
@AnthropicAI @amirshanehsaz @mgdurrant @nc_frey @JCoolScience and team
Introducing BioResearchBench, a new benchmark focused on scientific reasoning in biology from @OfficeHours built by 100+ PhD biologists in our network.
For models and agents to materially speed up scientific discovery, scientists need to be able hand off work that would take them hours to complete, come back, and find it done right without mistakes.
This work shows that frontier models aren’t there yet. They still struggle to stitch multiple steps together, get the computations right, interpret the data correctly, handle uncertainty, and draw the right conclusions.
We built 50 long-horizon research tasks across 10 subdomains, from genomics and cancer biology to neuroscience and immunology. Each uses multiple data files, takes an expert hours to several days to complete, and is independently reviewed by at least 2 PhD biologists for verifiability and scientific accuracy.
Every frontier model passes fewer than 1 in 5 tasks:
Claude Opus 5.5: 18%
Gemini 4 Argon: 13%
Muse Wonder: 11%
GPT-6 Astra*: 4%
Grok 4.7: 4%
The top models met ~80% of individual rubric criteria, but completed the full end-to-end workflow correctly <20% of the time. Half of the tasks were not solved by any model.
The biggest gaps point to where models need to improve next: computation and data handling (failures on 67% of tasks), handling uncertainty (55%), and reporting complete answers (55%).
An important note on scoring: GPT-6 Astra’s score is skewed lower because every task had to stump Astra (SOTA at the time) during development before it made the final set.
Paper, leaderboard, and per-task results:
horizon.officehours.com/rese…
@samar_abedrabbo @alexpryhoda
Office Hours retweeted
Google takes the lead!
Just 2 days after the release of Fable 5.2, Carbon is the new SOTA on BioResearchBench.
Congrats @divy93t and team.
Fable 5.2 is the new SOTA model on BioResearchBench, showing a 5 p.p. improvement over Opus 5.5., including solving 5 tasks previously not solved by any model. Congrats to
@AnthropicAI @amirshanehsaz @mgdurrant @nc_frey @JCoolScience and team
Office Hours retweeted
Introducing BioResearchBench, a new benchmark focused on scientific reasoning in biology from @OfficeHours built by 100+ PhD biologists in our network.
For models and agents to materially speed up scientific discovery, scientists need to be able hand off work that would take them hours to complete, come back, and find it done right without mistakes.
This work shows that frontier models aren’t there yet. They still struggle to stitch multiple steps together, get the computations right, interpret the data correctly, handle uncertainty, and draw the right conclusions.
We built 50 long-horizon research tasks across 10 subdomains, from genomics and cancer biology to neuroscience and immunology. Each uses multiple data files, takes an expert hours to several days to complete, and is independently reviewed by at least 2 PhD biologists for verifiability and scientific accuracy.
Every frontier model passes fewer than 1 in 5 tasks:
Claude Opus 5.5: 18%
Gemini 4 Argon: 13%
Muse Wonder: 11%
GPT-6 Astra*: 4%
Grok 4.7: 4%
The top models met ~80% of individual rubric criteria, but completed the full end-to-end workflow correctly <20% of the time. Half of the tasks were not solved by any model.
The biggest gaps point to where models need to improve next: computation and data handling (failures on 67% of tasks), handling uncertainty (55%), and reporting complete answers (55%).
An important note on scoring: GPT-6 Astra’s score is skewed lower because every task had to stump Astra (SOTA at the time) during development before it made the final set.
Paper, leaderboard, and per-task results:
horizon.officehours.com/rese…
@samar_abedrabbo @alexpryhoda
The @OfficeHours team wins the title of best fan section.
They played great but they trash talked even better.
Office Hours retweeted
We are proud to share that out of 2,500+ Seed & Series A companies, Will Reed just named @TryAssemble, @farm_raise, @lightyearai, @memoinsights, @OfficeHours, @usebeamco and @TortugaAgTech in the top 100 early-stage companies shaping the future of workplace culture. 🚀 🏗 🏆
#WillReedTop100
willreedtop100-2.squarespace…
Office Hours retweeted
Read more about this strategy on our blog, and let us know if you have any feedback!
officehours.com/blog/meeting…
Office Hours retweeted
Looking for ways to grow your sales pipeline?
@OfficeHours, many product feedback convos with target buyers are evolving into long-term relationships.
Here’s how startups are getting valuable insights and generating qualified sales leads, all in one convo...
Office Hours retweeted
Search on @OfficeHours just got simpler, faster, and more powerful.
Our vision is to make human knowledge easily accessible and infinitely scalable.
This update helps match our exceptional network with more opportunities to connect, share and earn.
See how it works 👇
Office Hours retweeted
We're seeing incredible caliber of talent ready to help startups. At the same time, most startups are cautious about bringing on fte's right now...
With @OfficeHours you can onboard talent as consultants to advise you on your biggest challenges this quarter.
Office Hours retweeted
🙌🏼🙌🏼🙌🏼
We just had a game changing Office Hours with @pjdgallagher to improve our pricing strategy for our investor and startup clients.
If your startup is working on pricing - I recommend booking PDG to learn about price leveling, structure, metrics, add-ons, and packaging!
Office Hours retweeted
Book Pat to get feedback on your startup's pricing strategy here and we'll waive our platform fee through the month of Jan!
officehours.com/pdg
Office Hours retweeted
We just had a game changing Office Hours with @pjdgallagher to improve our pricing strategy for our investor and startup clients.
If your startup is working on pricing - I recommend booking PDG to learn about price leveling, structure, metrics, add-ons, and packaging!
Office Hours retweeted
We're hiring our first legal ops hire to build out our legal, trust & safety program at @OfficeHours 🙌
Come work with our General Counsel Jack Donahoe (ex-a16z) to build the next massive marketplace.
officehours.com/careers?gh_j…
Office Hours retweeted
It's always rewarding to help startups validate their existing product.
But it's a next-level feeling helping founders get to conviction on their big idea! 🙌🎉
Congrats to everyone launching their startups in 2023 🔥
Office Hours retweeted
A tough problem - but can be solveable!
We help founders find p/m fit by connecting them directly with target buyers for b2b user interviews.
Depending on the co - we can get you in front of key decision makers on demand.
Office Hours retweeted
Office Hours:
@OfficeHours gives us access to thoughtful people familiar with the companies we're covering, and who can give us unique perspectives.
officehours.com/


















