@humansand, formerly @elicitorg
San Francisco
Joined November 2022
- Tweets147
- Following253
- Followers1.5K
- Likes484
As AI discovers new information about the universe at an exponentially increasing rate, teaching humans about the new discoveries will become a central bottleneck. Models like Ampersand tutor can help! Massive congrats to @alexisjross @314Bansal!
Go chat with ampersand!
Introducing Boba - the social intelligence that understands your goals and your community. Text Boba today or message it online at boba.app
Great thread from Noah with more details on the eval awareness results!
while evaluating Ampersand v0.1 for sycophancy @__Charlie_G found something interesting: deployment makes models more sycophantic, but good human simulators substantially reduce evaluation awareness.🧵
8
407
Cool result from the Ampersand system card: our user model Persimmon with test-time scaling reduced self-reported evaluation awareness to deployment-like levels, 40 percentage points below prompted Opus 5.5 for the most evaluation-aware models in our multi-turn sycophancy evals.
No one changes the world alone
Introducing Ampersand - the first model built for us. Ampersand is a collaborator trained to help people achieve their individual and collective goals. In multi-month simulations, users collaborating with Ampersand accomplished more than with Fable and other frontier models
Available to everyone through our API
I'm very excited about the direction of scaling compute to build higher fidelity simulations to reduce evaluation awareness in frontier systems. More details on this work: ampersand.humansand.ai/blog/…
7
148
Our new model trained to interact with groups of people over a long time horizon in high fidelity simulations is now out!
No one changes the world alone
Introducing Ampersand - the first model built for us. Ampersand is a collaborator trained to help people achieve their individual and collective goals. In multi-month simulations, users collaborating with Ampersand accomplished more than with Fable and other frontier models
Available to everyone through our API
1/7 Very excited about the Persimmon release!
Massive congrats to the entire team, particularly @314Bansal and @alexisjross, for leading this work.
Happy 250th birthday America!
I feel so blessed to live in such a beautiful and welcoming country where you can find the most amazing friends, work with the smartest people in the world and have a shot at building incredible things that aren't possible anywhere else.
17
910
Come join us on Saturday!
Announcing the humans& hackathon! Hack with us this Saturday - come experiment and build AI apps to help people collaborate and communicate, work with creative folks, learn a bit about what we're building, and win cool prizes
Apply here: luma.com/2pbif8t9
ALT watercolor of terminal with sf visible
10
1,150
We're hiring the best product team in the world. Come join us!
We’re building foundation models that enable humans to better collaborate, communicate, and coordinate with one another. That requires rethinking many interfaces we take for granted today. We’re hiring amazing product builders to join us on this mission - if that’s you, apply
ALT offsite photo converted into painting with large brush strokes
Massive thank you to @lisawehden and @plymouthstreet for getting me a new O-1 for @humansand in record time. It’s a massive privilege to work in the US with such talented people, and I’m really grateful for your help. If you need assistance with visas, please reach out to them!
Happy to share that I joined @humansand in November! I’m excited for a future where advanced AI is endlessly curious about humanity, deeply cares about us, and we care for it in turn. Let’s build AGI for the humans!
Today we introduce humans&, a human-centric frontier AI lab. We believe AI can be reimagined, centering around people and their relationships with each other. At its best, AI should serve as a deeper connective tissue that strengthens organizations and communities
ALT a photograph taken of dolores park with the "humans&" logo overlaid, then transformed to look like a painting
Today is my last at Elicit.
I joined then Ought at 18 as an intern in the summer of 2022. Over the past three years, we’ve done early factored cognition research, spun out into a for-profit, rebuilt the Elicit codebase from the ground up, and shipped many core platform features. We've scaled the user base 1000x during my tenure, and we went from 0 to millions in revenue. We’ve hired so many amazing new teammates, and I'm sure the Elicit team will do great without me. I'm so proud of what we've built together. What a journey!
I’m excited for new adventures, but wanted to take a minute to thank @stuhlmueller, @jungofthewon, @james_elicit, and Justin for the most wonderful mentorship. I couldn’t have done without you. So excited about what’s next for Elicit!
2
13
3,445
1/ How do you verify complex AI outputs at scale without expert-labelled data?
Working with @thinkymachines' new RL API Tinker, I've been expanding on some previous work I shared around using unstructured internet data to train models to grade IMO / USAMO solutions.
2/ What's Tinker?
Tinker is a new RL API that gives you the flexibility of writing code that runs directly on a GPU cluster, but with the ease of using a Python API.
The clever part is that you can define custom torch loss functions, run them on your CPU-based laptop device, and then send the gradients back to Tinker to update the model. This approach allows you to scale seamlessly from 1B to 100B parameter models and training runs that take minutes to days.
1
2
439
1/ How can we verify IMO solutions at scale without expert-labelled data? I've explored using GPT-4.1 & unstructured internet data to RL o4 mini, resulting in fine-tuned o4 mini outperforming o4 mini and GPT-4.1 at grading IMO solutions vs human experts. Here’s the process:
2/ Data Generation
- Start with real mathematical proofs from ProofWiki (2k from NaturalProofs dataset).
- Use GPT-4.1 to strategically inject subtle mistakes using CoT.
- GPT-4.1 also rates proofs' difficulty (1-5); I trained exclusively on difficulty level 2.
- No IMO specific data is used.
1
3
233
1/ Elicit can now help you radically speed up the creation of systematic reviews with the highest standards for rigour and accuracy.
Introducing Elicit Systematic Reviews!
Elicit now supports automated search, title & abstract screening, and data extraction, all in one step-by-step flow.
We built this to accelerate power researchers without asking them to sacrifice control. You can do the whole thing manually, or run an automated review in 9 clicks.
How it works:
1/ Your review starts with a research question. Elicit suggests ways to clarify your question and additional angles to analyze.
1/ Can GPT-3.5 supervise GPT-4o debates on hard closed QA tasks? We find some early results that suggest yes!
Just finished my first day back working full-time with the amazing @elicitorg team in Oakland! Huge thanks to @plymouthstreet and especially @lisawehden for making this possible by getting my O-1 approved super quickly.
1/ I trained a small GPT-2 style model using Kolmogorov-Arnold Networks (KANs) instead of MLPs on 134 million tokens of the TinyStories dataset using JAX. I found that KANs can achieve the same performance with 25% fewer parameters! Code below.
2/ I used a variant of KANs that uses Chebyshev polynomials (inspired by github.com/SynodicMonth/Cheb…) as the basis function and keeps the values between -1 and 1 by first applying tanh. This avoids the need to update the grid during training.
1/ Can large language models detect and correct their own hallucinations when summarizing academic papers? In our new paper, we explore a new method we call factored verification to help answer this question.
Blog: blog.elicit.com/factored-ver…
