Morgan (@morganlinton) header
Morgan (@morganlinton) profile picture

cofounder @boldmetrics, research @vulcanbench, winter lover - let it snow ❄️ (views are my own)

Incline Village, NV
Joined January 2009
Photo by Morgan (@morganlinton)Photo by Morgan (@morganlinton)Photo by Morgan (@morganlinton)Photo by Morgan (@morganlinton)Photo by Morgan (@morganlinton)Photo by Morgan (@morganlinton)Photo by Morgan (@morganlinton)Photo by Morgan (@morganlinton)Photo by Morgan (@morganlinton)Photo by Morgan (@morganlinton)
Current status
7
30
700
Okay, my benchmark of Gemini 4 Neon with @VulcanBench is now complete. 92 total runs of Gemini, and yeah, testing models on 23 tasks across effort levels is a lot. Lately I've been kinda envious of benchmarks that just benchmark models at Max effort. It would be so much easier, and cheaper for me to do these benchmarks at just one effort level, but at the same time, I think there's soooo much signal once you look at lower effort levels. So, I think it just means my benchmarks will come out later, and cost more, but this seems to be the path I've chosen, and me and my team have saved so much money not using Max effort, that it's worth it imo. I decided to compare Neon to Opus 5.5 and GPT 6.2 Sol as I think these are probably the three core options people are looking at in the execution layer today. As a reminder, scores between 80-90 on Frontier v4 are what I would consider to be totally strong enough for routine daily work. VulcanBench is a completely open source, independent benchmark, which means you can look at all the results, and traces, and if you want, you can run everything I did yourself. All the good stuff below: Full benchmark report: vulcanbench.com/benchmarks/s… Link to traces: github.com/morganlinton/vulc… Congrats to Google, safe to say, they are now very much in the game. And yeah, they were behind for a while there, but phew did they catch up fast. Live long and benchmark 🖖
58
13
10
180
27,819
Trying code mode, will report back.
Code Mode is an experimental new feature in Claude Code that can improve both speed and context efficiency when Claude needs to orchestrate multiple steps. Try it out and let us know what you think! It enables Claude to perform multi-step tool calls by writing and running JavaScript. Rather than calling one tool at a time (e.g. Bash, Read, or Edit) and waiting to see the result before deciding what to do next, Code Mode allows Claude to write a simple script to run multiple commands in parallel, passing arguments and filtering outputs for subsequent tool calls to reduce context. I'm excited about this work because Code Mode enables more efficient and streamlined multi-step coding flows. For example, rather than passing large amounts of raw search data into the context window or having to make a large number of sequential calls to perform the same refactoring across dozens of files, Claude can orchestrate these tasks programmatically in a single pass. Still, you will be able to interact with Claude as normal through natural prompts. And if you ever need to see the exact JavaScript that Claude generated, you can always use Ctrl+O to expand the collapsed Code rows in the transcript viewer. One of the most exciting practical improvements from Code Mode is its ability to perform long-running operations, like running test suites or complex builds, as background processes. When Code Mode is running in either auto or bypassPermissions mode, scripts can run in the background for up to two hours. You can continue to interact with Claude and perform other work within the same session. The /tasks command provides complete transparency and control over background scripts. Each tool call that is generated within a script is still governed by your current deny and ask permission policies, sandbox constraints, and PreToolUse hooks. Hooks can even be defined specifically for the REPL tool to inspect the generated JavaScript before running, or redact sensitive tokens from the output before reading it into the context. To enable Code Mode, just start your session with CLAUDE_CODE_EXPERIMENTAL_CODE_MODE=1 claude or set the environment variable in your ~/.claude/settings.json file. Give it a try by running multi-file audits, batch edits, or any type of parallel search-and-filter operation. Code Mode is experimental and the team is actively iterating on it, so please run /feedback in your terminal to let us know what works well and where you hit any edges. Hope it's helpful!
5
15
1,948
Just noticed that sandboxes are coming soon to @useblacksmith 👀 Given how insanely efficient BlackSmith is with CI, the combo w/sandboxes is going to be pretty wild.
7
1
24
2,613
Neon 🔥
6
1
66
3,252
So excited that ski season is finally back. And so much fun to share the excitement with everyone else. When the resorts first open, all of us are like little kids again, so many great conversations on the lifts, high fives in the lines, such an electric feeling. And for me, since skiing is my single favorite thing to do on the planet. I’m back to studying, analyzing and continuing to work on my form. I’m a super analytical person, and love the technical aspects of skiing, the physics, and the small changes you can make that impact how you ski. For me, it’s the ultimate hobby, getting into nature, exercising, hanging with friends, and having fun getting incrementally better. And, yeah, getting frustrated when I make a mistake, or days where I feel like my form is actually getting worse…but all part of the process. When I lived in SF, I was able to ski maybe 10 days a year, now I ski 50+, and I sure hope in the future, I can spend even more time on the slopes. It is my happy place. And as I tell people, I get through warm weather because I know it just means each day, I’m one day closer to winter. Here’s to an amazing winter, now please, send more snow to Tahoe! ❄️ ⛷️
12
1
1
69
3,551
Going up!
12
8
232
4,528
People have been asking me for an update on my Frontier v5 eval suite. It’s taking longer than I expected, but that’s okay, I’m actually really enjoying the process. But for those who have wanted an update, here ya go 🖖
2
6
1,282
Crazy beautiful morning walk. What I love about West Coast winters is we get tons of snow, then beautiful blue skies.
23
42
3
889
18,893
Another beautiful morning in Tahoe, time to hit the slopes, sooo glad ski season is finally back ⛷️
17
13
1
455
7,832
Just bought more shares of Anthropic. Buying below $90 is going to feel like a dream in a decade imo. (not financial advice)
11
1
55
3,870
Yesterday was a historic day, I truly believe Anthropic is changing the world, and now is only going to accelerate along this path. I also spent some time going down the rabbit hole, because as a founder myself, I'm interested in Dario's journey getting the company to where it is today, and so quickly. Learned a ton, really inspiring, thought I would share five things I learned about Anthropic that I didn't know: 1. @AnjneyMidha setup a little VC tour with @DarioAmodei in Silicon Valley, intro'd him to 22 VCs, 21 said no. 2. Anthropic didn't raise money from VCs until Series C. Their first VC investor was @sparkcapital who led a $450M round at a $4.5B valuation in May of 2023 3. And their Series B round, which as I said above, didn't involve any VCs, was led by...drumroll please, Sam Bankman-Fried, the founder of FTX 4. In total, Sam invested ~$500M in Anthropic's Series B, after he went to prison, his estate sold them for $1.34B...if they had waited a bit longer, they could have sold them for ~$40B yesterday 5. Dario himself has one direct report. When asked about this structure Dario said, "It’s incredibly freeing. It lets me do all the things that I do much more easily than I would otherwise." Also one bonus fact I learned as I did a deeper dive into my last point there. Dario says he spends about 40% of his time on company culture and direction. Company culture is so important, it's clear Dario really puts a lot into this, and I mean, looking at the results, I'd say it's working. Congrats again to Dario and the entire team at Anthropic, yesterday felt surreal in many ways. So excited for the journey ahead!
6
2
1
23
2,676
Very cool, got Vulcan 🖖
Muse has its own email address. Grab your muse.ai handle before someone else does 😉 Rolling out to everyone in supported markets, your Muse can now send and receive emails on your behalf. To get started, tap your avatar and head to the mail tab, or just ask your Muse.
3
2,099
Amazing backstory here, awesome of Byron to share this!
Anthropic’s IPO is a punctuation mark in the history of both humanity and computation. It’s also the largest bet in Bessemer’s history. While we all attempt to define just how large the impact of this company, and this day is, we'll put it this way: Anthropic is the most astonishing business we’ve ever seen. This is an important day not only for Dario, Daniela and the entire team, but also for Bessemer as well. Backstory →
5
2,481
So proud to have Bessemer as an investor, happy for Anthropic and them today. Dario had an amazing vision, Bessemer saw it early, and well, now we all know, the rest is history. Here's to continuing to make bets like this!
POV: your biggest bet just made history 🎉 We loved going BTS on @AnthropicAI’s big day!
1
24
3,375
Wild, this video just crossed 500,000 views. People either love Tahoe, or Narnia, or both 😅
Okay, it’s official, Tahoe has become Narnia.
41
19
661
24,887
Morgan retweeted
Behind the scenes of the Anthropic IPO @NasdaqExchange 📸
14
16
3
400
98,391
Huge congrats to Anthropic on their IPO. As a longtime customer I have always been in awe of the attention and detail the team puts into their models. Through VulcanBench I’ve had the chance to really do a deep dive into how models like Fable think, and it really is a thing of beauty. The team really deserves this, Anthropic has built something very special here. I am honored to become an investor today, and like other companies I have strong conviction in, I will be just buying monthly on autopilot for the next decade. Can’t wait to see how this all pans out in 2036, for the next ten years, I’ll just be buying. Onward 🚀
1
13
1,766
Okay, woke up to lots of snow, and the results of my GPT-6.2 Sol benchmark on @VulcanBench 🖖 Very impressed with this model, it is clearly a strong daily driver, and saw noticeable improvements at low and medium effort compared to GPT-6.1 Sol. As I've said many times, but worth saying again. A score between 80 - 90 on VulcanBench Frontier v4 is the range you want to be for routine daily work. And, once you compare it to Fable 5.2 and Opus 5.5, it's hard not to see how cost-effective GPT-6.2 Sol is. I'm impressed, will definitely make it into my stack for daily coding tasks. Congrats to the whole team at OpenAI, really neat to see the improvement specifically at lower effort levels too. Everything is now live on the VulcanBench site: vulcanbench.com/benchmarks/s… And of course, as a free, open source, independent benchmark, I also share the traces so you can see everything the model did across every task, every run: github.com/morganlinton/vulc… Live long and benchmark 🖖
12
4
3
106
16,220