Jonathan Hefner retweeted
Short example of discovering MCP Servers and Skills on Hugging Face and Vercel using the AI Catalog. Hosting resources at .well-known makes this easy for you - and your agents.
MCP and Skill Discovery through AI Catalog
Showing installation of a discovery plugin, live MCP Server connection (Hugging Face) and Vercel Skill installation.
In 7 months, the skills.sh registry reached 1 million agent skills and nearly 280 million installs.
See what people are teaching agents, and where skills are going ↓
vercel.com/blog/state-of-age…
In a future filled with agent-driven dependency upgrades, semantic versioning has less value.
(Ironically, agents are probably better at enforcing strict semantic versioning than humans are.)
So pretty...🤩
We built vgpu.sh to ship shaders on vercel.com. Now it's open source.
▪︎ Minimal agent-first WebGPU library
▪︎ Run in the browser or headless Node.js
▪︎ Render in CPU sandboxes and CI tests
▪︎ Create reusable .wgsl modules
Introducing Agent Plugins, an open standard for extending agents.
Supports Agent Skills and MCP, with more to come.
Built in collaboration with: @awsdevelopers, @code, @cursor_ai, @github, and @openaidevs.
vercel.com/blog/introducing-…
Completing my first month at Vercel today. Extremely proud to be part of this ship! ▲
Introducing Agent Plugins, an open standard for extending agents.
Supports Agent Skills and MCP, with more to come.
Built in collaboration with: @awsdevelopers, @code, @cursor_ai, @github, and @openaidevs.
vercel.com/blog/introducing-…
Jonathan Hefner retweeted
thinking of doing skillsbench 2.0. below are some of my personal ideas and would appreciate any feedback 🙌
topics we are thinking of covering:
- meta skills/tasks: agents self learning from environments / episodes (skillopt, skill self play)
- memory: skill as a media of memory for long running agents
- enterprise: skills are used more as a bespoke software at companies + becoming a way of distributing developer experiences: salesforce cli, gws cli (dead tho), hubspost cli etc. previously we have built out mock environments (clawsbench) thus we are thinking of creating sandboxes to test these skills
- more capability axes: covering using browsers (since we have mock envs), multimodal, deep search (general reasoning released back search)
- elo based: so the benchmark itself is longer lived
Jonathan Hefner retweeted
if you're not using skills you are wasting your time
repeatability has always been the goal of software efficiency, and skills are a version of that
even before skills were a thing, anyone who was getting shit done had some home grown version of it (embedded docs, macros, etc)
I don’t use loops, skills, fancy orchestration, plugins or anything like that. I raw dog the agent with its default harness, I read the code, and I’m shipping more than ever to production for real projects used by real people. Sometimes if it’s a low risk part of the application I might skip reading the code but only sometimes. I think most are either lying about the amount of value they are creating or just having fun while understanding they aren’t creating value but failing to distinguish that to their audience.
I agree more and more. Everything seems to point to doing work in passes -- code generation pass, review pass, etc. "Dreaming" is yet another pass.
I think we'd greatly benefit from more RL on memory extraction and/or discovery.
Jonathan Hefner retweeted
A big pain point in using AI benchmarks is encountering errors after its first release. Today, we're releasing SkillsBench 1.1, the first benchmark for how well AI agents use skills, now audited end to end and verified error-free. Prof. @dawnsongtweets joins 1.1 as advising author.
We worked through every task with several frontier labs to eliminate the errors in the previous version. We also added new tasks, moved the ones with external dependencies into a separate set so the core suite runs clean, and expanded coverage to more models.
Capability is climbing fast. The best with-skills resolution rate rose from ~36% (Claude Sonnet 4.5, Sep 2025) to 67% (GPT-5.5, May 2026), about +1.9 points per month. The frontier is hill-climbing SkillsBench fast.
The right skills still matter. Across the fleet, curated skills lift resolution rate by +16.6 points on average (33.9% → 50.5%), and by as much as +25.7 points for a single model. The top configuration is GPT-5.5 on OpenHands at 67.3%.
By popular demand (thx Nate @cursor_ai), we're now tracking skills invocation: how often an agent actually uses the skills it's given. Recent flagship configurations invoke them 90–99% of the time (Codex 99%, OpenHands + GPT-5.5 92%, Gemini CLI 90%), versus roughly 50% for older setups.
Also new in 1.1: @OpenHands joins as a fourth harness, alongside Claude Code, Codex, and Gemini CLI; a rebuilt leaderboard with refined categories, subdomain skill rankings, and Skill Lift; and native task . md on BenchFlow, with multi-scene environments and rollout branching. We also partnered with @k_dense_ai to add scientific skills to some science tasks.
One implication for deployment: skills can substitute for scale. GLM 5.1 with skills (58.4%) outperforms Opus 4.8 without (45.7%). A smaller model with the right procedural knowledge can beat a larger one running without it.
Huge thanks to @nick_kango @ivanleomk @kaggle @GoogleDeepMind for hosting a launch event with us. Thanks for everyone who's come on May 27!
Also thanks to our partners @gneubig @OpenHandsDev @ivanburazin @daytonaio @jackminong @johannes_hage @PrimeIntellect @TimothyKassis @k_dense_ai for providing support in credits, compute, and skills.
SkillsBench live leaderboard will also come to @ValsAI. Many people have told us they use SkillsBench as an index to measure models' agentic capability over diverse and high GDP value domains. Great work on Valkyrie as well! @ Jarett @nikilravi @langstonnashold @RayanKrishnan
SkillsBench is fully open-source. Explore the leaderboard and tasks, read the docs, or contribute your own skill set or harness and join the leaderboard. 🧵
How much of taste in code is visual? Like having a visceral reaction when you see nested ternary expressions or a pyramid of doom.
How much taste in code could models gain by having better vision and seeing rendered code during review?
In addition to leaking neuralese, models can have bothersome verbal or behavioral tics.
I can imagine a future where users only interact with "chief of staff" agents that embody and enforce user preferences, delegating all work to other agents and acting as a filter.
Replying to @tamaybes
fascinating because 5.5 does this too. invent weird technical jargon. perhaps you’re right and it’s a neuralese leakage
Jonathan Hefner retweeted
The skills.sh API is now generally available.
Power your agents, applications and platforms with access to over 600,000 skills ↓
vercel.com/changelog/the-ski…
Some people argue that skills will become obsolete as models improve. That may be true to some extent, but models will never know exactly how you (or your company) want things done. Personalization is a fantastic (and often overlooked) use case for skills.
Jonathan Hefner retweeted
I’ve seen people on X dunking on folks like @garrytan @doodlestein and others for sharing SKILL dot md files they've built. They are dismissing these files as "just a markdown file.”
I think this misses the point entirely and I'll try to address that here. Quick thread:
A bad skill file is just text, sure.
A good skill file is compressed expertise, packaged in a format an agent can actually use.
The value is not just in the “markdown file.” The value is the interaction between:
a huge neural network with latent capabilities
a precise, reusable, agent-readable procedure that steers those capabilities toward a specific outcome
That combination is the product.
Saying “it’s just markdown” is like saying Hamlet is “just ink on paper,” or Einstein’s relativity paper was “just a text.”
Technically true. Intellectually useless.
The medium is simple. The content is what matters. And more importantly, the effect of that content on the reader is what matters.
With humans, a book, a coach, a lecture, or painting can change how someone thinks and acts.
With LLMs, text is also the control surface. These models were trained on text, reason through text, call tools through text, and follow procedures through text.
So yes, the skill is “just text.”
But it is text designed to be read by an enormous neural net.
That matters.
A good skill is agent-ergonomic. It does not merely say “do this better.” It encodes workflow, constraints, examples, edge cases, tool usage, failure modes, and success criteria in a way the agent can reliably execute.
That is very different from a casual prompt.
A prompt is often a one-off request.
A skill can be reused, versioned, tested, improved, shared, and loaded at the exact moment an agent needs it.
That turns “vibes-based prompting” into something closer to operational knowledge.
Another way to think about it:
We have built these massive models, but much of their power is latent. Different people can extract very different levels of performance from the same model.
A good skill is a way to actualize a specific slice of that latent capability.
A refactoring skill.
A research skill.
A legal review skill.
A math explanation skill.
A codebase-navigation skill.
Each one can make the same model behave very differently.
I think of Cus D’Amato and Mike Tyson.
Tyson had enormous latent potential. But Cus gave him a system, a style, a discipline, a way to channel that potential.
That’s what good skills are for agents.
They are not magic. They are not all equally valuable. Many will be mediocre or useless.
But dismissing them right off the batt because they are “just markdown” shows a misunderstanding of what LLMs are.
Text is how we trained these systems. (for the most part)
Text is how we steer them.
Text is how we unlock parts of what they can do.
The question is not whether a skill file is “just text.”
The question is whether the text reliably makes the model perform better at a valuable task.
If yes, then it is not “just markdown.”
It is leverage.




















