evals @composio 🧑🏭 | building https://nitter.gq/t.co/pE3gIZHBPw | Do All Things
SF
Joined May 2012
- Tweets4.5K
- Following1.1K
- Followers3.2K
- Likes9.3K
jayesh retweeted
Five years ago @adamvpro and I started @zenml_io because getting AI from a notebook into production was painful.
It still is.
Only now it's agents, not just ML models, that make it hard.
Building an agent takes an afternoon. Keeping it working is the hard part. A prompt tweak breaks something that worked yesterday. A cheaper model quietly stops answering. You only hear about it when customers complain.
Every team we talked to said the same thing: we don't need another dashboard, we need someone to do the work.
So we built that.
ZenML Pro now comes with an always-on AI engineering team. It reads your traces, finds what your evals miss, and proves the fix works before you ship it. This tech is backed by ZenML and Kitaru, our two OSS products, that stay open source and freely available.
To kick off this new chapter, we have decided to rebrand. ZenML is now ZenML Labs, and our ambition is greater than ever.
Connect your agent traces today, free at zenml.io.
We are very proud of the onboarding ;-)
We wrote about it here: zenml.io/blog/introducing-ze…
jayesh retweeted
We are launching Open Dots, run the same capabilities in 1/10th of the cost
OpenAI shipped Dot yesterday for Pro users.
Ours is open source, runs on your Mac and cheaper
Here's how it works:
- each dot has its own browser that stays logged in securely
- it can use and setup triggers on your Gmail, Calendar and 1,500 other apps
- you can call it and give it things to do while you talk
- dots can coordinate and schedule reminders
built with @composio @OpenRouter
free and open source: github.com/composio-communit…
jayesh retweeted
i've been loving instinct but i find that i jump between a lot of agents, rn mostly claude code, codex and instinct
lots of context is lost between them, very often i want to start something on cc then handoff loop to instinct, etc
been hacking on chirp.wiki to solve this
jayesh retweeted
I spent the last six months trying to deconstruct Taalas's patents. We think it can be better: 100x fewer memfetches than their bitROM, better software by better quantization than their hardware team could.
So we wrote a compiler that:
> takes a huggingface checkpoint,
> quantizes the model
> descends the weights down metal layers to RTL and GDS, do your DRC, Yosys, PEX, make the electrical waveform execute a mat-vec from your huggingface checkpoint (thanks Cambricon tech papers)
in ~7000 lines of human-readable code.
we're looking for someone with contacts with a foundry/access to 7nm PDKs or contacts at a cheap EuroPTW shuttle? I'm broke and unemployed.
@itsclivetime this is what i think is the future of perplexity per picojoule. @zerohedge @zephyr_z9 you guys wanna see 40,000 tokens/sec?
jayesh retweeted
Hosted our very first ModelOps night and talked all about evals and benchmarks last week with @wjayesh @jomulr Jay Shu @gmi_cloud @oqoqo_ai
jayesh retweeted
Hermes and Pi Agent led on the average cost per task, while Claude Code cost about 3.7x as much as Pi:
- $0.39 Hermes Agent
- $0.40 Pi Agent
- $0.47 Codex
- $0.51 OpenCode
- $0.54 Kimi Code
- $1.47 Claude Code
The median cost tells the same story: $0.29 in Pi Agent and Hermes, $0.35 in OpenCode, $0.38 in Kimi Code, $0.39 in Codex and $0.72 in Claude Code, so the cost gap holds for a typical task and is not driven by a few expensive runs.
We calculated these costs using Kimi K3’s list prices: $3/1M input tokens, $0.30/1M cached input tokens, and $15/1M output tokens.
jayesh retweeted
Token efficiency is the next benchmark that matters.
jayesh retweeted
Everyone is posting the exact same launch photo today, so I'll talk about what’s actually cool about Vikram-1 that geeks me out.
First, the entire rocket is carbon composite, even the motor casings. Most launch vehicles still rely on aluminum and steel. Carbon fiber is incredibly light and strong, but it’s a total nightmare to engineer, which is why almost nobody builds a fully composite rocket. Skyroot nailed it on their very first try.
Second, the upper stage engine is entirely 3D printed; it completely changes the game for production. Instead of waiting months for hundreds of machined parts, they can print a finished engine in days at half the weight. That is the reason a monthly launch cadence is even on the table.
Plus, that upper stage can restart mid-flight. That means a single launch can drop different satellites into totally different orbits, which is exactly what small-sat companies pay a premium for.
But the biggest deal is the strategic angle. Because it uses solid fuel, you can keep the rocket ready to go and launch on short notice without needing a massive launchpad. Right now, that’s just a great commercial selling point. In a few years, it becomes a massive national asset - the ability to replace a satellite in days if you need to.
This is a huge moment for Indian engineering. Massive congrats to Pawan, Bharath, and the entire Skyroot team.
A historic new frontier for India’s space journey!
At 11:30 AM today, Skyroot Aerospace will undertake the maiden orbital launch of Vikram-1, India’s first privately developed launch vehicle.
This four-stage rocket is designed to provide rapid and on-demand launch services. This mission highlights the talent, determination and entrepreneurial spirit of our youth. It also shows how our space-sector reforms are unlocking new opportunities for innovation and enterprise.
My best wishes to the entire Skyroot Aerospace team for a successful launch. May Vikram-1 soar high, create history and inspire a generation of innovators.
I urge all Indians, especially my young friends, to follow this historic mission and join in wishing Team Skyroot success using #IndiaWithVikram1.
@SkyrootA
jayesh retweeted
We tested GLM-5.2 on 41 agentic tool-calling tasks across GitHub, Gmail, PostHog, Notion, and a few other production APIs.
GLM-5.2 completed 40 of those tasks (97.6%), the best score of the three models we ran, ahead of Claude Opus 4.8 and GPT-5.5.
Details below 🧵
jayesh retweeted
"If you're outside, if you're open, if you let yourself follow the music down into the meadow and barge into the group standing by the decks, the city does the rest; it accepts you."
the post so nice, you read it twice 🤩
jayesh retweeted
Your AI agent is in bed with you.
No protection. You just wanted it to work.
Gmail. Allow. Calendar. Allow.
Slack, Notion, GitHub.
Allow. Allow. Allow.
Every password, handed over. Your agent never needed a single one.
They just needed @Composio
Secure your agents in minutes ↓
composio.dev/protection
jayesh retweeted
𝗵𝗶𝗿𝗶𝗻𝗴 𝗳𝗼𝗿 𝘁𝗵𝗲 𝗮𝗴𝗲𝗻𝘁𝘀 𝘁𝗲𝗮𝗺 at @composio in bangalore.
the next 6 months are going to be one of the most turbulent times for tech. when the dust settles, only a few companies will remain. we intend to be a part of the few that define what comes next.
what i filter for:
• you talk to AI agents more than you talk to humans
• lazy by default, but maniacally obsessive when the right opportunity shows up
• you've already automated most of your life with agents
• good enough is never good enough. you seek perfection in everything you do
• bar raiser: having you on the team raises the intellect, intensity, and taste of everyone around you
@composio has 𝟮𝟰,𝟬𝟬𝟬+ ⭐ across the github ecosystem with 𝟭𝟬𝟬𝟬+ 𝘁𝗼𝗼𝗹𝗸𝗶𝘁𝘀. agent orchestrator (@aoagents) hit 𝟰,𝟱𝟬𝟬+ ⭐ in under a month. we shipped it before openai and anthropic shipped theirs.
real adoption from solo builders to enterprises like @Zoom , @awscloud , and @glean
if this sounds like you, link below.
jayesh retweeted
this is actually insane
> be tech guy in australia
> adopt cancer riddled rescue dog, months to live
> not_going_to_give_you_up.mp4
> pay $3,000 to sequence her tumor DNA
> feed it to ChatGPT and AlphaFold
> zero background in biology
> identify mutated proteins, match them to drug targets
> design a custom mRNA cancer vaccine from scratch
> genomics professor is “gobsmacked” that some puppy lover did this on his own
> need ethics approval to administer it
> red tape takes longer than designing the vaccine
> 3 months, finally approved
> drive 10 hours to get rosie her first injection
> tumor halves
> coat gets glossy again
> dog is alive and happy
> professor: “if we can do this for a dog, why aren’t we rolling this out to humans?”
one man with a chatbot, and $3,000 just outperformed the entire pharmaceutical discovery pipeline.
we are going to cure so many diseases.
I dont think people realize how good things are going to get
jayesh retweeted
We have similar workflows :) zenml.io/blog/how-i-rebuilt-…
TLDR think a lot upfront. always get codex to review Claude's plans + code. use skills when you're repeating yourself a lot. etc
