Mike Henry (@mikehenry8484) header
Mike Henry (@mikehenry8484) profile picture

Founder and CEO of https://nitter.gq/t.co/cXuCMfT2BF (@parasail_io). Previously Founder/CEO of Mythic and CPO at Groq.

San Francisco Bay Area
Joined August 2023
Photo by Mike Henry (@mikehenry8484)Photo by Mike Henry (@mikehenry8484)Photo by Mike Henry (@mikehenry8484)Photo by Mike Henry (@mikehenry8484)Photo by Mike Henry (@mikehenry8484)Photo by Mike Henry (@mikehenry8484)Photo by Mike Henry (@mikehenry8484)Photo by Mike Henry (@mikehenry8484)Photo by Mike Henry (@mikehenry8484)Photo by Mike Henry (@mikehenry8484)
Positron's Titan is joining the Parasail fleet. 5MW. Trillions of tokens a day of cost-optimized compute by late 2027. @parasail_io @positron_ai @parasail_io is the first multi-silicon inference cloud that routes your prompts to the best chip for the job, serving trillions of tokens a day to hundreds of customers. Positron fills a vital part of compute spectrum on the cost optimized side: 4X better cost-per token than next gen NVIDIA/AMD systems which is perfect for large-scale agent swarms, background agents, and physical AI. Positron has done incredible work building a chip and system that orchestrates across DRAM and SRAM allowing for massively scalable memory pools with commodity memory giving the best of both worlds for memory capacity and bandwidth. We've been excited to partner with them in the early days through the launch of their Atlas chip and now big capacity plans with Titan. @positron_ai
52
18
34
124
221,076
Congrats to @MeghanaMadhyas1 and Ben for the great talk on the many layers of agents we use to drive top tier performance with a small and mighty team.
Our agents tune LLM inference on production GPUs. Here’s what had to be built for that to work: - GPU leases + a queue so experiments don’t steal customer capacity - Cloned deployments on mirrored live traffic - Canaries with auto rollback and a human sign-off Checkout @MeghanaMadhyas1 and Ben Segal's talk at @QCon later today to learn more about our autonomous LLM performance optimization system.
3
1
12
534
Announcing the first of many GPU clusters that we own and operate ourselves. Given the volatility of the market, a hybrid strategy of owning compute and renting from the utility-scale GPU providers (i.e. not Neoclouds trying to claim software margins) is the way to deliver the best economics and flexible capacity to our customers.
Our newest owned cluster is live: NVIDIA B300s in Denver, in the @parasail_io fleet the day after burn-in. We have a mixed fleet of renting & owned capacity. Rented lets us scale fast and pick up new silicon as it lands. Owned clusters give us a base we control and tune for the largest customer workloads. An inference cloud locked into one way of getting compute gets stuck when the winds change.
9
3
40
2,700
Mike Henry retweeted
Implication - If the moat on your HW is actually SW, you no longer have a moat. If, otoh, the moat on your HW is actually HW, not only is your moat intact, but customers' ability and willingness to try your HW is dramatically increased. Btw, actual HW moats exist in two distinct flavors: 1. Roofline HW capability. Everyone can get much closer to roofline, much faster now. 2. Supply chain. Do you have the wafers, parts, manufacturing capacity, ... to deliver at scale? Both must be considered.
From the Anthropic Road Show video: 'This has also made us really good at bringing up new chips. As an example, this year, we started working with AMD GPUs for the first time. In the past, bringing on a new chip platform took dozens of engineers months of work. This time we had Mythos try it out. It got Claude running on AMD in just 3 days with basically no human intervention. Claude ported or wrote almost 100% of the kernels.'
25
28
20
279
36,833
Mike Henry retweeted
Why do you think Anthropic emphasized this?
151
183
61
5,414
732,965
Mike Henry retweeted
Megakernels are overrated
14
2
2
128
13,471
It feels like the 3-4X is achieved where the KV cache is very full and starts missing a lot. Is this just another way of saying it increases effective KV cache space by ~10%?
Introducing FlashState ⚡: an inference system that flexibly manages the prefix cache of hybrid LLMs, designed for agentic workloads. On GLM 5.3 Flash, FlashState more than doubles vLLM's throughput and provider profit, while cutting latency by 3.7× and user cost by 2.5×.
1
8
749
Solving utilization for heterogenous compute is the defining problem and a lot going on at @parasail_io behind the scenes on this. Stay tuned for some big launches later this year.
A chip that’s 3× faster but idle half the day isn’t 3× cheaper: A prefill or decode device only pays off when your traffic matches the ratio you bought, and that ratio keeps changing. Your GPUs can switch roles when that happens, but specialized silicon just sits idle while you keep paying for it. Software keeps moving the bar too, since megakernels alone have pushed low-concurrency decode on ordinary GPUs up by 40–75% this year, and every one of those gains is another knock on specialized chips. Consider: “how fast?” but “how fast, times how often it’s actually busy, after moving the KV cache over the network?” If your GPUs run at 70% and their chip averages 40% across a real year of traffic, it needs to be ~1.75× better just to tie. For a single model with very stable traffic (like a premium fast tier), that can work out, but in many case, general-purpose computing can be more useful.
1
1
14
1,234
Parasail is top tier! 👊🇺🇸🔥
Ranking inference providers six ways
2
3
1
36
7,343
Interesting times indeed
Was talking with the founder of a large inference cloud yesterday. His thoughts: 1/ lot's of rumours of some large chunks of 2027 capacity has disappeared - now no longer being built or not financed 2/ demand for tokens isn't going away it's exploding and pushing spot prices up 3/ whoever secures real compute and already has real compute in an even more supply constrained environment has the cards right now Interesting times
10
1,074
Inference providers who have already secured a ton of compute for next year:
1
7
894
Target acquired
Baseten’s first swing at an intelligent router: best in class (cheapest at every pass rate)
2
143
Stay tuned for technical details on how we pushed Kimi K3 to the limits to achieve the fastest endpoint on OpenRouter and 16% of all Kimi traffic. OpenRouter is the real battleground for performance and capacity under load, ignore the Benchmaxxed surveys out there.
3
1
9
813
the whole “we’re sota for american open weight models” when you’re still worse than china is quite odd energy
6
3
28
5,067
Mike Henry retweeted
Come join us for a happy hour during the @aiDotEngineer Code Summit next week! Stop by from 5:30-8pm at Bartlett Hall in SF. Link to reg in thread.
1
4
9
1,098
Old school founders still know how to grind out a fucking round
In venture rounds today you either raise the round in 2 hours with three tranches to come or you don’t raise a round at all.
3
291
If we were an airline, every seat in every flight would always be full. Imagine how profitable that would be. 99.4% of our available B300s in use this morning. B200s 99%, H100s 100%, H200s 98%. Split is 80% private traffic, 20% public traffic with the token marketplaces as the capacity buffer as private traffic scales up and down. Our new control plane cut our idle GPU hours in half. Inference is at its core a utilization business.
5
4
1
38
3,659
We have 3 AEs on our team and like 1.5B in pipeline built, I’m not worried about demand…
lot of people making stuff up so here's some more context you can imagine there's hidden enterprise demand for open source models that isn't visible but if you look at the top inference providers that aggressively go after this business, they have shared some numbers and they are all roughly the same size as us. and at best there's 5 of them if you think enterprises are self hosting inference on their own GPUs, there are so few people who can run reliable inference and they get paid a lot and work at ai labs. and if they really want sovereignty they're not getting that by leasing GPUs from a third party the root point here is this is entirely the wrong way to look at things, you just have to look at products how many products exist with a lot of traction that can use open models? this is what creates demand, it's not some magic thing that grows on its own OpenCode is one of those and we do see a lot of enterprise inbound. but it's so early, all of these companies are just starting to look at mixing in some OSS inference so yes demand will probably go up in the future but everyone in this industry is investing resources in preparing for that future. and so few people are building the products that will make it happen
3
666
Mike Henry retweeted
lot of people making stuff up so here's some more context you can imagine there's hidden enterprise demand for open source models that isn't visible but if you look at the top inference providers that aggressively go after this business, they have shared some numbers and they are all roughly the same size as us. and at best there's 5 of them if you think enterprises are self hosting inference on their own GPUs, there are so few people who can run reliable inference and they get paid a lot and work at ai labs. and if they really want sovereignty they're not getting that by leasing GPUs from a third party the root point here is this is entirely the wrong way to look at things, you just have to look at products how many products exist with a lot of traction that can use open models? this is what creates demand, it's not some magic thing that grows on its own OpenCode is one of those and we do see a lot of enterprise inbound. but it's so early, all of these companies are just starting to look at mixing in some OSS inference so yes demand will probably go up in the future but everyone in this industry is investing resources in preparing for that future. and so few people are building the products that will make it happen
110
72
37
1,590
143,133
"Invest in fast growing markets" amen
Why Hand Wringing about Defensibility Is Probably Pointless: - analyzing how product value grows as a company scales is really hard -- this generally involves reasoning about conversion properties of an adaptive feedback system - analyzing how product value is impacted by competitors' scale is really hard -- also adaptive feedback system - analyzing competitive equilibrium is really hard -- autonomous differential equation - stochastic nature of serendipity and setbacks are really hard to bound - all this shit is dynamic based on macro factors, team, cyber threat environment etc. - market growth trumps all this stuff anyway, as fast growing markets create new white space which totally changes the value functions I've now read a lot of the key papers on this stuff, and most of it is either post hoc, or makes mature market assumptions which I just don't think are all that relevant with early stage tech startups. The tl;dr is invest in fast growing markets.
252