all sorts of engineering
zsh 💻
Joined July 2013
- Tweets561
- Following133
- Followers65
- Likes6.7K
alberto retweeted
Getting benchmark numbers is easy. Getting the right ones is hard. KV cache offloading can cut TTFT from 60s to 10s and lift throughput up to 5.8x, but only when your working set and storage tier line up. Here's how we benchmarked it, caveats and all: developers.redhat.com/articl…
alberto retweeted
Join us to learn more about about the state of open source inference!
Hola, Madrid. vLLM and llm-d are coming to town: Red Hat and @nvidia are hosting an evening on open-source inference, with @vllm_project, LLM Compressor, vLLM-omni, safe agents, and llm-d on Kubernetes.
November 12 at MOA Institute. Register: luma.com/gl1d2loa
alberto retweeted
Today at PyTorch Conference 2026, we announced llm-d 1.0! 🎉
Distributed inference is too complex to solve behind closed doors.
17 months in, llm-d is now a @CloudNativeFdn project powering fleets worldwide.
Here’s what’s inside 1.0 🧵👇
alberto retweeted
"No man ever runs an experiment on the same infra twice, for it's not the same infra and he's not the same man." - Heraclitus
alberto retweeted
Right. Dario was already claiming that GPT2 was too dangerous to open source back in 2019.
I made fun of them then.
Everyone should make fun of them now.
alberto retweeted
Terence Tao just posted this really insightful thread which seems to have been written in reaction to the Navier Stokes announcement. Please give it a read. mathstodon.xyz/@tao/11723732…
alberto retweeted
It's honestly insane that Claude code is closed source. It's a JavaScript bundle that makes API calls and all your competitors are open. Are you a model harness company or an AI company
alberto retweeted
vLLM Office Hours #56 recording is up:
- What's new in vLLM 0.27
- Running Codex and Claude Code CLIs on open models
- Speculators v0.6 and v0.7 update
- GuideLLM v0.7.0: benchmarking reasoning, tool calling, and agentic traces
Watch + see slides: youtu.be/dQAByb7N5QQ
![[vLLM Office Hours #56] - GuideLLM v0.7: Benchmarking Reasoning and Agentic Traces - August 20, 2026](https://pbs.twimg.com/card_img/2117386975913123840/ujz3Co-b?format=jpg&name=800x419)
alberto retweeted
On behalf of the vLLM community, @inferact just signed the open letter. Let's build an open ecosystem together!
Inferact is proud to co-sign. We were founded to grow @vllm_project into the world’s leading open-source inference engine, making open-weight AI faster, cheaper, and accessible across hardware and clouds so intelligence can be on tap for everyone.
alberto retweeted
extremely unprofessional. if kimi wants to make it as a frontier lab, they need to act like one: perhaps silently route people to worse models, and maybe write a blog post about the collapse of humanity
Kimi K3 has received far more love than we expected, and our GPUs are feeling it.
Over the past 48 hours, demand has pushed close to the limits of our current capacity. To protect the experience of existing subscribers, we're temporarily pausing new subscriptions and prioritizing compute for current members. Existing subscribed users are not affected.
We're adding capacity as fast as we can and will reopen new subscription spots in batches.
Going forward, we'll also split membership into two more focused plans: Kimi Membership for Kimi Web, App, and Work; and Kimi Code Membership for coding workflows. This will help us match compute more precisely and keep the experience stable.
Thank you for your patience and understanding!
alberto retweeted
Introducing NEO’s 25 Degrees of Freedom, tendon-driven hands — nearing or surpassing human-level dexterity, strength, speed, and reliability.
For seventy years, robotics worked around the hand problem. The humanoid bet is the reverse: it lives or dies at the fingertips.
if you're running kv-offload experiments you're probably not offloading anything, the cache just chills in hbm the whole time 🥶 here's the simple math to size your experiments albertoperdomo.me/posts/kv-c…
alberto retweeted
🎉 llm-d v0.7 is officially live! 🚀
Earlier releases proved what llm-d could do, v0.7 is about making sure you can easily deploy it in production.
Backed by a massive 3.5x surge in community PR volume, this release hardens the stack for serious scale.
llm-d.ai/blog/llm-d-v0.7-fro…
alberto retweeted
in Spain they let the servers rest at night
Buenos días,
Te informamos que durante la noche la página web realiza un proceso automático de mantenimiento y reseteo de sistemas, por lo que es habitual que no se puedan procesar compras en esa franja horaria. No obstante, si necesitas adquirir tus billetes, tienes a tu disposición todos estos canales oficiales de venta alternativos: Atención Telefónica (las 24 horas): 912 320 320, App, en las estaciones: taquillas de venta física, máquinas Autoventa Multiproducto (disponibles en los vestíbulos) y Agencia de Viajes autorizada.
Un saludo,