Deploy and finetune LLMs easily: https://xitter.cf/t.co/SMYFJFbeTv
SF Bay Area
Joined June 2010
- Tweets380
- Following167
- Followers87
- Likes81
Sam Stoelinga retweeted
ALERT ALERT ALERT 🚨 🚨 🚨 VLLM MAINTAINERS HAVE JUST SHOWN THAT TPUv7 CAN GET 700 tok/s/user, 56% BETTER PERFORMANCE THAN NVIDIA GB200 NVL72 THROUGH MEGAKERNEL OPTIMIZATION ON KIMI K3.
As we said awhile ago, the TPU externalization of software is full steam ahead. This is ultra important to follow the progress of this.
Simple guide for anyone to deploy #Deepseek R1 14B on their laptop with a chat Web UI: medium.com/@samos123/deployi…
Sam Stoelinga retweeted
github.com/substratusai/kube… GitHub - substratusai/kubeai: AI Inference Operator for Kubernetes. The easiest way to serve ML mode
Sam Stoelinga retweeted
Making LLMs run efficiently can feel scary, but scaling isn’t magic, it’s math! We wanted to demystify the “systems view” of LLMs and wrote a little textbook called “How To Scale Your Model” which we’re releasing today. 1/n
Sam Stoelinga retweeted
535 ⭐️ KubeAI is a Kubernetes-native platform for running AI/ML workloads. It simplifies the deployment 🔧 and management 📝 of AI models in Kubernetes clusters, making it easier to scale AI applications.
star-history.com/#substratus…
#starhistory #GitHub #OpenSource
Tutorial: Deploying Llama 3.1 405B on #GKE Autopilot with 8 x A100 80GB substratus.ai/blog/deploying…
Sam Stoelinga retweeted
GitHub - substratusai/kubeai: Private Open AI on Kubernetes github.com/substratusai/kube…
Tutorial: Private RAG with Verba, #Weaviate, Embedding Model and LLMs all running inside your own K8s cluster. Copy paste-able instructions for all.
substratus.ai/blog/lingo-wea…
Need to provide a private OpenAI API compatible endpoint backed by OSS or fine-tuned LLMs? Lingo: an open source lightweight ML proxy and autoscaler for K8s makes this easy: github.com/substratusai/ling…
Kind + GPU: A Local K8s cluster that can run GPU pods youtu.be/O1683vzaJVE?feature… via #k8s #nvidia #gpu #ml
Tutorial: Converting HuggingFace Models to GGML/GGUF format substratus.ai/blog/convertin…
producthunt.com/posts/substr… Deploy and fine tune LLMs on K8s. Upvotes greatly appreciated!
Sam Stoelinga retweeted
Unlock the power of Apache Kafka on #Kubernetes!
Check out this tutorial to learn how to deploy and manage Kafka clusters with ease on #GKE for scalable and high-availability data streaming ↓
cloud.google.com/kubernetes-…
Annoyed about `gcloud container node-pools update --node-labels` overwriting all existing labels? Solve it with a simple bash script 😀
samos-it.com/posts/GKE-nodep…
Sam Stoelinga retweeted
⚡ Announcing the Spark Connector for Weaviate!
🗃️ The Spark Connector allows easy importing of data from @ApacheSpark into Weaviate
👀 Check out this tutorial to get started
👉 buff.ly/3jlleLp
👉 buff.ly/3YcAyc7
More in the 🧵
Made a new video: K8s Preventing Privileged Pods using Pod Security Admission / Standards youtu.be/D4Tw1blsQog