patapsco.ai retweeted
Thank you to our #CNAIDAY sponsors: @SUSE, @vmwarevcf, @vultr, @patapsco_ai, @RedHat, @spectrocloudinc, @AranyaInc & @lmcache! Dive in Nov 9 in Salt Lake City: bit.ly/3VQKD2J
patapsco.ai retweeted
Excited to introduce the platform we've been working on for the past year!
@Patapsco_AI believes you should be able to run your own intelligence. Running AI on a single node is easy. Running it on a cluster is harder. We're giving the community a way to run multiple replicas of very large models across a cluster, with KVCache-aware load balancing, API aggregation, VSCode extension, a custom-built stream caching filesystem, and a bunch of other goodies.
You can run it on a single node DGX Spark, or a 100-node B200 cluster.
We're making it freely available for anyone to download and use at home or at work.
If you already run Kubernetes, you can deploy it directly, there is a tool (pailab) to setup a cluster as a service.
If you have any issues, join our slack (slack.patapsco.ai) and hit us up.
I would consider this a beta-release, but it's been fairly well tested. Would love to know what people think.