Is $20/month for an LLM chatbot too much or too little? It's the perfect price if you're reading 2.5 books a day, i.e. 17 hours continuous reading every day of the year! Here is an oversimplified gross margin calculation. OpenAI offers GPT3.5-turbo API and charge $2 per million token. A token is 0.75 english words. So $240/year is 90 million words, which is 250k words per day. An average book is 100k words, so to break even compared to API calls you'd have to read 2.5 books a day worth of a chatbot. A human reads 250 words per minute, so you need to read about 1000 minutes = 17 hours per day. Obviously, this doesn't hold if the chatbot does way more than what GPT3.5-turbo provides. TLDR: API calls are cheap, chatbot subscriptions are not.

Today we released an open source model, DBRX, that beats all previous open source models on the standard benchmarks. The model itself is a Mixture of Experts (MoE), that's roughly twice the brains (132B) but half the cost (36B) of Llama2-70B. Making it both smart and cheap. Since only 36B expert parameters are used live, it's close to twice the speed (tokens/seconds) of Llama2-70B. We're excited to build custom versions of this for organizations that have proprietary data! Check it out! databricks.com/blog/announci…