Evaluation for LLM-Based Apps

Joined March 2021
Listen to The Ravit Show on Spotify for Creators open.spotify.com/episode/5vF… @awscloud @deepchecks @DavidArakelya13
2
3
93
We’re hosting an in-person AI Agents meetup next week in Menlo Park, CA with @p0 and @Snowflake.✨ If you’re interested in joining, you can register here: luma.com/z2qae6kx See you there next week!
1
1
4
265
The upcoming @deepchecks webinar is about "End-2-End Evaluation of RAG-Based Applications".🚀 We will cover the best practices for evaluating RAG apps, covering initial experiments, version comparison, and ongoing evaluation. ✅ Register here: linkedin.com/events/end-2-en… #LLMs
2
3
6
1,186
Last week, we released the @Deepchecks LLM Evaluation solution publically on LLMOps Space and Product Hunt! Since then, 𝐰𝐞 𝐡𝐚𝐯𝐞 𝐫𝐞𝐜𝐞𝐢𝐯𝐞𝐝 𝐝𝐨𝐳𝐞𝐧𝐬 𝐨𝐟 𝐫𝐞𝐪𝐮𝐞𝐬𝐭𝐬 𝐟𝐨𝐫 𝐠𝐞𝐭𝐭𝐢𝐧𝐠 𝐚𝐜𝐜𝐞𝐬𝐬 𝐭𝐨 𝐨𝐮𝐫 𝐋𝐋𝐌 𝐞𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 𝐩𝐥𝐚𝐭𝐟𝐨𝐫𝐦. 🚀 In the launch webinar, 150+ people tuned in and we had a super successful launch. 🥳 🏄‍♂️ You can request the Deepchecks LLM evaluation platform here: deepchecks.com/solutions/llm… 🎥 In case you missed the webinar, here's the link to the session recording: youtu.be/zSooEIrMf-c?feature… (And our next LLMOps Space event is with the Pinecone ~ happening in just 9 days!, 🚀 register here: linkedin.com/events/71375280…) Also, if you'd like to launch your LLM-related projects with LLMOps Space, feel free to send me a DM. We'd be happy to help you with your launch, 𝐟𝐨𝐫 𝐟𝐫𝐞𝐞! ❤️
2
4
3
40
3,336
🚀 I’m thrilled to share some exciting news: 𝐃𝐞𝐞𝐩𝐜𝐡𝐞𝐜𝐤𝐬' 𝐧𝐞𝐰 𝐋𝐋𝐌 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 𝐦𝐨𝐝𝐮𝐥𝐞 𝐢𝐬 𝐧𝐨𝐰 𝐥𝐢𝐯𝐞! ✨ 😊Show your support on ProductHunt: producthunt.com/posts/deepch… Since our open-source package launch in January 2022 for testing ML models, the response from the community has been incredible, with over 3,000 GitHub stars and more than 900,000 downloads. 📈 Today, we're proud to announce the launch of our LLM Evaluation module, designed to tackle the unique challenges posed by LLMs. 🧠💬 What makes this LLM Evaluation module special: ✅ 𝐃𝐮𝐚𝐥 𝐅𝐨𝐜𝐮𝐬: Assess both accuracy and model safety (bias, toxicity, PII leakage). 📝 𝐅𝐥𝐞𝐱𝐢𝐛𝐥𝐞 𝐓𝐞𝐬𝐭𝐢𝐧𝐠: Adapt to scenarios where multiple valid responses are possible. 👥 𝐃𝐢𝐯𝐞𝐫𝐬𝐞 𝐔𝐬𝐞𝐫 𝐁𝐚𝐬𝐞: Empower data curators, product managers, and business analysts as well as the SWEs and ML practitioners. 🚀 𝐌𝐮𝐥𝐭𝐢-𝐏𝐡𝐚𝐬𝐞 𝐀𝐩𝐩𝐫𝐨𝐚𝐜𝐡: Cover Experimentation, Staging, and Production phases. We believe this module will make a dent in how AI systems are validated, especially in the dynamic world of LLM-based applications. 🌐 #LLMs #opensource #ArtificialInteligence #ML #GPT4
2
5
26
2,546
The most recent LLMOps.Space webinar featured @yujian_tang from @zilliz_universe, where he talked about "Advanced RAG concepts: Chunking, Embeddings, and Vector Databases". 🙌 It was one of the most engaging sessions we hosted at LLMOps Space, 150+ 𝐋𝐋𝐌 𝐩𝐫𝐚𝐜𝐭𝐢𝐭𝐢𝐨𝐧𝐞𝐫𝐬 𝐚𝐫𝐨𝐮𝐧𝐝 𝐭𝐡𝐞 𝐰𝐨𝐫𝐥𝐝 𝐭𝐮𝐧𝐞𝐝 𝐢𝐧, and lots of interesting questions were being asked (the most asked question was about chunking 😮 ). 😊 In case you missed the event or want to revisit it, we got you, here's the link to the recording: youtu.be/tTW3dOfyCpE?feature… ---------------------------------------------------------- The next LLMOps Space event is with Itamar Golan, cofounder of Prompt Security ~ 𝐡𝐚𝐩𝐩𝐞𝐧𝐢𝐧𝐠 𝐧𝐞𝐱𝐭 𝐰𝐞𝐞𝐤!, 🚀 register here: linkedin.com/events/71247277… 𝐉𝐨𝐢𝐧 𝐭𝐡𝐞 𝐝𝐢𝐬𝐜𝐨𝐫𝐝 👉 llmops.space/discord #llmops #mlops #machinelearning #openai #generatieveai #largelanguagemodels #ai #opensource
10
1
46
3,381
#NLPSummit Stage 1 is now live! Join us for this insightful talk by @ShirChorev from @deepchecks on Explainable Data Drift for NLP.
1
2
396
.@Deepchecks CTO @ShirChorev explores the common pitfalls of ML models and best practices for testing them. By the end of the video, you'll be ready to integrate ML model testing into your workflow, ensuring your models are reliable and maintainable youtube.com/watch?v=jvRXPkym…
1
5
489