@dayoffdev

building: @reopenly_ai 🚀 prev: @LandingAI

 RJ, Brasil
reopenly.com
Joined September 2023
  • Tweets269
  • Following367
  • Followers232
12Media
RAG results still bad? Your embedding model may be too generic. Tuning it helps: up to 41% better retrieval in Google Cloud's tests (12% on average). New on ReOpenly: your server reports each search, you keep what helped, and you fine-tune and host your own model.
1
1
2
4
98
For people working with RAG, what is your biggest pain at the moment?
19
For all the people building search Have you noticed that a generic embedding model misses the words of your domain?
1
2
133
Try this first with free credits, before you train anything: 1. Similarity: your question vs three of your passages 2. Vector: six texts, see which ones group together 3. Report 20 real searches (free), review a few Train when you have reviewed searches.
1
3
3
89
The same three in code, one command each, and a comparison with the base model on searches you kept aside: github.com/reopenly/cookbook The guide: reopenly.com/blog/embedding-…

GitHub - reopenly/cookbook: Runnable ReOpenly examples for embeddings and model workflows

Runnable ReOpenly examples for embeddings and model workflows - reopenly/cookbook

github.com
1
16
What is a hard negative in the RAG context, in four posts 📝 1/ A hard negative is a result that looks right and is not: your search ranked it near the top, and it did not answer the question. "How long do refunds take?" answered with the return policy.
2
1
2
90
more replies
4/ A wrong label teaches the wrong thing, so nothing is used until you review it. A result you leave unchecked is never a label, a click never counts as one, and a score never picks one. Then you fine-tune on what you kept, and compare the new model with the base before you switch.
1
8
How the review works: reopenly.com/docs/datasets/s…

Search feedback | ReOpenly API

Reviewed searches promoted into Training pairs and pairs explicitly saved

reopenly.com
8
Sources: Google Cloud docs.cloud.google.com/vertex… · NVIDIA huggingface.co/blog/nvidia/d… · Databricks databricks.com/blog/improvin… The guide: reopenly.com/blog/embedding-… Reporting searches is free. Limits: up to 20 results per reported search; up to 512 tokens per input, 64 inputs per request.

Tune text embeddings  |  Gemini Enterprise Agent Platform  |  Google Cloud Documentation

Learn how to tune text embedding models on Gemini Enterprise Agent Platform for scenarios requiring specialized knowledge or highly tailored performance.

docs.cloud.google.com
32
ChatGPT Ads is now available in 26 more countries and 100k+ advertisers have already used it. But what does this mean for people building on top of these models? People will take decision based on ads openai.com/index/chatgpt-ads…

ChatGPT Ads expands to 26 new countries as over 100,000 advertisers have used the platform

From local businesses to global brands, more advertisers can now reach people as they explore, compare and decide what to buy in ChatGPT.

openai.com
2
120
RPA was so complex previously, now is soooo easy, we have so much power in our hands 🔥
1
48
Talking to some friends in tech I realized that most of them not even know what a harness is, we live in a completely different world here on X
1
4
133
Omg that is news to me, but not surprised tho'
Cyrus Maxx
@CyrusShepardOct 13
Incredible: Not only is Google working to block AI crawlers, but @nozzleio uncovered evidence that Google is deliberately serving FAKE search results to scrapers The "poisoned" search results look real, but w/ garbage data. This influences AI crawlers, scrapers, & rank trackers.
1
95
I realized that I don't need some SaaS to schedule my social media posts, nor to collect my analytics; in one claude session I built one with browser automation and some official libraries (e.g. linkedin offers for free)
4
1
142
LLMs + Monorepos = Peak Developer Velocity. Having complete full-stack context in one place allows for seamless, e2e application updates without the usual protocols or friction
1
73
Where Claude Sonnet 5 still won in our test: rare categories, 67.4% vs 60.4%. With almost no examples to learn from, broad knowledge helps. The fix is better examples for those cases. Fine-tune the stable, high-volume task. Keep the frontier model for the rare ones.
Dayanne Fernandes
@dayoffdevOct 12
A fine-tuned 8B model beat Claude Sonnet 5 on 437 contract clauses. 73.7% vs 70.0% accuracy 250 ms vs 2,230 ms per answer $0.125 vs about $2.00 per 1,000 answers Sonnet 5 still won the rare categories, 67.4% vs 60.4%. On a narrow task, the specialist wins.
1
4
109
A fine-tuned 8B model beat Claude Sonnet 5 on 437 contract clauses. 73.7% vs 70.0% accuracy 250 ms vs 2,230 ms per answer $0.125 vs about $2.00 per 1,000 answers Sonnet 5 still won the rare categories, 67.4% vs 60.4%. On a narrow task, the specialist wins.
3
3
4
11
862
The write-up, with the method: reopenly.com/blog/small-mode… The cost is GPU time: 1,000 answers at 250 ms each, at ReOpenly's rate, against Sonnet 5's API price for the same answers. The dataset is LEDGAR, from LexGLUE, public.

The $9 model that outperformed a frontier LLM · ReOpenly

A fine-tuned 8B model beat Claude Sonnet 5 on 437 contract clauses: 73.7% vs 70.0% accuracy, 9x faster and 16x cheaper. Here is where it lost, and why.

reopenly.com
4
47
Load more