LLMs have made exciting progress on hard tasks! But they still struggle to analyze complex, unstructured documents (including today's Gemini 1.5 Pro 002).
We (UC Berkeley) built 📜DocETL, an open-source, low-code system for LLM-powered data processing: data-people-group.github.io/…
This is why @ BuildBetter.ai we spend so much time in our marketing saying:
No waitlists, no unrealistic demo or promises.
We've spent the last 4 years building tools for product teams that we wish we had powered by AI with very real ROI.
Real value, today. Right now. – at Manhattan, NY
Hot tip: when using llms to generate structured outputs with libs like instructor, ai sdk, or openai's strict mode, the order of the properties passed into the schema really matters.
Remember that these autoregressive models can only generate one token at a time, and use the previously generated tokens to generate the next token.
You can use this to really improve performance by adding a "reason" field before any other structured outputs, giving it a scratch pad to iterate on the answer with chain of thought before giving you the real answer (the score in this case).
But the order really matters here, if you put the "reason" field after everything else, it won't do anything. The answer is already generated, the llm will just use the "reason" field to justify its own answer.