CamGross (@CamGross) profile picture

Software Product Guy. Singletrack rider. Tech Fan.

Minneapolis
Joined December 2007
Photo by CamGross (@CamGross)Photo by CamGross (@CamGross)Photo by CamGross (@CamGross)Photo by CamGross (@CamGross)Photo by CamGross (@CamGross)Photo by CamGross (@CamGross)Photo by CamGross (@CamGross)Photo by CamGross (@CamGross)
I think AI agentic workflows will drive massive AI progress this year — perhaps even more than the next generation of foundation models. This is an important trend, and I urge everyone who works in AI to pay attention to it. Today, we mostly use LLMs in zero-shot mode, prompting a model to generate final output token by token without revising its work. This is akin to asking someone to compose an essay from start to finish, typing straight through with no backspacing allowed, and expecting a high-quality result. Despite the difficulty, LLMs do amazingly well at this task! With an agentic workflow, however, we can ask the LLM to iterate over a document many times. For example, it might take a sequence of steps such as: - Plan an outline. - Decide what, if any, web searches are needed to gather more information. - Write a first draft. - Read over the first draft to spot unjustified arguments or extraneous information. - Revise the draft taking into account any weaknesses spotted. - And so on. This iterative process is critical for most human writers to write good text. With AI, such an iterative workflow yields much better results than writing in a single pass. Devin’s splashy demo recently received a lot of social media buzz. My team has been closely following the evolution of AI that writes code. We analyzed results from a number of research teams, focusing on an algorithm’s ability to do well on the widely used HumanEval coding benchmark. You can see our findings in the diagram below. GPT-3.5 (zero shot) was 48.1% correct. GPT-4 (zero shot) does better at 67.0%. However, the improvement from GPT-3.5 to GPT-4 is dwarfed by incorporating an iterative agent workflow. Indeed, wrapped in an agent loop, GPT-3.5 achieves up to 95.1%. Open source agent tools and the academic literature on agents are proliferating, making this an exciting time but also a confusing one. To help put this work into perspective, I’d like to share a framework for categorizing design patterns for building agents. My team AI Fund is successfully using these patterns in many applications, and I hope you find them useful. - Reflection: The LLM examines its own work to come up with ways to improve it. - Tool use: The LLM is given tools such as web search, code execution, or any other function to help it gather information, take action, or process data. - Planning: The LLM comes up with, and executes, a multistep plan to achieve a goal (for example, writing an outline for an essay, then doing online research, then writing a draft, and so on). - Multi-agent collaboration: More than one AI agent work together, splitting up tasks and discussing and debating ideas, to come up with better solutions than a single agent would. I’ll elaborate on these design patterns and offer suggested readings for each next week. [Original text: deeplearning.ai/the-batch/is…]
184
1,208
199
5,150
841,071
I had a realization yesterday over coffee with a friend. Product vs. distribution in a startup - what’s more important? It's the classic debate. I realized there's a seismic shift in the startup playbook that many aren't noticing. Every top 0.01% CEO, PM, designer know this…
47
86
8
624
262,013
What would it take to convert an Ethereum mining rig to one that mines for LLM tokens? -- Asking for a friend.
77
OK, community. It's happening! The first, largest, and only conference focused exclusively on Artificial Intelligence in Minnesota, the Applied AI Fall 2023 Conference is BACK on Friday, Nov 10th. Ealry-bird tickets and calls for speakers opens TODAY! appliedaiconf.com
1
2
3
6
821
Have you used @ProdPad ? How has it changed your team and/or your process? #productmanagement #product
1
81
Having worked with Nichole directly, I can tell you that you want her talent. lnkd.in/gCarumZD
27
There was a time when the three of us were colleagues. More than 15 years later, we're still friends. We pick up where we left off as if only a week has passed. We learned a lot about business in those early days of our careers. Much of those learnings…lnkd.in/gMPq2bwJ
1
55
With the right journey and process design, Reemo Health opens an opportunity to connect and respond in real-time to common SDoH issues seniors experience related to safety, food security, medication adherence, mental health and isolation. #medtech #weara…lnkd.in/gDHwx3HD
25
"As Benedict Evans points out: “There’s an old saying that when we get a new tool, we begin by making it fit the old way of working, and then we change the way we work to fit the new tool.” Perfect timing on this article from @cagan. These ten points wil…lnkd.in/g4a5kxp7
1
181
Brian Shulman -- have you seen Ōmcare yet? lnkd.in/egbcbxSS
32
Good morning, friends and fans of product management. In follow up to previous posts about bringing product minds together to talk about Applied AI in Product Management - please VOTE for your preferred time of day/week. Original post:…lnkd.in/gXnKiWUj
1
1
158
Product Management and Artificial Intelligence - Where are we going? - a workshop proposal If your feeds are anything like mine, you're finding it impossible to even skim all the latest developments, trends, and opinions related to AI. There's a lot to l…lnkd.in/gwZ2snqM
1
1
59
The Whitehouse has issued an AI Bill of Rights. Should this guidance (and future iterations) become part of a government department like the FDA that classifies automated/AI systems along with having the teeth to provide approval/r…lnkd.in/gVHaMhys lnkd.in/gjWXV75P
33
As a friendly addition: I would argue that 'A' can still be successful even if the tools aren't ideal and if organizational support is mixed. Psychological safety and team trust can go a long way on their own. Conversely, if just one of org chaos, lack…lnkd.in/geEurcwM
28
I'm so grateful for the kind words Joe Schneider. lnkd.in/g_uXpT9n
31