ai engineers ship great models and awful first impressions.
cache warming is the easiest, highest-leverage way to cut latency, yet almost everyone ignores it, even in voice ai. that obsession with the first second is one reason why @boardyai stands out!
interesting, yeah it’s fundamentally a different problem when you add vision. read, heard, see.
as soon as humans are actually looking at something, engagement and interaction rates skyrocket
1
1
97
It’s a completely different problem away from cascaded/full-duplex.
It’s more so context like you mention - just imagine talking to someone over the phone. You’re painting a picture in your head that could or could not be what the other person is describing.
When a voice-agent in realtime can visualize things for you, you’re more engaged. That’s where I see the future of conversational ai heading towards. No longer a voice problem, a conversation problem
2
2
85
that's what voice ai will def turn into. I'd rather read a bunch of text then forget what the agent said 30s ago.
2
2
196
Last month, we had 30,000+ hours of conversation. Yes, 30 thousand.
I’m the only engineer working on voice ai at @boardyai and here's what I'm noticing across the industry.
In the race for faster responses and more natural voices, I think we’re losing sight of what makes a conversation actually valuable. Helping someone understand.
Sometimes that takes more than words. It takes something you can see, point at, question, and explore together. That’s where I see conversational ai heading, and where I’m taking voice at Boardy.
honestly speaking most times one is enough. 2 is just a safe guard I have since prefix caching lives on one servers GPU's anyway. the only thing is that if it lands on replica 1 and the first turn routes to replica 2 it doesn't help.
so when I was benchmarking this and testing just kept 2. cost is minimal anyways but yes you can easily just do 1! good q
2
2
116