What do AI voice agents and self-driving cars have in common? Their performance can be evaluated similarly, argues Brooke Hopkins, former technology lead at Waymo. Coval, a new startup from Hopkins, looks to do just that.
“When I left Waymo, I realized that many of these problems we were having at Waymo were exactly what the rest of the AI industry was facing,” said Hopkins (pictured above). ) told TechCrunch. “But everyone was saying this is a new paradigm. We have to come up with testing practices from first principles, and basically we all have to reproduce everything. And I looked at it and said, wait, we’ve spent the last 10 years on our own trying to figure out how to do this.
In 2024, she plans to launch Coval, a platform for building simulations of AI voice and chat agents to test and evaluate how to perform tasks, similar to how Hopkins tested self-driving cars at Waymo. We have decided. Coval can run thousands of simulations simultaneously, such as having agents make restaurant reservations or having agents answer customer service questions asked to them indirectly.
Coval’s Tech evaluates agents on a common set of metrics, but companies can also customize what they’re looking for and continue evaluating regressions using Coval. Users can also retrieve this data. The insights also shine through and bring it to the end customer for demos or monitoring tools that show the customer that the agent is working as intended.
“One of the biggest blockers for agents employed in companies is to be confident that this is not a demo with smoke and mirrors,” Hopkins said. “Choosing between vendors is a very complex task for these executives because it’s very hard to know what to even ask or these agents don’t know what you’re expecting. Because it’s very difficult to know how to prove what you’re doing. And this gives our companies the ability to really show that and demonstrate that.”
Hopkins really formulated the idea behind Koval during the Summer 2024 batch before rolling out the product in October 2024. She said demand has been strong and explosive over the past two months. Evaluated.
The San Francisco-based startup is announcing a $3.3 million seed round led by Mac Venture Capital with participation from Y Combinator and General Catalyst. Startups use capital to build engineering teams and work to achieve product-market fit. Hopkins added that the company is working to allow users to evaluate other types of AI agents in the future, such as web-based agents.
Coval is coming to the scene, but it seems like both momentum and hype around the AI agent were at an all-time high. Enterprise technology leaders like Marc Benioff are praising (and marketing) the technology, saying Salesforce will deploy over 1 billion AI agents by next year. Openai is rumored to be releasing an AI agent view soon.
There are many startups being built in the space. There were over 100 startups building AI agents in Y Combinator’s three 2024 cohorts alone. Some AI agent startups have landed significant venture funding rounds. /dev /agents raised a $55 million seed round in November 2024 at a $500 million valuation.
This momentum means more companies are likely to seek help evaluating agents. Hopkins said Coval has a good shot at standing out from the pack because, unlike the inevitable newcomers, Coval has a head start.
“I think what really sets us apart is that I’ve been working in this field for 50 years and building these systems over and over again,” she said. “We’ve built multiple iterations. We’ve seen how they fail, how they scale, and build the same concepts into Coval and all learning.”
TechCrunch has a newsletter focused on AI! Sign up here to get it in your inbox every Wednesday.