Get the latest tech news
Using Agents to Not Use Agents: How we built our Text-to-SQL Q & A system
Ask-a-Metric is a WhatsApp-based AI data analyst that uses LLMs to answer SQL database queries, facilitating data access for decision-making in the development sector (GitHub). Initially, we used a simple pipeline for rapid feedback but faced challenges in accuracy and building it for scale. We tested an agentic approach with CrewAI, improving accuracy but ending up with high costs and slow response speeds. We used these results to develop a pseudo-agent pipeline that combines the best of both approaches, reducing costs and response times while maintaining accuracy.
We can use agents to implicitly perform a “search” over the parameter space we have described above and find the best “minima,” i.e. the set of actions, tools, and prompts that results in the highest response accuracy at the lowest cost. Figure 4 illustrates the key technical aspects of the newer pipeline which helped us keep the same level of accuracy as the agentic flows but reduce cost and speed by an order of magnitude. Going forward, we are working on improving our current solution along all three of our key metrics–accuracy, speed, and cost–while also building more features like multi-turn chat, easier user onboarding, multi-language support, etc.
Or read this on Hacker News
