Get the latest tech news

How well do agents use test/verification techniques?


We previously noted that, while it's easier than ever to hit a particular quality bar by having coding agents use effective test techniques, software quality seems to be getting worse, indicating that whatever defaults developers are using may not work very well. Here, we test if simple instructions to agents to use particular techniques or libraries improve implementation correctness, as a kind of test to see how effective agents are when guided by someone with no expertise in testing who's maybe heard that you should apply certain techniques or use certain libraries.

None

Get the Android app

Or read this on Hacker News

Read more on:

Photo of test

test

Photo of agents

agents

Related news:

News photo

USC Study Finds AI Agents Can Autonomously Coordinate Propaganda Campaigns Without Human Direction

News photo

What it feels like when OpenAI's agents hijack your website

News photo

Authors push back as publishers and agents make claims on Anthropic settlement