Get the latest tech news

LLMs and coding agents are a security nightmare

Things are about to get wild

Cognitive gaps in chatbots like that (to some degree addressable by guardrails) are bad enough, but there’s something new—and more dire—on the horizon, made possible by the recent arrival of “agents” that work on a user’s behalf, placing transactions, booking travel, writing and even fixing code and so on. The Nvidia talk by Becca Lynch and Rich Harang at Black Hat was a terrifying teaser of what is coming, and a master class in how attackers could use new variations on prompt injection to compromise systems such as coding agents. As a cybersecurity researcher who has been focusing more and more on AI, Nathan had already seen the writing on the wall and in fact, had been warning about the risk of these kinds of exploits for the past couple of years, proposing an attempt to mitigate these attacks and a simple technique he called RRT (Refrain Restrict Trap).

Get the Android app