Prompt Injection Isn’t Going Away—and OpenAI Knows It

It bears repeating, even if it feels like a broken record: Artificial Intelligence (AI) agents, no matter how advanced, are still vulnerable to manipulation (or prompt injection, to be exact)—especially when they are let loose on the open web.

Even as OpenAI works to harden its Atlas AI browser against cyberattacks, the company is conceding a reality many security professionals have long accepted: prompt injection attacks are not going away anytime soon. Prompt injection, a technique that manipulates AI agents into following malicious instructions hidden in web pages, documents, or emails, continues to raise uncomfortable questions about just how safely AI agents can operate with broad autonomy and deep access.

“Prompt injection, much like scams and social engineering on the web, is unlikely to ever be fully ‘solved,’” OpenAI wrote in a blog post published Monday, detailing how the company is reinforcing Atlas’ security posture to deal with a steady stream of attacks. The firm also acknowledged that ChatGPT Atlas’ so-called “agent mode” inevitably “expands the security threat surface.”

That admission is notable, but not surprising.

When OpenAI launched ChatGPT Atlas in October, security researchers wasted little time demonstrating its weaknesses. Within days, researchers showed that a few carefully placed words inside a Google Docs file could alter the behaviour of the underlying browser agent. Around the same time, Brave published its own blog post explaining that indirect prompt injection is a systemic problem for AI-powered browsers—not just Atlas, but competitors like Perplexity’s Comet as well.

And OpenAI is far from alone in sounding the alarm.

Prompt Injection: Experts Have Already Warned About It

Earlier this month, the UK’s National Cyber Security Centre warned that prompt injection attacks targeting generative AI applications “may never be totally mitigated,” cautioning organisations that the goal should be risk reduction, not outright prevention. In other words, mitigation—not elimination—is the realistic objective.

For its part, OpenAI is framing the issue as a long-term fight.

“We view prompt injection as a long-term AI security challenge, and we’ll need to continuously strengthen our defenses against it,” the company said.

So, how exactly is OpenAI going about this Sisyphean task?

The company says it is relying on a rapid, proactive security cycle designed to surface new attack techniques internally before they are exploited in the wild. This philosophy is broadly aligned with what rivals such as Anthropic and Google have been advocating: layered defenses that are continuously tested and refined. Google, for example, has been focusing on architectural and policy-level controls for agentic systems.

Where OpenAI diverges is in its use of what it calls an “LLM-based automated attacker.”

In simple terms, OpenAI has trained a bot—using reinforcement learning—to behave like a hacker. This automated attacker probes AI agents for weaknesses, testing malicious prompt strategies in simulation before they ever reach real users. The simulator allows OpenAI to see how a target AI reasons through an attack, what actions it would take, and where safeguards fail. The bot then iterates—tweaking the attack and trying again.

That visibility into an AI’s internal reasoning, OpenAI argues, gives it an edge that real-world attackers do not have.

“Our [reinforcement learning]-trained attacker can steer an agent into executing sophisticated, long-horizon harmful workflows that unfold over tens (or even hundreds) of steps,” OpenAI wrote. “We also observed novel attack strategies that did not appear in our human red teaming campaign or external reports.”

In one demonstration, OpenAI showed how the automated attacker slipped a malicious email into a user’s inbox via prompt injection. When the AI agent later scanned that inbox, it followed the hidden instructions in the message and sent a resignation email instead of drafting an out-of-office reply. Following the security update, however, “agent mode” successfully detected the injection attempt and flagged it to the user, according to the company.

A Problem That Still Needs a Fix

Still, OpenAI admits the problem is far from solved. The firm says it is leaning heavily on large-scale testing and faster patch cycles, while also working with third parties to improve Atlas’ resilience—a process that began even before the browser’s launch.

Not everyone is convinced the risk-reward balance currently makes sense.

“A useful way to reason about risk in AI systems is autonomy multiplied by access,” said Rami McCarthy, principal security researcher at cybersecurity firm Wiz. “Agentic browsers tend to sit in a challenging part of that space: moderate autonomy combined with very high access.”

“Many current recommendations reflect that trade-off,” McCarthy added. “Limiting logged-in access primarily reduces exposure, while requiring review of confirmation requests constrains autonomy.”

Those recommendations are echoed by OpenAI itself, which advises users to limit agent access, require confirmation before sending messages or payments, and give agents specific, narrowly scoped instructions.

“Wide latitude makes it easier for hidden or malicious content to influence the agent, even when safeguards are in place,” OpenAI noted.

For now, OpenAI insists that protecting Atlas users from prompt injection remains a top priority. But as McCarthy points out, skepticism is warranted.

“For most everyday use cases, agentic browsers don’t yet deliver enough value to justify their current risk profile,” he said. “The risk is high given their access to sensitive data like email and payment information. That balance will evolve—but today, the trade-offs are still very real.”