Large language models in operational roles query telemetry, propose configuration changes, and in some deployments execute those changes against live infrastructure. Ticket drafting and alert summarization were the starting point. Vendors describe this work as autonomous remediation or self-healing infrastructure. A recent survey on agentic AI in network and IT operations gives it a more useful name: a confused-deputy problem waiting to happen.

The shift matters because the agent is not merely advising a human operator. It sits inside change-management APIs, deployment pipelines, and network controllers. It reads tickets, runbooks, chat transcripts, and log entries. Those are the same artifacts an attacker can influence. The result is a security model in which compromising the tool is unnecessary: an attacker can compromise the text the agent reads before it uses the tool.

Key facts

• LLM operations agents query telemetry, propose configuration changes, and in some deployments execute changes against live infrastructure.
• Vendors market this as autonomous remediation or self-healing infrastructure.
• A recent survey describes agentic operations as a confused-deputy problem waiting to happen.
• Attack categories include prompt injection through operational artifacts, retrieval poisoning, retrieval jamming, and telemetry manipulation.
• Defense: strict propose-commit split, non-bypassable gate, policy-as-code, invariant verification, human approval, staged rollback, integrity-protected audit logs.
• Prompt-only defenses are brittle; an excessive-agency pattern reflects failure to implement the split cleanly.
• Evaluation should report tool-call traces, gate-violation rates, adversarial behavior, refusal-storm rates, rollback completeness.

The confused-deputy problem in agentic AI security

The classic confused-deputy attack tricks an authorized program into misusing its privileges. The program is not malicious. It is confused about who it is acting for. Agentic operations create an ideal substrate for this kind of abuse. The agent holds legitimate access to powerful systems. Its decisions are shaped by operational text that may be untrusted, stale, or deliberately manipulated.

Traditional privilege escalation tries to steal credentials or exploit a vulnerability. A confused-deputy attack against an AI operations agent can be quieter. The agent already has the credentials. It already has the API access. The attacker only needs to shape the context in which the agent decides to use them. A ticket can contain a hidden instruction. A wiki page can contain a poisoned procedure. A log entry can be altered. A metric can be skewed. The agent then acts with authority it legitimately possesses, but for a purpose the attacker chose.

Four attack categories targeting LLM operations

The survey catalogs several attack categories that deserve more attention. They differ in technique, but they share a common property: they target the agent's inputs rather than the agent's code.

Prompt injection through operational artifacts

Prompt injection through operational artifacts is the most familiar. A malicious instruction is embedded in a ticket, a wiki page, a runbook, or a chat message. The agent treats the text as operational context and follows the instruction. The result can be an unsafe action: a firewall rule change, a credential rotation, a deployment, or a data deletion. The attack surface is not a web form. It is the everyday paperwork of IT operations.

Retrieval poisoning

Retrieval poisoning corrupts the runbooks and incident histories the agent consults. If the agent retrieves a poisoned document, its diagnosis can be biased toward an attacker-chosen conclusion. The agent may then propose a mitigation that looks reasonable given the corrupted evidence. The human reviewer sees a coherent recommendation, not a visible attack.

Retrieval jamming

Retrieval jamming works in the opposite direction. It floods the knowledge base with blocker documents that trigger refusal loops. The agent repeatedly finds conflicting or incomplete guidance. Incident response stalls when it is most needed. This is a denial-of-service attack against the agent's reasoning process, not against a network port.

Telemetry manipulation

Telemetry manipulation works against LLM-driven operations agents. An attacker who can influence what metrics and logs say can steer mitigation decisions without touching the model. If the agent sees a false disk-full alert, it may expand storage. If it sees a false healthy signal, it may delay action. The model remains unchanged, but its view of reality is controlled.

These attacks are operationally dangerous because they do not look like attacks. They look like normal incident response that happens to go wrong. A ticket with a strange instruction can look like a rushed request. A poisoned runbook can look like outdated documentation. A refusal loop can look like an ambiguous incident. A skewed metric can look like a sensor issue. By the time the mistake is visible, the change may already be applied.

Why operational context raises the stakes

IT operations are a high-trust environment. Agents are often given broad access because narrow access is impractical. They may need to read logs across services, query configuration management databases, open pull requests, trigger pipelines, and call APIs. Each of those permissions is reasonable in isolation. Together they create a large blast radius.

The pressure of incident response makes the problem worse. When systems are down, operators want speed. An agent that can draft a fix quickly is attractive. An agent that can apply the fix without waiting for a human is even more attractive. But speed compresses the time available for verification. If the agent's input has been poisoned, speed turns a bad recommendation into an outage.

Agentic workflows also chain actions. An agent may read a ticket, retrieve a runbook, query metrics, propose a change, and then execute a sequence of API calls. Each step can be influenced. A small manipulation early in the chain can produce a large effect later. This is why security teams cannot evaluate only the model's final output. They must examine the tool-call trace, the retrieved evidence, and the gate decisions along the way.

The propose-commit split as an architectural defense

The defense proposed by the survey is architectural. The authors argue for a strict propose-commit split: the language model can reason, retrieve evidence, and draft change proposals, and it cannot execute writes. Every action that touches production passes through a non-bypassable gate the model has no authority over.

The gate covers policy-as-code checks, invariant verification, human approval for high-blast-radius changes, and rollback-ready staged deployment. The model's job is to draft a diff. The gate's job is to decide whether that diff is allowed to apply. This separation is not a prompt. It is an enforcement boundary outside the model's control.

The pattern resembles established engineering controls. Infrastructure as code uses pull requests and review. GitOps uses a controller to reconcile desired state. Policy-as-code uses deterministic rules to block noncompliant resources. The propose-commit split applies the same logic to AI-driven operations. The model can suggest. The gate can reject. The production environment only sees changes that pass the gate.

Audit logs that are integrity-protected round out the control set. Post-incident forensics need to reconstruct what happened. Which ticket did the agent read? Which runbook did it retrieve? Which metrics did it see? What did it propose? Which gate evaluated the proposal? Was the gate bypassed? Without tamper-evident logs, those questions may be impossible to answer.

The limits of prompt-based agentic AI security

This architecture matters because prompt-only defenses are brittle. Any system where the model's text generation can directly cause production changes has built its security perimeter inside the most unpredictable component in the stack. Prompt injection defenses can reduce risk, but they cannot be the last line of defense. The model is probabilistic. Its context is open-ended. Its inputs are not fully controlled.

An excessive-agency pattern is in practice a failure to implement the propose-commit split cleanly. Excessive agency is not only about giving an agent too many tools. It is about giving the agent the ability to commit changes without an independent gate. Least privilege matters, but least privilege alone is insufficient when the agent's legitimate privileges are broad enough to cause harm.

Security teams should treat the model as a proposal engine, not as a production authority. That framing changes procurement questions. Does the product allow the model to write directly to production? Can the gate be bypassed by configuration? Are emergency break-glass paths audited? Can a high-blast-radius change proceed without human approval? Are rollbacks automatic and verified? The answers should be architectural, not marketing claims.

The missing evidence for safe LLM autonomy

A measurement problem sits alongside the architectural one. Many claims about safe agentic operations cannot be falsified because the supporting evidence is missing. The survey identifies what evaluations should report: tool-call traces, gate-violation rates, behavior under adversarial inputs, refusal-storm rates under jamming attacks, and rollback completeness. Most current benchmarks omit these.

Success metrics on benign workloads are not enough. A system that performs well on clean incidents may collapse the moment someone embeds a hostile instruction in a ticket. A system that resolves alerts quickly may also be a system that bypasses verification quickly. A system that reports high accuracy may not report how often it tried to violate policy, how often it was stopped, or how often it recovered.

Security teams evaluating agentic products should ask for adversarial evaluation data alongside success metrics on benign workloads. They should ask for traces, not summaries. They should ask for gate-violation rates, not just incident-resolution rates. They should ask how the system behaves when the knowledge base is poisoned, when telemetry is lying, and when an attacker floods the retrieval layer with conflicting documents. If vendors cannot provide that evidence, the claim of safe autonomy is not yet testable.

Where autonomy earns trust and where it does not

The amount of autonomy an agent has is the amount of damage it can do when things go sideways. Read-only assistance is useful and low-risk. An agent that summarizes alerts, retrieves runbooks, and suggests next steps can reduce toil without holding the keys to production. Bounded execution with strong gates is defensible. An agent that drafts a change and submits it to an independent verification pipeline can be useful if the gate is real and the rollback is reliable.

Open-ended self-healing across large production environments, without the verification scaffolding the survey describes, is a harder problem than current deployments make it sound. Claims about it deserve skepticism. The gap between a demo and a production control is often the gap between a proposal and a commit. The propose-commit split makes that gap explicit. It forces vendors and operators to show where the model stops and where accountability begins.

Agentic AI in operations will continue to expand because the economics are compelling. Incident response is expensive. Toil is expensive. Downtime is expensive. But the security model must match the authority granted. If an AI assistant has the keys to production, the organization needs to know exactly which doors it can open, which doors require a second key, and what happens when someone slips a false instruction under the door. The operative question is not whether an agent can act. It is what prevents it from acting when its inputs have been poisoned.

Source: Help Net Security News