Cisco has taken a significant step to standardize the evaluation of agentic AI in cybersecurity by releasing its internally developed Foundry Security Spec to the open-source community on GitHub. The spec is designed to work with GitHub’s spec-kit, an industry-standard set of development workflows that can be tailored to different AI agents. The goal is to provide a common framework for evaluating and governing AI agents used in cybersecurity, moving beyond the current ad-hoc practices that often mix genuine insights with hallucinated findings.
Why a security spec for AI agents?
Agentic AI—systems that act autonomously to achieve goals—is rapidly transforming cybersecurity. However, current methods of using large language models (LLMs) for vulnerability detection often produce unverifiable results. Security teams typically throw a report at a frontier model and ask it to find bugs, but the output is often a wall of unbounded, unverifiable data. The Foundry Security Spec offers a structured alternative by wrapping the LLM in orchestration, roles, and guardrails. This ensures that detection, validation, and coverage are designed upfront rather than improvised in a chat window.
Anthony Grieco, Cisco’s senior vice president and chief security officer, emphasized the collaborative nature of cybersecurity. In a prerecorded video, he stated that the industry must come together for better collective defense. He noted that while frontier models like Anthropic’s Mythos and OpenAI’s GPT-5.5-Cyber can identify vulnerabilities at machine speed, most security teams lack the manpower or processes to verify findings. Foundry addresses this gap by providing a stable harness that can evaluate any model, making it model-agnostic.
Components of the Foundry Security Spec
The Foundry Security Spec is published as two main artifacts and a set of supporting documents. The first artifact, the “spec,” includes eight core agent roles: orchestrator, indexer, cartographer, detector, and others, plus five extension roles. It defines the finding lifecycle, the coordination substrate, and roughly 130 functional requirements, each with an inline rationale. The second artifact, the “constitution,” contains 11 firmly defined principles. Each principle encodes a real production failure that Cisco shipped, diagnosed, and fixed. These principles serve as guardrails that assume the model will eventually try to do the wrong thing and constrain it at the substrate level, not just through prompting.
Omar Santos, a distinguished engineer at Cisco focusing on AI security, wrote in a blog post that the difference between a raw LLM demo and a structured evaluation system is stark. The Foundry Security Spec produces a bounded, prioritized, verifiable set of findings, a clear “done” signal tied to coverage and yield thresholds, and an auditable provenance chain from detection through triage to publication. This makes it defensible in front of a CISO and auditors.
Relation to Project CodeGuard
The Foundry specification works hand-in-hand with Project CodeGuard, another Cisco-contributed open-source technology. CodeGuard is a security framework that builds secure-by-default rules into AI coding workflows. It offers a community-driven ruleset, translators for popular AI coding agents, and validators to enforce security automatically. The integration covers the entire AI coding lifecycle—from design and planning to code generation and post-generation review. Santos explained that rules can steer models toward secure patterns from the start, prevent issues during code generation, and help agents like Cursor, GitHub Copilot, Codex, Windsurf, and Claude Code review code.
Historical context and broader impact
Cisco’s move is part of a broader industry trend to formalize AI security. As agentic AI becomes more prevalent, the need for standardized evaluation frameworks grows. The Foundry Security Spec is designed to be future-proof, as it is built on functional requirements and roles rather than specific model parameters. Whether organizations use today’s frontier models or tomorrow’s more complex reasoning agents, the need for an orchestrator, detector, and validator will remain constant. This stability is crucial for security teams that must maintain consistent evaluation processes over time.
The open-sourcing of Foundry also reflects Cisco’s long-standing commitment to open platforms and community collaboration. By sharing this spec, Cisco hopes to raise the bar for everyone and break the cycle of ad-hoc AI testing. The company believes that a collective defense requires shared tools and knowledge.
In practice, the spec addresses a common pain point: security teams often struggle to know when they are done with an evaluation. The spec provides an economic yield threshold that, combined with an operator-defined coverage floor, signals completion. Additionally, the auditable provenance chain ensures that every finding can be traced back through detection, triage, validation, and publication. This transparency is vital for compliance and for building trust in AI-driven security tools.
Grieco noted that users do not have to wait for specific frontier LLM access to benefit from the spec; it works with any model. The protective software infrastructure surrounding an AI model—the harness—is what matters most.
As AI agents become more autonomous, the risk of unintended behavior grows. The Foundry Security Spec’s constitution encodes guardrails that assume the model will attempt to violate boundaries. By constraining at the substrate level, the spec prevents many common failure modes. This approach is more robust than relying solely on prompt engineering.
Ultimately, the Foundry Security Spec represents a maturation of AI security evaluation. It moves the industry from an experimental, demo-driven approach to a systematic, auditable practice. Cisco’s decision to open-source it encourages widespread adoption and community contributions, which in turn can help refine and extend the spec. The result is a more secure ecosystem for AI agents in cybersecurity, benefiting vendors, enterprises, and the broader internet community.
Source: Network World News