Written with contributions from Bert Amodol and Jason Santamaria.
Turning every conversation into a word salad of acronyms wasn’t bad enough. Now, we’re taking words that have one meaning and assigning a different meaning. This must stop!
First, it was “agent.” This always meant a piece of software that was installed on a computer. Now it means an autonomous system that can take goals, plan steps and carry out actions across multiple systems. Second is “Governance.” Most organizations are having two completely different conversations about AI governance and don't realize it.
When a CISO says "AI governance," they usually mean: how do we secure AI systems from attack? When a GRC team says, "AI governance," they usually mean: how do we ensure AI is used responsibly, ethically, and in compliance with regulations?
Both are legitimate. Both are necessary. And they are not the same thing.
Conflating them leads to a predictable failure: organizations build a governance policy document, check a compliance box, and assume they're covered, while leaving their AI systems technically exposed. Or, they lock down AI infrastructure with security controls but have no framework for deciding what AI should and shouldn't be allowed to do in the first place.
You need both. They just work at different layers.
AI Governance from a GRC Perspective
GRC-focused AI governance is about accountability, risk management, and regulatory alignment. It answers the question: are we using AI responsibly, and can we prove it?
This layer includes:
- Policy and accountability frameworks: Who owns AI decisions? Who is responsible when an AI system produces a harmful or incorrect output? What is the escalation path?
- Regulatory compliance: The EU AI Act categorizes AI systems by risk level and mandates documentation, human oversight, and transparency for high-risk applications. NIST's AI Risk Management Framework (AI RMF) provides a voluntary but widely adopted structure for managing AI risk across the organization.
- Bias, fairness, and ethics: Are AI models producing discriminatory outputs? Are training datasets representative? Are decisions explainable to affected parties?
- Third-party AI risk: What AI is embedded in your SaaS tools, vendor platforms, and supply chain? Do you know where AI is making decisions on your behalf?
- Data governance: What data feeds your AI models? Is it classified, retained appropriately, and used in compliance with privacy regulations like GDPR or HIPAA?
This is governance in the traditional sense: policies, frameworks, risk registers, audit trails, and human accountability for automated decisions.
AI Governance from a Security Control Perspective
Security-focused AI governance asks a different question: can we trust that our AI systems behave the way we intend, and that adversaries can't manipulate them?
This is where technical controls come in:
- Prompt injection and jailbreaking: Attackers craft inputs designed to override an AI system's instructions and make it behave in unintended ways. Without input validation and output filtering controls, your AI is exploitable.
- Model poisoning: If adversaries can influence the data used to train or fine-tune a model, they can embed backdoors that activate under specific conditions.
- Data exfiltration via AI: AI systems with access to sensitive data can become unintentional data leakage vectors. Users or attackers can extract information the model was trained on through carefully crafted queries.
- Supply chain risk in AI components: Pre-trained models, third-party APIs, and open-source libraries all introduce model supply chain risk that most organizations aren't scanning for.
- Logging and observability: Do you have audit logs for AI interactions? Can you reconstruct what a model was asked and what it produced during an incident?
- Access control around AI systems: Who can query your AI? Who can modify models or training data? Are these governed like any other privileged system?
These are security engineering problems. They live in your threat model, your SIEM, your vulnerability management program, not in your compliance spreadsheet.
Why the Distinction Matters
Here is where organizations fail: the GRC team builds an AI use policy and the security team adds AI to the asset inventory, and both groups believe AI governance is handled.
It isn't. Policies don't prevent prompt injection. Compliance frameworks don't catch model poisoning. And security controls alone don't ensure that an AI making hiring or lending decisions is producing fair, auditable, legally defensible outcomes.
Governance without controls is aspiration. Controls without governance is a technical fix to an organizational problem. A mature AI program requires both, running in parallel and informing each other.
What a Mature AI Governance Program Looks Like
Organizations that get this right treat AI governance as two parallel workstreams that feed each other:
- GRC workstream: Inventory all AI systems (including shadow AI), classify them by risk level, map to applicable regulations (EU AI Act, NIST AI RMF, state-level laws), establish ownership and accountability, and build review processes for AI decisions that affect people.
- Security workstream: Threat model AI systems like any other critical asset, apply input and output controls, establish logging and monitoring for AI interactions, test for adversarial manipulation, and govern access to models and training data.
- Bring them together: Security findings should inform governance risk registers. Governance policies should drive security control requirements. The two teams should be reviewing AI incidents together, not in isolation.
Final Thoughts
AI governance isn’t a single conversation. It is two parallel programs that need to be running simultaneously and talking to each other.
The organizations that will navigate this well are the ones who stop treating "AI governance" as one thing, and start recognizing it as two distinct disciplines with different owners, different tools, and different definitions of success.
Get both right and you have a program that is defensible from a compliance standpoint and resilient from a security standpoint.
That is the goal. Anything less is only half the job.
Shameless Marketing Information
Gotham Technology Group helps organizations build AI governance programs that span both GRC and security, from AI risk assessments and policy frameworks to technical controls for AI systems.
Our complimentary AI Readiness and AI Governance Health Check assessments can help your organization gain valuable insight into current AI usage, and our AI Governance Readiness Professional Services offering provides you with a clear path towards meeting your AI governance goals. Reach out to your Gotham Account Manager today to learn how we can help.