What is an AI security audit?
An AI security audit is a structured review of how the AI systems your organization builds or uses could be manipulated, abused, or made to expose data — and what controls stop that from happening.
It borrows from three older disciplines — application security testing, cloud and identity review, and data governance — and adds the threats that only exist because a language model is in the loop. The most important of those is that a model cannot reliably tell the difference between instructions and data. Anything the model reads — a user message, a web page, a PDF, an email, a support ticket — can carry instructions that change what it does next.
Good audits anchor to public frameworks so the findings mean something to auditors and boards: the OWASP Top 10 for LLM Applications for application-level risks, MITRE ATLAS for adversary techniques, and the NIST AI Risk Management Framework for governance.
What does an AI security audit cover?
A complete audit covers nine areas: AI inventory, data flows, prompt injection, agent and tool permissions, retrieval access control, output handling, the AI supply chain, logging and monitoring, and governance.
- Inventory. Every place AI is in use — sanctioned products, features inside SaaS you already pay for, and the “shadow AI” employees signed up for on their own.
- Data flows. What data goes to which model provider, under which contract, with what retention, and whether it can be used for training.
- Prompt injection. Direct attempts by users to override instructions, and indirect injection hidden in documents, web pages, or emails the model processes on someone’s behalf.
- Agent and tool permissions. What the AI can actually do — send email, query databases, call APIs, run code — and whether those permissions are scoped to the user it is acting for. OWASP calls the failure mode “excessive agency.”
- Retrieval (RAG) access control. Whether a chatbot connected to your document store respects the same permissions as the documents themselves, or lets an intern ask for the board minutes.
- Output handling. Whether model output is treated as untrusted before it is rendered in a browser, written to a database, or passed to a shell.
- Supply chain. Third-party models, plugins, MCP servers, and AI SDKs — who maintains them and what they can reach.
- Secrets, logging, and monitoring. API keys out of code and prompts; prompts, tool calls, and refusals logged somewhere you can investigate.
- Governance. An acceptable-use policy people can follow, an owner for each AI system, and a review gate before new AI features ship.
What does an AI security audit not cover?
It does not evaluate model accuracy, bias, or fairness; it is not a legal compliance opinion; and it is not a full penetration test of infrastructure unrelated to the AI system.
Those are all legitimate needs — they are just different work, often by different specialists. Accuracy and bias evaluation is a data-science exercise. Whether a use case is permitted under a specific regulation is a question for counsel. A general pen test of your network may be worth scheduling alongside the AI review, but folding it in hides the AI findings under a pile of unrelated ones.
Be wary of any proposal that claims to do all of it in one fixed-fee package. Scope that broad usually means each part gets a checklist rather than a test.
How is an AI red team different from a penetration test?
A penetration test looks for exploitable flaws in systems and code; an AI red team tries to make the AI itself misbehave — leak data, ignore its instructions, or misuse the tools it has been given.
The techniques overlap but the mindset differs. A pen tester asks “can I get a shell on this server?” An AI red-teamer asks “can I get the support bot to email me another customer’s invoice?” The second question often has a yes answer on systems that pass the first test cleanly, because the model is operating with legitimate credentials the whole time.
A sound AI security audit includes adversarial testing of this kind, but it is broader: it also reviews the design, permissions, and governance that decide how bad a successful attack can get.
Do we need an audit if we only use ChatGPT, Copilot, or Gemini?
Yes, but a lighter one. The risk with off-the-shelf AI is less about prompt injection and more about tenant settings, connected data sources, and what employees paste in.
For a business-tier subscription, the review focuses on: whether data retention and training settings match your obligations; which connectors (email, drive, CRM) are switched on and whose permissions they inherit; whether over-shared files in SharePoint or Drive are now one question away from anyone in the company; and whether you have a short, readable policy telling staff what not to paste.
Microsoft 365 Copilot is the common example. It honors existing file permissions — which is exactly the problem when those permissions were never cleaned up. The fix is usually permissions hygiene, not a Copilot setting.
What should we have ready before an AI security audit?
A list of the AI systems in scope, an architecture sketch for each, the identities and permissions each one uses, your model-provider contracts, and a test environment with realistic data.
None of it needs to be polished. A whiteboard photo of the data flow is fine. The most useful single artifact is an honest answer to “what can this AI do, as whom?” — because that one question decides how bad a successful prompt injection would be.
If you do not have that list, producing it is the first deliverable of the audit, and often the most valuable one.
What do you get at the end of an AI security audit?
A findings report ranked by severity with reproduction steps and specific fixes, an inventory of AI systems and data flows, and a retest once fixes are in place.
Look for findings that name the exact prompt, document, or tool call that caused the problem, and fixes that change architecture or permissions rather than just adding another line to the system prompt. “Tell the model not to do that” is a mitigation, not a control.
For the full picture of how we run these engagements, see the Cybersecurity & IT practice page. If you are weighing whether you need one, a short intro call is usually enough to scope it.
Part of the Llab Technologies Insights series — plain answers to the questions buyers ask us before they hire us.