Buy an AI security assessment when a customer or an auditor has asked whether the AI you are about to ship is safe, and a network penetration test will not answer them. The fixed-price way to test that layer is the AI Security Assessment: from $12,000, over 2–4 weeks. It threat-models and red-teams the agents, large language models, and RAG pipelines you already run, then returns findings graded by severity and a remediation plan. A pilot that dies in a security review is a different article — the five gates that stop a pilot. This post is one of those gates: what the assessment covers, what a pentest leaves out, and what you can hand the reviewer.
A penetration test leaves the AI path untested
A traditional penetration test covers network, application, and infrastructure vulnerabilities. It finds an open port, a broken login, a missing patch. It does not walk an untrusted sentence through a model, into a tool, and out as a business action. Prompt injection, jailbreaks, RAG poisoning, tool-use abuse, and system-prompt leakage sit on that path. A clean pentest report can be true and still say nothing about them.
Most teams shipping AI need both tests. The pentest still belongs on the network and the app. The assessment belongs on the AI layer and the application the model can reach. Treating the pentest PDF as coverage of the model is how a feature ships, then fails the questionnaire that asks how you tested prompt injection.
The failure mode is a questionnaire row marked done because a pentest PDF exists. The reviewer who knows the OWASP LLM Top 10 asks for the path from untrusted input to a tool call. Without that path, the launch waits.
Six layers, and the model is only the first
An AI feature inherits a new set of boundaries around the product you already run. Score all six. A finding on layer one with no look at layer three is a model demo, not an assessment.
| Layer | What has to hold |
|---|---|
| LLMs and prompts | Jailbreaks, system-prompt leakage, insecure output handling, and the guardrails that have to hold when a user gets creative |
| RAG pipelines | Poisoning, context manipulation, data leakage, and whether tenant identity survives every retrieval and inference boundary |
| AI agents and tool use | Agent identity, least-privilege authorisation, scoped credentials, tool allowlists, and a human checkpoint before a high-stakes action |
| Data and PHI | Every path patient or customer data takes, including model APIs and tracing tools that capture it in logs |
| AI supply chain | Model provenance, fine-tuning data, embeddings, and third-party dependencies a tampered component can enter through |
| Cloud, secrets, and identity | Secrets handling, API permissions, MCP servers, and the blast radius when one credential leaks |
Name only the model vendor and you have scoped one layer. The assessment starts from the other five: what is deployed, which systems it touches, and what done means for this review.
The tests follow one real execution path
Adversarial testing here means one path, repeated until it breaks or it holds: untrusted input, then model context, then an agent decision, then a tool, then a business action. The cases below are the ones every deployment shares. They are derived from the OWASP LLM Top 10 and then aimed at your system, not scored as a generic benchmark.
- Prompt injection, direct and indirect. A user types an instruction, or a document, email, or web page the model reads hides one. The model obeys it. In healthcare the high-risk variant is an instruction sitting inside a clinical note.
- Sensitive-data disclosure. The model returns a secret, an internal detail, or another tenant's data.
- RAG and embedding poisoning. Tampered content in the knowledge base bends what the system retrieves, or carries data out through the retrieval path.
- Excessive agency and tool misuse. An agent with more access than the task needs is talked into turning "summarise this ticket" into "export the customer table and send it."
- System-prompt leakage. The hidden prompt — rules, logic, sometimes keys — is coaxed out, and the attacker has a map of the guardrails.
- Insecure output handling. Model output goes straight into a browser, a shell, or a database, and a response that carries a script or a query becomes an ordinary injection in your own app.
- Model and supply-chain risk. A backdoored model, a compromised dependency, or a third-party integration is reviewed for provenance and for what it can execute in production.
- Cross-tenant isolation. A user authenticated for one tenant reaches another's data once the request crosses an embedding, an inference call, an agent tool, or a downstream API.
Authorisation decides whether an intended action may run. An assessment that stops at the sentence, and never calls the tool, misses the step where the damage happens.
What you hand the reviewer
The deliverable is a graded finding list and a prioritised remediation plan. If you want the fixes implemented, the same team that builds these systems can do that work. The assessment itself is the test and the plan. Implementation is a separate decision, named before it starts.
Reviewers ask for evidence, not a slogan. For healthcare the work is HIPAA-aware: data stays on your infrastructure, and a BAA is signed when the review requires one. For SaaS the work is on a SOC 2 path: audit logging, access control, and a trail a customer security questionnaire can point at. Neither phrase is a certification. We do not claim a HIPAA certification or a SOC 2 report from this engagement. We test and document to the controls the review expects, and we say what is actually in place.
The trail on an agent is four fields: input, tools called, data accessed, output. A reviewer who challenges one decision should be able to replay it. Alignment language on the report maps findings to the frameworks auditors cite — the OWASP LLM Top 10, NIST AI RMF, ISO 42001, and EU AI Act expectations. "Aligned with" means the findings are labelled in that vocabulary. It does not mean those bodies certified the system.
A published build shows what a pre-launch security pass looks like when it is not an AI red team. In month 5 of a telehealth platform, a third-party security assessment produced two medium findings, and both were remediated before launch. That was application and infrastructure security on a HIPAA platform. It was not a test of prompt injection or agent tool use. Use it as the pattern — findings graded, closed before go-live — and buy the AI-layer test separately when the system has a model in the path.
What the $12,000 buys, and what stays on a retainer
The AI Security Assessment is from $12,000, fixed, over 2–4 weeks. Larger or multi-system scopes are quoted before work starts. The number is named up front. There is no open meter hiding behind the floor.
Inside that window: a scoping call, an attack-surface map, adversarial testing along the real path, and a graded report with a remediation plan. Runtime guardrails, eval regressions, and monitoring sit on Managed AI Operations at $3,000–$20,000 a month. Prompts, tools, and models change after the report. The assessment fee does not include that retainer.
An unpatched host is a pentest. An agent that can call tools is this assessment. Where identity verification or specialist tooling is the right control, integrate it and say so.
What to do this week
1. List what is deployed. Models, agents, retrieval stores, and the systems they can reach. If the list is one vendor name, you have not scoped the work. 2. Mark the path. Any place untrusted text — a user, a document, an email, a web page — can become a tool call or a write. That path is the test. 3. Read the pentest against that path. If the report never mentions prompt injection, retrieval, or tool use, it does not cover this review. 4. Ask which control the reviewer will cite. OWASP LLM Top 10, a SOC 2 control, a HIPAA safeguard, or a customer questionnaire row. The report should use that label. 5. Price the test. The AI Security Assessment is from $12,000, fixed, over 2–4 weeks. Put monitoring on Managed AI Operations at $3,000–$20,000 a month after the findings are closed.
A penetration test answers a different question. An AI security assessment answers whether the model, the retrieval layer, and the tools can be talked into doing something your policy forbids — and it answers it at a fixed price, from $12,000. The telehealth proof shows the shape of a pre-launch pass: two medium findings, both remediated before launch. That pass was not an AI red team. When the system you are shipping has an agent or a RAG pipeline in it, start with the AI Security Assessment.
