What Is a Penetration Test and Does Your App Need One?
A penetration test is an assurance check, not a bug hunt. Here is what it covers, how to scope it, and when an app built with AI should pay for one.
A penetration test, or pen test, is an authorised simulated attack on a system, carried out with the tools an adversary would use, to check how well your defences hold. The UK National Cyber Security Centre calls it a method for gaining assurance in the security of an IT system.
What a pen test is, and what it is not
The key word in the NCSC's definition is assurance. It describes a test as a check on your own vulnerability management, comparable to an external financial audit that checks the work your internal finance team already did. It says a test should not be treated as a primary method for identifying vulnerabilities.
That matters for how you use it. The NCSC also points out a limit: a test can only confirm that your systems were free of known issues on the day, and a year or more may pass between tests. So routine security checks still have to happen between them.
A related idea, red teaming in AI, applies similar adversarial thinking to a model rather than to infrastructure.
When an AI-built app should buy one
Most apps do not need a pen test on day one. Some situations change that.
Situation | Reasonable move |
|---|---|
Prototype with no real user data | Run your own checks, no pen test yet |
App stores customer personal data | Self-check first, then consider a test before launch |
A client's security team asks for a report | Buy one, since it is a procurement requirement |
You are pursuing a compliance certificate | Check the standard, many expect independent testing |
Major change to login, payments or permissions | Test the changed area |
If a client asks for evidence, getting an AI product through a client security review shows what they usually want. For certificates, see SOC 2 for an AI product.
How to scope one
The NCSC says scoping should involve the people who own the risk, the technical staff who know the target, and a representative of the test team. The result should be a plan that covers:
the technical boundaries and test types
timeframe and effort, usually counted in days of work
scenarios or use cases, if relevant
preparation you need to do
compliance and reporting needs
constraints, such as testing out of hours
Share your current vulnerability assessment with the testers if you have one. If part of your system is unusual, say so when you ask for quotes so the provider can staff it properly.
Choosing a provider
Test quality depends heavily on the skill of the people doing it, so ask about their qualifications and experience. The NCSC recommends its CHECK scheme for UK government organisations. Outside that, ask for sample reports and who exactly will do the work.
What the report should contain
Expect the issues found, the testers' risk rating for each, a method to resolve each one, an opinion on how accurate your own vulnerability assessment was, and advice on improving that process. The NCSC stresses that the risk assessment and the decision to apply fixes are yours. Do not argue findings down because they are inconvenient.
What to do before you pay
Run the cheap checks first. Review your permissions, secrets and dependencies, and consider whether AI can find security vulnerabilities in your code as a pre-test pass. A tester's time is better spent on the hard problems than on a hard-coded key. The broader map sits in the AI risk overview.
FAQ
What is the difference between a pen test and a vulnerability scan?
A scan looks for known weaknesses with automated tools. A pen test simulates an attacker, and the NCSC frames it as a way to gain assurance about your security, not as the main way to find vulnerabilities.
How often should you run a penetration test?
The NCSC notes a year or more may pass between tests, so keep routine security testing running in between. A major change to login or payments is a sensible trigger.
How much does a penetration test cost?
It depends on scope and provider. The NCSC describes effort in days of work, so a smaller scope costs less. Ask for quotes against the same written scope.
Can AI do a penetration test for me?
AI can help with checks before a test. A formal test for a client or a certificate normally still needs an independent provider.
How did this land?
About the author
Senior Editor, AI & Product
Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.
