Three tiers ordered by what it costs to get wrong, plus the three AI-specific checks nobody inherits from conventional security practice. Stop partway and you have still covered the expensive part.
- One 2026 study found poisoning an agent's persistent state raised attack success from 24.6% to between 64% and 74%.
A checklist for software you did not fully write. Ordered by what it costs to get wrong, so you can stop partway and still have covered the expensive part.
Tier 1 before anything touches the internet. Tier 2 before it touches anyone else's data. Tier 3 before it touches money or credentials. Stopping early is fine. Skipping to tier 3 is not.
Tier 1 · Before it goes anywhere public
- Search the whole repository for secrets. API keys, tokens, passwords, connection strings. Generated code puts them inline because examples do.
- Check what is in version control. A key removed from the file is still in the history.
- Read every dependency in the manifest. Does each one exist, is it maintained, is the name what you expected? Models invent package names, and people register the ones they invent.
- Confirm debug mode is off and stack traces are not returned to users.
- Check CORS is not wide open. A generated example almost always uses a permissive setting.
Tier 2 · Before it touches anyone else's data
- Every input is validated server-side. Client-side validation is a convenience, not a control.
- Every database query is parameterised. String-concatenated SQL is the single most common generated flaw.
- Output is escaped where it is rendered. Not where it is stored.
- Authentication is checked on every protected route — not on the page that links to it.
- Authorisation is separate from authentication. Logged in is not the same as allowed.
- Rate limits exist on anything that sends, writes or costs money.
- Error messages say less than you think they need to. "Invalid credentials", not "no user with that email".
Tier 3 · Before money, credentials or personal records
- Nothing sensitive is logged. Verbose logging is a data exposure that outlives the system.
- Secrets come from the environment, and the environment is not in the repository.
- Anything irreversible has a human in front of it — see permission tiers.
- Test the restore, not the backup. An untested backup is a belief.
- Have someone who reads code read it. At this tier that is not optional, and no amount of prompting substitutes.
An untested backup is a belief, not a backup.
The AI-specific ones
Fetched content is data, never instruction
If your system reads pages, files, emails or comments and passes them to a model, that text can contain instructions and the model cannot reliably tell the difference. One 2026 study found poisoning an agent's persistent state raised attack success from 24.6% to between 64% and 74%.Wang et al., "Your Agent, Their Asset", arXiv 2604.04759
Read every skill and tool you install
An audit of 3,984 agent skills found 13.4% carried a critical-severity issue and 76 were live malicious payloads.Snyk, "ToxicSkills", 5 Feb 2026 Detail on skills are a supply chain.
Assume the model output is hostile input
If a model's output reaches a shell, a query, a file path or a browser, treat it exactly as you would treat text from a stranger — because functionally it is.
Tools that automate part of this
Some of the checklist can be run rather than read. None of it replaces the tier-three requirement that someone who reads code reads it.
Strix — open-source AI pentesting
Describes itself as autonomous AI agents that find and fix application vulnerabilities. Apache 2.0 — verified against the repository and the project's own site, 22 Aug 2026.github.com/usestrix/strix · strix.ai/open-source-pentesting
The star count is a moving target and worth showing rather than picking one. Reported at 28.9k on 1 July, 32.8k on 3 July, 36k on 5 July, 43k in early August, and above 50k by late August — roughly 4,000 a week.Range across AppSecSanta, CoddyKit, Rohit Raj, Trendshift and the project site, Jun–Aug 2026 Any single figure here would be wrong within a fortnight.
Worth knowing before you run it: one published account pointed it at a live site, found nothing, spent roughly $17 in tokens in ten minutes, and had their API key auto-disabled for the usage spike.Protego, "Strix AI Pentester: An Honest Review", Jul 2026 That is a token economics problem as much as a security one.
What it is good for: catching the common, well-documented classes — the tier-one and tier-two items on this page — faster and more consistently than a manual pass.
What it does not do: understand your business logic. An authorisation flaw where the wrong user can see the right-shaped data is invisible to a scanner and obvious to a person who knows the domain.
And the recursive caution: a security tool driven by an agent is still an agent. It reads your codebase, which is exactly the read-scope question raised in permission tiers. Run it on a copy first.
What this is not
This is not a penetration test and not legal advice. It catches the common, high-impact issues behind most real incidents in software assembled quickly.
If you are handling payments, health data or anything regulated, this checklist is where you start, not where you stop.