How much AI-generated code is insecure, and what to do about it before merge

In July 2025 an application security vendor gave 80 coding tasks to more than 100 large language models and checked the output for security defects. Models introduced an OWASP Top 10 vulnerability in 45% of cases. Java was the worst language at a failure rate over 70%. Python, C# and JavaScript sat between 38% and 45%. The code compiled and did what the prompt asked. It was also, almost half the time, exploitable.

The failure modes are not new. In the same study the models failed to protect against cross-site scripting (CWE-80) in 86% of cases and log injection (CWE-117) in 88%. These are the bugs the industry has been writing about for twenty years. The model learned from the same corpus that contains them, so it reproduces them at the rate the corpus does.

Meanwhile the volume is going up. GitHub's Octoverse 2025 counted 43.2 million merged pull requests a month on average, up 23% year over year, and reported that nearly 80% of new developers on the platform use GitHub Copilot in their first week. The question is no longer whether AI writes your code. It is what you check before that code merges.

#The trust gap is already measured

The DORA 2025 report, built on nearly 5,000 responses, put AI adoption among software professionals at 90%, up 14% year over year. The same report found that only 24% of respondents have "a lot" or "a great deal" of trust in AI output, and 30% trust it a little or not at all.

That is a reasonable position. The 45% figure above says a reviewer should expect a security defect in roughly every other security-relevant change a model writes. A reviewer who trusts the output less than the output deserves is not being paranoid. They are being accurate.

The problem is that distrust does not scale. A human cannot trace every data flow in a 400-line generated diff at the speed the diff arrives. Something deterministic has to do that part, and it has to give the same answer on the same code every time, or the reviewer is back to guessing which of two runs to believe.

#The bugs are the ones at the top of the CWE list

The 2025 CWE Top 25, built from 39,080 CVE records, ranks cross-site scripting (CWE-79) first, SQL injection (CWE-89) second, cross-site request forgery (CWE-352) third and missing authorization (CWE-862) fourth. Compare that with what the models got wrong: XSS at 86%, log injection at 88%, with injection and authorization gaps throughout.

This matters for tooling choice. These are the classes that static analysis with data-flow tracking handles well, because each one is a value travelling from an untrusted source to a sensitive sink without a sanitizer in between. A request parameter reaches a template without escaping. A request parameter reaches a query string by concatenation. A handler reads a record by id without checking who asked. The shape is the same each time and the path can be traced.

What the generated code typically lacks is not the function that would fix it. The escaping helper, the parameter binding, the ownership check all exist in the framework. The model simply did not call them, because the examples it learned from often did not either.

#What "same code, same answer" buys you

Vulkro has no model in the detection path. A finding comes from parsing the code, building the data flow and matching a traced path against a rule. Run it twice and you get the same findings, in the same order, with the same severity. That is the property you want when the author of the code is probabilistic: a second probabilistic opinion on top of it compounds the uncertainty rather than reducing it.

The cost of that approach is that it has limits, and we publish them. On the blind holdout for vulkro 0.36.0, measured 2026-10-19, Vulkro found 181 of 229 planted vulnerabilities (79%) and flagged 39 of 152 known-safe controls at Medium or above. The run used nine applications the detectors had never seen, with VULKRO_OFFLINE=1, VULKRO_DISABLE_CACHE=1 and VULKRO_DETERMINISTIC=1 set so the result is reproducible. Forty-eight planted bugs were missed. A scanner is one layer, not a replacement for review.

What the scanner does is remove the mechanical part of review. If the traced flow from req.query.name to the rendered template is shown as a list of hops, the reviewer's job is to read the hops, not to rediscover them.

#A pre-merge workflow for agent-written branches

Two commands do most of the work. The first is vulkro review, which scans the working tree against a base ref and tags each finding as NEW or existing. Only NEW findings count toward the exit code, and the default threshold is --fail-on low, so any real new finding fails the run. Documentation: vulkro review.

# On the branch the agent wrote, before opening the PR
vulkro review

The second is vulkro scan with the flags that exist for this situation. --ai-pr calibrates the scan for AI-generated pull request diffs. --ai-code-segregation emits a report alongside the normal output that breaks findings down by the AI tool fingerprint found in the commit. --attest-reviewed --reviewer <name> records that a human has reviewed the AI-touched code, as an evidence row on each finding in those files. Documentation: vulkro scan.

# In CI, on a PR a coding agent opened
vulkro scan . --ai-pr --ai-code-segregation

# After a human has read the diff and signed off
vulkro scan . --attest-reviewed --reviewer alice

The attestation does not change what the scanner finds. It records who looked, so that six months later the question "did anyone read this before it merged?" has an answer in the evidence rather than in someone's memory.

Exit codes follow the usual contract: 0 for a clean result, 1 when findings are reported, 2 for an error. A CI step can act on that without parsing output.

#What this does not solve

A scanner that traces data flow is good at injection and poor at intent. If the agent implemented a password reset that emails the token to an address the user supplied in the same request, no sink is reached and no rule fires. The logic is wrong, not the plumbing. That still needs a person who understands what the feature was supposed to do.

It also cannot tell you whether the 24% who trust AI output are right. It can only make sure that the code they merge has been checked for the things we know how to check, at a rate we have measured and published.

#What to do this week

• Run vulkro review on the next branch an agent writes and read the NEW findings before the PR is opened.
• Add vulkro scan . --ai-pr to the CI lane that runs on pull requests and watch what the segregation report attributes to generated code.
• Pick one reviewer per team to run --attest-reviewed on merged AI diffs so the evidence row exists.
• Read the benchmark page and decide whether 79% found and 26% of safe controls flagged is a trade you accept. If not, tell us what you need.

Vulkro Core comes with a 14-day trial. Start it here and run vulkro review on an agent-written branch today.