Reviewing AI pull requests: a checklist for the human who approves them

The volume is real. GitHub's Octoverse 2025 counts 43.2 million merged pull requests a month, up 23% in a year, and nearly a billion commits, up 25.1%. Eighty percent of new developers on the platform use GitHub Copilot in their first week. A growing share of the diffs landing in your queue were typed by a model.

The trust has not kept pace. The 2025 DORA report links AI adoption to higher throughput and finds that 59% of respondents see a positive influence on code quality, yet 30% trust AI output a little or not at all. DORA calls it the trust paradox. The reason is not hard to find: an application security vendor's 2025 study found 45% of AI-generated code samples insecure, with 86% failing on cross-site scripting and 88% on log injection.

So the reviewer is the bottleneck, and the reviewer is tired. This post is a checklist for the person who has to click approve, followed by the parts of that checklist a machine should do before the human opens the diff.

#Five things to read in every AI pull request

New sinks. Any line where data leaves your control: a SQL string, a shell command, a file path, an outbound URL, an HTML template, a log call. Agents reach for string concatenation because it is the shortest token path to a passing test. Ask where each argument came from.

New dependencies. A model that cannot find a function will sometimes invent a package that provides it. Check every added manifest line against the registry, and look at the publish date and maintainer of anything you do not recognise. Also read lockfile changes; an "unrelated" lockfile churn is where version pins get loosened.

Auth boundaries. Did the change add a route, a handler, a server action, a webhook or a tool? Every one of those is an entry point. Confirm it sits behind the same middleware as its neighbours, and that any object it loads by id is tied to the caller.

Removed checks. Agents fix failing tests by deleting the assertion, and fix a 403 by deleting the guard. Diff hunks that remove an if, a decorator, a try, or a validation call deserve more attention than the hunks that add code.

Scope. Compare what the PR description says with what the diff touches. A fix for a date-formatting bug that also edits the session module is not a fix for a date-formatting bug.

That is about twenty minutes of careful reading per PR. At the volumes above, you will not do it every time. The next sections are about which parts to hand off.

#Scan the change, and label what is actually new

The first thing to automate is the question "did this PR introduce anything, or is it inheriting the repo's existing debt?" A full scan on every PR produces the same hundred findings every time, and reviewers stop reading on day three.

vulkro review scans a commit range and tags each finding NEW or existing by fingerprint against a baseline ref:

vulkro review --range origin/main...HEAD --baseline origin/main

--fail-on defaults to low, so any NEW finding fails the check by default; raise it to critical,high if your team prefers to see Medium findings as comments rather than failures. One rule protects reviewers from noise: a NEW finding whose disposition is not-examined (a shape the scanner flagged but could not decide, typically an authorization question) never fails the check unless you set VULKRO_GATE_SEVERITY_ONLY=1. It is still reported, so the human sees it, but it does not block on a question the machine admits it cannot answer.

For a quick local pass on the agent's last commit, vulkro review --last-commit does the same for one commit.

#Catch the hallucinated package before it is installed

The dependency item on the checklist is mechanical and should never be done by eye. vulkro slopcheck reads a manifest or a pasted list of package names and flags names that do not exist in the registry as High (the hallucinated package, which an attacker can register tomorrow) and names within an edit distance of a popular package as Medium (the typosquat). Run it on the manifest diff of any PR that adds a dependency, before npm install or pip install runs anywhere.

#Changed lines are not the whole change

Scanning only the lines a PR touched is fast, and it misses one important case: the PR adds a new caller to an old, unsafe function. The sink has existed for a year and was unreachable from any request. The new three-line handler makes it reachable. A diff-scoped scan sees three harmless lines.

vulkro gate --base origin/main scans the whole working tree and the whole base, then reports only the findings that are new in the comparison. Because both sides are complete, the newly reachable old sink shows up as a new finding. It costs more than a changed-lines scan, so a reasonable layout is review on every push and gate on the merge check.

#Verify the fix before it is applied

When the scan finds something, the agent will offer a fix. The reviewer's instinct is to trust a fix that looks right. A better instinct is to check it the same way the bug was found.

The Vulkro MCP server exposes verify_fix, which applies a proposed diff to a temporary copy of the project, re-scans it and returns one of three verdicts: fixed, not-fixed or regressed. suggest_fixes works from the other direction and returns patches that already carry the same verdict. An agent wired to these tools can loop until it gets fixed, and a reviewer can ask for the verdict in the PR instead of re-deriving it. vulkro assess-change gives the same before-and-after view for a change made by hand.

#Put the result where the reviewer is

None of this helps if the output lives in a CI log nobody opens. The CI guide shows the pattern:

vulkro scan . --ai-pr --format gh-pr > comment.md
gh pr comment "$PR_NUMBER" --body-file comment.md

--ai-pr applies the checks that matter most for agent-written changes, and gh-pr renders a comment a reviewer can read in thirty seconds. gh-pr-inline-comments attaches findings to the lines themselves, and gitlab-mr and azure-pr cover the other two hosts. Teams on Vulkro Cloud get the same as a pull-request check, described in the Cloud overview.

#What the machine still cannot do

A scanner answers "does this change introduce a known class of weakness along a path I can trace?" It does not answer "should this feature exist?", "is this the right boundary for this service?", or "does this business rule match the contract?" Those are the reviewer's job and they stay so. The numbers are also worth keeping in mind: on the current blind holdout, Vulkro found 79% of planted vulnerabilities (181 of 229) and flagged 26% of known-safe controls at Medium or above (39 of 152). A clean check lowers the odds, it does not retire the checklist.

#What to do this week

• Add vulkro review --range origin/main...HEAD --baseline origin/main to the PR workflow and make NEW findings the only ones that fail.
• Run vulkro slopcheck on every manifest change before any install step.
• Add vulkro gate --base origin/main as the merge check so a new caller to an old sink is caught.
• Post results with --format gh-pr so the reviewer sees them inside the PR.
• Give your agents verify_fix and ask for its verdict in the PR description.

The CI/CD guide has copy-ready workflows for GitHub, GitLab and Azure. To run it on your own queue, start the 14-day trial.