Skip to main content

Checking another scanner's findings against a proof

· 8 min read
Vulkro
Vulkro
Security research

Most teams that care about security already run more than one tool. A query-based analyser in CI, a linter with security rules, a dependency checker, and lately an AI reviewer that comments on pull requests. Each one writes findings, and someone has to decide which of them are real.

That decision is the expensive part. Producing a finding is cheap. Confirming that untrusted input actually reaches the flagged line, with nothing in between that makes it safe, takes a person reading code. When a tool reports two hundred results, that reading is what stalls.

vulkro verify is built for that step. It takes another tool's results as a SARIF file, checks each one against Vulkro's own analysis of the same repository, and tells you which ones it can prove, which ones it looked at and cannot, and which ones it did not look at.

SARIF is the common language​

SARIF, the Static Analysis Results Interchange Format, is an OASIS standard, and version 2.1.0 is what most security tools emit today. A SARIF log holds one or more runs. Each run names the tool and its rules, and lists results. Each result has a rule id, a message and one or more locations, usually a file and a line. Many results also carry a CWE through the rule's tags or taxonomy references, and some carry a code flow from source to sink.

Because the format is shared, a single command can read the output of almost any scanner you already have, without a plugin for each one. And because SARIF defines property bags (section 3.8 of the spec), an open set of extra properties any object may carry, a second tool can annotate a result without changing anything the first tool wrote.

Three verdicts, never a guess​

Run it from the repository the results came from:

vulkro verify results.sarif
vulkro verify results.sarif --path ./service   # the repository the results were produced on

Vulkro maps each result to a vulnerability family: by its CWE first, then by its rule id, then by keywords in its message. It scans the repository once. Then every result gets exactly one verdict:

  • proven: a Vulkro finding of the same family sits at the result's line, or within three lines of it, and carries machine-checkable proof, such as a traced source-to-sink data flow. The proof chain, the entry point, and the actor, asset and severity from Vulkro's risk model are attached.
  • not provable: Vulkro analysed that location with a model of that family and holds no proof. The reason says what it found instead: only a heuristic match, an adequate guard, a parameterised query or other neutraliser, no recognised source of untrusted input, or no flow at all.
  • not checked (printed as not examined): the result is outside what this build covers. It might be a language Vulkro does not read, a file the scan skipped or could not find, or a family the result could not be mapped to. The reason comes from the scan's own coverage report.

The middle verdict is worded with care, and the command prints the caveat with every report: not provable is not a verdict that the result is a false positive. It means Vulkro analysed the code and could not build a proof. Sometimes that is because the code is safe. Sometimes the proof needs something static analysis cannot see.

What it looks like​

Here is a small Express service with two handlers that look alike to a pattern matcher:

app.get('/orders', async (req, res) => {
  const q = "SELECT * FROM orders WHERE customer = '" + req.query.customer + "'";
  const rows = await db.query(q);
  res.json(rows.rows);
});

app.get('/orders/by-id', async (req, res) => {
  const rows = await db.query('SELECT * FROM orders WHERE id = $1', [req.query.id]);
  res.json(rows.rows);
});

Suppose another tool reports SQL injection on both db.query lines, plus a third result in a Ruby file. Running vulkro verify on its SARIF output gives:

verify: other.sarif (3 result(s) from other-scanner) against .
========================================================================
  1. proven        sqli             app.js:8  [sql-injection]
       Vulkro proves this sqli result: `js-taint-sql-001` at app.js:8 (CRITICAL, 3 proof hop(s), reachable from GET /orders (unprotected)).
       risk: CRITICAL (actor anonymous, asset cross-user-data)
         app.js:6 [entry-point] entry point: GET /orders (no auth guard)
         app.js:6 [taint] source: express:req (param `req`)
         app.js:8 [taint] sink: db.query(q)
  2. not-provable  sqli             app.js:13  [sql-injection]
       parameterised query (a bound placeholder, the value passed separately) at line 13: `const rows = await db.query('SELECT * FROM orders WHERE id = $1', [req.query.id]);`.
  3. not-examined  sqli             lib/report.rb:4  [sql-injection]
       'lib/report.rb' is Ruby, a language this build has no extractor for, so no check read it (see `scan_coverage.unanalysed_languages`).

Summary: 1 proven, 1 not provable, 1 not examined (3 result(s)).

That is real output, with the scan's progress lines removed and the repository path shortened to .. The first result now comes with the path an attacker would take: an unauthenticated route, the request object, and the query it reaches. The second comes with the specific reason Vulkro could not prove it, the bound placeholder, which a reviewer can confirm in seconds. The third is honestly marked as outside coverage, so nobody mistakes silence for a clean bill.

Triage in that order​

The verdicts suggest a working order for a large backlog.

Proven first. These come with evidence a developer can follow line by line, so they are the cheapest to confirm and the most likely to be real. Fix them, or hand them over with the proof attached.

Not provable next, starting from the reason. A "parameterised query" or "adequate guard" reason is a pointer to the exact line that probably makes the result harmless. Read it and decide. A "heuristic only" reason means Vulkro also saw something there but has no flow to back it, which deserves a closer look rather than a dismissal.

Not checked last, and by someone else. These are results Vulkro has no opinion on. Keep the original tool's judgment for them, and do not count them as verified in either direction.

For any single location you want to dig into, vulkro why-not <path:line> explains why Vulkro reported nothing there, using the same reasons. We wrote about it in why did the scanner not flag this.

Keep the original file, add the verdicts​

Three output formats cover the common uses:

  • --format table (the default) for a person reading the result.
  • --format json for the vulkro-verify/1 report, for scripts and dashboards.
  • --format sarif re-emits the input log with every result annotated. Each result's property bag gains vulkro.verdict, vulkro.reasonCode, vulkro.reason, vulkro.family and vulkro.proof, plus the matching Vulkro finding's severity, rule, entry point, actor and asset when one answers for it. Each run gains a vulkro.verify summary with the counts. Nothing in the input is removed or renamed.

That last format means you can keep feeding your existing SARIF viewer or code-scanning upload, and filter on vulkro.verdict there.

The exit code follows the same contract as the rest of the CLI: 0 when no result was proven, 1 when at least one was, and 2 for an error such as an unreadable or invalid SARIF file. A pipeline can fail only on results that carry proof.

The command is deterministic and offline. No model is called, nothing leaves the machine, and it never edits a file: the annotated SARIF goes to standard output.

From an AI assistant​

The same check is available to AI coding agents through Vulkro's MCP server, as the verify_sarif tool. It takes the SARIF and the repository path and returns the same three verdicts. An agent asked to "fix the findings from the security scan" can start with the proven ones and the evidence behind them, instead of rewriting every line a tool pointed at. The MCP server reference lists the tools.

What it does not do​

vulkro verify can only prove what Vulkro itself models. It reads JavaScript, TypeScript, Python, Go, Java, PHP, C and C++. Results in other languages, or in vulnerability families Vulkro has no check for, are not checked. That is the right answer, and the report says so explicitly rather than folding them into a count. It also does not judge the other tool. A result it cannot prove may still be real, and the reasons are there so a person can decide.

Try it on the SARIF you already have​

If your pipeline already writes a SARIF file, you have everything you need: run vulkro verify results.sarif in the same checkout. Output formats covers Vulkro's own SARIF, and why deterministic analysis explains what we mean by proof. To run it on your code, start here.

Sources​