Vulkro vs the field, same corpus
We benchmarked Vulkro against Semgrep CE, Opengrep, Bearer, Bandit, Gosec, and njsscan on the same public ground-truth corpus, on the same machine, on the same day. The tables below include the comparisons we lose. One command reruns the whole thing on your laptop.
$ bench/comparison/run.sh --tier1 --tools vulkro,semgrep,opengrep,bearer,bandit,gosec,njsscan --min-confidence high
The four tools that scan more than one language, scored over the 12 supported-language repos of the Tier 1 corpus (76 catalogued bugs). The corpus also contains 3 PHP and Java repos: every tool scans them and the per-repo rows are published, but they are excluded from these totals because Vulkro does not analyze PHP or Java. Semgrep CE and Opengrep run the same p/security-audit ruleset, which is why their rows match.
| Tool | Precision | Recall | F1 | Bugs found (of 76) | False positives | 15-repo scan time |
|---|---|---|---|---|---|---|
| Vulkro 0.15.0 | 0.81 | 0.58 | 0.68 | 44 | 10 | 38.0s |
| Bearer 2.0.2 | 0.36 | 0.49 | 0.41 | 37 | 67 | 363.0s |
| Semgrep CE 1.136.0 | 0.62 | 0.20 | 0.30 | 15 | 9 | 48.9s |
| Opengrep 1.25.0 | 0.62 | 0.20 | 0.30 | 15 | 9 | 46.1s |
Vulkro finds 2.9x the catalogued bugs Semgrep CE finds (44 vs 15) at higher precision (0.81 vs 0.62), and more than Bearer (44 vs 37) with 10 false positives to Bearer's 67. A max-recall mode exists: at min-confidence low Vulkro reaches 0.72 recall at 0.15 precision, labeled as max-recall in the published scorecard. We grew our own ground truth from 69 to 90 bugs in July 2026 and published the recall drop that came with it; the corpus is versioned.
$ cat bench/comparison/scorecard-high.md # scores full methodology
Bandit only scans Python, Gosec only Go, njsscan only JS/TS, so rolling them up over a mixed corpus would manufacture zeros for them. Instead each specialist is scored only over the repos matching its language, and Vulkro is re-scored over the same subset. Fair fights only. We win two of the three and publish the one we lose.
7 Python repos, 42 catalogued bugs
| Precision | Recall | Bugs | FP | |
|---|---|---|---|---|
| Vulkro | 0.87 | 0.64 | 27 of 42 | 4 |
| Bandit | 0.56 | 0.43 | 18 of 42 | 14 |
On its home language Bandit finds 18 of the 42 catalogued Python bugs with 14 false positives. Vulkro finds 27 with 4.
1 Go repo (govwa), 4 catalogued bugs
| Precision | Recall | Bugs | FP | |
|---|---|---|---|---|
| Gosec | 0.20 | 0.75 | 3 of 4 | 12 |
| Vulkro | 0.00 | 0.00 | 0 of 4 | 1 |
On our one Go repo, Gosec finds 3 of 4 bugs; Vulkro finds 0. Go detection depth is a known gap we publish. The 4 misses (fmt.Sprintf SQL interpolation, template XSS, insecure cookies) stay in the corpus as catalogued false negatives.
4 JS/TS repos, 30 catalogued bugs
| Precision | Recall | Bugs | FP | |
|---|---|---|---|---|
| Vulkro | 0.77 | 0.57 | 17 of 30 | 5 |
| njsscan | 0.50 | 0.33 | 10 of 30 | 10 |
njsscan targets plain JavaScript, so on the TypeScript repo (juice-shop) it sees almost nothing in .ts files: 0 of 8 catalogued bugs there. The per-repo rows in the scorecard keep that visible.
$ cat bench/comparison/scorecard-high.md # section "Per-language specialists"
Every card ends with the concession. If a competitor is the better tool for your stack, the honest line below says so.
Semgrep CE and Opengrep (the Linux Foundation fork) run the same p/security-audit ruleset, so they score identically on our corpus: 15 of 76 catalogued bugs at 0.62 precision. The cross-file taint rules that catch the rest sit in the paid Semgrep tiers. Vulkro finds 44 of 76 at 0.81 precision with the deep rules included in Pro, per machine, no auto-renewal.
Honest Where they win: Semgrep is the gold standard for writing your own custom rules, and it parses far more languages than we analyze deeply. Vulkro detectors are compiled; you cannot author one without us shipping it. If your team writes bespoke rules every week, Semgrep earns its seat.
Bearer is the only multi-language tool in the run that comes close on recall: 37 of 76 catalogued bugs to our 44. It pays for that with 67 false positives to our 10, and it needs 363.0s for the corpus scan that takes Vulkro 38.0s.
Honest Where they win: Bearer scans PHP, Java, and Ruby, including the 14 catalogued PHP and Java bugs in our own corpus that Vulkro cannot scan at all (it finds 9 of them). Its privacy and data-flow reports for GDPR work are a real feature. And it is free under the Elastic License for in-house use.
Bandit, Gosec, and njsscan are free, install in seconds, and carry zero vendor risk. On the fair-fight subsets Vulkro beats Bandit on Python (27 vs 18 bugs at higher precision) and njsscan on JS/TS (17 vs 10), and loses to Gosec on Go outright: 0 of 4 to its 3 of 4.
Honest Where they win: Gosec beats us on Go today, and all three cost nothing. If you ship exactly one language and triage discipline is high, a specialist plus code review is a legitimate stack. We publish the subset tables so you can make that call from data.
Snyk Code has no column above: it requires an authenticated account, and publishing benchmark results may be restricted by its Terms of Service (a check we have not completed). The harness keeps a working Snyk runner: add snyk to --tools, authenticate, and produce the numbers locally for yourself after checking the current ToS. Akto and Pynt are DAST tools; they cannot scan a static code directory and exist only as documented stub lanes.
$ bench/comparison/REPRODUCE.md # section "Tools not included"
The harness, the pinned corpus, the ground truth, and the scoring code are public. One command reruns the whole comparison from a fresh machine; the walkthrough is bench/comparison/REPRODUCE.md.
# zero-to-scorecard on a fresh machine (needs vulkro, git, python) VULKRO="$(command -v vulkro)" bench/comparison/run.sh --tier1 # the published headline setting VULKRO="$(command -v vulkro)" bench/comparison/run.sh --tier1 --min-confidence high # read the result cat bench/comparison/results-*/scorecard.md
Same binary, same corpus SHAs, same scoring code: same TP/FP/FN counts on any machine. Competitor counts can drift because the Semgrep registry ruleset is not pinned upstream; the scorecard records the versions it ran. Full methodology and every table.
A score tells you who caught more bugs. This tells you what Vulkro ships that the others do not. Every cell is a real capability, checked against each tool. Snyk, Semgrep, SonarQube, and Socket are the reference set.
| Capability | Vulkro | Snyk | Semgrep | SonarQube | Socket |
|---|---|---|---|---|---|
| Reproducible benchmark ships with the toolThe harness, pinned repos, ground truth, and scoring code are public. You reproduce the exact number on your own laptop. | Yes | No | No | No | No |
| Cross-file taint dataflowSource-to-sink data flow, 3 passes, cross-file depth 4. Free tools mostly ship regex, not a dataflow ladder. | Yes | Yes | Pro tier only | Paid only | No |
| Reachability-ranked SCA (KEV / EPSS)SCA across npm, PyPI, Go, Cargo, Maven. Local CVE bundle of OSV + NVD + CISA KEV + FIRST.org EPSS. Ranks reachable code first. | Yes | Partial | No | No | Partial |
| In-loop AI write guardvulkro guard checks AI-written code as it lands, before it reaches your working tree. | Yes | No | No | No | No |
| Hallucinated-package gatevulkro slopcheck refuses install of packages an AI invented (slopsquatting) before they hit your lockfile. | Yes | No | No | No | Partial |
| Per-tool AI-origin auditTracks which code an AI wrote, with reviewer attestation. Incumbents do not ship this. | Yes | No | No | No | No |
| Reachability-backed VEX + CBOMOpenVEX and CycloneDX-VEX with not_affected backed by reachability, plus CycloneDX CBOM (crypto BOM). | Yes | Partial | No | No | No |
| Runs fully offline / no telemetryVULKRO_OFFLINE=1 hard-refuses every outbound call. No account, no upload, no phone-home. | Yes | No | Partial | Partial | No |
| Deterministic (same code, same result)No AI in the engine, so the same input always yields the same findings. Diffable in CI. | Yes | No | Yes | Yes | Yes |
| Local desktop console from one scan34 analysis views available over a single offline scan. The binary is also an MCP server, an LSP server, and the console backend. | Yes | No (cloud UI) | No | Server dashboard | No (cloud UI) |
| No auto-renewal / no per-seatPer-term license, priced per machine, that you choose to renew. Not billed per developer, not auto-charged. | Yes | No | No | No | No |
Beyond the table: 150+ core detectors, OWASP API Top 10 (all 10) and LLM Top 10, a 13-class DAST probe (authorization-gated), reachability-gated SCA across 5 ecosystems, and 23 machine-readable output formats including SARIF, CycloneDX SBOM, CycloneDX CBOM, SPDX 2.3 and 3.0.1, OpenVEX, and GDPR RoPA.
Pick your current tool. Get a one-line replacement for each common workflow.
Same workflow, no upload.
| Snyk Code workflow | Vulkro equivalent |
|---|---|
snyk code test | vulkro scan . |
Snyk PR check | vulkro scan . --since main --format gh-pr |
Snyk SARIF export | vulkro scan . --format sarif |
Snyk container scan | vulkro container <image> |
Long-form, one-on-one head-to-heads: the numbers, the workflow mapping, and where each tool genuinely wins.
Vulkro vs Snyk · Vulkro vs Semgrep · Vulkro vs Bearer · Vulkro vs Trivy · Vulkro for Salesforce vs CodeScan · Vulkro for Salesforce vs DigitSec · Best offline SAST
Community
Follow along on Substack or r/vulkro. You will hear about new detectors, benchmark refreshes, and CVEs that broke the things you scan.
Subscribe once, get two things: the weekly CVE + release digest in your inbox, and access to the live chat where in-between things land (detector ideas, weird findings, release heads-ups).
Public community at r/vulkro. Bug reports, scan-result war stories, CVE chat, AppSec questions. No email required, indexed by Google so threads stay useful.
Bug reports, install help, billing questions. One human reads every message. No web form, no chatbot, no AI summariser.
Don't take the table at face value. Install Vulkro, run it on the same project your current scanner already covers, and compare what each one reports. The public benchmark harness automates the comparison if you want a numeric answer.