Benchmark
Ranked on a suite we cannot tune against.
An independent public test of 2,740 generated Java cases that we did not write. It is scored on the suite's own metric, the true-positive rate minus the false-positive rate, and this is where that places us.
- Independent public suite
- Scored on true positives minus false positives
- Measured with the AI off
01 / The public suite
Second of 6 on the public Java accuracy suite
A separate, independent test of 2,740 generated Java cases that we did not write and cannot tune against. It is scored on the suite's own metric: the true-positive rate minus the false-positive rate.
bench/owasp-core/results/ - vulkro 0.21.0 (feat/proof-carrying-tier @ c485559507d1f3c2), measured 2026-08-18. OWASP BenchmarkJava v1.2, 2,740 cases, scored with AI off.
| Rank | Scorecard | True positives | False positives | Score |
|---|---|---|---|---|
| 1 | Published scorecard A | 96.84% | 57.74% | 39.10 |
| 2 | Vulkro | 62.60% | 24.30% | 38.35 |
| 3 | Published scorecard B | 50.36% | 17.02% | 33.34 |
| 4 | Published scorecard C | 19.95% | 0.12% | 19.84 |
| 5 | Published scorecard D | 5.12% | 5.19% | 0.07 |
| 6 | Published scorecard E | 0.00% | 0.00% | 0.00 |
The five comparison scorecards are the ones published alongside the suite, and they were produced against tool versions from around a decade ago. Our row was measured this year. That is not a like-for-like contest, so treat the placing as a public cross-check rather than a live head-to-head.
The row above ours reaches its score by flagging nearly everything: a 96.84% true-positive rate bought with a 57.74% false-positive rate. On a real codebase that is a scanner a team switches off in a week. Our row is 62.6% against 24.3%, which is the trade we make deliberately.
The suite is also synthetic. Generated test cases are useful precisely because nobody can tune to them, and useless as a description of how a scanner behaves on a real application.