Confidence model

Every finding Vulkro emits carries a confidence: High | Medium | Low value plus a confidence_reason: String that explains why the engine fired and what would mark this a false positive. This is the heart of how we keep a high-recall scanner usable.

Levels

Level	Meaning	When you'd see it
High	Strong, often runtime-confirmed evidence.	Provider-format secrets, taint-confirmed injection paths, KEV-listed CVEs, vulkro probe confirmations.
Medium	Pattern matches but with credible false-positive shape.	Heuristic IDOR (path takes an :id and no auth check is visible, but might be wrapped). Generic secret patterns without provider format.
Low	Heuristic. Often fires on test/example/migration code.	Pattern-match-only checks where context confirms little.

vulkro scan defaults to --min-confidence high. To see the rest:

vulkro scan . --min-confidence medium
vulkro scan . --all

Strict-confidence filter

For CI gates that want maximal precision, vulkro scan --strict-confidence (or VULKRO_STRICT_CONFIDENCE=1) layers an additional filter on top of the --min-confidence cut: any High finding whose evidence bag is empty is dropped at output time. A finding survives the strict cut only when it carries at least one populated Evidence item, which means at least one corroborating signal (a taint-flow source-to-sink, an exact-match invariant such as a SUP-COMPROMISE catalog row, a schema-vs-code contradiction, or a pair of pattern signals whose weights sum past the cumulative threshold).

Off by default; behaviour-preserving for callers who don't opt in. See the strict-confidence section on vulkro scan for the flag's surface and the env-var equivalent.

Per-detector-category tiers

The generic High | Medium | Low table above tells you the shape of a finding's confidence. What each level specifically means is detector-specific. The table below is the authoritative per-category reading; every rule page (under /docs/rules and siblings) links back here.

Detector category	High means	Medium means	Low means
Secrets (secrets, git_history)	Provider-format match (AWS AKIA, Stripe sk_live, GitHub ghp_, etc.) on a non-test path. The string is shaped like a real credential.	Generic high-entropy literal that matches no provider format, but assigned to a *_KEY | _TOKEN | _PASSWORD identifier. Could be a placeholder.	Long hex / base64 string with no naming signal. Frequently a hash, content checksum, or test fixture.
Auth (auth_dataflow, authz)	Route is reachable AND no auth middleware, dependency, decorator, or guard is visible in the import scope or call graph.	Route looks unprotected at the file level, but a global mount (app.use('/api', ...)) or framework convention (APP_GUARD) could be applying auth at runtime.	Route name pattern-matches admin/internal/private but no behavioural signal supports the claim.
IDOR / BOLA (idor)	Handler reads a path/query param like :id and resolves the object without a where(user=...), .filter(owner=...), or equivalent scoping clause; and taint flows from the param into the lookup.	Handler takes an :id and the lookup is unscoped, but no taint analysis confirmation (the param might be derived).	Heuristic only: an :id route exists with no visible check, used for forensic mode.
Mass assignment (mass_assignment)	Body-to-model bind via Model(**req.json), Object.assign(user, req.body), setattr loop over body, etc., on a model whose schema has fields the API surface does not allowlist.	Body-to-model bind on a model whose schema we cannot fully resolve (cross-file fragment, dynamic ORM).	Pattern match alone, no model resolution.
Injection (injection, injection_extra, taint)	End-to-end taint chain from request source to a SQL / shell / template / eval sink without intervening sanitiser.	Sink call with a likely-tainted argument but no full chain confirmation.	Sink call with no taint corroboration; pattern-only.
Crypto (crypto_weakness, crypto_new)	Exact-match invariant: md5(buf) for a password, MODE_ECB for confidentiality, verify=False on TLS. The construct is provably wrong regardless of context.	Weak primitive in use, but might be for a non-security purpose (e.g. md5 for cache key).	Reference to a weak primitive in a comment, docstring, or test name.
SCA / CVE (package_risk, slopsquat_known)	Direct dependency matches a KEV-listed advisory at the exact pinned version.	Transitive dependency matches an advisory; or the version match is a range hit not an exact one.	Heuristic typosquat / new-package signal without a CVE backing it.
SSRF (security::ssrf, taint)	Tainted URL flows into an HTTP client call without an allowlist check or hostname normalization between source and sink.	Tainted URL flows into a sink but the chain crosses a function we couldn't fully resolve.	Pattern only: URL-shaped argument to an HTTP client.
LLM (llm_security)	Tainted user input flows into a model SDK call (openai.chat, anthropic.messages, langchain) without isolation or structured-tool constraint. Or: an env var known to hold a secret flows into the prompt.	Tainted input near an LLM call but no full chain.	LLM call with any string-template usage that could in principle contain user data.
CSRF / cookies (csrf, cookies)	Session-cookie middleware is mounted AND CSRF middleware is absent or commented out; or cookie is set with SameSite: 'none' without Secure.	Session middleware present, CSRF middleware unclear; or cookie missing one of Secure/HttpOnly/SameSite but not all.	Pattern-only: a session-like cookie name with no other signal.

If a finding's tier surprises you, look up the category here first; the per-category meaning above is what the rule actually asserts. The matching rule page at /docs/rules walks through the OWASP category and its detectors in detail.

confidence_reason

Every finding has a one-line explanation:

confidence_reason = "taint flowed from req.body to db.query without sanitiser"
confidence_reason = "auth helper not found in import scope"
confidence_reason = "runtime-confirmed via active probe"
confidence_reason = "AKIA-prefix matches AWS access-key format"

Surfaced in JSON, SARIF (properties.confidence_reason), and the desktop console. Designed to be readable without re-reading source.

Calibration table

Vulkro holds a static calibration table that downgrades High -> Medium for detector categories scoring under 30% TP-rate on the 13-repo benchmark. The intent is that if a category is 70%+ false-positive in the wild, callers shouldn't be told "High confidence" by default.

The table is updated when benchmark numbers change. This keeps the default-mode signal-to-noise ratio honest as new detectors land.

Where the bar is

The published benchmark on 13 deliberately-vulnerable repos:

	vulkro default	vulkro --min-confidence high	open-source rules baseline	multi-language OSS scanner
precision	0.21	0.62	0.60	0.45
recall	0.91	0.76	0.22	0.45
F1	0.34	0.68	0.32	0.45

--min-confidence high is now the production-recommended cut: it leads the open-source rules baseline on precision (0.62 vs 0.60) and the broad multi-language scanner on recall (0.76 vs 0.45) on the same corpus. F1 0.68 beats both outright.

The lift came from three changes shipped on top of the Phase 4 AST- confirmation engine: the cumulative-weight High threshold was calibrated from 1.5 to 1.1 to match what shipped detectors actually emit (Phase 4 pairs at 0.7 + 0.5 = 1.2); four taint / template / autoescape emit sites were retrofitted to attach Evidence so the aggregator can survive the OWASP-category calibration downgrade; and a (file, line, message) dedup pass removes duplicate emissions from overlapping intra-/inter-procedural taint engines.

See Benchmark for the full methodology and per-repo breakdown.

Offline false-positive triage (--triage)

vulkro scan --triage adds an optional, default-off, fully-local pass that re-ranks findings by demoting the usual noise sources one confidence tier:

• test code (test/, tests/, __tests__/, *.spec.*, *_test.go, ...)
• fixtures / examples / samples / mocks
• generated / vendored / minified output (dist/, vendor/, node_modules/, *.min.js, ...)
• database migrations

It is deterministic (same input, same output) and makes no network call and uses no model - VULKRO_OFFLINE=1 is honoured. It is a path-based heuristic re-ranker, described as exactly that: it is not an ML classifier, and we do not call it "AI". (An optional on-device model layer is a future addition.)

Triage only ever lowers confidence and never drops a finding, so recall is preserved by construction: a demoted finding still appears under --all-confidence. The point is to combine it with --min-confidence high so the test/fixture/generated noise drops out of the default view while staying one flag away:

vulkro scan . --triage --min-confidence high