The security ratchet: a baseline that only goes down

Delivery speed has stopped being the trade-off it used to be. The 2025 DORA report finds that teams getting value from AI raise throughput without a higher change failure rate, and names the ongoing challenge plainly: making sure software works as intended before it reaches users. Volume is up with it; GitHub's Octoverse 2025 counts 43.2 million merged pull requests a month.

Remediation has not sped up to match. The 2025 Verizon DBIR, reported by Infosecurity Magazine, found that only 54% of vulnerabilities were fully remediated, with a median of 32 days. Every codebase therefore carries a stock of known debt, and a security gate that fails the build on all of it fails the build every time, until someone turns it off.

The pattern that survives contact with a real team is a ratchet: a baseline of accepted debt, a gate that fails only on findings new since that baseline, and a baseline that is allowed to move in one direction. This post is about the mechanics: the exact flags, the two ways to compare, and the trade-off between them. For the argument that new-only gating is the right policy, see the earlier post.

#Why gates fail politically

A gate that blocks a release has to be defensible to the person whose release it blocked. "Your change added a SQL injection" is defensible. "The build is red because of an authorization finding in a module last touched three years ago by someone who left" is not, and the engineer who hears it will ask for an exception. Grant three exceptions and the gate is a formality.

The fix is not a lower severity threshold. A threshold of Critical-only still fails the build on old Criticals, and it lets new Highs through. The fix is to change what the gate compares against: not zero, but what was already there.

#Compare against the base, not against zero

vulkro gate scans the working tree and the base ref, then reports only the findings present in the first and absent from the second:

vulkro gate --base origin/main

Existing debt does not fail the build; the docs say so in those words. The exit code is the standard contract: 0 when nothing new, 1 when new findings exist, 2 on error. Findings whose disposition is not-examined (authorization shapes the scanner could not decide from source) are reported but do not fail the gate by default. If your team prefers the strict reading, VULKRO_GATE_SEVERITY_ONLY=1 makes those findings fail again by severity alone.

The result is a gate whose every failure has a short explanation: this finding did not exist on origin/main and it does now.

#The ratchet: a baseline that only moves down

gate compares two trees. For a repository with a long history and a slow cleanup, a saved baseline file is the better reference, because it decouples "accepted debt" from "whatever is on main today".

vulkro scan --ratchet fails only on findings new since .vulkro-baseline.json. The baseline is created with vulkro baseline from a scan the team has triaged, committed to the repository, and regenerated when debt is paid down. The name says what is allowed to happen to it: the count of accepted findings goes down as fixes land, and it never goes up, because any new finding fails the build before it can be baselined.

Two flags protect the pattern from its common failure modes. --ratchet-strict makes a missing baseline an error instead of a silent "everything is new" or "nothing is new", which is what you want on a CI runner where a forgotten checkout step could otherwise pass every build. --baseline <FILE> points at a baseline kept somewhere other than the repository root, for monorepos or a central evidence store.

The thresholds compose with the ratchet rather than replacing it. --gate applies the [quality_gate] thresholds from vulkro.toml, so the team's agreed ceilings live in one versioned file. --fail-on critical,high sets the severities that fail. --fail-on-reachable-only excludes dependency CVEs whose vulnerable symbol your code never reaches, which removes the most frequent category of gate failure that nobody can act on.

A workable configuration for a team starting from debt:

# vulkro.toml
[quality_gate]
fail_on = ["critical", "high"]

vulkro baseline                 # once, after triage, then commit the file
vulkro scan . --ratchet --ratchet-strict --gate --fail-on-reachable-only

#Full-tree compare or changed-lines compare

There are two ways to decide what "new" means, and they differ in cost and in what they miss.

vulkro scan --gate-vs <ref> is the cheap one. It scans the lines the change touched and compares against the ref. It is fast enough for every push and it covers the common case, which is a new sink in new code.

vulkro gate --base origin/main is the full one. It scans both trees entirely. It costs more time, and it catches the case the cheap one cannot: a change that adds a new caller to an old, unreachable sink. The sink has not changed, so a changed-lines scan never looks at it. The full compare sees that a finding exists on the working tree that did not exist on the base, because reachability changed even though the vulnerable line did not.

The sensible layout uses both. Run --gate-vs on every push for fast feedback. Run gate --base on the merge check and on the nightly build, where the extra minutes are cheap and the missed case would be expensive.

#Keep the unproven visible without blocking

A ratchet that only shows what fails the build throws away information. Findings the scanner could not prove, and shapes it did not decide, still belong in front of the reviewer; they just should not block a release on a question the machine admits it cannot answer. Vulkro keeps them in the report at their disposition, counts them in the summary line, and leaves the gate decision to the findings it can stand behind. If a team wants a stricter posture later, the environment variable above tightens it without a change to the pipeline.

#Emit new-only SARIF so alerts do not multiply

When the gate output goes to a code scanning dashboard, the comparison has to survive the upload. GitHub's SARIF support deduplicates alerts only when the ruleId is stable and the result carries a fingerprint; otherwise every run creates a fresh alert for the same finding and the dashboard fills with duplicates of old debt.

vulkro gate --base origin/main --format sarif > results.sarif

The SARIF from gate contains only the new findings, each with a stable rule id and fingerprint, so the dashboard shows the delta the gate acted on and nothing else. Teams on Vulkro Cloud get the same view under scans, branches and gates, described in the Cloud overview.

#What to do this week

• Run one full scan, triage it with the team, and create .vulkro-baseline.json with vulkro baseline. Commit it.
• Switch the CI gate to vulkro scan . --ratchet --ratchet-strict with --fail-on critical,high and --fail-on-reachable-only.
• Add vulkro gate --base origin/main to the merge check so a newly reachable old sink is caught.
• Upload --format sarif from the gate, not from a full scan, so the dashboard shows the delta.
• Put a date on the calendar to regenerate the baseline after the next cleanup sprint, and watch the count go down.

The CI/CD guide has the workflow files for GitHub, GitLab and Azure. To put a ratchet on your own pipeline, start the 14-day trial.