Skip to main content

The pipeline gate

A gate is only worth adding if it survives the second week.

Adding a security check to a pipeline is easy. Adding one that is still switched on a month later is the hard part, and it usually fails for one of three reasons: the verdict moves on code that did not change, the first run buries the team in findings nobody asked for, or the whole repository leaves the building on every push. Vulkro is built against all three. The same commit always gets the same verdict, an existing codebase only fails on what a change introduced, and the check runs on your own runner.

Start the 14-day trialHow accurate it is →
  • Same commit, same verdict
  • Fails only on new findings
  • Runs on your runner
  • Nothing uploaded

01 / Why gates get switched off

Nobody disables a gate on purpose

It happens one exception at a time, and by the end the check is still in the pipeline configuration and no longer blocks anything.

The first failure is a verdict that moves on its own. A check that passes in the morning and fails in the afternoon on identical code teaches a team one lesson very quickly: when it goes red, run it again. After that the gate is decoration. Nobody investigates a red build from a check that has cried wolf, and the one time it is right about something serious, it gets re-run too.

The second is the first run on a codebase with history behind it. Thousands of findings arrive at once, none of them introduced by the person who happened to open the next change, and the only workable response is to make the check advisory. It never becomes blocking again, because there is no day on which somebody volunteers to clear a backlog that size.

The third is procurement. A check that uploads the repository to a third party on every push is a data-handling decision, and once someone in security or legal notices, the check waits on a review that outlasts everyone’s enthusiasm for it.

A gate that is switched off catches nothing, no matter how good the detection behind it is. So the properties that keep it switched on matter more than the ones that make a demo look impressive.

02 / Determinism

The verdict moves when the code moves, and not otherwise

Same version, same tree, same settings: the same findings, in the same order, on every run and on every machine.

Nothing in the review samples, and no model decides whether a finding exists. Re-running an unchanged commit returns the counts of the first run, which is what makes the difference between two runs mean something: the movement is your code, not the tool. That is the property a gate needs and the one that is hardest to retrofit.

It is asserted in our build rather than promised in a document. One test runs the review twice inside a single process and compares the results, which is a real test rather than a tautology, because a result that quietly depends on internal iteration order disagrees with itself there. A second test runs two full reviews of the same tree through the parallel pipeline and compares those. Both exist because the promise had already failed quietly once on a large repository, and a fix is only worth something if a test holds it in place.

console - project overviewcheckout-service
Health
82/100No change in findings since the previous scan

3 high, 9 medium, 14 low148 endpointslast scan re-run of the same commit

Deploy gate: pass0 critical (max 0)
The same commit reviewed a second time: the same counts, the same score, the same verdict. A red build from this gate is a reason to look rather than a reason to re-run.

03 / The retrofit

On day one, only fail on what this change introduced

A codebase with years behind it will not pass a fresh review, and pretending otherwise is how the gate ends up advisory forever.

There are two ways to hold the line, and they suit different teams. You can record the current state once and commit that record, after which the gate fails only on findings that were not in it. Existing work stays visible and gets burned down on whatever cadence you actually have, rather than becoming an assignment nobody can finish. Or you can compare each change against the branch it is merging into, which suits teams whose main branch moves fast enough that a recorded snapshot would be stale within a week.

Either way the contributor experience is the one that keeps the gate alive. Somebody opens a change, and the only findings they are asked about are the ones their change introduced. No inherited backlog, no argument about whose fault a seven-year-old handler is, and no incentive to route around the check.

Determinism is what makes that posture safe. Comparing two reviews is only meaningful if the review itself does not drift, otherwise the difference between them includes the tool’s own noise and the gate starts failing changes that introduced nothing at all.

vulkrobot · commented just now

HIGHBroken authorization on invoice download

routes/invoice.ts:47VULK-1042CWE-639

The handler looks the invoice up by the id in the path and returns the file. Nothing scopes that lookup to req.org, and requireAuth only proves the caller is signed in, not that the invoice belongs to them. Any authenticated user can download invoices from another organization by changing the number in the URL. The check belongs in the query, not in the response.

Suggested change

Line beforeLine afterChangeSource
@@ -45,7 +45,11 @@ router.get('/invoices/:id/download')
4545router.get('/invoices/:id/download', requireAuth, async (req, res) => {
46Removed line. const invoice = await db.invoice.findUnique({
47Removed line. where: {id: req.params.id},
46Added line. const invoice = await db.invoice.findFirst({
47Added line. where: {id: req.params.id, orgId: req.org.id},
4848 });
4949 
50Added line. if (!invoice) {
51Added line. return res.status(404).send('not found');
52Added line. }
53Added line.  
5054 return res.download(invoice.path);
5155});
Re-scanned. Finding no longer reported, file still parses.verified before posting

vulkro scan · exit 1 · 1 high, 0 criticalRan on your runner. The code never left it.

What a contributor sees. One finding, the file and line it sits on, the proposed change, and the receipt from re-checking the patched file.

04 / Where the report lands

Findings go where your team already looks

A report in a format your pipeline cannot read is a report nobody reads. The same review data comes out in whichever shape the destination expects.

Where a pipeline can send the findings, and what each destination receives.
DestinationWhat it gets
Your code-scanning tabThe standard interchange format for static analysis results. It is what a code-scanning view, a security dashboard and most managed pipelines read natively, so the findings appear where your team already looks rather than in a log nobody opens.
A comment on the change itselfThe finding rendered as a review comment, anchored to the file and the line, with the proposed fix inline. The conversation starts from something concrete instead of from a link somebody has to be granted access to.
Your test results panelThe same findings shaped as test results, so a pipeline that already surfaces failing tests surfaces failing security checks in the same place with no extra wiring.
A log or event pipelineOne finding per line, for a pipeline that forwards into a log platform or a security event system rather than into a human-facing dashboard.
A spreadsheet, or the terminalA flat table for the person who has to hand the list to somebody else, and the plain terminal report for the person watching the run.
These sit inside a catalogue of 24 output formats generated from the same review data, all written on the runner. The catalogue is printed by the binary itself, and a build test fails if the documented list and the emitted list disagree, so the two cannot drift apart.
.github/workflows/vulkro.ymlthe whole gate
name: vulkro
on:
  pull_request:
  push:
    branches: [main]
jobs:
  scan:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      security-events: write
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }   # full history, so the comparison works
      - run: curl -fsSL https://dist.vulkro.com/install.sh | bash
      - run: vulkro scan --ratchet --format sarif --output vulkro.sarif
      # then hand vulkro.sarif to your platform's own upload step, with
      # its always-run condition set, so the findings still reach the
      # dashboard on a run the gate failed
      - uses: <your-platform>/upload-sarif
        if: always()
        with: { sarif_file: vulkro.sarif }
The complete pipeline job. Install, review, upload the results, and keep uploading them even when the gate fails, so the findings reach the dashboard and the build still fails the way you want it to.

The same job is shorter on a pipeline that ingests the standard format natively as a security report: install, review, and declare the output file as the report. The result contract is the same everywhere. A clean run and a run with findings are distinguishable from an error, so your pipeline can treat a broken installation differently from a genuine failure instead of colouring both red.

05 / On the runner

Nothing leaves the runner, including on the runner

The check is a single binary that runs where the checkout already is. There is no upload step, so there is nothing to get a data-handling review for.

Your code, the findings and every report file stay in the runner’s workspace and are retained by your pipeline rather than by us. There is no queue to wait behind, no dashboard that has to be provisioned for every contractor, and no third party holding an inventory of your product. That is a property of where the software runs rather than a commitment in a contract.

The runner does need your licence, the same as any machine that runs a review. For ephemeral runners that is a licence file rather than an interactive sign-in, which is the arrangement most teams end up with because it needs no credential in the pipeline and no network round trip during the build. Licences are issued per seat directly by our team, and your first sign-in anywhere starts a 14-day trial of the full product.

Air-gapped and strict-egress pipelines are covered on the offline page.

06 / Honest scope

What a green build does not mean

The gate is a floor under every change. It is not an application security programme, and reading it as one is the expensive mistake.

It is a read of your code, not a test of your system
Nothing is executed and nothing is probed. A defect that only appears under load, only with production data, or only in the configuration of the environment you deploy into, is out of scope for this gate.
It is not a penetration test
A person paid to attack your product will find things a static read cannot. The gate is the floor you hold every change to, and it exists so the test finds interesting problems instead of the same four every time.
The order is triage, not proof
Findings are ranked by whether your own code reaches them, which is a defensible order to work down. It is not a claim that an attacker can reach any of them as you are deployed.
A green build means the known shapes are absent
It does not mean the change is safe. The right way to read a pass is that nothing the review knows how to recognise is present in what you changed.

The measured result, the method behind it and the places the review loses are published on the proof page. A gate is worth what its detection is worth, and that page is where the detection is argued rather than asserted.

The review that stands at the gate

Vulkro reviews your codebase and Vulkro for Salesforce reviews your Salesforce build. Vulkro Red is the other half of the team: it takes what the review found and works out what an attacker would actually do with it.

The reviewVulkro

Reviews your code

Goes through every line of your codebase before a release, the way a senior engineer would if they had the time, and tells you what a customer could exploit.

You get: what to fix, and a pass or fail on the releaseWhat it checks
 Vulkro for Salesforce

Reviews your Salesforce build

The same review for the part of your business that runs on Salesforce, including the settings in the org itself, and what the AppExchange security review will ask you for.

You get: a straight answer on whether you are ready to submitVulkro for Salesforce
The red teamVulkro RedComing soon

Attacks what they found

Takes the review and works out what an attacker would actually do with it: which small problems chain together into a real break-in, and which ones are noise.

You get: the attack, step by step, before someone else runs itHow it works

One engine behind all three, so the red team works from what the review already foundIt all runs on your machine. Your code never leaves it.