What did the agent's edit break?

An AI coding agent adds a tenantId parameter to findById, updates the two call sites in the file it was looking at, writes a cheerful summary and stops. There were five call sites. The other three are in files it never opened, they still pass one argument, and in a dynamically typed codebase nothing will complain until a request reaches one of them in production.

This is the most ordinary way an agent's edit goes wrong. Not a subtle logic error, not a security hole: a change whose consequences live somewhere the agent did not look. People make the same mistake, but people tend to make small changes and run the build. Agents make wide changes quickly and often declare victory before anything has been run.

vulkro ctx check is the question you would ask after any edit, answered mechanically: what did this change break? It works the same whichever agent or person made the edit, and it can run on every commit.

We covered how agents read code with vulkro ctx in the previous post, and how to stop agents writing vulnerable code in securing AI coding agents. This post is about the edit itself.

What it reports

Run it after an edit, with no arguments, and it compares the working tree (untracked files included) with HEAD:

vulkro ctx check

It lists:

• Call sites still on an old signature. Every call that passes the old number of arguments to a function whose parameters changed.
• Calls aimed at the wrong method. A call to a same-named method on another type is not blamed for your change. ctx check tells methods apart by the types of their receivers.
• Removed names that are still used. A function deleted or renamed in one file and still called from another.
• Syntax errors on the changed lines. Code that no longer parses, which a large search-and-replace can easily leave behind.
• What depends on the change, and the tests to run.
• Security findings on the lines you touched. The same detection as a regular scan, limited to your change, so the report is not buried under the backlog the repository already had.

It exits 1 when it finds a problem and 0 when it does not, so an agent, a script or a hook can act on the result.

An example

Take two repository classes in separate files, each with a findById method, and a routes file that calls both:

// src/routes.ts
import { invoiceRepo } from "./invoices";
import { userRepo } from "./users";

export function getInvoice(id: string) { return invoiceRepo.findById(id); }
export function getUser(id: string) { return userRepo.findById(id); }

The agent adds a tenantId parameter to InvoiceRepo.findById and nothing else. vulkro ctx check --fast (skipping the security scan, to keep the example short) says:

ctx check (uncommitted changes vs HEAD): 1 file changed, 1 problem
changed:
  src/invoices.ts: findById (signature (id: string) -> (id: string, tenantId: string))
problems:
  1. InvoiceRepo.findById (src/invoices.ts) now takes 2 arguments (was 1 argument). 1 call site still passes the old count:
     src/routes.ts:3 in getInvoice [passes 1]: export function getInvoice(id: string) { return invoiceRepo.findById(id); }
affected: 1 functions depend on what changed
tests to run: none found
security: not checked (--fast)

Two things to notice. The broken call is in a file the agent never edited. And userRepo.findById(id) on the next line is not reported, even though the name is identical, because userRepo is a UserRepo and that method did not change. A text search for findById would have flagged both, and an agent told "fix all the callers" might well have broken the second one.

The receiver types are resolved across files: a shared instance exported from one module (export const invoiceRepo = new InvoiceRepo()) and imported into another is matched to its class. Methods defined in object literals (const invoiceRepo = { findById(id) { ... } }) are recognised too.

Does it flag the right edits?

A check like this is only useful if it is quiet when nothing is wrong. An agent that gets a false alarm on every edit learns to ignore it, and so do people.

So we measured it on 62 recorded agent edits. Ten of them left the build broken. ctx check flagged all 10, and none of the 52 that did not.

That is a small set, and it is our own, so read it as evidence that the check is precise on the kinds of edit agents actually make rather than as a guarantee. It is also worth being clear about what "broken" means here: the four problem kinds above. A change that compiles, keeps every signature intact and is simply wrong is not something a structural check can see. That is what your tests are for, which is why ctx check lists the ones to run.

Before every commit

An agent can be told to run ctx check (and vulkro ctx init writes that instruction into AGENTS.md), but an instruction is a request. A git hook is not:

vulkro install-hook --pre-commit

The pre-commit hook runs vulkro ctx check on what is staged, whichever agent or person made the change, then the new-findings gate against HEAD. By default the ctx step warns: it prints the broken call sites and lets the commit go on. To make it stop the commit instead:

vulkro install-hook --pre-commit --block-on-ctx-check

We made warning the default on purpose. A hook that blocks on its first day gets uninstalled; one that has been right for a few weeks gets promoted. The hook manages a marked block inside the hook file, so it works alongside an existing husky or hand-written hook, and VULKRO_SKIP_HOOKS=1 skips it once (install-hook reference).

In the hook, the check runs on staged changes in its fast form, without the security scan, so a commit is not held up by a full scan. Outside the hook the flags are yours to choose:

vulkro ctx check --staged         # only what is staged
vulkro ctx check --base main      # everything on this branch since main
vulkro ctx check --fast           # skip the security scan

Inside the agent's loop

A commit hook catches problems at the end. It is better if the agent catches them itself, before it writes its summary.

For agents with a shell, that is just vulkro ctx check as the last step of a task. For MCP clients, vulkro mcp serve exposes the same check as the check_change tool, with optional base and fast arguments (MCP reference). An agent that calls it after an edit gets the broken call sites as a tool result and can fix them in the same turn, while it still has the change in mind.

For Salesforce projects, vulkro-sf ctx check does the same for Apex and Lightning Web Components: call sites still passing the old arguments to a method whose parameters changed, removed methods still used, syntax errors on changed lines, the test classes to run, and Salesforce findings such as CRUD and FLS on the lines you changed. vulkro-sf init --pre-commit installs it as a hook with the same --block-on-ctx-check option, and MCP clients get sf_check_change (reference).

What it cannot see

A call through a name the call graph cannot resolve (reflection, a string-keyed dispatch table, a method chosen at runtime) is not checked, so a clean report is not proof that every caller is fine. The impact list is a floor for the same reason. Receiver typing has limits too: when the type of a receiver cannot be worked out, a call to a same-named method can be listed when it should not be. The report shows each call line, so that is quick to judge.

And it checks structure, not behaviour. If the agent changed what a function returns without changing its signature, every caller is still "correct" as far as ctx check can tell. Run the tests it lists.

Where to see it

Install Vulkro, run vulkro install-hook --pre-commit in a repository where agents work, and let it warn for a week. Each warning points at a call site, a name or a line that the change left behind, before it reaches review. The vulkro ctx reference covers every flag.

To install, start here.