Pwnkemon
← All posts
Architecture·24 May 2026·6 min read

Why we don't run scanners on your GitHub runner

Every other security GitHub Action runs scanner code on your CI runner — the same runner that holds your deploy secrets, your registry tokens, and (depending on your workflow) your cloud credentials. We don't.

When you add the Pwnkemon Scan Action to a workflow, the GitHub-hosted runner makes one HTTP call, polls, prints a comment, and exits. The actual scan — the part that clones a repo and runs untrusted-input-handling tooling over it — happens on infrastructure we control, in an ephemeral container that's destroyed the moment the scan completes. Your runner never sees scanner code. Your runner never sees your repo source going through scanner code.

We think that's the right tradeoff. This post explains why.

What “runs in your CI” actually means

A typical security-vendor GitHub Action looks something like this:

- uses: someVendor/scan-action@v3
  with:
    api-key: ${{ secrets.VENDOR_API_KEY }}
    scan-type: code

Innocuous. What it actually does, behind the scenes, is:

  1. Download a multi-hundred-megabyte scanner binary onto the runner.
  2. Run that binary, as your runner's user, against the checked-out source.
  3. POST the findings to the vendor's API and exit.

Step 2 is the interesting one. The scanner binary is now a process inside your CI environment. It has the same ambient permissions every other step in your workflow has — which is to say, it can read every environment variable, see every secret currently bound to the job, touch the same filesystem your build artifacts live on, and call out to anything the runner's network policy permits.

For your build step that's fine. You wrote it; you trust it. For a third-party scanner binary published by a vendor you've outsourced your security posture to, it's a meaningful trust assumption. A compromised vendor build — or even a non-malicious bug that exfils environment snapshots in error reports — lands directly in the environment that holds your production credentials.

What we do instead

Our Action does only the three things the CI environment legitimately needs to do:

  1. Call our public API with a Pwnkemon token, asking us to scan a specific commit of a specific repo.
  2. Poll the same API until the scan is complete.
  3. Read the result, post a PR comment, fail the build if findings cross your configured severity floor.

The Action's entire source is open. It's a few hundred lines of TypeScript bundled into one file (dist/index.js in the public repo). Anyone who wants to audit what runs in their CI can read it in fifteen minutes. It imports only the two GitHub-provided helper libraries (@actions/core and @actions/github) and calls our API over HTTPS. That's it.

The scan — the part of this system that does anything complicated, that downloads scanner toolchains, that clones a repo, that interprets a Dockerfile or a package-lock.json — runs on our infrastructure. Each scan runs in a fresh Docker container that's destroyed when it exits. The container holds no long-lived credentials. The host running the containers accepts no inbound connections, polls our API for work, and has no way to reach our database or our other customers' data even if a scan somehow exploited the container boundary.

The four properties this buys

Concretely, here's what shipping a scan via our Action gives you that the alternative model doesn't:

1. Your CI secrets aren't in the threat model. Whatever's loaded into the runner's environment when the Pwnkemon step runs — npm tokens, AWS keys, deploy SSH keys — is irrelevant. Our code never reads any of it because our code never runs on your runner.

2. You don't pay CI minutes for scan runtime. A standard scan takes about thirty seconds of GitHub-runner time regardless of how big your repo is. The runner makes one call, polls, prints. The actual scanner pipeline — osv-scanner for dependencies, semgrep for SAST patterns, trivy for IaC and container layers, gitleaks for git history, an LLM triage pass — runs on infrastructure we've sized for it. On a large monorepo this can be the difference between $0.03 of CI cost per PR and $1.50.

3. Findings come pre-triaged. Most CI scanners dump raw output into the workflow log and let your team triage from there. By the time the result reaches your PR, ours has already had an LLM pass over it that downgrades unreachable transitive dependencies, deduplicates near-misses, and ranks by actual exploitability. A “high” in a Pwnkemon PR comment is one you actually need to look at — not noise dressed up in a severity label.

4. The scan is pinned to the commit, not the branch. A subtle one. When your CI fires on a PR, the “current state of the PR” is a specific SHA. But if your scanner is naive about this and just clones the branch by name, a fast-moving branch can advance between trigger and clone — and the scan you get back is of whatever was on the branch when the clone landed, not what the PR actually contains. We pin to the SHA from the workflow context, so the scan you get is always of the code the PR run was actually about.

What this costs us, honestly

The price of this architecture is that we have to run the scanner infrastructure ourselves. We can't lean on the free CPU sitting in your GitHub Actions allowance. That cost is real. We'd rather absorb it than ask you to absorb the risk of running unsandboxed third-party scanner code next to your deploy keys.

There's also a smaller drawback: a scan involves an outbound network round-trip from your runner to our API and back. If we're down, your CI step fails. We mitigate with the obvious: clear status communication, an uptime page, and an Action input that lets you set the scan as advisory rather than blocking if you'd prefer not to gate deploys on us being up.

Why we're writing this down

We don't love that “runs in your CI” is the default for our category. It's a tradeoff every other vendor made early, mostly to save on infrastructure costs, and the industry has normalised the resulting risk profile. We'd like that to be less normal. If you've added a security scanner Action to your workflow and never thought through what it has access to inside your runner, that's worth thinking through once — and the answer matters whichever vendor you choose.

If you want to see the Action's source before you adopt it, it's at github.com/Pwnkemon/pwnkemon-scan — MIT licensed, ~250 lines of TypeScript, the bundle that runs in your CI is checked in at dist/index.js and anyone can review it. Setup instructions in the Action docs.