What an LLM-triaged security report should look like

Most security scanners ship you a database query result and call it a report. A list of findings, each with a CVE ID, a severity label, and a link to the upstream advisory. Some will helpfully sort by severity. The really fancy ones add a CVSS breakdown.

It's not a report. It's a printout. A real report does the work the printout offloads onto you — deduplicating, prioritising, explaining why something matters in your context, and saying what should happen next.

We think the report is the product. This post walks through what's in a Pwnkemon report, why each section is there, and the four things we deliberately don't include — because the absence of certain ceremony is as much a design decision as what we do show.

What's in the report

A Pwnkemon scan output has five sections, in this order:

1. Executive summary

The first thing on the page. Three things, each one sentence: what was found, what the overall risk rating is, which findings matter most. Written by the LLM after it's seen every finding, not concatenated from individual finding titles.

The intended reader is a developer's manager (or a compliance reviewer) who has thirty seconds. They should be able to skim it and know whether to ping the on-call engineer or close the tab.

2. Findings by severity (table)

A small table: critical, high, medium, low, info, each with a count. Boring. Useful for at-a-glance triage decisions, and the only piece of the report we expect people to copy into a slide.

3. Dependency reachability (table)

For every dependency finding we've classified, the breakdown into reachable, unreachable, and unknown. With a one-line explanation of what each means.

Reachable findings should be fixed first. Unreachable findings are present in your dependency tree but not imported by your code — no current execution path to the vulnerability. Unknown is the honest answer when we can't resolve the import chain (dynamic imports, plugin loaders); the finding keeps its upstream severity and the report says so.

4. Attack chains

The one section that LLM triage genuinely makes possible and no rule-based scanner can do. A short list of cross-finding chains: finding A combined with finding B in this specific codebase yields exploit path C.

Concrete example from a recent scan: a private key in git history + a known public reference to the corresponding service + an unauthenticated endpoint that accepts signed tokens from that key = full account takeover, without triggering any single “critical” finding in isolation.

This is the section that's most useful to security teams and the one we get the most positive feedback on. It's also the section that prompted us to ship the triage layer in the first place — without cross-finding reasoning, the scanner is just a faster printout.

5. Detailed findings (one block per finding)

Each finding gets a block. Standardised structure:

• Title — one line, written by the triage step, not the raw scanner. Includes the file path when applicable.
• Severity — after triage. The original scanner severity is preserved in the audit log for anyone who wants to compare.
• Reachability — for dependency findings only; reachable / unreachable / unknown.
• Description — why this is a finding, what the impact is, what conditions are required to exploit. Written in plain English, no CVSS jargon.
• Evidence — the raw scanner output for this finding, redacted where the evidence is itself a secret. Always present, so the developer can verify the finding without re-running the scan.
• Recommendation — numbered steps to fix. Order matters: rotation before purge, patch before version bump.

The four things we deliberately leave out

Each absence is a design decision, not an oversight.

1. CVSS vector strings

No CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H in the body. The vector string is information designed to be parsed by other tools, not read by humans, and putting it in a customer report is cargo-cult security theatre. The severity label and the prose description carry the same information without the math notation. Anyone who wants the vector can pull it from the upstream advisory we cite.

2. The model used to triage

Non-admin reports don't name the LLM (it's Claude; we'll switch when there's a better one). The model is an implementation detail. Including it in the report invites the reader to spend time worrying about which model is “best” instead of reading the findings. We track the model in the audit log so we can reproduce a scan; we don't advertise it to the reader.

3. Total scanner runtime

We don't put “scan completed in 47.3 seconds” at the top of the report. Nobody making a decision based on this report cares. It's in the audit log for our own diagnostics.

4. A pile of obvious mitigations everyone already knows

We don't end every report with “Best practices: keep dependencies updated. Use strong passwords. Enable MFA.” That kind of boilerplate is the marker of a report written for compliance theatre rather than for the engineer who has to fix something. We respect your time more than that.

Why this matters more than the scanners

Every security vendor in our space ships semgrep, trivy, gitleaks, and the equivalent. The tools are open. The advisory databases are public. What separates a useful security product from a noisy one isn't the scanners — it's what happens to the scanner output before it reaches a human.

That post-processing step is where the report becomes the product. Triage that downgrades transitive-dep noise. Reachability analysis that tells you which findings touch your code. Cross-finding reasoning that surfaces attack chains a single scanner couldn't see. A summary written for a human reader, not extracted from a JSON field. A standardised finding structure so every entry carries the four things you need to triage it (what, how bad, why now, how to fix).

If a scanner ships you raw output and asks you to make sense of it, you're paying for distribution — not for security work. The work was supposed to be done for you.

See it on your own code

Every Pwnkemon scan produces a report in this shape. Quickest way to see one is to sign up, verify a repo, and run a Quick scan from the dashboard. Free tier gets you enough credits for one scan a month, with the full report format (minus the “not for compliance use” watermark).

For CI-driven scans on every PR, see the GitHub Action docs. The report posted as a PR comment uses the same triage path; only the formatting differs (table-and-summary in the PR comment; full per-finding blocks on the dashboard).