When three 'medium' findings are actually one critical

Give a raw scanner a repo with an outdated crypto library and it will hand you four rows:

HIGH   cryptography  CVE-2024-aaaa  upgrade to >= X
HIGH   cryptography  CVE-2024-bbbb  upgrade to >= X
MEDIUM cryptography  CVE-2024-cccc  upgrade to >= X
MEDIUM cryptography  CVE-2024-dddd  upgrade to >= X

Four rows, four severities, four “upgrade to” lines. A developer reads that as four problems. A senior pentester reads it as one problem, one package, one upgrade, and immediately asks the question the scanner didn't: does that upgrade actually install?

The gap between those two readings is the gap between a scanner and a report someone with security judgment would actually sign. Closing it is the entire point of our remediation work. Here's what “a real pentester wouldn't call this shit” looks like as engineering.

Problem 1: N rows that are really one action

Four CVEs in one package almost always have one fix: bump the package past the highest fixed version. Presented as four rows, they inflate your finding count, split your attention, and make a single-line change look like a four-item backlog.

Cross-CVE synthesis collapses them. Our report renders one combined action, “upgrade cryptography to the version that clears all four advisories”, with the individual CVEs listed underneath as what that action resolves. One decision, not four. The count on the summary table reflects actions you need to take, not rows a database returned.

Problem 2: the fix that doesn't resolve

Here's the one that separates a real report from a reformatted scanner dump. Suppose the fix is “upgrade cryptography to 50.0.0”, but another dependency in your tree pins sslyze<47, and sslyze in turn caps cryptography below 47. The advice is impossible. You cannot install the recommended version without a dependency conflict, and a scanner that never tried to resolve the upgrade will happily tell you to do it anyway.

We caught exactly this in our own self-scan, which is what kicked off the whole program. So the report now does impossible-fix detection: it checks the recommended upgrade against your actual manifest pins and, when a transitive constraint blocks it, says so, by name. Instead of “upgrade to 50.0.0” you get “fix blocked: sslyze<47 caps this package below the fixed version; the real remediation is to drop or replace sslyze.” That's advice a human can act on. “Upgrade to a version that doesn't install” is advice that wastes an afternoon.

Problem 3: confident severity on things you can't confirm

Some findings arrive with no CVE and no fixed version, a dependency flagged by reputation, or a heuristic, where the honest state of knowledge is “we can't independently confirm a severity.” Printing those as HIGH is fake precision. It's the fastest way to lose a security reader's trust, because the first thing they'll do is check one, find the body text contradicts the badge, and discount the whole report.

So we enforce severity honesty: a finding with no CVE and no confirmable fix is rated UNKNOWN and tagged unverified, not laundered into a number we can't defend. A report that admits what it doesn't know is more useful, and more trusted, than one that pretends to certainty. This is the same principle as our report philosophy applied to severity itself.

Problem 4: “reachable” that isn't

The hardest one. A classic example: a crypto library ships a CVE in its PKCS#7 handling, and the scanner marks it reachable because your code imports the library. But you only use that library's symmetric Fernet API, you never touch PKCS#7 at all. Import-level reachability (“do you import the package?”) says reachable. The truth is it isn't, and a pentester who reads the CVE and then greps your code will call that finding a false positive in about thirty seconds.

Our reachability analysis started at import level, which is already enough to cut most transitive-dependency noise. We're extending it to function level: a curated map from a CVE to the specific vulnerable symbol, plus call-site detection in your source, so “you import the package” becomes “you actually call the vulnerable function.” When the vulnerable symbol is never called, the finding is downgraded with that reasoning attached, rather than sitting at the top of your list forever.

Why this is an LLM job, partly

The synthesis, deciding that four rows are one action, that a pinned transitive dep blocks a fix, that a finding's own evidence doesn't support its severity, is exactly the reasoning a rules engine can't do and a human does in their head. It's the same argument we made in why we put an LLM in the triage loop, with one important guardrail: the deterministic checks (manifest-pin resolution, symbol call-site detection) run first and for free, and the model reasons over their output rather than being asked to guess. The expensive, fallible step only adjudicates what the cheap, exact steps can't settle.

The honest limits

• Function-level reachability needs a curated CVE-to-symbol map. It's accurate where the map exists and falls back to import-level, safely, where it doesn't. It is not a full taint analysis, and we don't claim it is.
• Fix verification proves the upgrade resolves, not that it won't break your tests. We can tell you the version installs cleanly against your manifest; we can't tell you your suite still passes. That's still your call.
• When we're unsure, we say so. Ambiguous reachability stays at the upstream severity and is labeled unknown, on purpose, so you never mistake “we didn't confirm” for “it's fine.”

The bar we hold ourselves to is simple: a competent pentester should never be able to look at a Pwnkemon report and immediately prove something in it false. Every recommendation is de-duplicated, checked against your real dependency tree, honestly rated, and reasoned as close to the function level as we can get, or explicitly marked unverified.

Run one against your own repo and read the remediation section, that's where the difference shows. See pricing, or add the GitHub Action to gate it on every PR.