DedSec Application Security · 2026-10-15
Two questions arrive at a security team in roughly this form. The first comes from a developer: "I committed an API key, noticed straight away and removed it in the next commit. Am I fine?" The second comes a few weeks later from someone else: "Why is the secrets scanner flagging a key that is not in the code any more? I searched the whole repository and it is not there."
They are the same question asked from opposite ends, and the answer to both is that removing a line from the current version of a file does not remove it from the repository.
The short answers, before the mechanism: if the commit was ever pushed anywhere, treat the credential as exposed and revoke it. And the scanner is flagging the key because it is still there, in a place a text search of the working tree does not look.
Git does not store changes. It stores snapshots, and it stores them as content-addressed objects.
Commit a configuration file containing a key, and git writes a blob with the key, a tree pointing at it, and a commit pointing at the tree. Delete the line and commit again, and git writes a new blob, a new tree and a new commit whose parent is the old one.
Nothing in that second step touches the first blob. The old commit still points at the old tree, which still points at the old blob, which still contains the key. Anyone with the repository can read it with a log command that prints patches. The diff for the second commit, a red line removing the key, is itself a signpost to where it was.
Cloning copies the object graph. Every full clone made after the push holds the blob, as does every fork, mirror, backup and CI cache that fetched that history while the commit was part of it. So "it was only up for a minute" is not much comfort: the exposure is not how long the key sat in HEAD, but every system that copied the object while it was reachable.
Suppose the key was added in one commit and removed in the next, both on the same branch, and the pull request is opened afterwards. A scan of the net difference between the base and the head of the branch sees neither event, because the net difference contains no key. The secret is in the branch's history and in no diff the scanner was shown. A scan of each commit in the pull request would catch it; a scan of the combined diff will not.
If the key went in through one pull request and came out through a later one, a diff scan of the second sees only a removal, which scanners looking for introduced secrets do not report.
Squash merging adds a twist: the default branch may never contain the commit with the key, but the pull request's original commits do not necessarily vanish: hosting platforms commonly keep references to pull-request heads, and the objects remain on the host and in any clone of the contributor's branch. Whether those references remain fetchable, and for how long, is host-specific, so treat the commits as published.
A history scan covers all of this by reading every commit's changes. The cost is that it also finds every secret ever committed, including ones revoked years ago.
The instinctive fix is to rewrite history: tools such as git filter-repo, or the BFG Repo-Cleaner, rewrite every commit that contained the secret so the blob is no longer referenced, then the rewritten branches are force-pushed. This is a legitimate operation, and occasionally a necessary one. It does not undo the exposure, for reasons that follow directly from the object model.
The credential has to be assumed copied from the moment of the first push, so the only remediation that ends the exposure is on the credential, not the repository: revoke it and issue a new one. Then work out what the old one could reach and check its usage logs for the period it was exposed, where the provider keeps them.
Rewriting history after that is an optional clean-up step, worth its cost of a re-clone for every collaborator mainly when the repository is public or widely forked and the key would otherwise keep being rediscovered.
The exception is a commit that never left your machine. If it was never pushed to any remote, amending or resetting it locally before pushing is enough, because nothing else holds a copy. Once it has been pushed, even to a private repository, everyone with read access, every integration that clones it and every CI run is in the set.
The same logic applies outside git: a container image layer that adds a key still ships it after a later layer deletes it, and CI logs, build artifacts and published packages hold copies no commit reaches.
Secret detection combines three techniques, plus the configuration that tunes them, and their failure modes differ enough that it matters which one produced a finding.
Provider-specific patterns. Many providers issue credentials in a recognisable shape. AWS access key IDs for long-term credentials begin with AKIA. GitHub personal access tokens in the classic format begin with ghp_, and fine-grained ones with github_pat_. Most Slack tokens begin with xox followed by a letter that indicates the token type. A PEM-encoded private key starts with a recognisable header line, such as one beginning BEGIN RSA PRIVATE KEY or BEGIN PRIVATE KEY. A rule for each shape, sometimes with a length and character-set check, is precise: a match is very likely to be the thing it claims to be.
The limits are in what has no shape. The AWS access key ID is an identifier; the part that actually authenticates is the secret access key, a forty-character string with no prefix, so a detector has to find it by context, such as proximity to an access key ID or a variable name. A database password in a connection string, a shared HMAC key, or an internal service token generated by your own code has no provider prefix at all. Pattern detection also only finds the providers someone wrote a rule for.
Generic entropy. The fallback for shapeless secrets is to look for strings that look random, measured by Shannon entropy over the characters, usually only in positions that suggest a secret, such as a value assigned to a name containing key, token, secret or password. This finds things pattern rules miss, at a cost in both directions.
False positives come from everything else in a repository that is legitimately random-looking: commit hashes and content hashes, lockfile integrity hashes, UUIDs, base64-encoded images and fonts, minified JavaScript, test fixtures and example values that are random by design. False negatives come from secrets that do not look random to a character-frequency measure: a short human-chosen password made of a word, a year and a punctuation mark cannot score highly, because a short string's Shannon entropy is capped by its length, and it passes straight through a threshold tuned to ignore ordinary identifiers.
Verification. Some tools go a step further and test a candidate against the provider. TruffleHog is the best-known open-source example: many of its detectors can make an authenticated request to the provider's API with the candidate credential and report whether the provider accepted it.
What verification buys is a change in kind: a verified secret is not a pattern match that might be a placeholder, it is a working credential in your repository, which is an incident rather than a finding. Its limits are worth stating exactly:
Rules, allowlists and ignores. Gitleaks illustrates the configurable rule-based approach. Its rules combine regular expressions, keywords and optional entropy thresholds, and its configuration supports allowlists by path, by regular expression and by commit. A line can be exempted with an inline gitleaks:allow comment, and individual findings can be ignored by listing their fingerprints in a .gitleaksignore file. Like any suppression, each is keyed to something, and the key decides how it decays: an inline comment travels with a copied line, a path allowlist for a test directory covers every future secret placed there, and a fingerprint list is exact but must be maintained as history and rules change. The post on suppression at scale covers that decay in general.
Running more than one detector is reasonable, because each catches things the other misses. It also multiplies the output in two directions.
Across engines. The same string in the same file at the same commit is matched by both tools, under different rule names, often with slightly different start and end positions. Without deduplication that is two findings for one credential.
Across history. History scanners generally read each commit's changes, so a secret is reported where it was introduced. It is reported again wherever it was introduced again: copied into a second configuration file, re-added after a revert, or pasted into a test. One carelessly handled credential can appear at many commits and paths.
The unit of work is the credential, because one revocation fixes every occurrence. A queue with one row per match makes the reviewer rediscover that repeatedly, so the choice of fingerprint is the central design decision. There are two families.
Keyed to location: commit, path, rule and line, which is roughly how a Gitleaks fingerprint is built. This is exact and needs no knowledge of the secret's value, but every occurrence is a separate fingerprint, the same secret under two engines' rule names is two fingerprints, and a suppression for one does not cover the others. It answers "which occurrence is this?" well and "which credential is this?" not at all.
Keyed to the value: a fingerprint derived from the secret string itself, optionally together with its type. All occurrences, across engines, commits, paths and even repositories, collapse into one record, and committing a rotated replacement correctly produces a new one. The trade-off is that a fingerprint derived from the secret has to be handled as sensitive. A plain unsalted hash of a low-entropy password can be reversed by guessing, and a plain hash shared across tenants would let one tenant test whether another holds a given value. A keyed hash, using a secret held by the system rather than in the findings store, avoids both. Normalisation matters too: the same key quoted, unquoted or with trailing whitespace should produce the same fingerprint.
In practice a useful model keeps both: the credential as the unit that is triaged and closed, and the occurrences underneath it as the evidence of where it appeared, because the investigation needs the first commit, its date and its author.
With the credential as the unit, the order of work mostly follows from what each finding proves.
Age matters in one direction only: a key committed years ago and never rotated has been exposed for years, and a new one is not lower priority for being new.
For most findings, "fixed" means the next scan no longer detects the problem, and since code findings are fixed by changing code, that is reasonable evidence. For a secret in history, the inference fails in both directions.
So the closing state of a secret finding should be tied to the credential, not the code: revoked, with a record of who revoked it and when, or not a credential, with a reason. Both are claims someone should have to stand behind, which is where a second approver earns its place. Removal from HEAD should be recorded as what it is, a clean-up, and should not close anything.
Re-detection deserves care too. The old occurrences of a revoked key should not reopen it on every history scan, but the same credential, by value, appearing in a new commit is a regression on something believed handled, and should come back as the same finding, reopened, rather than as an unrelated new one.
Before the commit. A pre-commit hook that scans staged changes stops a secret before it enters history, which is the only point at which removing it fully undoes the mistake. Both TruffleHog and Gitleaks document pre-commit use. It is cheap, and advisory: it runs only where someone installed it, and it can be skipped.
On push and in the pull request. A server-side or CI check cannot be skipped by the developer. It should scan each commit in the push or the pull request, not just the net diff, for the add-then-remove reason above. If it runs in CI, check the clone depth: a shallow checkout gives a history scanner almost no history to read and a clean result to report.
Full history, as the backstop. Run a full-history scan when a repository is first brought under scanning, because whatever was committed before any hook existed is only visible that way. After that, scanning incrementally from the last scanned commit is much cheaper, with two caveats. A force-push can replace the commit the incremental scan was anchored to, so the scanner needs to notice and fall back to a wider scan. And history is more than the default branch: other branches and tags carry their own commits, and a scan that only follows one branch misses secrets that never merged.
Re-run full history occasionally anyway: the history has not changed, but the rule set has, and a secret type nobody had a rule for at onboarding may have one now.
AppSecD's secret detection runs TruffleHog and Gitleaks, deduplicated, over the working tree and the full git history, covering API keys, tokens, private keys, and database and cloud credentials. It reads the git history, not just the diff, because a credential deleted in the latest commit is still in the repository. Its findings go through the same lifecycle as every other finding: deduplicated across scans, tracked through new, reopened and fixed states, and with maker-checker approval available, so that closing a finding can require a second person to confirm it.
Everything else in this post, from fingerprint design to triage order, is general practice, and the questions below apply to any secret scanner, ours included.
A scanner that keeps flagging a key that is no longer in the code is usually right. The key is still in the repository, and the useful question is not how to make the finding go away but whether the credential has been revoked.