Why did our certificate expire when we have auto-renew? TLS posture drift from the outside

DedSec Security Research ·

The alert arrives on a Saturday: the mobile app cannot reach api.example.com, and the error says the certificate has expired. Nobody expected it, because certificates here renew themselves. The renewal job's log says it succeeded seven weeks ago, and the certificate file on disk is new. The certificate the server presents is not.

This post is about the gap between the certificate you hold and the certificate you serve. An outside check sees only what a client is actually handed, and that is the side that causes outages.

What an outside handshake actually sees

Any client can connect to port 443, name a host, and learn which protocol versions and cipher suites the server negotiates, which certificate chain it sends, and, from the HTTP response, headers such as Strict-Transport-Security. None of this needs a credential.

Three inputs shape the answer: the address, the name sent in the Server Name Indication extension (RFC 6066), and what the client offers. Change any one and the answer can change, and four nodes behind one address can give four answers.

So your TLS is not one configuration but a set of them, and the useful checks compare members of the set with each other and over time.

Why the certificate expired when renewal is automated

Automated renewal is a pipeline: request, validate, issue, store, deploy, reload. Each stage can fail on its own, and a failure after the first one is usually silent.

Renewed but never served

Most web servers and proxies read the certificate at start or reload and keep it in memory, so a renewed file on disk changes nothing until the process reloads. A renewal client with no deploy hook, or one that reloads the web server but not the mail server or sidecar reading the same file, leaves the old certificate in service until it expires.

Load balancing makes this intermittent. Renewal runs on one node, or a sync job copies the new certificate everywhere and fails on one. Some connections get the expired certificate and retries succeed. From outside, the symptom is different serial numbers and expiry dates on repeated handshakes to the same name, and a check that makes one handshake per host passes or fails depending on which node it reached.

Or renewal happens at the wrong layer: the origin renews correctly, but TLS terminates at a load balancer or CDN holding a hand-uploaded copy nothing automates.

Renewal failing quietly

ACME, the protocol behind most automated issuance (RFC 8555), has the CA confirm control of the name before issuing. Both common challenge types can break after they have worked for years.

Clients typically start renewing weeks before expiry and retry each run, logging failures where nobody looks. The usual backstop was the CA's expiry email, and Let's Encrypt stopped sending those in 2025, so teams that relied on them lost their last warning without changing anything.

The host outside the automation

Automation covers the hosts someone enrolled. The microsite with a hand-bought certificate, the vendor-hosted status page under your name and the appliance with its own upload form are not in the renewal system, so its dashboard stays green. An outside view finds them because it starts from names, not from that system's list.

Shorter lifetimes make this more frequent

CA/Browser Forum Ballot SC-081, adopted in 2025, cuts the maximum validity of publicly trusted TLS certificates in steps, from 398 days to 200 days from 15 March 2026, 100 days from 15 March 2027 and 47 days from 15 March 2029. A host renewed by hand once a year becomes a recurring outage risk on a much shorter cycle, which is a reason to find those hosts now.

The chain that works in a browser

A server should send its leaf certificate and the intermediates that link it to a trusted root. If it sends only the leaf, many browsers still succeed. Some fetch the missing intermediate from the URL in the certificate's Authority Information Access extension, and some already hold common intermediates. Many non-browser TLS stacks do neither: curl and other OpenSSL-based clients, Java, many language runtimes, many embedded clients and some mobile apps, notably on Android, fail with an error such as unable to get local issuer certificate.

So the site works for every engineer with a browser and fails for the partner integration, the Android app and the webhook sender. Common causes are deploying the leaf file instead of the full-chain file, and a hand-assembled bundle still carrying an old intermediate after the CA changed issuing intermediates.

A useful external check validates what the server sent, in the order sent, against a plain trust store without fetching anything missing, and keeps the served chain as evidence.

Name coverage: SANs, wildcards and the default certificate

Clients match the hostname against the DNS names in the certificate's subjectAltName extension. Modern clients ignore the subject common name for this purpose. The rules are in RFC 9525, which replaced RFC 6125.

A wildcard covers exactly one label: *.example.com matches api.example.com but not example.com or v2.api.example.com. Mismatches come from that rule, from a new subdomain pointed at a load balancer whose certificate does not list it, and from a CNAME to a SaaS provider serving a certificate for its own name.

A client that sends no SNI, or a name the server does not recognise, gets the default certificate: perhaps a vendor's self-signed placeholder, or on shared infrastructure one listing other names hosted there. It is also what a scanner sees when it connects by address alone.

Certificate Transparency: your inventory, and theirs

Publicly trusted CAs record certificates in public, append-only Certificate Transparency logs (RFC 6962, with version 2 in RFC 9162), and major browsers require proof of logging before they trust a certificate. As a result, every name in a publicly trusted certificate is published and remains searchable by anyone, long after the certificate expires. That includes names that were never deployed.

For a defender, that is a discovery source: hosts outside the automation, certificates other teams obtained quietly, and certificates you never requested, from an issuer you do not use or for a name you do not recognise. Pair it with CAA records (RFC 8659), which tell CAs which of them may issue for your names.

For an attacker, it is the same list. Names such as vpn-legacy or jenkins-staging reveal internal structure and point at likely-neglected systems. Wildcard certificates keep labels out of the logs but usually share one private key across every host using them, so compromising one host exposes the key for all of them. That is a trade-off, not a fix.

CT has limits as an inventory. It covers only names that received publicly trusted certificates, so names under a private CA, or reachable only through a wildcard, never appear. And a name being logged does not mean it is deployed or still live.

Protocol and cipher drift

RFC 8996 (2021) deprecates TLS 1.0 and 1.1, leaving TLS 1.2 and TLS 1.3 (RFC 8446). Main endpoints were mostly fixed years ago; drift lives on the one nobody fixed, such as the legacy API kept for an old client, an appliance's management interface or a host built from an old image.

Beyond versions, look for key exchange without forward secrecy (static RSA, which TLS 1.3 removed) and RC4 (prohibited by RFC 7465). Which TLS 1.2 suites to accept is policy; the external question is whether every endpoint matches the policy you think you have.

Edge versus origin

With a CDN or cloud load balancer in front, there are two TLS configurations: the edge's, which clients normally see, and the origin's, used between the edge and your server. If the origin also answers on a public address, anyone can negotiate with it directly. It often has older protocols enabled, and a certificate that is expired or self-signed because only the CDN was supposed to connect. A direct connection also bypasses whatever the edge enforces, such as WAF rules and rate limits.

Origin addresses leak through historical DNS, CT entries for origin hostnames and internet-wide scans that map certificates to addresses. Hardening the origin's TLS helps, but the fix is to accept connections only from the edge, by address range or by having the edge authenticate to the origin.

HSTS: what it protects and where it stops

HSTS, specified in RFC 6797, is a response header such as Strict-Transport-Security: max-age=31536000; includeSubDomains. Once a browser has received it over HTTPS, for max-age seconds it rewrites http:// requests to that host as https://, and it makes certificate errors on the host fatal, with no click-through. Browsers ignore the header when it arrives over plain HTTP, and max-age=0 tells the browser to forget the host.

The gaps are specific and visible from outside:

includeSubDomains has a cost: every subdomain must then serve valid HTTPS, including the internal HTTP-only tool that resolves under your domain, or browsers that hold the policy will refuse to load it.

Preload is a one-way door, almost

The HSTS preload list is built into browsers, so it protects even the first visit. The preload directive is not part of RFC 6797; it is a convention of the list. Its submission requirements include a valid certificate, a redirect from HTTP to HTTPS on the same host if it listens on port 80, HTTPS on every subdomain, and an HSTS header on the base domain with a max-age of at least one year (31536000 seconds), includeSubDomains and preload. Removal means filing a request and waiting for it to reach browser releases, and installed browsers that are not updated keep the entry. Preloading commits the whole namespace to HTTPS for a long time, so inventory every subdomain before you submit.

Drift: why point-in-time checks miss it

Expiry is the TLS failure you can see coming from the certificate itself; most of the others are change. A point-in-time check records one handshake, at one moment, from one vantage point. It misses the node that answers only some connections, the configuration pushed after the scan and the proxy change that dropped the HSTS header.

Comparing runs catches what a single run cannot:

To be actionable, each observation needs its context: the served chain, the negotiated protocol and cipher, the headers, the time, the address and the SNI name. That lets the owner find the wrong node without repeating the investigation; the broader case is in why an EASM finding needs evidence attached.

What an external check cannot see

Fixing it, in order

  1. Make deployment part of renewal: a deploy hook that reloads every process reading the certificate, confirmed by handshake rather than by the file.
  2. Serve the full chain, tested with a client that does not fetch intermediates; openssl s_client with -servername prints the chain as served.
  3. Reconcile CT against your inventory, enrol or retire each unknown name, and publish CAA.
  4. Turn off TLS 1.0 and 1.1 everywhere, and accept origin connections only from the edge.
  5. Deploy HSTS per host, add includeSubDomains once every subdomain serves valid HTTPS, and preload last.

Where Maphra fits

Maphra starts from a domain name and works outward, enumerating subdomains and attributing each discovered asset back to the organisation that owns it. Certificate transparency monitoring is part of that discovery, and so is change detection between runs: Maphra re-runs discovery on a schedule, diffs the result against the last known surface, and reports what appeared, what changed and what quietly disappeared. Every discovered asset is checked for exposure, including TLS and certificate posture and security headers, and each finding carries the raw response that produced it. Findings are deduplicated across scans and tracked through new, reopened and fixed states.

What to ask of any external TLS check

These questions apply to any tool, this one included, and the limits above apply to any external check, ours included. Deploy hooks, origin lockdown and the preload decision are work for your own team.

A renewal job that reports success is a statement about a file. Only the handshake shows what your users actually get.

More about Maphra · All posts