Let’s Encrypt is a fairly popular service offering free SSL/TLS certificates to those who are uninterested in the value-add of traditional certificate resellers. Historically, this CA has partnered with IdenTrust to provide this service as it has existed in the root stores of client machines for many years. Let’s Encrypt’s strategy was dualfold: their certificates were signed by their own CA while it gained traction, and cross-signed by IdenTrust to maximize compatibility with older clients. This strategy was a clever one, and indeed, once this transition completes, Let’s Encrypt will be a more robust service. However, it will likely be a painful transition for those that use Let’s Encrypt for their own services, or those who consume services which use Let’s Encrypt.
When should I expect problems?
There are not one, but two dates savvy administrators must be aware of. While the root certificate Let’s Encrypt relies upon doesn’t expire until September 1st of 2021, as early as January of 2021 services may begin to phase out the certificate from client trust stores via software updates. Certain legacy software does not handle cross certification gracefully. SSL/TLS clients decide to trust or not trust a server by attempting to construct a valid “certificate chain” from a public key present in their root trust store, down through one or more intermediate certificates, eventually electing to trust or not trust the node (leaf) certificate they are attempting to verify. While robust client-side implementations continue to greedily validate possible paths AFTER finding a failure, many older implementations will return a failure if any invalid path is found! It is very difficult to ascertain which legacy client implementations suffer from this common flaw. This was an issue relatively recently when a commonly relied upon root certificate from Sectigo (formerly Comodo) expired. Unfortunately, legacy software is unlikely to receive regular software updates, so the pruning of expired certificates from the root certificate store is unlikely to occur.
What do I need to do?
Owners of legacy software as a best practice should be aware of the expiration of long-lived certificates in their client’s trust store and prune them regularly (starting in a test system for good measure).
Users providing services secured by Let’s Encrypt should prepare to alert their users of the possibility of problems for legacy clients. Specifically, Android 7.1.1 and below will receive trust errors after September 1st 2021. For companies that require compatibility with Android clients older than 7.1.1 on non-managed devices, the only recourse is to switch to a different provider of SSL/TLS certificates. For maximum client compatibility, DigiCert comes with the greatest client compatibility in the industry.
For managed devices, Administrators should insure that the new Let’s Encrypt root certificate is present on the device. MDM solutions allow for surgical insertion of trusted root certificates if needed, though as Android 7.1.1 is out of support, administrators should consider upgrading field devices sooner than later to avoid this complexity. Alternatively, while not ideal for many users, Firefox for mobile regularly updates root certificates out-of-band of the OS’s certificate store.
What about well-curated certificate stores?
Some organizations with very stringent security requirements do not take on faith the bundle of root authorities that ships with OSes or software. This is often wise – any root authority can issue a certificate for any website, and the compromise of any root CA’s private key undermines the entirety of the internet. While at first blush mechanisms such as DNS CAA records may appear to protect against this doomsday scenario, it is up to CA’s to respect them! If a CA’s private key were to become compromised, by an attacker or even a nation state, nothing prevents them from posing as any site. For those who groom their certificate stores carefully, the following powershell one-liner helps display any root certificate expiring within a year. While those who offload this decision to Microsoft receive root certificate store updates, those who refuse those updates need a way to stay on top of this.
get-childitem Cert:\CurrentUser\AuthRoot | where-object {$_.NotAfter -lt $(get-date).addDays(365)}For other client systems and software, similar scripted approaches may or may not be possible.
Looking Forward
Make no mistake, despite these growing pains, this represents the next chapter for Let’s Encrypt. Future root certificate expirations can be managed more gracefully by virtue of them no longer being dependent on a third party. Let’s Encrypt’s CA structure moving forward includes two “Generations” of root certificates – their active X1 root (expiring in 2035) and their upcoming X2 root leveraging elliptic curve cryptography (expiring in 2040). The root certificate private keys, as is best practice ,are kept offline. These root certificates sign a handful of intermediate certificates which are actually used to sign client certificates via the ACME protocol. The earliest of these certificates is set to expire in 2025. Hopefully by that time legacy client implementations are phased out, or else we will need to have this conversation with our end users once again!
Looking Forward
letsencrypt.org/2020/11/06/own-two-feet.html
www.ssltrust.com/blog/understanding-certificate-cross-signing