Specifically, certspotter no longer terminates unless it receives SIGTERM
or SIGINT or there is a serious error.
Although using cron made sense in the early days of Certificate
Transparency, certspotter now needs to run continuously to reliably keep
up with the high growth rate of contemporary CT logs, and to gracefully
handle the many transient errors that can arise when monitoring CT.
Closes: #63Closes: #37Closes: #32 (presumably by eliminating $DNS_NAMES and $IP_ADDRESSES)
Closes: #21 (with $WATCH_ITEM)
Closes: #25
This feature will likely be removed in the future.
This feature can help you identify certificates that are
issued before you take ownership of a domain, helping you identify
certificates that are definitely not yours.
However, in practice this doesn't have very much utility:
1. Such certificates are probably already in CT when you start monitoring,
requiring you to download ALL certificates (by omitting -start_at_end)
to find them, which is not very practical.
2. It doesn't detect certificates that are issued based on reused domain
validations that were completed before you took ownership of the domain.
Since first writing certspotter, I have witnessed many people
misunderstand precertificates, and do very bad things like ignore
precertificates under the invalid rationale that "precertificates
are not trusted by browsers." While it's true that precertificates
themselves are not trusted by browsers, a precertificate implies the
existence of a corresponding certificate that IS trusted by browsers,
and there is NO guarantee that this certificate will ever be logged to CT.
(Sectigo and Let's Encrypt do log certificates but it's on a best-effort
basis and I don't know of any other CAs that do so.) Therefore, if
you ignore precertificates you WILL fail to be alerted about potential
security threats.
While some PKI/CT researchers may care about the distinction between
certificates and precertificates, certspotter's primary purpose is to
help domain owners monitor their domains for misissued certificates.
Since there is no need to distinguish between certificates and
precertificates for that use case, I am removing $CERT_TYPE to prevent
people from shooting themselves in the foot. Those who do have a valid
use case for distinguishing between certificates and precertificates
can always parse $CERT_FILE themselves.
This permits us to detect JSON marshalling errors, and makes it easy to
retry the request.
Request bodies are short so this should have negligible performance impact.
RFC 6962 doesn't define how to generate a consistency proof in this case,
and it doesn't matter anyways since the tree is empty. The DigiCert logs
return a 400 error if we ask for such a proof.
When Cert Spotter starts monitoring a log that it has never monitored before,
it can either start monitoring it from the beginning, or seek to the end and
start monitoring there.
Monitoring from the beginning guarantees detection of all certificates, but
requires downloading hundreds of millions of certificates, which takes days.
With the new -start_at_end option, you can save significant time by
starting at the end. You will miss certificates that were added to a
log before Cert Spotter starts monitoring it, but you can always use the
Cert Spotter API <https://sslmate.com/certspotter/api> or crt.sh to find them.
Previously, the -start_at_end behavior was implied the first time you
ever ran Cert Spotter. This is no longer the case.
Andrew Ayer