From 04ea5c949f98ebb3edeb88e97c4e558549b46f2b Mon Sep 17 00:00:00 2001 From: Andrew Ayer Date: Sun, 5 Feb 2023 10:58:05 -0500 Subject: [PATCH] Update man pages --- man/certspotter-script.md | 276 +++++++++++++++++++++++++++++--------- man/certspotter.md | 204 ++++++++++++---------------- 2 files changed, 302 insertions(+), 178 deletions(-) diff --git a/man/certspotter-script.md b/man/certspotter-script.md index 081a7ac..978d645 100644 --- a/man/certspotter-script.md +++ b/man/certspotter-script.md @@ -2,104 +2,254 @@ **certspotter-script** - Certificate Transparency Log Monitor (hook script) -# SYNOPSIS - -**certspotter-script** - # DESCRIPTION -**certspotter-script** is *any* program that is called from **certspotter**'s -*-script* argument. **certspotter** executes this script when a file from the -CT log matches against the watchlist. +**certspotter-script** is *any* program that is called using **certspotter(8)**'s +*-script* argument. **certspotter** executes this program when it needs to notify +you about an event, such as detecting a certificate for a domain on your watch list. # ENVIRONMENT -The script will have the following variables defined in its environment: +## Event information -## Log entry information +The following environment variables are set for all types of events: -**CERT\_FILENAME** -: The path of the saved certificate on the local filesystem, if one exists. +`EVENT` -**CERT\_TYPE** -: The certificate's type (*cert* or *precert*). +: One of the following values, indicating the type of event: -**FINGERPRINT** -: The certificate's fingerprint. + * `discovered_cert` - certspotter has discovered a certificate for a + domain on your watch list. -**LOG\_URI** -: The URI of the log the certificate was found on. + * `malformed_cert` - certspotter can't determine if a certificate + matches your watch list because the certificate or the log entry + is malformed. -**ENTRY\_INDEX** -: The entry's index in the log. + * `error` - a problem is preventing certspotter from monitoring all + logs. -**CERT\_PARSEABLE** -: Whether the certificate could be parsed. + Additional event types may be defined in the future, so your script should + be able to handle unknown values. -## Identifiers +`SUMMARY` -**DNS\_NAMES** -: A comma-separated list of the certificate's dnsNames. +: A short human-readable string describing the event. -**IP\_ADDRESSES** -: A comma-separated list of the certificate's IP addresses. -## Certificate information +## Discovered certificate information -**PUBKEY\_HASH** -: The certificate public key's hash. +The following environment variables are set for `discovered_cert` events: -**SERIAL** -: The certificate's serial. +`WATCH_ITEM` -**NOT\_BEFORE**, **NOT\_AFTER** -: The certificate's validity information, as a string. +: The item from your watch list which matches this certificate. + (If more than one item matches, the first one is used.) -**NOT\_BEFORE\_UNIXTIME**, **NOT\_AFTER\_UNIXTIME** -: The certificate's validity information, as UNIX time. +`LOG_URI` -**SUBJECT\_DN** -: The certificate's subject distinguished name (DN). +: The URI of the log containing the certificate. -**ISSUER\_DN** -: the certificate issuer distinguished name (DN). +`ENTRY_INDEX` -## Errors +: The index of the log entry containing the certificate. -**PARSE\_ERROR** -: Set to the error that occurred when attempting to extract information about - the certificate. In this case, **CERT\_PARSEABLE** will also be set to "no" - and information such as **PUBKEY\_HASH**, **SERIAL**, as well as validity - and subject, will not be present. +`TBS_SHA256` -**SERIAL\_PARSE\_ERROR** -: Set to the error that occurred when attempting to extract the certificate's - serial. Emitted instead of **SERIAL**. +: The hex-encoded SHA-256 digest of the TBSCertificate, as defined in RFC 6962 Section 3.2. + Certificates and their corresponding precertificates have the same `TBS_SHA256` value. -**IDENTIFIERS\_PARSE\_ERROR** -: Set to the error that occurred when attempting to extract the certificate's - identifiers. Emitted instead of **DNS\_NAMES**, **IP\_ADDRESSES**. +`CERT_SHA256` -**VALIDITY\_PARSE\_ERROR** -: Set to the error that occurred when attempting to extract the certificate's - validity information. Emitted instead of **NOT\_BEFORE**, **NOT\_AFTER**. +: The hex-encoded SHA-256 digest (sometimes called fingerprint) of the certificate. + The digest is computed over the ASN.1 DER encoding. -**SUBJECT\_PARSE\_ERROR** -: Set to the error that occurred when attempting to extract the certificate's - subject information. Emitted instead of **SUBJECT\_DN**. +`PUBKEY_SHA256` -**ISSUER\_PARSE\_ERROR** -: Set to the error that occurred when attempting to extract the certificate's - issuer information. Emitted instead of **ISSUER\_DN**. +: The hex-encoded SHA-256 digest of the certificate's Subject Public Key Info. + +`CERT_FILENAME` + +: Path to a file containing the PEM-encoded certificate chain. Not set if `-no_save` was used. + +`JSON_FILENAME` + +: Path to a JSON containing additional information about the certificate. See below for the format of the JSON file. + Not set if `-no_save` was used. + +`TEXT_FILENAME` + +: Path to a file containing a text representation of the certificate. + Not set if `-no_save` was used. + +`NOT_BEFORE`, `NOT_BEFORE_UNIXTIME`, `NOT_BEFORE_RFC3339` + +: The not before time of the certificate, in a human-readable format, seconds since the UNIX epoch, and RFC3339, respectively. These variables may be unset if there was a parse error, in which case `VALIDITY_PARSE_ERROR` is set. + +`NOT_AFTER`, `NOT_AFTER_UNIXTIME`, `NOT_AFTER_RFC3339` + +: The not after (expiration) time of the certificate, in a human-readable format, seconds since the UNIX epoch, and RFC3339, respectively. These variables may be unset if there was a parse error, in which case `VALIDITY_PARSE_ERROR` is set. + +`VALIDITY_PARSE_ERROR` + +: Error parsing not before and not after, if any. If this variable is set, then the `NOT_BEFORE` and `NOT_AFTER` family of variables are unset. + +`SUBJECT_DN` + +: The distinguished name of the certificate's subject. This variable may be unset if there was a parse error, in which case `SUBJECT_PARSE_ERROR` is set. + +`SUBJECT_PARSE_ERROR` + +: Error parsing the subject, if any. If this variable is set, then `SUBJECT_DN` is unset. + +`ISSUER_DN` + +: The distinguished name of the certificate's issuer. This variable may be unset if there was a parse error, in which case `ISSUER_PARSE_ERROR` is set. + +`ISSUER_PARSE_ERROR` + +: Error parsing the issuer, if any. If this variable is set, then `ISSUER_DN` is unset. + +`SERIAL` + +: The hex-encoded serial number of the certificate. Prefixed with a minus (-) sign if negative. This variable may be unset if there was a parse error, in which case `SERIAL_PARSE_ERROR` is set. + +`SERIAL_PARSE_ERROR` + +: Error parsing the serial number, if any. If this variable is set, then `SERIAL` is unset. + +## Malformed certificate information + +The following environment variables are set for `malformed_cert` events: + +`LOG_URI` + +: The URI of the log containing the malformed certificate. + +`ENTRY_INDEX` + +: The index of the log entry containing the malformed certificate. + +`LEAF_HASH` + +: The base64-encoded Merkle hash of the leaf containing the malformed certificate. + +`PARSE_ERROR` + +: A human-readable string describing why the certificate is malformed. + +# JSON FILE FORMAT + +Unless `-no_save` is used, certspotter saves a JSON file for every discovered certificate +under `$CERTSPOTTER_STATE_DIR`, and puts the path to the file in `$JSON_FILENAME`. Your +script can read the JSON file, such as with the jq(1) command, to get additional information +about the certificate which isn't appropriate for environment variables. + +The JSON file contains an object with the following fields: + +`tbs_sha256` + +: A string containing the hex-encoded SHA-256 digest of the TBSCertificate, as defined in RFC 6962 Section 3.2. + Certificates and their corresponding precertificates have the same `tbs_sha256` value. + +`cert_sha256` + +: A string containing the hex-encoded SHA-256 digest (sometimes called fingerprint) of the certificate. + The digest is computed over the ASN.1 DER encoding. + +`pubkey_sha256` + +: A string containing the hex-encoded SHA-256 digest of the certificate's Subject Public Key Info. + +`issuer_der` + +: A base64 string containing the certificate's DER-encoded issuer distinguished name. + +`subject_der` + +: A base64 string containing the certificate's DER-encoded subject distinguished name. + +`dns_names` + +: An array of strings containing the DNS names for which the + certificate is valid, taken from both the DNS subject alternative names + (SANs) and the subject common name (CN). Internationalized domain names + are encoded in Punycode. + +`ip_addresses` + +: An array of strings containing the IP addresses for which the certificate is valid, + taken from both the IP subject alternative names (SANs) and the subject common name (CN). + +`not_before` + +: A string containing the not before time of the certificate in RFC3339 format. + Null if there was an error parsing the certificate's validity. + +`not_after` + +: A string containing the not after (expiration) time of the certificate in RFC3339 format. + Null if there was an error parsing the certificate's validity. + +`serial_number` + +: A string containing the hex-encoded serial number of the certificate. Prefixed with a minus (-) sign if negative. + Null if there was an error parsing the serial number. + +# EXAMPLES + +Example environment variables for a `discovered_cert` event: + +``` +CERT_FILENAME=/home/andrew/.certspotter/certs/3c/3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a.pem +CERT_SHA256=3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a +ENTRY_INDEX=6464843 +EVENT=discovered_cert +ISSUER_DN=C=GB, ST=Greater Manchester, L=Salford, O=Sectigo Limited, CN=Sectigo RSA Domain Validation Secure Server CA +JSON_FILENAME=/usr2/andrew/.certspotter/certs/3c/3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a.v1.json +LOG_URI=https://ct.cloudflare.com/logs/nimbus2024/ +NOT_AFTER='2024-01-26 03:47:26 +0000 UTC' +NOT_AFTER_RFC3339=2024-01-26T03:47:26Z +NOT_AFTER_UNIXTIME=1706240846 +NOT_BEFORE='2023-01-31 03:47:26 +0000 UTC' +NOT_BEFORE_RFC3339=2023-01-31T03:47:26Z +NOT_BEFORE_UNIXTIME=1675136846 +PUBKEY_SHA256=33ac1d9b9e56005ccac045eac2398b3e9dd6b3f5b66ae6260f2d478c7c0d82c8 +SERIAL=c170fbf3bf27481e5c351a4db6f2dc5f +SUBJECT_DN=CN=sslmate.com +SUMMARY='certificate discovered for .sslmate.com' +TBS_SHA256=2388ee81c6f45cffc73e68a35fa8921e839e20acc9a98e8e6dcaea07cbfbdef8 +TEXT_FILENAME=/usr2/andrew/.certspotter/certs/3c/3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a.txt +WATCH_ITEM=.sslmate.com +``` + +Example JSON file for a discovered certificate: + +``` +{ + "cert_sha256": "3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a", + "dns_names": [ + "sslmate.com", + "www.sslmate.com" + ], + "ip_addresses": [], + "issuer_der": "MIGPMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxNzA1BgNVBAMTLlNlY3RpZ28gUlNBIERvbWFpbiBWYWxpZGF0aW9uIFNlY3VyZSBTZXJ2ZXIgQ0E=", + "not_after": "2024-01-26T03:47:26Z", + "not_before": "2023-01-31T03:47:26Z", + "pubkey_sha256": "33ac1d9b9e56005ccac045eac2398b3e9dd6b3f5b66ae6260f2d478c7c0d82c8", + "serial_number": "c170fbf3bf27481e5c351a4db6f2dc5f", + "subject_der": "MBYxFDASBgNVBAMTC3NzbG1hdGUuY29t", + "tbs_sha256": "2388ee81c6f45cffc73e68a35fa8921e839e20acc9a98e8e6dcaea07cbfbdef8" +} +``` # SEE ALSO -**certspotter**(8), **x509**(1) +certspotter(8) # COPYRIGHT -Copyright (c) 2016-2022 Opsmate, Inc. +Copyright (c) 2016-2023 Opsmate, Inc. # BUGS -Report bugs to . +Report bugs to . diff --git a/man/certspotter.md b/man/certspotter.md index 3c39a3e..3cc90e7 100644 --- a/man/certspotter.md +++ b/man/certspotter.md @@ -4,13 +4,13 @@ # SYNOPSIS -**certspotter** [**-verbose**] [**-start\_at\_end**] [**-watchlist** _file_] `...` +**certspotter** [`-start_at_end`] [`-watchlist` *FILENAME*] [`-email` *ADDRESS*] `...` # DESCRIPTION **Cert Spotter** is a Certificate Transparency log monitor from SSLMate that -alerts you when a SSL/TLS certificate is issued for one of your domains. Cert -Spotter is easier than other open source CT monitors, since it does not require +alerts you when a SSL/TLS certificate is issued for one of your domains. +Cert Spotter is easier to use than other open source CT monitors, since it does not require a database. It's also more robust, since it uses a special certificate parser that ensures it won't miss certificates. @@ -30,163 +30,137 @@ You can use Cert Spotter to detect: # OPTIONS -**-all_time** -: Scan certs from all time, not just those logged since the previous run of - Cert Spotter. +-batch_size *NUMBER* -**-batch_size** _int_ -: Max number of entries to request at per call to get-entries. This is - advanced option. Defaults to 1000. +: Maximum number of entries to request per call to get-entries. + You should not generally need to change this. Defaults to 1000. -**-logs** _string_ -: Filename or HTTPS URL of a JSON file containing logs to monitor, in the - format documented at . +-email *ADDRESS* + +: Email address to contact when a matching certificate is discovered. + You can specify this option more than once to email multiple addresses. + Your system must have a working sendmail(1) command. + +-logs *ADDRESS* + +: Filename or HTTPS URL of a v2 or v3 JSON log list containing logs to monitor. + The schema for this file can be found at . Defaults to , which includes - the union of active logs recognized by Chrome and Apple. + the union of active logs recognized by Chrome and Apple. certspotter periodically + reloads the log list in case it has changed. + +-no_save -**-no\_save** : Do not save a copy of matching certificates. -**-num\_workers** _int_ -: Number of concurrent matchers. Default 2. +-script *COMMAND* -**-script** _string_ -: Script to execute when a matching certificate is found. See - **certspotter-script**(8) for information about the interface to scripts. +: Command to execute when a matching certificate is found. See + certspotter-script(8) for information about the interface to scripts. + +-start_at_end -**-start\_at\_end** : Start monitoring logs from the end rather than the beginning. **WARNING**: monitoring from the beginning guarantees detection of all certificates, but requires downloading hundreds of millions of certificates, which takes days. -**-state\_dir** _string_ -: Directory for storing state. Defaults to `~/.certspotter`. +-state_dir *PATH* + +: Directory for storing state. Defaults to `$CERTSPOTTER_STATE_DIR`, which is + "~/.certspotter" by default. + +-stdout + +: Write matching certificates to stdout. + +-verbose -**-verbose** : Be verbose. -**-version** +-version + : Print version and exit. -**-watchlist** _string_ -: File containing identifiers to watch. Use `-` for stdin. - Defaults to `~/.certspotter/watchlist`. +-watchlist *PATH* -# NOTES +: File containing DNS names to monitor, one per line. To monitor an entire + domain namespace (including the domain itself and all sub-domains) prefix + the domain name with a dot (e.g. ".example.com"). To monitor a single DNS + name only, do not prefix the name with a dot. + + Defaults to `$CERTSPOTTER_CONFIG_DIR/watchlist`, which is + "~/.certspotter/watchlist" by default. + Specify `-` to read the watch list from stdin. + + certspotter reads the watch list only when starting up, so you must restart + certspotter if you change it. -## Method of operation +# OPERATION -Every time you run Cert Spotter, it scans all browser-recognized -Certificate Transparency logs for certificates matching domains on -your watch list. When Cert Spotter detects a matching certificate, it -writes a report to standard out. +certspotter continuously monitors all browser-recognized Certificate +Transparency logs looking for certificates which are valid for any domain +on your watch list. When certspotter detects a matching certificate, it +emails you (if `-email` is specified), executes a script (if `-script` +is specified), and/or writes a report to standard out (if `-stdout` +is specified). -Cert Spotter also saves a copy of matching certificates in -`~/.certspotter/certs` (unless you specify the **-no\_save** option). +certspotter also saves a copy of matching certificates in +`$CERTSPOTTER_STATE_DIR/certs` ("~/.certspotter/certs" by default) +unless you specify the `-no_save` option. -When Cert Spotter has previously monitored a log, it scans the log -from the previous position, to avoid downloading the same log entry -more than once. (To override this behavior and scan all logs from the -beginning, specify the **-all\_time** option.) - -When Cert Spotter has not previously monitored a log, it can either start +When certspotter has not previously monitored a log, it can either start monitoring the log from the beginning, or seek to the end of the log and start monitoring from there. Monitoring from the beginning guarantees detection of all certificates, but requires downloading hundreds of millions of certificates, which takes days. The default behavior is to monitor from the beginning. To start monitoring new logs from the end, -specify the **-start\_at\_end** option. +specify the `-start_at_end` option. -You can add and remove domains on your watchlist at any time. However, -the certspotter command only notifies you of certificates that were -logged since adding a domain to the watchlist, unless you specify the -**-all\_time** option, which requires scanning the entirety of every log -and takes many days to complete with a fast Internet connection. -To examine preexisting certificates, it's better to use the Cert -Spotter service , the Cert Spotter -API , or a CT search engine such -as . - -## Coverage - -Any certificate that is logged to a Certificate Transparency log trusted by -Chromium will be detected by Cert Spotter. All certificates issued after April -30, 2018 must be logged to such a log to be trusted by Chromium. - -Generally, certificate authorities will automatically submit certificates -to logs so that they will work in Chromium. In addition, certificates -that are discovered during Internet-wide scans are submitted to Certificate -Transparency logs. - -## Bygone certificates - -Cert Spotter can also notify users of bygone SSL certificates, which are SSL -certificates that outlived their prior domain owner's registration into the -next owners registration. To detect these certificates add a **valid\_at** -argument to each domain in the watchlist followed by the date the domain was -registered in the following format YYYY-MM-DD. For example: - -``` -example.com valid_at:2014-05-02 -``` - -## Security considerations - -Cert Spotter assumes an adversarial model in which an attacker produces a -certificate that is accepted by at least some clients but goes undetected -because of an encoding error that prevents CT monitors from understanding it. -To defend against this attack, Cert Spotter uses a special certificate parser -that keeps the certificate unparsed except for the identifiers. If one of the -identifiers matches a domain on your watchlist, you will be notified, even if -other parts of the certificate are unparsable. - -Cert Spotter takes special precautions to ensure identifiers are parsed -correctly, and implements defenses against identifier-based attacks. For -instance, if a DNS identifier contains a null byte, Cert Spotter interprets it -as two identifiers: the complete identifier, and the identifier formed by -truncating at the first null byte. For example, a certificate for -*example.org\0.example.com* will alert the owners of both *example.org* and -*example.com*. This defends against null prefix attacks -. - -SSLMate continuously monitors CT logs to make sure every certificate's -identifiers can be successfully parsed, and will release updates to Cert -Spotter as necessary to fix parsing failures. - -Cert Spotter understands wildcard and redacted DNS names, and will alert you if -a wildcard or redacted certificate might match an identifier on your watchlist. -For example, a watchlist entry for *sub.example.com* would match certificates for -*\*.example.com* or *?.example.com*. - -Cert Spotter is not just a log monitor, but also a log auditor which checks -that the log is obeying its append-only property. A future release of Cert -Spotter will support gossiping with other log monitors to ensure the log is -presenting a single view. +If certspotter has previously monitored a log, it resumes monitoring +the log from the previous position. This means that if you add +a domain to your watch list, certspotter will not detect any certificates +that were logged prior to the addition. To detect such certificates, +you must delete `$CERTSPOTTER_STATE_DIR/logs`, which will cause certspotter +to restart monitoring from the very beginning of each log (provided +`-start_at_end` is not specified). This will cause certspotter to download +hundreds of millions of certificates, which takes days. To find preexisting +certificates, it's faster to use the Cert Spotter service +, SSLMate's Certificate Transparency Search +API , or a CT search engine such as +. # EXIT STATUS -certspotter exits 0 on success, 1 on any error. +certspotter exits 0 when it receives `SIGTERM` or `SIGINT`, +and non-zero when a serious error occurs. # ENVIRONMENT -**CERTSPOTTER\_STATE\_DIR** -: Directory for storing state. Overridden by **-state\_dir**. Defaults to +`CERTSPOTTER_STATE_DIR` + +: Directory for storing state. Overridden by `-state_dir`. Defaults to `~/.certspotter`. -**CERTSPOTTER\_CONFIG\_DIR** -: Directory from which any configuration, such as the watchlist, is read. +`CERTSPOTTER_CONFIG_DIR` + +: Directory from which any configuration, such as the watch list, is read. Defaults to `~/.certspotter`. +`HTTPS_PROXY` + +: URL of proxy server for making HTTPS requests. `http://`, `https://`, and + `socks5://` URLs are supported. By default, no proxy server is used. + # SEE ALSO -**certspotter-script**(8) +certspotter-script(8) # COPYRIGHT -Copyright (c) 2016-2022 Opsmate, Inc. +Copyright (c) 2016-2023 Opsmate, Inc. # BUGS -Report bugs to . +Report bugs to .