Tandem
/
certspotter
mirror of https://github.com/SSLMate/certspotter.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update man pages

This commit is contained in:
Andrew Ayer 2023-02-05 10:58:05 -05:00
parent 3c23ab4e34
commit 04ea5c949f
2 changed files with 302 additions and 178 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

276
man/certspotter-script.md
View File

@ -2,104 +2,254 @@
**certspotter-script** - Certificate Transparency Log Monitor (hook script) **certspotter-script** - Certificate Transparency Log Monitor (hook script)
# SYNOPSIS
**certspotter-script**
# DESCRIPTION # DESCRIPTION
**certspotter-script** is *any* program that is called from **certspotter**'s **certspotter-script** is *any* program that is called using **certspotter(8)**'s
*-script* argument. **certspotter** executes this script when a file from the *-script* argument. **certspotter** executes this program when it needs to notify
CT log matches against the watchlist. you about an event, such as detecting a certificate for a domain on your watch list.
# ENVIRONMENT # ENVIRONMENT
The script will have the following variables defined in its environment: ## Event information
## Log entry information The following environment variables are set for all types of events:
**CERT\_FILENAME** `EVENT`
: The path of the saved certificate on the local filesystem, if one exists.
**CERT\_TYPE** : One of the following values, indicating the type of event:
: The certificate's type (*cert* or *precert*).
**FINGERPRINT** * `discovered_cert` - certspotter has discovered a certificate for a
: The certificate's fingerprint. domain on your watch list.
**LOG\_URI** * `malformed_cert` - certspotter can't determine if a certificate
: The URI of the log the certificate was found on. matches your watch list because the certificate or the log entry
is malformed.
**ENTRY\_INDEX** * `error` - a problem is preventing certspotter from monitoring all
: The entry's index in the log. logs.
**CERT\_PARSEABLE** Additional event types may be defined in the future, so your script should
: Whether the certificate could be parsed. be able to handle unknown values.
## Identifiers `SUMMARY`
**DNS\_NAMES** : A short human-readable string describing the event.
: A comma-separated list of the certificate's dnsNames.
**IP\_ADDRESSES**
: A comma-separated list of the certificate's IP addresses.
## Certificate information ## Discovered certificate information
**PUBKEY\_HASH** The following environment variables are set for `discovered_cert` events:
: The certificate public key's hash.
**SERIAL** `WATCH_ITEM`
: The certificate's serial.
**NOT\_BEFORE**, **NOT\_AFTER** : The item from your watch list which matches this certificate.
: The certificate's validity information, as a string. (If more than one item matches, the first one is used.)
**NOT\_BEFORE\_UNIXTIME**, **NOT\_AFTER\_UNIXTIME** `LOG_URI`
: The certificate's validity information, as UNIX time.
**SUBJECT\_DN** : The URI of the log containing the certificate.
: The certificate's subject distinguished name (DN).
**ISSUER\_DN** `ENTRY_INDEX`
: the certificate issuer distinguished name (DN).
## Errors : The index of the log entry containing the certificate.
**PARSE\_ERROR** `TBS_SHA256`
: Set to the error that occurred when attempting to extract information about
the certificate. In this case, **CERT\_PARSEABLE** will also be set to "no"
and information such as **PUBKEY\_HASH**, **SERIAL**, as well as validity
and subject, will not be present.
**SERIAL\_PARSE\_ERROR** : The hex-encoded SHA-256 digest of the TBSCertificate, as defined in RFC 6962 Section 3.2.
: Set to the error that occurred when attempting to extract the certificate's Certificates and their corresponding precertificates have the same `TBS_SHA256` value.
serial. Emitted instead of **SERIAL**.
**IDENTIFIERS\_PARSE\_ERROR** `CERT_SHA256`
: Set to the error that occurred when attempting to extract the certificate's
identifiers. Emitted instead of **DNS\_NAMES**, **IP\_ADDRESSES**.
**VALIDITY\_PARSE\_ERROR** : The hex-encoded SHA-256 digest (sometimes called fingerprint) of the certificate.
: Set to the error that occurred when attempting to extract the certificate's The digest is computed over the ASN.1 DER encoding.
validity information. Emitted instead of **NOT\_BEFORE**, **NOT\_AFTER**.
**SUBJECT\_PARSE\_ERROR** `PUBKEY_SHA256`
: Set to the error that occurred when attempting to extract the certificate's
subject information. Emitted instead of **SUBJECT\_DN**.
**ISSUER\_PARSE\_ERROR** : The hex-encoded SHA-256 digest of the certificate's Subject Public Key Info.
: Set to the error that occurred when attempting to extract the certificate's
issuer information. Emitted instead of **ISSUER\_DN**. `CERT_FILENAME`
: Path to a file containing the PEM-encoded certificate chain. Not set if `-no_save` was used.
`JSON_FILENAME`
: Path to a JSON containing additional information about the certificate. See below for the format of the JSON file.
Not set if `-no_save` was used.
`TEXT_FILENAME`
: Path to a file containing a text representation of the certificate.
Not set if `-no_save` was used.
`NOT_BEFORE`, `NOT_BEFORE_UNIXTIME`, `NOT_BEFORE_RFC3339`
: The not before time of the certificate, in a human-readable format, seconds since the UNIX epoch, and RFC3339, respectively. These variables may be unset if there was a parse error, in which case `VALIDITY_PARSE_ERROR` is set.
`NOT_AFTER`, `NOT_AFTER_UNIXTIME`, `NOT_AFTER_RFC3339`
: The not after (expiration) time of the certificate, in a human-readable format, seconds since the UNIX epoch, and RFC3339, respectively. These variables may be unset if there was a parse error, in which case `VALIDITY_PARSE_ERROR` is set.
`VALIDITY_PARSE_ERROR`
: Error parsing not before and not after, if any. If this variable is set, then the `NOT_BEFORE` and `NOT_AFTER` family of variables are unset.
`SUBJECT_DN`
: The distinguished name of the certificate's subject. This variable may be unset if there was a parse error, in which case `SUBJECT_PARSE_ERROR` is set.
`SUBJECT_PARSE_ERROR`
: Error parsing the subject, if any. If this variable is set, then `SUBJECT_DN` is unset.
`ISSUER_DN`
: The distinguished name of the certificate's issuer. This variable may be unset if there was a parse error, in which case `ISSUER_PARSE_ERROR` is set.
`ISSUER_PARSE_ERROR`
: Error parsing the issuer, if any. If this variable is set, then `ISSUER_DN` is unset.
`SERIAL`
: The hex-encoded serial number of the certificate. Prefixed with a minus (-) sign if negative. This variable may be unset if there was a parse error, in which case `SERIAL_PARSE_ERROR` is set.
`SERIAL_PARSE_ERROR`
: Error parsing the serial number, if any. If this variable is set, then `SERIAL` is unset.
## Malformed certificate information
The following environment variables are set for `malformed_cert` events:
`LOG_URI`
: The URI of the log containing the malformed certificate.
`ENTRY_INDEX`
: The index of the log entry containing the malformed certificate.
`LEAF_HASH`
: The base64-encoded Merkle hash of the leaf containing the malformed certificate.
`PARSE_ERROR`
: A human-readable string describing why the certificate is malformed.
# JSON FILE FORMAT
Unless `-no_save` is used, certspotter saves a JSON file for every discovered certificate
under `$CERTSPOTTER_STATE_DIR`, and puts the path to the file in `$JSON_FILENAME`. Your
script can read the JSON file, such as with the jq(1) command, to get additional information
about the certificate which isn't appropriate for environment variables.
The JSON file contains an object with the following fields:
`tbs_sha256`
: A string containing the hex-encoded SHA-256 digest of the TBSCertificate, as defined in RFC 6962 Section 3.2.
Certificates and their corresponding precertificates have the same `tbs_sha256` value.
`cert_sha256`
: A string containing the hex-encoded SHA-256 digest (sometimes called fingerprint) of the certificate.
The digest is computed over the ASN.1 DER encoding.
`pubkey_sha256`
: A string containing the hex-encoded SHA-256 digest of the certificate's Subject Public Key Info.
`issuer_der`
: A base64 string containing the certificate's DER-encoded issuer distinguished name.
`subject_der`
: A base64 string containing the certificate's DER-encoded subject distinguished name.
`dns_names`
: An array of strings containing the DNS names for which the
certificate is valid, taken from both the DNS subject alternative names
(SANs) and the subject common name (CN). Internationalized domain names
are encoded in Punycode.
`ip_addresses`
: An array of strings containing the IP addresses for which the certificate is valid,
taken from both the IP subject alternative names (SANs) and the subject common name (CN).
`not_before`
: A string containing the not before time of the certificate in RFC3339 format.
Null if there was an error parsing the certificate's validity.
`not_after`
: A string containing the not after (expiration) time of the certificate in RFC3339 format.
Null if there was an error parsing the certificate's validity.
`serial_number`
: A string containing the hex-encoded serial number of the certificate. Prefixed with a minus (-) sign if negative.
Null if there was an error parsing the serial number.
# EXAMPLES
Example environment variables for a `discovered_cert` event:
```
CERT_FILENAME=/home/andrew/.certspotter/certs/3c/3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a.pem
CERT_SHA256=3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a
ENTRY_INDEX=6464843
EVENT=discovered_cert
ISSUER_DN=C=GB, ST=Greater Manchester, L=Salford, O=Sectigo Limited, CN=Sectigo RSA Domain Validation Secure Server CA
JSON_FILENAME=/usr2/andrew/.certspotter/certs/3c/3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a.v1.json
LOG_URI=https://ct.cloudflare.com/logs/nimbus2024/
NOT_AFTER='2024-01-26 03:47:26 +0000 UTC'
NOT_AFTER_RFC3339=2024-01-26T03:47:26Z
NOT_AFTER_UNIXTIME=1706240846
NOT_BEFORE='2023-01-31 03:47:26 +0000 UTC'
NOT_BEFORE_RFC3339=2023-01-31T03:47:26Z
NOT_BEFORE_UNIXTIME=1675136846
PUBKEY_SHA256=33ac1d9b9e56005ccac045eac2398b3e9dd6b3f5b66ae6260f2d478c7c0d82c8
SERIAL=c170fbf3bf27481e5c351a4db6f2dc5f
SUBJECT_DN=CN=sslmate.com
SUMMARY='certificate discovered for .sslmate.com'
TBS_SHA256=2388ee81c6f45cffc73e68a35fa8921e839e20acc9a98e8e6dcaea07cbfbdef8
TEXT_FILENAME=/usr2/andrew/.certspotter/certs/3c/3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a.txt
WATCH_ITEM=.sslmate.com
```
Example JSON file for a discovered certificate:
```
{
"cert_sha256": "3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a",
"dns_names": [
"sslmate.com",
"www.sslmate.com"
],
"ip_addresses": [],
"issuer_der": "MIGPMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxNzA1BgNVBAMTLlNlY3RpZ28gUlNBIERvbWFpbiBWYWxpZGF0aW9uIFNlY3VyZSBTZXJ2ZXIgQ0E=",
"not_after": "2024-01-26T03:47:26Z",
"not_before": "2023-01-31T03:47:26Z",
"pubkey_sha256": "33ac1d9b9e56005ccac045eac2398b3e9dd6b3f5b66ae6260f2d478c7c0d82c8",
"serial_number": "c170fbf3bf27481e5c351a4db6f2dc5f",
"subject_der": "MBYxFDASBgNVBAMTC3NzbG1hdGUuY29t",
"tbs_sha256": "2388ee81c6f45cffc73e68a35fa8921e839e20acc9a98e8e6dcaea07cbfbdef8"
}
```
# SEE ALSO # SEE ALSO
**certspotter**(8), **x509**(1) certspotter(8)
# COPYRIGHT # COPYRIGHT
Copyright (c) 2016-2022 Opsmate, Inc. Copyright (c) 2016-2023 Opsmate, Inc.
# BUGS # BUGS
Report bugs to <https://github.com/SSLmate/certspotter>. Report bugs to <https://github.com/SSLMate/certspotter>.

204
man/certspotter.md
View File

@ -4,13 +4,13 @@
# SYNOPSIS # SYNOPSIS
**certspotter** [**-verbose**] [**-start\_at\_end**] [**-watchlist** _file_] `...` **certspotter** [`-start_at_end`] [`-watchlist` *FILENAME*] [`-email` *ADDRESS*] `...`
# DESCRIPTION # DESCRIPTION
**Cert Spotter** is a Certificate Transparency log monitor from SSLMate that **Cert Spotter** is a Certificate Transparency log monitor from SSLMate that
alerts you when a SSL/TLS certificate is issued for one of your domains. Cert alerts you when a SSL/TLS certificate is issued for one of your domains.
Spotter is easier than other open source CT monitors, since it does not require Cert Spotter is easier to use than other open source CT monitors, since it does not require
a database. It's also more robust, since it uses a special certificate parser a database. It's also more robust, since it uses a special certificate parser
that ensures it won't miss certificates. that ensures it won't miss certificates.
@ -30,163 +30,137 @@ You can use Cert Spotter to detect:
# OPTIONS # OPTIONS
**-all_time** -batch_size *NUMBER*
: Scan certs from all time, not just those logged since the previous run of
Cert Spotter.
**-batch_size** _int_ : Maximum number of entries to request per call to get-entries.
: Max number of entries to request at per call to get-entries. This is You should not generally need to change this. Defaults to 1000.
advanced option. Defaults to 1000.
**-logs** _string_ -email *ADDRESS*
: Filename or HTTPS URL of a JSON file containing logs to monitor, in the
format documented at <https://www.certificate-transparency.org/known-logs>. : Email address to contact when a matching certificate is discovered.
You can specify this option more than once to email multiple addresses.
Your system must have a working sendmail(1) command.
-logs *ADDRESS*
: Filename or HTTPS URL of a v2 or v3 JSON log list containing logs to monitor.
The schema for this file can be found at <https://www.gstatic.com/ct/log_list/v3/log_list_schema.json>.
Defaults to <https://loglist.certspotter.org/monitor.json>, which includes Defaults to <https://loglist.certspotter.org/monitor.json>, which includes
the union of active logs recognized by Chrome and Apple. the union of active logs recognized by Chrome and Apple. certspotter periodically
reloads the log list in case it has changed.
-no_save
**-no\_save**
: Do not save a copy of matching certificates. : Do not save a copy of matching certificates.
**-num\_workers** _int_ -script *COMMAND*
: Number of concurrent matchers. Default 2.
**-script** _string_ : Command to execute when a matching certificate is found. See
: Script to execute when a matching certificate is found. See certspotter-script(8) for information about the interface to scripts.
**certspotter-script**(8) for information about the interface to scripts.
-start_at_end
**-start\_at\_end**
: Start monitoring logs from the end rather than the beginning. : Start monitoring logs from the end rather than the beginning.
**WARNING**: monitoring from the beginning guarantees detection of all **WARNING**: monitoring from the beginning guarantees detection of all
certificates, but requires downloading hundreds of millions of certificates, but requires downloading hundreds of millions of
certificates, which takes days. certificates, which takes days.
**-state\_dir** _string_ -state_dir *PATH*
: Directory for storing state. Defaults to `~/.certspotter`.
: Directory for storing state. Defaults to `$CERTSPOTTER_STATE_DIR`, which is
"~/.certspotter" by default.
-stdout
: Write matching certificates to stdout.
-verbose
**-verbose**
: Be verbose. : Be verbose.
**-version** -version
: Print version and exit. : Print version and exit.
**-watchlist** _string_ -watchlist *PATH*
: File containing identifiers to watch. Use `-` for stdin.
Defaults to `~/.certspotter/watchlist`.
# NOTES : File containing DNS names to monitor, one per line. To monitor an entire
domain namespace (including the domain itself and all sub-domains) prefix
the domain name with a dot (e.g. ".example.com"). To monitor a single DNS
name only, do not prefix the name with a dot.
Defaults to `$CERTSPOTTER_CONFIG_DIR/watchlist`, which is
"~/.certspotter/watchlist" by default.
Specify `-` to read the watch list from stdin.
certspotter reads the watch list only when starting up, so you must restart
certspotter if you change it.
## Method of operation # OPERATION
Every time you run Cert Spotter, it scans all browser-recognized certspotter continuously monitors all browser-recognized Certificate
Certificate Transparency logs for certificates matching domains on Transparency logs looking for certificates which are valid for any domain
your watch list. When Cert Spotter detects a matching certificate, it on your watch list. When certspotter detects a matching certificate, it
writes a report to standard out. emails you (if `-email` is specified), executes a script (if `-script`
is specified), and/or writes a report to standard out (if `-stdout`
is specified).
Cert Spotter also saves a copy of matching certificates in certspotter also saves a copy of matching certificates in
`~/.certspotter/certs` (unless you specify the **-no\_save** option). `$CERTSPOTTER_STATE_DIR/certs` ("~/.certspotter/certs" by default)
unless you specify the `-no_save` option.
When Cert Spotter has previously monitored a log, it scans the log When certspotter has not previously monitored a log, it can either start
from the previous position, to avoid downloading the same log entry
more than once. (To override this behavior and scan all logs from the
beginning, specify the **-all\_time** option.)
When Cert Spotter has not previously monitored a log, it can either start
monitoring the log from the beginning, or seek to the end of the log and monitoring the log from the beginning, or seek to the end of the log and
start monitoring from there. Monitoring from the beginning guarantees start monitoring from there. Monitoring from the beginning guarantees
detection of all certificates, but requires downloading hundreds of detection of all certificates, but requires downloading hundreds of
millions of certificates, which takes days. The default behavior is to millions of certificates, which takes days. The default behavior is to
monitor from the beginning. To start monitoring new logs from the end, monitor from the beginning. To start monitoring new logs from the end,
specify the **-start\_at\_end** option. specify the `-start_at_end` option.
You can add and remove domains on your watchlist at any time. However, If certspotter has previously monitored a log, it resumes monitoring
the certspotter command only notifies you of certificates that were the log from the previous position. This means that if you add
logged since adding a domain to the watchlist, unless you specify the a domain to your watch list, certspotter will not detect any certificates
**-all\_time** option, which requires scanning the entirety of every log that were logged prior to the addition. To detect such certificates,
and takes many days to complete with a fast Internet connection. you must delete `$CERTSPOTTER_STATE_DIR/logs`, which will cause certspotter
To examine preexisting certificates, it's better to use the Cert to restart monitoring from the very beginning of each log (provided
Spotter service <https://sslmate.com/certspotter>, the Cert Spotter `-start_at_end` is not specified). This will cause certspotter to download
API <https://sslmate.com/certspotter/api>, or a CT search engine such hundreds of millions of certificates, which takes days. To find preexisting
as <https://crt.sh>. certificates, it's faster to use the Cert Spotter service
<https://sslmate.com/certspotter>, SSLMate's Certificate Transparency Search
## Coverage API <https://sslmate.com/ct_search_api>, or a CT search engine such as
<https://crt.sh>.
Any certificate that is logged to a Certificate Transparency log trusted by
Chromium will be detected by Cert Spotter. All certificates issued after April
30, 2018 must be logged to such a log to be trusted by Chromium.
Generally, certificate authorities will automatically submit certificates
to logs so that they will work in Chromium. In addition, certificates
that are discovered during Internet-wide scans are submitted to Certificate
Transparency logs.
## Bygone certificates
Cert Spotter can also notify users of bygone SSL certificates, which are SSL
certificates that outlived their prior domain owner's registration into the
next owners registration. To detect these certificates add a **valid\_at**
argument to each domain in the watchlist followed by the date the domain was
registered in the following format YYYY-MM-DD. For example:
```
example.com valid_at:2014-05-02
```
## Security considerations
Cert Spotter assumes an adversarial model in which an attacker produces a
certificate that is accepted by at least some clients but goes undetected
because of an encoding error that prevents CT monitors from understanding it.
To defend against this attack, Cert Spotter uses a special certificate parser
that keeps the certificate unparsed except for the identifiers. If one of the
identifiers matches a domain on your watchlist, you will be notified, even if
other parts of the certificate are unparsable.
Cert Spotter takes special precautions to ensure identifiers are parsed
correctly, and implements defenses against identifier-based attacks. For
instance, if a DNS identifier contains a null byte, Cert Spotter interprets it
as two identifiers: the complete identifier, and the identifier formed by
truncating at the first null byte. For example, a certificate for
*example.org\0.example.com* will alert the owners of both *example.org* and
*example.com*. This defends against null prefix attacks
<http://www.thoughtcrime.org/papers/null-prefix-attacks.pdf>.
SSLMate continuously monitors CT logs to make sure every certificate's
identifiers can be successfully parsed, and will release updates to Cert
Spotter as necessary to fix parsing failures.
Cert Spotter understands wildcard and redacted DNS names, and will alert you if
a wildcard or redacted certificate might match an identifier on your watchlist.
For example, a watchlist entry for *sub.example.com* would match certificates for
*\*.example.com* or *?.example.com*.
Cert Spotter is not just a log monitor, but also a log auditor which checks
that the log is obeying its append-only property. A future release of Cert
Spotter will support gossiping with other log monitors to ensure the log is
presenting a single view.
# EXIT STATUS # EXIT STATUS
certspotter exits 0 on success, 1 on any error. certspotter exits 0 when it receives `SIGTERM` or `SIGINT`,
and non-zero when a serious error occurs.
# ENVIRONMENT # ENVIRONMENT
**CERTSPOTTER\_STATE\_DIR** `CERTSPOTTER_STATE_DIR`
: Directory for storing state. Overridden by **-state\_dir**. Defaults to
: Directory for storing state. Overridden by `-state_dir`. Defaults to
`~/.certspotter`. `~/.certspotter`.
**CERTSPOTTER\_CONFIG\_DIR** `CERTSPOTTER_CONFIG_DIR`
: Directory from which any configuration, such as the watchlist, is read.
: Directory from which any configuration, such as the watch list, is read.
Defaults to `~/.certspotter`. Defaults to `~/.certspotter`.
`HTTPS_PROXY`
: URL of proxy server for making HTTPS requests. `http://`, `https://`, and
`socks5://` URLs are supported. By default, no proxy server is used.
# SEE ALSO # SEE ALSO
**certspotter-script**(8) certspotter-script(8)
# COPYRIGHT # COPYRIGHT
Copyright (c) 2016-2022 Opsmate, Inc. Copyright (c) 2016-2023 Opsmate, Inc.
# BUGS # BUGS
Report bugs to <https://github.com/SSLmate/certspotter>. Report bugs to <https://github.com/SSLMate/certspotter>.