Commit Graph

313 Commits

Author SHA1 Message Date
Andrew Ayer 9342adcd93 Tighten up the cert information output
Remove subject and SANs since they are redundant with earlier identifier
listing.  Remove serial number because who cares?  Put type of entry
on same line as log entry info.

If people want this info they can always examine the saved file or the
crt.sh page.
2016-05-09 15:43:19 -07:00
Andrew Ayer b79cb31413 Move package to software.sslmate.com/src/certspotter 2016-05-04 12:19:59 -07:00
Andrew Ayer 1e582e2e0c License under the MPL 2.0 2016-05-04 11:56:13 -07:00
Andrew Ayer 670cddafbc Rename project to certspotter 2016-05-04 11:49:07 -07:00
Andrew Ayer ea3db97486 Only replace DNS label with placeholder if it's utterly unparsable
e.g. contains control characters, Punycode conversion fails

There are quite simply too many certs with bogus DNS labels out in the wild,
and it just doesn't make sense to bother every .com domain holder because
GoDaddy signed a cert with a DNS name like "www.        just4funpartyrentals.com"
It is highly unlikely any validator will ever match that DNS name.
2016-05-04 11:43:02 -07:00
Andrew Ayer 60636ba2d7 Move Identifiers from CertInfo to EntryInfo
It's more logical, and it avoids some redundant parsing.
2016-05-03 11:58:59 -07:00
Andrew Ayer 5f4e35843a Display SANs in output 2016-05-02 11:59:55 -07:00
Andrew Ayer df5ad71a40 Support for IP addresses encoded as strings in CNs/DNS SANs 2016-05-02 11:38:08 -07:00
Andrew Ayer 82167b8151 Additional handling of pathological DNS names
1. Trim leading and trailing whitespace of DNS names.

2. Trim http:// and https:// prefixes.

3. If DNS name contains a slash, ALSO process the DNS name up to
   the first slash, since it's probably a URL.
2016-05-01 17:02:52 -07:00
Andrew Ayer 3ec8a0a3db Ignore IP address SANs with an invalid length 2016-05-01 14:52:19 -07:00
Andrew Ayer ca8f60740a Trim trailing dots from DNS names 2016-05-01 12:49:26 -07:00
Andrew Ayer 3c220e56f6 scanner: reduce channel queue buffer length
To keep memory consumption down
2016-05-01 12:49:07 -07:00
Andrew Ayer a0859acad3 Remove defunct Certly log 2016-04-30 15:03:16 -07:00
Andrew Ayer 847b7129e8 Monitor for all DNS names that _might_ match a monitored domain
Wildcards, redacted labels, and unparseable labels.
2016-04-29 09:02:03 -07:00
Andrew Ayer ec68dde647 Only allow * and ? as entire DNS name labels 2016-04-29 08:45:54 -07:00
Andrew Ayer 2c9df274e9 Gracefully handle all manner of poorly encoded identifiers
Also add preliminary support for IP address identifiers.
2016-04-28 22:00:32 -07:00
Andrew Ayer a072440db8 Handle certificates with multiple Basic Constraints extensions 2016-04-26 18:06:59 -07:00
Andrew Ayer 65ed742477 Support wildcards
For example, if you're watching subdomain.example.com, a cert for
*.example.com will now match.
2016-04-26 14:49:39 -07:00
Andrew Ayer 4132ed5e9f Add support for IDNs
IDNs can be specified in either Unicode or ASCII (as Punycode).
Certs can specify the DNS name either way, and we'll match it.
2016-04-26 14:38:09 -07:00
Andrew Ayer 19c5f86d23 Allow DNS SANs to contain UTF-8
There are too many certs in the wild which have UTF-8 in their DNS SANs.
2016-04-26 14:14:08 -07:00
Andrew Ayer 2426817cd5 Raise parse error if certain strings are improperly encoded
If a UTF8String in the Subject CN isn't valid UTF-8, or if a DNS SAN is
not ASCII, raise a parse error, since we don't know how to interpret
the string.
2016-04-24 09:11:28 -07:00
Andrew Ayer 2d2aa37202 Parse common names separately from DNS names 2016-04-22 20:58:33 -07:00
Andrew Ayer ef0b46b7a5 Remove defunct Wosign log 2016-04-14 17:55:50 -07:00
Andrew Ayer e091186d83 Save consistency proof along with evidence of misbehavior
Although the consistency proof is neither necessary nor sufficient
to prove misbehavior by a log, this will help with debugging if a
log returns a bogus consistency proof erroneously (which seems to
be happening with the Rocketeer log lately...).
2016-04-06 08:10:06 -07:00
Andrew Ayer db2cd2c458 logclient: work around HTTP/2 issue
See https://github.com/google/certificate-transparency/issues/1136
2016-03-27 11:54:34 -07:00
Andrew Ayer 80bfe1321c Add helpers to get fingerprint/hashes in byte form 2016-03-26 18:04:22 -07:00
Andrew Ayer ef395b8e60 Add function to validate a pre-cert 2016-03-23 21:03:00 -07:00
Andrew Ayer 81bfa0bbd8 Add ctparsewatch
It watches for certificates which we can't fully parse
2016-03-23 20:19:39 -07:00
Andrew Ayer 786e9e3460 Add a relaxed ASN.1 Time parser
Since some certs contain invalid times in the validity
2016-03-23 20:18:26 -07:00
Andrew Ayer af14fca70f Add HasParseErrors method to EntryInfo 2016-03-23 20:18:12 -07:00
Andrew Ayer 616ac0cb83 Adjust gitignore 2016-03-23 20:04:55 -07:00
Andrew Ayer eded2ff458 Ensure ParseDNSNames does not return a nil slice 2016-03-22 17:17:38 -07:00
Andrew Ayer 3b59332bf1 Rename a function for clarity 2016-03-17 16:34:53 -07:00
Andrew Ayer a071e9490a Replace embedded X509 parser with my own lightweight parser 2016-03-16 16:59:37 -07:00
Andrew Ayer 5ccf9fdcd3 ctwatch: allow state dir to be set by $CTWATCH_STATE_DIR 2016-03-08 07:09:26 -08:00
Andrew Ayer f988d05b4b Decode JSON directly into []byte
Simplifies the code and hopefully reduces memory usage
2016-03-08 07:01:10 -08:00
Andrew Ayer 2608a74e66 Make trailing garbage a fatal error when extracting DNS names
Logging something to stderr was not helpful, and it's best to be
on the safe side anyways.

Whitelist a single null byte following the SAN extension.  This
is a harmless and common error.

As of now, all certificates in the CT logs parse successfully.
2016-02-22 19:37:03 -08:00
Andrew Ayer 08fa700d29 scanner: don't prefix log messages with log URI
It's redundant now that we're setting prefix with log.SetPrefix()
2016-02-22 19:23:08 -08:00
Andrew Ayer 5803389588 Fix some pointer inconsistencies in code 2016-02-22 15:29:52 -08:00
Andrew Ayer 09c37cfdfd Clarify a flag 2016-02-22 15:14:17 -08:00
Andrew Ayer 8f3bd3b6ff Improve logging 2016-02-22 14:58:11 -08:00
Andrew Ayer b297ba9967 Use bits in the exit code to convey what happened 2016-02-22 14:45:50 -08:00
Andrew Ayer 40123f9ba8 Allow . to be specified on stdin as well 2016-02-22 14:18:56 -08:00
Andrew Ayer 94ccbc0a4f Add backoff during fetch errors 2016-02-22 14:11:47 -08:00
Andrew Ayer df6527b165 Change -all_time to only affect logs we haven't seen before
It's more useful this way - there's no sense in scanning logs we've
already scanned.

I need a better name for this switch, though.
2016-02-20 12:04:07 -08:00
Andrew Ayer ff44576c87 Save old and new STHs if consistency proof fails 2016-02-18 12:40:21 -08:00
Andrew Ayer 672491e065 Fix bug where we were returning a nil tree builder 2016-02-18 11:58:00 -08:00
Andrew Ayer 16bf546258 Embed Google CT library, with my own changes 2016-02-18 10:44:56 -08:00
Andrew Ayer 3c33dc8277 Remove sha1watch 2016-02-18 10:41:55 -08:00
Andrew Ayer e91d7bacbd Minor cleanup to improve encapsulation 2016-02-18 10:23:07 -08:00