Commit Graph

254 Commits

Author SHA1 Message Date
Andrew Ayer 3ec8a0a3db Ignore IP address SANs with an invalid length 2016-05-01 14:52:19 -07:00
Andrew Ayer ca8f60740a Trim trailing dots from DNS names 2016-05-01 12:49:26 -07:00
Andrew Ayer 3c220e56f6 scanner: reduce channel queue buffer length
To keep memory consumption down
2016-05-01 12:49:07 -07:00
Andrew Ayer a0859acad3 Remove defunct Certly log 2016-04-30 15:03:16 -07:00
Andrew Ayer 847b7129e8 Monitor for all DNS names that _might_ match a monitored domain
Wildcards, redacted labels, and unparseable labels.
2016-04-29 09:02:03 -07:00
Andrew Ayer ec68dde647 Only allow * and ? as entire DNS name labels 2016-04-29 08:45:54 -07:00
Andrew Ayer 2c9df274e9 Gracefully handle all manner of poorly encoded identifiers
Also add preliminary support for IP address identifiers.
2016-04-28 22:00:32 -07:00
Andrew Ayer a072440db8 Handle certificates with multiple Basic Constraints extensions 2016-04-26 18:06:59 -07:00
Andrew Ayer 65ed742477 Support wildcards
For example, if you're watching subdomain.example.com, a cert for
*.example.com will now match.
2016-04-26 14:49:39 -07:00
Andrew Ayer 4132ed5e9f Add support for IDNs
IDNs can be specified in either Unicode or ASCII (as Punycode).
Certs can specify the DNS name either way, and we'll match it.
2016-04-26 14:38:09 -07:00
Andrew Ayer 19c5f86d23 Allow DNS SANs to contain UTF-8
There are too many certs in the wild which have UTF-8 in their DNS SANs.
2016-04-26 14:14:08 -07:00
Andrew Ayer 2426817cd5 Raise parse error if certain strings are improperly encoded
If a UTF8String in the Subject CN isn't valid UTF-8, or if a DNS SAN is
not ASCII, raise a parse error, since we don't know how to interpret
the string.
2016-04-24 09:11:28 -07:00
Andrew Ayer 2d2aa37202 Parse common names separately from DNS names 2016-04-22 20:58:33 -07:00
Andrew Ayer ef0b46b7a5 Remove defunct Wosign log 2016-04-14 17:55:50 -07:00
Andrew Ayer e091186d83 Save consistency proof along with evidence of misbehavior
Although the consistency proof is neither necessary nor sufficient
to prove misbehavior by a log, this will help with debugging if a
log returns a bogus consistency proof erroneously (which seems to
be happening with the Rocketeer log lately...).
2016-04-06 08:10:06 -07:00
Andrew Ayer db2cd2c458 logclient: work around HTTP/2 issue
See https://github.com/google/certificate-transparency/issues/1136
2016-03-27 11:54:34 -07:00
Andrew Ayer 80bfe1321c Add helpers to get fingerprint/hashes in byte form 2016-03-26 18:04:22 -07:00
Andrew Ayer ef395b8e60 Add function to validate a pre-cert 2016-03-23 21:03:00 -07:00
Andrew Ayer 81bfa0bbd8 Add ctparsewatch
It watches for certificates which we can't fully parse
2016-03-23 20:19:39 -07:00
Andrew Ayer 786e9e3460 Add a relaxed ASN.1 Time parser
Since some certs contain invalid times in the validity
2016-03-23 20:18:26 -07:00
Andrew Ayer af14fca70f Add HasParseErrors method to EntryInfo 2016-03-23 20:18:12 -07:00
Andrew Ayer 616ac0cb83 Adjust gitignore 2016-03-23 20:04:55 -07:00
Andrew Ayer eded2ff458 Ensure ParseDNSNames does not return a nil slice 2016-03-22 17:17:38 -07:00
Andrew Ayer 3b59332bf1 Rename a function for clarity 2016-03-17 16:34:53 -07:00
Andrew Ayer a071e9490a Replace embedded X509 parser with my own lightweight parser 2016-03-16 16:59:37 -07:00
Andrew Ayer 5ccf9fdcd3 ctwatch: allow state dir to be set by $CTWATCH_STATE_DIR 2016-03-08 07:09:26 -08:00
Andrew Ayer f988d05b4b Decode JSON directly into []byte
Simplifies the code and hopefully reduces memory usage
2016-03-08 07:01:10 -08:00
Andrew Ayer 2608a74e66 Make trailing garbage a fatal error when extracting DNS names
Logging something to stderr was not helpful, and it's best to be
on the safe side anyways.

Whitelist a single null byte following the SAN extension.  This
is a harmless and common error.

As of now, all certificates in the CT logs parse successfully.
2016-02-22 19:37:03 -08:00
Andrew Ayer 08fa700d29 scanner: don't prefix log messages with log URI
It's redundant now that we're setting prefix with log.SetPrefix()
2016-02-22 19:23:08 -08:00
Andrew Ayer 5803389588 Fix some pointer inconsistencies in code 2016-02-22 15:29:52 -08:00
Andrew Ayer 09c37cfdfd Clarify a flag 2016-02-22 15:14:17 -08:00
Andrew Ayer 8f3bd3b6ff Improve logging 2016-02-22 14:58:11 -08:00
Andrew Ayer b297ba9967 Use bits in the exit code to convey what happened 2016-02-22 14:45:50 -08:00
Andrew Ayer 40123f9ba8 Allow . to be specified on stdin as well 2016-02-22 14:18:56 -08:00
Andrew Ayer 94ccbc0a4f Add backoff during fetch errors 2016-02-22 14:11:47 -08:00
Andrew Ayer df6527b165 Change -all_time to only affect logs we haven't seen before
It's more useful this way - there's no sense in scanning logs we've
already scanned.

I need a better name for this switch, though.
2016-02-20 12:04:07 -08:00
Andrew Ayer ff44576c87 Save old and new STHs if consistency proof fails 2016-02-18 12:40:21 -08:00
Andrew Ayer 672491e065 Fix bug where we were returning a nil tree builder 2016-02-18 11:58:00 -08:00
Andrew Ayer 16bf546258 Embed Google CT library, with my own changes 2016-02-18 10:44:56 -08:00
Andrew Ayer 3c33dc8277 Remove sha1watch 2016-02-18 10:41:55 -08:00
Andrew Ayer e91d7bacbd Minor cleanup to improve encapsulation 2016-02-18 10:23:07 -08:00
Andrew Ayer b47d35a005 Rename some types/functions for clarity 2016-02-18 10:15:56 -08:00
Andrew Ayer 35eef25f4a Rename function for clarity 2016-02-18 10:09:33 -08:00
Andrew Ayer 9558efc955 Verify STH signatures 2016-02-17 16:03:49 -08:00
Andrew Ayer 4b304fd192 Audit Merkle tree when retrieving entries
Also add an -all_time command line option to retrieve all certificates,
not just the ones since the last scan.
2016-02-17 14:54:40 -08:00
Andrew Ayer b6dec7822d Overhaul to be more robust and simpler
All certificates are now parsed with a special, extremely
lax parser that extracts only the DNS names.  Only if the
DNS names match the domains we're interested in will we attempt
to parse the cert with the real X509 parser.  This ensures that
we won't miss a very badly encoded certificate that has been
issued for a monitored domain.

As of the time of commit, the lax parser is able to process every
logged certificate in the known logs.
2016-02-09 10:28:52 -08:00
Andrew Ayer 1dcbe91877 WriteCertRepository: avoid serializing precerts twice
With pre-certs, Chain[0] is the pre-cert itself.
2016-02-07 14:47:05 -08:00
Andrew Ayer a79cc26570 Include filename of saved cert in output/script invocation 2016-02-05 08:20:12 -08:00
Andrew Ayer cfaf126284 To monitor all domains, require "." to be specified
Now that we save all certs by default, we want to prevent people
from accidentally monitoring all domains, which could lead to MASSIVE
disk usage.

"." is used because it denotes the root zone in DNS.
2016-02-05 08:13:11 -08:00
Andrew Ayer e73a5a89a7 Ignore non-fatal errors when parsing root certificates 2016-02-05 07:57:15 -08:00