Commit Graph

35 Commits

Author SHA1 Message Date
Andrew Ayer 039339154f Move retry logic into LogClient
This allows retry logic to be used for all requests, not just get-entries

Also add context arguments
2022-06-02 10:02:32 -04:00
Andrew Ayer 4e4250dad2 Don't ask for consistency proofs based on an empty tree
RFC 6962 doesn't define how to generate a consistency proof in this case,
and it doesn't matter anyways since the tree is empty.  The DigiCert logs
return a 400 error if we ask for such a proof.
2021-08-17 15:00:48 -04:00
Andrew Ayer a147970db8 Use ct.SHA256Hash for log ID rather than []byte 2021-04-30 17:04:16 -04:00
Andrew Ayer 185445e158 Retrieve log list from certspotter.org at startup instead of embedding in source
The list of logs changes far too frequently (with annual shards and operators
dropping out of the ecosystem) to continue embedding in the source code.

Breaking change: the -logs option now expects a
JSON file in the v2 log list format, as documented at
<https://www.certificate-transparency.org/known-logs> and
<https://www.gstatic.com/ct/log_list/v2/log_list_schema.json>.

You can now specify an HTTPS URL to -logs in addition to a file path.

Breaking change: the -underwater option has been removed; if you want
this behavior then specify https://loglist.certspotter.org/underwater.json
as your log list.
2020-04-29 11:51:50 -04:00
Andrew Ayer c2099d6d49 Manually prefix all log messages with log URL
(Instead of using log.SetPrefix)

This will let us process logs in parallel.
2019-12-02 15:03:34 -05:00
Andrew Ayer 41ca1aaab8 Avoid unaligned atomic access on 32 bit platforms
Closes #23
2017-11-06 13:33:02 -08:00
Andrew Ayer 06c253a0ea Continue processing a log even if an STH failed to verify
It may still be possible to audit other STHs, and to scan new entries
up to the latest verified STH.  This allows Cert Spotter to continue
to make forward progress even if a log is persistently skewed (as the
DigiCert has been lately).

Also, rework some code to be simpler and less redundant.
2017-04-27 10:48:48 -07:00
Andrew Ayer 2f0833ac9c Apply gofmt 2017-01-08 10:17:00 -08:00
Andrew Ayer 9ceedea9ef Rename MerkleTreeBuilder to CollapsedMerkleTree 2017-01-06 14:39:08 -08:00
Andrew Ayer 1719aa5d8e Set log ID in STHs that we download
This will facilitate STH pollination.
2017-01-06 12:50:21 -08:00
Andrew Ayer 0c751f0294 Drop the MerkleTreeBuilder return value from VerifyConsistencyProof 2017-01-05 21:06:37 -08:00
Andrew Ayer 3673518b30 Add constructor functions for MerkleTreeBuilder 2017-01-05 14:48:35 -08:00
Andrew Ayer 39737f33ec Rename MerkleTreeBuilder.size to numLeaves 2017-01-05 14:41:54 -08:00
Andrew Ayer 0bd48084f0 Add MakeMerkleTreeBuilder 2016-11-25 20:13:17 -08:00
Jonathan Rudenberg acc6781f29 Run gofmt
Signed-off-by: Jonathan Rudenberg <jonathan@titanous.com>
2016-07-28 14:55:46 -04:00
Andrew Ayer 19e05b901a Remove some dead code from the scanner 2016-06-22 10:32:42 -07:00
Andrew Ayer b79cb31413 Move package to software.sslmate.com/src/certspotter 2016-05-04 12:19:59 -07:00
Andrew Ayer 1e582e2e0c License under the MPL 2.0 2016-05-04 11:56:13 -07:00
Andrew Ayer 670cddafbc Rename project to certspotter 2016-05-04 11:49:07 -07:00
Andrew Ayer 3c220e56f6 scanner: reduce channel queue buffer length
To keep memory consumption down
2016-05-01 12:49:07 -07:00
Andrew Ayer e091186d83 Save consistency proof along with evidence of misbehavior
Although the consistency proof is neither necessary nor sufficient
to prove misbehavior by a log, this will help with debugging if a
log returns a bogus consistency proof erroneously (which seems to
be happening with the Rocketeer log lately...).
2016-04-06 08:10:06 -07:00
Andrew Ayer 08fa700d29 scanner: don't prefix log messages with log URI
It's redundant now that we're setting prefix with log.SetPrefix()
2016-02-22 19:23:08 -08:00
Andrew Ayer 5803389588 Fix some pointer inconsistencies in code 2016-02-22 15:29:52 -08:00
Andrew Ayer 8f3bd3b6ff Improve logging 2016-02-22 14:58:11 -08:00
Andrew Ayer 94ccbc0a4f Add backoff during fetch errors 2016-02-22 14:11:47 -08:00
Andrew Ayer 16bf546258 Embed Google CT library, with my own changes 2016-02-18 10:44:56 -08:00
Andrew Ayer e91d7bacbd Minor cleanup to improve encapsulation 2016-02-18 10:23:07 -08:00
Andrew Ayer b47d35a005 Rename some types/functions for clarity 2016-02-18 10:15:56 -08:00
Andrew Ayer 9558efc955 Verify STH signatures 2016-02-17 16:03:49 -08:00
Andrew Ayer 4b304fd192 Audit Merkle tree when retrieving entries
Also add an -all_time command line option to retrieve all certificates,
not just the ones since the last scan.
2016-02-17 14:54:40 -08:00
Andrew Ayer b6dec7822d Overhaul to be more robust and simpler
All certificates are now parsed with a special, extremely
lax parser that extracts only the DNS names.  Only if the
DNS names match the domains we're interested in will we attempt
to parse the cert with the real X509 parser.  This ensures that
we won't miss a very badly encoded certificate that has been
issued for a monitored domain.

As of the time of commit, the lax parser is able to process every
logged certificate in the known logs.
2016-02-09 10:28:52 -08:00
Andrew Ayer 678e8bddc8 Include log URI in error messages 2016-02-05 07:47:42 -08:00
Andrew Ayer 1b17c25747 Decrease log severity of non-fatal parse errors
These errors are for things like unhandled critical extensions.  The cert
is still processed, so it's not such a bad thing.
2016-02-05 07:45:49 -08:00
Andrew Ayer 3f596730a0 New and simplified multi-log operation 2016-02-04 20:16:25 -08:00
Andrew Ayer a418a3686d Initial commit 2016-02-04 18:46:19 -08:00