Commit Graph

6 Commits

Author SHA1 Message Date
Andrew Ayer 185445e158 Retrieve log list from certspotter.org at startup instead of embedding in source
The list of logs changes far too frequently (with annual shards and operators
dropping out of the ecosystem) to continue embedding in the source code.

Breaking change: the -logs option now expects a
JSON file in the v2 log list format, as documented at
<https://www.certificate-transparency.org/known-logs> and
<https://www.gstatic.com/ct/log_list/v2/log_list_schema.json>.

You can now specify an HTTPS URL to -logs in addition to a file path.

Breaking change: the -underwater option has been removed; if you want
this behavior then specify https://loglist.certspotter.org/underwater.json
as your log list.
2020-04-29 11:51:50 -04:00
Andrew Ayer e8c4f10e97 Use a lock file to prevent certspotter from running concurrently 2017-01-10 10:50:41 -08:00
Andrew Ayer 2f0833ac9c Apply gofmt 2017-01-08 10:17:00 -08:00
Andrew Ayer 0eb6d199a4 Improve the name of a function 2017-01-06 12:24:09 -08:00
Andrew Ayer 8ea4003994 Add some additional logging 2017-01-06 10:31:34 -08:00
Andrew Ayer 0af0262498 Overhaul log processing and auditing
1. Instead of storing a single STH per log, we now store one verified
STH and any number of unverified STHs.  When we process a log, we verify
each unverified STH using a consistency proof with the verified STH,
and only delete it if it successfully verifies.  We set the verified
STH to the largest STH which we've successfully verified.

This has two important benefits.  First, we never ever delete an STH
unless we can successfully verify it (previously, we would forget about
an STH under certain error conditions).  Second, it lays the groundwork
for STH pollination.  Upon reception of an STH, we can simply drop it in
the log's unverified_sths directory (assuming the signature is valid),
and Cert Spotter will audit it.

There is no more "evidence" directory; if a consistency proof fails,
the STHs will already be present elsewhere in the state directory.

2. We now persist a MerkleTreeBuilder between each run of Cert Spotter,
instead of rebuilding it every time from the consistency proof.  This is
not intrinsically better, but it makes the code simpler considering we
can now fetch numerous consistency proofs per run.

3. To accommodate the above changes, the state directory has a brand
new layout.  The state directory is now versioned, and Cert Spotter
will automatically migrate old state directories to the new layout.
This migration logic will be removed in a future Cert Spotter release.

As a bonus, the code is generally cleaner now :-)
2017-01-05 21:00:35 -08:00