392 Commits

Author SHA1 Message Date
Andrew Ayer
b05a66f634 Only calculate root hash when needed to verify an STH 2025-06-17 10:45:56 -04:00
Andrew Ayer
b87b33a41b Upgrade dependencies 2025-06-16 23:33:51 -04:00
Andrew Ayer
3279459be2 Add Compare to LogID and merkletree.Hash 2025-06-16 14:24:26 -04:00
Andrew Ayer
d5bc1ef75b Simplify certspotterVersion
The old code is unnecessary now that go derives a version from the VCS info.
2025-06-13 16:26:10 -04:00
Andrew Ayer
38bcd36d98 Release v0.20.0 v0.20.0 2025-06-13 12:24:17 -04:00
Andrew Ayer
ca7b11ca96 Print a friendlier error message if -batch_size specified 2025-06-13 12:22:23 -04:00
Andrew Ayer
26439b4deb Remove unused code 2025-05-30 17:09:02 -04:00
Andrew Ayer
9544d8ab50 Imprve comment 2025-05-21 14:59:56 -04:00
Andrew Ayer
694eb276a6 Also check timestamp when comparing STHs
otherwise we might fail to delete unverified_sths if they have a different timestamp
2025-05-21 14:33:58 -04:00
Andrew Ayer
90ead642b0 Simplify context cancellation checks 2025-05-21 14:31:24 -04:00
Andrew Ayer
56af38ca70 Rewrite STH pipeline to avoid prematurely deleting STHs 2025-05-21 14:08:12 -04:00
Andrew Ayer
0c22448e5f Avoid spurious file not found errors loading STH dir if an STH is concurrently deleted 2025-05-20 15:29:23 -04:00
Andrew Ayer
61b037a708 Improve docs for -verbose 2025-05-19 13:47:04 -04:00
Andrew Ayer
15e35abdaa Only print log errors to stderr if -verbose specified
Log errors are so frequent that they are drowning out fatal errors. This commit will reserve stderr for fatal errors by default. See #104 for background.

This means that operators will need to enable -verbose if they want to get details about why a health check failed.  This seems better than making stderr noisy by default. The long-term solution is #106.
2025-05-19 13:46:16 -04:00
Andrew Ayer
ce80beb1d4 Document the directories used by certspotter in the man page
Closes: #103
2025-05-19 13:35:47 -04:00
Andrew Ayer
b06aecc56c Improve man pages 2025-05-19 13:35:43 -04:00
Andrew Ayer
46c8fc64fd Improve verbose logging 2025-05-19 13:24:51 -04:00
Andrew Ayer
b89afef32a In verbose mode, print a message when exiting due to signal 2025-05-19 13:13:18 -04:00
Andrew Ayer
e50476620c sequencer: improve Godocs 2025-05-14 18:44:25 -04:00
Andrew Ayer
63845b370d sequencer: add Reserve method 2025-05-14 18:44:16 -04:00
Andrew Ayer
bdc589762a Improve http.Client configuration
Ensure HTTP/2 can be used.

Set IdleConnTimeout to the net/http default.

Remove MaxIdleConns limit so that connections are more likely to be reused.
2025-05-14 18:43:47 -04:00
Andrew Ayer
0ba3b07bd9 Remove -batch_size option
It's obsolete due to the new parallel downloading system.
2025-05-08 08:39:32 -04:00
Andrew Ayer
996068385f Fail health check for logs have never been contacted 2025-05-07 21:31:43 -04:00
Andrew Ayer
37531001bc Improve formatting of an error message 2025-05-07 18:26:18 -04:00
Andrew Ayer
cfe7df0b9f Release v0.19.1 v0.19.1 2025-05-07 18:06:42 -04:00
Andrew Ayer
2a499552ae Retract v0.19.0 2025-05-07 18:06:36 -04:00
Andrew Ayer
d0f48efa91 Make an error message less verbose 2025-05-07 18:03:00 -04:00
Andrew Ayer
61b6c3bf2a Add a space after colon in log message 2025-05-07 18:01:19 -04:00
Andrew Ayer
62649aae08 Log errors contacting log 2025-05-07 17:58:17 -04:00
Andrew Ayer
e9c9ef8b43 Avoid integer overflow leading to panic in rand.N 2025-05-07 17:54:36 -04:00
Andrew Ayer
9ba1d4d915 Release v0.19.0 v0.19.0 2025-05-07 16:47:31 -04:00
Andrew Ayer
403d5e2f58 Apply gofmt 2025-05-07 09:58:22 -04:00
Andrew Ayer
8a655b8566 Avoid calling t.Fatalf from goroutine 2025-05-07 09:56:14 -04:00
Andrew Ayer
647b036ed1 Remove unreachable return statements 2025-05-07 09:55:45 -04:00
Andrew Ayer
61508d8bf1 Fix printf mistake 2025-05-07 09:49:18 -04:00
Andrew Ayer
560ab984e3 Update README 2025-05-07 09:32:55 -04:00
Andrew Ayer
300adf6608 Update copyright year in man pages 2025-05-07 09:27:27 -04:00
Andrew Ayer
344df03c6c Avoid generating download batches with an invalid range
Previously, if we rounded down the tree size to avoid downloading a
partial tile, but the log position was already within the partial tile
(which can happen with a brand new log and -start_at_end), we'd generate
a download batch where end < begin, which caused all sorts of problems.
2025-05-06 15:13:31 -04:00
Andrew Ayer
5769c83cf3 Revert "Avoid calling get-entries when range is invalid (end < begin)"
This reverts commit 71b296141ed278d9a808689aad0444324fe37cb6.
2025-05-06 15:10:58 -04:00
Andrew Ayer
71b296141e Avoid calling get-entries when range is invalid (end < begin)
end < begin can arise if we've rounded down end to avoid downloading a
partial tile, but the log position is already within the partial tile
(which can happen with a brand new log and -start_at_end).
2025-05-06 14:58:23 -04:00
Andrew Ayer
a6af6c54ba Avoid inclusive end bound until last possible moment
Inclusive end bounds are the devil.
2025-05-06 14:52:36 -04:00
Andrew Ayer
8119925c16 Store issuers cache under os.UserCacheDir 2025-05-06 14:25:41 -04:00
Andrew Ayer
6151cb26da Cache issuer certificates retrieved from static-ct-api logs 2025-05-06 14:19:25 -04:00
Andrew Ayer
958e7a9efb Avoid relying on STH timestamp during monitoring
Instead use the time at which the STH was observed (which for
FilesystemState is assumed to be the mtime of the STH file).  This is
easier to reason about: we don't have to worry about logs lying about
the time; we don't have to take into account the delay between STH fetch
and healthcheck; we won't raise spurious health checks about logs with
MMDs longer than the healthcheck interval.
2025-05-06 10:41:33 -04:00
Andrew Ayer
00fd77f6ed Rename certspotter-specific loglist fields, again 2025-05-05 10:29:20 -04:00
Andrew Ayer
56b190f7c0 Rename DownloadWorkers, revert to old defaults 2025-05-05 10:15:09 -04:00
Andrew Ayer
bc199bca4b Rename DownloadJobSize to GetEntriesSize 2025-05-05 10:04:50 -04:00
Andrew Ayer
c967253f80 monitor: fsync state files before renaming them
Without fsync, there's a risk of zero-length files being persisted if
there's a power failure.

Don't bother fsyncing the parent directory because it's OK if the data rolls
back to the previous version; we only need to avoid data corruption.

Closes: #101
2025-05-04 20:44:36 -04:00
Andrew Ayer
b856d7f163 static-ct-api support, parallel downloading 2025-05-04 20:41:33 -04:00
Andrew Ayer
84bd080553 Add a TODO 2025-05-04 20:32:38 -04:00