Commit Graph

243 Commits

Author SHA1 Message Date
Andrew Ayer 2f2ad094db Set User-Agent header when fetching log list 2023-02-03 15:24:55 -05:00
Andrew Ayer 2366c06ca6 Support ETag/Last-Modified when fetching loglist 2023-02-03 15:21:24 -05:00
Andrew Ayer 6bb03865fb Modernize loglist fetching, add context support 2023-02-03 14:55:09 -05:00
Andrew Ayer 29ed939006 Remove old code 2023-02-03 14:47:47 -05:00
Andrew Ayer 897c861451 Remove redundant information in an error message 2023-02-03 14:38:02 -05:00
Andrew Ayer 35555b769a Remove script directory support (for now)
The implementation was no good because it broke $PATH lookups.

I still like this feature but will defer it to a future version.
2023-02-03 14:35:26 -05:00
Andrew Ayer ef2a7698d7 Update a TODO comment 2023-02-03 14:32:44 -05:00
Andrew Ayer a5a9008de2 Add .v1 to file suffix of JSON files
If we add fields in the future this will make it clear that old
files don't have the new fields
2023-02-03 14:32:35 -05:00
Andrew Ayer 6848316a5b Make the .notified file a hidden file
Since it's an implementation detail that users shouldn't need
to know about.
2023-02-03 14:29:58 -05:00
Andrew Ayer 5e7fa8c079 Remove some TODOs that I'v decided not to do 2023-02-03 14:29:24 -05:00
Andrew Ayer 209cdb181b Convert to a daemon and make many other improvements
Specifically, certspotter no longer terminates unless it receives SIGTERM
or SIGINT or there is a serious error.

Although using cron made sense in the early days of Certificate
Transparency, certspotter now needs to run continuously to reliably keep
up with the high growth rate of contemporary CT logs, and to gracefully
handle the many transient errors that can arise when monitoring CT.

Closes: #63
Closes: #37
Closes: #32 (presumably by eliminating $DNS_NAMES and $IP_ADDRESSES)
Closes: #21 (with $WATCH_ITEM)
Closes: #25
2023-02-03 14:12:03 -05:00
Andrew Ayer e3835dea53 Add some comments about the nature of various errors 2023-02-03 13:59:40 -05:00
Andrew Ayer a2a2e40e15 Add merkletree package 2023-02-03 13:58:22 -05:00
Andrew Ayer 5236ac5ae8 Add Base64URLString to SHA256Hash 2023-02-03 13:58:01 -05:00
Andrew Ayer 57e9458ce5 Replace plain text NEWS file with Markdown-formatted CHANGELOG.md 2023-02-03 13:09:46 -05:00
Andrew Ayer 656fb065be logclient: improve some error messages 2023-01-29 13:22:17 -05:00
Andrew Ayer 5365450965 logclient: don't send User-Agent 2023-01-29 13:22:17 -05:00
Andrew Ayer 936a1ca8ed Remove ctparsewatch 2023-01-22 13:54:43 -05:00
Andrew Ayer b3d1b793c1 Remove unused helper functions 2023-01-21 17:20:39 -05:00
Andrew Ayer 95c823e86a logclient: optionally verify STH signatures 2023-01-21 16:53:43 -05:00
Andrew Ayer 654f8d4670 logclient: add GetEntriesRaw 2023-01-21 16:50:50 -05:00
Andrew Ayer 1cabee55e4 Remove an unused function 2023-01-20 16:50:49 -05:00
Andrew Ayer e682e1e9f8 Add some comments about script variables 2023-01-20 16:14:49 -05:00
Andrew Ayer 76d30c2033 Remove BygoneSSL documentation from the README
This feature will likely be removed in the future.

This feature can help you identify certificates that are
issued before you take ownership of a domain, helping you identify
certificates that are definitely not yours.

However, in practice this doesn't have very much utility:

1. Such certificates are probably already in CT when you start monitoring,
requiring you to download ALL certificates (by omitting -start_at_end)
to find them, which is not very practical.

2. It doesn't detect certificates that are issued based on reused domain
validations that were completed before you took ownership of the domain.
2023-01-16 18:07:28 -05:00
Andrew Ayer 34f5c857b6 Deprecate $FINGERPRINT and $PUBKEY_HASH -script variables
Replaced by $CERT_SHA256 and $PUBKEY_SHA256
2023-01-16 18:07:28 -05:00
Andrew Ayer fff3b01b26 Remove an obsolete TODO 2023-01-16 18:07:28 -05:00
Andrew Ayer cd2bb429fc Remove $CERT_TYPE environment variable for -script
Since first writing certspotter, I have witnessed many people
misunderstand precertificates, and do very bad things like ignore
precertificates under the invalid rationale that "precertificates
are not trusted by browsers."  While it's true that precertificates
themselves are not trusted by browsers, a precertificate implies the
existence of a corresponding certificate that IS trusted by browsers,
and there is NO guarantee that this certificate will ever be logged to CT.
(Sectigo and Let's Encrypt do log certificates but it's on a best-effort
basis and I don't know of any other CAs that do so.)  Therefore, if
you ignore precertificates you WILL fail to be alerted about potential
security threats.

While some PKI/CT researchers may care about the distinction between
certificates and precertificates, certspotter's primary purpose is to
help domain owners monitor their domains for misissued certificates.
Since there is no need to distinguish between certificates and
precertificates for that use case, I am removing $CERT_TYPE to prevent
people from shooting themselves in the foot.  Those who do have a valid
use case for distinguishing between certificates and precertificates
can always parse $CERT_FILE themselves.
2023-01-16 18:02:25 -05:00
Andrew Ayer 33ebbdfd07 Use os.UserHomeDir to determine home directory
Go provides this since Go 1.12; no need to reinvent the wheel.
2022-10-08 18:17:35 -04:00
Andrew Ayer 3d1bdb2b60 Release 0.14.0 2022-06-13 11:23:35 -04:00
Andrew Ayer 4c21e97208 Release 0.13 2022-06-13 08:50:07 -04:00
Andrew Ayer 0d29547d36 Update minimum Go version, install instructions 2022-06-13 08:45:23 -04:00
Andrew Ayer 270cdab44e Release 0.12 2022-06-07 08:23:35 -04:00
Andrew Ayer 039339154f Move retry logic into LogClient
This allows retry logic to be used for all requests, not just get-entries

Also add context arguments
2022-06-02 10:02:32 -04:00
Andrew Ayer f7f79f2600 logclient: buffer JSON request body
This permits us to detect JSON marshalling errors, and makes it easy to
retry the request.

Request bodies are short so this should have negligible performance impact.
2022-05-31 15:37:47 -04:00
Andrew Ayer c59eecfdec Switch to Go modules 2022-05-01 13:23:29 -04:00
Andrew Ayer 2335a57569 Drop dependency on github.com/mreiferson/go-httpclient 2022-05-01 13:22:01 -04:00
Andrew Ayer c0e79476ae Add .gitignore file 2022-02-28 08:11:13 -05:00
Andrew Ayer 31f0b8b830 Update loglist for Chrome's new v3 schema
See https://groups.google.com/a/chromium.org/d/msgid/ct-policy/f2958124-f679-406d-8bca-a063b7d37c3an%40chromium.org
2021-11-01 14:19:45 -04:00
Andrew Ayer 8c14597721 Add IsPreCert to CertInfo 2021-10-29 09:28:39 -04:00
Andrew Ayer c9aaa2782f Add -version flag 2021-10-12 10:28:34 -04:00
Andrew Ayer 54f34077d3 Release 0.11 2021-08-17 15:03:47 -04:00
Andrew Ayer 4e4250dad2 Don't ask for consistency proofs based on an empty tree
RFC 6962 doesn't define how to generate a consistency proof in this case,
and it doesn't matter anyways since the tree is empty.  The DigiCert logs
return a 400 error if we ask for such a proof.
2021-08-17 15:00:48 -04:00
Andrew Ayer 1a7622bfa6 loglist: add some helper functions 2021-05-01 17:35:18 -04:00
Andrew Ayer 4b280bdcd2 export loglist.Unmarshal 2021-05-01 16:53:56 -04:00
Andrew Ayer a147970db8 Use ct.SHA256Hash for log ID rather than []byte 2021-04-30 17:04:16 -04:00
Andrew Ayer 2cccf67601 Avoid leaving a file open for longer than necessary 2020-10-06 19:27:58 -04:00
Andrew Ayer 18b2d6d2a5 Add support for contacting logs via HTTP proxies
Just set the appropriate environment variable as documented at
https://golang.org/pkg/net/http/#ProxyFromEnvironment

Closes: #31
Closes: #41
2020-06-30 10:37:34 -04:00
Andrew Ayer 74a7329c00 Validate log list after loading it 2020-05-01 16:05:37 -04:00
Daniel Peukert 6d5e2395a1
Fix missing Printf 2020-05-01 00:25:39 +02:00
Andrew Ayer b01baf836d Release 0.10 2020-04-29 14:15:29 -04:00