ctcloud: CertSpotter-backed CT cloud-resource monitor
Single stateless Go process that consumes CT logs via embedded (patched) CertSpotter internals, classifies cloud/database/storage DNS identifiers in-process with a zero-allocation classifier, enriches via round-robin DNS resolvers with CNAME following, and batch-writes useful findings to PostgreSQL. - third_party/certspotter: 2-hunk IdentifierSink patch, no stdout/JSON/files - internal/classify: AWS/Azure/GCP/Firebase/Cloudflare R2/DigitalOcean/Supabase/OCI - internal/dns: sharded bounded TTL cache, multi-resolver load balancing + failover - internal/pipeline: bounded queues, dedup, direct-priority load shedding - internal/store: pgx batch writer, Postgres CT checkpoints (no cert contents) - internal/probe: optional non-destructive reachability checks (off by default) - Docker: distroless non-root, read-only rootfs, no app volume, resource limits - tests enforce 0 allocs/op on the classifier and sink hot path
This commit is contained in:
StasonJatham committed
1 parent
9e78a114ac
commit
e8f7d6bfdf
118 files changed
+16473
No files matched your search
@@ -0,0 +1 @@
|
||||
*.8
|
||||
Vendored
+9
@@ -0,0 +1,9 @@
|
||||
all: certspotter-script.8 certspotter.8 certspotter-authorize.8 certspotter-metrics.8
|
||||
|
||||
%.8: %.md
|
||||
lowdown -s -Tman \
|
||||
-M title:$(basename $(notdir $@)) \
|
||||
-M section:$(subst .,,$(suffix $@)) \
|
||||
-M date:$(if $(SOURCE_DATE_EPOCH),$(shell date -I -u -d "@$(SOURCE_DATE_EPOCH)"),$(shell date -I -u)) \
|
||||
-o $@ $<
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
# NAME
|
||||
|
||||
**certspotter-authorize** - Authorize certificates to suppress certspotter notifications
|
||||
|
||||
# SYNOPSIS
|
||||
|
||||
**certspotter-authorize** [`-printhash`] *CERTFILE*...
|
||||
|
||||
**certspotter-authorize** [`-printhash`] `-`
|
||||
|
||||
# DESCRIPTION
|
||||
|
||||
**certspotter-authorize** is a utility for preemptively authorizing
|
||||
certificates so that **certspotter(8)** will not send notifications when
|
||||
those certificates are discovered in Certificate Transparency logs.
|
||||
|
||||
This is useful for preventing false alarms when you know in advance that a
|
||||
certificate will be issued. For example, you might run **certspotter-authorize**
|
||||
immediately after receiving a certificate from your certificate authority, as
|
||||
part of your certificate issuance pipeline.
|
||||
|
||||
**certspotter-authorize** uses the TBSCertificate hash as defined by RFC
|
||||
6962 Section 3.2 to identify certificates. This hash is the same for a
|
||||
certificate and its corresponding precertificate. This means authorizing
|
||||
the certificate will suppress notifications for the precertificate
|
||||
as well. Certificates with different serial numbers, validity periods,
|
||||
or other changes to the TBSCertificate will not be covered by the authorization
|
||||
and will trigger notifications.
|
||||
|
||||
# ARGUMENTS
|
||||
|
||||
**certspotter-authorize** takes paths to one or more PEM-encoded certificates
|
||||
on the command line. If `-` is specified as the sole path, it reads the certificate
|
||||
from stdin.
|
||||
|
||||
# OPTIONS
|
||||
|
||||
-state\_dir *PATH*
|
||||
|
||||
: Directory where certspotter stores state. Defaults to
|
||||
`$CERTSPOTTER_STATE_DIR` if set, or `~/.certspotter` otherwise.
|
||||
This should be the same directory used by **certspotter(8)**.
|
||||
|
||||
-printhash
|
||||
|
||||
: Instead of authorizing the certificate, print its TBS hash (a hex-encoded SHA-256 digest) to stdout.
|
||||
|
||||
-version
|
||||
|
||||
: Print version information and exit.
|
||||
|
||||
# EXAMPLES
|
||||
|
||||
Authorize a certificate from a file:
|
||||
|
||||
$ certspotter-authorize /path/to/cert.pem
|
||||
|
||||
Authorize multiple certificates from files:
|
||||
|
||||
$ certspotter-authorize cert-a.pem cert-b.pem
|
||||
|
||||
Authorize a certificate from stdin:
|
||||
|
||||
$ cat cert.pem | certspotter-authorize -
|
||||
|
||||
Authorize a certificate in a custom state directory:
|
||||
|
||||
$ certspotter-authorize -state_dir /var/lib/certspotter cert.pem
|
||||
|
||||
Print the TBS hash without authorizing:
|
||||
|
||||
$ certspotter-authorize -printhash /path/to/cert.pem
|
||||
|
||||
# OPERATION
|
||||
|
||||
**certspotter-authorize** computes the SHA-256 hash of each
|
||||
certificate's TBSCertificate as defined by RFC 6962 Section 3.2 and
|
||||
creates a `.notified` marker file in the certspotter
|
||||
state directory. When certspotter later discovers a certificate with
|
||||
the same TBSCertificate in a CT log, it will skip sending notifications
|
||||
because the marker file is present.
|
||||
|
||||
# ENVIRONMENT
|
||||
|
||||
`CERTSPOTTER_STATE_DIR`
|
||||
|
||||
: Directory for storing state. Overridden by `-state_dir`. Defaults to
|
||||
`~/.certspotter`. This should be the same directory used by **certspotter(8)**.
|
||||
|
||||
# FILES
|
||||
|
||||
`$CERTSPOTTER_STATE_DIR/certs/XX/.HASH.notified`
|
||||
|
||||
: Marker files indicating that a certificate with TBS hash `HASH`
|
||||
has been authorized. `XX` is the first two hex digits of the hash.
|
||||
The file is empty; only its existence is checked.
|
||||
|
||||
# EXIT STATUS
|
||||
|
||||
**certspotter-authorize** exits with status 0 on success, 1 on error, or 2 on
|
||||
invalid usage.
|
||||
|
||||
# SEE ALSO
|
||||
|
||||
certspotter(8), certspotter-script(8)
|
||||
+128
@@ -0,0 +1,128 @@
|
||||
# NAME
|
||||
|
||||
**certspotter-metrics** - Output Prometheus metrics about certspotter's operation
|
||||
|
||||
# SYNOPSIS
|
||||
|
||||
**certspotter-metrics** [`-state_dir` *PATH*]
|
||||
|
||||
# DESCRIPTION
|
||||
|
||||
**certspotter-metrics** reads a certspotter state directory and writes
|
||||
Prometheus metrics, in the Prometheus text exposition format, about
|
||||
certspotter's progress monitoring each Certificate Transparency log.
|
||||
Metrics are written to standard output.
|
||||
|
||||
The intended use is to run **certspotter-metrics** periodically (e.g., from
|
||||
cron) and expose its output to Prometheus, for example with the
|
||||
prometheus-node-exporter(1) textfile collector.
|
||||
|
||||
**certspotter-metrics** only reads the state directory; it never modifies it.
|
||||
It is safe to run while certspotter is running, and it does not require
|
||||
certspotter to be running.
|
||||
|
||||
Metrics are output for every log for which certspotter has a state directory
|
||||
(`$STATE_DIR/logs/LOG_ID`, where `LOG_ID` is the base64url-encoded log ID).
|
||||
A log has a state directory once certspotter has started monitoring it.
|
||||
The `log_id` label on each metric is the standard base64 encoding (with
|
||||
padding) of the 32-byte log ID defined by RFC 6962. See <https://sslmate.com/app/ctlogs>
|
||||
for a list of human-friendly names for each log ID.
|
||||
|
||||
**certspotter-metrics** is experimental, and the metrics may change in a future release.
|
||||
|
||||
# OPTIONS
|
||||
|
||||
-state\_dir *PATH*
|
||||
|
||||
: Directory where certspotter stores state. Defaults to
|
||||
`$CERTSPOTTER_STATE_DIR` if set, or `~/.certspotter` otherwise.
|
||||
This should be the same directory used by **certspotter(8)**.
|
||||
|
||||
-version
|
||||
|
||||
: Print version information and exit.
|
||||
|
||||
# METRICS
|
||||
|
||||
All metrics are gauges with a `log_id` label identifying the log the
|
||||
metric pertains to. If a metric does not pertain to a log (because
|
||||
certspotter has not stored the relevant state for the log yet), the log
|
||||
does not have a sample for that metric.
|
||||
|
||||
certspotter\_log\_max\_sth\_size
|
||||
|
||||
: Tree size of the largest signed tree head (STH) certspotter has observed
|
||||
from the log; in other words, the number of entries in the log. Absent
|
||||
if certspotter has not yet downloaded an STH from the log.
|
||||
|
||||
certspotter\_log\_max\_sth\_timestamp
|
||||
|
||||
: Timestamp, in milliseconds since the Epoch, of the STH reported by
|
||||
certspotter\_log\_max\_sth\_size.
|
||||
|
||||
certspotter\_log\_download\_position
|
||||
|
||||
: Number of entries certspotter has downloaded from the log, including
|
||||
entries that have not yet been verified.
|
||||
|
||||
certspotter\_log\_verified\_position
|
||||
|
||||
: Number of entries certspotter has downloaded from the log and verified
|
||||
against an STH.
|
||||
|
||||
certspotter\_log\_verified\_sth\_timestamp
|
||||
|
||||
: Timestamp, in milliseconds since the Epoch, of the most recent STH
|
||||
certspotter has verified for the log. Absent if certspotter has not
|
||||
verified an STH for the log. The size of this STH is equal to
|
||||
certspotter\_log\_verified\_position.
|
||||
|
||||
certspotter\_log\_unverified\_sths
|
||||
|
||||
: Number of unverified STHs certspotter is currently storing for the log.
|
||||
|
||||
certspotter\_log\_malformed\_entries
|
||||
|
||||
: Number of malformed entries certspotter has recorded for the log.
|
||||
certspotter records an entry as malformed when it cannot parse the
|
||||
entry. See **certspotter(8)** for more information.
|
||||
|
||||
certspotter\_log\_health\_check\_failures
|
||||
|
||||
: Number of failed health checks certspotter has recorded for the log.
|
||||
See **certspotter(8)** for information about health checks.
|
||||
|
||||
# EXAMPLES
|
||||
|
||||
crontab(5) entry that exposes metrics to the node\_exporter textfile collector
|
||||
every five minutes (writing to a temporary file and renaming it
|
||||
ensures that node\_exporter never reads a partially-written file):
|
||||
|
||||
*/5 * * * * root certspotter-metrics > /var/lib/node_exporter/certspotter.prom.tmp && mv /var/lib/node_exporter/certspotter.prom.tmp /var/lib/node_exporter/certspotter.prom
|
||||
|
||||
Prometheus query to determine download backlog (number entries in the
|
||||
log that haven't been downloaded yet):
|
||||
|
||||
certspotter_log_max_sth_size-certspotter_log_download_position
|
||||
|
||||
# ENVIRONMENT
|
||||
|
||||
`CERTSPOTTER_STATE_DIR`
|
||||
|
||||
: Directory for storing state. Overridden by `-state_dir`. Defaults to
|
||||
`~/.certspotter`. This should be the same directory used by
|
||||
**certspotter(8)**.
|
||||
|
||||
`STATE_DIRECTORY`
|
||||
|
||||
: Directory for storing state, if set and certspotter-metrics is running
|
||||
under systemd.
|
||||
|
||||
# EXIT STATUS
|
||||
|
||||
**certspotter-metrics** exits with status 0 on success, 1 on error, or 2 on
|
||||
invalid usage.
|
||||
|
||||
# SEE ALSO
|
||||
|
||||
certspotter(8)
|
||||
+296
@@ -0,0 +1,296 @@
|
||||
# NAME
|
||||
|
||||
**certspotter-script** - Certificate Transparency Log Monitor (hook script)
|
||||
|
||||
# DESCRIPTION
|
||||
|
||||
**certspotter-script** is *any* program that is executed by **certspotter(8)**
|
||||
when it needs to notify you about an event, such as detecting a certificate for
|
||||
a domain on your watch list.
|
||||
|
||||
Scripts are placed in the `$CERTSPOTTER_CONFIG_DIR/hooks.d` directory
|
||||
(`~/.certspotter/hooks.d` by default), or specified on the command line
|
||||
using the `-script` argument.
|
||||
|
||||
# ENVIRONMENT
|
||||
|
||||
## Event information
|
||||
|
||||
The following environment variables are set for all types of events:
|
||||
|
||||
`EVENT`
|
||||
|
||||
: One of the following values, indicating the type of event:
|
||||
|
||||
* `discovered_cert` - certspotter has discovered a certificate for a
|
||||
domain on your watch list.
|
||||
|
||||
* `malformed_cert` - certspotter can't determine if a certificate
|
||||
matches your watch list because the certificate or the log entry
|
||||
is malformed.
|
||||
|
||||
* `error` - a problem is preventing certspotter from monitoring all
|
||||
logs.
|
||||
|
||||
Additional event types may be defined in the future, so your script should
|
||||
be able to handle unknown values.
|
||||
|
||||
`SUMMARY`
|
||||
|
||||
: A short human-readable string describing the event. This is the same string
|
||||
used in the subject line of emails sent by certspotter.
|
||||
|
||||
|
||||
## Discovered certificate information
|
||||
|
||||
The following environment variables are set for `discovered_cert` events:
|
||||
|
||||
`WATCH_ITEM`
|
||||
|
||||
: The item from your watch list which matches this certificate.
|
||||
(If more than one item matches, the first one is used.)
|
||||
|
||||
`LOG_URI`
|
||||
|
||||
: The URI of the log containing the certificate.
|
||||
|
||||
`ENTRY_INDEX`
|
||||
|
||||
: The index of the log entry containing the certificate.
|
||||
|
||||
`TBS_SHA256`
|
||||
|
||||
: The hex-encoded SHA-256 digest of the TBSCertificate, as defined in RFC 6962 Section 3.2.
|
||||
Certificates and their corresponding precertificates have the same `TBS_SHA256` value.
|
||||
You can compute the TBS digest of a certificate using the `-printhash` option of the certspotter-authorize(8) command.
|
||||
|
||||
`CERT_SHA256`
|
||||
|
||||
: The hex-encoded SHA-256 digest (sometimes called fingerprint) of the certificate or precertificate.
|
||||
The digest is computed over the ASN.1 DER encoding.
|
||||
|
||||
`PUBKEY_SHA256`
|
||||
|
||||
: The hex-encoded SHA-256 digest of the certificate's Subject Public Key Info.
|
||||
|
||||
`CERT_FILENAME`
|
||||
|
||||
: Path to a file containing the PEM-encoded certificate chain. When `-no_save` is used, this points to a temporary file that is automatically deleted after your script completes.
|
||||
|
||||
`JSON_FILENAME`
|
||||
|
||||
: Path to a JSON file containing additional information about the certificate. See below for the format of the JSON file.
|
||||
When `-no_save` is used, this points to a temporary file that is automatically deleted after your script completes.
|
||||
|
||||
`TEXT_FILENAME`
|
||||
|
||||
: Path to a text file containing information about the certificate. This file contains the same text that
|
||||
certspotter uses in emails. You should not attempt to parse this file because its format may change in the future.
|
||||
When `-no_save` is used, this points to a temporary file that is automatically deleted after your script completes.
|
||||
|
||||
`NOT_BEFORE`, `NOT_BEFORE_UNIXTIME`, `NOT_BEFORE_RFC3339`
|
||||
|
||||
: The not before time of the certificate, in a human-readable format, seconds since the UNIX epoch, and RFC3339, respectively. These variables may be unset if there was a parse error, in which case `VALIDITY_PARSE_ERROR` is set.
|
||||
|
||||
`NOT_AFTER`, `NOT_AFTER_UNIXTIME`, `NOT_AFTER_RFC3339`
|
||||
|
||||
: The not after (expiration) time of the certificate, in a human-readable format, seconds since the UNIX epoch, and RFC3339, respectively. These variables may be unset if there was a parse error, in which case `VALIDITY_PARSE_ERROR` is set.
|
||||
|
||||
`VALIDITY_PARSE_ERROR`
|
||||
|
||||
: Error parsing not before and not after, if any. If this variable is set, then the `NOT_BEFORE` and `NOT_AFTER` family of variables are unset.
|
||||
|
||||
`SUBJECT_DN`
|
||||
|
||||
: The distinguished name of the certificate's subject. This variable may be unset if there was a parse error, in which case `SUBJECT_PARSE_ERROR` is set.
|
||||
|
||||
`SUBJECT_PARSE_ERROR`
|
||||
|
||||
: Error parsing the subject, if any. If this variable is set, then `SUBJECT_DN` is unset.
|
||||
|
||||
`ISSUER_DN`
|
||||
|
||||
: The distinguished name of the certificate's issuer. This variable may be unset if there was a parse error, in which case `ISSUER_PARSE_ERROR` is set. Be aware that this value is controlled by the certificate authority and often contains [misleading, incorrect, or unhelpful information](https://www.agwa.name/blog/post/the_certificate_issuer_field_is_a_lie).
|
||||
|
||||
`ISSUER_PARSE_ERROR`
|
||||
|
||||
: Error parsing the issuer, if any. If this variable is set, then `ISSUER_DN` is unset.
|
||||
|
||||
`SERIAL`
|
||||
|
||||
: The hex-encoded serial number of the certificate. Prefixed with a minus (-) sign if negative. This variable may be unset if there was a parse error, in which case `SERIAL_PARSE_ERROR` is set. Be aware that malicious certificate authorities can issue certificates with duplicate serial numbers, so it is not safe to use the serial number to make security decisions (such as to determine if a certificate is legitimate or not).
|
||||
|
||||
`SERIAL_PARSE_ERROR`
|
||||
|
||||
: Error parsing the serial number, if any. If this variable is set, then `SERIAL` is unset.
|
||||
|
||||
`CHAIN_ERROR`
|
||||
|
||||
: Error building or verifying the certificate chain, if any. If this variable is set, then the certificate chain in `CERT_FILENAME` may be incomplete or invalid.
|
||||
|
||||
## Malformed certificate information
|
||||
|
||||
The following environment variables are set for `malformed_cert` events:
|
||||
|
||||
`LOG_URI`
|
||||
|
||||
: The URI of the log containing the malformed certificate.
|
||||
|
||||
`ENTRY_INDEX`
|
||||
|
||||
: The index of the log entry containing the malformed certificate.
|
||||
|
||||
`LEAF_HASH`
|
||||
|
||||
: The base64-encoded Merkle hash of the leaf containing the malformed certificate.
|
||||
|
||||
`PARSE_ERROR`
|
||||
|
||||
: A human-readable string describing why the certificate is malformed.
|
||||
|
||||
`ENTRY_FILENAME`
|
||||
|
||||
: Path to a file containing the JSON log entry. The file contains a JSON object with two fields, `leaf_input` and `extra_data`, as described in RFC 6962 Section 4.6.
|
||||
|
||||
`TEXT_FILENAME`
|
||||
|
||||
: Path to a text file containing a description of the malformed certificate. This file contains the same text that certspotter uses in emails.
|
||||
|
||||
## Error information
|
||||
|
||||
The following environment variables are set for `error` events:
|
||||
|
||||
`TEXT_FILENAME`
|
||||
|
||||
: Path to a text file containing a description of the error. This file contains the same text that certspotter uses in emails.
|
||||
|
||||
# JSON FILE FORMAT
|
||||
|
||||
certspotter saves a JSON file for every discovered certificate and puts the path to the file in `$JSON_FILENAME`.
|
||||
When `-no_save` is used, the JSON file is created in a temporary location and automatically deleted after your script completes.
|
||||
Otherwise, it is saved in the state directory for permanent storage. Your script can read the JSON file, such as with the jq(1) command,
|
||||
to get additional information about the certificate which isn't appropriate for environment variables.
|
||||
|
||||
The JSON file contains an object with the following fields:
|
||||
|
||||
`tbs_sha256`
|
||||
|
||||
: A string containing the hex-encoded SHA-256 digest of the TBSCertificate, as defined in RFC 6962 Section 3.2.
|
||||
Certificates and their corresponding precertificates have the same `tbs_sha256` value. You can compute the
|
||||
TBS digest of a certificate using the `-printhash` option of the certspotter-authorize(8) command.
|
||||
|
||||
`pubkey_sha256`
|
||||
|
||||
: A string containing the hex-encoded SHA-256 digest of the certificate's Subject Public Key Info.
|
||||
|
||||
`dns_names`
|
||||
|
||||
: An array of strings containing the DNS names for which the
|
||||
certificate is valid, taken from both the DNS subject alternative names
|
||||
(SANs) and the subject common name (CN). Internationalized domain names
|
||||
are encoded in Punycode.
|
||||
|
||||
`ip_addresses`
|
||||
|
||||
: An array of strings containing the IP addresses for which the certificate is valid,
|
||||
taken from both the IP subject alternative names (SANs) and the subject common name (CN).
|
||||
|
||||
`not_before`
|
||||
|
||||
: A string containing the not before time of the certificate in RFC3339 format.
|
||||
Null if there was an error parsing the certificate's validity.
|
||||
|
||||
`not_after`
|
||||
|
||||
: A string containing the not after (expiration) time of the certificate in RFC3339 format.
|
||||
Null if there was an error parsing the certificate's validity.
|
||||
|
||||
Additional fields will be added in the future based on user feedback. Please open
|
||||
an issue at <https://github.com/SSLMate/certspotter> if you have a use case for another field.
|
||||
|
||||
# DETERMINING IF A CERTIFICATE IS AUTHORIZED
|
||||
|
||||
You may want to compare a discovered certificate against a database of legitimate certificates
|
||||
to decide whether to raise an alarm about it. Beware the following caveats:
|
||||
|
||||
* Since certificates may be logged to CT in the form of a precertificate, it doesn't work
|
||||
to compare the discovered certificate's fingerprint against a list of legitimate fingerprints.
|
||||
|
||||
* Since malicious certificate authorities can issue certificates with duplicate serial numbers,
|
||||
it is not secure to compare the the discovered certificate's serial number against a list of
|
||||
legitimate serial numbers.
|
||||
|
||||
* Since malicious certificate authorities can put any value in a certificate's Subject Key Identifier (SKI)
|
||||
extension, it is not secure to compare a discovered certificate's Subject Key Identifier against
|
||||
a list of legitimate Subject Key Identifiers.
|
||||
|
||||
* Since malicious certificate authorities can put any value in the certificate's Issuer field,
|
||||
it is not secure to compare a discovered certificate's Issuer against a list of authorized Issuers.
|
||||
|
||||
To determine if a certificate is legitimate, you should do one of the following:
|
||||
|
||||
* Compare the discovered certificate's TBS hash (as provided in the `$TBS_SHA256` variable) against
|
||||
a list of authorized TBS hashes. The TBS hash is computed over the DER encoding of the certificate's
|
||||
TBSCertificate, *with any SCT extension removed*. You can use the `-printhash` option
|
||||
of the certspotter-authorize(8) command to compute a certificate's TBS hash.
|
||||
|
||||
* Compare the discovered certificate's public key hash (as provided in the `$PUBKEY_SHA256` variable) against
|
||||
a list of authorized public key hashes. The public key hash is computed over the DER encoding of the
|
||||
certificate's Subject Public Key Info. You can use the following openssl command to compute a certificate's
|
||||
public key hash: `openssl x509 -pubkey -noout | openssl pkey -pubin -outform DER | openssl sha256`
|
||||
|
||||
|
||||
# EXAMPLES
|
||||
|
||||
Example environment variables for a `discovered_cert` event:
|
||||
|
||||
```
|
||||
CERT_FILENAME=/home/andrew/.certspotter/certs/3c/3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a.pem
|
||||
CERT_SHA256=3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a
|
||||
ENTRY_INDEX=6464843
|
||||
EVENT=discovered_cert
|
||||
ISSUER_DN=C=GB, ST=Greater Manchester, L=Salford, O=Sectigo Limited, CN=Sectigo RSA Domain Validation Secure Server CA
|
||||
JSON_FILENAME=/usr2/andrew/.certspotter/certs/3c/3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a.v1.json
|
||||
LOG_URI=https://ct.cloudflare.com/logs/nimbus2024/
|
||||
NOT_AFTER='2024-01-26 03:47:26 +0000 UTC'
|
||||
NOT_AFTER_RFC3339=2024-01-26T03:47:26Z
|
||||
NOT_AFTER_UNIXTIME=1706240846
|
||||
NOT_BEFORE='2023-01-31 03:47:26 +0000 UTC'
|
||||
NOT_BEFORE_RFC3339=2023-01-31T03:47:26Z
|
||||
NOT_BEFORE_UNIXTIME=1675136846
|
||||
PUBKEY_SHA256=33ac1d9b9e56005ccac045eac2398b3e9dd6b3f5b66ae6260f2d478c7c0d82c8
|
||||
SERIAL=c170fbf3bf27481e5c351a4db6f2dc5f
|
||||
SUBJECT_DN=CN=sslmate.com
|
||||
SUMMARY='certificate discovered for .sslmate.com'
|
||||
TBS_SHA256=2388ee81c6f45cffc73e68a35fa8921e839e20acc9a98e8e6dcaea07cbfbdef8
|
||||
TEXT_FILENAME=/usr2/andrew/.certspotter/certs/3c/3cdc83b3932c194fcdf17aa2bf1abc34e8438b293c3d5c70693e175b38ff128a.txt
|
||||
WATCH_ITEM=.sslmate.com
|
||||
```
|
||||
|
||||
Example JSON file for a discovered certificate:
|
||||
|
||||
```
|
||||
{
|
||||
"dns_names": [
|
||||
"sslmate.com",
|
||||
"www.sslmate.com"
|
||||
],
|
||||
"ip_addresses": [],
|
||||
"not_after": "2024-01-26T03:47:26Z",
|
||||
"not_before": "2023-01-31T03:47:26Z",
|
||||
"pubkey_sha256": "33ac1d9b9e56005ccac045eac2398b3e9dd6b3f5b66ae6260f2d478c7c0d82c8",
|
||||
"tbs_sha256": "2388ee81c6f45cffc73e68a35fa8921e839e20acc9a98e8e6dcaea07cbfbdef8"
|
||||
}
|
||||
```
|
||||
|
||||
# SEE ALSO
|
||||
|
||||
certspotter(8)
|
||||
|
||||
# COPYRIGHT
|
||||
|
||||
Copyright (c) 2016-2025 Opsmate, Inc.
|
||||
|
||||
# BUGS
|
||||
|
||||
Report bugs to <https://github.com/SSLMate/certspotter>.
|
||||
+261
@@ -0,0 +1,261 @@
|
||||
# NAME
|
||||
|
||||
**certspotter** - Certificate Transparency Log Monitor
|
||||
|
||||
# SYNOPSIS
|
||||
|
||||
**certspotter** [`-start_at_end`] [`-watchlist` *FILENAME*] [`-email` *ADDRESS*] `...`
|
||||
|
||||
# DESCRIPTION
|
||||
|
||||
**Cert Spotter** is a Certificate Transparency log monitor from SSLMate that
|
||||
alerts you when a SSL/TLS certificate is issued for one of your domains.
|
||||
Cert Spotter is easier to use than other open source CT monitors, since it does not require
|
||||
a database. It's also more robust, since it uses a special certificate parser
|
||||
that ensures it won't miss certificates.
|
||||
|
||||
Cert Spotter is also available as a hosted service by SSLMate,
|
||||
<https://sslmate.com/certspotter>.
|
||||
|
||||
You can use Cert Spotter to detect:
|
||||
|
||||
* Certificates issued to attackers who have compromised your DNS and
|
||||
are redirecting your visitors to their malicious site.
|
||||
* Certificates issued to attackers who have taken over an abandoned
|
||||
sub-domain in order to serve malware under your name.
|
||||
* Certificates issued to attackers who have compromised a certificate
|
||||
authority and want to impersonate your site.
|
||||
* Certificates issued in violation of your corporate policy
|
||||
or outside of your centralized certificate procurement process.
|
||||
|
||||
# OPTIONS
|
||||
|
||||
-email *ADDRESS*
|
||||
|
||||
: Email address to contact when a matching certificate is discovered, or
|
||||
an error occurs. You can specify this option more than once to email
|
||||
multiple addresses. Your system must have a working sendmail(1) command.
|
||||
|
||||
Regardless of the `-email` option, certspotter also emails any address listed
|
||||
in `$CERTSPOTTER_CONFIG_DIR/email_recipients` file
|
||||
(`~/.certspotter/email_recipients` by default). (One address per line,
|
||||
blank lines are ignored.) This file is read only at startup, so you
|
||||
must restart certspotter if you change it.
|
||||
|
||||
-healthcheck *INTERVAL*
|
||||
|
||||
: Perform a health check at the given interval (default: "24h") as described
|
||||
below. *INTERVAL* must be a decimal number followed by "h" for hours or
|
||||
"m" for minutes.
|
||||
|
||||
-logs *ADDRESS*
|
||||
|
||||
: Filename or HTTPS URL of a v2 or v3 JSON log list containing logs to monitor.
|
||||
The schema for this file can be found at <https://www.gstatic.com/ct/log_list/v3/log_list_schema.json>.
|
||||
Defaults to <https://loglist.certspotter.org/monitor.json>, which includes
|
||||
the union of active logs recognized by Chrome and Apple. certspotter loads the
|
||||
log list when starting up, and periodically reloads it in case it has changed.
|
||||
|
||||
-no\_save
|
||||
|
||||
: Do not save a copy of matching certificates. Note that enabling this option
|
||||
will cause you to receive duplicate notifications, since certspotter will
|
||||
have no way of knowing if you've been previously notified about a certificate.
|
||||
|
||||
-script *COMMAND*
|
||||
|
||||
: Command to execute when a matching certificate is found or an error occurs. See
|
||||
certspotter-script(8) for information about the interface to scripts.
|
||||
|
||||
Regardless of the `-script` option, certspotter also executes any executable
|
||||
file in the `$CERTSPOTTER_CONFIG_DIR/hooks.d` directory
|
||||
(`~/.certspotter/hooks.d` by default).
|
||||
|
||||
-start\_at\_end
|
||||
|
||||
: Start monitoring logs from the end rather than the beginning.
|
||||
|
||||
**WARNING**: monitoring from the beginning guarantees detection of all
|
||||
certificates, but requires downloading hundreds of millions of
|
||||
certificates, which takes days.
|
||||
|
||||
-state\_dir *PATH*
|
||||
|
||||
: Directory for storing state. Defaults to `$CERTSPOTTER_STATE_DIR`, which is
|
||||
"~/.certspotter" by default.
|
||||
|
||||
-stdout
|
||||
|
||||
: Write matching certificates and errors to stdout.
|
||||
|
||||
-verbose
|
||||
|
||||
: Print detailed information about certspotter's operation to stderr.
|
||||
|
||||
-version
|
||||
|
||||
: Print version and exit.
|
||||
|
||||
-watchlist *PATH*
|
||||
|
||||
: File containing DNS names to monitor, one per line. To monitor an entire
|
||||
domain namespace (including the domain itself and all sub-domains) prefix
|
||||
the domain name with a dot (e.g. ".example.com"). To monitor a single DNS
|
||||
name only, do not prefix the name with a dot.
|
||||
|
||||
Defaults to `$CERTSPOTTER_CONFIG_DIR/watchlist`, which is
|
||||
"~/.certspotter/watchlist" by default.
|
||||
Specify `-` to read the watch list from stdin.
|
||||
|
||||
certspotter reads the watch list only when starting up, so you must restart
|
||||
certspotter if you change it.
|
||||
|
||||
# NOTIFICATIONS
|
||||
|
||||
When certspotter detects a certificate matching your watchlist, or encounters
|
||||
an error that is preventing it from discovering certificates, it notifies you
|
||||
as follows:
|
||||
|
||||
* Emails any address specified by the `-email` command line flag.
|
||||
|
||||
* Emails any address listed in the `$CERTSPOTTER_CONFIG_DIR/email_recipients`
|
||||
file (`~/.certspotter/email_recipients` by default). (One address per line,
|
||||
blank lines are ignored.) This file is read only at startup, so you
|
||||
must restart certspotter if you change it.
|
||||
|
||||
* Executes the script specified by the `-script` command line flag.
|
||||
|
||||
* Executes every executable file in the `$CERTSPOTTER_CONFIG_DIR/hooks.d`
|
||||
directory (`~/.certspotter/hooks.d` by default).
|
||||
|
||||
* Writes the notification to standard out if the `-stdout` flag was specified.
|
||||
|
||||
Sending email requires a working sendmail(1) command. For details about
|
||||
the script interface, see certspotter-script(8).
|
||||
|
||||
# OPERATION
|
||||
|
||||
certspotter continuously monitors all browser-recognized Certificate
|
||||
Transparency logs (both RFC6962 and static-ct-api) looking for certificates (including precertificates)
|
||||
which are valid for any domain on your watch list. When certspotter
|
||||
detects a matching certificate, it emails you, executes a script, and/or
|
||||
writes a report to standard out, as described above.
|
||||
|
||||
certspotter also saves a copy of matching certificates in
|
||||
`$CERTSPOTTER_STATE_DIR/certs` ("~/.certspotter/certs" by default)
|
||||
unless you specify the `-no_save` option.
|
||||
|
||||
When certspotter has not previously monitored a log, it can either start
|
||||
monitoring the log from the beginning, or seek to the end of the log and
|
||||
start monitoring from there. Monitoring from the beginning guarantees
|
||||
detection of all certificates, but requires downloading hundreds of
|
||||
millions of certificates, which takes days. The default behavior is to
|
||||
monitor from the beginning. To start monitoring new logs from the end,
|
||||
specify the `-start_at_end` option.
|
||||
|
||||
If certspotter has previously monitored a log, it resumes monitoring
|
||||
the log from the previous position. This means that if you add
|
||||
a domain to your watch list, certspotter will not detect any certificates
|
||||
that were logged prior to the addition. To detect such certificates,
|
||||
you must delete `$CERTSPOTTER_STATE_DIR/logs`, which will cause certspotter
|
||||
to restart monitoring from the very beginning of each log (provided
|
||||
`-start_at_end` is not specified). This will cause certspotter to download
|
||||
hundreds of millions of certificates, which takes days. To find preexisting
|
||||
certificates, it's faster to use the Cert Spotter service
|
||||
<https://sslmate.com/certspotter>, SSLMate's Certificate Transparency Search
|
||||
API <https://sslmate.com/ct_search_api>, or a CT search engine such as
|
||||
<https://crt.sh>.
|
||||
|
||||
# ERROR HANDLING
|
||||
|
||||
When certspotter encounters a problem with the local system (e.g. failure
|
||||
to write a file, send an email, or execute a script), it prints a message to stderr and
|
||||
exits with a non-zero status.
|
||||
|
||||
When certspotter encounters a problem loading the log list during startup, it
|
||||
prints a message to stderr and exits with a non-zero status. When certspotter encounters a problem
|
||||
reloading the log list, it prints a message to stderr and continues running with the previously-loaded
|
||||
log list. It will try reloading the log list again later.
|
||||
|
||||
When certspotter encounters a problem contacting a log, it writes the error to a file in
|
||||
the state directory and continues running. It will try contacting the log again later;
|
||||
most log errors are transient. The last 7 days of errors are kept.
|
||||
|
||||
Every 24 hours (unless overridden by `-healthcheck`), certspotter performs the
|
||||
following health checks:
|
||||
|
||||
* Ensure that the log list has been successfully retrieved at least once
|
||||
since the previous health check.
|
||||
* Ensure that every log has been successfully contacted at least once
|
||||
since the previous health check.
|
||||
* Ensure that certspotter is not falling behind monitoring any logs.
|
||||
|
||||
If any health check fails, certspotter notifies you by email, script, and/or
|
||||
standard out, as described above. The notification includes the last several errors
|
||||
encountered when contacting the log.
|
||||
|
||||
Health check failures should be rare, and you should take them seriously because it means
|
||||
certspotter might not detect all certificates. It might also be an indication
|
||||
of CT log misbehavior. Check the error files for details, and if
|
||||
you need help, file an issue at <https://github.com/SSLMate/certspotter>.
|
||||
|
||||
# EXIT STATUS
|
||||
|
||||
certspotter exits 0 when it receives `SIGTERM` or `SIGINT`,
|
||||
and non-zero when a serious error occurs.
|
||||
|
||||
# ENVIRONMENT
|
||||
|
||||
`CERTSPOTTER_STATE_DIR`
|
||||
|
||||
: Directory for storing state. Overridden by `-state_dir`. Defaults to
|
||||
`~/.certspotter`.
|
||||
|
||||
`CERTSPOTTER_CONFIG_DIR`
|
||||
|
||||
: Directory from which any configuration, such as the watch list, is read.
|
||||
Defaults to `~/.certspotter`.
|
||||
|
||||
`EMAIL`
|
||||
|
||||
: Email address from which to send emails. If not set, certspotter lets sendmail pick
|
||||
the address.
|
||||
|
||||
`HTTPS_PROXY`
|
||||
|
||||
: URL of proxy server for making HTTPS requests. `http://`, `https://`, and
|
||||
`socks5://` URLs are supported. By default, no proxy server is used.
|
||||
|
||||
`SENDMAIL_PATH`
|
||||
|
||||
: Path to the sendmail binary used for sending emails. Defaults to `/usr/sbin/sendmail`.
|
||||
|
||||
If the `SYSTEMD_EXEC_PID` environment variable equals the PID of certspotter, then certspotter assumes it is running under systemd and respects the `CONFIGURATION_DIRECTORY`, `STATE_DIRECTORY`, and `CACHE_DIRECTORY` environment variables, unless overridden by a certspotter-specific environment variable or flag.
|
||||
|
||||
certspotter respects the `XDG_CACHE_HOME` environment variable for locating its cache directory. For historical reasons, certspotter does not respect `XDG_CONFIG_HOME` or `XDG_STATE_HOME`.
|
||||
|
||||
# DIRECTORIES
|
||||
|
||||
Config directory
|
||||
|
||||
: Stores configuration, such as the watch list. The location is: (1) the `CERTSPOTTER_CONFIG_DIR` environment variable, if set, (2) the `CONFIGURATION_DIRECTORY` environment variable, if set and certspotter is running under systemd, or (3) the default location `~/.certspotter`. certspotter does not write to this directory.
|
||||
|
||||
State directory
|
||||
|
||||
: Stores state, such as the position of each log and a store of discovered certificates. The location is: (1) the `-state_dir` command line flag, if provided, (2) the `CERTSPOTTER_STATE_DIR` environment variable, if set, (3) the `STATE_DIRECTORY` environment variable, if set and certspotter is running under systemd, or (4) the default location `~/.certspotter`. certspotter creates this directory if necessary.
|
||||
|
||||
Cache directory
|
||||
|
||||
: Stores cached data. The location is (1) the `CERTSPOTTER_CACHE_DIR` environment variable if set, (2) the `CACHE_DIRECTORY` environment, if set and certspotter is running under systemd, or (3) `$XDG_CACHE_HOME/certspotter` (which on Linux is `~/.cache/certspotter` by default). You can delete this directory without impacting functionality, but certspotter may need to perform additional computation or network requests.
|
||||
|
||||
# SEE ALSO
|
||||
|
||||
certspotter-script(8)
|
||||
|
||||
# COPYRIGHT
|
||||
|
||||
Copyright (c) 2016-2025 Opsmate, Inc.
|
||||
|
||||
# BUGS
|
||||
|
||||
Report bugs to <https://github.com/SSLMate/certspotter>.
|
||||
Reference in new issue
Block a user