ctmon

Tells you when a new subdomain shows up in Certificate Transparency logs for the domains you watch, and sends it to Discord.

You add the domains once, leave the monitor running, and read a small SQLite file whenever you want the hostname list. There is no dashboard, no database server, and no notification framework: one fish script, one SQLite file, one cron entry.

Quick start

From /Users/quan.m.le/Tools/ctmon:

./ctmon.fish help                 # commands, environment variables, file locations
./ctmon.fish init                 # create the database and seed scopes (safe to re-run)
./ctmon.fish start                # begin monitoring in the foreground

In a second terminal, watch what it finds:

./ctmon.fish list                 # every hostname discovered so far
./ctmon.fish pending              # hostnames not yet sent to Discord

Notifications go out on their own through cron. To send now:

./ctmon.fish notify

To run the monitor after closing the terminal, keep it detached and log outside the project directory:

nohup /Users/quan.m.le/Tools/ctmon/ctmon.fish start >> ~/Library/Logs/ctmon/start.log 2>&1 &

What it does

Public CT logs
    |
    v
certspotter            continuous, resumable, state in ./state
    |  watchlist (scopes from SQLite, piped on stdin)
    |  -script ./ctmon.fish   (hook call per matching certificate)
    v
./ctmon.fish hook  --->  inventory.sqlite   (domains table, one row per hostname)
    |
    +--> ./ctmon.fish list / ./ctmon.fish pending / any sqlite3 query
    |
    +--> ./ctmon.fish notify  (cron)
              curl --> Discord webhook from pass

Three parts, each replaceable on its own: certspotter watches, SQLite stores, ct notify delivers. The hook never depends on Discord being reachable, so a webhook outage cannot lose a discovery.

Commands

Command	Behavior
./ctmon.fish init	Create the schema and seed scopes. Idempotent; never wipes data.
./ctmon.fish set-logs	Write a trimmed CT log list to state/logs.json. start uses it when present.
./ctmon.fish start	Run certspotter continuously in the foreground.
./ctmon.fish notify	Send pending Discord batches. Cron runs this.
./ctmon.fish list	Print all discovered hostnames.
./ctmon.fish pending	Print hostnames not yet sent to Discord.
./ctmon.fish help	Print usage. -h and --help do the same.

./ctmon.fish help prints the current list; the table above mirrors its output.

The same executable is the Cert Spotter hook. When EVENT is set in the environment it reads the certificate and exits, rather than treating the call as a command. You never invoke that mode by hand.

Scopes

A scope is one row in the scopes table. .example.com with excluded=0 monitors the apex and the whole subdomain tree. Without the leading dot it monitors only that exact host.

Names are compared on whole DNS labels, so lookalikes never slip in:

Hostname	.example.com scope
example.com	matches (apex)
api.example.com	matches
dev.api.example.com	matches
notexample.com	no match
example.com.evil.org	no match

Exclusion and inclusion resolve by the most specific matching scope, not by a blanket rule. A longer pattern beats a shorter one, and on a tie the excluded scope wins. That gives two useful behaviors:

• .internal.example.com excluded beats .example.com included, so the whole subtree is dropped.
• api.internal.example.com included beats .internal.example.com excluded, so one host can be pulled back out of an excluded subtree.

Scope rows	api.internal.example.com	Outcome
.example.com include only	matched	stored
plus .internal.example.com exclude	excluded wins	dropped
plus api.internal.example.com include	longest match	stored

# include a tree
sqlite3 inventory.sqlite "INSERT OR IGNORE INTO scopes(pattern, excluded, program) VALUES('.example.com', 0, 'capitalcom');"

# include one exact host
sqlite3 inventory.sqlite "INSERT OR IGNORE INTO scopes(pattern, excluded, program) VALUES('api.example.com', 0, 'capitalcom');"

# exclude a subtree (wins over an included parent)
sqlite3 inventory.sqlite "INSERT OR IGNORE INTO scopes(pattern, excluded, program) VALUES('.internal.example.com', 1, 'capitalcom');"

# list, then remove
sqlite3 inventory.sqlite "SELECT pattern, excluded, program FROM scopes;"
sqlite3 inventory.sqlite "DELETE FROM scopes WHERE pattern='.internal.example.com';"

Restart required. certspotter reads the watchlist once at startup, so scope changes take effect on the next ./ctmon.fish start. A running monitor keeps the old list until you restart it.

program groups notifications. Every scope currently uses capitalcom; if you add a second program label, notifications arrive as one digest per program.

Database

Three tables, one file, readable by any SQLite client.

scopes(pattern TEXT PRIMARY KEY,
       excluded INTEGER NOT NULL DEFAULT 0,     -- 0 include, 1 exclude
       program  TEXT NOT NULL DEFAULT 'capitalcom')

domains(fqdn TEXT PRIMARY KEY,                  -- unique, so dedup is free
        program TEXT NOT NULL DEFAULT 'capitalcom',
        first_seen TEXT NOT NULL,               -- UTC ISO 8601
        discord_sent_at TEXT)                   -- NULL means pending

webhooks(program TEXT PRIMARY KEY,              -- one row per program
         pass_entry TEXT NOT NULL)              -- pass entry holding the webhook URL

Indexes: domains(discord_sent_at) for the pending scan, domains(program) for digest grouping.

Queries you are likely to run:

# all hosts
sqlite3 inventory.sqlite "SELECT fqdn FROM domains ORDER BY fqdn;"

# discovered in the last 24 hours
sqlite3 inventory.sqlite "SELECT fqdn FROM domains WHERE first_seen >= datetime('now','-24 hours');"

# still awaiting Discord
sqlite3 inventory.sqlite "SELECT fqdn FROM domains WHERE discord_sent_at IS NULL;"

# counts per program
sqlite3 inventory.sqlite "SELECT program, count(*) FROM domains GROUP BY program;"

# newline list into the clipboard, ready for another tool
sqlite3 inventory.sqlite "SELECT fqdn FROM domains ORDER BY fqdn;" | pbcopy

SQLite is the single source of truth. There is no second scope file or export to keep in sync.

Notifications

• Each program sends to the webhook named in webhooks.pass_entry. The URL lives in pass and is read at send time. It is never written into this directory. capitalcom uses messaging/discord/webhook/ctmon.
• Two layouts are valid, and notify refuses any other:
  • 1-to-N: one pass entry for every program. All digests go to one channel.
  • N-to-N: one pass entry per program. Each program gets its own channel.
  • Anything in between (N-to-M) mixes programs in a channel, so notify logs notify skipped: webhook routing is N-to-M and sends nothing.
• A program with no webhooks row, or with a pass entry that cannot be read, stays pending. Other programs still send.
• A message never mixes programs. A program can span many messages.

To add a program in its own channel, store its webhook in pass, then:

sqlite3 inventory.sqlite "INSERT INTO webhooks(program, pass_entry) VALUES('newprog', 'messaging/discord/webhook/newprog');"

Use 1-to-N for all programs by pointing every row at the same pass entry. Changing a row takes effect on the next notify run.

• Pending hosts are grouped by program, formatted as a bold heading plus one Markdown bullet per host, and split into batches under 1900 characters (Discord allows 2000).
• A batch is marked sent only after HTTP 200 or 204. Anything else stays pending for the next run, so a failed send never loses a hostname.
• If the process dies after Discord accepted a message but before SQLite was updated, that batch is sent again. A rare duplicate is accepted in exchange for never losing a discovery.
• state/notify.lock prevents two runs from overlapping; the second run exits quietly.

Pick any schedule. A run only sends what is still pending, so a frequent entry costs one SQLite query when there is nothing to send:

# Every 6 hours; adjust to taste.
0 */6 * * * /Users/quan.m.le/Tools/ctmon/ctmon.fish notify >> /Users/quan.m.le/Library/Logs/ctmon/cron.log 2>&1

For local testing without touching the real channels, set CT_WEBHOOK_URL. It replaces every pass webhook, so all programs go to that one URL:

CT_WEBHOOK_URL=http://127.0.0.1:9000 ./ctmon.fish notify

Logs

Logs live in the macOS per-user log directory, not in this folder and not in /tmp, so they survive reboots and temp cleanup.

Path	Contents
~/Library/Logs/ctmon/ct.log	Hook inserts, start and exit, notify results
~/Library/Logs/ctmon/cron.log	stdout and stderr of the scheduled notify run
~/Library/Logs/ctmon/start.log	stdout and stderr of a detached nohup start
<name>.log.1	One previous copy of any of the above, after rotation

Each line is <UTC ISO 8601> <message>:

2026-11-06T10:03:26Z discovered api.capital.com (capitalcom)
2026-11-06T10:03:26Z hook discovered_cert inserted=3
2026-11-06T10:03:11Z notify sent 2 (capitalcom)
2026-11-06T10:03:12Z notify failed 2 (capitalcom)

tail -f ~/Library/Logs/ctmon/ct.log
grep 'notify failed' ~/Library/Logs/ctmon/ct.log

Every message is also mirrored to unified logging, so Console.app and log show see it:

log show --last 1h --info --debug --predicate 'process == "logger" AND eventMessage CONTAINS "ctmon:"'

Each mirrored message starts with ctmon: because logger's -t tag is not stored in eventMessage. The process == "logger" clause keeps the query from matching the log show invocation itself. --info --debug is required: logger writes at default level and log show hides that level otherwise. The text file is the durable record; unified logging rotates on its own schedule.

Logging never breaks a run. An unwritable log directory or a missing logger is ignored, and hook logging never changes the hook exit status. Override the directory with CT_LOG_DIR.

Memory

certspotter runs one worker set per monitored log, and the default log list has 89 of them. Measured peak RSS over 90 seconds with the same watchlist:

Run	Peak RSS
-start_at_end, 89 logs (normal operation)	48 MB
Backfilling history, 89 logs	479 MB
Backfilling history, 89 logs, GOMEMLIMIT=256MiB	362 MB
Backfilling history, 44 logs	223 MB
Backfilling history, 6 logs	55 MB

The big cost is replaying log history, not the number of scopes you watch. start passes -start_at_end, which skips history for any log without saved state, so normal operation stays near 48 MB.

start also sets GOMEMLIMIT=384MiB as a backstop for a log that does have to backfill. Go ignores a value below the live heap and never kills the process, so a tight limit slows it down rather than breaking it. Override with CT_MEMLIMIT, for example CT_MEMLIMIT=256MiB.

CPU and the log list

certspotter polls every log in its list forever, and a log whose validity window has not opened or has closed still costs a worker set and a STH request every five minutes.

Measured on copies of the same saved state with GOMEMLIMIT=384MiB, run side by side at the same time, 297 seconds each:

Log list	Logs	Download jobs	CPU
Default monitor.zip	89	151	66 s
Live plus 60 days ahead	45	87	65 s

Halving the log list did not reduce CPU. The two runs match within noise, and a separate 300-second pass put the smaller list higher (69.9 s against 46.1 s). Cost scales with how many certificate entries arrive, not with how many logs are configured.

Steady state on the live monitor is 10.2% of one core: 30.69 s of CPU over 300 seconds, in 30-second windows from 8% to 12%. The high numbers in the table above come from replaying saved state, not from ordinary operation.

Three findings from the measurements:

• Certificate authorities pre-populate future logs. Logs whose window opens in December or January publish entries now. In a 150-second sample, future-window logs added 695,508 entries against 170,092 for currently-open logs, so dropping them removes the busiest logs.
• Retry churn is not the cost either. The four logs responsible for 70% of a day's retries (2,171 retried lines, 1,177 timeouts, 899 HTTP 429s) cost 1.5% of one core when monitored alone.
• Steady-state CPU swings widely. The same 89-log list measured between 8% and 12% across consecutive 30-second windows, and separate short passes ranged from 5.6-7.4% to 15.4%. Certificate traffic is bursty, so treat any single short sample as a rough guide.

Where the CPU goes, from a 60-second sample of the live process, counting only samples that were on a CPU:

Work	Share of on-CPU time
gzip inflate and CRC	24%
Go reflect and type lookups	16%
ASN.1 certificate parsing	15%
Garbage collection and scheduling	14%
Byte and buffer handling	10%
JSON decode	7%
String handling	2%
Crypto	2%
Scope matching	0.8%
Precert TBS rebuild	0.4%

Matching a certificate against the watchlist is already nearly free, and no entry is ever written to disk. certspotter keeps per log only a position file, unverified tree heads, retry diagnostics, and healthchecks, about 8 MB in total for 89 logs. Entries are parsed, filtered, and dropped in memory.

That leaves decompression as the floor. CT logs serve gzipped batches and offer no server-side filtering, so an entry has to be inflated before its names can be read. A purpose-built streamer could skip the ASN.1 step, at most 15% of the total, but not the inflate, and skipping certspotter's Merkle tree and tree-head checks would trade away the proof that each log is consistent.

set-logs is available for reducing the list, but treat it as a coverage choice rather than a performance fix:

./ctmon.fish set-logs

It fetches monitor.json, keeps every log whose validity window overlaps the next CT_LOOKAHEAD days (default 60), and writes state/logs.json. start passes that file with -logs when it exists, and falls back to the default list when it does not.

A 60-day lookahead keeps every log that is currently publishing, including those that open next quarter, and drops only windows that closed or open after June 2027. Of the 44 logs it drops, 13 moved at all in a 75-second sample, and their movement was 2-12 entries; 38 of the 45 kept logs were active in the same sample. It also needs a refresh: run set-logs again after a window rolls over, or a log that opens later never gets picked up.

Rotation

No logrotate and no daemon. ct rotates its own files with wc -c, cp, and a truncate, so the same code works on macOS and Linux.

• Threshold is 5 MiB, set by _CT_LOG_MAX in the script.
• Checked at the start of each start, notify, and hook run, so a long-running monitor still rotates as certificates arrive.
• One backup per file: ct.log becomes ct.log.1, replacing the previous backup. Worst case is about 10 MiB per log.
• Files below the threshold are left alone.
• Copy-then-truncate keeps the inode. That matters because cron.log and start.log are written through a parent shell's >> redirect, and a rename would leave the running process writing to a hidden file.

Files

Path	Purpose
ctmon.fish	The macOS script: commands, hook, logging, rotation
ctmon.sh	The POSIX sh port for Linux and other hosts. Same commands and schema as ctmon.fish
inventory.sqlite	Scopes, discovered hostnames, delivery state
state/	certspotter log positions and saved certificates, plus notify lock
~/Library/Logs/ctmon/	Log files listed above

certspotter is installed as a mise global tool:

mise use -g go:software.sslmate.com/src/certspotter/cmd/certspotter@latest

Operational notes

• The first ./ctmon.fish start starts at the end of each log. -start_at_end skips log history, so it finds new certificates only, not ones logged before you started. This is the difference between 48 MB and 479 MB of memory. Progress lives in state/, so restarts resume where they left off rather than replay. Passing the flag on every start is safe: certspotter applies it only to a log with no saved state. To backfill one log's history on purpose, delete its directory under state/logs/ and start again without the flag.
• Nothing restarts it after a reboot. There is no LaunchAgent and no systemd unit. Start it again after logging in. Certificates logged while it was down are not seen, because starts skip history.
• Cron does not catch up after sleep. Missed runs simply leave rows pending until the next successful ./ctmon.fish notify.
• Cron uses a minimal PATH. Its shell has /usr/bin:/bin:/usr/sbin:/sbin, which contains neither Homebrew nor mise. The shebang is /opt/homebrew/bin/fish, and ct prepends /opt/homebrew/bin and ~/.local/share/mise/shims itself, so fish, pass, python3, curl, and certspotter all resolve without touching crontab.
• Deduplication is by hostname. Hostnames are lowercased and trailing dots are stripped before insert, so API.Capital.COM. and api.capital.com are the same row. The table key does the rest; INSERT OR IGNORE keeps the original first_seen and never resets delivery state.
• One certificate can yield many rows. A certificate naming five hosts produces five rows with the same first_seen. That is not a duplicate.
• Wildcards are not stored. *.example.com in a certificate is a pattern, not a host, so it is rejected. Only concrete names enter the inventory.
• Malformed names are rejected, not repaired. A host must be at least two labels, and each label must start and end with a letter or digit. So bad..example.com, -lead.example.com, trail-.example.com, and a_b.example.com are all dropped. One consequence worth knowing: an underscore is valid in DNS for records like _dmarc, but it is not a valid hostname, so those names never reach the inventory.

Troubleshooting

Symptom	Cause	Fix
certspotter not found	mise shims not on PATH	mise use -g go:software.sslmate.com/src/certspotter/cmd/certspotter@latest, then run ./ctmon.fish start again
notify failed: no webhook for (prog)	No webhooks row for that program	Add the row with sqlite3, as shown under Notifications
notify failed: webhook ... unavailable (prog)	The pass entry is absent or locked	pass show <pass_entry> for the entry in webhooks
routing is N-to-M	webhooks mixes programs in one channel	Use one pass entry for all programs, or one per program
notify already running	A previous run holds the lock	Wait; if the process died, remove state/notify.lock
notify nothing pending	Every host has been delivered	Nothing to do
No new hosts for a long time	Monitor stopped, or scopes changed without a restart	pgrep -fl certspotter, then ./ctmon.fish start
Notifications arrive twice for one batch	Process died between send and DB update	Expected tradeoff, not a defect

Start with the log when something looks wrong:

grep -E 'certspotter exited|notify failed|routing' ~/Library/Logs/ctmon/ct.log

Notes

Built as a personal reconnaissance tool for a bug bounty program. Deliberately boring: native tools, minimal state, no services to babysit.

Last reviewed: 2026-11-06.