Logs get pasted into issues, and the formatter from #1488 only masks
`?key=value` query parameters. I went through what else commonly ends up
in a log line and found these unmasked:
- credentials in a URL's userinfo, like a proxy's
`socks5://user:pass@host:1080` or `http://admin:secret@qbittorrent:8080`
- a passkey as a path segment, like `/rss/<key>` or
`/passkey/<key>/get.torrent`, which private trackers use
- the token in a Telegram bot, Discord webhook or Slack webhook URL,
which is what a notification URL looks like when it gets logged
All three are masked now, in `redact_secrets`, so they apply to every
handler. The path rule only fires after a segment that names a secret
(`passkey`, `apikey`, `rsskey`, `authkey`, `token`, `secret`, `rss`,
`torznab`) followed by 16 or more token characters. I left out generic
words like `download` on purpose, because `/api/download/<md5>/cancel`
appears in our own request log and the ids there are useful when
debugging. That is the tradeoff: a passkey in a `/download/<passkey>/`
path still gets through.
Separately, four modules created their logger with plain
`logging.getLogger`, so nothing they logged went through the formatter
at all: `api/websocket.py`, `download/clients/__init__.py`, and the
Jackett, Newznab and Prowlarr package inits. They use `setup_logger` now
like everything else.
Tests: ten cases, seven for the new shapes and three that must stay
readable (an md5 route id, a short `/rss/latest`, an email address),
plus two that check the stray modules now carry the redacting formatter.
Nine of them fail without the change. Full suite passes.
Not covered here: `/api/debug` still zips the existing log files as they
are on disk, so anything logged before this change rides along in a
bundle. That is a separate change in `genDebug.sh`.
This patch shares (for each log file) the `RotatingFileHandler` for
logging across all modules, reducing the number of open file descriptors
from ~78 to 1.
I had originally assumed this issue was a resource leak, but it seems to
just be a large fixed number of file descriptors. So this change mostly
just (1) shrinks the number of open file descriptors to a reasonable
level and (2) prevents two modules in the same process competing to
write to a log file.
splitsec2
CaliBrain
oliver