Two small changes that stop Shelfmark from opening a new HTTPS
connection for requests that go to the same host over and over.
Anna's Archive download counts. Every search result gets its count from
`/dyn/md5/inline_info/<md5>`, and each of those was a bare
`requests.get`, so a search with 50 results opened 50 TCP and TLS
connections to the mirror. `search_books` also runs once per title
variant, and the variants of one book mostly find the same files, so the
same md5 was looked up again for every variant that found it, and again
when the release list was reopened. The lookups now share one
`requests.Session`, and a successful count is kept for five minutes in a
bounded `CacheService` (the same class the metadata cache uses, as its
own instance, so it does not depend on the metadata cache toggle). A
lookup that fails or comes back without a count is not cached, so it is
retried on the next search just like today. The URL, headers and 5
second timeout are unchanged.
Debrid clients. Real-Debrid, AllDebrid, Debrid-Link and TorBox also used
the bare `requests` helpers. The download handler polls `get_status`
every two seconds for each active job, so every poll was a fresh TLS
handshake with the debrid API. Each of those modules now keeps one
`requests.Session` and sends its API calls through it, the way the
qBittorrent, Deluge, Prowlarr and Newznab clients already do. Headers,
timeouts, `verify` and error handling are still passed per call exactly
as before. The existing tests that patched
`<module>.requests.get/post/request` now patch
`<module>._session.<verb>` instead; nothing else in them changed.
How I checked it: each change has a test that fails on main and passes
here. They count the requests that reach `requests.Session.request` and
the sessions built while doing it, with no network. On main, three polls
of each debrid client build three sessions, and two overlapping batches
of AA results look `bbb` up twice. There are also tests for the count
cache expiring (with a faked clock) and for a failed lookup not being
cached. Full suite: 4432 passed, 48 skipped. ruff and basedpyright are
clean on the changed files.
I also looked at caching `/api/releases` on the server and left it out.
The release modal already caches each book's results per source and
content type in the browser for five minutes (longer for IRC and slskd
through `cache_ttl_seconds`), and that cache survives closing and
reopening the modal, so reopening it does not run a new search today. A
server cache would only help after a page reload, and it would quietly
undo the modal's "apply filters" and manual search refetches, which drop
the browser cache on purpose, unless the frontend also learned to send a
bypass. That did not seem worth it for this PR.
qBittorrent over HTTPS (#1417)
qbittorrent-api ignores the scheme in `host` and probes HTTP and HTTPS
itself. When the HTTPS probe fails, for instance on a certificate
Shelfmark doesn't trust, it falls back to plain HTTP against the TLS
port, and a reverse proxy answers "400 The plain HTTP request was sent
to HTTPS port". The download client and the Test Connection button now
pass FORCE_SCHEME_FROM_HOST for https:// URLs, so the configured scheme
is used as-is and a certificate problem is reported as one. http:// and
bare host:port URLs keep the probe: normalization adds http:// to a bare
host, and the probe follows a proxy's redirect to HTTPS.
Release sources leaked to non-admins (#1418)
A download queued from an approved request carries the release an admin
picked, and the requester could read it from their own activity feed:
the request's release_data (source_id, indexer, info URL, torrent
attributes), the download's full server path, and the download id
itself, which for Prowlarr is "<indexer id>:<guid>" and for a private
tracker is a URL into it. That id keys every status payload, the
/api/localdownload query and the cover proxy URL, so hiding
release_data alone would not have closed the leak.
For non-admin viewers:
- request rows keep only the release fields the activity cards display
- download_path is reduced to the file name, which the browser download
reveals anyway
- request-linked downloads are addressed by an opaque id (a keyed HMAC
of the task id) in the snapshot, history, /api/status, queue order,
active downloads, dismissed keys, cover URLs and the websocket status
and progress events. Routes that take a download id translate it
back, searching only the caller's own downloads.
Downloads a user queued directly keep their real id: the user picked
that release from search results that already showed it, and the
release list matches its buttons to the queue by that id. Admin views
are unchanged. The activity sidebar now links a fulfilled request to
its download by request_id instead of release_data.source_id.
Closes#1417Closes#1418
Add **TorBox** as a torrent download client for Prowlarr releases.
Users can select `TorBox` in the download client settings, configure it
with the new `TORBOX_API_KEY` environment variable, and verify their
credentials with the connection test button.
The integration supports both magnet links and `.torrent` files. It
tracks the torrent lifecycle through TorBox, downloads supported book
and audiobook files from the TorBox CDN, preserves safe nested file
paths, and cleans up remote and local download state.
Important: Shared HTTP download logs omit full download URLs and
URL-bearing exception text to avoid exposing credentials, following best
practices. This applies to all clients that use the shared
`download_url()` path; URLs remain available to the HTTP operations
themselves.
---
There is already related work in progress in #1173, which includes both
torrent and direct-download support for TorBox.
This PR is not intended to replace or compete with that contribution. It
offers the tested torrent client functionality as a smaller, focused
change that can make TorBox available to the community sooner. The
direct-download integration proposed in #1173 remains valuable and could
be reviewed or introduced separately.
Automated tests cover configuration, connection validation, magnet and
torrent-file submission, API errors, status and progress handling, file
retrieval, path traversal protection, cancellation, cleanup, and
sensitive URL redaction.
I also validated the complete flow locally with several magnet links and
`.torrent` downloads. TorBox processed the torrents and Shelfmark
downloaded the resulting files as expected.
AI was used to help with the implementation, with human validation. This
PR and long description? Took me some good minutes at night after work,
but gives me joy to open this PR to share with the community this
improvement.