Closes#1027
## Summary
- add a **Remove & Delete Files** (`remove_and_delete`) option to the
Torrent Completion Action setting
- after a successful import, remove the torrent and ask the client to
delete its downloaded data (`delete_files=True`), matching what the
usenet "move" flow already does
- clarify the setting description so it says **Remove** keeps the
downloaded files (raised in the issue comments)
- regenerate the `PROWLARR_TORRENT_ACTION` entry in
`docs/environment-variables.md`
## Behavior
The deletion runs from `post_process_cleanup`, the same place as the
existing Remove and Change Category actions, so it only happens after
output transfer and post-processing have succeeded. At that point every
file has already been copied or hardlinked into the library, and
transfer size checks have passed. A failed import never removes or
deletes anything. The torrent client deletes its own data, so Shelfmark
does not delete paths itself and remote path mappings are not involved.
Keep, Remove, and Change Category behave as before, and Keep is still
the default.
If a request matched a completed torrent that was already in the client,
this option deletes that torrent's data too. That is the same scope the
existing Remove action already applies to.
## Validation
- `make python-checks` (ruff check, ruff format, basedpyright on backend
and tests, vulture): passed
- `make python-test`: 3363 passed, 5 skipped, 9 failed. All 9 are in
`tests/config/test_entrypoint_permissions.py` and happen because macOS
`/bin/bash` 3.2 does not support `${1,,}` in `entrypoint.sh`. They do
not touch this change.
- new tests in `tests/prowlarr/test_handler.py`:
- Remove passes `delete_files=False` and Remove & Delete Files passes
`delete_files=True`
- a failed import with Remove & Delete Files does not call the client
- the delete case fails without the handler change
- manual smoke run: `PROWLARR_TORRENT_ACTION=remove_and_delete` set
through real config loading, with a stub client that deletes its folder.
The torrent folder was deleted and the hardlinked library file stayed
intact.
- pre-commit hooks (prek) passed
---------
Co-authored-by: Evan Kazakin <evan@Evans-MacBook-Pro.local>
## Why
Blackhole users need Shelfmark to hand a torrent file to their existing
downloader instead of importing the downloaded book itself.
## Change
- add Blackhole as a torrent client with a configurable watched
directory
- prefer a fetched `.torrent` file for Blackhole while preserving magnet
preference for other clients
- complete the queue task after the handoff without invoking book
post-processing
## Verification
- `uv run pytest -q tests/prowlarr/test_blackhole_client.py
tests/prowlarr/test_handler.py tests/newznab/test_handler.py
tests/download/test_orchestrator_lifecycle.py`
- `uv run basedpyright shelfmark/download/clients/blackhole.py
shelfmark/download/clients/__init__.py
shelfmark/download/clients/base_handler.py
shelfmark/download/clients/settings.py
shelfmark/download/orchestrator.py shelfmark/release_sources/__init__.py
shelfmark/release_sources/prowlarr/utils.py
shelfmark/release_sources/prowlarr/handler.py
shelfmark/release_sources/newznab/handler.py
tests/prowlarr/test_blackhole_client.py tests/prowlarr/test_handler.py
tests/newznab/test_handler.py
tests/download/test_orchestrator_lifecycle.py`
Fixes#1229
add_download() picks self._audiobook_label from a content_type kwarg,
but the only call site never passed one, so is_audiobook was always
False and every download got RTORRENT_LABEL. category does not fill
the gap: _get_category_for_task() returns None for rTorrent, which has
no category concept, leaving content_type as its only audiobook signal.
Pass task.content_type through from base_handler, and match it with the
shared is_audiobook() helper instead of == "audiobook".
normalize_content_type()
treats "book (audiobook)" as an audiobook, so the exact-string check
would have mislabeled that value even once it arrived.
The existing rTorrent tests passed content_type straight to the client,
which is why nothing caught the missing wiring; the new handler test
covers the call site itself.
Post-processing was never affected: destination.py reads
task.content_type directly, so files already landed in
DESTINATION_AUDIOBOOK correctly.
Fixes#1235
## Summary
- add a **Change Category** torrent completion action and conditionally
show its post-import category/label setting
- update kept torrents only after a successful library import
- support qBittorrent categories, Transmission labels, Deluge's Label
plugin, and rTorrent's `custom1` label
- preserve existing Keep/Remove behavior and document the new
environment setting
## Behavior
Category changes happen from `post_process_cleanup`, after output
transfer and post-processing complete. This keeps the existing hardlink
flow unchanged. An empty post-import category is a no-op, and client API
failures are logged without turning a successful library import into a
failure.
## Validation
- `pytest -n 0 tests/prowlarr/test_qbittorrent_client.py
tests/prowlarr/test_transmission_client.py
tests/prowlarr/test_deluge_client.py
tests/prowlarr/test_rtorrent_client.py tests/prowlarr/test_handler.py
tests/config/test_generate_env_docs.py` — 161 passed
- `ruff check` on all changed Python files — passed
- `basedpyright` on the changed client implementation files — passed
- full `basedpyright shelfmark/download/clients` currently reports two
pre-existing errors in the new debrid connection-test code at
`settings.py:546` and `settings.py:563`, outside this PR's diff
- multi-architecture Docker images built successfully for `linux/amd64`
and `linux/arm64`
Results were deduplicated on `guid` alone. When one tracker is
configured in Prowlarr as several indexer entries differing only by a
server-side search filter, all of them return the same guid for the same
torrent, so every entry but the first was silently discarded. Freeleech
and other filter-specific releases became invisible, replaced by the
unfiltered entry's copy, and which copy survived depended on query
ordering rather than user intent.
Include the indexer id in the dedup key so the entries stay distinct.
Release.source_id is qualified the same way. It keys the release cache
and becomes the download task id, so rows sharing a guid would otherwise
collide and a grab would route through whichever entry cached last,
defeating the point of showing them separately. The handler's task
matcher still accepts a bare guid or infoUrl so tasks queued before this
change still resolve.
Ordering now follows the priority already configured in Prowlarr (1-50,
lower preferred) rather than a new setting, since users curate that
ranking there and filtered entries are typically ranked ahead of their
unfiltered counterparts. The enabled-indexer list is fetched once and
reused for the enrichment check, so this costs no extra round trip.
Releases carry extra.indexer_priority, and the sort dropdown gains an
"Indexer priority" entry, ascending, alongside the existing alphabetical
"Indexer" sort; SortOption grew a default_direction for that, defaulting
to desc so "Peers" is unchanged.
PROWLARR_COLLAPSE_DUPLICATES (default off) optionally collapses a
release back to one row, resolved by the same Prowlarr priority. Left
off, every entry that carried a release keeps its own row, which is what
makes filtered results visible again.
Identity handling is defensive about partial payloads: a result that
cannot be identified is never dropped or merged, and collapse only
merges on a strong identifier (guid/downloadUrl/magnetUrl/infoUrl)
because merging on title alone would discard genuinely different
releases that share a name.
Fixes#1137
Co-authored-by: delize <4028612+delize@users.noreply.github.com>
## Summary
Fixes Prowlarr torrent downloads that fail with `Could not determine
torrent hash
from URL` when the result has no magnet link and no infohash (e.g.
MyAnonaMouse),
where fetching the .torrent from Prowlarr's proxy download link is the
only path.
Two problems compounded here:
1. **Every add attempt fetched the download link twice.**
`find_existing()`
prefetched the .torrent to compute a dedup hash, discarded the result,
and
`add_download()` fetched the same URL again seconds later. Private
tracker
links behind Prowlarr's proxy can be slow, rate-limited, or effectively
single-use, so the second hit could fail even when the link itself was
valid —
which is why the reporter's manual fetch of the same URL succeeded.
2. **The real failure reason was invisible.** When the fetch failed
(e.g.
Prowlarr returning HTTP 500 because the tracker rejected the request —
see the
2026-07-07 MAM report on #476, which turned out to be a MAM IP-settings
problem), the reason was logged at DEBUG only and the user saw the
misleading
generic hash error.
## What changed
- `extract_torrent_info()` now reuses a recent successful fetch of the
same URL
(short-TTL in-memory cache, successes only), so one add attempt hits the
tracker download link exactly once across `find_existing()` +
`add_download()`.
All four torrent clients (qBittorrent, Deluge, Transmission, rTorrent)
share
this path and benefit. Failures are never cached, so retries refetch.
- `TorrentInfo` gains a `fetch_error` field. qBittorrent and rTorrent
append it
to the hash error (`... (torrent file fetch failed: 500 Server Error
...)`),
Deluge to its "Failed to fetch torrent file" error. The enriched message
still
contains the exact substring the #1109 expired-link refresh hook matches
on,
so the refresh-and-retry path keeps working.
- Torrent fetch failures are logged at WARNING instead of DEBUG, so
non-debug
logs show the cause.
## Validation
- `uv run pytest tests/prowlarr tests/download -q` — 498 passed
- `uv run pytest tests/newznab tests/audiobookbay -q` — 147 passed
- `uv run ruff check` / `ruff format --check` on all changed files
- New tests: fetch-cache reuse, failure-not-cached + reason capture,
expected-hash fallback on failed/hashless fetches, TTL expiry,
magnet-redirect
reuse, and the enriched qBittorrent error message.
Fixes#1111
## Summary
Fixes Prowlarr downloads that fail after a queued torrent result’s
tracker download link expires.
Prowlarr torrent results can expose a `downloadUrl` that is only a
short-lived proxy to the upstream tracker. Some trackers, including MAM,
include expiring credentials in that URL. Shelfmark was persisting that
URL as retry data and later treating it as durable. If the in-memory
Prowlarr cache was gone, or if qBittorrent tried to add a stale URL,
Shelfmark could fail with a misleading torrent-hash error instead of
refreshing the release.
## What changed
- Stop persisting Prowlarr `downloadUrl` values as durable retry data.
- Persist only source context needed to refresh the release later.
- On a Prowlarr cache miss, re-query Prowlarr using the queued task
context.
- Accept refreshed results only when the stable identity matches the
original release:
- `guid == task.task_id`
- or `infoUrl == task.task_id`
- Cache the fresh raw Prowlarr result and build the download request
from its current `downloadUrl` / `magnetUrl`.
- If qBittorrent add fails with `Could not determine torrent hash from
URL`, remove the stale cached Prowlarr result, refresh once, and retry
with the fresh URL.
- Preserve existing magnet handling: torrent results continue to use
`magnetUrl` first, so the refresh path only targets Prowlarr proxy
`downloadUrl` values that can expire.
- Improve the user-facing failure when refresh cannot find the same
release:
`The indexer download link expired and the release could not be
refreshed. Search again for a fresh result.`
## Why this approach
The important constraint is avoiding accidental downloads of a different
edition or format after the original tracker link expires. Re-running a
search by title can return many plausible results, so the refresh path
deliberately requires an exact stable identity match before using any
new URL.
This treats Prowlarr `downloadUrl` as a short-lived hint, while still
allowing retries to recover when Prowlarr can find the same release
again. The one-shot retry after qBittorrent add failure handles the case
where Shelfmark still has a cached Prowlarr result, but that cached
result contains an expired proxy URL.
The refresh hook is source-specific and defaults to no-op for other
external download handlers, so Newznab and other sources keep their
existing retry behavior.
## Bug
Fixes#1012
Context:
https://github.com/calibrain/shelfmark/issues/1012#issuecomment-4917148398
## Validation
- `uv run pytest tests/prowlarr -q`
- `uv run pytest tests/download/test_orchestrator_user_output_mode.py
-q`
- `uv run pytest tests/newznab/test_handler.py
tests/audiobookbay/test_handler.py -q`
- `uv run ruff check shelfmark/core/models.py
shelfmark/download/orchestrator.py
shelfmark/download/clients/base_handler.py
shelfmark/release_sources/prowlarr/handler.py
tests/prowlarr/test_handler.py
tests/prowlarr/test_integration_handler.py
tests/prowlarr/test_failure_scenarios.py tests/prowlarr/test_source.py
tests/download/test_orchestrator_user_output_mode.py`
- after resolving conflicts with latest `main`: `uv run pytest
tests/prowlarr -q`
- after resolving conflicts with latest `main`: `uv run ruff check
shelfmark/release_sources/prowlarr/handler.py`
- after resolving conflicts with latest `main`: `uv run pytest
tests/prowlarr/test_handler.py tests/prowlarr/test_source.py -q`
Co-authored-by: Aidan Abbott <aidanabbott@Aidans-MacBook-Pro.local>
# fix(prowlarr): prevent silent loss of indexer seed limits
Related to #795, though not a fix for that specific (closed) report —
see note below.
## Problem
With "Use Prowlarr seed preferences" enabled, ~5–10% of torrent grabs
are added to the download client without their configured share limits
and seed indefinitely (∞ ETA in qBittorrent).
Seed limits are resolved once at search time.
`get_indexer_seed_settings()` builds on `get_indexers()`, which swallows
all API errors and returns `[]`. A transient failure of the
`/api/v1/indexer` call therefore produces an empty settings dict that is
indistinguishable from "no limits configured", while the search itself
(separate HTTP calls) still succeeds. Every result from that search is
cached without `configuredSeedTimeMinutes`; grabbing one sends the
torrent to the client with no limits.
Compounding factors: `cache_release()` is last-write-wins by GUID, so
one degraded search can strip enrichment from a previously good cache
entry; and the retry fields persisted at queue time snapshot the same
missing values, so retries reproduce the failure.
## Changes
- **`api.py`** — `get_indexers()` / `get_enabled_indexers_detailed()`
gain a keyword-only `raise_on_error` (default `False`, existing behavior
unchanged). `get_indexer_seed_settings()` uses it, so fetch failures now
propagate and an empty dict strictly means "nothing configured".
- **`source.py`** — searches fetch settings via
`_fetch_indexer_seed_settings()`, which maintains a module-level
last-known-good copy (merged on each success) and falls back to it with
a warning when the fetch fails. After one successful fetch, results can
no longer be cached un-enriched.
- **`handler.py`** — grab-time safety net in `_resolve_download()`: if
seed preferences are enabled, the release is a torrent, and no
configured limits are present in the cached result, the handler
re-resolves the limits from Prowlarr for that indexer
(`restrict_to=[indexerId]`) before adding to the client. If limits still
can't be resolved, a warning is logged so the condition is visible
instead of silent.
- **Tests** — regression coverage: last-known-good fallback (success
updates cache, failure falls back, failure with no history returns
empty, fallback copy is mutation-safe) and grab-time fallback (used when
enrichment is missing, tolerates Prowlarr being down, skipped when
enrichment is present). Existing test stubs for
`get_enabled_indexers_detailed` updated to accept the new kwarg.
## Testing
- `uv run pytest tests/prowlarr/test_handler.py
tests/prowlarr/test_source.py
tests/prowlarr/test_integration_handler.py` — 93 passed, 2 skipped
(Python 3.14.4)
- Full `tests/prowlarr` run has 11 pre-existing failures on this
environment (Windows path-separator assertions in the
qBittorrent/NZBGet/SABnzbd/Transmission client tests, e.g.
`/downloads/x` vs `\downloads\x`); confirmed these also fail on
unpatched `main` and are unrelated to this change
- `uv run ruff check` / `ruff format --check` — clean on touched files
- `uv run basedpyright` — 0 errors on touched files
No behavior change when `PROWLARR_USE_SEED_PREFERENCES` is disabled; the
fallback path only activates when the preference is on and enrichment is
missing for a torrent.
---
**Note on #795:** this PR references #795 for background context on the
seed-limits feature, but it does not fix that report — #795 was about
seed limits not being converted/applied at all (a units mismatch), and
was already fixed by #946 / #959. This PR fixes a separate,
still-present bug: `get_indexers()` silently swallowing transient API
errors, which intermittently drops seed limits even when the feature is
otherwise working correctly.
Clears up seedtime logic to use user-specified seedtime only, ignore the
indexer-defaults.
Adds a toggle to enable the seedtime feature, disabled by default.
Fixes#955
- Clean up a few uses of config options that may miss the env variable
if this is set
- Add enhanced retry availability utilising the DB to persist download
errors / retries across restarts, request failures, and pass Prowlarr
detail through the download task to maintain retry data.
- Strip back entrypoint permissions for less intensive chown operations.
Fixes#796
- Added a path for rootless permissions in the entrypoint script
- Routed prowlarr searches through torznab for seedtime info
- Added additional request flow for download permissions
- Migrate download client handling from /prowlarr to /download. Moves
all torrent/usenet handling to app-level and gives ABB this
functionality.
- ABB Scraper now uses shared HTTP infrastructure instead of raw
requests, adding retry and proxy support
- Added author, age and bitrate info to ABB search results
- Added "best match" sorting option for releases
- Added size and bitrate sorting options for ABB
- Removed bundled default ABB hostname, must be configured by the user
- Added URL normalisation for ABB hostname
- Rearranged settings UI, moved download clients to its own section.
- More tests
- Added heuristic-based author and title query creation, stripping out
unnecessary elements that could limit searches
- Improved language support when using Hardcover. Searches will now be
conducted on a per-language basis using localized book titles.
- Added manual search override option in the release modal.
- Added new book output option **upload to Booklore**, available in
download settings
- Got annoyed at my messy processing code while implementing Booklore so
refactored the whole thing
- Full black box file processing testing with randomised configuration
- Deluge: Connect via WebUI auth for simplified setup
- Added env vars documentation, auto generated via script, and unlocked
most settings to be used as env vars
- Updated Selenium to 4.45.6. Includes various crash and memory leak
fixes, plus new bypasser methods
- Bypasser now uses CDP captcha solving as priority - Faster, more
efficient, no PyAutoGUI needed. Fallback to existing methods.
- Better detection and cleanup of old Selenium instances to save memory.
- Added Hardcover graphQL API header detection
- Added AA download counts in details modal
- More robust switching of internal/external bypasser, fixed settings UI
toggle behavior.
## Headline features
### Prowlarr plugin - search trackers and download usenet/torrent books
- Search any usenet/torrent tracker via Prowlarr, returns books within
Universal search
- Configure download clients in the app settings (Qbittorrent, Deluge,
Transmission, NZBget, SABnzbd)
- Unified download and file handling within the app, same as AA.
### IRC plugin
- Search IRCHighway #ebooks channel for books and download right in the
app.
- No setup needed
- Credit to OpenBooks for the broad idea and inspiration for best
practices for ebook-specific search and download.
### Google Books Metadata Provider
- Create a Google Cloud API key and use Google Books as a metadata
provider
- Not the best source (Hardcover is still recommended), but another
option and further redundancy for universal search
### Book series support
- New "Series" search field in Hardcover provider
- "Series order" sort option - lists books in reading order
- "View Series" button in book details modal to search the full series
- Series info display (e.g., "3 of 12 in The Wheel of Time")
## Others:
- Better format filtering, helpful errors when formats rejected (e.g.,
"Found 3 ebooks but format not supported (.pdf). Enable in Settings >
Formats."
- Directory processing - Handles multi-file torrent/usenet downloads
properly
- Expand search toggle - Skip ISBN search to find more editions
- Filtered authors - Uses primary authors only (excludes
translators/narrators) for better search results
- Language multi-select - Filter releases by multiple languages
Docker / Build / Testing
- pip cache mounts - Faster Docker builds via BuildKit cache
- npm cache mounts - Faster frontend builds
- APT cleanup - Smaller final image size
- Added make restart command for quick restarts without rebuild
- New pytest-based test framework with proper configuration
(pyproject.toml)
- Unit tests for all download clients (qBittorrent, Transmission,
Deluge, NZBGet, SABnzbd)
- Bencode parsing tests
- Cache tests
- Integration tests for Prowlarr handler
- E2E test framework
Evan KazakinandEvan Kazakin
Atirna
CaliBrain
Tilian B
Andrew Doeringanddelize
Aidan AbbottandAidan Abbott
adman234
Alex
Alex