## Summary
Fixes Prowlarr torrent downloads that fail with `Could not determine
torrent hash
from URL` when the result has no magnet link and no infohash (e.g.
MyAnonaMouse),
where fetching the .torrent from Prowlarr's proxy download link is the
only path.
Two problems compounded here:
1. **Every add attempt fetched the download link twice.**
`find_existing()`
prefetched the .torrent to compute a dedup hash, discarded the result,
and
`add_download()` fetched the same URL again seconds later. Private
tracker
links behind Prowlarr's proxy can be slow, rate-limited, or effectively
single-use, so the second hit could fail even when the link itself was
valid —
which is why the reporter's manual fetch of the same URL succeeded.
2. **The real failure reason was invisible.** When the fetch failed
(e.g.
Prowlarr returning HTTP 500 because the tracker rejected the request —
see the
2026-07-07 MAM report on #476, which turned out to be a MAM IP-settings
problem), the reason was logged at DEBUG only and the user saw the
misleading
generic hash error.
## What changed
- `extract_torrent_info()` now reuses a recent successful fetch of the
same URL
(short-TTL in-memory cache, successes only), so one add attempt hits the
tracker download link exactly once across `find_existing()` +
`add_download()`.
All four torrent clients (qBittorrent, Deluge, Transmission, rTorrent)
share
this path and benefit. Failures are never cached, so retries refetch.
- `TorrentInfo` gains a `fetch_error` field. qBittorrent and rTorrent
append it
to the hash error (`... (torrent file fetch failed: 500 Server Error
...)`),
Deluge to its "Failed to fetch torrent file" error. The enriched message
still
contains the exact substring the #1109 expired-link refresh hook matches
on,
so the refresh-and-retry path keeps working.
- Torrent fetch failures are logged at WARNING instead of DEBUG, so
non-debug
logs show the cause.
## Validation
- `uv run pytest tests/prowlarr tests/download -q` — 498 passed
- `uv run pytest tests/newznab tests/audiobookbay -q` — 147 passed
- `uv run ruff check` / `ruff format --check` on all changed files
- New tests: fetch-cache reuse, failure-not-cached + reason capture,
expected-hash fallback on failed/hashless fetches, TTL expiry,
magnet-redirect
reuse, and the enriched qBittorrent error message.
Fixes#1111
## Summary
Fixes Prowlarr downloads that fail after a queued torrent result’s
tracker download link expires.
Prowlarr torrent results can expose a `downloadUrl` that is only a
short-lived proxy to the upstream tracker. Some trackers, including MAM,
include expiring credentials in that URL. Shelfmark was persisting that
URL as retry data and later treating it as durable. If the in-memory
Prowlarr cache was gone, or if qBittorrent tried to add a stale URL,
Shelfmark could fail with a misleading torrent-hash error instead of
refreshing the release.
## What changed
- Stop persisting Prowlarr `downloadUrl` values as durable retry data.
- Persist only source context needed to refresh the release later.
- On a Prowlarr cache miss, re-query Prowlarr using the queued task
context.
- Accept refreshed results only when the stable identity matches the
original release:
- `guid == task.task_id`
- or `infoUrl == task.task_id`
- Cache the fresh raw Prowlarr result and build the download request
from its current `downloadUrl` / `magnetUrl`.
- If qBittorrent add fails with `Could not determine torrent hash from
URL`, remove the stale cached Prowlarr result, refresh once, and retry
with the fresh URL.
- Preserve existing magnet handling: torrent results continue to use
`magnetUrl` first, so the refresh path only targets Prowlarr proxy
`downloadUrl` values that can expire.
- Improve the user-facing failure when refresh cannot find the same
release:
`The indexer download link expired and the release could not be
refreshed. Search again for a fresh result.`
## Why this approach
The important constraint is avoiding accidental downloads of a different
edition or format after the original tracker link expires. Re-running a
search by title can return many plausible results, so the refresh path
deliberately requires an exact stable identity match before using any
new URL.
This treats Prowlarr `downloadUrl` as a short-lived hint, while still
allowing retries to recover when Prowlarr can find the same release
again. The one-shot retry after qBittorrent add failure handles the case
where Shelfmark still has a cached Prowlarr result, but that cached
result contains an expired proxy URL.
The refresh hook is source-specific and defaults to no-op for other
external download handlers, so Newznab and other sources keep their
existing retry behavior.
## Bug
Fixes#1012
Context:
https://github.com/calibrain/shelfmark/issues/1012#issuecomment-4917148398
## Validation
- `uv run pytest tests/prowlarr -q`
- `uv run pytest tests/download/test_orchestrator_user_output_mode.py
-q`
- `uv run pytest tests/newznab/test_handler.py
tests/audiobookbay/test_handler.py -q`
- `uv run ruff check shelfmark/core/models.py
shelfmark/download/orchestrator.py
shelfmark/download/clients/base_handler.py
shelfmark/release_sources/prowlarr/handler.py
tests/prowlarr/test_handler.py
tests/prowlarr/test_integration_handler.py
tests/prowlarr/test_failure_scenarios.py tests/prowlarr/test_source.py
tests/download/test_orchestrator_user_output_mode.py`
- after resolving conflicts with latest `main`: `uv run pytest
tests/prowlarr -q`
- after resolving conflicts with latest `main`: `uv run ruff check
shelfmark/release_sources/prowlarr/handler.py`
- after resolving conflicts with latest `main`: `uv run pytest
tests/prowlarr/test_handler.py tests/prowlarr/test_source.py -q`
Co-authored-by: Aidan Abbott <aidanabbott@Aidans-MacBook-Pro.local>
# fix(prowlarr): prevent silent loss of indexer seed limits
Related to #795, though not a fix for that specific (closed) report —
see note below.
## Problem
With "Use Prowlarr seed preferences" enabled, ~5–10% of torrent grabs
are added to the download client without their configured share limits
and seed indefinitely (∞ ETA in qBittorrent).
Seed limits are resolved once at search time.
`get_indexer_seed_settings()` builds on `get_indexers()`, which swallows
all API errors and returns `[]`. A transient failure of the
`/api/v1/indexer` call therefore produces an empty settings dict that is
indistinguishable from "no limits configured", while the search itself
(separate HTTP calls) still succeeds. Every result from that search is
cached without `configuredSeedTimeMinutes`; grabbing one sends the
torrent to the client with no limits.
Compounding factors: `cache_release()` is last-write-wins by GUID, so
one degraded search can strip enrichment from a previously good cache
entry; and the retry fields persisted at queue time snapshot the same
missing values, so retries reproduce the failure.
## Changes
- **`api.py`** — `get_indexers()` / `get_enabled_indexers_detailed()`
gain a keyword-only `raise_on_error` (default `False`, existing behavior
unchanged). `get_indexer_seed_settings()` uses it, so fetch failures now
propagate and an empty dict strictly means "nothing configured".
- **`source.py`** — searches fetch settings via
`_fetch_indexer_seed_settings()`, which maintains a module-level
last-known-good copy (merged on each success) and falls back to it with
a warning when the fetch fails. After one successful fetch, results can
no longer be cached un-enriched.
- **`handler.py`** — grab-time safety net in `_resolve_download()`: if
seed preferences are enabled, the release is a torrent, and no
configured limits are present in the cached result, the handler
re-resolves the limits from Prowlarr for that indexer
(`restrict_to=[indexerId]`) before adding to the client. If limits still
can't be resolved, a warning is logged so the condition is visible
instead of silent.
- **Tests** — regression coverage: last-known-good fallback (success
updates cache, failure falls back, failure with no history returns
empty, fallback copy is mutation-safe) and grab-time fallback (used when
enrichment is missing, tolerates Prowlarr being down, skipped when
enrichment is present). Existing test stubs for
`get_enabled_indexers_detailed` updated to accept the new kwarg.
## Testing
- `uv run pytest tests/prowlarr/test_handler.py
tests/prowlarr/test_source.py
tests/prowlarr/test_integration_handler.py` — 93 passed, 2 skipped
(Python 3.14.4)
- Full `tests/prowlarr` run has 11 pre-existing failures on this
environment (Windows path-separator assertions in the
qBittorrent/NZBGet/SABnzbd/Transmission client tests, e.g.
`/downloads/x` vs `\downloads\x`); confirmed these also fail on
unpatched `main` and are unrelated to this change
- `uv run ruff check` / `ruff format --check` — clean on touched files
- `uv run basedpyright` — 0 errors on touched files
No behavior change when `PROWLARR_USE_SEED_PREFERENCES` is disabled; the
fallback path only activates when the preference is on and enrichment is
missing for a torrent.
---
**Note on #795:** this PR references #795 for background context on the
seed-limits feature, but it does not fix that report — #795 was about
seed limits not being converted/applied at all (a units mismatch), and
was already fixed by #946 / #959. This PR fixes a separate,
still-present bug: `get_indexers()` silently swallowing transient API
errors, which intermittently drops seed limits even when the feature is
otherwise working correctly.
- Add configurable completed-path wait for external clients : - Add a
configurable Advanced setting, DOWNLOAD_CLIENT_COMPLETED_PATH_TIMEOUT,
for how long Shelfmark waits after a torrent or usenet client reports
completion before treating the completed path as missing. - Keep the
default at the existing 60-second grace period, with a maximum of 3600
seconds
- Add e2e testing
- Add e2e testing
Should fix#861
## Backport bug fixes from `NemesisHubris/litfinder`
Forwards a curated set of bug fixes from
[NemesisHubris/litfinder](https://github.com/NemesisHubris/litfinder) —
a community fork of this project — that address open issues here. All
commits preserve original authorship via `git cherry-pick`; this PR is a
backport rather than original work. Each fix has been reviewed locally,
lint/format-cleaned to match this repo's existing ruff config, and
verified with the test suite. Rebrand strings, license switches, and
features have been deliberately excluded.
### Upstream issues addressed
- **#999** — Mirror URLs with query params no longer break search
requests (strip query string/fragment in `normalize_http_url`)
- **#956** — Apprise notifications now respect the configured proxy
(proxy env vars injected before dispatch)
- **#1025** — rTorrent: separate `RTORRENT_AUDIOBOOK_LABEL` setting,
falls back to book label if unset
- **#1010** — Stop button in Activity no longer makes the panel
disappear (snapshot refresh on cancel)
- **#1021** — Anna's Archive slow-download countdown now caps retries
instead of looping forever
- **#1040** — Empty destination directory cleaned up when write probe
fails
- **PR #1031** — Language detection from Anna's Archive distant path
when listing metadata is missing
### Additional fixes (no open issue but clear bugs)
- **fix: Python 2 `except` syntax across 27 files** — `except X, Y:` is
a SyntaxError in Python 3 and prevents affected modules from importing
at runtime. Mechanical sweep to `except (X, Y):`.
- **fix(abb): info hash validation with magnet fallback** — adds
SHA-1/SHA-256 hex validation on extracted info hashes; falls back to
scanning the full page for a magnet link (e.g. posted in comments) when
the table value is malformed. Also extends the exact-phrase fallback to
manual queries and defaults the ABB listing language to `en` when
missing, preventing valid results from being hidden by the language
filter. Includes a small test-fixture fix (`test(abb): use valid hex
info hashes in scraper test fixtures`) since the existing fixtures used
non-hex placeholders that the new validation correctly rejects.
- **fix: Anna's Archive title parser** — handles nested edition spans
and filters `lgli` catalog descriptor entries (e.g. "Book/Online Audio")
that were polluting search results.
### Deliberately not included
- LitFinder rebranding (UI strings, Apprise app ID, logo). The `fix:
three upstream bugs` commit (#999/#956/#1025) was cherry-picked with
Apprise app-id, description, and logo-URL strings reverted from
"LitFinder" back to "Shelfmark"; noted in the commit body.
- Features from the LitFinder fork (multi-variant title search,
multi-book flat-folder grouping, fuzzy text matching, "Leave in Place"
output handler, admin display name, custom-source plugin system). These
are larger behavior changes that each warrant their own focused review —
happy to send any of them separately if of interest.
- LitFinder-specific test environment and CI infrastructure.
### Verification
- Backend: **1879 passed**, 96 skipped (1 preexisting failure on
`seleniumbase`-dependent test in local venv; runs fine in the standard
Docker image with the `browser` extra)
- Lint, format, dead-code: all clean against this repo's existing
ruff/vulture config
- One follow-up cleanup commit (`style: ruff lint and format fixes for
ported commits`) brings the cherry-picked code into compliance with this
repo's ruff settings — no behavior changes there
### Etiquette / credit
Per-commit authorship preserved by cherry-pick. The only edits to the
original commits are:
- `fix: three upstream bugs` — Apprise rebrand strings reverted to
"Shelfmark" (noted in commit body, original author retained as
`Co-Authored-By` via cherry-pick)
- One follow-up `style:` commit for ruff config alignment
Big thanks to [@NemesisHubris](https://github.com/NemesisHubris) for the
original work in LitFinder; this PR exists to make sure these fixes
reach Shelfmark's wider user base. Happy to revise scope, split into
smaller PRs, or split off the Py2 cleanup separately if that's
preferable.
---------
Co-authored-by: NemesisHubris <155838970+NemesisHubris@users.noreply.github.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
Adds support for `content_type=combined` in URL search parameters,
letting users force combined-mode searches via a bookmarkable link
rather than relying on the last-used preference from localStorage.
The override is applied only in Universal mode and only when combined
mode is actually available (universal enabled, `show_combined_selector`
on, neither content type blocked by policy). Otherwise, it's silently
ignored, consistent with how `content_type=ebook`/`audiobook` already
behave outside Universal.
Existing `content_type=ebook`/`audiobook` URLs now also force combined
mode off, so the URL is authoritative regardless of prior preference.
Also adds a per-user `FORCE_COMBINED_SEARCH` setting that locks combined
mode on whenever it's available.
URL `content_type=ebook`/`audiobook` overrides are also superseded by
force-combined for the same reason: the search bar wouldn't let users
switch back, so honoring the URL param would leave them in a state they
couldn't escape from.
Clears up seedtime logic to use user-specified seedtime only, ignore the
indexer-defaults.
Adds a toggle to enable the seedtime feature, disabled by default.
Fixes#955
- Fixed internal bypasser startup with newer Chromium/SeleniumBase by
isolating the browser helper from Gunicorn/gevent, serialising helper
failures cleanly, and cleaning up orphan processes after a failure
- Stopped using /app as runtime home state, now moved to /home/shelfmark
or /tmp/shelfmark/home as fallback.
- Added tests
- Updated mirror selection
- Removed built-in mirror options, users must provide their own
configurations
- Set Universal search to default, added ability to disable direct
source
- Updated documentation
- Updated makefile
I've added a plugin using the same architecture as the prowlarr plugin
to enable Newznab as a source.
I've tested locally with nzbhydra2 and it all seems to work as intended.
I've added some unit tests for this feature, and found that a couple of
other unit tests weren't behaving so fixed those up while I was at it. I
also ran all of the linters in the makefile against it and fixed those
up, too, so hopefully this should be as clean and as compatible as it
can be.
CaliBrain
Aidan AbbottandAidan Abbott
adman234
Alex SchittkoandCaliBrain
Alex
blades