A request whose book turns out to be in the library is closed with the note `[auto] Already in the library.` instead of sitting in Needs Review. A request an admin handled by hand while a pass was running is skipped. A series label in front of a Calibre title no longer hides an owned book, and a Calibre title marked as another edition (full cast, dramatized, GraphicAudio) is not the book.
When no source matches, up to three simpler forms of the title are tried on each source under the same strict match. Optionally (off by default), an unmatched audiobook request is looked up on Audible US and UK through Audiobookshelf's metadata search and the request says what the store answered. It never declines anything.
A Soulseek peer whose recent transfers all failed, and an AudiobookBay post whose comments say it has no seeders, are tried after the other matches and never excluded. A failed or stalled release is kept out for AUTO_DOWNLOAD_RETRY_FAILED_RELEASE_DAYS and then tried again, and a torrent that died for one request is not offered to another. A request stuck on queued or downloading is settled from the download history. Optionally (off by default) a failed download is retried with another release within minutes.
A shelf book can be requested as an ebook, an audiobook or both (one request per format), and each request is checked against its own library, source priority list and format guard. Ranking within a source prefers the most downloaded copy, then size, and one m4b is preferred to a set of chapter files unless it is much smaller than the other copies. Soulseek peers are ranked by free slot, queue length and speed.
Automatic downloads refuse multi-book packs, sequels and other works named in the title, study guides, abridged or dramatised versions, two-in-one titles, releases in a language the requester didn't ask for, and an archive for an audiobook request.
Matching folds accents on both sides, drops an alternate title for the search and the match, and reads small numbers the same in any form. AudiobookBay releases with no author in the listing title are judged on the authors credited on the detail page.
Closes#1037.
Two new AudiobookBay settings, Seed Time (minutes) and Seed Ratio. Both
default to 0, which leaves the torrent client's own limit as it is
today, so nothing changes until someone sets them.
They go to the client with the magnet the same way the Prowlarr seed
preferences do (`DownloadRequest.seeding_time_limit` and `ratio_limit`),
so they apply with the clients that already take per-torrent limits
there: qBittorrent, Transmission and Deluge. A retry that reuses the
saved magnet carries them too. A value that isn't a positive number
(text, negative, NaN, inf) is ignored and the client's limit stays.
Tests cover reading both settings (unset, zero, each on its own, text
values, bad values) and that both the fresh and the cached-magnet
request carry them. Ruff, basedpyright and the backend suite pass. I
haven't tried it against a live client, since the client side is the
code Prowlarr already uses.
I added the two entries to `docs/environment-variables.md` by hand.
Running `scripts/generate_env_docs.py` here also changes unrelated parts
of that file (the Soulseek table of contents link and the
`SHELFMARK_API_KEY_READONLY` entry), so I left those alone.
Retrying an AudiobookBay download re-scraped the detail page for the
magnet link before checking the torrent client, so a torrent that had
already finished in the client could not be imported while AudiobookBay
was down.
The handler now keeps the magnet it resolved in the task's retry
context, which is persisted with the download history, and a retry hands
that magnet straight to the client's existing-download check. The
context is handler-owned, so a client-supplied download URL cannot seed
it.
Fixes#1388
## Multi-book packs: inspect a release before download and file each
book separately
Closes#576
### Problem
One queued release is always treated as one book. When a torrent is
actually a whole series
(`Series/Book 1 - Title/…`, or a flat folder of `Series 1.0 - Title.m4b`
files), post-processing
walks the whole tree, flattens every file into one list and renames them
`Title - 01…10` under the
searched book's `{Author}/{Title}`. Audiobookshelf then sees a single
10-file "book" and the user
has to re-file everything by hand.
### What this does
Most releases expose their file list *before* anything is downloaded, so
the split is decided up
front and approved by the user, then the download is fire-and-forget:
1. **Inspect** – clicking a release's download button now calls `POST
/api/releases/inspect`
first. A new optional `DownloadHandler.list_files(release_data)` hook
returns the release's
files without downloading:
- **AudiobookBay** reads the torrent file table off the detail page it
already fetches (the
page is now cached for 120 s, so inspect + download cost ABB one
request).
- **Prowlarr** parses `info.files` from the `.torrent` it already
fetches (the existing 120 s
torrent-fetch cache is reused). Magnet-only and usenet releases report
"can't inspect".
- Other sources default to `None`.
2. **Review** – if the plan contains more than one book, the Find
Releases modal swaps the list
for a review panel: one row per book with editable title / series
position / year, expandable
file lists, non-book sidecars (`.txt`, covers) shown as ignored, a
"Treat as a single book"
switch, and **Download N books**. Single-book releases queue
immediately, exactly as before.
3. **File** – the approved plan travels with the task
(`DownloadTask.book_plan`, retry-safe) and
post-processing files each book through the existing transfer code, one
book at a time
(`dataclasses.replace(task, title=…, series_position=…, year=…)`), so
organize/rename
templates, part numbering (now scoped per book), hardlinks, torrent
copy-preserve and usenet
handling are unchanged. Status reads `Complete (N books, M files)`.
4. **Fallback** – when a release can't be inspected the user gets a
toast, and a small
"Multi-book pack" toggle in the modal header forces a heuristic split
(subfolder = book, or
one book per file when the file names carry series positions).
Planning lives in `shelfmark/download/postprocess/packs.py` and is
shared by the inspect endpoint
and post-processing, so what the user approved is what gets filed. The
name parser strips
`Book 3 -`, `03 -`, `1.0 -`, `3.`, `[03]`, `#3`, a leading series name,
labels like
"An Expanse Novella -", repeated titles (`Gods of Risk 2.5 - Gods of
Risk`) and a trailing
`(Year)`; author and series name come from the book that was searched,
and the searched book's
own series position is never applied to its siblings.
### Files
- `shelfmark/download/postprocess/packs.py` (new) –
`PackFile/PackBook/PackPlan`, `plan_pack`,
`parse_pack_book_name`, `group_files_into_books`, `match_plan_to_files`
- `shelfmark/core/release_inspect_routes.py` (new) – `POST
/api/releases/inspect`
- `shelfmark/release_sources/__init__.py` – `DownloadHandler.list_files`
hook
- `shelfmark/release_sources/audiobookbay/{scraper,handler}.py` –
detail-page cache,
`extract_file_list`, `list_files`
- `shelfmark/release_sources/prowlarr/handler.py`,
`download/clients/torrent_utils.py` –
`extract_file_list_from_torrent`, `list_files`
- `shelfmark/core/models.py`, `download/orchestrator.py` – `multi_book`
/ `book_plan` fields,
queue + retry serialization
- `shelfmark/download/postprocess/transfer.py`, `pipeline.py`,
`outputs/folder.py` – per-book
transfer branch and status message
- `src/frontend`: `components/PackReviewPanel.tsx` (new),
`ReleaseModal.tsx`, `App.tsx`,
`services/api.ts`, `types/index.ts`, `utils/releasePayload.ts` (payload
builder moved out of
`App.tsx`), `utils/packReview.ts`
- `docs/dev/release-sources-plugin-guide.md` – documents the
`list_files` hook
### Out of scope (follow-ups)
- Listing files from an NZB (Shelfmark already fetches the bytes; `<file
subject>` names are noisy)
- Inspecting magnet links via qBittorrent's files API after a paused add
- BookLore / email outputs (they ignore `book_plan`; noted in code)
- The combined ebook + audiobook flow
### Testing
**Automated** (`make checks`, `make python-test`, `make frontend-test`
all green; the only
failures on my machine are the pre-existing
`tests/config/test_entrypoint_permissions.py` cases,
which need bash ≥ 4 and fail identically on `main` under macOS bash
3.2):
- `tests/download/test_packs.py` – name parsing (markers, series name,
novella labels, repeated
titles, bare numeric titles like `1984`), nested / flat / mixed /
deeper-nested packs, single
wrapping folder not treated as a pack, plan-to-disk matching with
basename fallback
- `tests/core/test_processing_packs.py` – full `post_process_download`
runs on a real temp
filesystem: approved plan files each book under its own
`{Author}/{Title}`, heuristic split
of a nested pack, searched book's series position does not leak,
multi-file book inside a pack
keeps `- 01/- 02` per book, hardlinked torrent pack leaves the seeding
tree intact, no pack
fields ⇒ behaviour unchanged, single group degrades to the searched
title, status message
- `tests/core/test_release_inspect_routes.py` – plan response,
not-inspectable, handler errors
never 500, unknown source / missing `source_id` ⇒ 400, login required
- `tests/audiobookbay/test_file_list.py` – file-table scraping from real
ABB markup (multi-file
and single-file pages), handler host validation, one page fetch shared
by magnet + file list
- `tests/prowlarr/test_torrent_file_list.py` – multi-file / single-file
`.torrent` parsing,
handler behaviour for torrent URL vs magnet vs usenet vs cache miss
- `tests/download/test_orchestrator_pack_fields.py` – queue-time parsing
and retry round-trip
- Frontend: `releasePayload.test.ts`, `packReview.test.ts` (vitest)
**Manual, on a real deployment** (arm64 image built from this branch,
run as a side container
next to production with the same qBittorrent / Audiobookshelf setup,
`FILE_ORGANIZATION_AUDIOBOOK=organize`,
hardlinks on):
- AudiobookBay "The Expanse Complete 2.0" (7.87 GB, 36 files): clicking
download opened the review
panel in ~1 s showing **18 books · 18 files · 18 files ignored** (the
`.txt` sidecars), with
series positions 0.1–9.5 and years parsed from the file names; novella
labels stripped
("The Churn", "The Butcher of Anderson Station"). Editing a title in the
panel works.
Confirming queued one task; the magnet resolved from the cached page in
~30 ms; after the
download the task reported `Complete (18 books, 18 files)`, 18 hardlinks
landed as
`audiobooks/James S. A. Corey/<Title>/<Title>.m4b`, the torrent kept
seeding, and
Audiobookshelf scanned each folder as its own book (title, author,
embedded chapters).
- A second pack ("Expanse [01 - 9.5]", `Title N - Title` naming) was
inspected to verify the
repeated-title rule and the Back button, without downloading.
- Single-book releases still queue immediately with no extra UI.
WordPress texturizes punctuation on output only, so a post stored as
"The
Stranger's Wife" renders as "The Stranger’s Wife". ABB's search matches
the
stored value and ANDs its terms, so one typographic character in the
query
empties the entire result set rather than merely ranking worse. Book
metadata
and mobile keyboards both hand us those characters.
Map curly quotes, dashes and ellipses to ASCII before a query goes out,
and on
both sides of the relevance comparison, since scraped titles carry the
rendered
forms. Release titles are still stored and displayed exactly as ABB
renders
them; only matching normalizes.
Also percent-encode the search query properly. The hand-rolled encoder
only
escaped double quotes and spaces, so a bare "&" started a new query
parameter
and silently truncated the search: "detective dan riley books 1 & 2
weatherley"
reached ABB as "detective dan riley books 1" and returned six
confident-looking
results without the requested book among them. "%" and "+" were mangled
too.
## Backport bug fixes from `NemesisHubris/litfinder`
Forwards a curated set of bug fixes from
[NemesisHubris/litfinder](https://github.com/NemesisHubris/litfinder) —
a community fork of this project — that address open issues here. All
commits preserve original authorship via `git cherry-pick`; this PR is a
backport rather than original work. Each fix has been reviewed locally,
lint/format-cleaned to match this repo's existing ruff config, and
verified with the test suite. Rebrand strings, license switches, and
features have been deliberately excluded.
### Upstream issues addressed
- **#999** — Mirror URLs with query params no longer break search
requests (strip query string/fragment in `normalize_http_url`)
- **#956** — Apprise notifications now respect the configured proxy
(proxy env vars injected before dispatch)
- **#1025** — rTorrent: separate `RTORRENT_AUDIOBOOK_LABEL` setting,
falls back to book label if unset
- **#1010** — Stop button in Activity no longer makes the panel
disappear (snapshot refresh on cancel)
- **#1021** — Anna's Archive slow-download countdown now caps retries
instead of looping forever
- **#1040** — Empty destination directory cleaned up when write probe
fails
- **PR #1031** — Language detection from Anna's Archive distant path
when listing metadata is missing
### Additional fixes (no open issue but clear bugs)
- **fix: Python 2 `except` syntax across 27 files** — `except X, Y:` is
a SyntaxError in Python 3 and prevents affected modules from importing
at runtime. Mechanical sweep to `except (X, Y):`.
- **fix(abb): info hash validation with magnet fallback** — adds
SHA-1/SHA-256 hex validation on extracted info hashes; falls back to
scanning the full page for a magnet link (e.g. posted in comments) when
the table value is malformed. Also extends the exact-phrase fallback to
manual queries and defaults the ABB listing language to `en` when
missing, preventing valid results from being hidden by the language
filter. Includes a small test-fixture fix (`test(abb): use valid hex
info hashes in scraper test fixtures`) since the existing fixtures used
non-hex placeholders that the new validation correctly rejects.
- **fix: Anna's Archive title parser** — handles nested edition spans
and filters `lgli` catalog descriptor entries (e.g. "Book/Online Audio")
that were polluting search results.
### Deliberately not included
- LitFinder rebranding (UI strings, Apprise app ID, logo). The `fix:
three upstream bugs` commit (#999/#956/#1025) was cherry-picked with
Apprise app-id, description, and logo-URL strings reverted from
"LitFinder" back to "Shelfmark"; noted in the commit body.
- Features from the LitFinder fork (multi-variant title search,
multi-book flat-folder grouping, fuzzy text matching, "Leave in Place"
output handler, admin display name, custom-source plugin system). These
are larger behavior changes that each warrant their own focused review —
happy to send any of them separately if of interest.
- LitFinder-specific test environment and CI infrastructure.
### Verification
- Backend: **1879 passed**, 96 skipped (1 preexisting failure on
`seleniumbase`-dependent test in local venv; runs fine in the standard
Docker image with the `browser` extra)
- Lint, format, dead-code: all clean against this repo's existing
ruff/vulture config
- One follow-up cleanup commit (`style: ruff lint and format fixes for
ported commits`) brings the cherry-picked code into compliance with this
repo's ruff settings — no behavior changes there
### Etiquette / credit
Per-commit authorship preserved by cherry-pick. The only edits to the
original commits are:
- `fix: three upstream bugs` — Apprise rebrand strings reverted to
"Shelfmark" (noted in commit body, original author retained as
`Co-Authored-By` via cherry-pick)
- One follow-up `style:` commit for ruff config alignment
Big thanks to [@NemesisHubris](https://github.com/NemesisHubris) for the
original work in LitFinder; this PR exists to make sure these fixes
reach Shelfmark's wider user base. Happy to revise scope, split into
smaller PRs, or split off the Py2 cleanup separately if that's
preferable.
---------
Co-authored-by: NemesisHubris <155838970+NemesisHubris@users.noreply.github.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
- Migrate download client handling from /prowlarr to /download. Moves
all torrent/usenet handling to app-level and gives ABB this
functionality.
- ABB Scraper now uses shared HTTP infrastructure instead of raw
requests, adding retry and proxy support
- Added author, age and bitrate info to ABB search results
- Added "best match" sorting option for releases
- Added size and bitrate sorting options for ABB
- Removed bundled default ABB hostname, must be configured by the user
- Added URL normalisation for ABB hostname
- Rearranged settings UI, moved download clients to its own section.
- More tests
## Summary
Adds AudiobookBay as a web-scraping release source for audiobook
torrents. Once enabled, a new tab shows up in the Find Releases modal.
## What's New
- **AudiobookBay source** – Search AudiobookBay for audiobook torrents
from the Shelfmark UI
- **Torrent downloads** – Extract magnet links from detail pages and add
them to the configured torrent client
- **Audiobook-only** – Source is limited to audiobooks
- **Download clients:** Currently uses the torrent client configured
under **Prowlarr > Download Clients**.
- Audiobook-specific categories (e.g. `QBITTORRENT_CATEGORY_AUDIOBOOK`)
are applied when set.
- **Settings → AudiobookBay**:
- Enable toggle
- Hostname
- Max pages to search (default 5)
- Rate limit delay in seconds (default 1)
## How It Works
1. User searches for an audiobook; AudiobookBay is queried if enabled.
2. Results show title, language, format, and size
3. User selects a release; the handler fetches the detail page and
extracts the magnet link.
4. Magnet link is sent to the configured torrent client.
## Testing
- Unit tests for source, handler, scraper, and utils
- Mocked HTTP requests and torrent client calls
- Coverage for search, relevance filtering, language mapping, size
parsing, and download flow
### Screenshots
<img width="600" alt="image"
src="https://github.com/user-attachments/assets/2e10a259-5c35-4065-980d-b59a1c961c9f"
/>