Supersedes #1353, per the discussion in #1352: one `API_KEY` environment
variable; when set, a request carrying it is authenticated as the first
admin, and cookie sessions keep working exactly as before (cookie **or**
key). Nothing else changes. No table, no UI, no settings-tab switch, no
per-user keys.
## What
- `API_KEY` (env). Unset → the feature is off and none of the new code
runs.
- `Authorization: Bearer <key>` or `X-Api-Key: <key>` on any existing
`/api/*` route authenticates that request as the first admin in
`users.db` (`ORDER BY id`), or as a bare admin identity
(`user_id="api"`, `is_admin=True`, no local user row) if the install has
no admin yet. Per request only; nothing is persisted; the admin's role
is read live, so deleting or demoting that user takes effect on the next
request.
- Both headers are checked and either may match. That is what makes the
key usable behind a reverse proxy that injects its own `Authorization`
header (oauth2-proxy, Authelia, forwardAuth): send the key in
`X-Api-Key`.
- A credential that is **not** the key is ignored and the request
continues on the normal session path, so proxy-forwarded tokens are
unaffected. Without a valid session such a request gets the usual `401
{"error": "Unauthorized"}`, identical to a request with no credential,
so there is nothing to probe.
## How
- `shelfmark/config/env.py`: `API_KEY = os.getenv("API_KEY",
"").strip()`.
- `shelfmark/core/api_key.py`: `extract_api_key_candidates()` (Bearer
token if the scheme is Bearer, then `X-Api-Key`) and `matches_api_key()`
using `hmac.compare_digest` on bytes.
- `shelfmark/core/user_db.py`: `UserDB.get_first_admin()`.
- `shelfmark/main.py`: `api_key_auth_middleware` (`before_request`,
registered before `proxy_auth_middleware`, which early-returns for keyed
requests). Only `/api/` paths; `/api/health` and `/api/auth/*` exempt;
no-op when `API_KEY` is unset or the auth mode is `none`. On a match it
mirrors the proxy-auth pattern: `session.clear()` then populate
`user_id` / `is_admin` / `db_user_id` for this request, `permanent =
False`, `modified = False`, `g.api_key_auth = True`. An `after_request`
hook guarantees no `Set-Cookie` is written for a keyed request even if a
handler dirties the session.
- `docs/api-access.md` (new), the `API_KEY` entry in
`docs/environment-variables.md`, and a README link.
## Security
- Constant-time compare; the key is never logged or echoed.
- Keyed requests never mint or refresh a session cookie and ignore any
cookie sent with them (a non-admin cookie plus the key yields admin for
that request; the browser's own session is left untouched and usable).
- The mismatch path touches neither the session nor `g`, so a stray
bearer on a browser request can neither log the user out nor change how
their cookie is refreshed.
- Store errors during the admin lookup fail closed (`500 {"error":
"Authentication error"}`), never to anonymous.
- Verified against Flask's `save_session` / `should_set_cookie`
ordering, and under auth modes `none`, `builtin`, `proxy`.
## Tests
`tests/core/test_api_key_env.py` (36): extraction and matching;
first-admin lookup; middleware behaviour on a guarded route and an admin
route, with and without a user_db, `X-Api-Key`, both-headers
combinations, no `Set-Cookie` when a handler dirties the session,
incoming non-admin cookie ignored, browser cookie still usable after a
keyed request, security headers, store error → 500, mismatch → guard's
401 / cookie path / permanent cookie untouched, unset → off, exempt
paths and path probes, `none` and `proxy` modes, deleted and demoted
first admin, a keyed write passing the guard. Existing auth suites
unchanged. All CI gates green on the fork:
https://github.com/gavinmcfall/shelfmark/pull/2 (CI-only draft).
Also exercised against a running instance: 47 scripted checks including
150 concurrent requests, proxy-mode switching through the key, an
unset-key restart, and a log scan for the key.
## Naming
`API_KEY` as discussed. If you'd rather namespace it
(`SHELFMARK_API_KEY`) to avoid clashing with other tools' env vars in
shared compose files, it is a one-line change; say the word.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
`POST /api/requests/batch` checks that `requests` is a non-empty list
and then hands each element to the shared preparation helper, which
calls `.get()` on it. A bare string, number or null in the list raises
`AttributeError` and the caller gets a 500, while `POST /api/requests`
answers 400 with a message for the same mistake.
This validates the element type beside the existing list check. One bad
item rejects the whole batch rather than being reported per item, which
matches the endpoint's current contract: every other failure path
already aborts the batch with a single error body, as
`test_batch_create_requests_is_atomic` asserts. Responses for valid
payloads are unchanged.
## Verification
- `tests/core/test_request_routes_api.py`: a case per bad shape (int,
str, null, list), plus a mixed valid and invalid batch that asserts
nothing was created. All fail on current main with a 500 and pass here.
- Full suite (3168), ruff, ruff format, basedpyright, vulture green.
`/api/localdownload` resolves the file through the live queue and
returns it before checking who owns the task; the owner check only runs
on the download-history fallback, once the task has aged out of the
queue. Task ids are source ids, so two users who searched the same book
can end up with the same id.
This applies the same rule on the queue path, reusing the
`_task_owned_by_actor` helper the cancel/retry/priority routes already
use, so both paths answer a non-owner with the same 404. Admin behaviour
is unchanged.
The 404 matches what this endpoint's history path already returns for a
non-owner rather than the 403 `download_not_owned` the
cancel/retry/priority routes use, happy to switch it if you prefer
consistency with the siblings instead.
## Verification
- `tests/core/test_activity_routes_api.py`: the owner still receives
their queued file; a different user receives 404. The new case fails on
current main and passes here; the existing history-fallback test is
unchanged.
- Full suite (3148), ruff, ruff format, basedpyright, vulture green.
`cancel_download` reads the task status under the queue lock, releases
it, and only then writes CANCELLED through `update_status`. A download
that finishes in that window has its COMPLETE overwritten. The queue and
the UI show the task as cancelled while the file is already on disk, and
the terminal hook fires for both statuses.
The check and the write now happen in a single lock hold. Because the
lock is non-reentrant and the terminal hook has to run after it is
released (the stall canceller depends on that), the lock-held part of
`update_status` moved into a small private helper that both paths share;
`update_status` is a thin wrapper over it. A cancel arriving once the
task is already terminal still returns `False`.
## Verification
-
`tests/core/test_queue.py::test_cancel_does_not_overwrite_a_download_that_finished_first`:
a worker thread completes the download while the cancel is in flight,
with the handover driven by events rather than sleeps. The task stays
complete. Fails on main, passes here.
- Full suite (3147), plus `tests/download/` and
`tests/core/test_download_api_guardrails.py`, ruff, ruff format,
basedpyright, vulture green.
The OIDC `return_to` sanitizer rejects values starting with `//` and
then relies on `urlsplit` to catch anything carrying a netloc. A value
such as `/\host` has no netloc, so it is stored in the session and used
as the post-login redirect target and browsers resolve the backslash as
a path separator, which lands the user outside the app after a
successful login.
`_normalize_return_to` now also rejects values whose path contains a
backslash. That matches the frontend sanitizer in `authRedirect.ts`,
which parses with `URL` and already discards those forms, so the two
ends agree again. The check covers the path only, so query and fragment
backslashes still round-trip, and it also catches the script-root case
where `/app/\host` strips to `/\host`.
## Verification
- New cases in `tests/core/test_oidc_routes.py` cover the rejected
forms, including under a script root, and confirm `/`, `/settings` and
`/search?q=x#frag` are unaffected. They fail on current main and pass
here.
- Full suite (3155), ruff, ruff format, basedpyright, vulture green.
`_resolve_policy_mode_for_current_user` reads `session.get("is_admin",
True)`, so a session carrying `user_id` but no `is_admin` key skips the
request policy entirely, while every other admin check in the codebase
defaults the key to `False`.
This uses the same default here. Every authenticated login path
(builtin, CWA, proxy, OIDC) writes `is_admin` into the session, and
`AUTH_METHOD=none` is already short-circuited one line earlier, so
sessions from those flows behave exactly as before.
## Verification
- `tests/core/test_request_routes_api.py::TestDownloadPolicyGuards`: a
session without `is_admin` now gets `policy_requires_request` and
nothing is queued. Fails on current main, passes here.
- Full suite (3147), ruff, ruff format, basedpyright, vulture green.
`PUT /api/users/me` and `PUT /api/admin/users/<id>` write the new
password hash, and then the profile fields, before the rest of the
payload is checked. When the request is rejected further down as an
invalid role, an admin-only setting, an invalid settings value, the
route answers 400 with those writes already committed, so the caller
sees an error while the password has in fact changed.
Both routes now validate the whole payload before touching the database,
and the password hash is folded into the same `update_user` call as the
other fields so the field write is a single transaction. Error messages,
status codes and the order they are reported in are unchanged.
## Verification
- New tests in `tests/core/test_self_user_routes.py` and
`tests/core/test_admin_users_api.py` assert that a rejected update
leaves the password, profile fields and role as they were, plus a
positive case that a valid payload still applies all three. They fail on
current main and pass here.
- Full suite (3150), ruff, ruff format, basedpyright, vulture green.
With `AUTH_METHOD=proxy` and no admin group configured, every user the
proxy authenticates for the first time is provisioned as an admin
(`is_admin = True` unless the user already exists in `users.db`). The
intent to never lock an instance out makes sense, but the effect is that
anyone the SSO gate lets through becomes an administrator. On an
instance shared with family or a small community that is a footgun; I
hit it when the first invited reader landed as an admin.
This keeps the guarantee and removes the footgun: the first account is
still provisioned as an admin while the instance has no admin at all,
and later first-time users follow a new `PROXY_AUTH_DEFAULT_ROLE`
setting (Security tab / env), default `user`. Known users keep their
stored role; the `PROXY_AUTH_ADMIN_GROUP_NAME` path is unchanged and
still takes precedence. I couldn't find a way with Cloudflare access to
pass this along.
Changes: `UserDB.has_admin()`, `_proxy_default_is_admin()` in the proxy
middleware, the new `SelectField` beside the other proxy settings, the
regenerated `docs/environment-variables.md` entry and a row in
`docs/reverse-proxy.md`.
Compatibility: the default moves from "everyone admin" to "first admin,
then users". Accounts already in `users.db` are unaffected; new SSO
users on an existing instance become regular users unless
`PROXY_AUTH_DEFAULT_ROLE=admin` is set. If you would rather ship this
purely opt-in I can flip the default to `admin`.
## Verification
- `tests/core/test_auth_api.py::TestProxyProvisioningRole`: first user
admin / second user not; `PROXY_AUTH_DEFAULT_ROLE=admin` restores the
old behaviour; an admin from another auth source counts as "an admin
exists"; a known user keeps their role whatever the default.
- Full suite (3094), ruff, ruff format, basedpyright, vulture green.
- Running on my own instance since 2026-09-19.
The `/api/releases` route carries an inner `_search_source_releases`
helper that builds the search plan for one source, logs the planned
query type, runs the search and turns
`SourceUnavailableError`/operational errors into an error message
instead of raising. Anything outside the route that wants to search one
source with exactly those semantics has to go through Flask today.
This moves that helper into `shelfmark/core/release_search.py` as
`search_source_releases()` and has the route delegate to it. Behaviour
is unchanged: same plan construction (including the caller's `user_id`,
so per-user default languages still apply), same logging, same
error-to-message handling.
It is the refactor half of #1047 by @InfiniteAvenger, split out on its
own as you asked for other PRs (#1318). Their authorship is preserved on
the commit; I rebased it onto current `main` and added tests.
## Verification
- `tests/core/test_release_search.py`: unknown source → `"Unknown
source: …"`, `SourceUnavailableError` and operational errors →
`"<source>: <error>"`, success path forwards `expand_search` /
`content_type` and returns the source instance, the plan receives
languages / manual query / indexers / `user_id`. (These tests are
type-annotated; happy to strip the annotations if you prefer the suite's
bare style.)
- Full suite, ruff, ruff format, basedpyright, vulture green; the
existing `/api/releases` route tests are unchanged and pass.
Co-authored-by: InfiniteAvenger <calebewest02@gmail.com>
DownloadHistoryService.record_download and updated the single production
caller, but not the eleven in the test suite, leaving main red with 32
failures. Pass None, which is what the pre-#1336 behaviour recorded.
For the Blackhole handoff (#1345): add_download publishes the torrent
before the cancel check runs, and BlackholeClient.remove() is a no-op,
so
the watcher picks the file up regardless. Reporting a bare "Cancelled"
hid
that from the user. Name the completed handoff in the cancellation
message
instead, drop the _handle_cancelled_download call whose usenet branch
cannot apply to a handoff-only client, and record why the orchestrator
no
longer verifies HandoffResult.path.
Finally, make tests/direct_download a package: test_libgen_extract.py
imports tests.libgen.sample_html across test directories, so without an
__init__.py pytest named its modules by bare basename and a same-named
module elsewhere would collide.
Closes#1230
## What
Adds a "Word Separator" setting (Space / Dot / Underscore / Hyphen /
Custom) that replaces internal whitespace in each naming-template
placeholder's rendered value — e.g. `{Author}` renders
"Arthur.Conan.Doyle" instead of "Arthur Conan Doyle" when Dot is
selected.
This follows option 2 from the issue rather than inventing new
dotted-keyword template syntax (`{Author.}`), since it's a smaller
surface: one setting applies uniformly across all four templates
(books/audiobooks × rename/organize) instead of needing a parallel token
for every existing one.
## How it works
- Literal characters typed into the template itself (e.g. the `.` in
`{Author}.-.{Title}`) are never touched — only whitespace *inside* a
placeholder's resolved value is affected.
- Default is "Space", which is a no-op: existing templates produce
byte-identical output after this change (verified via the existing test
suite, unmodified, still passing).
## Where
- `shelfmark/core/naming.py` — `word_separator` param on
`parse_naming_template` / `build_library_path`.
- `shelfmark/download/postprocess/policy.py` — `get_word_separator()`,
mirroring the existing `get_file_organization()` accessor.
- `shelfmark/download/postprocess/transfer.py` — wires the resolved
separator through the four existing template-rendering call sites.
- `shelfmark/config/settings.py` — new `Word Separator` / `Custom Word
Separator` fields next to the existing naming-template fields.
- `src/frontend/.../namingTemplatePreview.ts` +
`NamingTemplateField.tsx` — the settings UI has its own TS mirror of the
Python renderer for the live preview; updated it in lockstep so the
preview doesn't lie about what the separator will actually do.
- Tests added on both sides (pytest + vitest).
## Testing
- `uv run pytest tests/core/test_naming.py
tests/core/test_destination_file_organization.py` — all pass, including
new cases.
- `uv run pytest` (full suite) — same pre-existing failures as on `main`
before this change (browser/network-dependent bypass & e2e tests
unrelated to this diff), everything else green.
- `uv run ruff check` / `ruff format --check` / `basedpyright` — clean.
- `npm run lint` / `format:check` / `typecheck` / `test:unit` (196
tests) — clean.
Fixes#1331.
A search bot ANDs every term against a filename, so the given name is
the term
that empties the result set. Measured against irchighway's #ebooks:
"Revelations
David Petrie" is answered "no results", "Revelations Petrie" returns 9
matches,
6 of which parse, all filed as "D Petrie".
The query now carries the title and the surname, read off the search
variant so
the ISBN fallback and a manual query - which set author="" on purpose -
keep
their current shape.
Title-only, the shape #1295 settled on for Prowlarr, does not transfer:
the bot
caps an answer at 1000 matches, and a bare "Revelations" hits that cap
with 923
parsed rows across 500 authors, so the cap itself can drop the wanted
book. The
surname is the token the two spellings share and it keeps the answer
small.
The full author then orders what comes back, reusing author_affinity
from #1295,
since a surname also matches a different author who shares it. It sits
under
server availability the way indexer priority does in #1295: a download
addresses
one named bot and waits 120s for it, so a match from a bot that has left
the
channel must not outrank a mismatch that can answer. Ranking runs on the
way out
rather than before the cache, because one query identity is shared by
every book
that produced that query.
Two things found while testing:
- The parser writes the literal "Unknown" when a filename has no " - "
separator
(parser.py:168). Ranked literally that sorts as a wrong author, so
author_affinity's middle tier was unreachable here; it is now read as
absent.
5 of those 923 rows are affected.
- author_affinity moves to shelfmark/core/author_match.py, unchanged, so
IRC
does not import from the Prowlarr package. Prowlarr behaviour is
untouched and
its tests pass as they are.
The three IRC assertions in the #1252 regression file move to the
surname form.
The invariant they pin - one contributor's name reaches the query, never
the
whole credit list - is unchanged.
Tested with make python-lint, python-format, python-dead-code,
python-typecheck
and python-test, and end to end against irchighway with the patched
source: it
posts "Revelations Petrie" and returns 6 releases.
This patch shares (for each log file) the `RotatingFileHandler` for
logging across all modules, reducing the number of open file descriptors
from ~78 to 1.
I had originally assumed this issue was a resource leak, but it seems to
just be a large fixed number of file descriptors. So this change mostly
just (1) shrinks the number of open file descriptors to a reasonable
level and (2) prevents two modules in the same process competing to
write to a log file.
Closes#930.
## What
New `{FirstAuthor}` naming-template token. It renders only the first
author when metadata lists several ("Author1, Author2, Author3"), so
multi-author books can be filed alongside the rest of that author's work
instead of getting their own "Author1, Author2, ..." folder.
```
{Author} -> Terry Pratchett, Neil Gaiman
{FirstAuthor} -> Terry Pratchett
```
## How
- Added to `KNOWN_TOKENS` in `shelfmark/core/naming.py`, positioned
before `author` so `{FirstAuthor}` isn't parsed as literal `First` +
`{Author}`.
- Derived inside `parse_naming_template` from the existing `Author`
value (split on `,` / `;`), so every caller — folder transfer, rename,
the settings preview — picks it up with no extra wiring. An explicit
`FirstAuthor` key in the metadata still wins if one is ever passed.
- `{Author}` behaviour is unchanged.
- Frontend `namingTemplatePreview.ts` token list + `KNOWN_TOKENS` kept
in lockstep (there's a test enforcing that), with a matching
`firstAuthor` helper.
- Settings field descriptions + `docs/environment-variables.md` list the
new token.
## Known limitation
A lone author written `Last, First` is split on the comma too and
renders as `Last` — the source metadata doesn't mark which form it is.
Called out in the token help text and covered by a test. `{Author}`
remains available for anyone who wants the raw string.
## Checks
- `make python-test` — 2963 passed
- `make python-lint` / `make python-format` / `make python-typecheck` /
vulture — clean
- `make frontend-test` — 187 passed · `frontend-lint` /
`frontend-format` / `frontend-typecheck` — clean
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Two independent reasons a working search reported failure to the user.
1. The client gave up before the server did (#1285)
`/api/releases` bounds one release search with RELEASE_SEARCH_TIMEOUT
(default
300s) and answers a spent budget with a sentence naming the real cause -
the
machinery added for #1276. The frontend then aborted the direct_download
search
at a hard-coded 180s, so it always won the race: the user saw "Request
timed
out. Check your network connection or proxy configuration." instead, and
raising RELEASE_SEARCH_TIMEOUT changed nothing they could observe, the
180s
being baked into the hashed bundle inside the image.
- /api/config reports the effective (clamped) budget, and the client
derives its
abort from it plus a margin, so the server always answers first.
- Direct-mode search shows what the server actually said. Every non-auth
failure
was relabelled "Unable to reach download source. Network may be
restricted or
mirrors blocked.", which discarded the explanation and blamed the user's
network. ApiResponseError now carries `serverMessage`, set only when the
server
explained itself, so the status-line placeholder still falls back.
Two latency fixes for the cost that made the timeout reachable at all:
- Fetch each distinct AA search URL once per search. The language-filter
retry
re-runs every title variant, and with DIRECT_DOWNLOAD_LANGUAGE_FROM_PATH
on
both passes build a byte-identical URL - behind DDoS-Guard each repeat
is a
fresh browser solve.
- Drop the solve-only bypass method. `_bypass_method_cdp_gui_click`
opens with
exactly that call and returns the moment it works, so the entry ahead of
it
could only repeat the half that had already failed, plus the backoff
before
the method that does work started. Reported at 0/19 successes and ~5.5s
of
each ~26s solve against DDoS-Guard.
2. The query carried every contributor, not one author (#1252)
`_pick_search_author` returned `book.search_author` verbatim while the
authors[]
fallback beside it deliberately narrowed to the first name before a
comma. Both
fields routinely arrive holding every contributor joined with ", ": the
frontend
builds `book.author` as `authors.join(', ')` for display
(bookTransformers.ts)
and the release modal sends that display string straight back as the
`author`
parameter, and `browse_record_to_book_metadata` and the manual-search
branch
both split the joined text into `authors` while still passing the
unsplit string
as `search_author`, so the split was never used.
A book whose metadata lists translators was therefore searched for as
Blindness Jose Saramago, Giovanni Pontiero, <persian translator>
which matches nothing on Anna's Archive. The bypass succeeds, the search
comes
back empty, and the user is told the book has no releases.
Narrowed in one place, `search_plan.first_author`, so the two branches
cannot
drift apart again, and applied to the IRC source, which built its query
with the
same verbatim preference. Hardcover is unaffected: it already sets
`search_author` from `_simplify_author_for_search(authors[0])`, which
resolves
"Last, First" itself and never yields a multi-author string.
Fixes#1252 for the case in the second report.
`_pick_search_author` returns `search_author` untouched but trims
`authors[0]` to its first comma-separated name. So the same credit list
searches differently depending on which field carries it:
```
via authors[0] -> "Blindness Jose Saramago"
via search_author -> "Blindness Jose Saramago, Giovanni Pontiero, Zohreh Eftekhari"
```
Anna's Archive answers the second one with nothing. That is the query in
@theDoz12's log, and it explains the shape of the report: the bypass
succeeds, the search runs, and the UI still says no releases. Nothing in
the download path is broken, the query simply cannot match.
Measured against live AA on 1.3.14, same book, same source, only the
field carrying the author changed:
| query | releases |
| --- | --- |
| `Blindness Jose Saramago, Giovanni Pontiero, Zohreh Eftekhari` | 0 |
| `Blindness Jose Saramago` | 49 |
| `Blindness` | 50 |
With the patch the second form is produced from either field, and the
same search returns 49.
Three regression tests added, including one that asserts both fields
yield the same query. On `tests/core/test_search_plan.py` the run goes
from 5 failures to 3; the 3 that remain are the language tests, which
fail identically with and without this change on my machine.
Worth saying what this does not cover: the first report in that issue
ends with `Found 2 releases via ISBN` and still shows nothing, so that
one is a different fault further along. I could not reproduce it here.
## Multi-book packs: inspect a release before download and file each
book separately
Closes#576
### Problem
One queued release is always treated as one book. When a torrent is
actually a whole series
(`Series/Book 1 - Title/…`, or a flat folder of `Series 1.0 - Title.m4b`
files), post-processing
walks the whole tree, flattens every file into one list and renames them
`Title - 01…10` under the
searched book's `{Author}/{Title}`. Audiobookshelf then sees a single
10-file "book" and the user
has to re-file everything by hand.
### What this does
Most releases expose their file list *before* anything is downloaded, so
the split is decided up
front and approved by the user, then the download is fire-and-forget:
1. **Inspect** – clicking a release's download button now calls `POST
/api/releases/inspect`
first. A new optional `DownloadHandler.list_files(release_data)` hook
returns the release's
files without downloading:
- **AudiobookBay** reads the torrent file table off the detail page it
already fetches (the
page is now cached for 120 s, so inspect + download cost ABB one
request).
- **Prowlarr** parses `info.files` from the `.torrent` it already
fetches (the existing 120 s
torrent-fetch cache is reused). Magnet-only and usenet releases report
"can't inspect".
- Other sources default to `None`.
2. **Review** – if the plan contains more than one book, the Find
Releases modal swaps the list
for a review panel: one row per book with editable title / series
position / year, expandable
file lists, non-book sidecars (`.txt`, covers) shown as ignored, a
"Treat as a single book"
switch, and **Download N books**. Single-book releases queue
immediately, exactly as before.
3. **File** – the approved plan travels with the task
(`DownloadTask.book_plan`, retry-safe) and
post-processing files each book through the existing transfer code, one
book at a time
(`dataclasses.replace(task, title=…, series_position=…, year=…)`), so
organize/rename
templates, part numbering (now scoped per book), hardlinks, torrent
copy-preserve and usenet
handling are unchanged. Status reads `Complete (N books, M files)`.
4. **Fallback** – when a release can't be inspected the user gets a
toast, and a small
"Multi-book pack" toggle in the modal header forces a heuristic split
(subfolder = book, or
one book per file when the file names carry series positions).
Planning lives in `shelfmark/download/postprocess/packs.py` and is
shared by the inspect endpoint
and post-processing, so what the user approved is what gets filed. The
name parser strips
`Book 3 -`, `03 -`, `1.0 -`, `3.`, `[03]`, `#3`, a leading series name,
labels like
"An Expanse Novella -", repeated titles (`Gods of Risk 2.5 - Gods of
Risk`) and a trailing
`(Year)`; author and series name come from the book that was searched,
and the searched book's
own series position is never applied to its siblings.
### Files
- `shelfmark/download/postprocess/packs.py` (new) –
`PackFile/PackBook/PackPlan`, `plan_pack`,
`parse_pack_book_name`, `group_files_into_books`, `match_plan_to_files`
- `shelfmark/core/release_inspect_routes.py` (new) – `POST
/api/releases/inspect`
- `shelfmark/release_sources/__init__.py` – `DownloadHandler.list_files`
hook
- `shelfmark/release_sources/audiobookbay/{scraper,handler}.py` –
detail-page cache,
`extract_file_list`, `list_files`
- `shelfmark/release_sources/prowlarr/handler.py`,
`download/clients/torrent_utils.py` –
`extract_file_list_from_torrent`, `list_files`
- `shelfmark/core/models.py`, `download/orchestrator.py` – `multi_book`
/ `book_plan` fields,
queue + retry serialization
- `shelfmark/download/postprocess/transfer.py`, `pipeline.py`,
`outputs/folder.py` – per-book
transfer branch and status message
- `src/frontend`: `components/PackReviewPanel.tsx` (new),
`ReleaseModal.tsx`, `App.tsx`,
`services/api.ts`, `types/index.ts`, `utils/releasePayload.ts` (payload
builder moved out of
`App.tsx`), `utils/packReview.ts`
- `docs/dev/release-sources-plugin-guide.md` – documents the
`list_files` hook
### Out of scope (follow-ups)
- Listing files from an NZB (Shelfmark already fetches the bytes; `<file
subject>` names are noisy)
- Inspecting magnet links via qBittorrent's files API after a paused add
- BookLore / email outputs (they ignore `book_plan`; noted in code)
- The combined ebook + audiobook flow
### Testing
**Automated** (`make checks`, `make python-test`, `make frontend-test`
all green; the only
failures on my machine are the pre-existing
`tests/config/test_entrypoint_permissions.py` cases,
which need bash ≥ 4 and fail identically on `main` under macOS bash
3.2):
- `tests/download/test_packs.py` – name parsing (markers, series name,
novella labels, repeated
titles, bare numeric titles like `1984`), nested / flat / mixed /
deeper-nested packs, single
wrapping folder not treated as a pack, plan-to-disk matching with
basename fallback
- `tests/core/test_processing_packs.py` – full `post_process_download`
runs on a real temp
filesystem: approved plan files each book under its own
`{Author}/{Title}`, heuristic split
of a nested pack, searched book's series position does not leak,
multi-file book inside a pack
keeps `- 01/- 02` per book, hardlinked torrent pack leaves the seeding
tree intact, no pack
fields ⇒ behaviour unchanged, single group degrades to the searched
title, status message
- `tests/core/test_release_inspect_routes.py` – plan response,
not-inspectable, handler errors
never 500, unknown source / missing `source_id` ⇒ 400, login required
- `tests/audiobookbay/test_file_list.py` – file-table scraping from real
ABB markup (multi-file
and single-file pages), handler host validation, one page fetch shared
by magnet + file list
- `tests/prowlarr/test_torrent_file_list.py` – multi-file / single-file
`.torrent` parsing,
handler behaviour for torrent URL vs magnet vs usenet vs cache miss
- `tests/download/test_orchestrator_pack_fields.py` – queue-time parsing
and retry round-trip
- Frontend: `releasePayload.test.ts`, `packReview.test.ts` (vitest)
**Manual, on a real deployment** (arm64 image built from this branch,
run as a side container
next to production with the same qBittorrent / Audiobookshelf setup,
`FILE_ORGANIZATION_AUDIOBOOK=organize`,
hardlinks on):
- AudiobookBay "The Expanse Complete 2.0" (7.87 GB, 36 files): clicking
download opened the review
panel in ~1 s showing **18 books · 18 files · 18 files ignored** (the
`.txt` sidecars), with
series positions 0.1–9.5 and years parsed from the file names; novella
labels stripped
("The Churn", "The Butcher of Anderson Station"). Editing a title in the
panel works.
Confirming queued one task; the magnet resolved from the cached page in
~30 ms; after the
download the task reported `Complete (18 books, 18 files)`, 18 hardlinks
landed as
`audiobooks/James S. A. Corey/<Title>/<Title>.m4b`, the torrent kept
seeding, and
Audiobookshelf scanned each folder as its own book (title, author,
embedded chapters).
- A second pack ("Expanse [01 - 9.5]", `Title N - Title` naming) was
inspected to verify the
repeated-title rule and the Back button, without downloading.
- Single-book releases still queue immediately with no extra UI.
Review follow-ups to #1255, all in the code that PR touched.
Drop the dead user_id from the Prowlarr retry path.
ProwlarrSource.search
never reads plan.languages, and _refresh_release builds a synthetic book
with no titles_by_language, so the title variants came out identical
with
and without it. It also should not language-filter: it re-finds one
exact
release by its guid.
Pin the tab move in tests. BOOK_LANGUAGE moved from the General tab to
Search Mode with no migration, which only works because both tabs
persist
into the same settings.json. Nothing asserted that, so splitting the
files
later would silently reset every install to ["en"]. Covers the stored
value, a fresh install, and ENV precedence.
Stop the UI inventing a default language. An empty BOOK_LANGUAGE is a
deliberate "no default filter" that the backend preserves, but the two
frontend call sites replaced it with the first supported language, so
the
filter said English where the server filtered nothing.
resolveDefaultLanguageCodes
now falls back only when the value is absent.
Keep the normalized value for every validated search key.
validate_user_settings
gated the write-back on a hand-maintained subset of the keys the search
validator recognises, so METADATA_PROVIDER_COMBINED,
SHOW_COMBINED_SELECTOR
and FORCE_COMBINED_SEARCH were validated and then stored raw -- a padded
provider name was accepted and persisted with its padding. Reuse the
validator's own key set instead.
Skip blank language entries rather than rejecting them, so "" and "en,"
mean the same as [] and ["en"] instead of erroring on an unnamed
language.
Extract resolveListOverride for the list-override detection that was
copy-pasted between the two user-settings sections, and mention
languages
in the Search Preferences section description.
## Why
`BOOK_LANGUAGE` is a per-reader property, not a per-instance one. On a
shared install one household member searches in German while another
wants English and German — today whoever changes the setting changes it
for everyone, and the only escape is re-picking languages in the filter
on every single search.
The per-user override machinery already carries `SEARCH_MODE`, the
metadata providers and the default release sources, so the language
default mostly had to opt into it.
## What changed
**The field.** `BOOK_LANGUAGE` becomes `user_overridable` and moves from
the **General** tab to **Search Mode**, next to the other
user-overridable search defaults (per
[review](https://github.com/calibrain/shelfmark/pull/1255#issuecomment-5391189094)
— the first version had the Search section span two tabs, this one
doesn't). Admins set it per user in the user editor, users set it in
**My Account → Search Preferences**, and the Search Mode tab carries the
usual "N users override this" summary.
**No migration for the move.** `general` and `search_mode` both persist
into `settings.json`, and a field's value is resolved through
`load_config_file(tab)` for the tab it's declared on — so an install
that already stores `BOOK_LANGUAGE` keeps its value. Checked against a
`settings.json` written while the field still lived on General: the
stored value resolves unchanged, a fresh install still gets `["en"]`,
and `BOOK_LANGUAGE` in the environment still overrides both.
**The two places the default is read.**
- `/api/config` seeds the frontend's language filter, so it now resolves
`BOOK_LANGUAGE` for the session user.
- `build_release_search_plan` falls back to the default whenever a
request carries no language filter — which is exactly what the filter's
"Default" option sends. It takes an optional `user_id`, passed by
`/api/releases` from the session and by the Prowlarr retry path from
`task.user_id`, so a retry re-searches in the languages of whoever
queued the download.
**Validation.** Overrides go through `normalize_language()`, so
`"German"`, `"ger"` and `"de"` all store as `de`, and an unknown
language is rejected with a message naming it instead of being silently
searched for. An empty list stays an empty list (a deliberate "no
default filter"), `null` clears the override as everywhere else, and ENV
still wins: with `BOOK_LANGUAGE` set in the environment the field
reports `fromEnv` and overrides are ignored.
**Scope.** Only the language default becomes overridable. The two format
lists left behind under "Default Search Filters" stay admin-only — they
describe what the library and its post-processing accept, not what a
reader wants to read. There's a test pinning that.
## Verification
- 2681 unit tests pass (2670 before, 11 added)
- `ruff check`, `ruff format`, `basedpyright` over backend and tests,
and `vulture` all clean; frontend lint, format, typecheck and 126 unit
tests clean
- `docs/environment-variables.md` regenerated via
`scripts/generate_env_docs.py` (the `BOOK_LANGUAGE` row follows the
field into the Search Mode section)
- Manually against a two-user instance with builtin auth (first round,
before the tab move): with user A on German and user B on
English+German, `/api/config` returns each reader their own
`default_language` and an unfiltered `/api/releases` plans the matching
languages; an admin can set and read the same override for another user;
clearing it falls back to the global value; a stray `"klingon"` is
rejected; and `BOOK_LANGUAGE` in the environment overrides both users
with the field marked `fromEnv`
- After the tab move I re-ran the suites above plus the
stored-value/fresh-install/ENV check described under "No migration for
the move"; the behaviour it exercises is what the move could have broken
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
## Problem
Some trackers — MyAnonamouse in particular — distribute AAC audiobooks
as per-chapter `.mp4` files. That's the same ISO-BMFF container as
`.m4a`/`.m4b`, just with the generic extension (`ftyp isom`,
audio-only).
Today those releases:
1. show up in Prowlarr search results with **no format chip** — only the
generic "Audiobook" icon, because no format could be inferred;
2. download successfully; then
3. fail post-processing with **"No book files found in download"**,
because `.mp4` isn't in `AUDIOBOOK_FORMATS` (`shelfmark/core/utils.py`).
Real example: MAM #627978, *The Martian* (Andy Weir, 2020 edition) — 142
files `0001 … 0142 Andy Weir (2020) The Martian.mp4` + `cover.jpg`, 305
MB. Every file is a valid AAC-in-MP4 chapter.
Adding `mp4` to `SUPPORTED_AUDIOBOOK_FORMATS` in `settings.json` doesn't
help since the hard-coded tuple is what post-processing scans against.
## Change
- Add `"mp4"` to `AUDIOBOOK_FORMATS` (single source of truth — settings
UI, Prowlarr parsing, IRC parser, archive extraction and post-download
scan all derive from it), with a comment explaining why.
- Add `".mp4"` to the two hand-maintained debrid `_BOOK_EXTENSIONS`
lists (AllDebrid / Real-Debrid) so file selection matches.
- Slot `mp4` into the IRC `AUDIOBOOK_FORMAT_PRIORITY` table right after
`m4a` (same container family).
- Update the documented default in `docs/environment-variables.md`.
- New regression test
`test_audiobook_multifile_mp4_chapters_are_book_files` modelled on the
existing multi-file usenet test.
### Note for existing installs
The legacy-default migration only widens configs that still hold the old
`m4b,mp3` list, so users on the current widened default won't pick up
`mp4` automatically — they'll need to tick it in Settings → Audiobook
formats. New installs get it by default. Happy to extend the migration
if you'd rather it be automatic.
## Testing
- `ruff check` / `ruff format --check`: clean
- `pytest tests/core tests/config tests/irc tests/prowlarr
tests/download -m "not integration and not e2e"`: 2296 passed, new test
+ `test_audiobook_format_consistency.py` all green. The 10 failures in
`test_entrypoint_permissions.py` / `test_orchestrator_stall.py`
reproduce identically on untouched `main` on macOS and are unrelated.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Follow-up to #1237. \`rename_and_group\` only grouped when the source
root was a directory, so a multi-file audiobook delivered as a single
archive fell through to the flat path: a \`Book.zip\` of twelve chapters
landed loose in the destination root with its original chapter names —
the layout #1181 is about.
The \`is_dir()\` guard was there to keep \`Book.zip/\` from becoming the
folder name, but skipping the file case gives up the grouping instead of
naming it. A non-directory source can only produce several book files by
having been extracted (\`collect_staged_files\` returns a single-element
list for every other file shape), so the archive stem is the release
name and the suffix is packaging: group under \`Book/\`.
Also regenerates the env docs for the new option and gives it the same
\"do not use with ingest folders\" caveat Rename and Organize carries,
since both now create directories in the destination.
Tested: reverting only the source fix makes both new tests fail and the
\`rename\` control case pass, so grouping stays opt-in. Full non-e2e
suite green (2653 passed).
Pass the effective source root from `process_folder_output` into
`transfer_book_files`, and have the transfer layer select a sanitized
child directory named after that source root when an audiobook has
multiple files and its organization mode is `none` or `rename`. Create
that grouping directory before applying the existing hardlink/copy/move
logic so operation accounting, torrent seeding preservation, collision
handling, cleanup, and custom-script final paths continue to use the
established production path. Completed multi-file audiobook torrents
arrive as a directory whose chapter filenames may not identify the book,
but folder output currently sends every discovered chapter directly to
the configured destination in `none` and `rename` modes. This flattens
chapters from unrelated books together and causes directory-oriented
consumers such as Audiobookshelf to interpret individual chapters as
separate books.
A multi-file audiobook torrent in the default `rename` mode copies or
hardlinks all supported chapter files beneath `<destination>/<original
torrent directory>/` with their original chapter filenames, and places
no chapters directly in the destination root; A multi-file audiobook in
`none` mode receives the same source-folder grouping without renaming
its chapter files.
Fixes#1181
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
## Summary
- sync an existing CWA-backed user's username when CWA renames it
- keep username collisions safe by assigning a stable `__cwa` alias
instead of overwriting a local account
- allow username updates through `UserDB` and cover
rename/collision/repeat-sync behavior
Fixes#1197.
## Testing
- `uv run ruff check shelfmark tests`
- `uv run ruff format --check shelfmark tests`
- `uv run vulture shelfmark`
- `uv run pytest tests/core/test_cwa_user_sync.py
tests/core/test_user_db.py tests/core/test_admin_users_api.py
tests/core/test_auth_api.py -k "cwa or update_user"` (36 passed)
- `uv run pytest tests/ -x --tb=short -m "not integration and not e2e"
--ignore=tests/config/test_entrypoint_permissions.py -q` (2445 passed, 5
skipped)
The entrypoint permission tests were excluded locally because macOS
ships Bash 3.2, which does not support the `${1,,}` expansion used by
`entrypoint.sh`; the same failure reproduces on an unchanged checkout.
`make python-typecheck` also currently reports the existing
`settings.py:147` callback return-type mismatch on the unchanged base.
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
An IRC audiobook search returned nothing while OpenBooks, reading the
same @search answer from the same channel, listed results. Three
separate defects were discarding them.
The audiobook format list was maintained by hand in four places and had
drifted. The settings UI offered only m4b/mp3/m4a/zip/rar, and that list
is the only one a user's config can be built from, so flac, opus, ogg,
aac, wav and wma were unreachable everywhere — even though the IRC
parser recognized them, the IRC sorter ranked them (dead code that could
never fire), archive extraction knew them and Prowlarr searched for
them. A FLAC audiobook was invisible in search and, if it arrived
anyway, rejected after download as "format not supported".
AUDIOBOOK_FORMATS and ARCHIVE_FORMATS now live once in core.utils and
every layer derives from them, which also restored the missing .opus in
the post-download scan's trackable extensions.
Widening the default alone would not have reached anyone already
affected: initialize_default_configs() writes field defaults only when a
tab has no config file yet, so an existing install keeps its persisted
m4b/mp3 list forever. migrate_audiobook_formats rewrites a list that
still matches the old default exactly and leaves every other value
alone — re-enabling formats someone had deliberately turned off would be
worse than leaving them narrow.
The IRC parser filtered by file extension alone. Multi-file audiobooks
ship as a .rar or .zip of MP3s, which matched neither SUPPORTED_FORMATS
nor SUPPORTED_AUDIOBOOK_FORMATS, so they fell out of the ebook bucket
and the audiobook bucket both. Results are now classified before the
format filter is applied: an audio extension means audiobook, an ebook
extension means ebook, and for a container — where the extension says
nothing about the contents — the release name decides. An ebook archive
stays out of audiobook results.
RESULT_LINE_REGEX matched \w+ after any dot, so a line carrying no file
extension parsed as format "5mb" out of "::INFO:: 620.5MB", taking the
title and the size down with it and guaranteeing every downstream filter
dropped it. Any decimal size did this. The extension is now matched
against the known formats, so such a line falls through to the simple
pattern and comes back as "unknown", which the rest of the parser
already handles. ALL_RECOGNIZED_FORMATS became an ordered tuple in the
process: it was a set, so which extension won for a line naming two of
them depended on set iteration order and could vary between restarts.
Refs #1129
LOG_LEVEL never reached the app logger: env.py hardcoded the level to
DEBUG or INFO, so INFO lines kept appearing under LOG_LEVEL=error. Read
it from the env var and advanced settings, normalize unknown values to
INFO, and expose it as a setting. entrypoint.sh now normalizes
gunicorn's level too, so a typo falls back to info instead of stopping
the container from booting.
Z-Library gates the first hit on /md5/<hash> with a 503 whose only
payload is a Set-Cookie; echoing that cookie back returns the 302 to the
real page. html_get_page dropped it and re-ran the same rejected request
on every retry, ending in "No download URL resolved". Retry once with
the
cookies the 503 issued.
Fixes#1185Fixes#1187
Fixes#1175
Preserves the forwarded host and port when Shelfmark builds OIDC
callback URLs. The reverse-proxy examples now retain custom ports as
well.
Tests:
- `uv run pytest -n 0 -q tests/core/test_proxy_headers.py
tests/core/test_oidc_routes.py`
- `uv run ruff check shelfmark/main.py tests/core/test_proxy_headers.py`
- `uv run ruff format --check shelfmark/main.py
tests/core/test_proxy_headers.py`
Fixes#1138Fixes#1141
## Problem
Two language editions of one book resolve to the same canonical title,
so they render to the same path and the second gets a `_1` collision
suffix. Audiobookshelf treats a folder as exactly one library item, so
the pair becomes a single book with both files as tracks and a summed
runtime.
Shelfmark already parses and displays the language. It just never
reached the template engine.
## `{Language}` template variable
A template like `{Author}/{Title}{ (Language)}/{Author} - {Title}` now
yields:
```
/library/J K Rowling/Harry Potter (sv)/J K Rowling - Harry Potter.m4b
/library/J K Rowling/Harry Potter/J K Rowling - Harry Potter.m4b
```
The untagged edition's path is byte-identical to today, so no existing
layout shifts.
Three details worth flagging:
**The value is casefolded.** On a case-insensitive filesystem `(SV)` and
`(sv)` would collapse back into one folder, reintroducing the exact
collision being fixed.
**Values meaning "we don't know" render nothing** rather than producing
`Project Hail Mary (unknown)` folders. Anna's Archive reports that
string literally (`direct_download.py`, `language = detected or
"unknown"`).
**The frontend wasn't sending the release language at all**, so the
token would have stayed empty for exactly the audiobook sources in the
report. Prowlarr and AudiobookBay do not put language in `extra` the way
`direct_download` does, hence the payload plumbing. It reads
`release.language`, never `book.language` — the latter is the provider's
canonical edition and would mislabel a translation, with a regression
test for that specifically.
Not gated to audiobooks: Calibre-Web-Automated stages ingested files by
basename and discards folder structure, so the rename (filename)
template is the only lever those users have. Verified that form works:
`J K Rowling - Harry Potter (sv).epub`.
## Language consolidation (#1141)
Three release sources each carried their own alias map, all resolving to
the same ISO 639-1 codes, alongside a bundled database that only one of
them used. Adding a language meant editing three places.
Aliases now live in `data/book-languages.json` beside the code and name
they belong to, and `shelfmark/core/languages.py` resolves any of them —
two-letter code, ISO 639-2 three-letter in either the bibliographic or
terminological form, or English name. Prowlarr and AudiobookBay drop
their tables. Direct Download keeps its own path-parsing heuristics,
including the ambiguous short codes that collide with English words
(`de`, `en`, `no`, `in`), and takes only the alias data.
This also closes a coverage gap. MyAnonamouse offers 62 languages;
Prowlarr mapped 37, and an unmapped code is *dropped* rather than passed
through, so the other 25 carried no language at all — leaving
`{Language}` empty and the collision unfixed for Latin, Farsi, Tamil,
Urdu and the rest. Seven languages MAM offers had no database entry at
all: Bosnian, Burmese, Estonian, Icelandic, Manx, Scottish Gaelic,
Sanskrit.
Also fixes the Traditional Chinese code, which used a U+2011
non-breaking hyphen. Nothing compares against the ASCII spelling today
so it was latent, but it would silently defeat the first thing that did.
## Validation
Verified end to end against a live Prowlarr and MyAnonamouse, not just
unit tests. A real search returning both an English and a Swedish
edition, through the actual `queue_release` → `DownloadTask` → naming
path:
```
STEP 1 real MAM search -> 37 releases, languages: ['en', 'sv']
STEP 3 queue_release -> task.language='sv'
STEP 4 build_metadata_dict -> metadata['Language']='sv'
STEP 5 build_library_path -> /library/J K Rowling/Harry Potter (sv)/...
two language editions resolve to DIFFERENT folders: True
```
The refactor is pinned by a snapshot of both per-source maps taken
*before* they were deleted. All 131 aliases are asserted to still
resolve to the same code, one parametrised test each, so a regression
names the specific alias.
Also verified: the filename-only template, the retry round-trip
(`serialize_task_for_retry` → `_restore_task_from_retry_payload`, plus a
legacy payload with no `language` key), and placeholder handling.
Added a `KNOWN_TOKENS` ordering invariant test — `find_placeholder()`
does a substring `.find()` in list order and nothing protected that
contract, so a future token in the wrong position could silently shadow
an existing one. And a lockstep guard on the frontend, since
`KNOWN_TOKENS` is hand-duplicated in TypeScript.
**One caveat worth stating.** Three MAM codes are confirmed by
observation (`ENG`→`en`, `SWE`→`sv`, `MAL`→`ml`, the last from a real
`[MAL / EPUB]` Tagore release). The remaining ~59 are derived from ISO
639-2 rather than observed, because MAM's catalogue is overwhelmingly
English — enabling 27 extra languages still yielded only one non-English
hit across 258 results. Mitigated rather than closed: both 639-2
variants are present for every language where they differ, and a wrong
alias is an unused entry while a missing one loses the language. Happy
to correct any code a maintainer knows differs.
## Test results
2056 Python tests pass (up from 1906). Frontend typecheck, lint, format
and 126 unit tests pass.
Pre-existing failures on my machine, unchanged by this branch and
unrelated: `tests/bypass/` needs `seleniumbase`, and
`tests/config/test_entrypoint_permissions.py` uses bash-4 syntax that
macOS bash 3.2 rejects.
---------
Co-authored-by: delize <4028612+delize@users.noreply.github.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
## Summary
Audiobooks whose files are split across nested subfolders now keep their
on-disk folder order when "Rename and Organize" assigns part numbers.
Previously `natural_sort_key` keyed on `Path(path).name`, so identically
named files from different folders (`001.mp3` in `00_Introduction/` vs
`06_Side 6/`) collapsed together and the part sequence scrambled.
## Background
zazizou reported in #1007 that the total file count was right but
`06_Side 6/001.mp3` came out as part 1 instead of
`00_Introduction/001_About.mp3`. `scan.py` flattens every nested file
into one list and `assign_part_numbers` sorts it with
`natural_sort_key`, which discarded the parent-folder prefix. The fix
pads the existing `PAD_NUMBERS_PATTERN` numbers over the full relative
path (`str(path).lower()`) rather than the basename, so nested
audiobooks sort by folder first and then by filename. Flat directories
are unaffected because their files share a prefix, and
`natural_sort_key` is only called from `assign_part_numbers`, so nothing
else changes. I updated the test that pinned the old basename-only
ordering and added nested-subfolder and duplicate-basename cases;
`pytest tests/core/test_naming.py
tests/core/test_part_number_extraction.py` passes (102 tests).
Closes#1007
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
## Backport bug fixes from `NemesisHubris/litfinder`
Forwards a curated set of bug fixes from
[NemesisHubris/litfinder](https://github.com/NemesisHubris/litfinder) —
a community fork of this project — that address open issues here. All
commits preserve original authorship via `git cherry-pick`; this PR is a
backport rather than original work. Each fix has been reviewed locally,
lint/format-cleaned to match this repo's existing ruff config, and
verified with the test suite. Rebrand strings, license switches, and
features have been deliberately excluded.
### Upstream issues addressed
- **#999** — Mirror URLs with query params no longer break search
requests (strip query string/fragment in `normalize_http_url`)
- **#956** — Apprise notifications now respect the configured proxy
(proxy env vars injected before dispatch)
- **#1025** — rTorrent: separate `RTORRENT_AUDIOBOOK_LABEL` setting,
falls back to book label if unset
- **#1010** — Stop button in Activity no longer makes the panel
disappear (snapshot refresh on cancel)
- **#1021** — Anna's Archive slow-download countdown now caps retries
instead of looping forever
- **#1040** — Empty destination directory cleaned up when write probe
fails
- **PR #1031** — Language detection from Anna's Archive distant path
when listing metadata is missing
### Additional fixes (no open issue but clear bugs)
- **fix: Python 2 `except` syntax across 27 files** — `except X, Y:` is
a SyntaxError in Python 3 and prevents affected modules from importing
at runtime. Mechanical sweep to `except (X, Y):`.
- **fix(abb): info hash validation with magnet fallback** — adds
SHA-1/SHA-256 hex validation on extracted info hashes; falls back to
scanning the full page for a magnet link (e.g. posted in comments) when
the table value is malformed. Also extends the exact-phrase fallback to
manual queries and defaults the ABB listing language to `en` when
missing, preventing valid results from being hidden by the language
filter. Includes a small test-fixture fix (`test(abb): use valid hex
info hashes in scraper test fixtures`) since the existing fixtures used
non-hex placeholders that the new validation correctly rejects.
- **fix: Anna's Archive title parser** — handles nested edition spans
and filters `lgli` catalog descriptor entries (e.g. "Book/Online Audio")
that were polluting search results.
### Deliberately not included
- LitFinder rebranding (UI strings, Apprise app ID, logo). The `fix:
three upstream bugs` commit (#999/#956/#1025) was cherry-picked with
Apprise app-id, description, and logo-URL strings reverted from
"LitFinder" back to "Shelfmark"; noted in the commit body.
- Features from the LitFinder fork (multi-variant title search,
multi-book flat-folder grouping, fuzzy text matching, "Leave in Place"
output handler, admin display name, custom-source plugin system). These
are larger behavior changes that each warrant their own focused review —
happy to send any of them separately if of interest.
- LitFinder-specific test environment and CI infrastructure.
### Verification
- Backend: **1879 passed**, 96 skipped (1 preexisting failure on
`seleniumbase`-dependent test in local venv; runs fine in the standard
Docker image with the `browser` extra)
- Lint, format, dead-code: all clean against this repo's existing
ruff/vulture config
- One follow-up cleanup commit (`style: ruff lint and format fixes for
ported commits`) brings the cherry-picked code into compliance with this
repo's ruff settings — no behavior changes there
### Etiquette / credit
Per-commit authorship preserved by cherry-pick. The only edits to the
original commits are:
- `fix: three upstream bugs` — Apprise rebrand strings reverted to
"Shelfmark" (noted in commit body, original author retained as
`Co-Authored-By` via cherry-pick)
- One follow-up `style:` commit for ruff config alignment
Big thanks to [@NemesisHubris](https://github.com/NemesisHubris) for the
original work in LitFinder; this PR exists to make sure these fixes
reach Shelfmark's wider user base. Happy to revise scope, split into
smaller PRs, or split off the Py2 cleanup separately if that's
preferable.
---------
Co-authored-by: NemesisHubris <155838970+NemesisHubris@users.noreply.github.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
Adds support for `content_type=combined` in URL search parameters,
letting users force combined-mode searches via a bookmarkable link
rather than relying on the last-used preference from localStorage.
The override is applied only in Universal mode and only when combined
mode is actually available (universal enabled, `show_combined_selector`
on, neither content type blocked by policy). Otherwise, it's silently
ignored, consistent with how `content_type=ebook`/`audiobook` already
behave outside Universal.
Existing `content_type=ebook`/`audiobook` URLs now also force combined
mode off, so the URL is authoritative regardless of prior preference.
Also adds a per-user `FORCE_COMBINED_SEARCH` setting that locks combined
mode on whenever it's available.
URL `content_type=ebook`/`audiobook` overrides are also superseded by
force-combined for the same reason: the search bar wouldn't let users
switch back, so honoring the URL param would leave them in a state they
couldn't escape from.
- Updated mirror selection
- Removed built-in mirror options, users must provide their own
configurations
- Set Universal search to default, added ability to disable direct
source
- Updated documentation
- Updated makefile
CaliBrain
Gavin McFallandClaude Fable 5.1
splitsec2
Vinicius Gabriel
Zoltán Szabó
oliver
Nicholas Velten
Lance Marks
jakesterpdxandClaude Fable 5
FlozeandCaliBrain
Guflly
Alex