Commit Graph
126 Commits
Author SHA1 Message Date
splitsec2andCaliBrain 8fd477bf50 fix(releases): apply the request policy to release listing, not only to download (#1506)
The request policy is applied when a user downloads
(`/api/releases/download`) or requests (`/api/requests`), but not when
they list releases. A user whose mode is `request_book` or `blocked` can
still search releases and list a torrent's files through the shared
Prowlarr and qBittorrent credentials. Only the UI hides the list.

This applies the same gate to `/api/releases`, `/api/releases/inspect`
and the release-source records route, with the same codes as the
download route (`policy_blocked`, `policy_requires_request`,
`required_mode`).

- `blocked` and `request_book` are refused. `request_release` is
allowed, because that mode lists releases so the user can pick one. The
UI opens the request dialog directly under `request_book` and never
calls `/api/releases` there.
- `/api/releases` resolves the mode per source for the content type. A
refused source is dropped from the search, so a user blocked on Prowlarr
no longer reaches Prowlarr even when other sources are allowed. It
answers 403 only when no source is left, or when the one source asked
for is refused.
- The records route doesn't know the content type before it fetches, so
it refuses only when every content type the source supports is refused.
- Admins, auth mode `none` and an instance with requests off bypass the
gate, as they do on the download route.

A user whose default is `blocked` now gets a 403 toast in the release
modal, where they used to get a list of "Unavailable" rows. I made no
frontend change.

Tests (14, new file) cover each route for blocked, request_book,
request_release, an admin, and auth mode none, the per-source drop, and
the records route. Ruff, basedpyright and the backend suite pass.

Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2026-10-20 23:38:40 -04:00
splitsec2 b05bbc8c58 fix(covers): key the cover cache by the image URL, not the caller-supplied id (#1505)
`/api/covers/<id>?url=...` cached the fetched image under the id in the
path, and the caller chooses that id. A request with another book's id
and a different URL put its image in that book's cache entry, and every
user then got it.

The key is now derived on the server from the image URL (a hash of it),
and the id in the path stays only so links keep their shape. The SSRF
checks and `fetch_and_cache` are unchanged, and `transform_cover_url`
links already carry the URL, so they keep working. The URL is decoded
first now, so a missing one is a 404 and bad encoding a 400, both before
any fetch.

I made the key the URL alone, not URL plus id. Either stops the
overwrite, and the URL alone lets one image be shared when the same
cover appears under two ids (the viewer redaction rewrites ids and still
hits the cache).

Entries cached under the old keys are not read any more, since there is
no telling which were overwritten. They age out of the LRU and the cache
warms once after the upgrade.

Tests: a request with another id can't replace the legitimate image, in
either order; the normal miss then hit; the same URL under two ids hits;
no URL gives 404; bad encoding gives 400 without a fetch; and a unit
test for the key. Ruff, basedpyright and the backend suite pass.
2026-10-20 23:35:42 -04:00
splitsec2 71cd030484 Merge commit from fork
GET /api/users/me/edit-context served every user-overridable key with the
admin's global value in globalValues and effective, so any user could read
the global destination paths, the admin's EMAIL_RECIPIENT, the Booklore
library and path ids and the global USER_NOTIFICATION_ROUTES table, whose
Apprise URLs embed tokens.

build_user_preferences_payload gains a for_self_service mode that replaces
a global value with None unless the user set the key themselves or the
field is a checkbox, number, select or multi-select switch the form needs
to lay itself out (BOOKLORE_LIBRARY_ID and BOOKLORE_PATH_ID are masked
despite being selects). The self-service route uses it; the admin routes
keep full data.
2026-10-20 23:35:02 -04:00
splitsec2 386fdb313c Merge commit from fork
GET /api/debug returns a bundle with application logs, process listings
and environment names, and GET /api/restart exits the process. Both are
registered only when DEBUG is on, but any logged-in user could call them.

Add an admin_required decorator built on login_required and use it for
both routes. Non-admin sessions get 403 "Admin access required";
authentication mode "none" keeps working as before.
2026-10-20 23:34:32 -04:00
splitsec2andCaliBrain 77b80eb31a fix(metadata): only an admin may use the provider's own account, its lists and shelves (#1501)
Refs #981.

Hardcover is signed in as one account, the instance owner's, and every
logged-in user shares it. A non-admin can see the owner's lists and
shelves in the List filter, browse one by asking for it, and add or
remove books and change read status from search results, which is what
#981 describes.

Now only an admin may. With authentication set to none the one user
keeps everything.

- `GET` targets and the batch lookup return no options for a non-admin,
so the buttons don't show and the account isn't asked.
- `PUT` targets answers 403.
- The List filter (`hardcover_list`) has no options for a non-admin, and
a request that asks for it has the field dropped before the search.
Providers name these fields in `account_search_fields`, and Hardcover
lists `hardcover_list`.
- Public things are unchanged: author, title and series suggestions, and
a pasted list URL in the search box.

That pasted URL is the one gap I know of. It resolves any list the
account can see, so a user who knows the slug of one of the owner's
private lists could still open it. Closing it means the provider has to
know who is asking, which is a bigger change than this one, so I left
it.

Not included from the issue: a toggle for letting users view lists and
limiting which lists they see. I kept this to the admin-only part so it
stays small, and I can do the toggle as a follow-up if you want it.

Tests cover the routes as an admin, a user and with no authentication
(targets read, batch and write, list options, a crafted list search, and
public fields still passing). Ruff, basedpyright and the backend suite
pass.

Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2026-10-20 23:29:05 -04:00
splitsec2 9ba5816b83 fix(requests): an older failed attempt no longer undoes a later delivery (#1493)
A request that was retried after a failure has two tasks in the status
built from download history. `sync_delivery_states_from_queue_status`
walks the status buckets in `QueueStatus` order and keeps the last one
it sees for a request. Error comes after complete, so a request that was
delivered by its retry is set back to error and, when no retry is left,
reopened as pending. The next pass can then fetch the book again.

When a request has more than one task, an attempt that is still running
now decides its state, else a completed one, else the failure. Ties keep
the old order, so a request with a single failed task still reopens as
before.

Tests: an earlier failure doesn't undo a later delivery, a running retry
outranks an earlier failure, and a lone failure still reopens. It
happened on my instance to a request whose first try failed and whose
retry succeeded. Ruff and the backend suite pass.
2026-10-20 23:21:16 -04:00
splitsec2 a1ac017487 feat(requests): let an admin reopen a rejected request (#1491)
Closes #1445, where you said you were open to it.

A rejected request was final. `update_request` won't move a terminal
status and the admin routes only fulfil or reject, so an admin who
changed their mind had to ask the user to request the book again, and
the user kept seeing the old rejected one.

`POST /api/admin/requests/<id>/reopen` moves a rejected request back to
pending and clears the review fields. What the user asked for (request
level, release, note) is kept. It refuses anything that isn't rejected,
and refuses when that user already has a pending request for the same
book.

Admins get a Reopen action on rejected cards in the activity sidebar and
in History. A rejected request has usually been cleared from the list by
the time anyone wants it back, so reopening also clears the dismissal,
the way the failed-download reopen does. Without that the pending
request stays hidden.

Tests cover the service (state change, what is kept, the refusals, a
non-string admin note) and the route, and the card model for the new
action. Ruff, tsc, oxlint, vitest and the backend suite pass. It has
been running on my instance since Oct 8.
2026-10-20 23:20:26 -04:00
splitsec2 88cc734db2 fix(activity): a reused release follows the new request, and stale rows of a delivered request are closed (#1496)
Two things that go wrong when a second request picks a release a first
request already used. A task id comes from the release, so the history
row is reused.

**The row stays with the first request.** Recording a download kept the
first request's id, notes and state. On my instance a card for one book
showed up under the request for a different book, with that request's
note on it. Recording a download now moves the row to the request that
queued it and restarts its queue time. A retry with no request keeps the
request the row had.

**Old rows of a delivered request show as failures forever.** The
startup sweep leaves an old request-linked row alone so its retry stays,
but one whose request already has its book (delivered, or another
download of it finished) is now closed as "Superseded by a completed
download" instead of showing as an interrupted failure for ever.

Also in the first commit: History cards used the saved cover URL as it
was, an external address the page's own-origin image policy blocks,
while running downloads use the cover proxy. They use it too.

The second commit is about restarts. After a restart nothing is in
flight, but a request in downloading, resolving or locating whose row
had moved to another request was stranded, because only queued was
reopened. Those states are reopened too.

Tests cover the row takeover and the retry case, the cover through the
proxy (and a missing cover), the superseded cases (delivered request,
another row finished, an undelivered request is left alone) and the
in-flight states with no row. Ruff and the backend suite pass.
2026-10-20 23:20:04 -04:00
splitsec2 06a67a7150 feat(download): a concurrency limit per source (#1495)
With one global limit, a few tasks that are only waiting hold every
slot. A Soulseek download sitting in a peer's queue is the example I
hit: three of those and a torrent that could start waits behind them.

`MAX_CONCURRENT_DOWNLOADS_PER_SOURCE` (0 keeps the single global limit,
which is the default). With it set, the coordinator starts a download
only when its source is below the limit and passes over tasks of a full
source without losing their place in the queue. `get_next` gets an
optional filter for this, and a cancelled task is still dropped when one
is given. The start-up stagger now applies only between downloads from
the same source, and the pool is sized so every source can reach its
limit.

The second commit: qBittorrent keeps its own queue of torrents waiting
for a free slot. The task polling one of those does no work, so it no
longer counts against its source's limit and the next task of that
source can start. The client does the pacing, as it does for its own
queue.

Tests cover the queue pass-over (order and priority kept, nothing
eligible, cancelled tasks), the coordinator (a full source doesn't block
the others, each source has its own limit, the global limit when unset)
and the client-queue case. Ruff and the backend suite pass.
2026-10-20 23:19:33 -04:00
splitsec2 3b5afb1a04 Merge commit from fork
PUT /api/users/me set a new password from the password field alone, so a live session could replace the password and lock the owner out. A builtin user now has to send current_password with it, checked against the stored hash. A wrong or missing one is a 400, and counts toward the same lockout as a failed login (a locked account gets a 429 without being checked), so the endpoint cannot be used to guess the password.

A password or current_password that is not a string is a 400. A password that was not a string raised a TypeError from len() and answered 500.

The account form asks for the current password when a new one is typed. Profile and settings changes without a password are unchanged.
2026-10-20 23:18:13 -04:00
CaliBrain 10710abd4e fix(download): send every file of a multi-file download to the browser (#1489)
Download to Browser only ever sent the first file of a multi-file
release
(audiobook chapters, packs): the folder output returned final_paths[0]
and
/api/localdownload served that one path.

The folder output now keeps every final path on the in-memory task
(download_paths), and /api/localdownload streams them as one
uncompressed
ZIP named after the book when there is more than one. The archive is
never
staged on disk or held in memory, and ZIP64 covers large audiobooks.
History still stores only the first path, so a re-download from history
after a dismiss or restart serves a single file, as before.

Also brings every linter back to green on all files:
- pre-commit: bump the ruff hook to v0.17.1 to match uv.lock;
ruff-format
  the Python examples in docs/dev and skills
- ruff: fix the remaining issues in skills/shelfmark/download_books.py;
  format base_handler.py
- basedpyright: Optional and attrs errors in databazeknih.py
- knip: drop 31 unused exports, reuse ColumnColorHint in colorMaps
- oxfmt 0.72: styles.css
- direct download: fix the "Welib, mirrors" typo and the test that still
  expected the pre-OceanofPDF message

Fixes #1467
2026-10-20 02:33:53 -04:00
amura11andClaude Sonnet 5.5 51f32b374d feat(packs): let users choose which files, books and formats to download (#1480)
## Summary
- **Skip files or books in a multi-book pack.** The pack review panel
gets a switch per file and a "Skip this book" switch per book, using the
shared `ToggleSwitch`. A partly skipped book shows e.g. "3 of 5 files"
in the Files column. A fully skipped book is greyed out.
- **Choose formats for a single book.** A new "Choose formats" header
toggle, next to the multi-book one, opens a format picker for a release
that has several book formats (EPUB, MOBI, PDF and so on). It has one
switch per format with file count and size. Releases with one format, or
that can't be inspected, download as before.
- **Skipped files are sent as `ignored_files`.** They are stored on the
task (including restart-safe retry) and removed before grouping and
transfer. This matters for packs because files missing from `book_plan`
would otherwise be filed by the heuristic fallback. Ignored entries
match by release-relative path, then path suffix, then unique basename.
- A pack that ends up with one book after skips keeps the titles you
reviewed instead of falling back to the searched title.

## Screenshots
<img width="1277" height="413" alt="Screenshot 2026-10-19 at 09-39-28
Shelfmark"
src="https://github.com/user-attachments/assets/c005ca4b-3dac-45dd-b2ff-d02fc0f9fcc9"
/>
<img width="1277" height="1210" alt="Screenshot 2026-10-19 at 09-40-23
Shelfmark"
src="https://github.com/user-attachments/assets/2145f3ba-e5fc-4635-9364-1790feb0c8b8"
/>
<img width="1277" height="748" alt="Screenshot 2026-10-19 at 09-39-03
Shelfmark"
src="https://github.com/user-attachments/assets/39023fac-70e3-46d8-af89-1160ebd6e415"
/>


## Test plan
- [x] `pytest` pack tests (new `drop_ignored_files` cases added)
- [x] `vitest`, `tsc`, `oxlint`, `oxfmt`, `ruff`
- [x] Manual check in the UI with a real multi-book pack

🤖 (mostly) Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 02:03:05 -04:00
CaliBrain 3c5cfcb77d fix/redact log secrets (#1488)
- fix(logging): mask tracker API keys in log output (#1486)
- fix(welib): follow slow_download links through the countdown (#1447)
2026-10-20 01:55:49 -04:00
Amalkrishna A S b38e4e4a00 feat(naming): add {AuthorSort} template variable (#1484)
Closes #1178.

Adds `{AuthorSort}`, the first author surname-first ("Arthur Conan
Doyle" → "Doyle, Arthur Conan"), for rename and organize templates,
alongside `{FirstAuthor}`.

- A value already in "Last, First" form (one word before the comma) is
kept as is; Jr/Sr/II/III/IV stay at the end.
- Known limits, same as calibre's default: the last word is treated as
the surname, so particles ("Le Guin") and surname-first names aren't
detected.
- Added to the settings help, env docs, the frontend variable
picker/preview, and tests.
2026-10-20 01:22:05 -04:00
HeihoffJandClaude Opus 5.5 756bdec19b fix(metadata): show the translation a search matched on Open Library and Hardcover (#1451)
Searching a translated title on Open Library or Hardcover returned the
original-language work. "Der Weg der Könige" came back as "The Way of
Kings" (2010), and that English title then went into the release search
and the download name. Both providers keep one record per work, titled
after the original, and list translations underneath it.

#483 already searches releases per language with Hardcover's localized
titles, which fixed the release side of #360 when the matching language
is selected. This PR fixes the search result itself, so the book a user
picks is the translation they searched for, and it covers Open Library,
the default provider, which has no titles_by_language.

- Open Library: request the matched edition (editions.*) and prefer its
title, ISBNs, cover, language and publication year when its title
differs from the work's.
- Hardcover: use the alternative title named in the hit's highlights
(alternative_titles, best index first), without the original's subtitle.
A hit whose title also matched keeps the work title.
- /api/releases: when the modal sends a title that differs from the book
record's, it becomes search_title. Before, Hardcover's search_title from
the original won, so picking "Kinder des Nebels" still searched "The
Final Empire" first.
- Release cache: keyed on title and language as well as the provider ID.
All translations share that ID, so opening "The Way of Kings" after "Der
Weg der Könige" showed the German releases until a reload.

Known limitations: neither provider has descriptions per edition, so the
description stays the work's (usually English). Hardcover's search hits
carry no year or language for alternative titles, so those stay the
work's; Open Library's do not. Google Books is unaffected, since it
already returns one volume per edition.

Tests: Open Library edition parsing, Hardcover alternative-title
matching, the /api/releases title override, and the release cache, each
written to fail first. The Python suite passes apart from the 9
test_entrypoint_permissions.py tests, which fail the same way on an
unmodified main on macOS. The frontend typecheck, lint, format check and
unit tests pass. Checked on a live instance with Open Library, Hardcover
and Google Books.

Refs #360, #483

Disclaimer:
This was developed with the help of AI but reviewed and tested as well
as deployed on my server by a real human.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 01:08:45 -04:00
splitsec2 f0f0d46381 fix(queue): first terminal status wins, so a late COMPLETE cannot overwrite a cancel (#1429)
Kept small on purpose so it's easy to review. I have a few more small
fixes like this from going back over recently merged PRs. Would you
rather get them as separate PRs like this one, or combined into one?

#1361 made `cancel_download` write under the queue lock, which stops a
cancel from overwriting a download that finished first. The other
direction is still open. The worker checks `cancel_flag.is_set()` and
then calls `update_status(COMPLETE)` a few lines later in
`_process_single_download`. If a cancel lands between those two lines
the task goes CANCELLED, then the worker writes COMPLETE over it.
`_apply_status_locked` allows terminal to terminal moves, so the
terminal hook also fires a second time and records two events for one
task.

This makes the first terminal status win: `_apply_status_locked` ignores
a different terminal status once the task already has one. Requeueing
from a terminal status still works because QUEUED isn't terminal.

I replaced the existing race test. It patched `queue.update_status`,
which `cancel_download` no longer calls, so it never exercised the path
it names, and its last assertion only looked at the final hook event.
The new tests call the queue directly: complete then cancel is refused,
cancel then complete stays cancelled with one hook event, error then
complete stays error, and a terminal task can still be requeued. The two
late-write tests fail on current `main`.

Ruff and basedpyright are clean. `tests/core` and `tests/download` pass
apart from 4 `seleniumbase` import failures in my environment that also
fail on `main`.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.
2026-10-20 01:00:21 -04:00
splitsec2 65dbf42a86 fix(release-search): report a ValueError from a source as that source's error (#1433)
Kept small on purpose so it's easy to review. This is the same kind of
small fix as the ones in #1429 to #1432, so see the question in #1429
about separate PRs versus combined. I'm sending them separately until I
hear otherwise.

In `search_source_releases` the `except ValueError` comes before the
branch that handles the operational errors, and `ValueError` is in that
tuple. A `ValueError` from `build_release_search_plan` or
`source.search()` is therefore reported as `Unknown source: <name>`
instead of `<name>: <message>`. The only `ValueError` meant for that
branch is the one `get_source` raises for a name that isn't registered,
so that call now has its own `try` and the rest goes back to the
operational errors branch.

The failure test already covered `OSError` and `RuntimeError`, so I
added `ValueError` to it, and that case fails on current `main`. Ruff,
basedpyright and `tests/core` pass.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.
2026-10-20 00:59:54 -04:00
splitsec2 93f58caf7a fix(debug): keep SHELFMARK_API_KEY_READONLY out of the debug bundle (#1432)
Kept small on purpose so it's easy to review.

`genDebug.sh` filters secrets out of `environment.txt` by matching
`_KEY=`, `_SECRET=`, `_PASSWORD=` and `_TOKEN=`, which only catches
names that end in those words. `SHELFMARK_API_KEY_READONLY` ends in
`_READONLY`, so its value lands in the file, and people attach these
bundles to issues. That key only reads counters, so the impact is small,
but any variable like `DB_PASSWORD_FILE` would leak the same way.

The pattern now matches the secret word anywhere in the variable name.
The new test runs the real filter line from the script against a made-up
environment, and it fails on the old pattern.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.
2026-10-20 00:59:10 -04:00
splitsec2 e8f6505bea fix(library): back off after a failed index instead of retrying for every result (#1430)
Kept small on purpose so it's easy to review.

`library_index._entries_for` retries `fetch_entries` on every call while
the index is failing. The lookup runs once per search result, so with
the Calibre path unreadable a page of 40 results makes 40 attempts and
logs 40 warnings. The entrypoint runs one gevent worker and the Calibre
read is synchronous sqlite, so each attempt also holds up every other
request.

After a failure it now answers from the stale cache (or an empty list)
for 60 seconds before trying again, and a successful index clears that.
The new tests cover 40 lookups against a failing library with a stale
cache (one failed attempt, one warning), a failing library with no cache
(tried once), and a retry once the 60 seconds have passed. The first and
second fail on current `main`.

I left a few other things in this module out to keep this one small:
re-indexing blocks the worker for as long as a big library takes (I
haven't measured how long), `test_connection` stores entries from the
unsaved form path under the shared cache key, and a title with a
collection word in it, like "Books of Blood", shows as "In a
collection". I can send any of those separately.

Ruff and basedpyright are clean and `tests/core` passes.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.
2026-10-20 00:58:49 -04:00
splitsec2 755f28a6a4 feat(api): add a read-only API key and a /api/stats endpoint (#1419)
This is the read-only API key from #1410, where you said to go ahead.

I run Shelfmark behind Homepage and wanted more on the dashboard tile
than up or down, without putting an admin API key in the dashboard's
config. This adds a second key, `SHELFMARK_API_KEY_READONLY`, that can
read one new endpoint.

`GET /api/stats` returns counts only: books added over the last 7 and 30
days by format, the queue, requests by outcome, and download failures
over the last 7 days. No titles and no user names. The read-only key,
the admin key or an admin session can read it.

The read-only key gets a 403 on every other path and on any write,
including a POST to `/api/stats` itself, and it never gets a session or
a cookie. If a request carries both keys, the admin one wins, so a
proxy's own `Authorization` header can't downgrade a correct
`X-Api-Key`. Unset means off, like the existing key. The endpoint exists
either way, but without a key only an admin session can read it.

It also adds a `db_path` property on `UserDB`, so the stats module can
open the database read-only.

Tests cover the key scope, the 403s for the read-only key, both keys at
once, who can reach the endpoint, and the counters. The full suite
passes, and I broke each rule on purpose to check a test catches it. A
version of this has been running on my own install behind Homepage.

If you'd rather have the stats in a different shape or under a different
path, I'm happy to change it.
2026-10-02 22:45:48 -04:00
adman234andClaude Opus 5.5 32abaff21e feat(naming): {Narrator} placeholder and MyAnonamouse series fallback (#1407)
## Human-Written explanation:

Following up on previous PR to use MAM ID to get nararrator and series
to show up in search, this PR will allow the series and nararrator
fields to be added to the path when saving an audiobook.

I have tested my ghcr.io image on my instance and it seemed to work
properly.

All the text below is written by Claude.

# feat(naming): {Narrator} placeholder and MyAnonamouse series fallback

Follow-up to #1390 and #1399. Related to #605 and #934 (narrator
support).

## Problem

Several narrations of the same audiobook currently land in the same
folder, because nothing in the path template tells them apart. Keeping
more than one version means renaming folders by hand after every
download.

Since #1390, MyAnonamouse results carry the release's narrator and
series in `extra`, but the naming templates can't use them: there is no
`{Narrator}` placeholder, and `{Series}` / `{SeriesPosition}` only come
from the metadata provider.

## Changes

- **`{Narrator}` placeholder** (`core/naming.py`,
`download/postprocess/transfer.py`, `core/models.py`): `DownloadTask`
gains `narrator`, read from `narrator` or `extra.narrator` when a
release is queued, and kept in the restart-safe retry payload. It's
available in audiobook templates like any other placeholder, including
prefix/suffix blocks such as `{ - Narrator}`.
- **Audiobookshelf folder style**: Audiobookshelf reads the narrator
from folder names like `Title {Narrator}`. The template `{Title}
{{Narrator}}` already matches as `{` + `{Narrator}`; the parser now also
drops the closing brace when the narrator is empty, so the folder is
`Title` rather than `Title }`.
- **No stray spaces around `/`**: an empty placeholder at the start or
end of a folder name no longer leaves a space there (e.g. `Title
/Title`). This applies to all templates.
- **Series fallback** (`release_sources/prowlarr/mam.py`, `source.py`,
`download/orchestrator.py`): MAM enrichment also stores the first
series' name and number as `extra.series_name` /
`extra.series_position`, which queueing already falls back to when the
metadata provider has no series. The provider's series still wins. A
release-level series number is only used when it names the same series
as the provider, so a provider series is never paired with the number of
a different MAM series (an omnibus, for example).
- **Settings and preview** (`config/settings.py`,
`namingTemplatePreview.ts`): the audiobook template descriptions list
`{Narrator}` and explain `{{Narrator}}`. The settings page preview
mirrors the parser changes and offers `{Narrator}` under "Insert
variable" for audiobook templates only.
- **Docs**: regenerated `environment-variables.md`. This also picked up
the Library Check settings, which weren't in the generated docs yet;
happy to drop that part to keep the diff focused.

## Example

Audiobook Path Template `{Author}/{Title} {{Narrator}}/{Title}`:

| Narrator | Result |
|----------|--------|
| Samuel Roukin | `Christopher Ruocchio/Empire of Silence {Samuel
Roukin}/Empire of Silence.m4b` |
| none | `Christopher Ruocchio/Empire of Silence/Empire of Silence.m4b`
|

## Testing

- `tests/core/test_narrator_template_variable.py` (new, 16 tests):
`{{Narrator}}` with and without a value, the `{ {Narrator}}` prefix
form, prefix/suffix blocks, sanitizing, `build_library_path`, task
metadata, retry payload round trip, MAM series name/number parsing
(including a `1-3` range having no number), and queueing (narrator from
`extra`, series fallback, no cross-series number, blank narrator).
- `namingTemplatePreview.test.ts`: the same `{{Narrator}}` cases for the
preview.
- Updated `test_generate_env_docs.py` for the new template description.
- Full `pytest -m "not integration and not e2e"` compared with an
upstream `main` worktree on the same machine: no new failures (the
remaining ones are Windows-only on both).
- `ruff check`, `ruff format --check`, `basedpyright` (0 errors) and
`vulture` on touched files; frontend `tsc --noEmit`, `oxlint`, `oxfmt
--check`, `vitest` (218 passed).
- Checked the settings page locally: the audiobook preview renders `...
{Simon Vance}.mp3` and `{Narrator}` is listed for audiobook templates
only.

No behavior change for existing templates, apart from spaces next to `/`
being trimmed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 22:45:10 -04:00
CaliBrain 119d3374e3 fix(downloads): reconcile downloads a restart interrupted (#1423) 2026-10-02 22:35:09 -04:00
splitsec2andCaliBrain 8392b43dd3 fix(downloads): reconcile downloads a restart interrupted (#1409)
I found this while grabbing new books and working on my fork. I
redeployed the container while downloads were running, and it left a
mess in my queue. It's an edge case, but a few rows were stuck until I
cleared them by hand.

The download queue lives in memory, so stopping the process with a
download in flight leaves its history row "active" and its request
"queued", with nothing working on either. The activity API shows that
row as "Interrupted" when it's read, but nothing is stored and nothing
looks at it again. Upstream only reopens an errored request when no
manual retry is offered, and an interrupted row always offers one, so
the request just sits there.

This closes those rows out once at startup, before the coordinator
starts, as an error with the message "Interrupted". The retry payload
stays, so the manual retry still works. A request whose download was
queued in the last 24 hours goes back to pending through
`reopen_failed_request`. Older ones get closed out but their requests
are left alone, because I didn't think a startup should reopen a backlog
or fill an admin's approval queue with old requests. A request that
already arrived is left alone, and running it twice changes nothing.

It assumes a single worker process, which `entrypoint.sh` already sets.

On my install the first start after deploying it closed 53 rows that had
been orphaned for nine days. Tests are in
`tests/core/test_startup_reconcile.py`, and the full suite passes
locally.

If the 24 hour window feels off, or you'd rather handle this another
way, I'm happy to change it or drop the PR.

Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2026-10-02 22:23:28 -04:00
CaliBrain bac0b93566 fix: keep https:// for qBittorrent and hide release sources from non-admins (#1422)
qBittorrent over HTTPS (#1417)

qbittorrent-api ignores the scheme in `host` and probes HTTP and HTTPS
itself. When the HTTPS probe fails, for instance on a certificate
Shelfmark doesn't trust, it falls back to plain HTTP against the TLS
port, and a reverse proxy answers "400 The plain HTTP request was sent
to HTTPS port". The download client and the Test Connection button now
pass FORCE_SCHEME_FROM_HOST for https:// URLs, so the configured scheme
is used as-is and a certificate problem is reported as one. http:// and
bare host:port URLs keep the probe: normalization adds http:// to a bare
host, and the probe follows a proxy's redirect to HTTPS.

Release sources leaked to non-admins (#1418)

A download queued from an approved request carries the release an admin
picked, and the requester could read it from their own activity feed:
the request's release_data (source_id, indexer, info URL, torrent
attributes), the download's full server path, and the download id
itself, which for Prowlarr is "<indexer id>:<guid>" and for a private
tracker is a URL into it. That id keys every status payload, the
/api/localdownload query and the cover proxy URL, so hiding
release_data alone would not have closed the leak.

For non-admin viewers:
- request rows keep only the release fields the activity cards display
- download_path is reduced to the file name, which the browser download
  reveals anyway
- request-linked downloads are addressed by an opaque id (a keyed HMAC
  of the task id) in the snapshot, history, /api/status, queue order,
  active downloads, dismissed keys, cover URLs and the websocket status
  and progress events. Routes that take a download id translate it
  back, searching only the caller's own downloads.

Downloads a user queued directly keep their real id: the user picked
that release from search results that already showed it, and the
release list matches its buttons to the queue by that id. Admin views
are unchanged. The activity sidebar now links a fulfilled request to
its download by request_id instead of release_data.source_id.

Closes #1417
Closes #1418
2026-10-02 20:38:58 -04:00
a05e002305 feat: narrator, series and bitrate columns for MyAnonamouse results (#1390)
# feat: narrator, series and bitrate columns for MyAnonamouse results

Addresses #605 and #934 (narrator in the release list).

## Problem

Both requests were closed because the narrator isn't in Torznab results,
which is correct. Prowlarr's MyAnonamouse indexer reads `author_info`
but drops the `narrator_info` and `series_info` MAM returns next to it,
and neither Prowlarr's `ReleaseInfo` nor the Torznab output has a field
for them. Shelfmark's size tooltip already looks for a `narrator`
Torznab attribute, but nothing ever sends one.

When a book has several narrations, choosing one means going back and
forth between Shelfmark and the tracker.

## Approach

Optional, opt-in enrichment using the user's own MAM session (`mam_id`),
the same approach AudioBookRequest's MAM indexer uses:

1. After the Prowlarr search, releases whose info URL is
`…myanonamouse.net/t/<id>` are collected.
2. Shelfmark sends the same query text to MAM's JSON search
(`/tor/js/loadSearchJSONbasic.php`, normally one request, at most 3),
and matches torrents back to Prowlarr's results by torrent ID. The MAM
origin is taken from the result URL, so a custom Prowlarr MAM base URL
is respected, and the cookie is only ever sent to `*.myanonamouse.net`.
3. Adds `extra.narrator`, `extra.series` (`The Sun Eater #1`) and
`extra.bitrate` / `extra.bitrate_value`. MAM has no bitrate field, so
it's parsed from the uploader's free-text tags (`64 kbps`) and some
releases won't have one.

Lookups are cached per torrent for an hour, respect the existing
Prowlarr search deadline, go through the configured proxy
(`get_proxies`), and never fail the search: a 403, timeout or bad JSON
is logged and the list renders without the extra details.

## Changes

- **`release_sources/prowlarr/mam.py`** (new): small MAM client
(`search`, and `get_username` for the test button), `narrator_info` /
`series_info` / tags parsing, cached best-effort
`lookup_torrent_details()`. A 403 includes MAM's reply and a note about
the IP/ASN lock.
- **`release_sources/prowlarr/source.py`**: enrichment after the result
loop. Series, Narrator and Bitrate columns only when a MAM ID is
configured, since otherwise they would be empty for every row. A Torznab
`bitrate` attribute from other indexers is also mapped to
`extra.bitrate`.
- **`release_sources/prowlarr/settings.py`**: "MyAnonamouse Enrichment"
section with `PROWLARR_MAM_ID` (password field, env-overridable like
every setting) and a **Test MAM Session** button.
- **`release_sources/__init__.py`**: `ColumnSchema` gains optional
`setting_key` and `content_types`. The new `apply_column_visibility()`
drops gated columns and their grid tracks. Neither field is serialized.
- **`main.py`**: `/api/releases` applies `apply_column_visibility()`
with the request's content type and the user's effective settings.
- **`config/settings.py` / `users_settings.py`**: Search Mode › "Release
List Columns" with `SHOW_SERIES_COLUMN`, `SHOW_NARRATOR_COLUMN` and
`SHOW_BITRATE_COLUMN`, all default on and user-overridable. Narrator and
bitrate are audiobook-only, series shows for both. AudiobookBay's
existing bitrate column now follows the bitrate toggle.
- **Frontend**: text cells truncate with a hover title; the mobile info
line wraps and skips empty text/number cells so blank optional columns
don't leave orphan `·` separators; the size tooltip no longer lists
Bitrate twice.
- **Docs**: new `docs/myanonamouse-enrichment.md` (linked from the
index), and a regenerated `environment-variables.md`. The regeneration
also picked up a few pre-existing drifts from `main` (the Libgen
section, `AA_DEFAULT_SORT` default, a duplicate `BOOK_LANGUAGE` row). I
can drop those if you'd rather keep this diff focused.

## ⚠️ MAM sessions are IP/ASN-locked

MyAnonamouse locks each session to one IP or ASN. Reusing the session
Prowlarr (or a seedbox script) uses will often **403**. **A separate MAM
session for Shelfmark will likely be needed** when Shelfmark reaches MAM
from a different IP (another host, a VPN container, or a proxy in
Shelfmark's Network settings), or when the existing session is
ASN-locked to another network. The setting's description, the error
message and the new doc all say so.

## Testing

- `tests/prowlarr/test_mam_enrichment.py` (new, 19 tests): parsing
(narrator dedupe, multiple series, missing numbers, malformed JSON, tag
bitrate), lookup (stops once all IDs are found, cache, 403 and
connection errors return empty, expired deadline skips the request),
only MAM releases enriched, Torznab bitrate mapping, column config with
and without a MAM ID for audiobook and ebook, toggles, grid-track
removal, and gates not serialized.
- `tests/core/test_admin_users_api.py`: the curated search-preference
key list now includes the three toggles.
- Full `pytest -m "not integration and not e2e"` compared with an
upstream `main` worktree on the same machine: no new failures. The
remaining ~115 failures on both are Windows-only (tor/entrypoint shell
tests, path separators).
- `ruff check` / `ruff format --check` / `basedpyright` (0 errors) /
`vulture` on touched files; frontend `tsc --noEmit`, `oxlint`, `oxfmt
--check`, `vitest` (201 passed).
- Manually verified with a real MAM account on a Docker build of this
branch: the test button, then narrator, series and bitrate on
MyAnonamouse audiobook results.

No behavior change unless `PROWLARR_MAM_ID` is set, apart from the
bitrate toggle on AudiobookBay (default on, same as today).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2026-09-25 19:17:23 -04:00
CaliBrain ece6b8f341 fix(auth): fail closed when auth prerequisites are missing (#1387) (#1397)
Auth mode resolution fell back to "none" (anonymous full admin) whenever
the configured method's prerequisites were missing: no local password
admin for builtin/OIDC, no Calibre-Web database, a blank proxy header,
an
unrecognized AUTH_METHOD (including "OIDC" in uppercase), or any error
while reading the config. Deleting or demoting the last local admin was
allowed on purpose because of that fallback, which exposed OIDC
instances publicly.

- Only an explicit AUTH_METHOD=none disables authentication. A
configured
  method stays active when its prerequisites are missing, so sign-in
  fails instead of opening up.
- An unrecognized or unreadable AUTH_METHOD resolves to "unavailable",
  which still requires a session and accepts no login. Values are
  normalized, so AUTH_METHOD=OIDC works.
- Restore the guard against deleting or demoting the last local password
  admin while builtin/OIDC is active (unless DISABLE_LOCAL_AUTH is set).
- Require a local admin before enabling Local auth, as OIDC already did.
- Log a recovery hint at startup when builtin/OIDC runs without a local
  admin, and document recovery via AUTH_METHOD=none.
- Drop the "will fall back to No Authentication" UI toasts and hints.



Fixes https://github.com/calibrain/shelfmark/issues/1387
2026-09-25 18:56:16 -04:00
splitsec2 37a77e9562 feat(library): mark search results already in a Calibre library (#1377)
Per discussion #1372, where you said you were fine with this specific
implementation: check whether metadata.db exists, read it if so, and
show a check mark saying the book is already there.

Searching for a book you already own gives no hint that you own it, so
the easiest way to end up with a second copy is to not remember you have
the first. This reads a Calibre `metadata.db`, read only, and marks
matching results with an **In library** badge in the card, list and
compact views and in the details dialog.

Off by default. It sits in Settings, General beside the existing Library
URL, with a test button that reports how many books it indexed. No HTTP
call, no token, nothing written back.

Matching runs most to least confident: a shared external id, then an
ISBN compared in both ISBN-10 and ISBN-13 form, then fuzzy title tokens
plus the author surname. The check fails open, so an unreadable database
degrades the badge and never blocks a search, and entries are cached for
ten minutes with an early refresh when the file changes, so a large
library costs one read rather than one per search.

`text_match.py` is new and shared by the index and the provider, so
title, author and ISBN matching stays consistent in one place.

## On the provider interface

`library_index` talks only to a `LibraryProvider` protocol and knows
nothing about Calibre. That is deliberate but it is not speculative
generality, it is what let me send you the Calibre half on its own: I
run an Audiobookshelf provider on the same interface in my fork, which
is where the audiobook side of the badge comes from. I have left that
out because it is a new service integration rather than something
already in the codebase, which is the line your non-goals draw. Happy to
send it separately if you ever want it, and equally happy for the answer
to be no.

Adding a library is a module with the `LibraryProvider` shape plus one
line in `all_providers()`.

## Verification

- `tests/core/test_library_index.py`: id, ISBN and fuzzy matching,
per-content-type provider selection, fail-open on provider errors,
stale-cache reuse, TTL and fingerprint refresh, per-provider cache
isolation, and the test-connection path including unsaved form values.
- `tests/core/test_text_match.py`: ISBN variants and token matching.
- `src/frontend/src/tests/libraryBadge.test.ts` and the added cases in
`bookTransformers.test.ts`.
- Python suite (3269) and frontend suite (206) green, plus ruff, ruff
format, basedpyright, vulture, tsc, oxlint, oxfmt and the production
build.
2026-09-25 18:18:10 -04:00
splitsec2 fe99d4bb5b feat(search): add a configurable default content type (#1371)
Closes #1018.

Worth correcting the issue first: the search tab is not hardcoded.
`useContentTypePreferences` has persisted the user's choice to
localStorage since #564, so a browser that has picked a tab already
keeps it. What is missing is the other half the issue asks for, a
default for a browser that has stored nothing, and a per-user override.

`DEFAULT_CONTENT_TYPE` is a user overridable select next to
`BOOK_LANGUAGE`, so it follows the same path: global value in Settings,
per-user value in Search Preferences, resolved with `user_id` in
`/api/config`. The frontend uses it only when this browser has no stored
choice, which is captured before the existing effect writes one, so
nothing changes for anyone who has already picked a tab.

The resolution is a pure function in `utils/contentTypePreference.ts`
rather than logic inside the hook, since vitest here has no jsdom and
the existing tests cover resolvers like `resolveDefaultLanguageCodes`
the same way.

One small move in `App.tsx`: the `config` state was declared below the
hook that now reads it, so it moved above it.

## Verification

- `tests/core/test_config_api.py`: the payload carries
`default_content_type` and reads it with the user's id.
- `tests/core/test_admin_users_api.py`: the key appears in the per-user
search preferences list.
- `src/frontend/src/tests/contentTypePreference.test.ts`: a stored tab
wins, combined mode survives, the server default applies when nothing is
stored, and an unrecognised value falls back to ebook.
- Python suite (3163) and frontend suite (201) green, plus ruff, ruff
format, basedpyright, vulture, tsc, oxlint, oxfmt and the production
build.
2026-09-25 18:14:42 -04:00
CaliBrain d978896142 fix(auth): rename the API_KEY env var to SHELFMARK_API_KEY (#1374) 2026-09-21 00:10:10 -04:00
Gavin McFallandClaude Fable 5.1 3b280009ae feat(auth): static API_KEY (env) accepted as Bearer or X-Api-Key, cookie or key (#1366)
Supersedes #1353, per the discussion in #1352: one `API_KEY` environment
variable; when set, a request carrying it is authenticated as the first
admin, and cookie sessions keep working exactly as before (cookie **or**
key). Nothing else changes. No table, no UI, no settings-tab switch, no
per-user keys.

## What

- `API_KEY` (env). Unset → the feature is off and none of the new code
runs.
- `Authorization: Bearer <key>` or `X-Api-Key: <key>` on any existing
`/api/*` route authenticates that request as the first admin in
`users.db` (`ORDER BY id`), or as a bare admin identity
(`user_id="api"`, `is_admin=True`, no local user row) if the install has
no admin yet. Per request only; nothing is persisted; the admin's role
is read live, so deleting or demoting that user takes effect on the next
request.
- Both headers are checked and either may match. That is what makes the
key usable behind a reverse proxy that injects its own `Authorization`
header (oauth2-proxy, Authelia, forwardAuth): send the key in
`X-Api-Key`.
- A credential that is **not** the key is ignored and the request
continues on the normal session path, so proxy-forwarded tokens are
unaffected. Without a valid session such a request gets the usual `401
{"error": "Unauthorized"}`, identical to a request with no credential,
so there is nothing to probe.

## How

- `shelfmark/config/env.py`: `API_KEY = os.getenv("API_KEY",
"").strip()`.
- `shelfmark/core/api_key.py`: `extract_api_key_candidates()` (Bearer
token if the scheme is Bearer, then `X-Api-Key`) and `matches_api_key()`
using `hmac.compare_digest` on bytes.
- `shelfmark/core/user_db.py`: `UserDB.get_first_admin()`.
- `shelfmark/main.py`: `api_key_auth_middleware` (`before_request`,
registered before `proxy_auth_middleware`, which early-returns for keyed
requests). Only `/api/` paths; `/api/health` and `/api/auth/*` exempt;
no-op when `API_KEY` is unset or the auth mode is `none`. On a match it
mirrors the proxy-auth pattern: `session.clear()` then populate
`user_id` / `is_admin` / `db_user_id` for this request, `permanent =
False`, `modified = False`, `g.api_key_auth = True`. An `after_request`
hook guarantees no `Set-Cookie` is written for a keyed request even if a
handler dirties the session.
- `docs/api-access.md` (new), the `API_KEY` entry in
`docs/environment-variables.md`, and a README link.

## Security

- Constant-time compare; the key is never logged or echoed.
- Keyed requests never mint or refresh a session cookie and ignore any
cookie sent with them (a non-admin cookie plus the key yields admin for
that request; the browser's own session is left untouched and usable).
- The mismatch path touches neither the session nor `g`, so a stray
bearer on a browser request can neither log the user out nor change how
their cookie is refreshed.
- Store errors during the admin lookup fail closed (`500 {"error":
"Authentication error"}`), never to anonymous.
- Verified against Flask's `save_session` / `should_set_cookie`
ordering, and under auth modes `none`, `builtin`, `proxy`.

## Tests

`tests/core/test_api_key_env.py` (36): extraction and matching;
first-admin lookup; middleware behaviour on a guarded route and an admin
route, with and without a user_db, `X-Api-Key`, both-headers
combinations, no `Set-Cookie` when a handler dirties the session,
incoming non-admin cookie ignored, browser cookie still usable after a
keyed request, security headers, store error → 500, mismatch → guard's
401 / cookie path / permanent cookie untouched, unset → off, exempt
paths and path probes, `none` and `proxy` modes, deleted and demoted
first admin, a keyed write passing the guard. Existing auth suites
unchanged. All CI gates green on the fork:
https://github.com/gavinmcfall/shelfmark/pull/2 (CI-only draft).

Also exercised against a running instance: 47 scripted checks including
150 concurrent requests, proxy-mode switching through the key, an
unset-key restart, and a log scan for the key.

## Naming

`API_KEY` as discussed. If you'd rather namespace it
(`SHELFMARK_API_KEY`) to avoid clashing with other tools' env vars in
shared compose files, it is a one-line change; say the word.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 00:00:42 -04:00
splitsec2 ab3aa9a8b0 fix(requests): reject non-object items in the batch endpoint (#1369)
`POST /api/requests/batch` checks that `requests` is a non-empty list
and then hands each element to the shared preparation helper, which
calls `.get()` on it. A bare string, number or null in the list raises
`AttributeError` and the caller gets a 500, while `POST /api/requests`
answers 400 with a message for the same mistake.

This validates the element type beside the existing list check. One bad
item rejects the whole batch rather than being reported per item, which
matches the endpoint's current contract: every other failure path
already aborts the batch with a single error body, as
`test_batch_create_requests_is_atomic` asserts. Responses for valid
payloads are unchanged.

## Verification

- `tests/core/test_request_routes_api.py`: a case per bad shape (int,
str, null, list), plus a mixed valid and invalid batch that asserts
nothing was created. All fail on current main with a 500 and pass here.
- Full suite (3168), ruff, ruff format, basedpyright, vulture green.
2026-09-20 23:03:31 -04:00
splitsec2 c1315a2b23 fix(download): check task ownership before serving queued files (#1357)
`/api/localdownload` resolves the file through the live queue and
returns it before checking who owns the task; the owner check only runs
on the download-history fallback, once the task has aged out of the
queue. Task ids are source ids, so two users who searched the same book
can end up with the same id.

This applies the same rule on the queue path, reusing the
`_task_owned_by_actor` helper the cancel/retry/priority routes already
use, so both paths answer a non-owner with the same 404. Admin behaviour
is unchanged.

The 404 matches what this endpoint's history path already returns for a
non-owner rather than the 403 `download_not_owned` the
cancel/retry/priority routes use, happy to switch it if you prefer
consistency with the siblings instead.

## Verification

- `tests/core/test_activity_routes_api.py`: the owner still receives
their queued file; a different user receives 404. The new case fails on
current main and passes here; the existing history-fallback test is
unchanged.
- Full suite (3148), ruff, ruff format, basedpyright, vulture green.
2026-09-20 12:56:23 -04:00
splitsec2 7934924678 fix(queue): don't stamp CANCELLED over a finished download (#1361)
`cancel_download` reads the task status under the queue lock, releases
it, and only then writes CANCELLED through `update_status`. A download
that finishes in that window has its COMPLETE overwritten. The queue and
the UI show the task as cancelled while the file is already on disk, and
the terminal hook fires for both statuses.

The check and the write now happen in a single lock hold. Because the
lock is non-reentrant and the terminal hook has to run after it is
released (the stall canceller depends on that), the lock-held part of
`update_status` moved into a small private helper that both paths share;
`update_status` is a thin wrapper over it. A cancel arriving once the
task is already terminal still returns `False`.

## Verification

-
`tests/core/test_queue.py::test_cancel_does_not_overwrite_a_download_that_finished_first`:
a worker thread completes the download while the cancel is in flight,
with the handover driven by events rather than sleeps. The task stays
complete. Fails on main, passes here.
- Full suite (3147), plus `tests/download/` and
`tests/core/test_download_api_guardrails.py`, ruff, ruff format,
basedpyright, vulture green.
2026-09-20 12:54:05 -04:00
splitsec2 a6204a318e fix(oidc): reject backslash paths in the return_to sanitizer (#1359)
The OIDC `return_to` sanitizer rejects values starting with `//` and
then relies on `urlsplit` to catch anything carrying a netloc. A value
such as `/\host` has no netloc, so it is stored in the session and used
as the post-login redirect target and browsers resolve the backslash as
a path separator, which lands the user outside the app after a
successful login.

`_normalize_return_to` now also rejects values whose path contains a
backslash. That matches the frontend sanitizer in `authRedirect.ts`,
which parses with `URL` and already discards those forms, so the two
ends agree again. The check covers the path only, so query and fragment
backslashes still round-trip, and it also catches the script-root case
where `/app/\host` strips to `/\host`.

## Verification

- New cases in `tests/core/test_oidc_routes.py` cover the rejected
forms, including under a script root, and confirm `/`, `/settings` and
`/search?q=x#frag` are unaffected. They fail on current main and pass
here.
- Full suite (3155), ruff, ruff format, basedpyright, vulture green.
2026-09-20 12:51:21 -04:00
splitsec2 545480c557 fix(download): default is_admin to False in the request policy guard (#1358)
`_resolve_policy_mode_for_current_user` reads `session.get("is_admin",
True)`, so a session carrying `user_id` but no `is_admin` key skips the
request policy entirely, while every other admin check in the codebase
defaults the key to `False`.

This uses the same default here. Every authenticated login path
(builtin, CWA, proxy, OIDC) writes `is_admin` into the session, and
`AUTH_METHOD=none` is already short-circuited one line earlier, so
sessions from those flows behave exactly as before.

## Verification

- `tests/core/test_request_routes_api.py::TestDownloadPolicyGuards`: a
session without `is_admin` now gets `policy_requires_request` and
nothing is queued. Fails on current main, passes here.
- Full suite (3147), ruff, ruff format, basedpyright, vulture green.
2026-09-20 12:50:53 -04:00
splitsec2 127dd82615 fix(users): apply user updates only after the payload validates (#1360)
`PUT /api/users/me` and `PUT /api/admin/users/<id>` write the new
password hash, and then the profile fields, before the rest of the
payload is checked. When the request is rejected further down as an
invalid role, an admin-only setting, an invalid settings value, the
route answers 400 with those writes already committed, so the caller
sees an error while the password has in fact changed.

Both routes now validate the whole payload before touching the database,
and the password hash is folded into the same `update_user` call as the
other fields so the field write is a single transaction. Error messages,
status codes and the order they are reported in are unchanged.

## Verification

- New tests in `tests/core/test_self_user_routes.py` and
`tests/core/test_admin_users_api.py` assert that a rejected update
leaves the password, profile fields and role as they were, plus a
positive case that a valid payload still applies all three. They fail on
current main and pass here.
- Full suite (3150), ruff, ruff format, basedpyright, vulture green.
2026-09-20 12:50:14 -04:00
splitsec2 acd59f7cbb feat(auth): provision proxy users as non-admin once an admin exists (#1356)
With `AUTH_METHOD=proxy` and no admin group configured, every user the
proxy authenticates for the first time is provisioned as an admin
(`is_admin = True` unless the user already exists in `users.db`). The
intent to never lock an instance out makes sense, but the effect is that
anyone the SSO gate lets through becomes an administrator. On an
instance shared with family or a small community that is a footgun; I
hit it when the first invited reader landed as an admin.

This keeps the guarantee and removes the footgun: the first account is
still provisioned as an admin while the instance has no admin at all,
and later first-time users follow a new `PROXY_AUTH_DEFAULT_ROLE`
setting (Security tab / env), default `user`. Known users keep their
stored role; the `PROXY_AUTH_ADMIN_GROUP_NAME` path is unchanged and
still takes precedence. I couldn't find a way with Cloudflare access to
pass this along.

Changes: `UserDB.has_admin()`, `_proxy_default_is_admin()` in the proxy
middleware, the new `SelectField` beside the other proxy settings, the
regenerated `docs/environment-variables.md` entry and a row in
`docs/reverse-proxy.md`.

Compatibility: the default moves from "everyone admin" to "first admin,
then users". Accounts already in `users.db` are unaffected; new SSO
users on an existing instance become regular users unless
`PROXY_AUTH_DEFAULT_ROLE=admin` is set. If you would rather ship this
purely opt-in I can flip the default to `admin`.

## Verification

- `tests/core/test_auth_api.py::TestProxyProvisioningRole`: first user
admin / second user not; `PROXY_AUTH_DEFAULT_ROLE=admin` restores the
old behaviour; an admin from another auth source counts as "an admin
exists"; a known user keeps their role whatever the default.
- Full suite (3094), ruff, ruff format, basedpyright, vulture green.
- Running on my own instance since 2026-09-19.
2026-09-19 23:27:13 -04:00
splitsec2andInfiniteAvenger 44f4e13cce refactor: extract the per-source release search out of /api/releases (#1355)
The `/api/releases` route carries an inner `_search_source_releases`
helper that builds the search plan for one source, logs the planned
query type, runs the search and turns
`SourceUnavailableError`/operational errors into an error message
instead of raising. Anything outside the route that wants to search one
source with exactly those semantics has to go through Flask today.

This moves that helper into `shelfmark/core/release_search.py` as
`search_source_releases()` and has the route delegate to it. Behaviour
is unchanged: same plan construction (including the caller's `user_id`,
so per-user default languages still apply), same logging, same
error-to-message handling.

It is the refactor half of #1047 by @InfiniteAvenger, split out on its
own as you asked for other PRs (#1318). Their authorship is preserved on
the commit; I rebased it onto current `main` and added tests.

## Verification

- `tests/core/test_release_search.py`: unknown source → `"Unknown
source: …"`, `SourceUnavailableError` and operational errors →
`"<source>: <error>"`, success path forwards `expand_search` /
`content_type` and returns the source instance, the plan receives
languages / manual query / indexers / `user_id`. (These tests are
type-annotated; happy to strip the annotations if you prefer the suite's
bare style.)
- Full suite, ruff, ruff format, basedpyright, vulture green; the
existing `/api/releases` route tests are unchanged and pass.

Co-authored-by: InfiniteAvenger <calebewest02@gmail.com>
2026-09-19 23:16:53 -04:00
CaliBrain e5dd34ae0e fix: unbreak main and follow up on the Blackhole handoff review (#1346)
DownloadHistoryService.record_download and updated the single production
caller, but not the eleven in the test suite, leaving main red with 32
failures. Pass None, which is what the pre-#1336 behaviour recorded.

For the Blackhole handoff (#1345): add_download publishes the torrent
before the cancel check runs, and BlackholeClient.remove() is a no-op,
so
the watcher picks the file up regardless. Reporting a bare "Cancelled"
hid
that from the user. Name the completed handoff in the cancellation
message
instead, drop the _handle_cancelled_download call whose usenet branch
cannot apply to a handoff-only client, and record why the orchestrator
no
longer verifies HandoffResult.path.

Finally, make tests/direct_download a package: test_libgen_extract.py
imports tests.libgen.sample_html across test directories, so without an
__init__.py pytest named its modules by bare basename and a same-named
module elsewhere would collide.
2026-09-17 16:27:53 -04:00
Vinicius Gabriel c53545d9fe Add configurable word separator for naming templates (#1333)
Closes #1230

## What
Adds a "Word Separator" setting (Space / Dot / Underscore / Hyphen /
Custom) that replaces internal whitespace in each naming-template
placeholder's rendered value — e.g. `{Author}` renders
"Arthur.Conan.Doyle" instead of "Arthur Conan Doyle" when Dot is
selected.

This follows option 2 from the issue rather than inventing new
dotted-keyword template syntax (`{Author.}`), since it's a smaller
surface: one setting applies uniformly across all four templates
(books/audiobooks × rename/organize) instead of needing a parallel token
for every existing one.

## How it works
- Literal characters typed into the template itself (e.g. the `.` in
`{Author}.-.{Title}`) are never touched — only whitespace *inside* a
placeholder's resolved value is affected.
- Default is "Space", which is a no-op: existing templates produce
byte-identical output after this change (verified via the existing test
suite, unmodified, still passing).

## Where
- `shelfmark/core/naming.py` — `word_separator` param on
`parse_naming_template` / `build_library_path`.
- `shelfmark/download/postprocess/policy.py` — `get_word_separator()`,
mirroring the existing `get_file_organization()` accessor.
- `shelfmark/download/postprocess/transfer.py` — wires the resolved
separator through the four existing template-rendering call sites.
- `shelfmark/config/settings.py` — new `Word Separator` / `Custom Word
Separator` fields next to the existing naming-template fields.
- `src/frontend/.../namingTemplatePreview.ts` +
`NamingTemplateField.tsx` — the settings UI has its own TS mirror of the
Python renderer for the live preview; updated it in lockstep so the
preview doesn't lie about what the separator will actually do.
- Tests added on both sides (pytest + vitest).

## Testing
- `uv run pytest tests/core/test_naming.py
tests/core/test_destination_file_organization.py` — all pass, including
new cases.
- `uv run pytest` (full suite) — same pre-existing failures as on `main`
before this change (browser/network-dependent bypass & e2e tests
unrelated to this diff), everything else green.
- `uv run ruff check` / `ruff format --check` / `basedpyright` — clean.
- `npm run lint` / `format:check` / `typecheck` / `test:unit` (196
tests) — clean.
2026-09-17 15:52:21 -04:00
Zoltán Szabó 35037b35fd fix(irc): search by surname, and rank the answer by author (#1331) (#1332)
Fixes #1331.

A search bot ANDs every term against a filename, so the given name is
the term
that empties the result set. Measured against irchighway's #ebooks:
"Revelations
David Petrie" is answered "no results", "Revelations Petrie" returns 9
matches,
6 of which parse, all filed as "D Petrie".

The query now carries the title and the surname, read off the search
variant so
the ISBN fallback and a manual query - which set author="" on purpose -
keep
their current shape.

Title-only, the shape #1295 settled on for Prowlarr, does not transfer:
the bot
caps an answer at 1000 matches, and a bare "Revelations" hits that cap
with 923
parsed rows across 500 authors, so the cap itself can drop the wanted
book. The
surname is the token the two spellings share and it keeps the answer
small.

The full author then orders what comes back, reusing author_affinity
from #1295,
since a surname also matches a different author who shares it. It sits
under
server availability the way indexer priority does in #1295: a download
addresses
one named bot and waits 120s for it, so a match from a bot that has left
the
channel must not outrank a mismatch that can answer. Ranking runs on the
way out
rather than before the cache, because one query identity is shared by
every book
that produced that query.

Two things found while testing:

- The parser writes the literal "Unknown" when a filename has no " - "
separator
  (parser.py:168). Ranked literally that sorts as a wrong author, so
author_affinity's middle tier was unreachable here; it is now read as
absent.
  5 of those 923 rows are affected.
- author_affinity moves to shelfmark/core/author_match.py, unchanged, so
IRC
does not import from the Prowlarr package. Prowlarr behaviour is
untouched and
  its tests pass as they are.

The three IRC assertions in the #1252 regression file move to the
surname form.
The invariant they pin - one contributor's name reaches the query, never
the
whole credit list - is unchanged.

Tested with make python-lint, python-format, python-dead-code,
python-typecheck
and python-test, and end to end against irchighway with the patched
source: it
posts "Revelations Petrie" and returns 6 releases.
2026-09-11 22:14:37 -04:00
oliver 1e3fd48b8b fix: share rotating log file handlers (#1316)
This patch shares (for each log file) the `RotatingFileHandler` for
logging across all modules, reducing the number of open file descriptors
from ~78 to 1.

I had originally assumed this issue was a resource leak, but it seems to
just be a large fixed number of file descriptors. So this change mostly
just (1) shrinks the number of open file descriptors to a reasonable
level and (2) prevents two modules in the same process competing to
write to a log file.
2026-09-11 00:48:22 -04:00
Vinicius GabrielandClaude Sonnet 5 c576003319 feat(naming): add {FirstAuthor} template token (#1322)
Closes #930.

## What

New `{FirstAuthor}` naming-template token. It renders only the first
author when metadata lists several ("Author1, Author2, Author3"), so
multi-author books can be filed alongside the rest of that author's work
instead of getting their own "Author1, Author2, ..." folder.

```
{Author}       -> Terry Pratchett, Neil Gaiman
{FirstAuthor}  -> Terry Pratchett
```

## How

- Added to `KNOWN_TOKENS` in `shelfmark/core/naming.py`, positioned
before `author` so `{FirstAuthor}` isn't parsed as literal `First` +
`{Author}`.
- Derived inside `parse_naming_template` from the existing `Author`
value (split on `,` / `;`), so every caller — folder transfer, rename,
the settings preview — picks it up with no extra wiring. An explicit
`FirstAuthor` key in the metadata still wins if one is ever passed.
- `{Author}` behaviour is unchanged.
- Frontend `namingTemplatePreview.ts` token list + `KNOWN_TOKENS` kept
in lockstep (there's a test enforcing that), with a matching
`firstAuthor` helper.
- Settings field descriptions + `docs/environment-variables.md` list the
new token.

## Known limitation

A lone author written `Last, First` is split on the comma too and
renders as `Last` — the source metadata doesn't mark which form it is.
Called out in the token help text and covered by a test. `{Author}`
remains available for anyone who wants the raw string.

## Checks

- `make python-test` — 2963 passed
- `make python-lint` / `make python-format` / `make python-typecheck` /
vulture — clean
- `make frontend-test` — 187 passed · `frontend-lint` /
`frontend-format` / `frontend-typecheck` — clean

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-11 00:47:37 -04:00
CaliBrain 3d7ea40088 fix(search): reach the server's deadline, query one author (#1285, #1252) (#1287)
Two independent reasons a working search reported failure to the user.

1. The client gave up before the server did (#1285)

`/api/releases` bounds one release search with RELEASE_SEARCH_TIMEOUT
(default
300s) and answers a spent budget with a sentence naming the real cause -
the
machinery added for #1276. The frontend then aborted the direct_download
search
at a hard-coded 180s, so it always won the race: the user saw "Request
timed
out. Check your network connection or proxy configuration." instead, and
raising RELEASE_SEARCH_TIMEOUT changed nothing they could observe, the
180s
being baked into the hashed bundle inside the image.

- /api/config reports the effective (clamped) budget, and the client
derives its
  abort from it plus a margin, so the server always answers first.
- Direct-mode search shows what the server actually said. Every non-auth
failure
was relabelled "Unable to reach download source. Network may be
restricted or
mirrors blocked.", which discarded the explanation and blamed the user's
network. ApiResponseError now carries `serverMessage`, set only when the
server
  explained itself, so the status-line placeholder still falls back.

Two latency fixes for the cost that made the timeout reachable at all:

- Fetch each distinct AA search URL once per search. The language-filter
retry
re-runs every title variant, and with DIRECT_DOWNLOAD_LANGUAGE_FROM_PATH
on
both passes build a byte-identical URL - behind DDoS-Guard each repeat
is a
  fresh browser solve.
- Drop the solve-only bypass method. `_bypass_method_cdp_gui_click`
opens with
exactly that call and returns the moment it works, so the entry ahead of
it
could only repeat the half that had already failed, plus the backoff
before
the method that does work started. Reported at 0/19 successes and ~5.5s
of
  each ~26s solve against DDoS-Guard.

2. The query carried every contributor, not one author (#1252)

`_pick_search_author` returned `book.search_author` verbatim while the
authors[]
fallback beside it deliberately narrowed to the first name before a
comma. Both
fields routinely arrive holding every contributor joined with ", ": the
frontend
builds `book.author` as `authors.join(', ')` for display
(bookTransformers.ts)
and the release modal sends that display string straight back as the
`author`
parameter, and `browse_record_to_book_metadata` and the manual-search
branch
both split the joined text into `authors` while still passing the
unsplit string
as `search_author`, so the split was never used.

A book whose metadata lists translators was therefore searched for as

    Blindness Jose Saramago, Giovanni Pontiero, <persian translator>

which matches nothing on Anna's Archive. The bypass succeeds, the search
comes
back empty, and the user is told the book has no releases.

Narrowed in one place, `search_plan.first_author`, so the two branches
cannot
drift apart again, and applied to the IRC source, which built its query
with the
same verbatim preference. Hardcover is unaffected: it already sets
`search_author` from `_simplify_author_for_search(authors[0])`, which
resolves
"Last, First" itself and never yields a multi-author string.
2026-09-01 12:38:57 -04:00
Nicholas Velten 69ff0d6a78 fix: trim a credit list in search_author to the first name (#1290)
Fixes #1252 for the case in the second report.

`_pick_search_author` returns `search_author` untouched but trims
`authors[0]` to its first comma-separated name. So the same credit list
searches differently depending on which field carries it:

```
via authors[0]     -> "Blindness Jose Saramago"
via search_author  -> "Blindness Jose Saramago, Giovanni Pontiero, Zohreh Eftekhari"
```

Anna's Archive answers the second one with nothing. That is the query in
@theDoz12's log, and it explains the shape of the report: the bypass
succeeds, the search runs, and the UI still says no releases. Nothing in
the download path is broken, the query simply cannot match.

Measured against live AA on 1.3.14, same book, same source, only the
field carrying the author changed:

| query | releases |
| --- | --- |
| `Blindness Jose Saramago, Giovanni Pontiero, Zohreh Eftekhari` | 0 |
| `Blindness Jose Saramago` | 49 |
| `Blindness` | 50 |

With the patch the second form is produced from either field, and the
same search returns 49.

Three regression tests added, including one that asserts both fields
yield the same query. On `tests/core/test_search_plan.py` the run goes
from 5 failures to 3; the 3 that remain are the language tests, which
fail identically with and without this change on my machine.

Worth saying what this does not cover: the first report in that issue
ends with `Found 2 releases via ISBN` and still shows nothing, so that
one is a different fault further along. I could not reproduce it here.
2026-09-01 11:08:39 -04:00
CaliBrain 97e289ae13 fix: search, Prowlarr and qBittorrent follow-ups (#1276, #1283) (#1284) 2026-08-30 03:09:13 -04:00
Lance Marks f441b85da2 feat(packs): inspect multi-book releases and file each book separately (#1270)
## Multi-book packs: inspect a release before download and file each
book separately

Closes #576

### Problem

One queued release is always treated as one book. When a torrent is
actually a whole series
(`Series/Book 1 - Title/…`, or a flat folder of `Series 1.0 - Title.m4b`
files), post-processing
walks the whole tree, flattens every file into one list and renames them
`Title - 01…10` under the
searched book's `{Author}/{Title}`. Audiobookshelf then sees a single
10-file "book" and the user
has to re-file everything by hand.

### What this does

Most releases expose their file list *before* anything is downloaded, so
the split is decided up
front and approved by the user, then the download is fire-and-forget:

1. **Inspect** – clicking a release's download button now calls `POST
/api/releases/inspect`
first. A new optional `DownloadHandler.list_files(release_data)` hook
returns the release's
   files without downloading:
- **AudiobookBay** reads the torrent file table off the detail page it
already fetches (the
page is now cached for 120 s, so inspect + download cost ABB one
request).
- **Prowlarr** parses `info.files` from the `.torrent` it already
fetches (the existing 120 s
torrent-fetch cache is reused). Magnet-only and usenet releases report
"can't inspect".
   - Other sources default to `None`.
2. **Review** – if the plan contains more than one book, the Find
Releases modal swaps the list
for a review panel: one row per book with editable title / series
position / year, expandable
file lists, non-book sidecars (`.txt`, covers) shown as ignored, a
"Treat as a single book"
switch, and **Download N books**. Single-book releases queue
immediately, exactly as before.
3. **File** – the approved plan travels with the task
(`DownloadTask.book_plan`, retry-safe) and
post-processing files each book through the existing transfer code, one
book at a time
(`dataclasses.replace(task, title=…, series_position=…, year=…)`), so
organize/rename
templates, part numbering (now scoped per book), hardlinks, torrent
copy-preserve and usenet
   handling are unchanged. Status reads `Complete (N books, M files)`.
4. **Fallback** – when a release can't be inspected the user gets a
toast, and a small
"Multi-book pack" toggle in the modal header forces a heuristic split
(subfolder = book, or
   one book per file when the file names carry series positions).

Planning lives in `shelfmark/download/postprocess/packs.py` and is
shared by the inspect endpoint
and post-processing, so what the user approved is what gets filed. The
name parser strips
`Book 3 -`, `03 -`, `1.0 -`, `3.`, `[03]`, `#3`, a leading series name,
labels like
"An Expanse Novella -", repeated titles (`Gods of Risk 2.5 - Gods of
Risk`) and a trailing
`(Year)`; author and series name come from the book that was searched,
and the searched book's
own series position is never applied to its siblings.

### Files

- `shelfmark/download/postprocess/packs.py` (new) –
`PackFile/PackBook/PackPlan`, `plan_pack`,
`parse_pack_book_name`, `group_files_into_books`, `match_plan_to_files`
- `shelfmark/core/release_inspect_routes.py` (new) – `POST
/api/releases/inspect`
- `shelfmark/release_sources/__init__.py` – `DownloadHandler.list_files`
hook
- `shelfmark/release_sources/audiobookbay/{scraper,handler}.py` –
detail-page cache,
  `extract_file_list`, `list_files`
- `shelfmark/release_sources/prowlarr/handler.py`,
`download/clients/torrent_utils.py` –
  `extract_file_list_from_torrent`, `list_files`
- `shelfmark/core/models.py`, `download/orchestrator.py` – `multi_book`
/ `book_plan` fields,
  queue + retry serialization
- `shelfmark/download/postprocess/transfer.py`, `pipeline.py`,
`outputs/folder.py` – per-book
  transfer branch and status message
- `src/frontend`: `components/PackReviewPanel.tsx` (new),
`ReleaseModal.tsx`, `App.tsx`,
`services/api.ts`, `types/index.ts`, `utils/releasePayload.ts` (payload
builder moved out of
  `App.tsx`), `utils/packReview.ts`
- `docs/dev/release-sources-plugin-guide.md` – documents the
`list_files` hook

### Out of scope (follow-ups)

- Listing files from an NZB (Shelfmark already fetches the bytes; `<file
subject>` names are noisy)
- Inspecting magnet links via qBittorrent's files API after a paused add
- BookLore / email outputs (they ignore `book_plan`; noted in code)
- The combined ebook + audiobook flow

### Testing

**Automated** (`make checks`, `make python-test`, `make frontend-test`
all green; the only
failures on my machine are the pre-existing
`tests/config/test_entrypoint_permissions.py` cases,
which need bash ≥ 4 and fail identically on `main` under macOS bash
3.2):

- `tests/download/test_packs.py` – name parsing (markers, series name,
novella labels, repeated
titles, bare numeric titles like `1984`), nested / flat / mixed /
deeper-nested packs, single
wrapping folder not treated as a pack, plan-to-disk matching with
basename fallback
- `tests/core/test_processing_packs.py` – full `post_process_download`
runs on a real temp
filesystem: approved plan files each book under its own
`{Author}/{Title}`, heuristic split
of a nested pack, searched book's series position does not leak,
multi-file book inside a pack
keeps `- 01/- 02` per book, hardlinked torrent pack leaves the seeding
tree intact, no pack
fields ⇒ behaviour unchanged, single group degrades to the searched
title, status message
- `tests/core/test_release_inspect_routes.py` – plan response,
not-inspectable, handler errors
  never 500, unknown source / missing `source_id` ⇒ 400, login required
- `tests/audiobookbay/test_file_list.py` – file-table scraping from real
ABB markup (multi-file
and single-file pages), handler host validation, one page fetch shared
by magnet + file list
- `tests/prowlarr/test_torrent_file_list.py` – multi-file / single-file
`.torrent` parsing,
  handler behaviour for torrent URL vs magnet vs usenet vs cache miss
- `tests/download/test_orchestrator_pack_fields.py` – queue-time parsing
and retry round-trip
- Frontend: `releasePayload.test.ts`, `packReview.test.ts` (vitest)

**Manual, on a real deployment** (arm64 image built from this branch,
run as a side container
next to production with the same qBittorrent / Audiobookshelf setup,
`FILE_ORGANIZATION_AUDIOBOOK=organize`,
hardlinks on):

- AudiobookBay "The Expanse Complete 2.0" (7.87 GB, 36 files): clicking
download opened the review
panel in ~1 s showing **18 books · 18 files · 18 files ignored** (the
`.txt` sidecars), with
series positions 0.1–9.5 and years parsed from the file names; novella
labels stripped
("The Churn", "The Butcher of Anderson Station"). Editing a title in the
panel works.
Confirming queued one task; the magnet resolved from the cached page in
~30 ms; after the
download the task reported `Complete (18 books, 18 files)`, 18 hardlinks
landed as
`audiobooks/James S. A. Corey/<Title>/<Title>.m4b`, the torrent kept
seeding, and
Audiobookshelf scanned each folder as its own book (title, author,
embedded chapters).
- A second pack ("Expanse [01 - 9.5]", `Title N - Title` naming) was
inspected to verify the
  repeated-title rule and the Back button, without downloading.
- Single-book releases still queue immediately with no extra UI.
2026-08-27 00:40:08 -04:00
CaliBrain ff06a1a581 fix(search): follow-ups to per-user book languages (#1267)
Review follow-ups to #1255, all in the code that PR touched.

Drop the dead user_id from the Prowlarr retry path.
ProwlarrSource.search
never reads plan.languages, and _refresh_release builds a synthetic book
with no titles_by_language, so the title variants came out identical
with
and without it. It also should not language-filter: it re-finds one
exact
release by its guid.

Pin the tab move in tests. BOOK_LANGUAGE moved from the General tab to
Search Mode with no migration, which only works because both tabs
persist
into the same settings.json. Nothing asserted that, so splitting the
files
later would silently reset every install to ["en"]. Covers the stored
value, a fresh install, and ENV precedence.

Stop the UI inventing a default language. An empty BOOK_LANGUAGE is a
deliberate "no default filter" that the backend preserves, but the two
frontend call sites replaced it with the first supported language, so
the
filter said English where the server filtered nothing.
resolveDefaultLanguageCodes
now falls back only when the value is absent.

Keep the normalized value for every validated search key.
validate_user_settings
gated the write-back on a hand-maintained subset of the keys the search
validator recognises, so METADATA_PROVIDER_COMBINED,
SHOW_COMBINED_SELECTOR
and FORCE_COMBINED_SEARCH were validated and then stored raw -- a padded
provider name was accepted and persisted with its padding. Reuse the
validator's own key set instead.

Skip blank language entries rather than rejecting them, so "" and "en,"
mean the same as [] and ["en"] instead of erroring on an unnamed
language.

Extract resolveListOverride for the list-override detection that was
copy-pasted between the two user-settings sections, and mention
languages
in the Search Preferences section description.
2026-08-24 18:11:29 -04:00
463ef49ac3 feat(search): let each user pick their own default book languages (#1255)
## Why

`BOOK_LANGUAGE` is a per-reader property, not a per-instance one. On a
shared install one household member searches in German while another
wants English and German — today whoever changes the setting changes it
for everyone, and the only escape is re-picking languages in the filter
on every single search.

The per-user override machinery already carries `SEARCH_MODE`, the
metadata providers and the default release sources, so the language
default mostly had to opt into it.

## What changed

**The field.** `BOOK_LANGUAGE` becomes `user_overridable` and moves from
the **General** tab to **Search Mode**, next to the other
user-overridable search defaults (per
[review](https://github.com/calibrain/shelfmark/pull/1255#issuecomment-5391189094)
— the first version had the Search section span two tabs, this one
doesn't). Admins set it per user in the user editor, users set it in
**My Account → Search Preferences**, and the Search Mode tab carries the
usual "N users override this" summary.

**No migration for the move.** `general` and `search_mode` both persist
into `settings.json`, and a field's value is resolved through
`load_config_file(tab)` for the tab it's declared on — so an install
that already stores `BOOK_LANGUAGE` keeps its value. Checked against a
`settings.json` written while the field still lived on General: the
stored value resolves unchanged, a fresh install still gets `["en"]`,
and `BOOK_LANGUAGE` in the environment still overrides both.

**The two places the default is read.**

- `/api/config` seeds the frontend's language filter, so it now resolves
`BOOK_LANGUAGE` for the session user.
- `build_release_search_plan` falls back to the default whenever a
request carries no language filter — which is exactly what the filter's
"Default" option sends. It takes an optional `user_id`, passed by
`/api/releases` from the session and by the Prowlarr retry path from
`task.user_id`, so a retry re-searches in the languages of whoever
queued the download.

**Validation.** Overrides go through `normalize_language()`, so
`"German"`, `"ger"` and `"de"` all store as `de`, and an unknown
language is rejected with a message naming it instead of being silently
searched for. An empty list stays an empty list (a deliberate "no
default filter"), `null` clears the override as everywhere else, and ENV
still wins: with `BOOK_LANGUAGE` set in the environment the field
reports `fromEnv` and overrides are ignored.

**Scope.** Only the language default becomes overridable. The two format
lists left behind under "Default Search Filters" stay admin-only — they
describe what the library and its post-processing accept, not what a
reader wants to read. There's a test pinning that.

## Verification

- 2681 unit tests pass (2670 before, 11 added)
- `ruff check`, `ruff format`, `basedpyright` over backend and tests,
and `vulture` all clean; frontend lint, format, typecheck and 126 unit
tests clean
- `docs/environment-variables.md` regenerated via
`scripts/generate_env_docs.py` (the `BOOK_LANGUAGE` row follows the
field into the Search Mode section)
- Manually against a two-user instance with builtin auth (first round,
before the tab move): with user A on German and user B on
English+German, `/api/config` returns each reader their own
`default_language` and an unfiltered `/api/releases` plans the matching
languages; an admin can set and read the same override for another user;
clearing it falls back to the global value; a stray `"klingon"` is
rejected; and `BOOK_LANGUAGE` in the environment overrides both users
with the field marked `fromEnv`
- After the tab move I re-ran the suites above plus the
stored-value/fresh-install/ENV check described under "No migration for
the move"; the behaviour it exercises is what the move could have broken

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2026-08-24 17:57:21 -04:00
jakesterpdxandClaude Fable 5 65e2e3be20 feat(audiobook): recognise .mp4 as an audiobook format (#1264)
## Problem

Some trackers — MyAnonamouse in particular — distribute AAC audiobooks
as per-chapter `.mp4` files. That's the same ISO-BMFF container as
`.m4a`/`.m4b`, just with the generic extension (`ftyp isom`,
audio-only).

Today those releases:
1. show up in Prowlarr search results with **no format chip** — only the
generic "Audiobook" icon, because no format could be inferred;
2. download successfully; then
3. fail post-processing with **"No book files found in download"**,
because `.mp4` isn't in `AUDIOBOOK_FORMATS` (`shelfmark/core/utils.py`).

Real example: MAM #627978, *The Martian* (Andy Weir, 2020 edition) — 142
files `0001 … 0142 Andy Weir (2020) The Martian.mp4` + `cover.jpg`, 305
MB. Every file is a valid AAC-in-MP4 chapter.

Adding `mp4` to `SUPPORTED_AUDIOBOOK_FORMATS` in `settings.json` doesn't
help since the hard-coded tuple is what post-processing scans against.

## Change

- Add `"mp4"` to `AUDIOBOOK_FORMATS` (single source of truth — settings
UI, Prowlarr parsing, IRC parser, archive extraction and post-download
scan all derive from it), with a comment explaining why.
- Add `".mp4"` to the two hand-maintained debrid `_BOOK_EXTENSIONS`
lists (AllDebrid / Real-Debrid) so file selection matches.
- Slot `mp4` into the IRC `AUDIOBOOK_FORMAT_PRIORITY` table right after
`m4a` (same container family).
- Update the documented default in `docs/environment-variables.md`.
- New regression test
`test_audiobook_multifile_mp4_chapters_are_book_files` modelled on the
existing multi-file usenet test.

### Note for existing installs

The legacy-default migration only widens configs that still hold the old
`m4b,mp3` list, so users on the current widened default won't pick up
`mp4` automatically — they'll need to tick it in Settings → Audiobook
formats. New installs get it by default. Happy to extend the migration
if you'd rather it be automatic.

## Testing

- `ruff check` / `ruff format --check`: clean
- `pytest tests/core tests/config tests/irc tests/prowlarr
tests/download -m "not integration and not e2e"`: 2296 passed, new test
+ `test_audiobook_format_consistency.py` all green. The 10 failures in
`test_entrypoint_permissions.py` / `test_orchestrator_stall.py`
reproduce identically on untouched `main` on macOS and are unrelated.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 17:45:54 -04:00