Commit Graph
264 Commits
Author SHA1 Message Date
splitsec2 88cc734db2 fix(activity): a reused release follows the new request, and stale rows of a delivered request are closed (#1496)
Two things that go wrong when a second request picks a release a first
request already used. A task id comes from the release, so the history
row is reused.

**The row stays with the first request.** Recording a download kept the
first request's id, notes and state. On my instance a card for one book
showed up under the request for a different book, with that request's
note on it. Recording a download now moves the row to the request that
queued it and restarts its queue time. A retry with no request keeps the
request the row had.

**Old rows of a delivered request show as failures forever.** The
startup sweep leaves an old request-linked row alone so its retry stays,
but one whose request already has its book (delivered, or another
download of it finished) is now closed as "Superseded by a completed
download" instead of showing as an interrupted failure for ever.

Also in the first commit: History cards used the saved cover URL as it
was, an external address the page's own-origin image policy blocks,
while running downloads use the cover proxy. They use it too.

The second commit is about restarts. After a restart nothing is in
flight, but a request in downloading, resolving or locating whose row
had moved to another request was stranded, because only queued was
reopened. Those states are reopened too.

Tests cover the row takeover and the retry case, the cover through the
proxy (and a missing cover), the superseded cases (delivered request,
another row finished, an undelivered request is left alone) and the
in-flight states with no row. Ruff and the backend suite pass.
2026-10-20 23:20:04 -04:00
splitsec2 06a67a7150 feat(download): a concurrency limit per source (#1495)
With one global limit, a few tasks that are only waiting hold every
slot. A Soulseek download sitting in a peer's queue is the example I
hit: three of those and a torrent that could start waits behind them.

`MAX_CONCURRENT_DOWNLOADS_PER_SOURCE` (0 keeps the single global limit,
which is the default). With it set, the coordinator starts a download
only when its source is below the limit and passes over tasks of a full
source without losing their place in the queue. `get_next` gets an
optional filter for this, and a cancelled task is still dropped when one
is given. The start-up stagger now applies only between downloads from
the same source, and the pool is sized so every source can reach its
limit.

The second commit: qBittorrent keeps its own queue of torrents waiting
for a free slot. The task polling one of those does no work, so it no
longer counts against its source's limit and the next task of that
source can start. The client does the pacing, as it does for its own
queue.

Tests cover the queue pass-over (order and priority kept, nothing
eligible, cancelled tasks), the coordinator (a full source doesn't block
the others, each source has its own limit, the global limit when unset)
and the client-queue case. Ruff and the backend suite pass.
2026-10-20 23:19:33 -04:00
splitsec2 3b5afb1a04 Merge commit from fork
PUT /api/users/me set a new password from the password field alone, so a live session could replace the password and lock the owner out. A builtin user now has to send current_password with it, checked against the stored hash. A wrong or missing one is a 400, and counts toward the same lockout as a failed login (a locked account gets a 429 without being checked), so the endpoint cannot be used to guess the password.

A password or current_password that is not a string is a 400. A password that was not a string raised a TypeError from len() and answered 500.

The account form asks for the current password when a new one is typed. Profile and settings changes without a password are unchanged.
2026-10-20 23:18:13 -04:00
CaliBrain 10710abd4e fix(download): send every file of a multi-file download to the browser (#1489)
Download to Browser only ever sent the first file of a multi-file
release
(audiobook chapters, packs): the folder output returned final_paths[0]
and
/api/localdownload served that one path.

The folder output now keeps every final path on the in-memory task
(download_paths), and /api/localdownload streams them as one
uncompressed
ZIP named after the book when there is more than one. The archive is
never
staged on disk or held in memory, and ZIP64 covers large audiobooks.
History still stores only the first path, so a re-download from history
after a dismiss or restart serves a single file, as before.

Also brings every linter back to green on all files:
- pre-commit: bump the ruff hook to v0.17.1 to match uv.lock;
ruff-format
  the Python examples in docs/dev and skills
- ruff: fix the remaining issues in skills/shelfmark/download_books.py;
  format base_handler.py
- basedpyright: Optional and attrs errors in databazeknih.py
- knip: drop 31 unused exports, reuse ColumnColorHint in colorMaps
- oxfmt 0.72: styles.css
- direct download: fix the "Welib, mirrors" typo and the test that still
  expected the pre-OceanofPDF message

Fixes #1467
2026-10-20 02:33:53 -04:00
c8d66db22e feat(direct_download): add OceanofPDF as a standalone search and download provider (#1485)
Closes #1149

## What this adds

OceanofPDF as a second Direct Download provider, alongside Anna's
Archive. Unlike Z-Library, LibGen and Welib, which only download files
Anna's Archive found, OceanofPDF searches and downloads on its own. It
works with or without an AA mirror configured; with both, one search
queries both and the results are merged.

It plugs into the provider registry from #1337, so search, record
routing and the download handler need no changes.

**Search:** WordPress `/?s=` search, "title author" first, then the bare
title if that finds nothing. One release per advertised format (`[PDF]
[EPUB]`), with title, author and cover. Shelfmark's supported-format
filter applies as usual.

**Download** follows the site's three hops:

1. The book page has one POST form per format, targeting
`Fetching_Resource.php`.
2. That returns an interstitial whose `<meta http-equiv="refresh">`
points at a signed, expiring file URL.
3. The file URL serves the book.

Pages go through `html_get_page` and its Cloudflare bypass. The form
POST replays the clearance the bypasser stored, and re-solves once if
challenged. Bypassers only replay GETs, so it can't be handed over
directly.

**Settings:**
- `OCEANOFPDF_MIRROR_URLS` under Mirrors. Only the first URL is used.
- An OceanofPDF toggle under Download Sources → Slow downloads. It's
locked until a mirror is set and while the Cloudflare bypass is off. Its
position in the list doesn't matter, and its description says so.
- A saved list from before this change has no OceanofPDF entry. The
settings page shows a missing option as off, so the provider treats it
as off too, keeping the page and the behaviour in agreement. New
installs default to on.

## Fixes found along the way

- **`challenge_marker` rejected real pages behind Cloudflare** (separate
commit, `fix(bypass)`). Cloudflare's bot-detection script injects
`/cdn-cgi/challenge-platform/scripts/jsd/main.js` into every page it
serves, after load. Bypassers return the rendered page, so a successful
FlareSolverr solve of any page under 64 KB came back carrying
`/cdn-cgi/challenge-platform` and was rejected as unsolved. The detector
now ignores that one script path. Real challenge pages load from
`/cdn-cgi/challenge-platform/h/…` and keep their other markers, so they
are still caught. This also affects any other Cloudflare-fronted host,
not just OceanofPDF. Happy to split it into its own PR if you prefer.
- **Brotli on a plain `requests` call.** Copying `DOWNLOAD_HEADERS` into
the form POST offered `br`. Cloudflare then answered in Brotli, which
`requests` can't decode without the optional `brotli` package. The POST
now leaves `Accept-Encoding` to `requests`. If a response still has no
file link, its status, encoding, title and challenge marker are logged.

## Testing

- **New tests:** 41 in `tests/direct_download/test_oceanofpdf.py` and 4
in `tests/bypass/test_cloudflare_beacon.py`. They cover search parsing,
format and language filtering, form selection, the interstitial, the
download flow (including a challenged POST, an HTML stub served as the
file, and cancellation), the toggle, and registry wiring. Fixtures
mirror the live site's markup.
- **Existing tests:** `tests/direct_download`, `tests/bypass`,
`tests/config`, `tests/core` and `tests/download` show no new failures
against `main`. Some tests in those folders can't run on Windows; those
fail identically on both.
- **Lint:** `ruff check` and `ruff format --check` are clean on every
changed file.
- **Live:** verified end to end on a v1.4.0 Docker deployment behind
FlareSolverr (external bypasser), with only an OceanofPDF mirror
configured: search, then the book page, the form POST and the completed
download.

## Known limits

- OceanofPDF doesn't label languages, so its results always pass a
language filter.
- PDFs only appear when `pdf` is in the supported formats.
- Each query reads the first results page only (roughly 7–10 books).
- Downloaded files carry OceanofPDF's own tags/watermark, as noted in
#1149. They aren't stripped.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Adam Stone <196691089+ast0ne1@users.noreply.github.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2026-10-20 02:07:23 -04:00
amura11andClaude Sonnet 5.5 51f32b374d feat(packs): let users choose which files, books and formats to download (#1480)
## Summary
- **Skip files or books in a multi-book pack.** The pack review panel
gets a switch per file and a "Skip this book" switch per book, using the
shared `ToggleSwitch`. A partly skipped book shows e.g. "3 of 5 files"
in the Files column. A fully skipped book is greyed out.
- **Choose formats for a single book.** A new "Choose formats" header
toggle, next to the multi-book one, opens a format picker for a release
that has several book formats (EPUB, MOBI, PDF and so on). It has one
switch per format with file count and size. Releases with one format, or
that can't be inspected, download as before.
- **Skipped files are sent as `ignored_files`.** They are stored on the
task (including restart-safe retry) and removed before grouping and
transfer. This matters for packs because files missing from `book_plan`
would otherwise be filed by the heuristic fallback. Ignored entries
match by release-relative path, then path suffix, then unique basename.
- A pack that ends up with one book after skips keeps the titles you
reviewed instead of falling back to the searched title.

## Screenshots
<img width="1277" height="413" alt="Screenshot 2026-10-19 at 09-39-28
Shelfmark"
src="https://github.com/user-attachments/assets/c005ca4b-3dac-45dd-b2ff-d02fc0f9fcc9"
/>
<img width="1277" height="1210" alt="Screenshot 2026-10-19 at 09-40-23
Shelfmark"
src="https://github.com/user-attachments/assets/2145f3ba-e5fc-4635-9364-1790feb0c8b8"
/>
<img width="1277" height="748" alt="Screenshot 2026-10-19 at 09-39-03
Shelfmark"
src="https://github.com/user-attachments/assets/39023fac-70e3-46d8-af89-1160ebd6e415"
/>


## Test plan
- [x] `pytest` pack tests (new `drop_ignored_files` cases added)
- [x] `vitest`, `tsc`, `oxlint`, `oxfmt`, `ruff`
- [x] Manual check in the UI with a real multi-book pack

🤖 (mostly) Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 02:03:05 -04:00
CaliBrain 3c5cfcb77d fix/redact log secrets (#1488)
- fix(logging): mask tracker API keys in log output (#1486)
- fix(welib): follow slow_download links through the countdown (#1447)
2026-10-20 01:55:49 -04:00
bb1655238a feat(slskd): add Soulseek (slskd) release source, building on #1335 (#1428)
## Add Soulseek (slskd) as a release source, building on #1335

This is @dskvr's work from #1335 rebased onto current `main`, with my
changes on top. #1335 has been quiet since 2026-09-13 and I needed the
changes below, so I carried it forward instead of starting over. Their
two commits are the first two in this branch with their authorship
intact, and the source plugin, handler, API client, settings tab, tests,
`ReleaseProtocol.SOULSEEK`, test client compose file and frontend bits
are all theirs. I haven't changed how a search turns into releases
beyond what is listed here.

If @dskvr would rather keep #1335 alive and take these as commits on
their branch, I can do that instead and close this.

### What I changed

**Downloads survive other apps using the same slskd.** The handler found
its files by listing slskd's downloads, and another app on the same
instance (SoulSync here) sends `DELETE
/transfers/downloads/all/completed` after each of its batches. Once that
ran, a finished download no longer appeared in the list and the task
never saw it complete, even though the files were on disk. With the new
`SLSKD_ISOLATE_DOWNLOADS` setting (on by default) each task:

- enqueues one batch (`POST /transfers/downloads/batches`) into its own
folder, `<SLSKD_DESTINATION_PREFIX>/<task id>`
- polls each transfer by id, which slskd still answers after the list
has been cleared
- decides it is done from the files on disk (sizes, not names), with the
`/events` log as a fallback when the transfers are gone and the files
aren't visible yet
- copies out of that folder, and removes it only if
`SLSKD_KEEP_COMPLETED` is off and the import worked

With isolation off the handler behaves as it did in #1335. If the batch
endpoint isn't there (404/405) it falls back to the original per-file
enqueue.

**Source changes.** m4b/m4a files are their own release next to loose
chapter files, so auto-download can prefer one file over a chapter set.
Results from the query variants are merged per peer. The folder text is
exposed as `extra["author"]` and `extra["title_raw"]`, which is what
auto-download's strict match and pack detection read. `is_available()`
also needs an API key now.

**Smaller fixes.** Cancelling while files are still being located now
removes the transfer records it created. In the settings tab, a cleared
form field is no longer replaced by the saved value when testing the
connection. The new settings are in `docs/environment-variables.md`.

### Testing

`tests/slskd` has 192 passing and 6 live tests that skip without an
instance. I wrote the new tests first and checked the handler and source
rules by breaking them and confirming a test failed. Ruff is clean. On
the whole suite the only failures are 4 in `tests/download` that import
`seleniumbase`, which isn't installed in my environment, and they fail
the same on `main`.

I've been running this on my own deployment against slskd 0.26.0 behind
a VPN, with SoulSync active on a second instance. A search returned a
couple hundred results from about 70 peers, an epub came down and was
imported, and auto-download fetched five audiobooks as single m4b files.
That is a day of use, so I don't know how it behaves on other slskd
versions or on Windows path mappings.

---------

Co-authored-by: sandwich <dskvr@users.noreply.github.com>
Co-authored-by: sandwich <299465+dskvr@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-10-20 01:33:12 -04:00
Amalkrishna A S b38e4e4a00 feat(naming): add {AuthorSort} template variable (#1484)
Closes #1178.

Adds `{AuthorSort}`, the first author surname-first ("Arthur Conan
Doyle" → "Doyle, Arthur Conan"), for rename and organize templates,
alongside `{FirstAuthor}`.

- A value already in "Last, First" form (one word before the comma) is
kept as is; Jr/Sr/II/III/IV stay at the end.
- Known limits, same as calibre's default: the last word is treated as
the surname, so particles ("Le Guin") and surname-first names aren't
detected.
- Added to the settings help, env docs, the frontend variable
picker/preview, and tests.
2026-10-20 01:22:05 -04:00
misko903andClaude Opus 5.5 b83e35ee0a feat(metadata): add Databáze knih metadata provider (#1479)
## What

Adds [Databáze knih](https://www.databazeknih.cz), the Czech and Slovak
community book catalog, as a metadata provider. It follows the Moly.hu
provider (#1172): HTML scraping, no API key, disabled by default, and a
settings tab with an enable checkbox and a Test Connection button.

## Why

Hardcover, Open Library and Google Books cover Czech and Slovak editions
poorly. Databáze knih is the de facto catalog for them, including Czech
and Slovak translations of foreign works. With this provider, Universal
mode works end to end for Czech and Slovak titles: the localized title
and author from Databáze knih feed the release search.

## How

- HTML scraping with BeautifulSoup, which is already a dependency. Book
pages carry schema.org JSON-LD, so most fields come from structured data
rather than CSS selectors.
- **Title search** uses `/vyhledavani/knihy`, with the site's pagination
(`page`) and language filter (`lang` for cs/sk/en/de). A search with a
single match redirects straight to the book page, and that case is
handled.
- **Author search:** the site search only matches titles, so an
author-only search finds the author (accent- and word-order-insensitive)
and lists their books page, which is also paginated. Title + author
filters the title results on the author.
- **ISBN search** follows the site's redirect from `/search?q=<isbn>` to
the book page.
- **`get_book`** reads the full description (the JSON-LD one is
truncated), ISBN-10/13, publisher, edition year, language, genres, cover
(placeholder images dropped), series with id and position, and rating
and rating count. It also reads the page count and original title from
the more-info fragment.
- The `RateLimiter` from Open Library (30 requests/min) and the standard
`@cacheable` decorators. Fetch failures return `None`, so they are not
cached.
- `docs/environment-variables.md`: added the `DATABAZEKNIH_ENABLED`
section.

## Testing

- `tests/metadata/test_databazeknih_parse.py`: offline tests with
shortened copies of real pages. They cover search parsing, pagination
and language parameters, redirect-to-book, author listing, title +
author filtering, uncached failures, book details, ISBN resolution and
helpers.
- `uv run pytest tests/metadata` is green (89 passed), and `ruff check`
and `ruff format --check` are clean.
- Verified live against databazeknih.cz: connection test, title search
(pages 1 and 2), author books, title + author, `get_book`, and ISBN
lookup.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 01:20:26 -04:00
sean0407andSean Li a0ffb268b2 fix: wait for synced files before processing completed downloads (#1464)
## Summary

```text
wait for completed download path
  regular file -> keep existing immediate handling
  directory -> inspect files enabled for this task's content type
              wait until the supported-file signature is stable across checks
```

Fixes the remote-sync race in #1463. A mapped torrent folder can appear
before its large media file finishes copying, so path existence alone
does not mean the download is ready.

## Evidence

- **Before:** In #1463, qBittorrent reported a multi-file audiobook
complete while the destination contained only `.cue` and `.jpg`;
Shelfmark immediately post-processed the folder and failed with “No book
files found in download.” Retrying after the `.m4b` sync finished
imported it.
- **After:** A regression test starts with the cue sheet and cover
present, then simulates the `.m4b` file appearing and growing. Path
waiting returns only after the supported file's size and modification
time remain stable across checks.
- Credential-free isolated `python -m pytest
tests/prowlarr/test_remote_path_mappings.py -q -n 0 -o addopts=''`: 11
passed.
- `ruff check shelfmark/download/clients/base_handler.py
tests/prowlarr/test_remote_path_mappings.py`: passed.

## Merge Danger

**Door:** Two-way

**Blast Radius:** Download handling

The wait uses the existing completed-path timeout and leaves regular
single-file downloads on their current path.

Co-authored-by: Sean Li <22557931+sean0407@users.noreply.github.com>
2026-10-20 01:12:49 -04:00
splitsec2 0ade063db5 fix(download): extract RAR archives in place, not in the IO threadpool (#1459)
Fixes #1458.

A RAR download fails with `Download failed: TypeError`, and the log says
`child watchers are only available on the default loop`. rarfile
extracts by running `unrar` as a subprocess, and `extract_archive_files`
calls it through `run_blocking_io`, so it runs in gevent's IO
threadpool. gevent only lets a subprocess start on the thread that owns
the default loop. `run_blocking_io` already avoids the threadpool for
`subprocess.run`, but a RAR gets to the subprocess through rarfile, so
nothing catches it.

This extracts a RAR on the calling greenlet, where the patched
subprocess waits for `unrar` without blocking anything else. ZIP is pure
Python and stays in the threadpool. I checked it with a real RAR4 file
on a build of main with the stock image's `unrar`: from the threadpool
it raises the TypeError, in place it extracts all 7 files.

Two tests. One checks that a RAR doesn't go through `run_blocking_io`.
The other runs under real gevent patching with an extractor that starts
a process the way rarfile does. Both fail without the change, and the
full suite passes.

I keyed it on the `.rar` suffix to keep it small. Teaching
`run_blocking_io` about rarfile's subprocess would be the more general
fix, but that's a bigger change to a helper a lot of code goes through,
so I left it for you to say if you want it.
2026-10-20 01:12:10 -04:00
ultraviolet150 5f3647a496 fix(clients): wait for qBittorrent-compatible clients to finish before post-processing (#1465)
## Summary

`QBittorrentClient.get_status()` treats a torrent as complete as soon as
`progress >= 1.0`, excluding only the `moving` state. Some
qBittorrent-compatible clients report full progress before they are
finished. rdt-client (v2.0.142) reports `progress = 1.0` once the debrid
and local downloads are done, but keeps `state = "downloading"` for a
few seconds while it finalises the torrent, and until then reports a
guessed `content_path`. It only switches to `pausedUP` with the real
path afterwards.

Shelfmark picks the torrent up inside that window, so post-processing
runs against a path that isn't there yet and fails within milliseconds
("No book files found" / `post-processing failed`). Clicking Retry then
imports it fine. Every rdt-client download hits this.

## Change

- Add `_INCOMPLETE_STATES` (all download-side states, plus `moving`) and
only mark a torrent complete when `progress >= 1.0` **and** its state is
not one of those.
- Real qBittorrent leaves these states when a torrent reaches 100%, so
its behaviour is unchanged.

## Example log (before)

```
12:04:43,215 base_handler - Remapped download path for qbittorrent (<hash>): <rdt-client download dir>\<torrent name>\ -> /downloads/<torrent name>
12:04:43,216 orchestrator - download finished; starting post-processing
12:04:43,232 orchestrator - WARNING - post-processing failed
```

## Testing

- New test
`test_get_status_full_progress_still_downloading_not_complete`: fails
before the change, passes after.
- `tests/prowlarr/`: all passing. `ruff check`, `ruff format --check`
and `basedpyright` clean on the changed files.
- Running the patched client against rdt-client: downloads now import
without Retry.

Related to #1463 / #1464, which handle the same symptom when the client
really is finished but files are still syncing to local disk. This PR
addresses a different cause: the client reports 100% progress while
still in a downloading state, so Shelfmark shouldn't treat it as
complete yet. The two changes are independent and complementary. Not
covered by #741, which fixed add-time failures. Possibly related: #607.
2026-10-20 01:11:33 -04:00
Matt Van HornandMatt Van Horn d3bcf0674d fix(direct_download): Anna's Archive search 404 on ISBN queries (#1456)
When ISBN filters are present, the search query is now the deduplicated
bare ISBN strings, with the free-text query appended only if it is not
already one of those tokens. Searches with no ISBN still send the quoted
free-text query alone. An Anna's Archive lookup that includes an ISBN
comes back as 404 Not Found, so the direct-download release search fails
and the book cannot be retrieved. The failing request puts an
isbn13:/isbn10: boolean expression in the q parameter.

The direct-download search query tests passed.

Closes #1444

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
2026-10-20 01:10:03 -04:00
splitsec2 84d1d7196c fix(irc): pin the TLS context to TLS 1.2 or newer (#1455)
CodeQL on my fork flagged `ssl.create_default_context()` in the IRC
client as allowing TLS 1.0 and 1.1 (`py/insecure-protocol`, rated high).

In practice the shipped image is already fine. Python 3.10+ sets a TLS
1.2 floor on the default context, and on the current image (Python
3.14.7, OpenSSL 3.5.7) `minimum_version` comes back as TLS 1.2. An
OpenSSL config on the host can still lower that floor, so this pins it
with one line, `context.minimum_version = ssl.TLSVersion.TLSv1_2`.

Added `tests/irc/test_client_tls.py`. It drops the floor to the minimum
before `connect()` runs, then checks that `connect()` raised it back to
1.2. It fails without the change.

Certificate and hostname verification are still off (`CERT_NONE`). I
left that alone because of the comment about self-signed certs on IRC
servers. Turning verification on, or making it a setting, would be the
bigger fix, but that is ultimately a call for @calibrain if you want a
PR for it.
2026-10-20 01:09:26 -04:00
Som Samantray 2b8e0da2f3 fix(direct-download): clarify required search mirror (#1454)
Direct Download now tells users who configure only a Z-Library mirror
that they need an Anna's Archive mirror for search. Z-Library, LibGen,
and Welib mirrors download results found through Anna's Archive.

Fixes #1408. Validation: the Direct mode browser check, 18 focused
tests, Ruff, formatting, and backend/test type checks passed. The
CI-style Python suite passed 3,569 tests;
`tests/core/test_hardlink.py::TestSameFilesystem::test_string_paths` was
excluded because this container reports different `st_dev` values for a
file and its parent directory. AI disclosure: OpenAI Codex assisted with
diagnosis, implementation, and validation.
2026-10-20 01:09:09 -04:00
HeihoffJandClaude Opus 5.5 756bdec19b fix(metadata): show the translation a search matched on Open Library and Hardcover (#1451)
Searching a translated title on Open Library or Hardcover returned the
original-language work. "Der Weg der Könige" came back as "The Way of
Kings" (2010), and that English title then went into the release search
and the download name. Both providers keep one record per work, titled
after the original, and list translations underneath it.

#483 already searches releases per language with Hardcover's localized
titles, which fixed the release side of #360 when the matching language
is selected. This PR fixes the search result itself, so the book a user
picks is the translation they searched for, and it covers Open Library,
the default provider, which has no titles_by_language.

- Open Library: request the matched edition (editions.*) and prefer its
title, ISBNs, cover, language and publication year when its title
differs from the work's.
- Hardcover: use the alternative title named in the hit's highlights
(alternative_titles, best index first), without the original's subtitle.
A hit whose title also matched keeps the work title.
- /api/releases: when the modal sends a title that differs from the book
record's, it becomes search_title. Before, Hardcover's search_title from
the original won, so picking "Kinder des Nebels" still searched "The
Final Empire" first.
- Release cache: keyed on title and language as well as the provider ID.
All translations share that ID, so opening "The Way of Kings" after "Der
Weg der Könige" showed the German releases until a reload.

Known limitations: neither provider has descriptions per edition, so the
description stays the work's (usually English). Hardcover's search hits
carry no year or language for alternative titles, so those stay the
work's; Open Library's do not. Google Books is unaffected, since it
already returns one volume per edition.

Tests: Open Library edition parsing, Hardcover alternative-title
matching, the /api/releases title override, and the release cache, each
written to fail first. The Python suite passes apart from the 9
test_entrypoint_permissions.py tests, which fail the same way on an
unmodified main on macOS. The frontend typecheck, lint, format check and
unit tests pass. Checked on a live instance with Open Library, Hardcover
and Google Books.

Refs #360, #483

Disclaimer:
This was developed with the help of AI but reviewed and tested as well
as deployed on my server by a real human.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 01:08:45 -04:00
splitsec2andCaliBrain 45548687b7 fix(hardcover): defensive rate limiting and 429 backoff (#1439)
Kept as small as I could so it's easy to review. It is still about 140
lines of code plus tests, in two commits so the second can be dropped on
its own. Same kind of fix as #1429, so see the question there about
separate PRs versus combined.

This is a defensive change. I found the problem on my own deployment,
which has a more automated ingestion on top of this provider: Hardcover
want-to-read items turn into requests, and each pending request gets one
`get_book` lookup. One pass made around 38 lookups back to back, which
is more than Hardcover's burst allows, and a few came back as 429.
`_execute_query` swallowed the 429 and returned `None`, so `get_book`
returned `None`, and the caller read that as "no such book". Nothing in
the logs said a lookup had been throttled.

**What it does**
- Every POST goes through one shared token bucket (burst 5, 0.8 requests
per second), set under the documented Free limits. The limit is per user
across all of the account's apps, so I kept well below it.
- A 429 is retried up to 3 attempts in total. It waits for `Retry-After`
when Hardcover sends one, otherwise 2^attempt seconds plus up to a
second of jitter, and the wait holds back other threads too. A
`Retry-After` over 30 seconds (a spent daily quota) is not waited out.
- When the retries run out it raises `HardcoverRateLimitError`, a
`RuntimeError`, instead of returning `None`.
- The second commit makes `get_book` ask for that error, so a throttled
or failed lookup is no longer taken for a missing book. GraphQL
rejections still return `None`.

**Behaviour change in the second commit.** `/api/metadata/book` already
turns a `RuntimeError` into a 503, so a timeout or HTTP error there now
returns 503 instead of 404. In `/api/releases` the same error goes
through the operational errors handler and returns 500 with the message,
where it used to return 404 "Book not found". If you'd rather keep
returning `None` and only have the pacing and backoff, I can drop the
second commit.

**Limits.** I read them from the `hardcover-docs` repo
(`api/Getting-Started.mdx`, Rate Limits), because the docs site blocks
my headless browser: Free is 60 per minute, a burst of 10 and 5,000 a
day, with `Retry-After` on a 429. Personal Access Tokens get double the
burst. I didn't parse the `RateLimit-Policy` headers, so the bucket
constants are fixed.

**Testing.** The tests use a fake clock and never sleep. I broke each
rule in turn (pacing, `Retry-After`, the 30 second cap, the shared
hold-back, and `get_book` raising) and each break failed a test. Ruff,
basedpyright, `tests/metadata` and `tests/core` pass. I haven't seen a
real 429 from Hardcover with this change running, so the `Retry-After`
handling comes from the docs.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.

---------

Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2026-10-20 01:06:56 -04:00
splitsec2 61cf0792bd fix(googlebooks): report more pages when limit is above the 40 result maximum (#1434)
Kept small on purpose so it's easy to review. Same kind of small fix as
#1429 to #1432, so see the question in #1429 about separate PRs versus
combined. I'm sending them separately until I hear otherwise.

`search_paginated` guesses `has_more` as `len(books) >= options.limit`.
Google Books returns at most 40 per page and `search` already caps
`maxResults` at 40, but `/api/metadata/search` accepts a limit up to
100. With a limit over 40 a full page of 40 never reaches `limit`, so
`has_more` is always False. The UI sends 40, so this only affects direct
API callers.

The Google provider now overrides `search_paginated` to apply the same
cap before the base heuristic runs. A full page of 40 with a limit of
100 reports more results and a short page doesn't. The first test fails
on current `main`. Ruff, basedpyright and `tests/metadata` pass.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.
2026-10-20 01:01:38 -04:00
splitsec2 f0f0d46381 fix(queue): first terminal status wins, so a late COMPLETE cannot overwrite a cancel (#1429)
Kept small on purpose so it's easy to review. I have a few more small
fixes like this from going back over recently merged PRs. Would you
rather get them as separate PRs like this one, or combined into one?

#1361 made `cancel_download` write under the queue lock, which stops a
cancel from overwriting a download that finished first. The other
direction is still open. The worker checks `cancel_flag.is_set()` and
then calls `update_status(COMPLETE)` a few lines later in
`_process_single_download`. If a cancel lands between those two lines
the task goes CANCELLED, then the worker writes COMPLETE over it.
`_apply_status_locked` allows terminal to terminal moves, so the
terminal hook also fires a second time and records two events for one
task.

This makes the first terminal status win: `_apply_status_locked` ignores
a different terminal status once the task already has one. Requeueing
from a terminal status still works because QUEUED isn't terminal.

I replaced the existing race test. It patched `queue.update_status`,
which `cancel_download` no longer calls, so it never exercised the path
it names, and its last assertion only looked at the final hook event.
The new tests call the queue directly: complete then cancel is refused,
cancel then complete stays cancelled with one hook event, error then
complete stays error, and a terminal task can still be requeued. The two
late-write tests fail on current `main`.

Ruff and basedpyright are clean. `tests/core` and `tests/download` pass
apart from 4 `seleniumbase` import failures in my environment that also
fail on `main`.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.
2026-10-20 01:00:21 -04:00
splitsec2 65dbf42a86 fix(release-search): report a ValueError from a source as that source's error (#1433)
Kept small on purpose so it's easy to review. This is the same kind of
small fix as the ones in #1429 to #1432, so see the question in #1429
about separate PRs versus combined. I'm sending them separately until I
hear otherwise.

In `search_source_releases` the `except ValueError` comes before the
branch that handles the operational errors, and `ValueError` is in that
tuple. A `ValueError` from `build_release_search_plan` or
`source.search()` is therefore reported as `Unknown source: <name>`
instead of `<name>: <message>`. The only `ValueError` meant for that
branch is the one `get_source` raises for a name that isn't registered,
so that call now has its own `try` and the rest goes back to the
operational errors branch.

The failure test already covered `OSError` and `RuntimeError`, so I
added `ValueError` to it, and that case fails on current `main`. Ruff,
basedpyright and `tests/core` pass.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.
2026-10-20 00:59:54 -04:00
splitsec2 93f58caf7a fix(debug): keep SHELFMARK_API_KEY_READONLY out of the debug bundle (#1432)
Kept small on purpose so it's easy to review.

`genDebug.sh` filters secrets out of `environment.txt` by matching
`_KEY=`, `_SECRET=`, `_PASSWORD=` and `_TOKEN=`, which only catches
names that end in those words. `SHELFMARK_API_KEY_READONLY` ends in
`_READONLY`, so its value lands in the file, and people attach these
bundles to issues. That key only reads counters, so the impact is small,
but any variable like `DB_PASSWORD_FILE` would leak the same way.

The pattern now matches the secret word anywhere in the variable name.
The new test runs the real filter line from the script against a made-up
environment, and it fails on the old pattern.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.
2026-10-20 00:59:10 -04:00
splitsec2 be2653bdb7 fix(debridlink): reuse the file list from the status check (#1431)
Kept small on purpose so it's easy to review.

When a torrent is ready, `get_status` has already fetched the full
record with `/seedbox/list?ids=` and `_handle_torrent_info` has checked
that every file is at 100%. The retrieval thread was then started
without that list, so `_process_and_download` called `_fetch_file_list`,
which asks the same endpoint again. That second call runs inside the
thread's generic `except`, without the flood and transient error
handling the status path has, so a rate limit or a hiccup there sets the
phase to error for a torrent that finished fine.

This passes the files already in hand into the thread.
`_process_and_download` already took an optional `files` argument. The
new test starts the thread from a ready status and checks it uses the
given list and never calls `_fetch_file_list`.

Separately, `download_url` returns a `BytesIO`, so a whole file sits in
memory. AllDebrid, TorBox and Real-Debrid do the same, so I left that
for a streaming variant of the shared helper instead of fixing it for
one client here.

Ruff and basedpyright are clean and `tests/prowlarr` passes.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.
2026-10-20 00:59:00 -04:00
splitsec2 e8f6505bea fix(library): back off after a failed index instead of retrying for every result (#1430)
Kept small on purpose so it's easy to review.

`library_index._entries_for` retries `fetch_entries` on every call while
the index is failing. The lookup runs once per search result, so with
the Calibre path unreadable a page of 40 results makes 40 attempts and
logs 40 warnings. The entrypoint runs one gevent worker and the Calibre
read is synchronous sqlite, so each attempt also holds up every other
request.

After a failure it now answers from the stale cache (or an empty list)
for 60 seconds before trying again, and a successful index clears that.
The new tests cover 40 lookups against a failing library with a stale
cache (one failed attempt, one warning), a failing library with no cache
(tried once), and a retry once the 60 seconds have passed. The first and
second fail on current `main`.

I left a few other things in this module out to keep this one small:
re-indexing blocks the worker for as long as a big library takes (I
haven't measured how long), `test_connection` stores entries from the
unsaved form path under the shared cache key, and a title with a
collection word in it, like "Books of Blood", shows as "In a
collection". I can send any of those separately.

Ruff and basedpyright are clean and `tests/core` passes.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.
2026-10-20 00:58:49 -04:00
splitsec2 47707b827a feat(download): make the torrent stall window a setting (#1426)
#1420 gave a torrent 15 minutes to get going and 15 minutes after each
step forward. That suits my swarms, but other people will want it
shorter or longer, so this makes the number a setting.

`TORRENT_STALL_TIMEOUT_MINUTES` sits in Download Clients with the other
torrent options and takes 5 to 60. The default is 15, so nothing changes
unless someone sets it. 5 is the orchestrator's own stall timeout, so
choosing it asks for no extra time and gives the old behaviour. The
value can arrive as text from an env var, so anything that isn't a
number falls back to the default and anything out of range is clamped.

The orchestrator caps one activity grace at 16 minutes, so I raised that
to an hour to fit the top of the range. As far as I can tell the
protection bypass is the only caller that asks for more than a few
minutes, but I haven't audited every caller. If you'd rather keep the
cap, I can lower the maximum to 16 minutes instead.

Tests cover the default, 5 and 30, text values, out of range values and
NaN, the start and movement windows following the setting, and the
setting showing up in the Download Clients tab. I broke each rule on
purpose to check a test catches it. The full suite fails the same tests
with and without this change (the bypass and seleniumbase tests don't
run in my environment).
2026-10-20 00:58:24 -04:00
CaliBrainandClaude Opus 5.5 9321096032 fix(clients): correct Debrid-Link file listing and rate-limit handling (#1425)
Follow-up to #1380.

- Fetch the file list with ?ids=, the documented parameter. There is no
  ?id=, so a many-file torrent was not expanded and the call could fail
  with badArguments at the last step of every download.
- Back off status checks for 60s after floodDetected, doubling to a 240s
cap, instead of an hour. The orchestrator cancels a download after five
minutes without a change, so the hour-long pause cancelled every active
  download. Only a flood on the status check starts it, and a successful
  check resets it.
- Drop the client-side one-hour stall timer. The orchestrator stall
timer
  always fires first.
- Keep progress at 50% when file retrieval starts instead of dropping to
  0% until the first file arrives.
- Report queued, paused and verifying torrents as QUEUED, PAUSED and
  CHECKING, so a queued torrent gets the handler queue grace.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 08:08:32 -04:00
splitsec2 cad18b8019 feat(clients): add Debrid-Link as a torrent client (#1380)
calibrain said in #1315 that he'd review and merge a Debrid-Link client
if someone took a stab at it, so here it is. It's the fourth debrid
service alongside AllDebrid, Real-Debrid and TorBox and it uses the same
DownloadClient interface as the other three. No frontend changes, the
settings UI picks it up from the backend field definitions.

**I can't test this against the live API.** I don't use debrid and I
don't have a Debrid-Link key. Everything here is built from their v2 API
docs and cross-checked against two independent reference
implementations, so the request shapes should be right, but nobody has
run a real magnet through it yet.

@Tatsu941 you opened the issue and mentioned they give developers free
premium. Would you be able to try it? The things most worth confirming
are the field names on a seedbox torrent, that `premiumLeft` is the
right premium check, and that an API error comes back the way the client
expects. Happy to fix whatever turns up.

Two things about their API shaped how this is written.

Every v2 endpoint wraps its payload in a `{success, value}` envelope and
a failure can arrive with an HTTP 200, so `_request_value` decides the
outcome from the body instead of the status code.

A completed seedbox torrent already carries a direct `downloadUrl` on
every file. The other three clients each need a separate unrestrict or
link-request call per file and this one doesn't, which is why it came
out shorter than them despite doing the same job.

Endpoints used:

| Endpoint | Used for |
|---|---|
| `GET account/infos` | test the key, confirm premium time remains |
| `POST seedbox/add` | magnet as JSON, .torrent as multipart |
| `GET seedbox/list` | poll progress |
| `DELETE seedbox/{id}/remove` | clean up |

Ruff and basedpyright are clean and the 36 new tests pass. The two
failures in `tests/e2e/test_proxy_auth_middleware.py` are already on
main and fail the same way on a pristine checkout. They assume the first
proxy-authenticated user becomes an admin, which only holds while the
user table has no admin in it, so they depend on how many users earlier
tests created.

One thing for review: the download loop uses `download_url` like the
other three clients, which holds each file in memory. I kept the
existing pattern rather than do something different in a new-provider
PR, but say so if you'd rather it streamed here.

Closes #1315
2026-10-03 07:55:36 -04:00
CaliBrain f773ac6029 fix(sabnzbd): strip Prowlarr API key on cross-host NZB redirects (#1424)
_fetch_nzb_content sent X-Api-Key
then followed redirects with the header still attached, leaking the
Prowlarr key when Prowlarr 302s to the indexer download link. Follow
redirects manually with allow_redirects=False and re-evaluate headers
 per hop, mirroring the torrent path.
2026-10-02 22:46:16 -04:00
splitsec2 755f28a6a4 feat(api): add a read-only API key and a /api/stats endpoint (#1419)
This is the read-only API key from #1410, where you said to go ahead.

I run Shelfmark behind Homepage and wanted more on the dashboard tile
than up or down, without putting an admin API key in the dashboard's
config. This adds a second key, `SHELFMARK_API_KEY_READONLY`, that can
read one new endpoint.

`GET /api/stats` returns counts only: books added over the last 7 and 30
days by format, the queue, requests by outcome, and download failures
over the last 7 days. No titles and no user names. The read-only key,
the admin key or an admin session can read it.

The read-only key gets a 403 on every other path and on any write,
including a POST to `/api/stats` itself, and it never gets a session or
a cookie. If a request carries both keys, the admin one wins, so a
proxy's own `Authorization` header can't downgrade a correct
`X-Api-Key`. Unset means off, like the existing key. The endpoint exists
either way, but without a key only an admin session can read it.

It also adds a `db_path` property on `UserDB`, so the stats module can
open the database read-only.

Tests cover the key scope, the 403s for the read-only key, both keys at
once, who can reach the endpoint, and the counters. The full suite
passes, and I broke each rule on purpose to check a test catches it. A
version of this has been running on my own install behind Homepage.

If you'd rather have the stats in a different shape or under a different
path, I'm happy to change it.
2026-10-02 22:45:48 -04:00
adman234andClaude Opus 5.5 32abaff21e feat(naming): {Narrator} placeholder and MyAnonamouse series fallback (#1407)
## Human-Written explanation:

Following up on previous PR to use MAM ID to get nararrator and series
to show up in search, this PR will allow the series and nararrator
fields to be added to the path when saving an audiobook.

I have tested my ghcr.io image on my instance and it seemed to work
properly.

All the text below is written by Claude.

# feat(naming): {Narrator} placeholder and MyAnonamouse series fallback

Follow-up to #1390 and #1399. Related to #605 and #934 (narrator
support).

## Problem

Several narrations of the same audiobook currently land in the same
folder, because nothing in the path template tells them apart. Keeping
more than one version means renaming folders by hand after every
download.

Since #1390, MyAnonamouse results carry the release's narrator and
series in `extra`, but the naming templates can't use them: there is no
`{Narrator}` placeholder, and `{Series}` / `{SeriesPosition}` only come
from the metadata provider.

## Changes

- **`{Narrator}` placeholder** (`core/naming.py`,
`download/postprocess/transfer.py`, `core/models.py`): `DownloadTask`
gains `narrator`, read from `narrator` or `extra.narrator` when a
release is queued, and kept in the restart-safe retry payload. It's
available in audiobook templates like any other placeholder, including
prefix/suffix blocks such as `{ - Narrator}`.
- **Audiobookshelf folder style**: Audiobookshelf reads the narrator
from folder names like `Title {Narrator}`. The template `{Title}
{{Narrator}}` already matches as `{` + `{Narrator}`; the parser now also
drops the closing brace when the narrator is empty, so the folder is
`Title` rather than `Title }`.
- **No stray spaces around `/`**: an empty placeholder at the start or
end of a folder name no longer leaves a space there (e.g. `Title
/Title`). This applies to all templates.
- **Series fallback** (`release_sources/prowlarr/mam.py`, `source.py`,
`download/orchestrator.py`): MAM enrichment also stores the first
series' name and number as `extra.series_name` /
`extra.series_position`, which queueing already falls back to when the
metadata provider has no series. The provider's series still wins. A
release-level series number is only used when it names the same series
as the provider, so a provider series is never paired with the number of
a different MAM series (an omnibus, for example).
- **Settings and preview** (`config/settings.py`,
`namingTemplatePreview.ts`): the audiobook template descriptions list
`{Narrator}` and explain `{{Narrator}}`. The settings page preview
mirrors the parser changes and offers `{Narrator}` under "Insert
variable" for audiobook templates only.
- **Docs**: regenerated `environment-variables.md`. This also picked up
the Library Check settings, which weren't in the generated docs yet;
happy to drop that part to keep the diff focused.

## Example

Audiobook Path Template `{Author}/{Title} {{Narrator}}/{Title}`:

| Narrator | Result |
|----------|--------|
| Samuel Roukin | `Christopher Ruocchio/Empire of Silence {Samuel
Roukin}/Empire of Silence.m4b` |
| none | `Christopher Ruocchio/Empire of Silence/Empire of Silence.m4b`
|

## Testing

- `tests/core/test_narrator_template_variable.py` (new, 16 tests):
`{{Narrator}}` with and without a value, the `{ {Narrator}}` prefix
form, prefix/suffix blocks, sanitizing, `build_library_path`, task
metadata, retry payload round trip, MAM series name/number parsing
(including a `1-3` range having no number), and queueing (narrator from
`extra`, series fallback, no cross-series number, blank narrator).
- `namingTemplatePreview.test.ts`: the same `{{Narrator}}` cases for the
preview.
- Updated `test_generate_env_docs.py` for the new template description.
- Full `pytest -m "not integration and not e2e"` compared with an
upstream `main` worktree on the same machine: no new failures (the
remaining ones are Windows-only on both).
- `ruff check`, `ruff format --check`, `basedpyright` (0 errors) and
`vulture` on touched files; frontend `tsc --noEmit`, `oxlint`, `oxfmt
--check`, `vitest` (218 passed).
- Checked the settings page locally: the audiobook preview renders `...
{Simon Vance}.mp3` and `{Narrator}` is listed for audiobook templates
only.

No behavior change for existing templates, apart from spaces next to `/`
being trimmed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 22:45:10 -04:00
CaliBrain 119d3374e3 fix(downloads): reconcile downloads a restart interrupted (#1423) 2026-10-02 22:35:09 -04:00
HeihoffJandClaude Opus 5.5 95d9f1da57 fix(sabnzbd): prefetch NZBs served from the indexer's own domain (#1416)
Indexers often serve NZB downloads from a different host than their API
(e.g. file.indexer.example for indexer.example/api). The exact-origin
check added in #967 rejected those, so SABnzbd silently fell back to
addurl.

Trust any host in the configured indexer's registrable domain, using the
Public Suffix List so shared suffixes like co.uk or duckdns.org never
widen trust. Scheme and port must still match, IP literals and
single-label hosts still require an exact match, and the Prowlarr API
key header is unchanged.

This closes this issue:
https://github.com/calibrain/shelfmark/issues/1411 

Disclaimer: Implemented by Claude and Co-Authored by me. Tested and
verified by me alone (Why is it always this way around and not the
other)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 22:25:36 -04:00
splitsec2andCaliBrain 8392b43dd3 fix(downloads): reconcile downloads a restart interrupted (#1409)
I found this while grabbing new books and working on my fork. I
redeployed the container while downloads were running, and it left a
mess in my queue. It's an edge case, but a few rows were stuck until I
cleared them by hand.

The download queue lives in memory, so stopping the process with a
download in flight leaves its history row "active" and its request
"queued", with nothing working on either. The activity API shows that
row as "Interrupted" when it's read, but nothing is stored and nothing
looks at it again. Upstream only reopens an errored request when no
manual retry is offered, and an interrupted row always offers one, so
the request just sits there.

This closes those rows out once at startup, before the coordinator
starts, as an error with the message "Interrupted". The retry payload
stays, so the manual retry still works. A request whose download was
queued in the last 24 hours goes back to pending through
`reopen_failed_request`. Older ones get closed out but their requests
are left alone, because I didn't think a startup should reopen a backlog
or fill an admin's approval queue with old requests. A request that
already arrived is left alone, and running it twice changes nothing.

It assumes a single worker process, which `entrypoint.sh` already sets.

On my install the first start after deploying it closed 53 rows that had
been orphaned for nine days. Tests are in
`tests/core/test_startup_reconcile.py`, and the full suite passes
locally.

If the 24 hour window feels off, or you'd rather handle this another
way, I'm happy to change it or drop the PR.

Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2026-10-02 22:23:28 -04:00
Evan KazakinandEvan Kazakin 7888217380 feat(download): add Remove & Delete Files torrent completion action (#1401)
Closes #1027

## Summary

- add a **Remove & Delete Files** (`remove_and_delete`) option to the
Torrent Completion Action setting
- after a successful import, remove the torrent and ask the client to
delete its downloaded data (`delete_files=True`), matching what the
usenet "move" flow already does
- clarify the setting description so it says **Remove** keeps the
downloaded files (raised in the issue comments)
- regenerate the `PROWLARR_TORRENT_ACTION` entry in
`docs/environment-variables.md`

## Behavior

The deletion runs from `post_process_cleanup`, the same place as the
existing Remove and Change Category actions, so it only happens after
output transfer and post-processing have succeeded. At that point every
file has already been copied or hardlinked into the library, and
transfer size checks have passed. A failed import never removes or
deletes anything. The torrent client deletes its own data, so Shelfmark
does not delete paths itself and remote path mappings are not involved.
Keep, Remove, and Change Category behave as before, and Keep is still
the default.

If a request matched a completed torrent that was already in the client,
this option deletes that torrent's data too. That is the same scope the
existing Remove action already applies to.

## Validation

- `make python-checks` (ruff check, ruff format, basedpyright on backend
and tests, vulture): passed
- `make python-test`: 3363 passed, 5 skipped, 9 failed. All 9 are in
`tests/config/test_entrypoint_permissions.py` and happen because macOS
`/bin/bash` 3.2 does not support `${1,,}` in `entrypoint.sh`. They do
not touch this change.
- new tests in `tests/prowlarr/test_handler.py`:
- Remove passes `delete_files=False` and Remove & Delete Files passes
`delete_files=True`
  - a failed import with Remove & Delete Files does not call the client
  - the delete case fails without the handler change
- manual smoke run: `PROWLARR_TORRENT_ACTION=remove_and_delete` set
through real config loading, with a stub client that deletes its folder.
The torrent folder was deleted and the hardlinked library file stayed
intact.
- pre-commit hooks (prek) passed

---------

Co-authored-by: Evan Kazakin <evan@Evans-MacBook-Pro.local>
2026-10-02 22:15:17 -04:00
splitsec2 d99e9d7c4f fix(download): remove a cancelled torrent that never started (#1421)
Cancelling a torrent download leaves the torrent in the client.
`_safe_remove_download` documents the rule: "torrents: never remove or
delete client data (avoid breaking seeding)". That makes sense for a
torrent that has data. A torrent that is still at 0% has nothing to seed
and nothing to resume, and leaving it behind holds a download slot in
qBittorrent's queue or sits there as a dead entry.

I hit this on a qBittorrent that is shared with Sonarr, Radarr and
Lidarr and has a download limit. Shelfmark cancelled downloads that were
queued (#1420) and left them in the client. Three torrents that never
fetched metadata held every active slot, and each torrent added after
them waited behind them. Removing those by hand freed the slots.

This removes the torrent and its files when a download is cancelled,
Shelfmark added the torrent, and the client reports 0% progress. A
torrent with any data is left alone as before. So is one that was
already in the client when the download started, since Shelfmark didn't
add it. If the removal fails it is logged and the cancel carries on.

Tests cover removal at 0% for a queued and a downloading torrent,
leaving one that has data, leaving one the user already had, and a
failing removal. The full suite passes, and I broke each rule on purpose
to check a test catches it.

This changes a rule you wrote down, so I've kept it apart from the queue
fix. If you'd rather have it as a setting, or only remove a torrent that
is still queued or fetching metadata, I'm happy to rework it.
2026-10-02 22:12:51 -04:00
splitsec2 88f73e3bc3 fix(download): do not cancel a queued torrent as stalled (#1420)
qBittorrent only runs a few downloads at once (`max_active_downloads`
defaults to 3) and holds the rest as `queuedDL`. Shelfmark's stall timer
cancels a download after five minutes without a changed status or
progress. A queued torrent reports "Queued" on every poll, so it looks
idle, and Shelfmark cancels a download that was only waiting for a slot.

I run Shelfmark against the same qBittorrent that Sonarr, Radarr and
Lidarr use, which I assume is a common setup. Their torrents and
Shelfmark's share one limit, so anything that fills the slots makes
Shelfmark's torrent wait. In my case three torrents that never fetched
metadata held every slot. I'd expect a busy Sonarr queue to hold them
the same way, but I haven't reproduced that. Once the stall timer had
cancelled the waiting downloads they stayed in the client, still queued
behind the same blockers. On my instance 10 AudioBookBay downloads were
cancelled while qBittorrent still had them queued.

This changes the torrent poll loop. When the client reports a queued
torrent, it asks for an activity grace (the mechanism `html_get_page`
already uses for protection bypasses) and releases it as soon as the
torrent leaves the queue. The grace is renewed every ten minutes and
stops at two hours, so a queue that never moves still ends. A torrent
that is downloading and moving is timed as before until the new window
below applies.

The second commit sets the stall message before the cancel runs. The
terminal hook copies the message into the history row, and it was set
afterwards, so a download the timer ended was recorded with the last
poll's "Queued". That left no way to tell it apart from a person
cancelling.

The last two commits are about slow torrents. The timer only counts a
change in progress, so a torrent that needs more than five minutes to
fetch metadata or find its first peer is cancelled before it has any
progress to count, and one that moves a few megabytes at a time can be
cancelled between bursts. I hit this with a torrent that had a single
peer. A torrent now gets one 15 minute window the first time it is seen
outside the queue, and each step forward in progress renews it. One that
makes no progress for 15 minutes is still cancelled, so a dead torrent
now takes about 15 minutes to end instead of five. That trade is the
part I'm least sure about, so the number is easy to change.

Tests cover the grace being requested once, released when the torrent
starts, renewed, and capped, the order of the message and the cancel,
and the start and movement windows. The full suite passes, and I broke
each rule on purpose to check a test catches it.

The two hour ceiling is a guess. If you'd rather have a setting for it,
or a different number, I'm happy to change it.
2026-10-02 22:12:25 -04:00
CaliBrain bac0b93566 fix: keep https:// for qBittorrent and hide release sources from non-admins (#1422)
qBittorrent over HTTPS (#1417)

qbittorrent-api ignores the scheme in `host` and probes HTTP and HTTPS
itself. When the HTTPS probe fails, for instance on a certificate
Shelfmark doesn't trust, it falls back to plain HTTP against the TLS
port, and a reverse proxy answers "400 The plain HTTP request was sent
to HTTPS port". The download client and the Test Connection button now
pass FORCE_SCHEME_FROM_HOST for https:// URLs, so the configured scheme
is used as-is and a certificate problem is reported as one. http:// and
bare host:port URLs keep the probe: normalization adds http:// to a bare
host, and the probe follows a proxy's redirect to HTTPS.

Release sources leaked to non-admins (#1418)

A download queued from an approved request carries the release an admin
picked, and the requester could read it from their own activity feed:
the request's release_data (source_id, indexer, info URL, torrent
attributes), the download's full server path, and the download id
itself, which for Prowlarr is "<indexer id>:<guid>" and for a private
tracker is a URL into it. That id keys every status payload, the
/api/localdownload query and the cover proxy URL, so hiding
release_data alone would not have closed the leak.

For non-admin viewers:
- request rows keep only the release fields the activity cards display
- download_path is reduced to the file name, which the browser download
  reveals anyway
- request-linked downloads are addressed by an opaque id (a keyed HMAC
  of the task id) in the snapshot, history, /api/status, queue order,
  active downloads, dismissed keys, cover URLs and the websocket status
  and progress events. Routes that take a download id translate it
  back, searching only the caller's own downloads.

Downloads a user queued directly keep their real id: the user picked
that release from search results that already showed it, and the
release list matches its buttons to the queue by that id. Admin views
are unchanged. The activity sidebar now links a fulfilled request to
its download by request_id instead of release_data.source_id.

Closes #1417
Closes #1418
2026-10-02 20:38:58 -04:00
vansh 41a4df01c4 fix: hand DiamWall 513 challenges to the bypasser (#1400)
## Summary
Recognize DiamWall interstitials in both the HTTP retry path and the
internal browser helper. HTTP 513 challenge responses go directly to the
configured bypasser, while ordinary 513 responses keep the existing
error behavior.

Fixes #1386

## Testing
- `uv run --frozen --extra browser pytest -q -n 0
tests/download/test_http_challenge_513.py
tests/bypass/test_internal_bypasser.py`: 27 passed
- Ruff lint and format checks passed
2026-10-02 16:26:04 -04:00
CaliBrain 893f7bdb92 fix(mam): keep the session ID on MAM and rerun Prowlarr's exact search (#1399)
Follow-up to #1390.

- The origin came from each result's infoUrl, matched by a regex that
did
  not check the host, and the mam_id cookie had no domain. Any Prowlarr
  indexer returning a URL like https://evil.example?myanonamouse.net/t/1
  sent the session ID to evil.example. Requests now always go to
https://www.myanonamouse.net (Prowlarr's only MAM URL), with the cookie
  as a header and redirects off. Only results from Prowlarr's
  MyAnonamouse indexer are looked up, and their URLs must be on
  myanonamouse.net.
- The lookup searched every category and read one page, so for common
  titles most of Prowlarr's results were missed (a "Dune" audiobook
  search: 56 audiobooks on the first 100 of 328 matches). It now reruns
  Prowlarr's exact search: the same query clean-up, the MAM main
  categories behind the Torznab categories searched (13/15/16 for
  audiobooks, 14 for e-books, all once expanded), and the MAM indexer's
  own search type, search-in options and languages. Further pages are
  read while IDs are missing, page 1 of every title first, at most 4
  requests per search.
- Failed requests back off for 1, 2, 4 ... up to 30 minutes. The 10th
  consecutive failure stops enrichment until Test MAM Session passes,
  the session ID changes, or Shelfmark restarts.
- The detail cache prunes expired entries instead of growing for as long
  as Shelfmark runs.
2026-09-25 21:21:54 -04:00
a05e002305 feat: narrator, series and bitrate columns for MyAnonamouse results (#1390)
# feat: narrator, series and bitrate columns for MyAnonamouse results

Addresses #605 and #934 (narrator in the release list).

## Problem

Both requests were closed because the narrator isn't in Torznab results,
which is correct. Prowlarr's MyAnonamouse indexer reads `author_info`
but drops the `narrator_info` and `series_info` MAM returns next to it,
and neither Prowlarr's `ReleaseInfo` nor the Torznab output has a field
for them. Shelfmark's size tooltip already looks for a `narrator`
Torznab attribute, but nothing ever sends one.

When a book has several narrations, choosing one means going back and
forth between Shelfmark and the tracker.

## Approach

Optional, opt-in enrichment using the user's own MAM session (`mam_id`),
the same approach AudioBookRequest's MAM indexer uses:

1. After the Prowlarr search, releases whose info URL is
`…myanonamouse.net/t/<id>` are collected.
2. Shelfmark sends the same query text to MAM's JSON search
(`/tor/js/loadSearchJSONbasic.php`, normally one request, at most 3),
and matches torrents back to Prowlarr's results by torrent ID. The MAM
origin is taken from the result URL, so a custom Prowlarr MAM base URL
is respected, and the cookie is only ever sent to `*.myanonamouse.net`.
3. Adds `extra.narrator`, `extra.series` (`The Sun Eater #1`) and
`extra.bitrate` / `extra.bitrate_value`. MAM has no bitrate field, so
it's parsed from the uploader's free-text tags (`64 kbps`) and some
releases won't have one.

Lookups are cached per torrent for an hour, respect the existing
Prowlarr search deadline, go through the configured proxy
(`get_proxies`), and never fail the search: a 403, timeout or bad JSON
is logged and the list renders without the extra details.

## Changes

- **`release_sources/prowlarr/mam.py`** (new): small MAM client
(`search`, and `get_username` for the test button), `narrator_info` /
`series_info` / tags parsing, cached best-effort
`lookup_torrent_details()`. A 403 includes MAM's reply and a note about
the IP/ASN lock.
- **`release_sources/prowlarr/source.py`**: enrichment after the result
loop. Series, Narrator and Bitrate columns only when a MAM ID is
configured, since otherwise they would be empty for every row. A Torznab
`bitrate` attribute from other indexers is also mapped to
`extra.bitrate`.
- **`release_sources/prowlarr/settings.py`**: "MyAnonamouse Enrichment"
section with `PROWLARR_MAM_ID` (password field, env-overridable like
every setting) and a **Test MAM Session** button.
- **`release_sources/__init__.py`**: `ColumnSchema` gains optional
`setting_key` and `content_types`. The new `apply_column_visibility()`
drops gated columns and their grid tracks. Neither field is serialized.
- **`main.py`**: `/api/releases` applies `apply_column_visibility()`
with the request's content type and the user's effective settings.
- **`config/settings.py` / `users_settings.py`**: Search Mode › "Release
List Columns" with `SHOW_SERIES_COLUMN`, `SHOW_NARRATOR_COLUMN` and
`SHOW_BITRATE_COLUMN`, all default on and user-overridable. Narrator and
bitrate are audiobook-only, series shows for both. AudiobookBay's
existing bitrate column now follows the bitrate toggle.
- **Frontend**: text cells truncate with a hover title; the mobile info
line wraps and skips empty text/number cells so blank optional columns
don't leave orphan `·` separators; the size tooltip no longer lists
Bitrate twice.
- **Docs**: new `docs/myanonamouse-enrichment.md` (linked from the
index), and a regenerated `environment-variables.md`. The regeneration
also picked up a few pre-existing drifts from `main` (the Libgen
section, `AA_DEFAULT_SORT` default, a duplicate `BOOK_LANGUAGE` row). I
can drop those if you'd rather keep this diff focused.

## ⚠️ MAM sessions are IP/ASN-locked

MyAnonamouse locks each session to one IP or ASN. Reusing the session
Prowlarr (or a seedbox script) uses will often **403**. **A separate MAM
session for Shelfmark will likely be needed** when Shelfmark reaches MAM
from a different IP (another host, a VPN container, or a proxy in
Shelfmark's Network settings), or when the existing session is
ASN-locked to another network. The setting's description, the error
message and the new doc all say so.

## Testing

- `tests/prowlarr/test_mam_enrichment.py` (new, 19 tests): parsing
(narrator dedupe, multiple series, missing numbers, malformed JSON, tag
bitrate), lookup (stops once all IDs are found, cache, 403 and
connection errors return empty, expired deadline skips the request),
only MAM releases enriched, Torznab bitrate mapping, column config with
and without a MAM ID for audiobook and ebook, toggles, grid-track
removal, and gates not serialized.
- `tests/core/test_admin_users_api.py`: the curated search-preference
key list now includes the three toggles.
- Full `pytest -m "not integration and not e2e"` compared with an
upstream `main` worktree on the same machine: no new failures. The
remaining ~115 failures on both are Windows-only (tor/entrypoint shell
tests, path separators).
- `ruff check` / `ruff format --check` / `basedpyright` (0 errors) /
`vulture` on touched files; frontend `tsc --noEmit`, `oxlint`, `oxfmt
--check`, `vitest` (201 passed).
- Manually verified with a real MAM account on a Docker build of this
branch: the test button, then narrator, series and bitrate on
MyAnonamouse audiobook results.

No behavior change unless `PROWLARR_MAM_ID` is set, apart from the
bitrate toggle on AudiobookBay (default on, same as today).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2026-09-25 19:17:23 -04:00
CaliBrain dce91e6972 fix(audiobookbay): reuse the resolved magnet when retrying (#1388) (#1398)
Retrying an AudiobookBay download re-scraped the detail page for the
magnet link before checking the torrent client, so a torrent that had
already finished in the client could not be imported while AudiobookBay
was down.

The handler now keeps the magnet it resolved in the task's retry
context, which is persisted with the download history, and a retry hands
that magnet straight to the client's existing-download check. The
context is handler-owned, so a client-supplied download URL cannot seed
it.

Fixes #1388
2026-09-25 19:11:56 -04:00
CaliBrain ece6b8f341 fix(auth): fail closed when auth prerequisites are missing (#1387) (#1397)
Auth mode resolution fell back to "none" (anonymous full admin) whenever
the configured method's prerequisites were missing: no local password
admin for builtin/OIDC, no Calibre-Web database, a blank proxy header,
an
unrecognized AUTH_METHOD (including "OIDC" in uppercase), or any error
while reading the config. Deleting or demoting the last local admin was
allowed on purpose because of that fallback, which exposed OIDC
instances publicly.

- Only an explicit AUTH_METHOD=none disables authentication. A
configured
  method stays active when its prerequisites are missing, so sign-in
  fails instead of opening up.
- An unrecognized or unreadable AUTH_METHOD resolves to "unavailable",
  which still requires a session and accepts no login. Values are
  normalized, so AUTH_METHOD=OIDC works.
- Restore the guard against deleting or demoting the last local password
  admin while builtin/OIDC is active (unless DISABLE_LOCAL_AUTH is set).
- Require a local admin before enabling Local auth, as OIDC already did.
- Log a recovery hint at startup when builtin/OIDC runs without a local
  admin, and document recovery via AUTH_METHOD=none.
- Drop the "will fall back to No Authentication" UI toasts and hints.



Fixes https://github.com/calibrain/shelfmark/issues/1387
2026-09-25 18:56:16 -04:00
splitsec2 37a77e9562 feat(library): mark search results already in a Calibre library (#1377)
Per discussion #1372, where you said you were fine with this specific
implementation: check whether metadata.db exists, read it if so, and
show a check mark saying the book is already there.

Searching for a book you already own gives no hint that you own it, so
the easiest way to end up with a second copy is to not remember you have
the first. This reads a Calibre `metadata.db`, read only, and marks
matching results with an **In library** badge in the card, list and
compact views and in the details dialog.

Off by default. It sits in Settings, General beside the existing Library
URL, with a test button that reports how many books it indexed. No HTTP
call, no token, nothing written back.

Matching runs most to least confident: a shared external id, then an
ISBN compared in both ISBN-10 and ISBN-13 form, then fuzzy title tokens
plus the author surname. The check fails open, so an unreadable database
degrades the badge and never blocks a search, and entries are cached for
ten minutes with an early refresh when the file changes, so a large
library costs one read rather than one per search.

`text_match.py` is new and shared by the index and the provider, so
title, author and ISBN matching stays consistent in one place.

## On the provider interface

`library_index` talks only to a `LibraryProvider` protocol and knows
nothing about Calibre. That is deliberate but it is not speculative
generality, it is what let me send you the Calibre half on its own: I
run an Audiobookshelf provider on the same interface in my fork, which
is where the audiobook side of the badge comes from. I have left that
out because it is a new service integration rather than something
already in the codebase, which is the line your non-goals draw. Happy to
send it separately if you ever want it, and equally happy for the answer
to be no.

Adding a library is a module with the `LibraryProvider` shape plus one
line in `all_providers()`.

## Verification

- `tests/core/test_library_index.py`: id, ISBN and fuzzy matching,
per-content-type provider selection, fail-open on provider errors,
stale-cache reuse, TTL and fingerprint refresh, per-provider cache
isolation, and the test-connection path including unsaved form values.
- `tests/core/test_text_match.py`: ISBN variants and token matching.
- `src/frontend/src/tests/libraryBadge.test.ts` and the added cases in
`bookTransformers.test.ts`.
- Python suite (3269) and frontend suite (206) green, plus ruff, ruff
format, basedpyright, vulture, tsc, oxlint, oxfmt and the production
build.
2026-09-25 18:18:10 -04:00
splitsec2 fe99d4bb5b feat(search): add a configurable default content type (#1371)
Closes #1018.

Worth correcting the issue first: the search tab is not hardcoded.
`useContentTypePreferences` has persisted the user's choice to
localStorage since #564, so a browser that has picked a tab already
keeps it. What is missing is the other half the issue asks for, a
default for a browser that has stored nothing, and a per-user override.

`DEFAULT_CONTENT_TYPE` is a user overridable select next to
`BOOK_LANGUAGE`, so it follows the same path: global value in Settings,
per-user value in Search Preferences, resolved with `user_id` in
`/api/config`. The frontend uses it only when this browser has no stored
choice, which is captured before the existing effect writes one, so
nothing changes for anyone who has already picked a tab.

The resolution is a pure function in `utils/contentTypePreference.ts`
rather than logic inside the hook, since vitest here has no jsdom and
the existing tests cover resolvers like `resolveDefaultLanguageCodes`
the same way.

One small move in `App.tsx`: the `config` state was declared below the
hook that now reads it, so it moved above it.

## Verification

- `tests/core/test_config_api.py`: the payload carries
`default_content_type` and reads it with the user's id.
- `tests/core/test_admin_users_api.py`: the key appears in the per-user
search preferences list.
- `src/frontend/src/tests/contentTypePreference.test.ts`: a stored tab
wins, combined mode survives, the server default applies when nothing is
stored, and an unrecognised value falls back to ebook.
- Python suite (3163) and frontend suite (201) green, plus ruff, ruff
format, basedpyright, vulture, tsc, oxlint, oxfmt and the production
build.
2026-09-25 18:14:42 -04:00
splitsec2 ce1092db7f test(auth): stop proxy provisioning tests depending on run order (#1381)
I hit this while building the Debrid-Link client in #1380. The new tests
there changed the test count, that reshuffled the xdist workers, and
these two went red. They were related to this branch, but were tests
related to another PR I did which didn't have proper tests..

`test_sets_session_from_header` and
`test_reads_remote_user_wsgi_fallback` both assert that a
proxy-authenticated user comes back with `is_admin is True`. Since #1356
that only holds for the bootstrap account, because
`_proxy_default_is_admin` returns True only while `has_admin()` is
False. Both tests assume they're provisioning the first account, and
only one of them can be.

Run with `-n 0` so nothing is sharded:

| | |
|---|---|
| either test alone | passes |
| both, file order | the second fails |
| both, reversed | the second fails |
| both, across 2 workers | both pass |

Reversing the order moving which one breaks is what makes it an ordering
problem rather than a real one.

It stays green on CI because the suite runs with `-n auto` and
`tests/conftest.py` calls `mkdtemp` at module level, which runs once per
worker process. Each worker gets its own `CONFIG_DIR` and its own
`users.db`, the two tests land on different workers, and each one is
genuinely first in its own database. That passed but it's luck rather
than design, and any change to the test count can put them back
together.

So this gives every test in the file an empty user table and stops the
question of who ran first from mattering.

While I was fixed that I added coverage for the rule itself, which I had
failed to test for:

- the bootstrap account is an admin and the next one isn't
- `PROXY_AUTH_DEFAULT_ROLE=admin` promotes later accounts
- a user already in the database keeps its stored role instead of
picking up the default

Tests only, no source changes. The full suite passes serially now, where
it had those two failures before, and it's still green under `-n auto`.
2026-09-25 18:07:46 -04:00
splitsec2 b690832659 fix(download): stream a completed book instead of buffering it in RAM - lowering memory needs significantly (#1378)
Credit where it is due: this defect was found and measured by **@DrNgo**
in
[DrNgo/shelfmark-fork@ae2185c](https://github.com/DrNgo/shelfmark-fork/commit/ae2185c8a83d537e55f1dc03f745ab844b5cdcdb).
He recorded a 493 MB audiobook peaking near 963 MB and OOM-killing a 1
GiB container, with the proxy access log showing a single `GET
/api/localdownload` returning 502 at the exact second of the kill. The
analysis is his; I am sending it because it is still open here.

Serving a completed book from the live queue calls `get_book_data`,
which reads the whole file into bytes so the route can wrap it in a
`BytesIO` for `send_file`. That is two copies of the book, with one
resident for the length of the client transfer, to hand over a file that
is already sitting on disk.

`get_book_path` returns the path the task already holds and `send_file`
streams it. The history fallback a few lines up in the same route has
always worked this way, so this makes the two paths consistent rather
than introducing anything new. `get_book_data` stays for callers that
genuinely want the bytes, now documented as the expensive option.

## The fetch side has the same problem, and this PR does not fix it

I should note: `download_url` in `shelfmark/download/http.py` still
builds the inbound file in a `BytesIO` and returns it, so a large
download is fully resident while it fetches. @DrNgo's commit fixes that
too, with a `tempfile.SpooledTemporaryFile(max_size=...)` so small
payloads stay in memory exactly as they do now and large ones spill to
disk.

I left it out deliberately. It changes the return type of `download_url`
from `BytesIO` to a file object, which touches several callers, and it
lands in a file that has been reworked around bypass handling, waiting
rooms and resume since his branch point. That deserves its own PR rather
than riding along with a two-function change. I may send it later; if
@DrNgo sends it first, his should win, and if you would rather have both
together say so and I will hold this one.

## Verification

- `tests/download/test_orchestrator_retry.py`: `get_book_path` returns
the path without opening the file (the test fails the run if it does),
and reports a missing file rather than handing back a dead path.
- The existing `/api/localdownload` tests still pass unchanged,
including the history fallback and the ownership checks.
- Full suite (3222), ruff, ruff format, basedpyright, vulture green.
2026-09-25 18:06:15 -04:00
CaliBrain e1c3f057ab fix: bypass recordings, welib wrong-md5 links, footer build sha (#1364) (#1373)
Debug screen recordings never started. Every bypass logged "Capturearea
1540x1050 at position 0.0 outside the screen size 1440x1880".We ask
ffmpeg for the fingerprint screen size plus margin, the size wealso pass
SeleniumBase as xvfb_metrics. SeleniumBase builds thatdisplay with
use_xauth=True, the image ships no xauth binary, so itfalls back to a
fixed 1440x1880 Xvfb and the requested size neverexists. Drop
-video_size so x11grab records the whole screen, whateversize it turned
out to be.

welib could hand back a link for a different book. Welib
answers/md5/<md5> with a search for that md5; when it does not have the
file,the resolver took the first "Download" on the results page
(md5a2c1dc0c... resolved to auto_download/9c8cf85d...). On an
/md5/<md5>page a GET/Download link is now only taken when its href names
thatmd5; otherwise the source is reported as not having the file.
Alsoremoves _get_download_urls_from_welib and _is_source_enabled:
themd5-template branch in _get_urls_for_source always handles welib
first,so that resolver could never run.

The footer showed the build date instead of the commit. CI
stampsBUILD_VERSION as <yyyy-mm-dd>-<sha> (pr-<sha> for PR images) and
thefooter kept its first seven characters, so dev images read
"Shelfmarkmain (2026-09)". Take the trailing commit sha instead: "main
(1a5b37d)".The full BUILD_VERSION stays in the hover title
2026-09-21 00:15:41 -04:00
CaliBrain d978896142 fix(auth): rename the API_KEY env var to SHELFMARK_API_KEY (#1374) 2026-09-21 00:10:10 -04:00
Gavin McFallandClaude Fable 5.1 3b280009ae feat(auth): static API_KEY (env) accepted as Bearer or X-Api-Key, cookie or key (#1366)
Supersedes #1353, per the discussion in #1352: one `API_KEY` environment
variable; when set, a request carrying it is authenticated as the first
admin, and cookie sessions keep working exactly as before (cookie **or**
key). Nothing else changes. No table, no UI, no settings-tab switch, no
per-user keys.

## What

- `API_KEY` (env). Unset → the feature is off and none of the new code
runs.
- `Authorization: Bearer <key>` or `X-Api-Key: <key>` on any existing
`/api/*` route authenticates that request as the first admin in
`users.db` (`ORDER BY id`), or as a bare admin identity
(`user_id="api"`, `is_admin=True`, no local user row) if the install has
no admin yet. Per request only; nothing is persisted; the admin's role
is read live, so deleting or demoting that user takes effect on the next
request.
- Both headers are checked and either may match. That is what makes the
key usable behind a reverse proxy that injects its own `Authorization`
header (oauth2-proxy, Authelia, forwardAuth): send the key in
`X-Api-Key`.
- A credential that is **not** the key is ignored and the request
continues on the normal session path, so proxy-forwarded tokens are
unaffected. Without a valid session such a request gets the usual `401
{"error": "Unauthorized"}`, identical to a request with no credential,
so there is nothing to probe.

## How

- `shelfmark/config/env.py`: `API_KEY = os.getenv("API_KEY",
"").strip()`.
- `shelfmark/core/api_key.py`: `extract_api_key_candidates()` (Bearer
token if the scheme is Bearer, then `X-Api-Key`) and `matches_api_key()`
using `hmac.compare_digest` on bytes.
- `shelfmark/core/user_db.py`: `UserDB.get_first_admin()`.
- `shelfmark/main.py`: `api_key_auth_middleware` (`before_request`,
registered before `proxy_auth_middleware`, which early-returns for keyed
requests). Only `/api/` paths; `/api/health` and `/api/auth/*` exempt;
no-op when `API_KEY` is unset or the auth mode is `none`. On a match it
mirrors the proxy-auth pattern: `session.clear()` then populate
`user_id` / `is_admin` / `db_user_id` for this request, `permanent =
False`, `modified = False`, `g.api_key_auth = True`. An `after_request`
hook guarantees no `Set-Cookie` is written for a keyed request even if a
handler dirties the session.
- `docs/api-access.md` (new), the `API_KEY` entry in
`docs/environment-variables.md`, and a README link.

## Security

- Constant-time compare; the key is never logged or echoed.
- Keyed requests never mint or refresh a session cookie and ignore any
cookie sent with them (a non-admin cookie plus the key yields admin for
that request; the browser's own session is left untouched and usable).
- The mismatch path touches neither the session nor `g`, so a stray
bearer on a browser request can neither log the user out nor change how
their cookie is refreshed.
- Store errors during the admin lookup fail closed (`500 {"error":
"Authentication error"}`), never to anonymous.
- Verified against Flask's `save_session` / `should_set_cookie`
ordering, and under auth modes `none`, `builtin`, `proxy`.

## Tests

`tests/core/test_api_key_env.py` (36): extraction and matching;
first-admin lookup; middleware behaviour on a guarded route and an admin
route, with and without a user_db, `X-Api-Key`, both-headers
combinations, no `Set-Cookie` when a handler dirties the session,
incoming non-admin cookie ignored, browser cookie still usable after a
keyed request, security headers, store error → 500, mismatch → guard's
401 / cookie path / permanent cookie untouched, unset → off, exempt
paths and path probes, `none` and `proxy` modes, deleted and demoted
first admin, a keyed write passing the guard. Existing auth suites
unchanged. All CI gates green on the fork:
https://github.com/gavinmcfall/shelfmark/pull/2 (CI-only draft).

Also exercised against a running instance: 47 scripted checks including
150 concurrent requests, proxy-mode switching through the key, an
unset-key restart, and a log scan for the key.

## Naming

`API_KEY` as discussed. If you'd rather namespace it
(`SHELFMARK_API_KEY`) to avoid clashing with other tools' env vars in
shared compose files, it is a one-line change; say the word.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 00:00:42 -04:00
splitsec2 1a5b37d9d3 fix(deluge): send seeding ratio limit under Deluge's own keys (#1367)
Deluge's per-torrent options are `stop_at_ratio` (bool) and `stop_ratio`
(float), and `torrentmanager` checks `options['stop_at_ratio'] and
get_ratio() >= options['stop_ratio']`. We were putting the indexer's
float into `stop_at_ratio`, which only switched stopping on and left the
daemon's global ratio (default 2.0) as the one actually enforced. A
`ratio_limit` of 0 turned stopping off entirely. `stop_at_ratio_enabled`
is not a Deluge option at all.

Deluge has no per-torrent seeding time limit, `seed_time_limit` is a
global core preference, so the value is logged as unapplied instead of
sent as a key the daemon drops.

qBittorrent and Transmission already honour both indexer limits, so this
removes a silent difference between clients.

## Verification

- `tests/prowlarr/test_deluge_client.py`: the ratio arrives as
`stop_ratio` with `stop_at_ratio` set, and no key Deluge does not define
is sent. Both fail on current main and pass here.
- Full suite (3165), ruff, ruff format, basedpyright, vulture green.
2026-09-20 23:06:14 -04:00