mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-11-15 17:58:11 +00:00
d8879426308a418edc1cfcd5d3ffbb8b0baffdee
134
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  splitsec2andCaliBrain
|
f837506017 |
Merge commit from fork
A signed-in user picks the release fields of POST /api/releases/download, and a record id with the "oceanofpdf:" prefix reached the provider without handles() or is_enabled(). The provider then fetched source_url, POSTed to the form action found on that page and fetched the meta-refresh target, with no check on any of them. The provider now refuses to run when it is switched off, and the book page, the form action and every POST redirect hop must be on the configured mirror (same scheme, host and port). The signed file link may sit on another host but must be http(s). network.ensure_public_destination refuses a host that resolves to a loopback, private, link-local or otherwise non-global address (IPv4-mapped IPv6 judged by its IPv4 address). Hosts the operator configured as mirrors and names that do not resolve pass as before. html_get_page and download_url call it on their first URL and check each redirect hop through a response hook. The image cache uses the shared address check too. Co-authored-by: CaliBrain <calibrain@l4n.xyz> |
||
|
splitsec2andCaliBrain
|
cb72ce807c |
Merge commit from fork
A user could set any value for their own overridable settings through
PUT /api/users/me. DESTINATION and DESTINATION_AUDIOBOOK accepted any
string, so a download (including one an admin fulfilled for that user)
could be written to any directory the container can reach, and
get_destination created it on demand.
A user's own destination must now resolve inside the global one (with
{User} expanded on both sides). Absolute paths outside it and any ".."
segment are rejected at save time, and get_destination applies the same
check when the path is used, so a stored or hand-written value outside
the global destination is ignored with a warning and the global
destination is used instead. A relative value is taken relative to the
global destination.
Other self-service values are checked as well: select fields must use
one of their options, EMAIL_RECIPIENT must be a plain address (the same
check the orchestrator and the downloads tab use, now shared from
request_helpers), and Grimmory library and path ids must be non-negative
integers. Admin saves on a user's behalf are unchanged.
The self-service 400 response now carries a message line so the reason
reaches the user in the toast.
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
|
||
|
CaliBrain
|
c269230ef3 |
fix: review regressions in metadata, notifications, activity redaction, search options and genDebug (#1510)
- **Open Library (#1451): keep the work unless the search matched a translation.** Open Library shows an English edition whenever one matches the query. So "Mistborn" returned the 2023 "Mistborn" reissue (2023) instead of "The Final Empire" (2001), and "Children of Dune" returned "Children of Dune (Dune Chronicles, Book 3)" (1987). The matched edition's title, ISBNs, cover, language and year now win only when the edition is in another language and the query doesn't name the work's own title. Checked against live Open Library: "Children of Dune" and "Mistborn" give the v1.4.0 results again, and "Der Weg der Könige" and "Kinder des Nebels" still return the German editions. - **Google Books (#1434): the ISBN cache decorator is back on `search_by_isbn`.** It had moved to the new `search_paginated`. Failed searches were cached as empty pages, and ISBN lookups lost their cache. - **Notifications: Test notification shows the plugin's failure reason again.** Apprise 2 delivers on its own worker threads, even for a single service when that service has a deadline. `_AppriseLogCapture` filtered records by thread id and dropped all of them, so the UI only said "json: delivery failed". Apprise runs those workers in a copy of the caller's context, so the capture now filters on a context variable. That still keeps concurrent dispatches apart. `AppriseAsset(async_mode=False)` would not fix this, because the sequential path also uses the worker pool when a service has a deadline. - **Activity: `activity_update` no longer sends the raw task id to non-admins.** The owner's `user_<id>` room got the raw task id. For a request-linked download that id can hold a private tracker's passkey. It now goes through `viewer_download_id`, the same mapping `/api/status` and `download_progress` use. The admins room still gets the raw id. - **SearchBar (#1476): Escape and an outside click close the homepage's inline options panel again,** as they did in v1.4.0. - **genDebug.sh: packages are listed through `importlib.metadata`,** since #1404 removed pip from the image. The script also now redacts: - `PROWLARR_MAM_ID` - `user:pass@` credentials in URLs (proxies), in both the environment and the config files - notification route URLs, which carry their service's token (Telegram, Discord and others) - `make python-checks`, `make python-test`, `make frontend-checks` and `make frontend-test` pass on top of current `main`. - New tests cover: - Open Library edition selection - Google Books caching - log capture against a real Apprise 2 delivery to a local server that answers 500, plus a concurrent dispatch on another thread - the `activity_update` payload sent to each room - genDebug redaction and the package listing Each one fails on the old code. - I checked the SearchBar fix in the running app, because the frontend has no DOM test setup: Escape closes the panel, an outside click closes it, and a click inside keeps it open. - I ran the genDebug redaction under GNU sed (Debian image) and BSD sed. The package listing ran in the pip-less image. |
||
|
splitsec2andCaliBrain
|
520b6b16d0 |
fix(activity): Clear Completed leaves out an item that is no longer finished instead of failing the batch (#1508)
Clear Completed can fail outright with "Only terminal requests can be dismissed". `dismiss-many` answers 409 and dismisses nothing when any one request or download in the batch is not terminal. The list a user clears from can be older than the server, so a request that was reopened or retried since, or a download that started again, takes the whole batch down with it. The earlier fix for pending requests (#1438) filters on the client, which only helps when the list is current. `dismiss-many` now skips such an item and carries on with the rest. The response adds `skipped_item_keys` when something was left out, and `count` is what was actually dismissed. A single `dismiss` of a non-terminal item still answers 409. I hit it with a request whose download had completed while the request itself was still pending, but any reopen between the list and the click does it. This touches the same handler as #1507, so whichever lands second needs a small rebase. Tests: a batch with a finished and a reopened request dismisses the finished one and reports the other, and a batch with a finished and a running download does the same. Ruff, basedpyright and the backend suite pass (3933 passed, e2e excluded). Co-authored-by: CaliBrain <calibrain@l4n.xyz> |
||
|
splitsec2
|
61c7ad3c62 |
Merge commit from fork
* fix(torrent): check each redirect hop when fetching a torrent file The torrent file fetch followed redirects by hand without looking at where a hop pointed, so a download link could make the server request a loopback, link-local or LAN address. Each hop (the initial URL and every redirect Location) must now be http or https and resolve only to global addresses. The configured Prowlarr/Newznab origin is exempt, as it usually lives on the LAN. The API key still only goes to that origin. * fix(socketio): only accept websocket handshakes from the app's own origin The Socket.IO server allowed every origin. It now uses the same-origin check python-engineio applies by default (Host, or X-Forwarded-Proto and X-Forwarded-Host behind a proxy). SOCKETIO_ALLOWED_ORIGINS takes a comma separated list of extra origins for proxies that change Host without forwarding it. * fix(custom-script): do not pass a file name that starts with a dash as the first argument In relative path mode the script got the organized file's relative path as its first argument, which many scripts read as an option when the name starts with a dash. Such a name is now passed with a ./ prefix. Absolute mode is unchanged. |
||
|
splitsec2
|
6f65d6e20a |
Merge commit from fork
A release format such as /../../x was appended to the staging file name unchanged, so the direct-download and libgen handlers could write to, and on cancel unlink, a path outside TMP_DIR. Normalise the format to a plain extension at queue time and in build_filename, reject path-like source ids for the sources that use them as file names, and make both handlers resolve the staging path and refuse anything that is not a direct child of TMP_DIR. |
||
|
splitsec2
|
89663e32f6 |
Merge commit from fork
A user's personal notification routes were sent to any Apprise URL, from the server. Through the test endpoint or a saved route a signed-in user could make the server POST to loopback, link-local and LAN hosts, choosing the path and, with Apprise's + and : parameters, extra headers and body fields. User routes are now limited to the hosted services that talk to a fixed vendor host (discord, slack, tgram, pover, pbul) and to ntfy and gotify, whose host must resolve only to public addresses (checked on the host urlsplit reads and on the host Apprise reads, including IPv4-mapped and numeric forms). The check runs when a route is tested, when an event is queued and again when it is sent. Admin routes are unchanged. |
||
|
splitsec2
|
d34b200414 |
fix(activity): answer another user's item like a missing one, and skip foreign history rows (#1507)
Two small fixes in `activity_routes.py`. **Another user's item answers like a missing one.** Dismissing a task or request id answered 404 when it didn't exist and 403 when it belonged to someone else, so a user could test whether an arbitrary id exists. A non-admin now gets the same 404 for both. In dismiss-many a foreign key is reported with the missing ones and the rest carry on, and the owner is still written to the log. Admins are unchanged. **A foreign row in someone's viewer state no longer gives a 500.** If a non-admin's viewer state referenced another user's download or request row, building history raised a `RuntimeError`. The row is now skipped with a warning. Not changed: history still raises when a row that viewer state points to has been deleted. That is a different case, and I didn't want to widen this. Tests: the existing forbidden test now expects the 404 and the log context, and four new ones cover a foreign download and a foreign request answering like a missing one (including dismiss-many), an admin still dismissing a foreign row, and history skipping foreign rows. Ruff, basedpyright and the backend suite pass. |
||
|
splitsec2andCaliBrain
|
8fd477bf50 |
fix(releases): apply the request policy to release listing, not only to download (#1506)
The request policy is applied when a user downloads (`/api/releases/download`) or requests (`/api/requests`), but not when they list releases. A user whose mode is `request_book` or `blocked` can still search releases and list a torrent's files through the shared Prowlarr and qBittorrent credentials. Only the UI hides the list. This applies the same gate to `/api/releases`, `/api/releases/inspect` and the release-source records route, with the same codes as the download route (`policy_blocked`, `policy_requires_request`, `required_mode`). - `blocked` and `request_book` are refused. `request_release` is allowed, because that mode lists releases so the user can pick one. The UI opens the request dialog directly under `request_book` and never calls `/api/releases` there. - `/api/releases` resolves the mode per source for the content type. A refused source is dropped from the search, so a user blocked on Prowlarr no longer reaches Prowlarr even when other sources are allowed. It answers 403 only when no source is left, or when the one source asked for is refused. - The records route doesn't know the content type before it fetches, so it refuses only when every content type the source supports is refused. - Admins, auth mode `none` and an instance with requests off bypass the gate, as they do on the download route. A user whose default is `blocked` now gets a 403 toast in the release modal, where they used to get a list of "Unavailable" rows. I made no frontend change. Tests (14, new file) cover each route for blocked, request_book, request_release, an admin, and auth mode none, the per-source drop, and the records route. Ruff, basedpyright and the backend suite pass. Co-authored-by: CaliBrain <calibrain@l4n.xyz> |
||
|
splitsec2
|
b05bbc8c58 |
fix(covers): key the cover cache by the image URL, not the caller-supplied id (#1505)
`/api/covers/<id>?url=...` cached the fetched image under the id in the path, and the caller chooses that id. A request with another book's id and a different URL put its image in that book's cache entry, and every user then got it. The key is now derived on the server from the image URL (a hash of it), and the id in the path stays only so links keep their shape. The SSRF checks and `fetch_and_cache` are unchanged, and `transform_cover_url` links already carry the URL, so they keep working. The URL is decoded first now, so a missing one is a 404 and bad encoding a 400, both before any fetch. I made the key the URL alone, not URL plus id. Either stops the overwrite, and the URL alone lets one image be shared when the same cover appears under two ids (the viewer redaction rewrites ids and still hits the cache). Entries cached under the old keys are not read any more, since there is no telling which were overwritten. They age out of the LRU and the cache warms once after the upgrade. Tests: a request with another id can't replace the legitimate image, in either order; the normal miss then hit; the same URL under two ids hits; no URL gives 404; bad encoding gives 400 without a fetch; and a unit test for the key. Ruff, basedpyright and the backend suite pass. |
||
|
splitsec2
|
71cd030484 |
Merge commit from fork
GET /api/users/me/edit-context served every user-overridable key with the admin's global value in globalValues and effective, so any user could read the global destination paths, the admin's EMAIL_RECIPIENT, the Booklore library and path ids and the global USER_NOTIFICATION_ROUTES table, whose Apprise URLs embed tokens. build_user_preferences_payload gains a for_self_service mode that replaces a global value with None unless the user set the key themselves or the field is a checkbox, number, select or multi-select switch the form needs to lay itself out (BOOKLORE_LIBRARY_ID and BOOKLORE_PATH_ID are masked despite being selects). The self-service route uses it; the admin routes keep full data. |
||
|
splitsec2
|
386fdb313c |
Merge commit from fork
GET /api/debug returns a bundle with application logs, process listings and environment names, and GET /api/restart exits the process. Both are registered only when DEBUG is on, but any logged-in user could call them. Add an admin_required decorator built on login_required and use it for both routes. Non-admin sessions get 403 "Admin access required"; authentication mode "none" keeps working as before. |
||
|
splitsec2andCaliBrain
|
77b80eb31a |
fix(metadata): only an admin may use the provider's own account, its lists and shelves (#1501)
Refs #981. Hardcover is signed in as one account, the instance owner's, and every logged-in user shares it. A non-admin can see the owner's lists and shelves in the List filter, browse one by asking for it, and add or remove books and change read status from search results, which is what #981 describes. Now only an admin may. With authentication set to none the one user keeps everything. - `GET` targets and the batch lookup return no options for a non-admin, so the buttons don't show and the account isn't asked. - `PUT` targets answers 403. - The List filter (`hardcover_list`) has no options for a non-admin, and a request that asks for it has the field dropped before the search. Providers name these fields in `account_search_fields`, and Hardcover lists `hardcover_list`. - Public things are unchanged: author, title and series suggestions, and a pasted list URL in the search box. That pasted URL is the one gap I know of. It resolves any list the account can see, so a user who knows the slug of one of the owner's private lists could still open it. Closing it means the provider has to know who is asking, which is a bigger change than this one, so I left it. Not included from the issue: a toggle for letting users view lists and limiting which lists they see. I kept this to the admin-only part so it stays small, and I can do the toggle as a follow-up if you want it. Tests cover the routes as an admin, a user and with no authentication (targets read, batch and write, list options, a crafted list search, and public fields still passing). Ruff, basedpyright and the backend suite pass. Co-authored-by: CaliBrain <calibrain@l4n.xyz> |
||
|
splitsec2
|
9ba5816b83 |
fix(requests): an older failed attempt no longer undoes a later delivery (#1493)
A request that was retried after a failure has two tasks in the status built from download history. `sync_delivery_states_from_queue_status` walks the status buckets in `QueueStatus` order and keeps the last one it sees for a request. Error comes after complete, so a request that was delivered by its retry is set back to error and, when no retry is left, reopened as pending. The next pass can then fetch the book again. When a request has more than one task, an attempt that is still running now decides its state, else a completed one, else the failure. Ties keep the old order, so a request with a single failed task still reopens as before. Tests: an earlier failure doesn't undo a later delivery, a running retry outranks an earlier failure, and a lone failure still reopens. It happened on my instance to a request whose first try failed and whose retry succeeded. Ruff and the backend suite pass. |
||
|
splitsec2
|
a1ac017487 |
feat(requests): let an admin reopen a rejected request (#1491)
Closes #1445, where you said you were open to it. A rejected request was final. `update_request` won't move a terminal status and the admin routes only fulfil or reject, so an admin who changed their mind had to ask the user to request the book again, and the user kept seeing the old rejected one. `POST /api/admin/requests/<id>/reopen` moves a rejected request back to pending and clears the review fields. What the user asked for (request level, release, note) is kept. It refuses anything that isn't rejected, and refuses when that user already has a pending request for the same book. Admins get a Reopen action on rejected cards in the activity sidebar and in History. A rejected request has usually been cleared from the list by the time anyone wants it back, so reopening also clears the dismissal, the way the failed-download reopen does. Without that the pending request stays hidden. Tests cover the service (state change, what is kept, the refusals, a non-string admin note) and the route, and the card model for the new action. Ruff, tsc, oxlint, vitest and the backend suite pass. It has been running on my instance since Oct 8. |
||
|
splitsec2
|
88cc734db2 |
fix(activity): a reused release follows the new request, and stale rows of a delivered request are closed (#1496)
Two things that go wrong when a second request picks a release a first request already used. A task id comes from the release, so the history row is reused. **The row stays with the first request.** Recording a download kept the first request's id, notes and state. On my instance a card for one book showed up under the request for a different book, with that request's note on it. Recording a download now moves the row to the request that queued it and restarts its queue time. A retry with no request keeps the request the row had. **Old rows of a delivered request show as failures forever.** The startup sweep leaves an old request-linked row alone so its retry stays, but one whose request already has its book (delivered, or another download of it finished) is now closed as "Superseded by a completed download" instead of showing as an interrupted failure for ever. Also in the first commit: History cards used the saved cover URL as it was, an external address the page's own-origin image policy blocks, while running downloads use the cover proxy. They use it too. The second commit is about restarts. After a restart nothing is in flight, but a request in downloading, resolving or locating whose row had moved to another request was stranded, because only queued was reopened. Those states are reopened too. Tests cover the row takeover and the retry case, the cover through the proxy (and a missing cover), the superseded cases (delivered request, another row finished, an undelivered request is left alone) and the in-flight states with no row. Ruff and the backend suite pass. |
||
|
splitsec2
|
06a67a7150 |
feat(download): a concurrency limit per source (#1495)
With one global limit, a few tasks that are only waiting hold every slot. A Soulseek download sitting in a peer's queue is the example I hit: three of those and a torrent that could start waits behind them. `MAX_CONCURRENT_DOWNLOADS_PER_SOURCE` (0 keeps the single global limit, which is the default). With it set, the coordinator starts a download only when its source is below the limit and passes over tasks of a full source without losing their place in the queue. `get_next` gets an optional filter for this, and a cancelled task is still dropped when one is given. The start-up stagger now applies only between downloads from the same source, and the pool is sized so every source can reach its limit. The second commit: qBittorrent keeps its own queue of torrents waiting for a free slot. The task polling one of those does no work, so it no longer counts against its source's limit and the next task of that source can start. The client does the pacing, as it does for its own queue. Tests cover the queue pass-over (order and priority kept, nothing eligible, cancelled tasks), the coordinator (a full source doesn't block the others, each source has its own limit, the global limit when unset) and the client-queue case. Ruff and the backend suite pass. |
||
|
splitsec2
|
3b5afb1a04 |
Merge commit from fork
PUT /api/users/me set a new password from the password field alone, so a live session could replace the password and lock the owner out. A builtin user now has to send current_password with it, checked against the stored hash. A wrong or missing one is a 400, and counts toward the same lockout as a failed login (a locked account gets a 429 without being checked), so the endpoint cannot be used to guess the password. A password or current_password that is not a string is a 400. A password that was not a string raised a TypeError from len() and answered 500. The account form asks for the current password when a new one is typed. Profile and settings changes without a password are unchanged. |
||
|
CaliBrain
|
10710abd4e |
fix(download): send every file of a multi-file download to the browser (#1489)
Download to Browser only ever sent the first file of a multi-file release (audiobook chapters, packs): the folder output returned final_paths[0] and /api/localdownload served that one path. The folder output now keeps every final path on the in-memory task (download_paths), and /api/localdownload streams them as one uncompressed ZIP named after the book when there is more than one. The archive is never staged on disk or held in memory, and ZIP64 covers large audiobooks. History still stores only the first path, so a re-download from history after a dismiss or restart serves a single file, as before. Also brings every linter back to green on all files: - pre-commit: bump the ruff hook to v0.17.1 to match uv.lock; ruff-format the Python examples in docs/dev and skills - ruff: fix the remaining issues in skills/shelfmark/download_books.py; format base_handler.py - basedpyright: Optional and attrs errors in databazeknih.py - knip: drop 31 unused exports, reuse ColumnColorHint in colorMaps - oxfmt 0.72: styles.css - direct download: fix the "Welib, mirrors" typo and the test that still expected the pre-OceanofPDF message Fixes #1467 |
||
  amura11andClaude Sonnet 5.5
|
51f32b374d |
feat(packs): let users choose which files, books and formats to download (#1480)
## Summary - **Skip files or books in a multi-book pack.** The pack review panel gets a switch per file and a "Skip this book" switch per book, using the shared `ToggleSwitch`. A partly skipped book shows e.g. "3 of 5 files" in the Files column. A fully skipped book is greyed out. - **Choose formats for a single book.** A new "Choose formats" header toggle, next to the multi-book one, opens a format picker for a release that has several book formats (EPUB, MOBI, PDF and so on). It has one switch per format with file count and size. Releases with one format, or that can't be inspected, download as before. - **Skipped files are sent as `ignored_files`.** They are stored on the task (including restart-safe retry) and removed before grouping and transfer. This matters for packs because files missing from `book_plan` would otherwise be filed by the heuristic fallback. Ignored entries match by release-relative path, then path suffix, then unique basename. - A pack that ends up with one book after skips keeps the titles you reviewed instead of falling back to the searched title. ## Screenshots <img width="1277" height="413" alt="Screenshot 2026-10-19 at 09-39-28 Shelfmark" src="https://github.com/user-attachments/assets/c005ca4b-3dac-45dd-b2ff-d02fc0f9fcc9" /> <img width="1277" height="1210" alt="Screenshot 2026-10-19 at 09-40-23 Shelfmark" src="https://github.com/user-attachments/assets/2145f3ba-e5fc-4635-9364-1790feb0c8b8" /> <img width="1277" height="748" alt="Screenshot 2026-10-19 at 09-39-03 Shelfmark" src="https://github.com/user-attachments/assets/39023fac-70e3-46d8-af89-1160ebd6e415" /> ## Test plan - [x] `pytest` pack tests (new `drop_ignored_files` cases added) - [x] `vitest`, `tsc`, `oxlint`, `oxfmt`, `ruff` - [x] Manual check in the UI with a real multi-book pack 🤖 (mostly) Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
CaliBrain
|
3c5cfcb77d |
fix/redact log secrets (#1488)
- fix(logging): mask tracker API keys in log output (#1486) - fix(welib): follow slow_download links through the countdown (#1447) |
||
 Amalkrishna A S
|
b38e4e4a00 |
feat(naming): add {AuthorSort} template variable (#1484)
Closes #1178. Adds `{AuthorSort}`, the first author surname-first ("Arthur Conan Doyle" → "Doyle, Arthur Conan"), for rename and organize templates, alongside `{FirstAuthor}`. - A value already in "Last, First" form (one word before the comma) is kept as is; Jr/Sr/II/III/IV stay at the end. - Known limits, same as calibre's default: the last word is treated as the surname, so particles ("Le Guin") and surname-first names aren't detected. - Added to the settings help, env docs, the frontend variable picker/preview, and tests. |
||
 HeihoffJandClaude Opus 5.5
|
756bdec19b |
fix(metadata): show the translation a search matched on Open Library and Hardcover (#1451)
Searching a translated title on Open Library or Hardcover returned the original-language work. "Der Weg der Könige" came back as "The Way of Kings" (2010), and that English title then went into the release search and the download name. Both providers keep one record per work, titled after the original, and list translations underneath it. #483 already searches releases per language with Hardcover's localized titles, which fixed the release side of #360 when the matching language is selected. This PR fixes the search result itself, so the book a user picks is the translation they searched for, and it covers Open Library, the default provider, which has no titles_by_language. - Open Library: request the matched edition (editions.*) and prefer its title, ISBNs, cover, language and publication year when its title differs from the work's. - Hardcover: use the alternative title named in the hit's highlights (alternative_titles, best index first), without the original's subtitle. A hit whose title also matched keeps the work title. - /api/releases: when the modal sends a title that differs from the book record's, it becomes search_title. Before, Hardcover's search_title from the original won, so picking "Kinder des Nebels" still searched "The Final Empire" first. - Release cache: keyed on title and language as well as the provider ID. All translations share that ID, so opening "The Way of Kings" after "Der Weg der Könige" showed the German releases until a reload. Known limitations: neither provider has descriptions per edition, so the description stays the work's (usually English). Hardcover's search hits carry no year or language for alternative titles, so those stay the work's; Open Library's do not. Google Books is unaffected, since it already returns one volume per edition. Tests: Open Library edition parsing, Hardcover alternative-title matching, the /api/releases title override, and the release cache, each written to fail first. The Python suite passes apart from the 9 test_entrypoint_permissions.py tests, which fail the same way on an unmodified main on macOS. The frontend typecheck, lint, format check and unit tests pass. Checked on a live instance with Open Library, Hardcover and Google Books. Refs #360, #483 Disclaimer: This was developed with the help of AI but reviewed and tested as well as deployed on my server by a real human. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
splitsec2
|
f0f0d46381 |
fix(queue): first terminal status wins, so a late COMPLETE cannot overwrite a cancel (#1429)
Kept small on purpose so it's easy to review. I have a few more small fixes like this from going back over recently merged PRs. Would you rather get them as separate PRs like this one, or combined into one? #1361 made `cancel_download` write under the queue lock, which stops a cancel from overwriting a download that finished first. The other direction is still open. The worker checks `cancel_flag.is_set()` and then calls `update_status(COMPLETE)` a few lines later in `_process_single_download`. If a cancel lands between those two lines the task goes CANCELLED, then the worker writes COMPLETE over it. `_apply_status_locked` allows terminal to terminal moves, so the terminal hook also fires a second time and records two events for one task. This makes the first terminal status win: `_apply_status_locked` ignores a different terminal status once the task already has one. Requeueing from a terminal status still works because QUEUED isn't terminal. I replaced the existing race test. It patched `queue.update_status`, which `cancel_download` no longer calls, so it never exercised the path it names, and its last assertion only looked at the final hook event. The new tests call the queue directly: complete then cancel is refused, cancel then complete stays cancelled with one hook event, error then complete stays error, and a terminal task can still be requeued. The two late-write tests fail on current `main`. Ruff and basedpyright are clean. `tests/core` and `tests/download` pass apart from 4 `seleniumbase` import failures in my environment that also fail on `main`. The one red check is the existing clock-dependent test `test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner that booted recently. It is fixed in #1426 and is not caused by this change. |
||
|
splitsec2
|
65dbf42a86 |
fix(release-search): report a ValueError from a source as that source's error (#1433)
Kept small on purpose so it's easy to review. This is the same kind of small fix as the ones in #1429 to #1432, so see the question in #1429 about separate PRs versus combined. I'm sending them separately until I hear otherwise. In `search_source_releases` the `except ValueError` comes before the branch that handles the operational errors, and `ValueError` is in that tuple. A `ValueError` from `build_release_search_plan` or `source.search()` is therefore reported as `Unknown source: <name>` instead of `<name>: <message>`. The only `ValueError` meant for that branch is the one `get_source` raises for a name that isn't registered, so that call now has its own `try` and the rest goes back to the operational errors branch. The failure test already covered `OSError` and `RuntimeError`, so I added `ValueError` to it, and that case fails on current `main`. Ruff, basedpyright and `tests/core` pass. The one red check is the existing clock-dependent test `test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner that booted recently. It is fixed in #1426 and is not caused by this change. |
||
|
splitsec2
|
93f58caf7a |
fix(debug): keep SHELFMARK_API_KEY_READONLY out of the debug bundle (#1432)
Kept small on purpose so it's easy to review. `genDebug.sh` filters secrets out of `environment.txt` by matching `_KEY=`, `_SECRET=`, `_PASSWORD=` and `_TOKEN=`, which only catches names that end in those words. `SHELFMARK_API_KEY_READONLY` ends in `_READONLY`, so its value lands in the file, and people attach these bundles to issues. That key only reads counters, so the impact is small, but any variable like `DB_PASSWORD_FILE` would leak the same way. The pattern now matches the secret word anywhere in the variable name. The new test runs the real filter line from the script against a made-up environment, and it fails on the old pattern. The one red check is the existing clock-dependent test `test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner that booted recently. It is fixed in #1426 and is not caused by this change. |
||
|
splitsec2
|
e8f6505bea |
fix(library): back off after a failed index instead of retrying for every result (#1430)
Kept small on purpose so it's easy to review. `library_index._entries_for` retries `fetch_entries` on every call while the index is failing. The lookup runs once per search result, so with the Calibre path unreadable a page of 40 results makes 40 attempts and logs 40 warnings. The entrypoint runs one gevent worker and the Calibre read is synchronous sqlite, so each attempt also holds up every other request. After a failure it now answers from the stale cache (or an empty list) for 60 seconds before trying again, and a successful index clears that. The new tests cover 40 lookups against a failing library with a stale cache (one failed attempt, one warning), a failing library with no cache (tried once), and a retry once the 60 seconds have passed. The first and second fail on current `main`. I left a few other things in this module out to keep this one small: re-indexing blocks the worker for as long as a big library takes (I haven't measured how long), `test_connection` stores entries from the unsaved form path under the shared cache key, and a title with a collection word in it, like "Books of Blood", shows as "In a collection". I can send any of those separately. Ruff and basedpyright are clean and `tests/core` passes. The one red check is the existing clock-dependent test `test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner that booted recently. It is fixed in #1426 and is not caused by this change. |
||
|
splitsec2
|
755f28a6a4 |
feat(api): add a read-only API key and a /api/stats endpoint (#1419)
This is the read-only API key from #1410, where you said to go ahead. I run Shelfmark behind Homepage and wanted more on the dashboard tile than up or down, without putting an admin API key in the dashboard's config. This adds a second key, `SHELFMARK_API_KEY_READONLY`, that can read one new endpoint. `GET /api/stats` returns counts only: books added over the last 7 and 30 days by format, the queue, requests by outcome, and download failures over the last 7 days. No titles and no user names. The read-only key, the admin key or an admin session can read it. The read-only key gets a 403 on every other path and on any write, including a POST to `/api/stats` itself, and it never gets a session or a cookie. If a request carries both keys, the admin one wins, so a proxy's own `Authorization` header can't downgrade a correct `X-Api-Key`. Unset means off, like the existing key. The endpoint exists either way, but without a key only an admin session can read it. It also adds a `db_path` property on `UserDB`, so the stats module can open the database read-only. Tests cover the key scope, the 403s for the read-only key, both keys at once, who can reach the endpoint, and the counters. The full suite passes, and I broke each rule on purpose to check a test catches it. A version of this has been running on my own install behind Homepage. If you'd rather have the stats in a different shape or under a different path, I'm happy to change it. |
||
 adman234andClaude Opus 5.5
|
32abaff21e |
feat(naming): {Narrator} placeholder and MyAnonamouse series fallback (#1407)
## Human-Written explanation:
Following up on previous PR to use MAM ID to get nararrator and series
to show up in search, this PR will allow the series and nararrator
fields to be added to the path when saving an audiobook.
I have tested my ghcr.io image on my instance and it seemed to work
properly.
All the text below is written by Claude.
# feat(naming): {Narrator} placeholder and MyAnonamouse series fallback
Follow-up to #1390 and #1399. Related to #605 and #934 (narrator
support).
## Problem
Several narrations of the same audiobook currently land in the same
folder, because nothing in the path template tells them apart. Keeping
more than one version means renaming folders by hand after every
download.
Since #1390, MyAnonamouse results carry the release's narrator and
series in `extra`, but the naming templates can't use them: there is no
`{Narrator}` placeholder, and `{Series}` / `{SeriesPosition}` only come
from the metadata provider.
## Changes
- **`{Narrator}` placeholder** (`core/naming.py`,
`download/postprocess/transfer.py`, `core/models.py`): `DownloadTask`
gains `narrator`, read from `narrator` or `extra.narrator` when a
release is queued, and kept in the restart-safe retry payload. It's
available in audiobook templates like any other placeholder, including
prefix/suffix blocks such as `{ - Narrator}`.
- **Audiobookshelf folder style**: Audiobookshelf reads the narrator
from folder names like `Title {Narrator}`. The template `{Title}
{{Narrator}}` already matches as `{` + `{Narrator}`; the parser now also
drops the closing brace when the narrator is empty, so the folder is
`Title` rather than `Title }`.
- **No stray spaces around `/`**: an empty placeholder at the start or
end of a folder name no longer leaves a space there (e.g. `Title
/Title`). This applies to all templates.
- **Series fallback** (`release_sources/prowlarr/mam.py`, `source.py`,
`download/orchestrator.py`): MAM enrichment also stores the first
series' name and number as `extra.series_name` /
`extra.series_position`, which queueing already falls back to when the
metadata provider has no series. The provider's series still wins. A
release-level series number is only used when it names the same series
as the provider, so a provider series is never paired with the number of
a different MAM series (an omnibus, for example).
- **Settings and preview** (`config/settings.py`,
`namingTemplatePreview.ts`): the audiobook template descriptions list
`{Narrator}` and explain `{{Narrator}}`. The settings page preview
mirrors the parser changes and offers `{Narrator}` under "Insert
variable" for audiobook templates only.
- **Docs**: regenerated `environment-variables.md`. This also picked up
the Library Check settings, which weren't in the generated docs yet;
happy to drop that part to keep the diff focused.
## Example
Audiobook Path Template `{Author}/{Title} {{Narrator}}/{Title}`:
| Narrator | Result |
|----------|--------|
| Samuel Roukin | `Christopher Ruocchio/Empire of Silence {Samuel
Roukin}/Empire of Silence.m4b` |
| none | `Christopher Ruocchio/Empire of Silence/Empire of Silence.m4b`
|
## Testing
- `tests/core/test_narrator_template_variable.py` (new, 16 tests):
`{{Narrator}}` with and without a value, the `{ {Narrator}}` prefix
form, prefix/suffix blocks, sanitizing, `build_library_path`, task
metadata, retry payload round trip, MAM series name/number parsing
(including a `1-3` range having no number), and queueing (narrator from
`extra`, series fallback, no cross-series number, blank narrator).
- `namingTemplatePreview.test.ts`: the same `{{Narrator}}` cases for the
preview.
- Updated `test_generate_env_docs.py` for the new template description.
- Full `pytest -m "not integration and not e2e"` compared with an
upstream `main` worktree on the same machine: no new failures (the
remaining ones are Windows-only on both).
- `ruff check`, `ruff format --check`, `basedpyright` (0 errors) and
`vulture` on touched files; frontend `tsc --noEmit`, `oxlint`, `oxfmt
--check`, `vitest` (218 passed).
- Checked the settings page locally: the audiobook preview renders `...
{Simon Vance}.mp3` and `{Narrator}` is listed for audiobook templates
only.
No behavior change for existing templates, apart from spaces next to `/`
being trimmed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
CaliBrain
|
119d3374e3 | fix(downloads): reconcile downloads a restart interrupted (#1423) | ||
|
splitsec2andCaliBrain
|
8392b43dd3 |
fix(downloads): reconcile downloads a restart interrupted (#1409)
I found this while grabbing new books and working on my fork. I redeployed the container while downloads were running, and it left a mess in my queue. It's an edge case, but a few rows were stuck until I cleared them by hand. The download queue lives in memory, so stopping the process with a download in flight leaves its history row "active" and its request "queued", with nothing working on either. The activity API shows that row as "Interrupted" when it's read, but nothing is stored and nothing looks at it again. Upstream only reopens an errored request when no manual retry is offered, and an interrupted row always offers one, so the request just sits there. This closes those rows out once at startup, before the coordinator starts, as an error with the message "Interrupted". The retry payload stays, so the manual retry still works. A request whose download was queued in the last 24 hours goes back to pending through `reopen_failed_request`. Older ones get closed out but their requests are left alone, because I didn't think a startup should reopen a backlog or fill an admin's approval queue with old requests. A request that already arrived is left alone, and running it twice changes nothing. It assumes a single worker process, which `entrypoint.sh` already sets. On my install the first start after deploying it closed 53 rows that had been orphaned for nine days. Tests are in `tests/core/test_startup_reconcile.py`, and the full suite passes locally. If the 24 hour window feels off, or you'd rather handle this another way, I'm happy to change it or drop the PR. Co-authored-by: CaliBrain <calibrain@l4n.xyz> |
||
|
CaliBrain
|
bac0b93566 |
fix: keep https:// for qBittorrent and hide release sources from non-admins (#1422)
qBittorrent over HTTPS (#1417) qbittorrent-api ignores the scheme in `host` and probes HTTP and HTTPS itself. When the HTTPS probe fails, for instance on a certificate Shelfmark doesn't trust, it falls back to plain HTTP against the TLS port, and a reverse proxy answers "400 The plain HTTP request was sent to HTTPS port". The download client and the Test Connection button now pass FORCE_SCHEME_FROM_HOST for https:// URLs, so the configured scheme is used as-is and a certificate problem is reported as one. http:// and bare host:port URLs keep the probe: normalization adds http:// to a bare host, and the probe follows a proxy's redirect to HTTPS. Release sources leaked to non-admins (#1418) A download queued from an approved request carries the release an admin picked, and the requester could read it from their own activity feed: the request's release_data (source_id, indexer, info URL, torrent attributes), the download's full server path, and the download id itself, which for Prowlarr is "<indexer id>:<guid>" and for a private tracker is a URL into it. That id keys every status payload, the /api/localdownload query and the cover proxy URL, so hiding release_data alone would not have closed the leak. For non-admin viewers: - request rows keep only the release fields the activity cards display - download_path is reduced to the file name, which the browser download reveals anyway - request-linked downloads are addressed by an opaque id (a keyed HMAC of the task id) in the snapshot, history, /api/status, queue order, active downloads, dismissed keys, cover URLs and the websocket status and progress events. Routes that take a download id translate it back, searching only the caller's own downloads. Downloads a user queued directly keep their real id: the user picked that release from search results that already showed it, and the release list matches its buttons to the queue by that id. Admin views are unchanged. The activity sidebar now links a fulfilled request to its download by request_id instead of release_data.source_id. Closes #1417 Closes #1418 |
||
|
|
a05e002305 |
feat: narrator, series and bitrate columns for MyAnonamouse results (#1390)
# feat: narrator, series and bitrate columns for MyAnonamouse results Addresses #605 and #934 (narrator in the release list). ## Problem Both requests were closed because the narrator isn't in Torznab results, which is correct. Prowlarr's MyAnonamouse indexer reads `author_info` but drops the `narrator_info` and `series_info` MAM returns next to it, and neither Prowlarr's `ReleaseInfo` nor the Torznab output has a field for them. Shelfmark's size tooltip already looks for a `narrator` Torznab attribute, but nothing ever sends one. When a book has several narrations, choosing one means going back and forth between Shelfmark and the tracker. ## Approach Optional, opt-in enrichment using the user's own MAM session (`mam_id`), the same approach AudioBookRequest's MAM indexer uses: 1. After the Prowlarr search, releases whose info URL is `…myanonamouse.net/t/<id>` are collected. 2. Shelfmark sends the same query text to MAM's JSON search (`/tor/js/loadSearchJSONbasic.php`, normally one request, at most 3), and matches torrents back to Prowlarr's results by torrent ID. The MAM origin is taken from the result URL, so a custom Prowlarr MAM base URL is respected, and the cookie is only ever sent to `*.myanonamouse.net`. 3. Adds `extra.narrator`, `extra.series` (`The Sun Eater #1`) and `extra.bitrate` / `extra.bitrate_value`. MAM has no bitrate field, so it's parsed from the uploader's free-text tags (`64 kbps`) and some releases won't have one. Lookups are cached per torrent for an hour, respect the existing Prowlarr search deadline, go through the configured proxy (`get_proxies`), and never fail the search: a 403, timeout or bad JSON is logged and the list renders without the extra details. ## Changes - **`release_sources/prowlarr/mam.py`** (new): small MAM client (`search`, and `get_username` for the test button), `narrator_info` / `series_info` / tags parsing, cached best-effort `lookup_torrent_details()`. A 403 includes MAM's reply and a note about the IP/ASN lock. - **`release_sources/prowlarr/source.py`**: enrichment after the result loop. Series, Narrator and Bitrate columns only when a MAM ID is configured, since otherwise they would be empty for every row. A Torznab `bitrate` attribute from other indexers is also mapped to `extra.bitrate`. - **`release_sources/prowlarr/settings.py`**: "MyAnonamouse Enrichment" section with `PROWLARR_MAM_ID` (password field, env-overridable like every setting) and a **Test MAM Session** button. - **`release_sources/__init__.py`**: `ColumnSchema` gains optional `setting_key` and `content_types`. The new `apply_column_visibility()` drops gated columns and their grid tracks. Neither field is serialized. - **`main.py`**: `/api/releases` applies `apply_column_visibility()` with the request's content type and the user's effective settings. - **`config/settings.py` / `users_settings.py`**: Search Mode › "Release List Columns" with `SHOW_SERIES_COLUMN`, `SHOW_NARRATOR_COLUMN` and `SHOW_BITRATE_COLUMN`, all default on and user-overridable. Narrator and bitrate are audiobook-only, series shows for both. AudiobookBay's existing bitrate column now follows the bitrate toggle. - **Frontend**: text cells truncate with a hover title; the mobile info line wraps and skips empty text/number cells so blank optional columns don't leave orphan `·` separators; the size tooltip no longer lists Bitrate twice. - **Docs**: new `docs/myanonamouse-enrichment.md` (linked from the index), and a regenerated `environment-variables.md`. The regeneration also picked up a few pre-existing drifts from `main` (the Libgen section, `AA_DEFAULT_SORT` default, a duplicate `BOOK_LANGUAGE` row). I can drop those if you'd rather keep this diff focused. ## ⚠️ MAM sessions are IP/ASN-locked MyAnonamouse locks each session to one IP or ASN. Reusing the session Prowlarr (or a seedbox script) uses will often **403**. **A separate MAM session for Shelfmark will likely be needed** when Shelfmark reaches MAM from a different IP (another host, a VPN container, or a proxy in Shelfmark's Network settings), or when the existing session is ASN-locked to another network. The setting's description, the error message and the new doc all say so. ## Testing - `tests/prowlarr/test_mam_enrichment.py` (new, 19 tests): parsing (narrator dedupe, multiple series, missing numbers, malformed JSON, tag bitrate), lookup (stops once all IDs are found, cache, 403 and connection errors return empty, expired deadline skips the request), only MAM releases enriched, Torznab bitrate mapping, column config with and without a MAM ID for audiobook and ebook, toggles, grid-track removal, and gates not serialized. - `tests/core/test_admin_users_api.py`: the curated search-preference key list now includes the three toggles. - Full `pytest -m "not integration and not e2e"` compared with an upstream `main` worktree on the same machine: no new failures. The remaining ~115 failures on both are Windows-only (tor/entrypoint shell tests, path separators). - `ruff check` / `ruff format --check` / `basedpyright` (0 errors) / `vulture` on touched files; frontend `tsc --noEmit`, `oxlint`, `oxfmt --check`, `vitest` (201 passed). - Manually verified with a real MAM account on a Docker build of this branch: the test button, then narrator, series and bitrate on MyAnonamouse audiobook results. No behavior change unless `PROWLARR_MAM_ID` is set, apart from the bitrate toggle on AudiobookBay (default on, same as today). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: CaliBrain <calibrain@l4n.xyz> |
||
|
CaliBrain
|
ece6b8f341 |
fix(auth): fail closed when auth prerequisites are missing (#1387) (#1397)
Auth mode resolution fell back to "none" (anonymous full admin) whenever the configured method's prerequisites were missing: no local password admin for builtin/OIDC, no Calibre-Web database, a blank proxy header, an unrecognized AUTH_METHOD (including "OIDC" in uppercase), or any error while reading the config. Deleting or demoting the last local admin was allowed on purpose because of that fallback, which exposed OIDC instances publicly. - Only an explicit AUTH_METHOD=none disables authentication. A configured method stays active when its prerequisites are missing, so sign-in fails instead of opening up. - An unrecognized or unreadable AUTH_METHOD resolves to "unavailable", which still requires a session and accepts no login. Values are normalized, so AUTH_METHOD=OIDC works. - Restore the guard against deleting or demoting the last local password admin while builtin/OIDC is active (unless DISABLE_LOCAL_AUTH is set). - Require a local admin before enabling Local auth, as OIDC already did. - Log a recovery hint at startup when builtin/OIDC runs without a local admin, and document recovery via AUTH_METHOD=none. - Drop the "will fall back to No Authentication" UI toasts and hints. Fixes https://github.com/calibrain/shelfmark/issues/1387 |
||
|
splitsec2
|
37a77e9562 |
feat(library): mark search results already in a Calibre library (#1377)
Per discussion #1372, where you said you were fine with this specific implementation: check whether metadata.db exists, read it if so, and show a check mark saying the book is already there. Searching for a book you already own gives no hint that you own it, so the easiest way to end up with a second copy is to not remember you have the first. This reads a Calibre `metadata.db`, read only, and marks matching results with an **In library** badge in the card, list and compact views and in the details dialog. Off by default. It sits in Settings, General beside the existing Library URL, with a test button that reports how many books it indexed. No HTTP call, no token, nothing written back. Matching runs most to least confident: a shared external id, then an ISBN compared in both ISBN-10 and ISBN-13 form, then fuzzy title tokens plus the author surname. The check fails open, so an unreadable database degrades the badge and never blocks a search, and entries are cached for ten minutes with an early refresh when the file changes, so a large library costs one read rather than one per search. `text_match.py` is new and shared by the index and the provider, so title, author and ISBN matching stays consistent in one place. ## On the provider interface `library_index` talks only to a `LibraryProvider` protocol and knows nothing about Calibre. That is deliberate but it is not speculative generality, it is what let me send you the Calibre half on its own: I run an Audiobookshelf provider on the same interface in my fork, which is where the audiobook side of the badge comes from. I have left that out because it is a new service integration rather than something already in the codebase, which is the line your non-goals draw. Happy to send it separately if you ever want it, and equally happy for the answer to be no. Adding a library is a module with the `LibraryProvider` shape plus one line in `all_providers()`. ## Verification - `tests/core/test_library_index.py`: id, ISBN and fuzzy matching, per-content-type provider selection, fail-open on provider errors, stale-cache reuse, TTL and fingerprint refresh, per-provider cache isolation, and the test-connection path including unsaved form values. - `tests/core/test_text_match.py`: ISBN variants and token matching. - `src/frontend/src/tests/libraryBadge.test.ts` and the added cases in `bookTransformers.test.ts`. - Python suite (3269) and frontend suite (206) green, plus ruff, ruff format, basedpyright, vulture, tsc, oxlint, oxfmt and the production build. |
||
|
splitsec2
|
fe99d4bb5b |
feat(search): add a configurable default content type (#1371)
Closes #1018. Worth correcting the issue first: the search tab is not hardcoded. `useContentTypePreferences` has persisted the user's choice to localStorage since #564, so a browser that has picked a tab already keeps it. What is missing is the other half the issue asks for, a default for a browser that has stored nothing, and a per-user override. `DEFAULT_CONTENT_TYPE` is a user overridable select next to `BOOK_LANGUAGE`, so it follows the same path: global value in Settings, per-user value in Search Preferences, resolved with `user_id` in `/api/config`. The frontend uses it only when this browser has no stored choice, which is captured before the existing effect writes one, so nothing changes for anyone who has already picked a tab. The resolution is a pure function in `utils/contentTypePreference.ts` rather than logic inside the hook, since vitest here has no jsdom and the existing tests cover resolvers like `resolveDefaultLanguageCodes` the same way. One small move in `App.tsx`: the `config` state was declared below the hook that now reads it, so it moved above it. ## Verification - `tests/core/test_config_api.py`: the payload carries `default_content_type` and reads it with the user's id. - `tests/core/test_admin_users_api.py`: the key appears in the per-user search preferences list. - `src/frontend/src/tests/contentTypePreference.test.ts`: a stored tab wins, combined mode survives, the server default applies when nothing is stored, and an unrecognised value falls back to ebook. - Python suite (3163) and frontend suite (201) green, plus ruff, ruff format, basedpyright, vulture, tsc, oxlint, oxfmt and the production build. |
||
|
CaliBrain
|
d978896142 | fix(auth): rename the API_KEY env var to SHELFMARK_API_KEY (#1374) | ||
 Gavin McFallandClaude Fable 5.1
|
3b280009ae |
feat(auth): static API_KEY (env) accepted as Bearer or X-Api-Key, cookie or key (#1366)
Supersedes #1353, per the discussion in #1352: one `API_KEY` environment variable; when set, a request carrying it is authenticated as the first admin, and cookie sessions keep working exactly as before (cookie **or** key). Nothing else changes. No table, no UI, no settings-tab switch, no per-user keys. ## What - `API_KEY` (env). Unset → the feature is off and none of the new code runs. - `Authorization: Bearer <key>` or `X-Api-Key: <key>` on any existing `/api/*` route authenticates that request as the first admin in `users.db` (`ORDER BY id`), or as a bare admin identity (`user_id="api"`, `is_admin=True`, no local user row) if the install has no admin yet. Per request only; nothing is persisted; the admin's role is read live, so deleting or demoting that user takes effect on the next request. - Both headers are checked and either may match. That is what makes the key usable behind a reverse proxy that injects its own `Authorization` header (oauth2-proxy, Authelia, forwardAuth): send the key in `X-Api-Key`. - A credential that is **not** the key is ignored and the request continues on the normal session path, so proxy-forwarded tokens are unaffected. Without a valid session such a request gets the usual `401 {"error": "Unauthorized"}`, identical to a request with no credential, so there is nothing to probe. ## How - `shelfmark/config/env.py`: `API_KEY = os.getenv("API_KEY", "").strip()`. - `shelfmark/core/api_key.py`: `extract_api_key_candidates()` (Bearer token if the scheme is Bearer, then `X-Api-Key`) and `matches_api_key()` using `hmac.compare_digest` on bytes. - `shelfmark/core/user_db.py`: `UserDB.get_first_admin()`. - `shelfmark/main.py`: `api_key_auth_middleware` (`before_request`, registered before `proxy_auth_middleware`, which early-returns for keyed requests). Only `/api/` paths; `/api/health` and `/api/auth/*` exempt; no-op when `API_KEY` is unset or the auth mode is `none`. On a match it mirrors the proxy-auth pattern: `session.clear()` then populate `user_id` / `is_admin` / `db_user_id` for this request, `permanent = False`, `modified = False`, `g.api_key_auth = True`. An `after_request` hook guarantees no `Set-Cookie` is written for a keyed request even if a handler dirties the session. - `docs/api-access.md` (new), the `API_KEY` entry in `docs/environment-variables.md`, and a README link. ## Security - Constant-time compare; the key is never logged or echoed. - Keyed requests never mint or refresh a session cookie and ignore any cookie sent with them (a non-admin cookie plus the key yields admin for that request; the browser's own session is left untouched and usable). - The mismatch path touches neither the session nor `g`, so a stray bearer on a browser request can neither log the user out nor change how their cookie is refreshed. - Store errors during the admin lookup fail closed (`500 {"error": "Authentication error"}`), never to anonymous. - Verified against Flask's `save_session` / `should_set_cookie` ordering, and under auth modes `none`, `builtin`, `proxy`. ## Tests `tests/core/test_api_key_env.py` (36): extraction and matching; first-admin lookup; middleware behaviour on a guarded route and an admin route, with and without a user_db, `X-Api-Key`, both-headers combinations, no `Set-Cookie` when a handler dirties the session, incoming non-admin cookie ignored, browser cookie still usable after a keyed request, security headers, store error → 500, mismatch → guard's 401 / cookie path / permanent cookie untouched, unset → off, exempt paths and path probes, `none` and `proxy` modes, deleted and demoted first admin, a keyed write passing the guard. Existing auth suites unchanged. All CI gates green on the fork: https://github.com/gavinmcfall/shelfmark/pull/2 (CI-only draft). Also exercised against a running instance: 47 scripted checks including 150 concurrent requests, proxy-mode switching through the key, an unset-key restart, and a log scan for the key. ## Naming `API_KEY` as discussed. If you'd rather namespace it (`SHELFMARK_API_KEY`) to avoid clashing with other tools' env vars in shared compose files, it is a one-line change; say the word. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
splitsec2
|
ab3aa9a8b0 |
fix(requests): reject non-object items in the batch endpoint (#1369)
`POST /api/requests/batch` checks that `requests` is a non-empty list and then hands each element to the shared preparation helper, which calls `.get()` on it. A bare string, number or null in the list raises `AttributeError` and the caller gets a 500, while `POST /api/requests` answers 400 with a message for the same mistake. This validates the element type beside the existing list check. One bad item rejects the whole batch rather than being reported per item, which matches the endpoint's current contract: every other failure path already aborts the batch with a single error body, as `test_batch_create_requests_is_atomic` asserts. Responses for valid payloads are unchanged. ## Verification - `tests/core/test_request_routes_api.py`: a case per bad shape (int, str, null, list), plus a mixed valid and invalid batch that asserts nothing was created. All fail on current main with a 500 and pass here. - Full suite (3168), ruff, ruff format, basedpyright, vulture green. |
||
|
splitsec2
|
c1315a2b23 |
fix(download): check task ownership before serving queued files (#1357)
`/api/localdownload` resolves the file through the live queue and returns it before checking who owns the task; the owner check only runs on the download-history fallback, once the task has aged out of the queue. Task ids are source ids, so two users who searched the same book can end up with the same id. This applies the same rule on the queue path, reusing the `_task_owned_by_actor` helper the cancel/retry/priority routes already use, so both paths answer a non-owner with the same 404. Admin behaviour is unchanged. The 404 matches what this endpoint's history path already returns for a non-owner rather than the 403 `download_not_owned` the cancel/retry/priority routes use, happy to switch it if you prefer consistency with the siblings instead. ## Verification - `tests/core/test_activity_routes_api.py`: the owner still receives their queued file; a different user receives 404. The new case fails on current main and passes here; the existing history-fallback test is unchanged. - Full suite (3148), ruff, ruff format, basedpyright, vulture green. |
||
|
splitsec2
|
7934924678 |
fix(queue): don't stamp CANCELLED over a finished download (#1361)
`cancel_download` reads the task status under the queue lock, releases it, and only then writes CANCELLED through `update_status`. A download that finishes in that window has its COMPLETE overwritten. The queue and the UI show the task as cancelled while the file is already on disk, and the terminal hook fires for both statuses. The check and the write now happen in a single lock hold. Because the lock is non-reentrant and the terminal hook has to run after it is released (the stall canceller depends on that), the lock-held part of `update_status` moved into a small private helper that both paths share; `update_status` is a thin wrapper over it. A cancel arriving once the task is already terminal still returns `False`. ## Verification - `tests/core/test_queue.py::test_cancel_does_not_overwrite_a_download_that_finished_first`: a worker thread completes the download while the cancel is in flight, with the handover driven by events rather than sleeps. The task stays complete. Fails on main, passes here. - Full suite (3147), plus `tests/download/` and `tests/core/test_download_api_guardrails.py`, ruff, ruff format, basedpyright, vulture green. |
||
|
splitsec2
|
a6204a318e |
fix(oidc): reject backslash paths in the return_to sanitizer (#1359)
The OIDC `return_to` sanitizer rejects values starting with `//` and then relies on `urlsplit` to catch anything carrying a netloc. A value such as `/\host` has no netloc, so it is stored in the session and used as the post-login redirect target and browsers resolve the backslash as a path separator, which lands the user outside the app after a successful login. `_normalize_return_to` now also rejects values whose path contains a backslash. That matches the frontend sanitizer in `authRedirect.ts`, which parses with `URL` and already discards those forms, so the two ends agree again. The check covers the path only, so query and fragment backslashes still round-trip, and it also catches the script-root case where `/app/\host` strips to `/\host`. ## Verification - New cases in `tests/core/test_oidc_routes.py` cover the rejected forms, including under a script root, and confirm `/`, `/settings` and `/search?q=x#frag` are unaffected. They fail on current main and pass here. - Full suite (3155), ruff, ruff format, basedpyright, vulture green. |
||
|
splitsec2
|
545480c557 |
fix(download): default is_admin to False in the request policy guard (#1358)
`_resolve_policy_mode_for_current_user` reads `session.get("is_admin",
True)`, so a session carrying `user_id` but no `is_admin` key skips the
request policy entirely, while every other admin check in the codebase
defaults the key to `False`.
This uses the same default here. Every authenticated login path
(builtin, CWA, proxy, OIDC) writes `is_admin` into the session, and
`AUTH_METHOD=none` is already short-circuited one line earlier, so
sessions from those flows behave exactly as before.
## Verification
- `tests/core/test_request_routes_api.py::TestDownloadPolicyGuards`: a
session without `is_admin` now gets `policy_requires_request` and
nothing is queued. Fails on current main, passes here.
- Full suite (3147), ruff, ruff format, basedpyright, vulture green.
|
||
|
splitsec2
|
127dd82615 |
fix(users): apply user updates only after the payload validates (#1360)
`PUT /api/users/me` and `PUT /api/admin/users/<id>` write the new password hash, and then the profile fields, before the rest of the payload is checked. When the request is rejected further down as an invalid role, an admin-only setting, an invalid settings value, the route answers 400 with those writes already committed, so the caller sees an error while the password has in fact changed. Both routes now validate the whole payload before touching the database, and the password hash is folded into the same `update_user` call as the other fields so the field write is a single transaction. Error messages, status codes and the order they are reported in are unchanged. ## Verification - New tests in `tests/core/test_self_user_routes.py` and `tests/core/test_admin_users_api.py` assert that a rejected update leaves the password, profile fields and role as they were, plus a positive case that a valid payload still applies all three. They fail on current main and pass here. - Full suite (3150), ruff, ruff format, basedpyright, vulture green. |
||
|
splitsec2
|
acd59f7cbb |
feat(auth): provision proxy users as non-admin once an admin exists (#1356)
With `AUTH_METHOD=proxy` and no admin group configured, every user the proxy authenticates for the first time is provisioned as an admin (`is_admin = True` unless the user already exists in `users.db`). The intent to never lock an instance out makes sense, but the effect is that anyone the SSO gate lets through becomes an administrator. On an instance shared with family or a small community that is a footgun; I hit it when the first invited reader landed as an admin. This keeps the guarantee and removes the footgun: the first account is still provisioned as an admin while the instance has no admin at all, and later first-time users follow a new `PROXY_AUTH_DEFAULT_ROLE` setting (Security tab / env), default `user`. Known users keep their stored role; the `PROXY_AUTH_ADMIN_GROUP_NAME` path is unchanged and still takes precedence. I couldn't find a way with Cloudflare access to pass this along. Changes: `UserDB.has_admin()`, `_proxy_default_is_admin()` in the proxy middleware, the new `SelectField` beside the other proxy settings, the regenerated `docs/environment-variables.md` entry and a row in `docs/reverse-proxy.md`. Compatibility: the default moves from "everyone admin" to "first admin, then users". Accounts already in `users.db` are unaffected; new SSO users on an existing instance become regular users unless `PROXY_AUTH_DEFAULT_ROLE=admin` is set. If you would rather ship this purely opt-in I can flip the default to `admin`. ## Verification - `tests/core/test_auth_api.py::TestProxyProvisioningRole`: first user admin / second user not; `PROXY_AUTH_DEFAULT_ROLE=admin` restores the old behaviour; an admin from another auth source counts as "an admin exists"; a known user keeps their role whatever the default. - Full suite (3094), ruff, ruff format, basedpyright, vulture green. - Running on my own instance since 2026-09-19. |
||
 splitsec2andInfiniteAvenger
|
44f4e13cce |
refactor: extract the per-source release search out of /api/releases (#1355)
The `/api/releases` route carries an inner `_search_source_releases` helper that builds the search plan for one source, logs the planned query type, runs the search and turns `SourceUnavailableError`/operational errors into an error message instead of raising. Anything outside the route that wants to search one source with exactly those semantics has to go through Flask today. This moves that helper into `shelfmark/core/release_search.py` as `search_source_releases()` and has the route delegate to it. Behaviour is unchanged: same plan construction (including the caller's `user_id`, so per-user default languages still apply), same logging, same error-to-message handling. It is the refactor half of #1047 by @InfiniteAvenger, split out on its own as you asked for other PRs (#1318). Their authorship is preserved on the commit; I rebased it onto current `main` and added tests. ## Verification - `tests/core/test_release_search.py`: unknown source → `"Unknown source: …"`, `SourceUnavailableError` and operational errors → `"<source>: <error>"`, success path forwards `expand_search` / `content_type` and returns the source instance, the plan receives languages / manual query / indexers / `user_id`. (These tests are type-annotated; happy to strip the annotations if you prefer the suite's bare style.) - Full suite, ruff, ruff format, basedpyright, vulture green; the existing `/api/releases` route tests are unchanged and pass. Co-authored-by: InfiniteAvenger <calebewest02@gmail.com> |
||
|
CaliBrain
|
e5dd34ae0e |
fix: unbreak main and follow up on the Blackhole handoff review (#1346)
DownloadHistoryService.record_download and updated the single production caller, but not the eleven in the test suite, leaving main red with 32 failures. Pass None, which is what the pre-#1336 behaviour recorded. For the Blackhole handoff (#1345): add_download publishes the torrent before the cancel check runs, and BlackholeClient.remove() is a no-op, so the watcher picks the file up regardless. Reporting a bare "Cancelled" hid that from the user. Name the completed handoff in the cancellation message instead, drop the _handle_cancelled_download call whose usenet branch cannot apply to a handoff-only client, and record why the orchestrator no longer verifies HandoffResult.path. Finally, make tests/direct_download a package: test_libgen_extract.py imports tests.libgen.sample_html across test directories, so without an __init__.py pytest named its modules by bare basename and a same-named module elsewhere would collide. |
||
 Vinicius Gabriel
|
c53545d9fe |
Add configurable word separator for naming templates (#1333)
Closes #1230 ## What Adds a "Word Separator" setting (Space / Dot / Underscore / Hyphen / Custom) that replaces internal whitespace in each naming-template placeholder's rendered value — e.g. `{Author}` renders "Arthur.Conan.Doyle" instead of "Arthur Conan Doyle" when Dot is selected. This follows option 2 from the issue rather than inventing new dotted-keyword template syntax (`{Author.}`), since it's a smaller surface: one setting applies uniformly across all four templates (books/audiobooks × rename/organize) instead of needing a parallel token for every existing one. ## How it works - Literal characters typed into the template itself (e.g. the `.` in `{Author}.-.{Title}`) are never touched — only whitespace *inside* a placeholder's resolved value is affected. - Default is "Space", which is a no-op: existing templates produce byte-identical output after this change (verified via the existing test suite, unmodified, still passing). ## Where - `shelfmark/core/naming.py` — `word_separator` param on `parse_naming_template` / `build_library_path`. - `shelfmark/download/postprocess/policy.py` — `get_word_separator()`, mirroring the existing `get_file_organization()` accessor. - `shelfmark/download/postprocess/transfer.py` — wires the resolved separator through the four existing template-rendering call sites. - `shelfmark/config/settings.py` — new `Word Separator` / `Custom Word Separator` fields next to the existing naming-template fields. - `src/frontend/.../namingTemplatePreview.ts` + `NamingTemplateField.tsx` — the settings UI has its own TS mirror of the Python renderer for the live preview; updated it in lockstep so the preview doesn't lie about what the separator will actually do. - Tests added on both sides (pytest + vitest). ## Testing - `uv run pytest tests/core/test_naming.py tests/core/test_destination_file_organization.py` — all pass, including new cases. - `uv run pytest` (full suite) — same pre-existing failures as on `main` before this change (browser/network-dependent bypass & e2e tests unrelated to this diff), everything else green. - `uv run ruff check` / `ruff format --check` / `basedpyright` — clean. - `npm run lint` / `format:check` / `typecheck` / `test:unit` (196 tests) — clean. |
||
 Zoltán Szabó
|
35037b35fd |
fix(irc): search by surname, and rank the answer by author (#1331) (#1332)
Fixes #1331. A search bot ANDs every term against a filename, so the given name is the term that empties the result set. Measured against irchighway's #ebooks: "Revelations David Petrie" is answered "no results", "Revelations Petrie" returns 9 matches, 6 of which parse, all filed as "D Petrie". The query now carries the title and the surname, read off the search variant so the ISBN fallback and a manual query - which set author="" on purpose - keep their current shape. Title-only, the shape #1295 settled on for Prowlarr, does not transfer: the bot caps an answer at 1000 matches, and a bare "Revelations" hits that cap with 923 parsed rows across 500 authors, so the cap itself can drop the wanted book. The surname is the token the two spellings share and it keeps the answer small. The full author then orders what comes back, reusing author_affinity from #1295, since a surname also matches a different author who shares it. It sits under server availability the way indexer priority does in #1295: a download addresses one named bot and waits 120s for it, so a match from a bot that has left the channel must not outrank a mismatch that can answer. Ranking runs on the way out rather than before the cache, because one query identity is shared by every book that produced that query. Two things found while testing: - The parser writes the literal "Unknown" when a filename has no " - " separator (parser.py:168). Ranked literally that sorts as a wrong author, so author_affinity's middle tier was unreachable here; it is now read as absent. 5 of those 923 rows are affected. - author_affinity moves to shelfmark/core/author_match.py, unchanged, so IRC does not import from the Prowlarr package. Prowlarr behaviour is untouched and its tests pass as they are. The three IRC assertions in the #1252 regression file move to the surname form. The invariant they pin - one contributor's name reaches the query, never the whole credit list - is unchanged. Tested with make python-lint, python-format, python-dead-code, python-typecheck and python-test, and end to end against irchighway with the patched source: it posts "Revelations Petrie" and returns 6 releases. |
||
 oliver
|
1e3fd48b8b |
fix: share rotating log file handlers (#1316)
This patch shares (for each log file) the `RotatingFileHandler` for logging across all modules, reducing the number of open file descriptors from ~78 to 1. I had originally assumed this issue was a resource leak, but it seems to just be a large fixed number of file descriptors. So this change mostly just (1) shrinks the number of open file descriptors to a reasonable level and (2) prevents two modules in the same process competing to write to a log file. |