mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-10-04 08:31:11 +01:00
main
51
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 splitsec2
|
cad18b8019 |
feat(clients): add Debrid-Link as a torrent client (#1380)
calibrain said in #1315 that he'd review and merge a Debrid-Link client if someone took a stab at it, so here it is. It's the fourth debrid service alongside AllDebrid, Real-Debrid and TorBox and it uses the same DownloadClient interface as the other three. No frontend changes, the settings UI picks it up from the backend field definitions. **I can't test this against the live API.** I don't use debrid and I don't have a Debrid-Link key. Everything here is built from their v2 API docs and cross-checked against two independent reference implementations, so the request shapes should be right, but nobody has run a real magnet through it yet. @Tatsu941 you opened the issue and mentioned they give developers free premium. Would you be able to try it? The things most worth confirming are the field names on a seedbox torrent, that `premiumLeft` is the right premium check, and that an API error comes back the way the client expects. Happy to fix whatever turns up. Two things about their API shaped how this is written. Every v2 endpoint wraps its payload in a `{success, value}` envelope and a failure can arrive with an HTTP 200, so `_request_value` decides the outcome from the body instead of the status code. A completed seedbox torrent already carries a direct `downloadUrl` on every file. The other three clients each need a separate unrestrict or link-request call per file and this one doesn't, which is why it came out shorter than them despite doing the same job. Endpoints used: | Endpoint | Used for | |---|---| | `GET account/infos` | test the key, confirm premium time remains | | `POST seedbox/add` | magnet as JSON, .torrent as multipart | | `GET seedbox/list` | poll progress | | `DELETE seedbox/{id}/remove` | clean up | Ruff and basedpyright are clean and the 36 new tests pass. The two failures in `tests/e2e/test_proxy_auth_middleware.py` are already on main and fail the same way on a pristine checkout. They assume the first proxy-authenticated user becomes an admin, which only holds while the user table has no admin in it, so they depend on how many users earlier tests created. One thing for review: the download loop uses `download_url` like the other three clients, which holds each file in memory. I kept the existing pattern rather than do something different in a new-provider PR, but say so if you'd rather it streamed here. Closes #1315 |
||
|
splitsec2
|
755f28a6a4 |
feat(api): add a read-only API key and a /api/stats endpoint (#1419)
This is the read-only API key from #1410, where you said to go ahead. I run Shelfmark behind Homepage and wanted more on the dashboard tile than up or down, without putting an admin API key in the dashboard's config. This adds a second key, `SHELFMARK_API_KEY_READONLY`, that can read one new endpoint. `GET /api/stats` returns counts only: books added over the last 7 and 30 days by format, the queue, requests by outcome, and download failures over the last 7 days. No titles and no user names. The read-only key, the admin key or an admin session can read it. The read-only key gets a 403 on every other path and on any write, including a POST to `/api/stats` itself, and it never gets a session or a cookie. If a request carries both keys, the admin one wins, so a proxy's own `Authorization` header can't downgrade a correct `X-Api-Key`. Unset means off, like the existing key. The endpoint exists either way, but without a key only an admin session can read it. It also adds a `db_path` property on `UserDB`, so the stats module can open the database read-only. Tests cover the key scope, the 403s for the read-only key, both keys at once, who can reach the endpoint, and the counters. The full suite passes, and I broke each rule on purpose to check a test catches it. A version of this has been running on my own install behind Homepage. If you'd rather have the stats in a different shape or under a different path, I'm happy to change it. |
||
  adman234andClaude Opus 5.5
|
32abaff21e |
feat(naming): {Narrator} placeholder and MyAnonamouse series fallback (#1407)
## Human-Written explanation:
Following up on previous PR to use MAM ID to get nararrator and series
to show up in search, this PR will allow the series and nararrator
fields to be added to the path when saving an audiobook.
I have tested my ghcr.io image on my instance and it seemed to work
properly.
All the text below is written by Claude.
# feat(naming): {Narrator} placeholder and MyAnonamouse series fallback
Follow-up to #1390 and #1399. Related to #605 and #934 (narrator
support).
## Problem
Several narrations of the same audiobook currently land in the same
folder, because nothing in the path template tells them apart. Keeping
more than one version means renaming folders by hand after every
download.
Since #1390, MyAnonamouse results carry the release's narrator and
series in `extra`, but the naming templates can't use them: there is no
`{Narrator}` placeholder, and `{Series}` / `{SeriesPosition}` only come
from the metadata provider.
## Changes
- **`{Narrator}` placeholder** (`core/naming.py`,
`download/postprocess/transfer.py`, `core/models.py`): `DownloadTask`
gains `narrator`, read from `narrator` or `extra.narrator` when a
release is queued, and kept in the restart-safe retry payload. It's
available in audiobook templates like any other placeholder, including
prefix/suffix blocks such as `{ - Narrator}`.
- **Audiobookshelf folder style**: Audiobookshelf reads the narrator
from folder names like `Title {Narrator}`. The template `{Title}
{{Narrator}}` already matches as `{` + `{Narrator}`; the parser now also
drops the closing brace when the narrator is empty, so the folder is
`Title` rather than `Title }`.
- **No stray spaces around `/`**: an empty placeholder at the start or
end of a folder name no longer leaves a space there (e.g. `Title
/Title`). This applies to all templates.
- **Series fallback** (`release_sources/prowlarr/mam.py`, `source.py`,
`download/orchestrator.py`): MAM enrichment also stores the first
series' name and number as `extra.series_name` /
`extra.series_position`, which queueing already falls back to when the
metadata provider has no series. The provider's series still wins. A
release-level series number is only used when it names the same series
as the provider, so a provider series is never paired with the number of
a different MAM series (an omnibus, for example).
- **Settings and preview** (`config/settings.py`,
`namingTemplatePreview.ts`): the audiobook template descriptions list
`{Narrator}` and explain `{{Narrator}}`. The settings page preview
mirrors the parser changes and offers `{Narrator}` under "Insert
variable" for audiobook templates only.
- **Docs**: regenerated `environment-variables.md`. This also picked up
the Library Check settings, which weren't in the generated docs yet;
happy to drop that part to keep the diff focused.
## Example
Audiobook Path Template `{Author}/{Title} {{Narrator}}/{Title}`:
| Narrator | Result |
|----------|--------|
| Samuel Roukin | `Christopher Ruocchio/Empire of Silence {Samuel
Roukin}/Empire of Silence.m4b` |
| none | `Christopher Ruocchio/Empire of Silence/Empire of Silence.m4b`
|
## Testing
- `tests/core/test_narrator_template_variable.py` (new, 16 tests):
`{{Narrator}}` with and without a value, the `{ {Narrator}}` prefix
form, prefix/suffix blocks, sanitizing, `build_library_path`, task
metadata, retry payload round trip, MAM series name/number parsing
(including a `1-3` range having no number), and queueing (narrator from
`extra`, series fallback, no cross-series number, blank narrator).
- `namingTemplatePreview.test.ts`: the same `{{Narrator}}` cases for the
preview.
- Updated `test_generate_env_docs.py` for the new template description.
- Full `pytest -m "not integration and not e2e"` compared with an
upstream `main` worktree on the same machine: no new failures (the
remaining ones are Windows-only on both).
- `ruff check`, `ruff format --check`, `basedpyright` (0 errors) and
`vulture` on touched files; frontend `tsc --noEmit`, `oxlint`, `oxfmt
--check`, `vitest` (218 passed).
- Checked the settings page locally: the audiobook preview renders `...
{Simon Vance}.mp3` and `{Narrator}` is listed for audiobook templates
only.
No behavior change for existing templates, apart from spaces next to `/`
being trimmed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
  Evan KazakinandEvan Kazakin
|
7888217380 |
feat(download): add Remove & Delete Files torrent completion action (#1401)
Closes #1027 ## Summary - add a **Remove & Delete Files** (`remove_and_delete`) option to the Torrent Completion Action setting - after a successful import, remove the torrent and ask the client to delete its downloaded data (`delete_files=True`), matching what the usenet "move" flow already does - clarify the setting description so it says **Remove** keeps the downloaded files (raised in the issue comments) - regenerate the `PROWLARR_TORRENT_ACTION` entry in `docs/environment-variables.md` ## Behavior The deletion runs from `post_process_cleanup`, the same place as the existing Remove and Change Category actions, so it only happens after output transfer and post-processing have succeeded. At that point every file has already been copied or hardlinked into the library, and transfer size checks have passed. A failed import never removes or deletes anything. The torrent client deletes its own data, so Shelfmark does not delete paths itself and remote path mappings are not involved. Keep, Remove, and Change Category behave as before, and Keep is still the default. If a request matched a completed torrent that was already in the client, this option deletes that torrent's data too. That is the same scope the existing Remove action already applies to. ## Validation - `make python-checks` (ruff check, ruff format, basedpyright on backend and tests, vulture): passed - `make python-test`: 3363 passed, 5 skipped, 9 failed. All 9 are in `tests/config/test_entrypoint_permissions.py` and happen because macOS `/bin/bash` 3.2 does not support `${1,,}` in `entrypoint.sh`. They do not touch this change. - new tests in `tests/prowlarr/test_handler.py`: - Remove passes `delete_files=False` and Remove & Delete Files passes `delete_files=True` - a failed import with Remove & Delete Files does not call the client - the delete case fails without the handler change - manual smoke run: `PROWLARR_TORRENT_ACTION=remove_and_delete` set through real config loading, with a stub client that deletes its folder. The torrent folder was deleted and the hardlinked library file stayed intact. - pre-commit hooks (prek) passed --------- Co-authored-by: Evan Kazakin <evan@Evans-MacBook-Pro.local> |
||
|
a05e002305 |
feat: narrator, series and bitrate columns for MyAnonamouse results (#1390)
# feat: narrator, series and bitrate columns for MyAnonamouse results Addresses #605 and #934 (narrator in the release list). ## Problem Both requests were closed because the narrator isn't in Torznab results, which is correct. Prowlarr's MyAnonamouse indexer reads `author_info` but drops the `narrator_info` and `series_info` MAM returns next to it, and neither Prowlarr's `ReleaseInfo` nor the Torznab output has a field for them. Shelfmark's size tooltip already looks for a `narrator` Torznab attribute, but nothing ever sends one. When a book has several narrations, choosing one means going back and forth between Shelfmark and the tracker. ## Approach Optional, opt-in enrichment using the user's own MAM session (`mam_id`), the same approach AudioBookRequest's MAM indexer uses: 1. After the Prowlarr search, releases whose info URL is `…myanonamouse.net/t/<id>` are collected. 2. Shelfmark sends the same query text to MAM's JSON search (`/tor/js/loadSearchJSONbasic.php`, normally one request, at most 3), and matches torrents back to Prowlarr's results by torrent ID. The MAM origin is taken from the result URL, so a custom Prowlarr MAM base URL is respected, and the cookie is only ever sent to `*.myanonamouse.net`. 3. Adds `extra.narrator`, `extra.series` (`The Sun Eater #1`) and `extra.bitrate` / `extra.bitrate_value`. MAM has no bitrate field, so it's parsed from the uploader's free-text tags (`64 kbps`) and some releases won't have one. Lookups are cached per torrent for an hour, respect the existing Prowlarr search deadline, go through the configured proxy (`get_proxies`), and never fail the search: a 403, timeout or bad JSON is logged and the list renders without the extra details. ## Changes - **`release_sources/prowlarr/mam.py`** (new): small MAM client (`search`, and `get_username` for the test button), `narrator_info` / `series_info` / tags parsing, cached best-effort `lookup_torrent_details()`. A 403 includes MAM's reply and a note about the IP/ASN lock. - **`release_sources/prowlarr/source.py`**: enrichment after the result loop. Series, Narrator and Bitrate columns only when a MAM ID is configured, since otherwise they would be empty for every row. A Torznab `bitrate` attribute from other indexers is also mapped to `extra.bitrate`. - **`release_sources/prowlarr/settings.py`**: "MyAnonamouse Enrichment" section with `PROWLARR_MAM_ID` (password field, env-overridable like every setting) and a **Test MAM Session** button. - **`release_sources/__init__.py`**: `ColumnSchema` gains optional `setting_key` and `content_types`. The new `apply_column_visibility()` drops gated columns and their grid tracks. Neither field is serialized. - **`main.py`**: `/api/releases` applies `apply_column_visibility()` with the request's content type and the user's effective settings. - **`config/settings.py` / `users_settings.py`**: Search Mode › "Release List Columns" with `SHOW_SERIES_COLUMN`, `SHOW_NARRATOR_COLUMN` and `SHOW_BITRATE_COLUMN`, all default on and user-overridable. Narrator and bitrate are audiobook-only, series shows for both. AudiobookBay's existing bitrate column now follows the bitrate toggle. - **Frontend**: text cells truncate with a hover title; the mobile info line wraps and skips empty text/number cells so blank optional columns don't leave orphan `·` separators; the size tooltip no longer lists Bitrate twice. - **Docs**: new `docs/myanonamouse-enrichment.md` (linked from the index), and a regenerated `environment-variables.md`. The regeneration also picked up a few pre-existing drifts from `main` (the Libgen section, `AA_DEFAULT_SORT` default, a duplicate `BOOK_LANGUAGE` row). I can drop those if you'd rather keep this diff focused. ## ⚠️ MAM sessions are IP/ASN-locked MyAnonamouse locks each session to one IP or ASN. Reusing the session Prowlarr (or a seedbox script) uses will often **403**. **A separate MAM session for Shelfmark will likely be needed** when Shelfmark reaches MAM from a different IP (another host, a VPN container, or a proxy in Shelfmark's Network settings), or when the existing session is ASN-locked to another network. The setting's description, the error message and the new doc all say so. ## Testing - `tests/prowlarr/test_mam_enrichment.py` (new, 19 tests): parsing (narrator dedupe, multiple series, missing numbers, malformed JSON, tag bitrate), lookup (stops once all IDs are found, cache, 403 and connection errors return empty, expired deadline skips the request), only MAM releases enriched, Torznab bitrate mapping, column config with and without a MAM ID for audiobook and ebook, toggles, grid-track removal, and gates not serialized. - `tests/core/test_admin_users_api.py`: the curated search-preference key list now includes the three toggles. - Full `pytest -m "not integration and not e2e"` compared with an upstream `main` worktree on the same machine: no new failures. The remaining ~115 failures on both are Windows-only (tor/entrypoint shell tests, path separators). - `ruff check` / `ruff format --check` / `basedpyright` (0 errors) / `vulture` on touched files; frontend `tsc --noEmit`, `oxlint`, `oxfmt --check`, `vitest` (201 passed). - Manually verified with a real MAM account on a Docker build of this branch: the test button, then narrator, series and bitrate on MyAnonamouse audiobook results. No behavior change unless `PROWLARR_MAM_ID` is set, apart from the bitrate toggle on AudiobookBay (default on, same as today). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: CaliBrain <calibrain@l4n.xyz> |
||
|
splitsec2
|
fe99d4bb5b |
feat(search): add a configurable default content type (#1371)
Closes #1018. Worth correcting the issue first: the search tab is not hardcoded. `useContentTypePreferences` has persisted the user's choice to localStorage since #564, so a browser that has picked a tab already keeps it. What is missing is the other half the issue asks for, a default for a browser that has stored nothing, and a per-user override. `DEFAULT_CONTENT_TYPE` is a user overridable select next to `BOOK_LANGUAGE`, so it follows the same path: global value in Settings, per-user value in Search Preferences, resolved with `user_id` in `/api/config`. The frontend uses it only when this browser has no stored choice, which is captured before the existing effect writes one, so nothing changes for anyone who has already picked a tab. The resolution is a pure function in `utils/contentTypePreference.ts` rather than logic inside the hook, since vitest here has no jsdom and the existing tests cover resolvers like `resolveDefaultLanguageCodes` the same way. One small move in `App.tsx`: the `config` state was declared below the hook that now reads it, so it moved above it. ## Verification - `tests/core/test_config_api.py`: the payload carries `default_content_type` and reads it with the user's id. - `tests/core/test_admin_users_api.py`: the key appears in the per-user search preferences list. - `src/frontend/src/tests/contentTypePreference.test.ts`: a stored tab wins, combined mode survives, the server default applies when nothing is stored, and an unrecognised value falls back to ebook. - Python suite (3163) and frontend suite (201) green, plus ruff, ruff format, basedpyright, vulture, tsc, oxlint, oxfmt and the production build. |
||
|
CaliBrain
|
d978896142 | fix(auth): rename the API_KEY env var to SHELFMARK_API_KEY (#1374) | ||
 Gavin McFallandClaude Fable 5.1
|
3b280009ae |
feat(auth): static API_KEY (env) accepted as Bearer or X-Api-Key, cookie or key (#1366)
Supersedes #1353, per the discussion in #1352: one `API_KEY` environment variable; when set, a request carrying it is authenticated as the first admin, and cookie sessions keep working exactly as before (cookie **or** key). Nothing else changes. No table, no UI, no settings-tab switch, no per-user keys. ## What - `API_KEY` (env). Unset → the feature is off and none of the new code runs. - `Authorization: Bearer <key>` or `X-Api-Key: <key>` on any existing `/api/*` route authenticates that request as the first admin in `users.db` (`ORDER BY id`), or as a bare admin identity (`user_id="api"`, `is_admin=True`, no local user row) if the install has no admin yet. Per request only; nothing is persisted; the admin's role is read live, so deleting or demoting that user takes effect on the next request. - Both headers are checked and either may match. That is what makes the key usable behind a reverse proxy that injects its own `Authorization` header (oauth2-proxy, Authelia, forwardAuth): send the key in `X-Api-Key`. - A credential that is **not** the key is ignored and the request continues on the normal session path, so proxy-forwarded tokens are unaffected. Without a valid session such a request gets the usual `401 {"error": "Unauthorized"}`, identical to a request with no credential, so there is nothing to probe. ## How - `shelfmark/config/env.py`: `API_KEY = os.getenv("API_KEY", "").strip()`. - `shelfmark/core/api_key.py`: `extract_api_key_candidates()` (Bearer token if the scheme is Bearer, then `X-Api-Key`) and `matches_api_key()` using `hmac.compare_digest` on bytes. - `shelfmark/core/user_db.py`: `UserDB.get_first_admin()`. - `shelfmark/main.py`: `api_key_auth_middleware` (`before_request`, registered before `proxy_auth_middleware`, which early-returns for keyed requests). Only `/api/` paths; `/api/health` and `/api/auth/*` exempt; no-op when `API_KEY` is unset or the auth mode is `none`. On a match it mirrors the proxy-auth pattern: `session.clear()` then populate `user_id` / `is_admin` / `db_user_id` for this request, `permanent = False`, `modified = False`, `g.api_key_auth = True`. An `after_request` hook guarantees no `Set-Cookie` is written for a keyed request even if a handler dirties the session. - `docs/api-access.md` (new), the `API_KEY` entry in `docs/environment-variables.md`, and a README link. ## Security - Constant-time compare; the key is never logged or echoed. - Keyed requests never mint or refresh a session cookie and ignore any cookie sent with them (a non-admin cookie plus the key yields admin for that request; the browser's own session is left untouched and usable). - The mismatch path touches neither the session nor `g`, so a stray bearer on a browser request can neither log the user out nor change how their cookie is refreshed. - Store errors during the admin lookup fail closed (`500 {"error": "Authentication error"}`), never to anonymous. - Verified against Flask's `save_session` / `should_set_cookie` ordering, and under auth modes `none`, `builtin`, `proxy`. ## Tests `tests/core/test_api_key_env.py` (36): extraction and matching; first-admin lookup; middleware behaviour on a guarded route and an admin route, with and without a user_db, `X-Api-Key`, both-headers combinations, no `Set-Cookie` when a handler dirties the session, incoming non-admin cookie ignored, browser cookie still usable after a keyed request, security headers, store error → 500, mismatch → guard's 401 / cookie path / permanent cookie untouched, unset → off, exempt paths and path probes, `none` and `proxy` modes, deleted and demoted first admin, a keyed write passing the guard. Existing auth suites unchanged. All CI gates green on the fork: https://github.com/gavinmcfall/shelfmark/pull/2 (CI-only draft). Also exercised against a running instance: 47 scripted checks including 150 concurrent requests, proxy-mode switching through the key, an unset-key restart, and a log scan for the key. ## Naming `API_KEY` as discussed. If you'd rather namespace it (`SHELFMARK_API_KEY`) to avoid clashing with other tools' env vars in shared compose files, it is a one-line change; say the word. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
splitsec2
|
acd59f7cbb |
feat(auth): provision proxy users as non-admin once an admin exists (#1356)
With `AUTH_METHOD=proxy` and no admin group configured, every user the proxy authenticates for the first time is provisioned as an admin (`is_admin = True` unless the user already exists in `users.db`). The intent to never lock an instance out makes sense, but the effect is that anyone the SSO gate lets through becomes an administrator. On an instance shared with family or a small community that is a footgun; I hit it when the first invited reader landed as an admin. This keeps the guarantee and removes the footgun: the first account is still provisioned as an admin while the instance has no admin at all, and later first-time users follow a new `PROXY_AUTH_DEFAULT_ROLE` setting (Security tab / env), default `user`. Known users keep their stored role; the `PROXY_AUTH_ADMIN_GROUP_NAME` path is unchanged and still takes precedence. I couldn't find a way with Cloudflare access to pass this along. Changes: `UserDB.has_admin()`, `_proxy_default_is_admin()` in the proxy middleware, the new `SelectField` beside the other proxy settings, the regenerated `docs/environment-variables.md` entry and a row in `docs/reverse-proxy.md`. Compatibility: the default moves from "everyone admin" to "first admin, then users". Accounts already in `users.db` are unaffected; new SSO users on an existing instance become regular users unless `PROXY_AUTH_DEFAULT_ROLE=admin` is set. If you would rather ship this purely opt-in I can flip the default to `admin`. ## Verification - `tests/core/test_auth_api.py::TestProxyProvisioningRole`: first user admin / second user not; `PROXY_AUTH_DEFAULT_ROLE=admin` restores the old behaviour; an admin from another auth source counts as "an admin exists"; a known user keeps their role whatever the default. - Full suite (3094), ruff, ruff format, basedpyright, vulture green. - Running on my own instance since 2026-09-19. |
||
 Marcelo Rodrigo
|
b7002a6eca |
feat: Add TorBox client support and settings integration (#1342)
Add **TorBox** as a torrent download client for Prowlarr releases. Users can select `TorBox` in the download client settings, configure it with the new `TORBOX_API_KEY` environment variable, and verify their credentials with the connection test button. The integration supports both magnet links and `.torrent` files. It tracks the torrent lifecycle through TorBox, downloads supported book and audiobook files from the TorBox CDN, preserves safe nested file paths, and cleans up remote and local download state. Important: Shared HTTP download logs omit full download URLs and URL-bearing exception text to avoid exposing credentials, following best practices. This applies to all clients that use the shared `download_url()` path; URLs remain available to the HTTP operations themselves. --- There is already related work in progress in #1173, which includes both torrent and direct-download support for TorBox. This PR is not intended to replace or compete with that contribution. It offers the tested torrent client functionality as a smaller, focused change that can make TorBox available to the community sooner. The direct-download integration proposed in #1173 remains valuable and could be reviewed or introduced separately. Automated tests cover configuration, connection validation, magnet and torrent-file submission, API errors, status and progress handling, file retrieval, path traversal protection, cancellation, cleanup, and sensitive URL redaction. I also validated the complete flow locally with several magnet links and `.torrent` downloads. TorBox processed the torrents and Shelfmark downloaded the resulting files as expected. AI was used to help with the implementation, with human validation. This PR and long description? Took me some good minutes at night after work, but gives me joy to open this PR to share with the community this improvement. |
||
 Vinicius Gabriel
|
c53545d9fe |
Add configurable word separator for naming templates (#1333)
Closes #1230 ## What Adds a "Word Separator" setting (Space / Dot / Underscore / Hyphen / Custom) that replaces internal whitespace in each naming-template placeholder's rendered value — e.g. `{Author}` renders "Arthur.Conan.Doyle" instead of "Arthur Conan Doyle" when Dot is selected. This follows option 2 from the issue rather than inventing new dotted-keyword template syntax (`{Author.}`), since it's a smaller surface: one setting applies uniformly across all four templates (books/audiobooks × rename/organize) instead of needing a parallel token for every existing one. ## How it works - Literal characters typed into the template itself (e.g. the `.` in `{Author}.-.{Title}`) are never touched — only whitespace *inside* a placeholder's resolved value is affected. - Default is "Space", which is a no-op: existing templates produce byte-identical output after this change (verified via the existing test suite, unmodified, still passing). ## Where - `shelfmark/core/naming.py` — `word_separator` param on `parse_naming_template` / `build_library_path`. - `shelfmark/download/postprocess/policy.py` — `get_word_separator()`, mirroring the existing `get_file_organization()` accessor. - `shelfmark/download/postprocess/transfer.py` — wires the resolved separator through the four existing template-rendering call sites. - `shelfmark/config/settings.py` — new `Word Separator` / `Custom Word Separator` fields next to the existing naming-template fields. - `src/frontend/.../namingTemplatePreview.ts` + `NamingTemplateField.tsx` — the settings UI has its own TS mirror of the Python renderer for the live preview; updated it in lockstep so the preview doesn't lie about what the separator will actually do. - Tests added on both sides (pytest + vitest). ## Testing - `uv run pytest tests/core/test_naming.py tests/core/test_destination_file_organization.py` — all pass, including new cases. - `uv run pytest` (full suite) — same pre-existing failures as on `main` before this change (browser/network-dependent bypass & e2e tests unrelated to this diff), everything else green. - `uv run ruff check` / `ruff format --check` / `basedpyright` — clean. - `npm run lint` / `format:check` / `typecheck` / `test:unit` (196 tests) — clean. |
||
|
Vinicius GabrielandClaude Sonnet 5
|
c576003319 |
feat(naming): add {FirstAuthor} template token (#1322)
Closes #930. ## What New `{FirstAuthor}` naming-template token. It renders only the first author when metadata lists several ("Author1, Author2, Author3"), so multi-author books can be filed alongside the rest of that author's work instead of getting their own "Author1, Author2, ..." folder. ``` {Author} -> Terry Pratchett, Neil Gaiman {FirstAuthor} -> Terry Pratchett ``` ## How - Added to `KNOWN_TOKENS` in `shelfmark/core/naming.py`, positioned before `author` so `{FirstAuthor}` isn't parsed as literal `First` + `{Author}`. - Derived inside `parse_naming_template` from the existing `Author` value (split on `,` / `;`), so every caller — folder transfer, rename, the settings preview — picks it up with no extra wiring. An explicit `FirstAuthor` key in the metadata still wins if one is ever passed. - `{Author}` behaviour is unchanged. - Frontend `namingTemplatePreview.ts` token list + `KNOWN_TOKENS` kept in lockstep (there's a test enforcing that), with a matching `firstAuthor` helper. - Settings field descriptions + `docs/environment-variables.md` list the new token. ## Known limitation A lone author written `Last, First` is split on the comma too and renders as `Last` — the source metadata doesn't mark which form it is. Called out in the token help text and covered by a test. `{Author}` remains available for anyone who wants the raw string. ## Checks - `make python-test` — 2963 passed - `make python-lint` / `make python-format` / `make python-typecheck` / vulture — clean - `make frontend-test` — 187 passed · `frontend-lint` / `frontend-format` / `frontend-typecheck` — clean 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
 Jorge Lima
|
d7fe28595c |
fix(bypass): wait for the solved page before reading its source (#1286)
Follow-up to #1276 with a measurement from the instance I reported there. v1.3.13 solves the challenge again, but on my setup the solve was being thrown away immediately afterwards: ``` 19:26:08 Bypass successful using _bypass_method_cdp_gui_click 19:26:16 Bypass failed (attempt 1/10): TimeoutError: Time ran out while waiting for: {html} ``` `_get()` ends with `return await page.get_page_source()`, which is `find("html", timeout=1)` in SeleniumBase. One second is enough for a page that is already sitting on its content, but Anna's Archive answers a cleared check with a redirect to the real page, so the document is not there yet. The solve is discarded, the whole attempt restarts, and the extra requests are what earn the 429 that `note_rate_limited()` then parks the host for — 120 s, then 300 s. ## Change `_read_page_source()` waits for the document itself, with a `BYPASS_PAGE_SOURCE_TIMEOUT` setting (default 20 s, min 1, max 120) in Direct Download → Cloudflare Bypass, next to the existing bypasser timeouts. ## Measured on a live instance I patched the wait in the running container (`find("html", timeout=1)` → `timeout=20` in the installed seleniumbase, which is the same effect as this PR) and re-ran the same searches on the same host, k3s behind a Surfshark WireGuard exit, internal bypasser, v1.3.13: | | 1 s wait | 20 s wait | |---|---|---| | `Time ran out while waiting for: {html}` | one per solve | none | | 429 backoffs | 2 (120 s, then 300 s) | none | | Search for a book AA has | 199 s and 200 s, both errored | 61 s, 2 epub releases | A download after that took 5 s from LibGen, so the search was the whole cost. ## Tests Two tests in `tests/bypass/test_bypass_budgets.py`, the file already covering #1276: a page that needs longer than a second still yields its HTML, and `BYPASS_PAGE_SOURCE_TIMEOUT` overrides the default. `uv run pytest tests/ --ignore=tests/e2e`: 2848 passed, 47 skipped. Ruff check and format clean. The docs table is auto-generated, but running `scripts/generate_env_docs.py` here rewrote unrelated entries (Newznab, BOOK_LANGUAGE), so I added only the new entry by hand in the generator's format rather than commit that churn. One thing I could not judge from outside: whether 20 s is the right default for hosts other than AA. It only costs anything when a solve would otherwise be discarded, but I have measured it on one site. |
||
|
CaliBrain
|
97e289ae13 | fix: search, Prowlarr and qBittorrent follow-ups (#1276, #1283) (#1284) | ||
  zab1996andRyan
|
02b7e9d958 |
feat(newznab): support multiple named indexers (#1271)
## Summary - add a named Newznab indexer table with per-indexer URL and API key settings - search every configured indexer and retain the originating indexer name on each result - namespace cached release IDs across connections and isolate individual indexer failures - preserve the legacy single-indexer settings as a fallback - support masked API-key cells and trusted SABnzbd prefetching for named indexers ## Validation - 121 Newznab and SABnzbd backend tests passed on Python 3.14 - Ruff passed for all changed Python files - frontend TypeScript and strict lint checks passed - all 134 frontend unit tests passed - frontend formatting check passed ## Compatibility Existing `NEWZNAB_URL` and `NEWZNAB_API_KEY` configurations continue to work whenever `NEWZNAB_INDEXERS` is empty. Co-authored-by: Ryan <zab1996@users.noreply.github.com> |
||
|
463ef49ac3 |
feat(search): let each user pick their own default book languages (#1255)
## Why `BOOK_LANGUAGE` is a per-reader property, not a per-instance one. On a shared install one household member searches in German while another wants English and German — today whoever changes the setting changes it for everyone, and the only escape is re-picking languages in the filter on every single search. The per-user override machinery already carries `SEARCH_MODE`, the metadata providers and the default release sources, so the language default mostly had to opt into it. ## What changed **The field.** `BOOK_LANGUAGE` becomes `user_overridable` and moves from the **General** tab to **Search Mode**, next to the other user-overridable search defaults (per [review](https://github.com/calibrain/shelfmark/pull/1255#issuecomment-5391189094) — the first version had the Search section span two tabs, this one doesn't). Admins set it per user in the user editor, users set it in **My Account → Search Preferences**, and the Search Mode tab carries the usual "N users override this" summary. **No migration for the move.** `general` and `search_mode` both persist into `settings.json`, and a field's value is resolved through `load_config_file(tab)` for the tab it's declared on — so an install that already stores `BOOK_LANGUAGE` keeps its value. Checked against a `settings.json` written while the field still lived on General: the stored value resolves unchanged, a fresh install still gets `["en"]`, and `BOOK_LANGUAGE` in the environment still overrides both. **The two places the default is read.** - `/api/config` seeds the frontend's language filter, so it now resolves `BOOK_LANGUAGE` for the session user. - `build_release_search_plan` falls back to the default whenever a request carries no language filter — which is exactly what the filter's "Default" option sends. It takes an optional `user_id`, passed by `/api/releases` from the session and by the Prowlarr retry path from `task.user_id`, so a retry re-searches in the languages of whoever queued the download. **Validation.** Overrides go through `normalize_language()`, so `"German"`, `"ger"` and `"de"` all store as `de`, and an unknown language is rejected with a message naming it instead of being silently searched for. An empty list stays an empty list (a deliberate "no default filter"), `null` clears the override as everywhere else, and ENV still wins: with `BOOK_LANGUAGE` set in the environment the field reports `fromEnv` and overrides are ignored. **Scope.** Only the language default becomes overridable. The two format lists left behind under "Default Search Filters" stay admin-only — they describe what the library and its post-processing accept, not what a reader wants to read. There's a test pinning that. ## Verification - 2681 unit tests pass (2670 before, 11 added) - `ruff check`, `ruff format`, `basedpyright` over backend and tests, and `vulture` all clean; frontend lint, format, typecheck and 126 unit tests clean - `docs/environment-variables.md` regenerated via `scripts/generate_env_docs.py` (the `BOOK_LANGUAGE` row follows the field into the Search Mode section) - Manually against a two-user instance with builtin auth (first round, before the tab move): with user A on German and user B on English+German, `/api/config` returns each reader their own `default_language` and an unfiltered `/api/releases` plans the matching languages; an admin can set and read the same override for another user; clearing it falls back to the global value; a stray `"klingon"` is rejected; and `BOOK_LANGUAGE` in the environment overrides both users with the field marked `fromEnv` - After the tab move I re-ran the suites above plus the stored-value/fresh-install/ENV check described under "No migration for the move"; the behaviour it exercises is what the move could have broken Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: CaliBrain <calibrain@l4n.xyz> |
||
 jakesterpdxandClaude Fable 5
|
65e2e3be20 |
feat(audiobook): recognise .mp4 as an audiobook format (#1264)
## Problem Some trackers — MyAnonamouse in particular — distribute AAC audiobooks as per-chapter `.mp4` files. That's the same ISO-BMFF container as `.m4a`/`.m4b`, just with the generic extension (`ftyp isom`, audio-only). Today those releases: 1. show up in Prowlarr search results with **no format chip** — only the generic "Audiobook" icon, because no format could be inferred; 2. download successfully; then 3. fail post-processing with **"No book files found in download"**, because `.mp4` isn't in `AUDIOBOOK_FORMATS` (`shelfmark/core/utils.py`). Real example: MAM #627978, *The Martian* (Andy Weir, 2020 edition) — 142 files `0001 … 0142 Andy Weir (2020) The Martian.mp4` + `cover.jpg`, 305 MB. Every file is a valid AAC-in-MP4 chapter. Adding `mp4` to `SUPPORTED_AUDIOBOOK_FORMATS` in `settings.json` doesn't help since the hard-coded tuple is what post-processing scans against. ## Change - Add `"mp4"` to `AUDIOBOOK_FORMATS` (single source of truth — settings UI, Prowlarr parsing, IRC parser, archive extraction and post-download scan all derive from it), with a comment explaining why. - Add `".mp4"` to the two hand-maintained debrid `_BOOK_EXTENSIONS` lists (AllDebrid / Real-Debrid) so file selection matches. - Slot `mp4` into the IRC `AUDIOBOOK_FORMAT_PRIORITY` table right after `m4a` (same container family). - Update the documented default in `docs/environment-variables.md`. - New regression test `test_audiobook_multifile_mp4_chapters_are_book_files` modelled on the existing multi-file usenet test. ### Note for existing installs The legacy-default migration only widens configs that still hold the old `m4b,mp3` list, so users on the current widened default won't pick up `mp4` automatically — they'll need to tick it in Settings → Audiobook formats. New installs get it by default. Happy to extend the migration if you'd rather it be automatic. ## Testing - `ruff check` / `ruff format --check`: clean - `pytest tests/core tests/config tests/irc tests/prowlarr tests/download -m "not integration and not e2e"`: 2296 passed, new test + `test_audiobook_format_consistency.py` all green. The 10 failures in `test_entrypoint_permissions.py` / `test_orchestrator_stall.py` reproduce identically on untouched `main` on macOS and are unrelated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
CaliBrain
|
7d56624ab6 |
fix: group multi-file audiobooks that arrive as an archive (#1254)
Follow-up to #1237. \`rename_and_group\` only grouped when the source root was a directory, so a multi-file audiobook delivered as a single archive fell through to the flat path: a \`Book.zip\` of twelve chapters landed loose in the destination root with its original chapter names — the layout #1181 is about. The \`is_dir()\` guard was there to keep \`Book.zip/\` from becoming the folder name, but skipping the file case gives up the grouping instead of naming it. A non-directory source can only produce several book files by having been extracted (\`collect_staged_files\` returns a single-element list for every other file shape), so the archive stem is the release name and the suffix is packaging: group under \`Book/\`. Also regenerates the env docs for the new option and gives it the same \"do not use with ingest folders\" caveat Rename and Organize carries, since both now create directories in the destination. Tested: reverting only the source fix makes both new tests fail and the \`rename\` control case pass, so grouping stays opt-in. Full non-e2e suite green (2653 passed). |
||
|
CaliBrain
|
e7007865a4 |
fix(prowlarr): stop turning indexer failures into empty results and 404s (#1251)
Two independent bugs, both from an indexer that Prowlarr proxies rather
than answers for itself: the search never reported that it had failed,
and the grab never resolved what it was handed.
Search. A Torznab search is Prowlarr proxying a live request out to the
tracker, so for a Cloudflare-fronted indexer it waits on FlareSolverr.
The client gave it the 30s budget sized for Prowlarr's own JSON
endpoints, then swallowed every failure -- the timeout, the 429 Prowlarr
returns once it has disabled an indexer, a parse error -- into the same
empty list that means "this indexer has nothing". A cold challenge
routinely runs past a minute, so the UI said "No releases found for this
book" while FlareSolverr was still solving. That empty list also drove
the auto-expand retry, which fires on "no results with the category
filter". A timeout satisfies it, so Shelfmark sent a second search to an
indexer still busy with the first -- two Chromes at once, enough to take
FlareSolverr's down on a small host.
torznab_search now raises ProwlarrSearchError, and an empty list
strictly
means the indexer answered with no matches. The source records which
indexer searches failed: one dead indexer no longer sinks the others,
auto-expand runs only when every indexer genuinely answered, and zero
results with at least one failure raises SourceUnavailableError, which
the releases endpoint already turns into a 503 carrying a real message.
Prowlarr being unreachable was the same lie by another route -- the
indexer list came back empty, leaving nothing to query -- and now says
so.
Indexer searches also get their own timeout, PROWLARR_INDEXER_TIMEOUT,
defaulting to 90s and clamped to 5-300. Prowlarr's status and indexer
list keep 30s so Test Connection stays responsive, and the connect
timeout is split out at 10s so an unreachable Prowlarr fails fast rather
than hanging for the whole read budget. The overall per-request search
budget now scales to twice the indexer timeout, capped at 240s, so
raising the setting is not undone by the cap one level up while staying
under the 300s gunicorn worker timeout.
Grab. Prowlarr hands out a proxy download URL, with no magnetUrl and no
infoHash, for any indexer that only publishes torrent files. The native
Real-Debrid client built its magnet as "if not
url.startswith('magnet:') and expected_hash", so with no hash to work
from it left the URL alone and POSTed it to /torrents/addMagnet as the
magnet field. Real-Debrid answered 404 and the grab died on a raw HTTP
error. AllDebrid carried the same line and the same bug.
Both now resolve the URL first, through the extract_torrent_info path
the
torrent clients have used since #1108: pass a magnet through untouched,
follow a redirect or a response body that turns out to be a magnet,
otherwise upload the fetched .torrent, and fall back to a magnet built
from the infoHash only when the fetch failed. The file is preferred over
a synthesized urn:btih: magnet because it carries the tracker list; a
bare hash leaves the service to find the swarm on DHT alone. Fetches are
shared with the rest of the add path through the torrent fetch cache, so
resolving costs at most one request. Real-Debrid takes the file on PUT
/torrents/addTorrent with the raw bytes as the request body, AllDebrid
on
POST /magnet/upload/file as multipart files[]. A URL that resolves to
neither form now raises before any request reaches the service, so the
user reads why instead of a 404. Neither debrid client had any test
coverage; both have some now.
Fixes #1249
Fixes #1250
|
||
 helgehelge123andhelgehelge123
|
7b9c416df8 |
perf(bypass): keep the helper subprocess alive between bypasses (#1222)
Every protected request spawns a fresh helper subprocess, paying interpreter start and imports before any work begins. Measured inside the container, five consecutive runs of `python -c "import shelfmark.bypass.internal_bypasser"`: ``` 3.53s 3.45s 3.55s 3.54s 3.46s ``` A single search issues several protected requests, so that is paid several times over per search. ## What changed The helper now serves one JSON request per line of stdin until the parent closes the pipe, and an idle timer (`BYPASS_BROWSER_IDLE_TIMEOUT`, default 180s) shuts it down once searching stops. Answers still travel by result file, but the file is now written to a `.part` path and renamed into place — the parent treats the file's existence as the answer, so it must never observe a half-written one. stdout and stderr stay attached to the parent's, so helper logs keep appearing in `docker logs` exactly as before. Failure handling, since a warm helper is exposed to more of it than a per-request one ever was: | Situation | Handling | | --- | --- | | Helper died between requests | Detected via `poll()`, respawned | | Pipe broken at write time (`poll()` can miss this) | One retry on a fresh process; a fresh one failing there is a real failure | | Helper exits without writing a result | `RuntimeError` naming the exit code | | Wedged past the timeout, or cancelled mid-bypass | Helper killed, then `_cleanup_orphan_processes` because a killed helper never got to close Chrome | | Idle reaper racing an arriving request | Re-checks the deadline under the lock and re-arms instead of killing a helper that just did work | The DNS config now travels with every request rather than only at spawn: a warm helper outlives changes the parent makes to its provider. ## `BYPASS_REUSE_BROWSER`, off by default This parks the CDP driver between bypasses. A driver's websockets are bound to the loop that opened them and cannot outlive their process, so the persistent helper is what makes this possible at all — and the warm path runs on `_CDP_WORKER`'s long-lived loop rather than `asyncio.run` for the same reason. The mechanism works. With it on, the browser start disappears from the second request onward: 0.7s from `Reusing warm Chrome browser` to the first bypass attempt, against roughly 16s cold. **It still ships off, because a matched-pair test shows it is a net loss against DDoS-Guard.** Each round primed with one cold bypass, waited 10s, then measured a second — identical timing in both arms, only the browser strategy differing, order balanced (fresh, warm, warm, fresh) so drift over the session cannot masquerade as an effect: | Arm | Measured request | | --- | --- | | fresh browser | 42.8s, 40.6s | | warm browser | 57.1s, 59.6s | Spread within each arm is 2.2s and 2.5s, against 16.7s between them. Reuse removes the ~15s browser start and then gives back roughly twice that in solving: a returning browser draws a harder challenge. Where the cold browser is through on the second bypass method, the warm one fails the first three and only `_bypass_method_humanlike` gets it, at ~30s for that method alone. Worth separating from a second effect I ran into while measuring: five back-to-back searches slow from ~32s to 51–98s with reuse **disabled** as well, so DDoS-Guard escalates on request rate independently of any of this. That is why the pairs above are timed identically rather than simply run in sequence. It is the larger of the two effects, but not something this project can patch around. Reuse is left available rather than dropped because Cloudflare sites may not respond the same way, and because the two concerns are independent: the helper start is pure overhead and always worth removing, the browser is not. ## Verification - 2559 unit tests pass (2542 before, 17 added in `tests/bypass/test_warm_browser.py`) - `ruff check`, `ruff format`, `basedpyright` over backend and tests, and `vulture` all clean - `docs/environment-variables.md` regenerated via `scripts/generate_env_docs.py` - Live against Anna's Archive on a warm helper: searches return their usual ~760KB and 667 results, the app's own search warm-up completes with 50 results, and the container is left with no orphan chrome/Xvfb/ffmpeg processes Happy to drop the `BYPASS_REUSE_BROWSER` half entirely if you would rather not carry a default-off path — the helper persistence stands on its own. Co-authored-by: helgehelge123 <helge.neumann@zollsoft.de> |
||
|
CaliBrain
|
646b531669 |
fix(hardcover): accept the short hc_pat_ keys Hardcover issues now (#1241)
Hardcover replaced its ~500 char JWTs with short opaque personal access
tokens ("hc_pat_..."), and the connection test rejected anything under
100 chars before a request ever left Shelfmark, so every newly created
key failed with "API key seems too short".
The length floor now applies only to keys without the hc_pat_ prefix; a
prefixed key goes straight to Hardcover, which is the authority on
whether it is valid. Also strip a pasted "bearer " prefix regardless of
casing -- Hardcover's docs tell users to paste the token into an
"authorization" header, so the prefix rides along on the copy, and the
old case-sensitive removeprefix() sent it through as part of the token.
The API key field now names the expected shape.
Note that Hardcover's PAT path currently answers every hc_pat_ token
with a 500, a fabricated one included, while non-PAT tokens still get a
clean 401. So a new key cannot connect yet regardless of this change --
that failure is server-side and not something this code can reach.
Refs #1240
|
||
|
CaliBrain
|
7345f6be1a | Fix README and hints for audiobooks (#1215) | ||
|
CaliBrain
|
2b8b35bb52 |
fix(newznab): make indexer book categories configurable (#1214)
Newznab searches hardcoded category 7000 for ebooks and 3030 for audiobooks, so indexers using custom IDs returned no results or the wrong ones. Add NEWZNAB_EBOOK_CATEGORIES and NEWZNAB_AUDIOBOOK_CATEGORIES (tag lists, defaulting to 7000 and 3030) and resolve the search categories from config. Values are parsed leniently — list or comma/whitespace separated, non-numeric entries skipped, duplicates dropped — and fall back to the standard IDs when empty, so a cleared field can't silently widen the search to every category. NEWZNAB_AUTO_EXPAND remains the way to do that on purpose. Results carrying a custom ID outside the standard 7000-7999 / 3030 ranges were typed as "other", which routed custom-category audiobooks as ebooks. Trust the searched content type when a result carries a category we explicitly asked for. Also drop the unused NEWZNAB_BOOKS / NEWZNAB_AUDIOBOOKS constants from api.py — a third copy of the same hardcoding. Closes #1208 |
||
|
CaliBrain
|
d0e008adde |
Stop dropping audiobook releases that are not m4b or mp3 (#1199)
An IRC audiobook search returned nothing while OpenBooks, reading the same @search answer from the same channel, listed results. Three separate defects were discarding them. The audiobook format list was maintained by hand in four places and had drifted. The settings UI offered only m4b/mp3/m4a/zip/rar, and that list is the only one a user's config can be built from, so flac, opus, ogg, aac, wav and wma were unreachable everywhere — even though the IRC parser recognized them, the IRC sorter ranked them (dead code that could never fire), archive extraction knew them and Prowlarr searched for them. A FLAC audiobook was invisible in search and, if it arrived anyway, rejected after download as "format not supported". AUDIOBOOK_FORMATS and ARCHIVE_FORMATS now live once in core.utils and every layer derives from them, which also restored the missing .opus in the post-download scan's trackable extensions. Widening the default alone would not have reached anyone already affected: initialize_default_configs() writes field defaults only when a tab has no config file yet, so an existing install keeps its persisted m4b/mp3 list forever. migrate_audiobook_formats rewrites a list that still matches the old default exactly and leaves every other value alone — re-enabling formats someone had deliberately turned off would be worse than leaving them narrow. The IRC parser filtered by file extension alone. Multi-file audiobooks ship as a .rar or .zip of MP3s, which matched neither SUPPORTED_FORMATS nor SUPPORTED_AUDIOBOOK_FORMATS, so they fell out of the ebook bucket and the audiobook bucket both. Results are now classified before the format filter is applied: an audio extension means audiobook, an ebook extension means ebook, and for a container — where the extension says nothing about the contents — the release name decides. An ebook archive stays out of audiobook results. RESULT_LINE_REGEX matched \w+ after any dot, so a line carrying no file extension parsed as format "5mb" out of "::INFO:: 620.5MB", taking the title and the size down with it and guaranteeing every downstream filter dropped it. Any decimal size did this. The extension is now matched against the known formats, so such a line falls through to the simple pattern and comes back as "unknown", which the rest of the parser already handles. ALL_RECOGNIZED_FORMATS became an ordered tuple in the process: it was a set, so which extension won for a line naming two of them depended on set iteration order and could vary between restarts. Refs #1129 |
||
|
CaliBrain
|
e320b7623d |
Fix LOG_LEVEL being ignored and Z-Library 503 cookie gate (#1188)
LOG_LEVEL never reached the app logger: env.py hardcoded the level to DEBUG or INFO, so INFO lines kept appearing under LOG_LEVEL=error. Read it from the env var and advanced settings, normalize unknown values to INFO, and expose it as a setting. entrypoint.sh now normalizes gunicorn's level too, so a typo falls back to info instead of stopping the container from booting. Z-Library gates the first hit on /md5/<hash> with a 503 whose only payload is a Set-Cookie; echoing that cookie back returns the 302 to the real page. html_get_page dropped it and re-ran the same rejected request on every retry, ending in "No download URL resolved". Retry once with the cookies the 503 issued. Fixes #1185 Fixes #1187 |
||
 Tilian B
|
ff770940ca |
Add torrent post-import category action (#1148)
## Summary - add a **Change Category** torrent completion action and conditionally show its post-import category/label setting - update kept torrents only after a successful library import - support qBittorrent categories, Transmission labels, Deluge's Label plugin, and rTorrent's `custom1` label - preserve existing Keep/Remove behavior and document the new environment setting ## Behavior Category changes happen from `post_process_cleanup`, after output transfer and post-processing complete. This keeps the existing hardlink flow unchanged. An empty post-import category is a no-op, and client API failures are logged without turning a successful library import into a failure. ## Validation - `pytest -n 0 tests/prowlarr/test_qbittorrent_client.py tests/prowlarr/test_transmission_client.py tests/prowlarr/test_deluge_client.py tests/prowlarr/test_rtorrent_client.py tests/prowlarr/test_handler.py tests/config/test_generate_env_docs.py` — 161 passed - `ruff check` on all changed Python files — passed - `basedpyright` on the changed client implementation files — passed - full `basedpyright shelfmark/download/clients` currently reports two pre-existing errors in the new debrid connection-test code at `settings.py:546` and `settings.py:563`, outside this PR's diff - multi-architecture Docker images built successfully for `linux/amd64` and `linux/arm64` |
||
 
|
816a735cde |
Add a {Language} naming template variable, and consolidate language resolution (#1142)
Fixes #1138 Fixes #1141 ## Problem Two language editions of one book resolve to the same canonical title, so they render to the same path and the second gets a `_1` collision suffix. Audiobookshelf treats a folder as exactly one library item, so the pair becomes a single book with both files as tracks and a summed runtime. Shelfmark already parses and displays the language. It just never reached the template engine. ## `{Language}` template variable A template like `{Author}/{Title}{ (Language)}/{Author} - {Title}` now yields: ``` /library/J K Rowling/Harry Potter (sv)/J K Rowling - Harry Potter.m4b /library/J K Rowling/Harry Potter/J K Rowling - Harry Potter.m4b ``` The untagged edition's path is byte-identical to today, so no existing layout shifts. Three details worth flagging: **The value is casefolded.** On a case-insensitive filesystem `(SV)` and `(sv)` would collapse back into one folder, reintroducing the exact collision being fixed. **Values meaning "we don't know" render nothing** rather than producing `Project Hail Mary (unknown)` folders. Anna's Archive reports that string literally (`direct_download.py`, `language = detected or "unknown"`). **The frontend wasn't sending the release language at all**, so the token would have stayed empty for exactly the audiobook sources in the report. Prowlarr and AudiobookBay do not put language in `extra` the way `direct_download` does, hence the payload plumbing. It reads `release.language`, never `book.language` — the latter is the provider's canonical edition and would mislabel a translation, with a regression test for that specifically. Not gated to audiobooks: Calibre-Web-Automated stages ingested files by basename and discards folder structure, so the rename (filename) template is the only lever those users have. Verified that form works: `J K Rowling - Harry Potter (sv).epub`. ## Language consolidation (#1141) Three release sources each carried their own alias map, all resolving to the same ISO 639-1 codes, alongside a bundled database that only one of them used. Adding a language meant editing three places. Aliases now live in `data/book-languages.json` beside the code and name they belong to, and `shelfmark/core/languages.py` resolves any of them — two-letter code, ISO 639-2 three-letter in either the bibliographic or terminological form, or English name. Prowlarr and AudiobookBay drop their tables. Direct Download keeps its own path-parsing heuristics, including the ambiguous short codes that collide with English words (`de`, `en`, `no`, `in`), and takes only the alias data. This also closes a coverage gap. MyAnonamouse offers 62 languages; Prowlarr mapped 37, and an unmapped code is *dropped* rather than passed through, so the other 25 carried no language at all — leaving `{Language}` empty and the collision unfixed for Latin, Farsi, Tamil, Urdu and the rest. Seven languages MAM offers had no database entry at all: Bosnian, Burmese, Estonian, Icelandic, Manx, Scottish Gaelic, Sanskrit. Also fixes the Traditional Chinese code, which used a U+2011 non-breaking hyphen. Nothing compares against the ASCII spelling today so it was latent, but it would silently defeat the first thing that did. ## Validation Verified end to end against a live Prowlarr and MyAnonamouse, not just unit tests. A real search returning both an English and a Swedish edition, through the actual `queue_release` → `DownloadTask` → naming path: ``` STEP 1 real MAM search -> 37 releases, languages: ['en', 'sv'] STEP 3 queue_release -> task.language='sv' STEP 4 build_metadata_dict -> metadata['Language']='sv' STEP 5 build_library_path -> /library/J K Rowling/Harry Potter (sv)/... two language editions resolve to DIFFERENT folders: True ``` The refactor is pinned by a snapshot of both per-source maps taken *before* they were deleted. All 131 aliases are asserted to still resolve to the same code, one parametrised test each, so a regression names the specific alias. Also verified: the filename-only template, the retry round-trip (`serialize_task_for_retry` → `_restore_task_from_retry_payload`, plus a legacy payload with no `language` key), and placeholder handling. Added a `KNOWN_TOKENS` ordering invariant test — `find_placeholder()` does a substring `.find()` in list order and nothing protected that contract, so a future token in the wrong position could silently shadow an existing one. And a lockstep guard on the frontend, since `KNOWN_TOKENS` is hand-duplicated in TypeScript. **One caveat worth stating.** Three MAM codes are confirmed by observation (`ENG`→`en`, `SWE`→`sv`, `MAL`→`ml`, the last from a real `[MAL / EPUB]` Tagore release). The remaining ~59 are derived from ISO 639-2 rather than observed, because MAM's catalogue is overwhelmingly English — enabling 27 extra languages still yielded only one non-English hit across 258 results. Mitigated rather than closed: both 639-2 variants are present for every language where they differ, and a wrong alias is an unused entry while a missing one loses the language. Happy to correct any code a maintainer knows differs. ## Test results 2056 Python tests pass (up from 1906). Frontend typecheck, lint, format and 126 unit tests pass. Pre-existing failures on my machine, unchanged by this branch and unrelated: `tests/bypass/` needs `seleniumbase`, and `tests/config/test_entrypoint_permissions.py` uses bash-4 syntax that macOS bash 3.2 rejects. --------- Co-authored-by: delize <4028612+delize@users.noreply.github.com> Co-authored-by: CaliBrain <calibrain@l4n.xyz> |
||
|
CaliBrain
|
a1367f431d |
Ship Prowlarr per-entry rows as opt-in, not the default (#1145)
#1140 fixed the guid-only dedup that hid results from filter-specific indexer entries, but shipped the new behaviour on by default: PROWLARR_COLLAPSE_DUPLICATES defaulted off, so every existing Prowlarr user got extra rows for any release that two indexer entries both returned, and the setting only let them opt back into what they already had. Default it on. The dedup key stays indexer-qualified, so the entries are still distinct internally; collapse then merges them back to one row, resolved by the Prowlarr priority rather than by query order as before. The visible result set matches what users had prior to #1140, and anyone who wants the per-entry rows (freeleech and the like) turns the setting off. Beyond the noisier list, the default mattered because split rows differ only by indexer name while sharing a title, size and peer count. Two of them have distinct source_ids, so the queue's duplicate guard does not fire, and the second grab's find_existing() matches the first by infohash and runs post-processing over the same download again, delivering the book twice. The search-side fallback in config.get(..., True) is flipped to agree with the field default. In production the field default governs, since the config cache is seeded from the registry and the fallback only applies to an unregistered key. The source tests monkeypatch config.get with a plain dict lookup, though, so the fallback is what they exercise: leaving it False would have kept every default-behaviour test asserting the opposite of what ships. A new test pins the two together. The deduplication tests now opt out explicitly, since they assert the split itself. test_collapse_off_by_default_keeps_both_rows becomes a pair, one for the untouched setting collapsing to a single row and one for opting out. docs/environment-variables.md is regenerated rather than hand-edited. It was already stale on main, so it also picks up RTORRENT_AUDIOBOOK_LABEL, DIRECT_DOWNLOAD_LANGUAGE_FROM_PATH, and reworded IRC_SEARCH_BOT and RTORRENT_LABEL text from earlier merges. The rest is fallout from the ruff 0.16.0 bump in #1139, which enabled a much larger default rule set and started formatting Python code blocks in Markdown: _find_existing_alias_user() uses min() instead of sorted()[0] (FURB192, currently failing Python Quality on main), and the two READMEs get their code blocks reformatted. |
||
|
Andrew Doeringanddelize
|
a4086f5e06 |
Keep Prowlarr results from distinct indexer entries separate (#1140)
Results were deduplicated on `guid` alone. When one tracker is configured in Prowlarr as several indexer entries differing only by a server-side search filter, all of them return the same guid for the same torrent, so every entry but the first was silently discarded. Freeleech and other filter-specific releases became invisible, replaced by the unfiltered entry's copy, and which copy survived depended on query ordering rather than user intent. Include the indexer id in the dedup key so the entries stay distinct. Release.source_id is qualified the same way. It keys the release cache and becomes the download task id, so rows sharing a guid would otherwise collide and a grab would route through whichever entry cached last, defeating the point of showing them separately. The handler's task matcher still accepts a bare guid or infoUrl so tasks queued before this change still resolve. Ordering now follows the priority already configured in Prowlarr (1-50, lower preferred) rather than a new setting, since users curate that ranking there and filtered entries are typically ranked ahead of their unfiltered counterparts. The enabled-indexer list is fetched once and reused for the enrichment check, so this costs no extra round trip. Releases carry extra.indexer_priority, and the sort dropdown gains an "Indexer priority" entry, ascending, alongside the existing alphabetical "Indexer" sort; SortOption grew a default_direction for that, defaulting to desc so "Peers" is unchanged. PROWLARR_COLLAPSE_DUPLICATES (default off) optionally collapses a release back to one row, resolved by the same Prowlarr priority. Left off, every entry that carried a release keeps its own row, which is what makes filtered results visible again. Identity handling is defensive about partial payloads: a result that cannot be identified is never dropped or merged, and collapse only merges on a strong identifier (guid/downloadUrl/magnetUrl/infoUrl) because merging on title alone would discard genuinely different releases that share a name. Fixes #1137 Co-authored-by: delize <4028612+delize@users.noreply.github.com> |
||
 Adam Vigneaux
|
0d02c6db47 |
Add qBittorrent API key authentication (#1143)
[qBittorrent 5.2.0](https://www.qbittorrent.org/news#sun-may-03rd-2026---qbittorrent-v5.2.0-release) (May 2026) added support for API key-based authentication in addition to the existing username/password-based authentication. This commit adds support for qBittorrent API key authentication to Shelfmark, configurable via environment variable or settings UI. If an API key is set at the same time as the username/password, API key will be preferred for authentication. Requires `qbittorrent-api` 2026.5.3, the version that added the `api_key` argument, or newer. `403 Forbidden` responses are not retried with API key authentication because a retry has no chance of succeeding. Tested end-to-end with my live qBittorrent 5.2.3 instance (WebAPI v2.15.1). <img width="785" height="616" alt="image" src="https://github.com/user-attachments/assets/8064e10e-9a7f-49f0-804e-4c441d23a1fc" /> |
||
|
CaliBrain
|
7fdaf3f67d | Add Audiobook support to IRC (#1136) | ||
|
CaliBrain
|
404e8cc5c5 |
path timeouts (#1103)
- Add configurable completed-path wait for external clients : - Add a configurable Advanced setting, DOWNLOAD_CLIENT_COMPLETED_PATH_TIMEOUT, for how long Shelfmark waits after a torrent or usenet client reports completion before treating the completed path as missing. - Keep the default at the existing 60-second grace period, with a maximum of 3600 seconds - Add e2e testing - Add e2e testing Should fix #861 |
||
 Alex SchittkoandCaliBrain
|
40d6a179b7 |
feat: native WireGuard VPN egress mode (USING_WIREGUARD) (#1097)
Add an opt-in WireGuard egress path alongside the existing Tor mode --------- Co-authored-by: CaliBrain <calibrain@l4n.xyz> |
||
|
CaliBrain
|
c64c2d374a |
Make IRC less spammy and require a bot name for conversatons (#1065)
First step towards fixing the friction created by shelfmark in #997 |
||
 spindrift
|
0633cfde87 |
Add content_type=combined URL parameter and FORCE_COMBINED_SEARCH user setting (#1058)
Adds support for `content_type=combined` in URL search parameters, letting users force combined-mode searches via a bookmarkable link rather than relying on the last-used preference from localStorage. The override is applied only in Universal mode and only when combined mode is actually available (universal enabled, `show_combined_selector` on, neither content type blocked by policy). Otherwise, it's silently ignored, consistent with how `content_type=ebook`/`audiobook` already behave outside Universal. Existing `content_type=ebook`/`audiobook` URLs now also force combined mode off, so the URL is authoritative regardless of prior preference. Also adds a per-user `FORCE_COMBINED_SEARCH` setting that locks combined mode on whenever it's available. URL `content_type=ebook`/`audiobook` overrides are also superseded by force-combined for the same reason: the search bar wouldn't let users switch back, so honoring the URL param would leave them in a state they couldn't escape from. |
||
 Alex
|
9b8402c9a7 |
Add DISABLE_LOCAL_AUTH env variable (#962)
Adds a new env var to disable local auth entirely when using OIDC authentication Fixes #922 #834 |
||
|
Alex
|
3554d01c81 | Change path default for audiobooks + description fixes (#933) | ||
|
Alex
|
e35b4c47a7 |
Direct source refactor (#895)
- Updated mirror selection - Removed built-in mirror options, users must provide their own configurations - Set Universal search to default, added ability to disable direct source - Updated documentation - Updated makefile |
||
|
Alex
|
87d5f127d6 | Add prek + pytest-cov (#873) | ||
|
Alex
|
41c4aa1d72 |
Updated permissions model and non-root support (#871)
- Adds a non-root startup path at user 1000:1000 - skips privilege escalation and ownership checks. Works e.g. for kubernetes deployments (user 1000:1000 and runAsNonRoot enabled). - Remove startup check/chown commands for user-owned folders. Checks can be done with a "Test destination" button in settings which performs a test write. Users are responsible for fixing their own permissions. - Update docs |
||
|
Alex
|
8f949a73d5 | Remove audible provider (#778) | ||
 cadric
|
3295be82a7 |
Add Audible metadata provider via Audimeta (#762)
Closes #515 ## Summary This adds a new `audible` metadata provider backed by the Audimeta API. The provider supports: - Audible/Audimeta metadata lookup without authentication - region selection (`us`, `ca`, `uk`, `au`, `fr`, `de`, `jp`, `it`, `in`, `es`, `br`) - ASIN book lookup - ISBN lookup with fallback search - series suggestions and series-order browsing - richer audiobook metadata such as narrators, runtime, rating, subtitle, cover, publisher, and series info - configurable Audimeta base URL, timeout, cache usage, default sort, and unreleased filtering ## Notes A few Audimeta-specific integration details were needed: - send a meaningful `User-Agent`, otherwise Audimeta rejects requests with `403` - send the `cache` parameter in the format Audimeta expects - use `keywords` for general search instead of `query`, which gave poor/irrelevant results for title-style searches ## Validation Tested locally with: - `python -m py_compile shelfmark/metadata_providers/audible.py` - `python -m pytest tests/metadata/test_audible.py -v` - `python -m pytest tests/metadata/test_metadata_provider_capabilities.py -v` Also verified manually in a Podman test container: - searching for `Discount Dan` returns Audible title `B0DXLXRNGG` - book details and series metadata load correctly ## Scope This PR intentionally keeps the change localized to the provider layer and docs: - new Audible provider - provider registration - provider docs - generated environment variable docs |
||
|
Alex
|
3d72f9e258 |
Various requested small features (#741)
- Added torrent removal option - Pass Prowlarr seedtimes to download clients (excluding rTorrent) - Split default release source option by content type - Split download to browser option by content type - Add "hide links" option |
||
|
Alex
|
ba92ad90bc |
Refine UI and adjust content type settings (#705)
- Tweak manual search toggle position - Refinements to the Hardcover list dropdown behavior - Hide the content type dropdown when a content type is blocked for a user - Fixes to Hardcover author parsing to strip out initialed names - Remove `env_supported=false` for security config options. |
||
|
Alex
|
554f5fcbe7 |
Patch: Various feature additions (#625)
- Add admin config for self-settings options visibility. Remove delivery preferences or notifications from the view. - Add option to use Booklore's Bookdrop API destination instead of a specific library - Add download path options for all torrent clients |
||
 arjunsrinivasan1997
|
a7064939ce |
feat: Add tag support to qBittorrent (#610)
Added support for adding tag(s) to torrents sent to qBittorrent via shelfmark.   |
||
 Alex
|
a560089ce3 |
Patch: Script improvements + bug fixes (#591)
- Add new booklore API file formats - Renamed cookie for better login persistence with reverse proxy - Updated fs.py to try hardlink before atomic move from tmp dir - Fix transmission URL parsing - Fix scenario where file processing of huge files starves the healthcheck - Large enhancements to custom scripting, including passing JSON download info, more consistent activation across output types, decoupling from staging behavior, and added full documentation. |
||
|
Alex
|
f84fb082ad |
Fix: AA mirror behavior (#589)
- Refreshed available AA URLs - Fixed potential redirect from AA itself causing mirror cache errors - Added fully customizable mirror list in UI - Segmented rotation behavior to Auto mode only Fixes #588 |
||
|
Alex
|
b10458a48b |
Patch: Migrate bypasser to pure CDP + Misc fixes (#575)
Bypasser: - Refactored internal bypasser logic to use SeleniumBase Pure CDP mode, removed chromedriver dependencies and UC code. - Added dedicated threading for internal bypasser functions, fixes any potential asyncio CPU spike behavior - Fixed WebGL issue with Chromium 144. Reverted 1.0.3 hotfix and updated to latest Chromium Misc: - Added M4A color mapping - Fix frontend language filtering with multi-language releases - Added "days" age for usenet/torrent releases - Improved entrypoint chown efficiency - Added `ONBOARDING` env variable, default true |
||
|
Alex
|
3be99effe4 |
Base url additions and bug fixes (#519)
- Base URL option in settings for reverse proxy setups - Fix NZB downloads not deleting on completion - Fix handling for audiobook files over 100+ parts - Fix prowlarr search timeout - Fix prowlarr categorisation for expanded searches |