Two independent reasons a working search reported failure to the user.
1. The client gave up before the server did (#1285)
`/api/releases` bounds one release search with RELEASE_SEARCH_TIMEOUT
(default
300s) and answers a spent budget with a sentence naming the real cause -
the
machinery added for #1276. The frontend then aborted the direct_download
search
at a hard-coded 180s, so it always won the race: the user saw "Request
timed
out. Check your network connection or proxy configuration." instead, and
raising RELEASE_SEARCH_TIMEOUT changed nothing they could observe, the
180s
being baked into the hashed bundle inside the image.
- /api/config reports the effective (clamped) budget, and the client
derives its
abort from it plus a margin, so the server always answers first.
- Direct-mode search shows what the server actually said. Every non-auth
failure
was relabelled "Unable to reach download source. Network may be
restricted or
mirrors blocked.", which discarded the explanation and blamed the user's
network. ApiResponseError now carries `serverMessage`, set only when the
server
explained itself, so the status-line placeholder still falls back.
Two latency fixes for the cost that made the timeout reachable at all:
- Fetch each distinct AA search URL once per search. The language-filter
retry
re-runs every title variant, and with DIRECT_DOWNLOAD_LANGUAGE_FROM_PATH
on
both passes build a byte-identical URL - behind DDoS-Guard each repeat
is a
fresh browser solve.
- Drop the solve-only bypass method. `_bypass_method_cdp_gui_click`
opens with
exactly that call and returns the moment it works, so the entry ahead of
it
could only repeat the half that had already failed, plus the backoff
before
the method that does work started. Reported at 0/19 successes and ~5.5s
of
each ~26s solve against DDoS-Guard.
2. The query carried every contributor, not one author (#1252)
`_pick_search_author` returned `book.search_author` verbatim while the
authors[]
fallback beside it deliberately narrowed to the first name before a
comma. Both
fields routinely arrive holding every contributor joined with ", ": the
frontend
builds `book.author` as `authors.join(', ')` for display
(bookTransformers.ts)
and the release modal sends that display string straight back as the
`author`
parameter, and `browse_record_to_book_metadata` and the manual-search
branch
both split the joined text into `authors` while still passing the
unsplit string
as `search_author`, so the split was never used.
A book whose metadata lists translators was therefore searched for as
Blindness Jose Saramago, Giovanni Pontiero, <persian translator>
which matches nothing on Anna's Archive. The bypass succeeds, the search
comes
back empty, and the user is told the book has no releases.
Narrowed in one place, `search_plan.first_author`, so the two branches
cannot
drift apart again, and applied to the IRC source, which built its query
with the
same verbatim preference. Hardcover is unaffected: it already sets
`search_author` from `_simplify_author_for_search(authors[0])`, which
resolves
"Last, First" itself and never yields a multi-author string.
Anna's Archive put a DDoS-Guard JS challenge in front of /search: the
homepage still returns 200, but /search and /md5/<id> answer 403 on
every mirror (.gl, .pk, .gd all confirmed). Search fetched both with
allow_bypasser_fallback=False, which rotates mirrors on a 403 instead of
invoking the bypasser, so it walked the whole mirror list, exhausted it,
and surfaced "Unable to reach download source. Network restricted or
mirrors are blocked." as a 503 on every query. Adding mirrors could not
help — they sit behind the same gate — and neither could USE_CF_BYPASS,
since search never reached that branch.
Fetch search and the detail page with allow_bypasser_fallback=True so a
403 hands over to the bypasser, which already detects this challenge
(DDOS_GUARD_INDICATORS matches the live page). Echoing the __ddg cookies
back does not clear it; it needs real JS execution. The download-count
fetch keeps allow_bypasser_fallback=False: it is decoration on the
details modal and not worth holding the modal open for a browser solve.
Fixes#1196
PR #1169 (python:3.14.6-slim -> python:3.15.0b3-slim) ran for 6h before
GitHub's max job limit killed it, then did it again on re-run. Two
independent defects.
Dependabot proposed a beta at all: the config already excluded python
from the docker digest group for dependabot-core#9496, but the comment
claimed ungrouped python updates get their pre-release filtered. They
don't. dependabot-core#13815 rewrote the Docker pre-release heuristic to
catch PEP 440 tags (its tests cover 3.15.0a2 and 3.5.0b3), yet the
suffixed real tag still got through seven months later. CPython spells
pre-releases without a separator, so 3.15.0b3 parses as an ordinary
version sorting above 3.14.6. Ignore python semver-minor/major instead
of trusting the heuristic; patch and digest updates still flow.
The run took hours rather than failing: the health wait looked bounded
at 60 iterations x 2s, but bare `curl` has no timeout. The 3.15 image
booted a container that bound 8084 without ever serving (greenlet has no
3.15 wheel, so the gevent gunicorn worker was wedged), so curl blocked
on
read forever and the loop never reached iteration 2. Every job's orphan
process at cancellation was that curl. Bound each probe and switch to a
wall-clock deadline, and add timeout-minutes so a hang can never reach
6h
again.
Verified against a socket that accepts and never responds: the old loop
was still hung at 30s, the new one exits at 120s with HEALTHY=0 into the
existing log-dump path, and a responsive endpoint is still detected
immediately.
- Add configurable completed-path wait for external clients : - Add a
configurable Advanced setting, DOWNLOAD_CLIENT_COMPLETED_PATH_TIMEOUT,
for how long Shelfmark waits after a torrent or usenet client reports
completion before treating the completed path as missing. - Keep the
default at the existing 60-second grace period, with a maximum of 3600
seconds
- Add e2e testing
- Add e2e testing
Should fix#861
- Adds `uv`, `ruff`, `pyright`, `vulture` and `pytest-xdist`
- Move project, lockfile, docker build etc to uv
- Align python tooling on 3.14
- Huge bulk of ruff linter fixes applied. Still in progress but all the
core types are now enforced
- Update CI and test helpers
- Clean up a few uses of config options that may miss the env variable
if this is set
- Add enhanced retry availability utilising the DB to persist download
errors / retries across restarts, request failures, and pass Prowlarr
detail through the download task to maintain retry data.
- Strip back entrypoint permissions for less intensive chown operations.
Fixes#796
- Much simpler handling of downloads in the activity sidebar, and
improved storage, persistence and UI behavior.
- Replace `ActivityService` with direct storage on
`DownloadHistoryService` and `download_requests` and removes the
activity_log/activity_dismissals tables
- Simplify no-auth mode by removing the fake user row pattern, handled
internally
- Add local download fallback so history entries can still serve files
after tasks leave the queue
- Downloads, requests and history are now entirely persistent between
updates / restarts, and correctly tied to each user.
- Added migration for builtin auth users who used dev builds during
multi-user development
- Display apprise errors in logging
- Fix user provisioning in reverse proxy auth setups
- Refactor scoring and release modal utils
- Refactored activity backend for full user-level management, using the
db file
- Revamped the activity sidebar UX and categorisation
- Added download history and user filtering
- Added User Preferences modal, giving limited configuration for
non-admins - replaces the "restrict settings" config option.
- Many many bug fixes
- Many many new tests
- Moved backend OIDC functionality to external library Authlib to help
maintainability
- Separated User settings UI into individual components, allowing for
standard settings UI decorator components to be used.
- Added full support for reverse proxy and CWA users alongside local and
OIDC
- Added mapping and syncing functionality for OIDC, CWA and reverse
proxy users
- Added per-user settings into the app-wide config system. Each config
can be declared as user-overrideable, and app-wide functionality can now
receive user-specific options via standard config calls.
- Added per-user audiobook destination config
- Updated login modal UI for simplified login, plus custom labels for
OIDC login
- Added user visibility in header dropdown
- Unified "restrict settings to admin" to use app-wide user roles.
Closes#552
## Summary
Adds OIDC authentication and multi-user support to Shelfmark. Users can
now be managed individually with per-user download settings, while
maintaining full backwards compatibility with existing auth modes
(no-auth, builtin, proxy, CWA).
### Authentication
- **OIDC login** with PKCE, auto-discovery, group-based admin mapping
- **Password fallback** when OIDC is enabled (prevents admin lockout)
- **Auto-provisioning** of OIDC users (configurable on/off)
- **Email-based linking** of pre-created users to OIDC accounts
- **Lockout prevention** — requires a local admin before OIDC can be
enabled
### User Management
- **SQLite user database** (`users.db`) with admin CRUD API
- **Users management tab** in settings UI (admin-only)
- **Settings restricted to admins** in multi-user modes (builtin/OIDC) —
non-admin users cannot access settings
- Create, edit, and delete users with role assignment (admin/user)
- Password management for builtin auth users
- OIDC users shown with provider badge (password fields hidden)
- Per-user configurable settings:
- **Download destination** — custom folder path per user
- **BookLore library & path** — dropdown select, each user's books go to
their own library
- **Email recipients** — per-user email delivery targets
- **`{User}` template variable** — use in destination paths (e.g.,
`/books/{User}/`)
- Settings override model: per-user values override globals, empty/unset
falls back to global defaults
### Download Scoping
- **Per-user download visibility** — non-admins only see their own
downloads
- **Username display** in downloads sidebar (shows who requested each
download)
- **WebSocket room-based filtering** — admins see all, users see only
their own
- **Download progress scoping** — progress events routed to correct user
rooms
### BookLore Integration
- **Dynamic dropdown selects** for library/path (replaces text inputs)
- **Per-user library/path overrides** via user settings
- **Options cache refresh** after Test Connection
### Security
- SQL injection prevention (column whitelist on user updates)
- Generic OIDC error messages (no internal detail leakage)
- Admin self-deletion and last-local-admin deletion guards
- OIDC role overwrite fix (only updates role when admin_group is
configured)
## Migration
**No migration script needed.** The `users.db` is created automatically
on first startup. Existing builtin auth users are auto-migrated to the
database on their first login. All other auth modes (no-auth, proxy,
CWA) continue working unchanged.
## Test Plan
- [x] All 519 tests passing, 0 failures
- [ ] Test no-auth mode: settings accessible, downloads work without
login
- [ ] Test builtin auth: legacy credentials auto-migrate on login, new
users can be created
- [ ] Test OIDC auth: login flow, callback, auto-provisioning,
group-based admin
- [ ] Test CWA auth: unchanged behavior
- [ ] Test proxy auth: unchanged behavior
- [ ] Test per-user downloads: non-admin sees only own downloads
- [ ] Test BookLore dropdowns: library/path selection, per-user
overrides
- [ ] Test Docker build: no Dockerfile changes needed
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Added new book output option **upload to Booklore**, available in
download settings
- Got annoyed at my messy processing code while implementing Booklore so
refactored the whole thing
- Full black box file processing testing with randomised configuration
- Deluge: Connect via WebUI auth for simplified setup
- Added env vars documentation, auto generated via script, and unlocked
most settings to be used as env vars
- Changes the auth settings to support more than two auth types
- Added a proxy auth type with settings for user and optionally group
headers
- Added a global middleware `proxy_auth_middleware` to handle proxy auth
(it does nothing if any other auth mode is set)
- Added support for proxy auth to `get_auth_mode`, `login_required`,
`api_login/out`, and `api_auth_check`
- Added a backend check to make protect the API for settings when admin
is required
---------
Co-authored-by: Joshua Tag Howard <git@jthoward.dev>
Co-authored-by: Alex <alex.bilbie1@gmail.com>
## Headline features
### Prowlarr plugin - search trackers and download usenet/torrent books
- Search any usenet/torrent tracker via Prowlarr, returns books within
Universal search
- Configure download clients in the app settings (Qbittorrent, Deluge,
Transmission, NZBget, SABnzbd)
- Unified download and file handling within the app, same as AA.
### IRC plugin
- Search IRCHighway #ebooks channel for books and download right in the
app.
- No setup needed
- Credit to OpenBooks for the broad idea and inspiration for best
practices for ebook-specific search and download.
### Google Books Metadata Provider
- Create a Google Cloud API key and use Google Books as a metadata
provider
- Not the best source (Hardcover is still recommended), but another
option and further redundancy for universal search
### Book series support
- New "Series" search field in Hardcover provider
- "Series order" sort option - lists books in reading order
- "View Series" button in book details modal to search the full series
- Series info display (e.g., "3 of 12 in The Wheel of Time")
## Others:
- Better format filtering, helpful errors when formats rejected (e.g.,
"Found 3 ebooks but format not supported (.pdf). Enable in Settings >
Formats."
- Directory processing - Handles multi-file torrent/usenet downloads
properly
- Expand search toggle - Skip ISBN search to find more editions
- Filtered authors - Uses primary authors only (excludes
translators/narrators) for better search results
- Language multi-select - Filter releases by multiple languages
Docker / Build / Testing
- pip cache mounts - Faster Docker builds via BuildKit cache
- npm cache mounts - Faster frontend builds
- APT cleanup - Smaller final image size
- Added make restart command for quick restarts without rebuild
- New pytest-based test framework with proper configuration
(pyproject.toml)
- Unit tests for all download clients (qBittorrent, Transmission,
Deluge, NZBGet, SABnzbd)
- Bencode parsing tests
- Cache tests
- Integration tests for Prowlarr handler
- E2E test framework
CaliBrain
Alex
Michael Joshua SaulandClaude Opus 4.6
Alex
