Commit Graph
27 Commits
Author SHA1 Message Date
Alex 5bed0b20f4 Patch: Multi-user and OIDC polish (#612)
- Moved backend OIDC functionality to external library Authlib to help
maintainability
- Separated User settings UI into individual components, allowing for
standard settings UI decorator components to be used.
- Added full support for reverse proxy and CWA users alongside local and
OIDC
- Added mapping and syncing functionality for OIDC, CWA and reverse
proxy users
- Added per-user settings into the app-wide config system. Each config
can be declared as user-overrideable, and app-wide functionality can now
receive user-specific options via standard config calls.
- Added per-user audiobook destination config
- Updated login modal UI for simplified login, plus custom labels for
OIDC login
- Added user visibility in header dropdown
- Unified "restrict settings to admin" to use app-wide user roles.
2026-02-12 14:38:28 +00:00
Michael Joshua SaulandClaude Opus 4.6 2d2f54729f Add OIDC authentication and multi-user support (#606)
Closes #552

## Summary

Adds OIDC authentication and multi-user support to Shelfmark. Users can
now be managed individually with per-user download settings, while
maintaining full backwards compatibility with existing auth modes
(no-auth, builtin, proxy, CWA).

### Authentication
- **OIDC login** with PKCE, auto-discovery, group-based admin mapping
- **Password fallback** when OIDC is enabled (prevents admin lockout)
- **Auto-provisioning** of OIDC users (configurable on/off)
- **Email-based linking** of pre-created users to OIDC accounts
- **Lockout prevention** — requires a local admin before OIDC can be
enabled

### User Management
- **SQLite user database** (`users.db`) with admin CRUD API
- **Users management tab** in settings UI (admin-only)
- **Settings restricted to admins** in multi-user modes (builtin/OIDC) —
non-admin users cannot access settings
- Create, edit, and delete users with role assignment (admin/user)
- Password management for builtin auth users
- OIDC users shown with provider badge (password fields hidden)
- Per-user configurable settings:
  - **Download destination** — custom folder path per user
- **BookLore library & path** — dropdown select, each user's books go to
their own library
  - **Email recipients** — per-user email delivery targets
- **`{User}` template variable** — use in destination paths (e.g.,
`/books/{User}/`)
- Settings override model: per-user values override globals, empty/unset
falls back to global defaults

### Download Scoping
- **Per-user download visibility** — non-admins only see their own
downloads
- **Username display** in downloads sidebar (shows who requested each
download)
- **WebSocket room-based filtering** — admins see all, users see only
their own
- **Download progress scoping** — progress events routed to correct user
rooms

### BookLore Integration
- **Dynamic dropdown selects** for library/path (replaces text inputs)
- **Per-user library/path overrides** via user settings
- **Options cache refresh** after Test Connection

### Security
- SQL injection prevention (column whitelist on user updates)
- Generic OIDC error messages (no internal detail leakage)
- Admin self-deletion and last-local-admin deletion guards
- OIDC role overwrite fix (only updates role when admin_group is
configured)

## Migration

**No migration script needed.** The `users.db` is created automatically
on first startup. Existing builtin auth users are auto-migrated to the
database on their first login. All other auth modes (no-auth, proxy,
CWA) continue working unchanged.

## Test Plan

- [x] All 519 tests passing, 0 failures
- [ ] Test no-auth mode: settings accessible, downloads work without
login
- [ ] Test builtin auth: legacy credentials auto-migrate on login, new
users can be created
- [ ] Test OIDC auth: login flow, callback, auto-provisioning,
group-based admin
- [ ] Test CWA auth: unchanged behavior
- [ ] Test proxy auth: unchanged behavior
- [ ] Test per-user downloads: non-admin sees only own downloads
- [ ] Test BookLore dropdowns: library/path selection, per-user
overrides
- [ ] Test Docker build: no Dockerfile changes needed

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 17:44:27 +00:00
Alex 022e50a0ba Add threading to file system operations (#602) 2026-02-09 18:04:10 +00:00
Alex a560089ce3 Patch: Script improvements + bug fixes (#591)
- Add new booklore API file formats
- Renamed cookie for better login persistence with reverse proxy
- Updated fs.py to try hardlink before atomic move from tmp dir
- Fix transmission URL parsing 
- Fix scenario where file processing of huge files starves the
healthcheck
- Large enhancements to custom scripting, including passing JSON
download info, more consistent activation across output types,
decoupling from staging behavior, and added full documentation.
2026-02-06 13:51:23 +00:00
Alex f84fb082ad Fix: AA mirror behavior (#589)
- Refreshed available AA URLs
- Fixed potential redirect from AA itself causing mirror cache errors
- Added fully customizable mirror list in UI
- Segmented rotation behavior to Auto mode only

Fixes #588
2026-02-06 10:04:31 +00:00
Alex b10458a48b Patch: Migrate bypasser to pure CDP + Misc fixes (#575)
Bypasser:
- Refactored internal bypasser logic to use SeleniumBase Pure CDP mode,
removed chromedriver dependencies and UC code.
- Added dedicated threading for internal bypasser functions, fixes any
potential asyncio CPU spike behavior
- Fixed WebGL issue with Chromium 144. Reverted 1.0.3 hotfix and updated
to latest Chromium

Misc: 
- Added M4A color mapping
- Fix frontend language filtering with multi-language releases
- Added "days" age for usenet/torrent releases
- Improved entrypoint chown efficiency
- Added `ONBOARDING` env variable, default true
2026-02-02 20:32:19 +00:00
Alex e5ccabe1ef Patch: Various additions (#564)
- Added rich Prowlarr search results for whitelisted indexers
- Added torznab query for whitelisted indexers
- Added flags for all Prowlarr indexers
- Added completed external download retry mechanism and "locating" state
- Added client side preference storage of Book/Audiobook search
preference
- Fixed reverse proxy base URL in edge cases
- Added gevent locking for I/O operations, keeps healthcheck alive on
intensive processing operations
- Added M4A supported audiobook option
- Improved file transfer counting and logging with hardlink fallback
warnings
- Fixed proxy auth header for REMOTE_USER scenario
- Dependency tweak for internal bypasser
2026-01-31 12:53:11 +00:00
Marcel Meier 86082c999c Enhance naming templates with arbitrary prefix/suffix support (#560)
As described in https://github.com/calibrain/shelfmark/issues/559
I would like the option to use prefix/suffix text as part of my file
handling.

I appreciate every feedack
2026-01-30 13:53:05 +00:00
Alex 3be99effe4 Base url additions and bug fixes (#519)
- Base URL option in settings for reverse proxy setups
- Fix NZB downloads not deleting on completion
- Fix handling for audiobook files over 100+ parts
- Fix prowlarr search timeout 
- Fix prowlarr categorisation for expanded searches
2026-01-23 13:03:02 +00:00
Alex 03c364e375 Fix: Magnet and hash handling + Various bug fixes (#511) 2026-01-21 19:41:12 +00:00
Alex edf25150bd Fix: various external client issues (#505) 2026-01-20 19:34:18 +00:00
Alex a030bca5d3 Fix: Use info hash for clients (#495)
Passes prowlarr's info hash to download client, existing behavior as
fallback
2026-01-19 20:23:45 +00:00
Alex 8470095534 URL normalization and path mapping tweaks (#489)
- URL normalization (WIP) for external clients / prowlarr / booklore
URLs used.
- More robust handling of Windows path directories in mapping 
- UI tweaks
- Compose clean-ups
2026-01-18 17:36:16 +00:00
Alex f7375d56e2 Heuristic searches, full language support, manual search override (#483)
- Added heuristic-based author and title query creation, stripping out
unnecessary elements that could limit searches
- Improved language support when using Hardcover. Searches will now be
conducted on a per-language basis using localized book titles.
- Added manual search override option in the release modal.
2026-01-17 18:56:10 +00:00
Alex 5a6db5f8a8 Remote path mappings, Client handling improvements (#481) 2026-01-17 14:52:06 +00:00
Alex fd74021594 File processing refactor and Booklore upload support (#474)
- Added new book output option **upload to Booklore**, available in
download settings
- Got annoyed at my messy processing code while implementing Booklore so
refactored the whole thing
- Full black box file processing testing with randomised configuration
- Deluge: Connect via WebUI auth for simplified setup
- Added env vars documentation, auto generated via script, and unlocked
most settings to be used as env vars
2026-01-16 14:45:00 +00:00
Patrick Veverka ba906c45df Feature: rTorrent client support (#463)
This adds in rtorrent for
https://github.com/calibrain/shelfmark/issues/420

The one weird thing I noticed is that the download path needs to be the
same for both (that's not how I typically set it up)

But it definitely adds to rtorrent and gives progress.

**rTorrent client integration:**

* Added a new `RTorrentClient` class in
`shelfmark/release_sources/prowlarr/clients/rtorrent.py` that implements
the download client interface for rTorrent using XML-RPC, supporting
adding, removing, and querying torrent status.
* Registered the rTorrent client in the client registry in
`shelfmark/release_sources/prowlarr/clients/__init__.py`.

**Settings and configuration:**

* Extended the Prowlarr client settings UI and backend
(`shelfmark/release_sources/prowlarr/settings.py`) to add rTorrent as a
selectable client, provide rTorrent-specific configuration fields (URL,
username, password, label, download directory), and implement a
connection test action.
[[1]](diffhunk://#diff-052272b85804cb61162870f262cc7544ef321596ff3ebf08117a6c25afaa3ec5R390)
[[2]](diffhunk://#diff-052272b85804cb61162870f262cc7544ef321596ff3ebf08117a6c25afaa3ec5R539-R582)
[[3]](diffhunk://#diff-052272b85804cb61162870f262cc7544ef321596ff3ebf08117a6c25afaa3ec5R198-R225)

**Test environment and scripts:**

* Updated `docker-compose.test-clients.yml` to add an rTorrent service
for local testing, including configuration, ports, and documentation
updates.
[[1]](diffhunk://#diff-a9fe4200dec6a29947e21c338305d04c8b64a7bddd9b0e519f4ab5382c478ba6R17-L19)
[[2]](diffhunk://#diff-a9fe4200dec6a29947e21c338305d04c8b64a7bddd9b0e519f4ab5382c478ba6R39)
[[3]](diffhunk://#diff-a9fe4200dec6a29947e21c338305d04c8b64a7bddd9b0e519f4ab5382c478ba6R66)
[[4]](diffhunk://#diff-a9fe4200dec6a29947e21c338305d04c8b64a7bddd9b0e519f4ab5382c478ba6R164-R181)
* Enhanced `scripts/test_clients.py` to include rTorrent in the test
suite, with logic for connecting, adding, and removing torrents via
XML-RPC.
[[1]](diffhunk://#diff-c7146552cddc9665e380aec1473363fd8592535ab85f06443478464da8f5a99eR26)
[[2]](diffhunk://#diff-c7146552cddc9665e380aec1473363fd8592535ab85f06443478464da8f5a99eR57)
[[3]](diffhunk://#diff-c7146552cddc9665e380aec1473363fd8592535ab85f06443478464da8f5a99eR88-R90)
[[4]](diffhunk://#diff-c7146552cddc9665e380aec1473363fd8592535ab85f06443478464da8f5a99eR395-R468)
[[5]](diffhunk://#diff-c7146552cddc9665e380aec1473363fd8592535ab85f06443478464da8f5a99eR492)

Closes https://github.com/calibrain/shelfmark/issues/420
2026-01-16 14:28:21 +00:00
0d7a12ca7c Feature: Reverse proxy authentication (#455)
- Changes the auth settings to support more than two auth types
- Added a proxy auth type with settings for user and optionally group
headers
- Added a global middleware `proxy_auth_middleware` to handle proxy auth
(it does nothing if any other auth mode is set)
- Added support for proxy auth to `get_auth_mode`, `login_required`,
`api_login/out`, and `api_auth_check`
- Added a backend check to make protect the API for settings when admin
is required

---------

Co-authored-by: Joshua Tag Howard <git@jthoward.dev>
Co-authored-by: Alex <alex.bilbie1@gmail.com>
2026-01-15 13:27:50 +00:00
Alex bd1ad3495c Proxy, prowlarr, ingest dir fixes (#437) 2026-01-13 14:18:23 +00:00
Alex afeae46821 Rename to Shelfmark and IRC adjustments (#415) 2026-01-11 19:38:38 +00:00
Alex 06778184af Fix: Directory config and init process (#396) 2026-01-05 17:56:00 +00:00
Alex 3d84c5b42f Final tweaks and code cleanup (#392) 2026-01-04 14:12:36 +00:00
Alex 8cb5335234 Fix: Auto parsing of Qbittorrent hash character lengths (#386) 2026-01-01 20:55:12 +00:00
Alex b2887eb4b0 Template based file naming and torrent hardlinking (#385)
- Added alternative file processing mode. Save files directly into a
library folder and set up file names / directories based on user
preference.
- Uses template based naming and directory creation. E.g. {Author} /
{Series} {Title} {Part} etc. Works for saving correctly to libraries
such as Audiobookshelf.
- Use torrent hardlinking directly into library directories.
2026-01-01 12:35:22 +00:00
Alex 91dd479edb Prowlarr non-category search fallback, bypass optimizations, and code cleanup (#379)
- Prowlarr: Added automatic fallback to search without category filter
when indexers return no results with book category (7000), improving
compatibility with indexers that don't support category filtering
- Prowlarr: Hide language filter in UI since Prowlarr has unreliable
language metadata
- Bypass: Refactored internal bypasser with code cleanup, extracted
helper functions, and added health check capability
  - Bypass: Added fingerprint module for screen size handling
- qBittorrent: Fixed connection test to use web API version instead of
app version
- Frontend: Added supported_filters config to control which filters
display per source
- Auth: Improved CWA database path validation (now uses Path object
properly)
2025-12-30 23:19:25 +00:00
Alex 98aada2f55 Selenium update and bypasser enhancements, various bug fixes and tests (#375)
- Updated Selenium to 4.45.6. Includes various crash and memory leak
fixes, plus new bypasser methods
- Bypasser now uses CDP captcha solving as priority - Faster, more
efficient, no PyAutoGUI needed. Fallback to existing methods.
- Better detection and cleanup of old Selenium instances to save memory.
- Added Hardcover graphQL API header detection
- Added AA download counts in details modal
- More robust switching of internal/external bypasser, fixed settings UI
toggle behavior.
2025-12-30 09:42:06 +00:00
Alex a99dc1501d Prowlarr and IRC sources, Google Books, book series support + more (#361)
## Headline features 

### Prowlarr plugin - search trackers and download usenet/torrent books

- Search any usenet/torrent tracker via Prowlarr, returns books within
Universal search
- Configure download clients in the app settings (Qbittorrent, Deluge,
Transmission, NZBget, SABnzbd)
- Unified download and file handling within the app, same as AA. 

### IRC plugin 
- Search IRCHighway #ebooks channel for books and download right in the
app.
- No setup needed
- Credit to OpenBooks for the broad idea and inspiration for best
practices for ebook-specific search and download.

### Google Books Metadata Provider
- Create a Google Cloud API key and use Google Books as a metadata
provider
- Not the best source (Hardcover is still recommended), but another
option and further redundancy for universal search

### Book series support
  - New "Series" search field in Hardcover provider
  - "Series order" sort option - lists books in reading order
  - "View Series" button in book details modal to search the full series
  - Series info display (e.g., "3 of 12 in The Wheel of Time")

## Others: 

- Better format filtering, helpful errors when formats rejected (e.g.,
"Found 3 ebooks but format not supported (.pdf). Enable in Settings >
Formats."
- Directory processing - Handles multi-file torrent/usenet downloads
properly
- Expand search toggle - Skip ISBN search to find more editions
- Filtered authors - Uses primary authors only (excludes
translators/narrators) for better search results
- Language multi-select - Filter releases by multiple languages

 Docker / Build / Testing

  - pip cache mounts - Faster Docker builds via BuildKit cache
  - npm cache mounts - Faster frontend builds
  - APT cleanup - Smaller final image size
  - Added make restart command for quick restarts without rebuild
- New pytest-based test framework with proper configuration
(pyproject.toml)
- Unit tests for all download clients (qBittorrent, Transmission,
Deluge, NZBGet, SABnzbd)
  - Bencode parsing tests
  - Cache tests
  - Integration tests for Prowlarr handler
  - E2E test framework
2025-12-27 14:59:06 +00:00