Compare commits

..
31 Commits
Author SHA1 Message Date
Alex a2a5a22324 Fix theming bug (#722) 2026-03-07 19:13:28 +00:00
Alex a7db7f04e9 Hardcover tweaks (#720) 2026-03-07 18:16:40 +00:00
Alex 9d08bb3ef1 Expanded Hardcover list features (#719)
- Adds full interaction with Hardcover lists, including adding and
removing from lists + want to read status
- List selection exposed in search results, details modal and release
modal
- Added automatic list dropdown when selecting "list" search
- Added auto-removal of books from a list when downloading from that
specific list page
- Changed search selector to hover-activated
2026-03-07 15:33:46 +00:00
Alex 80aa289a64 Misc fixes (#718)
- Update file movement to prefer copy
- Improved mirror config overwriting on app updates
- Request / user DB hardening
2026-03-07 10:30:47 +00:00
Alex edb437e905 Fix sorting + update readme (#715)
- Harden default sort preference use + fix series ordering use
- Update readme with contribution and project scope disclaimers
2026-03-06 17:06:20 +00:00
Alex 72464e32b8 Update makefile test (#713) 2026-03-06 15:00:05 +00:00
Alex 60893b19c6 Search UI revamp, series search and search suggestions (#712)
- Restructured search field options into the left-hand selector.
Includes dynamic options for each provider.
- Moved Hardcover list and manual search mode into the left hand
selector
- Added search mode and metadata provider into the search options area
- Added new Hardcover series API query and live series suggestions
- Added live Hardcover author and title suggestions
2026-03-06 14:44:55 +00:00
Alex 8bb188c903 Refactor direct source to use universal API (#711) 2026-03-06 12:59:37 +00:00
Alex d6d10a450e Enhance Hardcover lists (#710) 2026-03-06 10:45:15 +00:00
Alex 4b0d1aef13 Download history refactor pt3 (#706)
- Added canonical per-user visibility of requests and downloads via new
activity view table. Users get fully independent activity and history
views, while admins still see all.
- Replaces janky frontend + backend combination
2026-03-05 19:53:22 +00:00
Giovanni Scieri 447ed1a924 fix(search): include default language in search query filters (#704)
## Bug description

When a default language was configured, it was **not passed as a search
filter**.
This occurred regardless of configuration via UI or environment
variables.

## Fix

Updated filter logic so that the default language is always applied when
no explicit filter is provided:

```python
for value in filters.lang if filters.lang else config.BOOK_LANGUAGE or []:
    if value and value != "all":
        filters_query += f"&lang={quote(value)}"
```

This ensures:
- the default language is used when available
- empty or invalid values are ignored
- "all" does not apply a language filter

## Testing
- default language via UI → search filters correctly
- default language via environment variable → search filters correctly
- "all" value → no language filter applied
2026-03-05 16:25:19 +00:00
Alex ba92ad90bc Refine UI and adjust content type settings (#705)
- Tweak manual search toggle position
- Refinements to the Hardcover list dropdown behavior
- Hide the content type dropdown when a content type is blocked for a
user
- Fixes to Hardcover author parsing to strip out initialed names
- Remove `env_supported=false` for security config options.
2026-03-05 16:24:03 +00:00
Alex cce2c10704 Download history refactor pt.2 (#703)
Two-phase download history: downloads are now recorded in the DB at
queue time (not just at terminal time), eliminating the need to
reconstruct metadata in the terminal hook and removing the
`_is_graduated_request_download()` request-scan mess
2026-03-05 13:06:34 +00:00
Alex fbe25725d3 Download history refactor (#700)
- Much simpler handling of downloads in the activity sidebar, and
improved storage, persistence and UI behavior.
- Replace `ActivityService` with direct storage on
`DownloadHistoryService` and `download_requests` and removes the
activity_log/activity_dismissals tables
- Simplify no-auth mode by removing the fake user row pattern, handled
internally
- Add local download fallback so history entries can still serve files
after tasks leave the queue
- Downloads, requests and history are now entirely persistent between
updates / restarts, and correctly tied to each user.
2026-03-04 19:10:06 +00:00
Alex bd65bccf52 Fix: Refresh mirrors (#695) 2026-03-03 22:14:18 +00:00
Alex 71900e00db Feature: Hardcover list search (#694)
- Adds the functionality to search Hardcover lists, either public lists
or user's private lists
- Paste a list URL into the search box to view results
- Select a specific list from user's collection from advanced fields
dropdown
- Fixes content_type parameter in URL search query to use book/audiobook
2026-03-03 21:53:43 +00:00
Alex de18f2b9fe Fix: File movement trigger event (#691)
Creates IN_MOVED_TO event, fixes CWA ingest folder detection when file
movement fallback occurs
2026-03-03 18:00:48 +00:00
Alex 6718848cfb Feature: Manual search option (#687)
- Adds a toggle to advanced search fields to search sources manually
instead of using metadata
- Hidden for users when "Request Book" or "Blocked" default policy is in
effect.
2026-03-02 18:41:03 +00:00
Alex 7d992c3918 User DB cleanup and refactor (#686)
- Refactored user and request code to avoid any database conflicts
- Fix threading behavior with custom script execution
- Harden the no_auth activity user filtering
- Add a hint to add local admin if none is created
- Added secret key to persist login states across updates / restarts
2026-03-02 15:41:15 +00:00
Alex 9593c040b0 Misc features: Retries, user search config, sort by format, admin download control (#679)
- Added the manual retry option for failed downloads
- Added the ability to retry failed post-processing using existing
downloaded file
- Added admin-visible "Download as" selector, admin chooses a user to
download on-behalf of - inherits their output preferences.
- Added search mode and default metadata provider / release source
options to User Preferences and My Account settings.
- Added sort by format option in release results
- Added {OriginalName} renaming field option, to retain the exact
downloaded filename
- Frontend dependency updates - fixes rollup vulnerability from this
week

Closes #662 #656 #649 #562
2026-03-01 19:47:57 +00:00
Alex ea0d06ae08 Further notification tweaks (#671)
- Improved multi-URL notification handling
- Tweak Apprise validation to catch errors earlier
- Much improved notification logging and UI response
- More robust notification tests
2026-02-28 10:16:01 +00:00
Alex 0f3a06bc9c Fix: User DB hardening and apprise tweak (#668) 2026-02-27 21:06:39 +00:00
Alex d78aad066b Fix: Apprise logging and no_auth hardening (#667)
- Passes apprise logging into shelfmark logs
- Update UI activity dismissal when no authentication is active
2026-02-27 15:49:14 +00:00
Alex e7d2845235 Fixes: Auth edge cases, apprise logging, scoring and release refactors (#665)
- Added migration for builtin auth users who used dev builds during
multi-user development
- Display apprise errors in logging
- Fix user provisioning in reverse proxy auth setups
- Refactor scoring and release modal utils
2026-02-27 10:21:06 +00:00
Alex ac36d539c8 Patch: Various fixes (#660)
Various fixes from the last couple days: 

- Add manual approval option for book/audiobook requests (#651)
- Add flagged HTTP headers 
- Add filesystem fallback - copy + delete when hardlink/move fails
across filesystems (#647)
- Dependency updates
- Tweak frontend test config (simplified tsconfig for tests)
- Fix overlapping sort scoring in release modal - duplicate scoring keys
caused incorrect release ordering (#654 )
- Fix stale search session after download - search state was not
refreshed when returning from a download (#659)
- Fix multi-format release filtering - releases with multiple formats
were incorrectly excluded by the format filter (#658)
- Fix config persistence when action button is used - clicking "Test
connection" reset unsaved settings (#657)
- Fix request grid text positioning in admin request policy panel 
- Fix rTorrent path discovery (#653)
- Fix `/login` API check (#650)
2026-02-25 18:44:46 +00:00
Alex 91cbd51b67 Fix flask version (#646) 2026-02-23 10:09:16 +00:00
Alex c80c88676c Fix direct search request flow (#644)
Fixes #643
2026-02-23 09:27:55 +00:00
Alex 0d271f1f69 Patch: Certificate validation setting + Misc fixes (#642)
- Add certificate validation setting
- Fix some OIDC providers not linking emails to local users
- Reintroduce sort by peers option for prowlarr results
- Fix "All languages" search query reverting to default language
- Fix download/request dismissal with multiple admin users
- Fix download / request behavior on details modal
2026-02-22 23:07:55 +00:00
Alex 014fc38b48 Patch: OIDC polish (#636)
- Added two env vars for OIDC login: 
- HIDE_LOCAL_AUTH - Remove the "password" option on login page when OIDC
enabled
  - OIDC_AUTO_REDIRECT - Immediately launch OIDC provider page
- Improved UX for initial OIDC setup, including creating a local admin
user
- Added callback URL label to OIDC setup page
- Fix Qbittorrent save path bug
2026-02-21 11:51:11 +00:00
Alex fdd46852f2 Add new docs (#633) 2026-02-20 18:07:43 +00:00
Alex a57d081caa Fix OIDC name fallback with limited responses (#632) 2026-02-20 16:24:37 +00:00
242 changed files with 19464 additions and 7285 deletions
+1
View File
@@ -233,3 +233,4 @@ AGENTS.md
.claude/
.playwright-mcp/
frontend-dist/
node_modules/
+2 -3
View File
@@ -68,7 +68,7 @@ RUN apt-get update && \
# For debug
zip iputils-ping \
# For user switching
sudo \
gosu \
# --- Tor support (activated via USING_TOR=true) ---
tor \
supervisor \
@@ -151,8 +151,7 @@ RUN --mount=type=cache,target=/root/.cache/pip \
pip install -r requirements-shelfmark.txt
# Grant read/execute permissions to others
RUN chmod -R o+rx /usr/bin/chromium && \
chmod -R o+rwx /usr/local/lib/python3.10/site-packages/seleniumbase/drivers/
RUN chmod -R o+rx /usr/bin/chromium
# Default command to run the application entrypoint script
CMD ["/app/entrypoint.sh"]
+7 -1
View File
@@ -1,4 +1,4 @@
.PHONY: help install dev build preview typecheck clean up down docker-build refresh restart
.PHONY: help install dev build preview typecheck frontend-test clean up down docker-build refresh restart
# Frontend directory
FRONTEND_DIR := src/frontend
@@ -16,6 +16,7 @@ help:
@echo " build - Build frontend for production"
@echo " preview - Preview production build"
@echo " typecheck - Run TypeScript type checking"
@echo " frontend-test - Run frontend unit tests"
@echo " clean - Remove node_modules and build artifacts"
@echo ""
@echo "Backend (Docker):"
@@ -50,6 +51,11 @@ typecheck:
@echo "Running TypeScript type checking..."
cd $(FRONTEND_DIR) && npm run typecheck
# Run frontend unit tests
frontend-test:
@echo "Running frontend unit tests..."
cd $(FRONTEND_DIR) && npm run test:unit
# Clean build artifacts and dependencies
clean:
@echo "Cleaning build artifacts and dependencies..."
Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.1 MiB

After

Width:  |  Height:  |  Size: 2.0 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 151 KiB

After

Width:  |  Height:  |  Size: 148 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 854 KiB

After

Width:  |  Height:  |  Size: 848 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.3 MiB

After

Width:  |  Height:  |  Size: 2.1 MiB

+2 -1
View File
@@ -1,6 +1,7 @@
services:
shelfmark-lite:
image: ghcr.io/calibrain/shelfmark-lite:latest
container_name: shelfmark-lite
environment:
# EXT_BYPASSER_URL: http://flaresolverr:8191 #If using Flaresolverr
PUID: 1000
@@ -12,4 +13,4 @@ services:
- /path/to/books:/books # Default destination for book downloads
- /path/to/config:/config # App configuration
# Required for torrent / usenet - path must match your download client's volume exactly
# - /path/to/downloads:/path/to/downloads
# - /path/to/downloads:/path/to/downloads
+2
View File
@@ -12,6 +12,8 @@ services:
- SYS_PTRACE
environment:
DEBUG: true
# HIDE_LOCAL_AUTH: true
OIDC_AUTO_REDIRECT: true
volumes:
- ./.local/config:/config
- ./.local/books:/books
+177 -11
View File
@@ -10,6 +10,7 @@ This document lists all configuration options that can be set via environment va
- [General](#general)
- [Search Mode](#search-mode)
- [Downloads](#downloads)
- [Security](#security)
- [Network](#network)
- [Advanced](#advanced)
- [Prowlarr](#prowlarr)
@@ -251,8 +252,8 @@ The release source tab to open by default in the release modal.
| `BOOKS_OUTPUT_MODE` | Choose where completed book files are sent. | string (choice) | `folder` |
| `INGEST_DIR` | Directory where downloaded files are saved. Use {User} for per-user folders (e.g. /books/{User}). | string | `/books` |
| `FILE_ORGANIZATION` | Choose how downloaded book files are named and organized. | string (choice) | `rename` |
| `TEMPLATE_RENAME` | Variables: {Author}, {Title}, {Year}, {User}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders. | string | `{Author} - {Title} ({Year})` |
| `TEMPLATE_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. | string | `{Author}/{Title} ({Year})` |
| `TEMPLATE_RENAME` | Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension). Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders. Applies to single-file downloads. | string | `{Author} - {Title} ({Year})` |
| `TEMPLATE_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension). Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. | string | `{Author}/{Title} ({Year})` |
| `HARDLINK_TORRENTS` | Create hardlinks instead of copying. Preserves seeding but archives won't be extracted. Don't use if destination is a library ingest folder. | boolean | `false` |
| `BOOKLORE_HOST` | Base URL of your Booklore instance | string | _none_ |
| `BOOKLORE_USERNAME` | Booklore account username | string | _none_ |
@@ -273,8 +274,8 @@ The release source tab to open by default in the release modal.
| `EMAIL_ALLOW_UNVERIFIED_TLS` | Disable TLS certificate verification (not recommended). | boolean | `false` |
| `DESTINATION_AUDIOBOOK` | Directory where downloaded audiobook files are saved. Leave empty to use the Books destination. | string | _none_ |
| `FILE_ORGANIZATION_AUDIOBOOK` | Choose how downloaded audiobook files are named and organized. | string (choice) | `rename` |
| `TEMPLATE_AUDIOBOOK_RENAME` | Variables: {Author}, {Title}, {Year}, {User}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders. | string | `{Author} - {Title}` |
| `TEMPLATE_AUDIOBOOK_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. | string | `{Author}/{Title}` |
| `TEMPLATE_AUDIOBOOK_RENAME` | Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension), {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders. Applies to single-file downloads. | string | `{Author} - {Title}` |
| `TEMPLATE_AUDIOBOOK_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension), {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. | string | `{Author}/{Title}` |
| `HARDLINK_TORRENTS_AUDIOBOOK` | Create hardlinks instead of copying. Preserves seeding but archives won't be extracted. Don't use if destination is a library ingest folder. | boolean | `true` |
| `AUTO_OPEN_DOWNLOADS_SIDEBAR` | Automatically open the downloads sidebar when a new download is queued. | boolean | `false` |
| `DOWNLOAD_TO_BROWSER` | Automatically download completed files to your browser. | boolean | `false` |
@@ -318,7 +319,7 @@ Choose how downloaded book files are named and organized.
**Naming Template**
Variables: {Author}, {Title}, {Year}, {User}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders.
Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension). Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders. Applies to single-file downloads.
- **Type:** string
- **Default:** `{Author} - {Title} ({Year})`
@@ -327,7 +328,7 @@ Variables: {Author}, {Title}, {Year}, {User}. Universal adds: {Series}, {SeriesP
**Path Template**
Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.
Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension). Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.
- **Type:** string
- **Default:** `{Author}/{Title} ({Year})`
@@ -528,7 +529,7 @@ Choose how downloaded audiobook files are named and organized.
**Naming Template**
Variables: {Author}, {Title}, {Year}, {User}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders.
Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension), {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders. Applies to single-file downloads.
- **Type:** string
- **Default:** `{Author} - {Title}`
@@ -537,7 +538,7 @@ Variables: {Author}, {Title}, {Year}, {User}, {Series}, {SeriesPosition}, {Subti
**Path Template**
Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.
Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension), {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.
- **Type:** string
- **Default:** `{Author}/{Title}`
@@ -592,10 +593,165 @@ How long to keep completed/failed downloads in the queue display.
</details>
## Security
| Variable | Description | Type | Default |
|----------|-------------|------|---------|
| `AUTH_METHOD` | Select the authentication method for accessing Shelfmark. | string (choice) | `none` |
| `PROXY_AUTH_USER_HEADER` | The HTTP header your proxy uses to pass the authenticated username. | string | `X-Auth-User` |
| `PROXY_AUTH_LOGOUT_URL` | The URL to redirect users to for logging out. Leave empty to disable logout functionality. | string | _empty string_ |
| `PROXY_AUTH_ADMIN_GROUP_HEADER` | Optional: header your proxy uses to pass user groups/roles. | string | `X-Auth-Groups` |
| `PROXY_AUTH_ADMIN_GROUP_NAME` | Optional: users in this group are treated as admins. Leave blank to skip group-based admin detection. | string | _empty string_ |
| `OIDC_DISCOVERY_URL` | OpenID Connect discovery endpoint URL. Usually ends with /.well-known/openid-configuration. | string | _none_ |
| `OIDC_CLIENT_ID` | OAuth2 client ID from your identity provider. | string | _none_ |
| `OIDC_CLIENT_SECRET` | OAuth2 client secret from your identity provider. | string (secret) | _none_ |
| `OIDC_SCOPES` | OAuth2 scopes to request from the identity provider. Managed automatically: includes essential scopes and the group claim when using admin group authorization. | string | `openid,email,profile` |
| `OIDC_GROUP_CLAIM` | The name of the claim in the ID token that contains user groups. | string | `groups` |
| `OIDC_ADMIN_GROUP` | Users in this group will be given admin access (if enabled below). Leave empty to use database roles only. | string | _empty string_ |
| `OIDC_USE_ADMIN_GROUP` | When enabled, users in the Admin Group are granted admin access. When disabled, admin access is determined solely by database roles. | boolean | `true` |
| `OIDC_AUTO_PROVISION` | Automatically create a user account on first OIDC login. When disabled, users must be pre-created by an admin. | boolean | `true` |
| `OIDC_BUTTON_LABEL` | Custom label for the OIDC sign-in button on the login page. | string | _empty string_ |
<details>
<summary>Detailed descriptions</summary>
#### `AUTH_METHOD`
**Authentication Method**
Select the authentication method for accessing Shelfmark.
- **Type:** string (choice)
- **Default:** `none`
- **Options:** `none` (No Authentication), `builtin` (Local), `proxy` (Proxy Authentication), `oidc` (OIDC (OpenID Connect)), `cwa` (Calibre-Web Database)
#### `PROXY_AUTH_USER_HEADER`
**Proxy Auth User Header**
The HTTP header your proxy uses to pass the authenticated username.
- **Type:** string
- **Default:** `X-Auth-User`
#### `PROXY_AUTH_LOGOUT_URL`
**Proxy Auth Logout URL**
The URL to redirect users to for logging out. Leave empty to disable logout functionality.
- **Type:** string
- **Default:** _empty string_
#### `PROXY_AUTH_ADMIN_GROUP_HEADER`
**Proxy Auth Admin Group Header**
Optional: header your proxy uses to pass user groups/roles.
- **Type:** string
- **Default:** `X-Auth-Groups`
#### `PROXY_AUTH_ADMIN_GROUP_NAME`
**Proxy Auth Admin Group**
Optional: users in this group are treated as admins. Leave blank to skip group-based admin detection.
- **Type:** string
- **Default:** _empty string_
#### `OIDC_DISCOVERY_URL`
**Discovery URL**
OpenID Connect discovery endpoint URL. Usually ends with /.well-known/openid-configuration.
- **Type:** string
- **Default:** _none_
- **Required:** Yes
#### `OIDC_CLIENT_ID`
**Client ID**
OAuth2 client ID from your identity provider.
- **Type:** string
- **Default:** _none_
- **Required:** Yes
#### `OIDC_CLIENT_SECRET`
**Client Secret**
OAuth2 client secret from your identity provider.
- **Type:** string (secret)
- **Default:** _none_
- **Required:** Yes
#### `OIDC_SCOPES`
**Scopes**
OAuth2 scopes to request from the identity provider. Managed automatically: includes essential scopes and the group claim when using admin group authorization.
- **Type:** string
- **Default:** `openid,email,profile`
#### `OIDC_GROUP_CLAIM`
**Group Claim Name**
The name of the claim in the ID token that contains user groups.
- **Type:** string
- **Default:** `groups`
#### `OIDC_ADMIN_GROUP`
**Admin Group Name**
Users in this group will be given admin access (if enabled below). Leave empty to use database roles only.
- **Type:** string
- **Default:** _empty string_
#### `OIDC_USE_ADMIN_GROUP`
**Use Admin Group for Authorization**
When enabled, users in the Admin Group are granted admin access. When disabled, admin access is determined solely by database roles.
- **Type:** boolean
- **Default:** `true`
#### `OIDC_AUTO_PROVISION`
**Auto-Provision Users**
Automatically create a user account on first OIDC login. When disabled, users must be pre-created by an admin.
- **Type:** boolean
- **Default:** `true`
#### `OIDC_BUTTON_LABEL`
**Login Button Label**
Custom label for the OIDC sign-in button on the login page.
- **Type:** string
- **Default:** _empty string_
</details>
## Network
| Variable | Description | Type | Default |
|----------|-------------|------|---------|
| `CERTIFICATE_VALIDATION` | Controls SSL/TLS certificate verification for outbound connections. Disable for self-signed certificates on internal services (e.g. OIDC providers, Prowlarr). | string (choice) | `enabled` |
| `CUSTOM_DNS` | DNS provider for domain resolution. 'Auto' rotates through providers on failure. | string (choice) | `auto` |
| `CUSTOM_DNS_MANUAL` | Comma-separated list of DNS server IP addresses (e.g., 8.8.8.8, 1.1.1.1). | string | _none_ |
| `USE_DOH` | Use encrypted DNS queries for improved reliability and privacy. | boolean | `true` |
@@ -609,6 +765,16 @@ How long to keep completed/failed downloads in the queue display.
<details>
<summary>Detailed descriptions</summary>
#### `CERTIFICATE_VALIDATION`
**Certificate Validation**
Controls SSL/TLS certificate verification for outbound connections. Disable for self-signed certificates on internal services (e.g. OIDC providers, Prowlarr).
- **Type:** string (choice)
- **Default:** `enabled`
- **Options:** `enabled` (Enabled (Recommended)), `disabled_local` (Disabled for Local Addresses), `disabled` (Disabled)
#### `CUSTOM_DNS`
**DNS Provider**
@@ -1773,7 +1939,7 @@ Timeout for external bypasser requests in milliseconds.
| Variable | Description | Type | Default |
|----------|-------------|------|---------|
| `AA_BASE_URL` | Select 'Auto' to try mirrors from your list on startup and fall back on failures. Choosing a specific mirror locks Shelfmark to that mirror (no fallback). | string (choice) | `auto` |
| `AA_MIRROR_URLS` | Editable list of AA mirrors. Used to populate the Primary Mirror dropdown and the order used when Auto is selected. Type a URL and press Enter to add. Order matters for auto-rotation | string | `https://annas-archive.gl,https://annas-archive.li` |
| `AA_MIRROR_URLS` | Editable list of AA mirrors. Used to populate the Primary Mirror dropdown and the order used when Auto is selected. Type a URL and press Enter to add. Order matters for auto-rotation | string | `https://annas-archive.gl,https://annas-archive.pk,https://annas-archive.vg,https://annas-archive.gd` |
| `AA_ADDITIONAL_URLS` | Deprecated. Use Mirrors instead. This is kept for backwards compatibility with existing installs and environment variables. | string | _none_ |
| `LIBGEN_ADDITIONAL_URLS` | Comma-separated list of custom LibGen mirrors to add to the defaults. | string | _none_ |
| `ZLIB_PRIMARY_URL` | Z-Library mirror to use for downloads. | string (choice) | `https://z-lib.fm` |
@@ -1792,7 +1958,7 @@ Select 'Auto' to try mirrors from your list on startup and fall back on failures
- **Type:** string (choice)
- **Default:** `auto`
- **Options:** `auto` (Auto (Recommended)), `https://annas-archive.gl` (annas-archive.gl), `https://annas-archive.li` (annas-archive.li)
- **Options:** `auto` (Auto (Recommended)), `https://annas-archive.gl` (annas-archive.gl), `https://annas-archive.pk` (annas-archive.pk), `https://annas-archive.vg` (annas-archive.vg), `https://annas-archive.gd` (annas-archive.gd)
#### `AA_MIRROR_URLS`
@@ -1801,7 +1967,7 @@ Select 'Auto' to try mirrors from your list on startup and fall back on failures
Editable list of AA mirrors. Used to populate the Primary Mirror dropdown and the order used when Auto is selected. Type a URL and press Enter to add. Order matters for auto-rotation
- **Type:** string
- **Default:** `https://annas-archive.gl,https://annas-archive.li`
- **Default:** `https://annas-archive.gl,https://annas-archive.pk,https://annas-archive.vg,https://annas-archive.gd`
#### `AA_ADDITIONAL_URLS`
+50
View File
@@ -0,0 +1,50 @@
# OpenID Connect (OIDC) Authentication
## Callback URL
```
https://<your-shelfmark-domain>/api/auth/oidc/callback
```
With a subpath (`URL_BASE=/shelfmark/`):
```
https://<your-shelfmark-domain>/shelfmark/api/auth/oidc/callback
```
The callback URL is constructed from the incoming request, so your reverse proxy must forward `X-Forwarded-Proto` and `X-Forwarded-Host` correctly. PKCE (S256) is used automatically.
## Settings
Configure in **Settings → Security → Authentication Method → OIDC**.
| Setting | Description | Default |
|---------|-------------|---------|
| Discovery URL | `/.well-known/openid-configuration` endpoint | — |
| Client ID | OAuth2 client ID | — |
| Client Secret | OAuth2 client secret | — |
| Scopes | Scopes to request. The group claim is added automatically when admin group authorization is enabled | `openid email profile` |
| Group Claim Name | Claim containing user groups | `groups` |
| Admin Group Name | Group granted admin access. Leave empty for database-only roles | — |
| Use Admin Group for Authorization | Toggle group-based admin detection | `true` |
| Auto-Provision Users | Create accounts on first login | `true` |
| Login Button Label | Custom text for the sign-in button | — |
Use **Test Connection** to verify discovery and client configuration before attempting login.
## Environment Variables
These optional environment variables control login page behavior when OIDC is enabled.
| Variable | Description | Default |
|----------|-------------|---------|
| `HIDE_LOCAL_AUTH` | Hide the username/password login option, so only the OIDC button is shown | `false` |
| `OIDC_AUTO_REDIRECT` | Automatically redirect to the OIDC provider instead of showing the login page | `false` |
If both are enabled, users are redirected straight to the OIDC provider. On failure they return to the login page with an error message but no password fallback.
## Troubleshooting
- **Issuer validation failed** — The issuer in the token doesn't match the discovery document. Check your provider's external URL / issuer configuration.
- **Callback URL mismatch** — Reverse proxy isn't forwarding `X-Forwarded-Proto` or `X-Forwarded-Host`, so the constructed callback URL doesn't match what's registered in the provider.
- **Account not found** — Auto-provision is disabled and the user hasn't been pre-created by an admin.
+8 -1
View File
@@ -19,6 +19,7 @@ http://your-server:8084/?q=harry+potter
| `lang` | Filter by language (ISO 639-1 code) | `/?lang=en` |
| `format` | Filter by file format | `/?format=epub` |
| `content` | Filter by content type | `/?content=fiction` |
| `content_type` | Select media type (`ebook` or `audiobook`) in Universal mode only | `/?q=dune&content_type=audiobook` |
| `sort` | Sort order for results | `/?sort=newest` |
## Multiple Values
@@ -57,15 +58,21 @@ Some parameters support multiple values by repeating the parameter:
/?q=science+fiction&sort=newest
```
**Universal search as audiobook:**
```
/?q=dune&content_type=audiobook
```
## Search Mode Behavior
### Direct Download Mode (default)
All parameters are used to filter results from the direct download source.
`content_type` is ignored in Direct mode.
### Universal Mode
Only `q` and `sort` are used. Other parameters (author, title, format, etc.) are silently ignored since metadata providers have their own search capabilities.
`q`, `sort`, and `content_type` are used. Other parameters (author, title, format, etc.) are silently ignored since metadata providers have their own search capabilities.
## Notes
+98
View File
@@ -0,0 +1,98 @@
# Users & Requests
Configure in **Settings → Users & Requests**.
## Authentication Methods
Shelfmark supports four authentication methods, configured in **Settings → Security**.
### Local
You create user accounts directly in Shelfmark with a username and password. At least one local admin account must exist before this mode can be enabled.
### Proxy Authentication
Your reverse proxy handles authentication and passes the username to Shelfmark via a header (e.g. `Remote-User`). Accounts are created automatically on first sign-in. If a local user with the same username already exists, the proxy identity will be linked to that account rather than creating a duplicate. Admin status can optionally be derived from a groups header.
### OIDC (OpenID Connect)
Users sign in through your identity provider. Accounts are created automatically on first login (unless auto-provisioning is disabled, in which case you need to pre-create them). If a local user with a matching verified email already exists, the OIDC identity will be linked to that account on first sign-in. Admin status can optionally be derived from a group claim.
A local admin account is required as a fallback. See [OIDC](oidc.md) for provider setup.
### Calibre-Web Database
User accounts are synced from your Calibre-Web `app.db`. If a local user with a matching email already exists, the CWA identity will be linked to that account. Roles are kept in sync with CWA. Users removed from CWA are cleaned up on the next sync.
Requires mounting your Calibre-Web `app.db` to `/auth/app.db`.
## Per-User Settings
Admins can configure per-user settings by editing a user in the user management panel. Non-admin users can also edit their own settings through **My Account** (accessible from the user menu). Admins control which sections are visible in My Account via the **Visible Self-Settings Sections** option.
There are three categories of per-user settings:
### Delivery Preferences
Override where a user's downloads are sent. Options depend on the global output mode configuration:
- **Output mode** — Folder, Email (SMTP), or BookLore (API)
- **Destination** — A custom folder path for this user's ebook downloads
- **Audiobook destination** — A custom folder path for audiobook downloads
- **BookLore library/path** — Per-user BookLore target (when using BookLore output mode)
- **Email recipient** — Per-user email address (when using Email output mode)
### Notifications
Users can configure personal notification routes, separate from the global notification settings. Each route targets a URL (e.g. an Apprise-compatible endpoint) and can be scoped to specific event types or all events.
### Request Policy (admin-only)
Admins can override the default ebook/audiobook modes and request rules for individual users. See [Per-User Overrides](#per-user-overrides) below.
---
## Requests
The request system controls whether users can download directly or need admin approval first.
### Policy Modes
Each content type (ebook, audiobook) has a default mode that sets the baseline:
| Mode | Behaviour |
|------|-----------|
| **Download** | Users download directly, no approval needed |
| **Request Release** | Users pick a specific release, then submit it for admin approval |
| **Request Book** | Users request the book itself — an admin picks the release and fulfils it |
| **Blocked** | No downloads or requests allowed |
### Settings
| Setting | Description | Default |
|---------|-------------|---------|
| Enable Requests | Master toggle. When off, everyone downloads directly | Off |
| Default Ebook Mode | Baseline mode for all ebook sources | Download |
| Default Audiobook Mode | Baseline mode for all audiobook sources | Download |
| Request Rules | Per-source overrides (see below) | None |
| Max Pending Requests Per User | Open request limit per user | 20 |
| Allow Notes on Requests | Let users attach a note when submitting | On |
### Request Rules
The rules matrix lets you override the mode for specific source + content type combinations. Rules can only be **equal to or more restrictive** than the content-type default — they cannot grant more access than the baseline.
For example, if the default ebook mode is "Download", a rule can restrict a specific source to "Request Release" or "Blocked", but not the other way around. If no rule matches, the content-type default applies.
### Per-User Overrides
Admins can override the default ebook/audiobook modes and request rules for individual users. Per-user rules are overlaid on the global rules, not replacing them.
### Request Lifecycle
1. User submits a request (book or release level, depending on the resolved policy mode)
2. Request appears in the admin request queue as **pending**
3. Admin either **fulfils** (queues a download) or **rejects** the request
4. For fulfilled requests, delivery state is tracked through the download pipeline
5. If delivery fails, an admin can reopen the request to try a different release
6. Users can cancel their own pending requests
+24 -12
View File
@@ -45,9 +45,9 @@ if is_truthy "$ENABLE_LOGGING_VALUE"; then
LOG_DIR=${LOG_ROOT:-/var/log/}/shelfmark
mkdir -p "$LOG_DIR"
LOG_FILE="${LOG_DIR}/shelfmark_entrypoint.log"
# Cleanup any existing files or folders in the log directory
rm -rf "$LOG_DIR"/*
# Keep the previous entrypoint log instead of deleting all history on boot.
[ -f "${LOG_FILE}.prev" ] && rm -f "${LOG_FILE}.prev"
[ -f "$LOG_FILE" ] && mv "$LOG_FILE" "${LOG_FILE}.prev"
fi
(
@@ -127,14 +127,26 @@ USERNAME=$(getent passwd "$RUN_UID" | cut -d: -f1)
echo "Username for UID $RUN_UID is $USERNAME"
test_write() {
folder=$1
test_file=$folder/shelfmark_TEST_WRITE
mkdir -p $folder
(
echo 0123456789_TEST | sudo -E -u "$USERNAME" HOME=/app tee $test_file > /dev/null
)
FILE_CONTENT=$(cat $test_file || echo "")
rm -f $test_file
local folder=$1
local test_file="$folder/shelfmark_TEST_WRITE"
local FILE_CONTENT
local result
local result_text
if ! mkdir -p "$folder"; then
echo "Failed to create directory for write test: $folder"
return 1
fi
if ! (
echo 0123456789_TEST | gosu "$USERNAME" env HOME=/app tee "$test_file" > /dev/null
); then
echo "Failed to write test file in $folder as $USERNAME"
return 1
fi
FILE_CONTENT=$(cat "$test_file" 2>/dev/null || echo "")
rm -f "$test_file"
[ "$FILE_CONTENT" = "0123456789_TEST" ]
result=$?
if [ $result -eq 0 ]; then
@@ -342,4 +354,4 @@ echo "Setting umask to $UMASK_VALUE"
umask $UMASK_VALUE
stop_file_logging
exec sudo -E -u "$USERNAME" HOME=/app $command
exec gosu "$USERNAME" env HOME=/app $command
-1
View File
@@ -1 +0,0 @@
../baseline-browser-mapping/dist/cli.js
-17
View File
@@ -1,17 +0,0 @@
{
"name": "shelfmark",
"lockfileVersion": 3,
"requires": true,
"packages": {
"node_modules/baseline-browser-mapping": {
"version": "2.9.19",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.9.19.tgz",
"integrity": "sha512-ipDqC8FrAl/76p2SSWKSI+H9tFwm7vYqXQrItCuiVPt26Km0jS+NzSsBWAaBusvSbQcfJG+JitdMm+wZAgTYqg==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"baseline-browser-mapping": "dist/cli.js"
}
}
}
}
-201
View File
@@ -1,201 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-463
View File
@@ -1,463 +0,0 @@
# [`baseline-browser-mapping`](https://github.com/web-platform-dx/web-features/packages/baseline-browser-mapping)
By the [W3C WebDX Community Group](https://www.w3.org/community/webdx/) and contributors.
`baseline-browser-mapping` provides:
- An `Array` of browsers compatible with Baseline Widely available and Baseline year feature sets via the [`getCompatibleVersions()` function](#get-baseline-widely-available-browser-versions-or-baseline-year-browser-versions).
- An `Array`, `Object` or `CSV` as a string describing the Baseline feature set support of all browser versions included in the module's data set via the [`getAllVersions()` function](#get-data-for-all-browser-versions).
You can use `baseline-browser-mapping` to help you determine minimum browser version support for your chosen Baseline feature set; or to analyse the level of support for different Baseline feature sets in your site's traffic by joining the data with your analytics data.
## Install for local development
To install the package, run:
`npm install --save-dev baseline-browser-mapping`
`baseline-browser-mapping` depends on `web-features` and `@mdn/browser-compat-data` for core browser version selection, but the data is pre-packaged and minified. This package checks for updates to those modules and the supported [downstream browsers](#downstream-browsers) on a daily basis and is updated frequently. Consider adding a script to your `package.json` to update `baseline-browser-mapping` and using it as part of your build process to ensure your data is as up to date as possible:
```javascript
"scripts": [
"refresh-baseline-browser-mapping": "npm i --save-dev baseline-browser-mapping@latest"
]
```
The minimum supported NodeJS version for `baseline-browser-mapping` is v8 in alignment with `browserslist`. For NodeJS versions earlier than v13.2, the [`require('baseline-browser-mapping')`](https://nodejs.org/api/modules.html#requireid) syntax should be used to import the module.
## Keeping `baseline-browser-mapping` up to date
If you are only using this module to generate minimum browser versions for Baseline Widely available or Baseline year feature sets, you don't need to update this module frequently, as the backward looking data is reasonably stable.
However, if you are targeting Newly available, using the [`getAllVersions()`](#get-data-for-all-browser-versions) function or heavily relying on the data for downstream browsers, you should update this module more frequently. If you target a feature cut off date within the last two months and your installed version of `baseline-browser-mapping` has data that is more than 2 months old, you will receive a console warning advising you to update to the latest version when you call `getCompatibleVersions()` or `getAllVersions()`.
If you want to suppress these warnings you can use the `suppressWarnings: true` option in the configuration object passed to `getCompatibleVersions()` or `getAllVersions()`. Alternatively, you can use the `BASELINE_BROWSER_MAPPING_IGNORE_OLD_DATA=true` environment variable when running your build process. This module also respects the `BROWSERSLIST_IGNORE_OLD_DATA=true` environment variable. Environment variables can also be provided in a `.env` file from Node 20 onwards; however, this module does not load .env files automatically to avoid conflicts with other libraries with different requirements. You will need to use `process.loadEnvFile()` or a library like `dotenv` to load .env files before `baseline-browser-mapping` is called.
If you want to ensure [reproducible builds](https://www.wikiwand.com/en/articles/Reproducible_builds), we strongly recommend using the `widelyAvailableOnDate` option to fix the Widely available date on a per build basis to ensure dependent tools provide the same output and you do not produce data staleness warnings. If you are using [`browserslist`](https://github.com/browserslist/browserslist) to target Baseline Widely available, consider automatically updating your `browserslist` configuration in `package.json` or `.browserslistrc` to `baseline widely available on {YYYY-MM-DD}` as part of your build process to ensure the same or sufficiently similar list of minimum browsers is reproduced for historical builds.
## Importing `baseline-browser-mapping`
This module exposes two functions: `getCompatibleVersions()` and `getAllVersions()`, both which can be imported directly from `baseline-browser-mapping`:
```javascript
import {
getCompatibleVersions,
getAllVersions,
} from "baseline-browser-mapping";
```
If you want to load the script and data directly in a web page without hosting it yourself, consider using a CDN:
```html
<script type="module">
import {
getCompatibleVersions,
getAllVersions,
} from "https://cdn.jsdelivr.net/npm/baseline-browser-mapping";
</script>
```
## Get Baseline Widely available browser versions or Baseline year browser versions
To get the current list of minimum browser versions compatible with Baseline Widely available features from the core browser set, call the `getCompatibleVersions()` function:
```javascript
getCompatibleVersions();
```
Executed on 7th March 2025, the above code returns the following browser versions:
```javascript
[
{ browser: "chrome", version: "105", release_date: "2022-09-02" },
{
browser: "chrome_android",
version: "105",
release_date: "2022-09-02",
},
{ browser: "edge", version: "105", release_date: "2022-09-02" },
{ browser: "firefox", version: "104", release_date: "2022-08-23" },
{
browser: "firefox_android",
version: "104",
release_date: "2022-08-23",
},
{ browser: "safari", version: "15.6", release_date: "2022-09-02" },
{
browser: "safari_ios",
version: "15.6",
release_date: "2022-09-02",
},
];
```
> [!NOTE]
> The minimum versions of each browser are not necessarily the final release before the Widely available cutoff date of `TODAY - 30 MONTHS`. Some earlier versions will have supported the full Widely available feature set.
### `getCompatibleVersions()` configuration options
`getCompatibleVersions()` accepts an `Object` as an argument with configuration options. The defaults are as follows:
```javascript
{
targetYear: undefined,
widelyAvailableOnDate: undefined,
includeDownstreamBrowsers: false,
listAllCompatibleVersions: false,
suppressWarnings: false
}
```
#### `targetYear`
The `targetYear` option returns the minimum browser versions compatible with all **Baseline Newly available** features at the end of the specified calendar year. For example, calling:
```javascript
getCompatibleVersions({
targetYear: 2020,
});
```
Returns the following versions:
```javascript
[
{ browser: "chrome", version: "87", release_date: "2020-11-19" },
{
browser: "chrome_android",
version: "87",
release_date: "2020-11-19",
},
{ browser: "edge", version: "87", release_date: "2020-11-19" },
{ browser: "firefox", version: "83", release_date: "2020-11-17" },
{
browser: "firefox_android",
version: "83",
release_date: "2020-11-17",
},
{ browser: "safari", version: "14", release_date: "2020-09-16" },
{ browser: "safari_ios", version: "14", release_date: "2020-09-16" },
];
```
> [!NOTE]
> The minimum version of each browser is not necessarily the final version released in that calendar year. In the above example, Firefox 84 was the final version released in 2020; however Firefox 83 supported all of the features that were interoperable at the end of 2020.
> [!WARNING]
> You cannot use `targetYear` and `widelyAavailableDate` together. Please only use one of these options at a time.
#### `widelyAvailableOnDate`
The `widelyAvailableOnDate` option returns the minimum versions compatible with Baseline Widely available on a specified date in the format `YYYY-MM-DD`:
```javascript
getCompatibleVersions({
widelyAvailableOnDate: `2023-04-05`,
});
```
> [!TIP]
> This option is useful if you provide a versioned library that targets Baseline Widely available on each version's release date and you need to provide a statement on minimum supported browser versions in your documentation.
#### `includeDownstreamBrowsers`
Setting `includeDownstreamBrowsers` to `true` will include browsers outside of the Baseline core browser set where it is possible to map those browsers to an upstream Chromium or Gecko version:
```javascript
getCompatibleVersions({
includeDownstreamBrowsers: true,
});
```
For more information on downstream browsers, see [the section on downstream browsers](#downstream-browsers) below.
#### `includeKaiOS`
KaiOS is an operating system and app framework based on the Gecko engine from Firefox. KaiOS is based on the Gecko engine and feature support can be derived from the upstream Gecko version that each KaiOS version implements. However KaiOS requires other considerations beyond feature compatibility to ensure a good user experience as it runs on device types that do not have either mouse and keyboard or touch screen input in the way that all the other browsers supported by this module do.
```javascript
getCompatibleVersions({
includeDownstreamBrowsers: true,
includeKaiOS: true,
});
```
> [!NOTE]
> Including KaiOS requires you to include all downstream browsers using the `includeDownstreamBrowsers` option.
#### `listAllCompatibleVersions`
Setting `listAllCompatibleVersions` to true will include the minimum versions of each compatible browser, and all the subsequent versions:
```javascript
getCompatibleVersions({
listAllCompatibleVersions: true,
});
```
#### `suppressWarnings`
Setting `suppressWarnings` to `true` will suppress the console warning about old data:
```javascript
getCompatibleVersions({
suppressWarnings: true,
});
```
## Get data for all browser versions
You may want to obtain data on all the browser versions available in this module for use in an analytics solution or dashboard. To get details of each browser version's level of Baseline support, call the `getAllVersions()` function:
```javascript
import { getAllVersions } from "baseline-browser-mapping";
getAllVersions();
```
By default, this function returns an `Array` of `Objects` and excludes downstream browsers:
```javascript
[
...
{
browser: "firefox_android", // Browser name
version: "125", // Browser version
release_date: "2024-04-16", // Release date
year: 2023, // Baseline year feature set the version supports
wa_compatible: true // Whether the browser version supports Widely available
},
...
]
```
For browser versions in `@mdn/browser-compat-data` that were released before Baseline can be defined, i.e. Baseline 2015, the `year` property is always the string: `"pre_baseline"`.
### Understanding which browsers support Newly available features
You may want to understand which recent browser versions support all Newly available features. You can replace the `wa_compatible` property with a `supports` property using the `useSupport` option:
```javascript
getAllVersions({
useSupports: true,
});
```
The `supports` property is optional and has two possible values:
- `widely` for browser versions that support all Widely available features.
- `newly` for browser versions that support all Newly available features.
Browser versions that do not support Widely or Newly available will not include the `support` property in the `array` or `object` outputs, and in the CSV output, the `support` column will contain an empty string. Browser versions that support all Newly available features also support all Widely available features.
### `getAllVersions()` Configuration options
`getAllVersions()` accepts an `Object` as an argument with configuration options. The defaults are as follows:
```javascript
{
includeDownstreamBrowsers: false,
outputFormat: "array",
suppressWarnings: false
}
```
#### `includeDownstreamBrowsers` (in `getAllVersions()` output)
As with `getCompatibleVersions()`, you can set `includeDownstreamBrowsers` to `true` to include the Chromium and Gecko downstream browsers [listed below](#list-of-downstream-browsers).
```javascript
getAllVersions({
includeDownstreamBrowsers: true,
});
```
Downstream browsers include the same properties as core browsers, as well as the `engine`they use and `engine_version`, for example:
```javascript
[
...
{
browser: "samsunginternet_android",
version: "27.0",
release_date: "2024-11-06",
engine: "Blink",
engine_version: "125",
year: 2023,
supports: "widely"
},
...
]
```
#### `includeKaiOS` (in `getAllVersions()` output)
As with `getCompatibleVersions()` you can include KaiOS in your output. The same requirement to have `includeDownstreamBrowsers: true` applies.
```javascript
getAllVersions({
includeDownstreamBrowsers: true,
includeKaiOS: true,
});
```
#### `suppressWarnings` (in `getAllVersions()` output)
As with `getCompatibleVersions()`, you can set `suppressWarnings` to `true` to suppress the console warning about old data:
```javascript
getAllVersions({
suppressWarnings: true,
});
```
#### `outputFormat`
By default, this function returns an `Array` of `Objects` which can be manipulated in Javascript or output to JSON.
To return an `Object` that nests keys , set `outputFormat` to `object`:
```javascript
getAllVersions({
outputFormat: "object",
});
```
In thise case, `getAllVersions()` returns a nested object with the browser [IDs listed below](#list-of-downstream-browsers) as keys, and versions as keys within them:
```javascript
{
"chrome": {
"53": {
"year": 2016,
"release_date": "2016-09-07"
},
...
}
```
Downstream browsers will include extra fields for `engine` and `engine_versions`
```javascript
{
...
"webview_android": {
"53": {
"year": 2016,
"release_date": "2016-09-07",
"engine": "Blink",
"engine_version": "53"
},
...
}
```
To return a `String` in CSV format, set `outputFormat` to `csv`:
```javascript
getAllVersions({
outputFormat: "csv",
});
```
`getAllVersions` returns a `String` with a header row and comma-separated values for each browser version that you can write to a file or pass to another service. Core browsers will have "NULL" as the value for their `engine` and `engine_version`:
```csv
"browser","version","year","supports","release_date","engine","engine_version"
...
"chrome","24","pre_baseline","","2013-01-10","NULL","NULL"
...
"chrome","53","2016","","2016-09-07","NULL","NULL"
...
"firefox","135","2024","widely","2025-02-04","NULL","NULL"
"firefox","136","2024","newly","2025-03-04","NULL","NULL"
...
"ya_android","20.12","2020","year_only","2020-12-20","Blink","87"
...
```
> [!NOTE]
> The above example uses `"includeDownstreamBrowsers": true`
### Static resources
The outputs of `getAllVersions()` are available as JSON or CSV files generated on a daily basis and hosted on GitHub pages:
- Core browsers only
- [Array](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions_array.json)
- [Object](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions_object.json)
- [CSV](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions.csv)
- Core browsers only, with `supports` property
- [Array](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions_array_with_supports.json)
- [Object](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions_object_with_supports.json)
- [CSV](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions_with_supports.csv)
- Including downstream browsers
- [Array](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions_array.json)
- [Object](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions_object.json)
- [CSV](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions.csv)
- Including downstream browsers with `supports` property
- [Array](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions_array_with_supports.json)
- [Object](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions_object_with_supports.json)
- [CSV](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions_with_supports.csv)
These files are updated on a daily basis.
## CLI
`baseline-browser-mapping` includes a command line interface that exposes the same data and options as the `getCompatibleVersions()` function. To learn more about using the CLI, run:
```sh
npx baseline-browser-mapping --help
```
## Downstream browsers
### Limitations
The browser versions in this module come from two different sources:
- MDN's `browser-compat-data` module.
- Parsed user agent strings provided by [useragents.io](https://useragents.io/)
MDN `browser-compat-data` is an authoritative source of information for the browsers it contains. The release dates for the Baseline core browser set and the mapping of downstream browsers to Chromium versions should be considered accurate.
Browser mappings from useragents.io are provided on a best effort basis. They assume that browser vendors are accurately stating the Chromium version they have implemented. The initial set of version mappings was derived from a bulk export in November 2024. This version was iterated over with a Regex match looking for a major Chrome version and a corresponding version of the browser in question, e.g.:
`Mozilla/5.0 (Linux; U; Android 10; en-US; STK-L21 Build/HUAWEISTK-L21) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/100.0.4896.58 UCBrowser/13.8.2.1324 Mobile Safari/537.36`
Shows UC Browser Mobile 13.8 implementing Chromium 100, and:
`Mozilla/5.0 (Linux; arm_64; Android 11; Redmi Note 8 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.6613.123 YaBrowser/24.10.2.123.00 SA/3 Mobile Safari/537.36`
Shows Yandex Browser Mobile 24.10 implementing Chromium 128. The Chromium version from this string is mapped to the corresponding Chrome version from MDN `browser-compat-data`.
> [!NOTE]
> Where possible, approximate release dates have been included based on useragents.io "first seen" data. useragents.io does not have "first seen" dates prior to June 2020. However, these browsers' Baseline compatibility is determined by their Chromium or Gecko version, so their release dates are more informative than critical.
This data is updated on a daily basis using a [script](https://github.com/web-platform-dx/web-features/tree/main/scripts/refresh-downstream.ts) triggered by a GitHub [action](https://github.com/web-platform-dx/web-features/tree/main/.github/workflows/refresh_downstream.yml). Useragents.io provides a private API for this module which exposes the last 7 days of newly seen user agents for the currently tracked browsers. If a new major version of one of the tracked browsers is encountered with a Chromium version that meets or exceeds the previous latest version of that browser, it is added to the [src/data/downstream-browsers.json](src/data/downstream-browsers.json) file with the date it was first seen by useragents.io as its release date.
KaiOS is an exception - its upstream version mappings are handled separately from the other browsers because they happen very infrequently.
### List of downstream browsers
| Browser | ID | Core | Source |
| --------------------- | ------------------------- | ------- | ------------------------- |
| Chrome | `chrome` | `true` | MDN `browser-compat-data` |
| Chrome for Android | `chrome_android` | `true` | MDN `browser-compat-data` |
| Edge | `edge` | `true` | MDN `browser-compat-data` |
| Firefox | `firefox` | `true` | MDN `browser-compat-data` |
| Firefox for Android | `firefox_android` | `true` | MDN `browser-compat-data` |
| Safari | `safari` | `true` | MDN `browser-compat-data` |
| Safari on iOS | `safari_ios` | `true` | MDN `browser-compat-data` |
| Opera | `opera` | `false` | MDN `browser-compat-data` |
| Opera Android | `opera_android` | `false` | MDN `browser-compat-data` |
| Samsung Internet | `samsunginternet_android` | `false` | MDN `browser-compat-data` |
| WebView Android | `webview_android` | `false` | MDN `browser-compat-data` |
| QQ Browser Mobile | `qq_android` | `false` | useragents.io |
| UC Browser Mobile | `uc_android` | `false` | useragents.io |
| Yandex Browser Mobile | `ya_android` | `false` | useragents.io |
| KaiOS | `kai_os` | `false` | Manual |
| Facebook for Android | `facebook_android` | `false` | useragents.io |
| Instagram for Android | `instagram_android` | `false` | useragents.io |
> [!NOTE]
> All the non-core browsers currently included implement Chromium or Gecko. Their inclusion in any of the above methods is based on the Baseline feature set supported by the Chromium or Gecko version they implement, not their release date.
-64
View File
@@ -1,64 +0,0 @@
{
"name": "baseline-browser-mapping",
"main": "./dist/index.cjs",
"version": "2.9.19",
"description": "A library for obtaining browser versions with their maximum supported Baseline feature set and Widely Available status.",
"exports": {
".": {
"require": "./dist/index.cjs",
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
},
"./legacy": {
"require": "./dist/index.cjs",
"types": "./dist/index.d.ts"
}
},
"jsdelivr": "./dist/index.js",
"files": [
"dist/*",
"!dist/scripts/*",
"LICENSE.txt",
"README.md"
],
"types": "./dist/index.d.ts",
"type": "module",
"bin": {
"baseline-browser-mapping": "dist/cli.js"
},
"scripts": {
"fix-cli-permissions": "output=$(npx baseline-browser-mapping 2>&1); path=$(printf '%s\n' \"$output\" | sed -n 's/^.*: \\(.*\\): Permission denied$/\\1/p; t; s/^\\(.*\\): Permission denied$/\\1/p'); if [ -n \"$path\" ]; then echo \"Permission denied for: $path\"; echo \"Removing $path ...\"; rm -rf \"$path\"; else echo \"$output\"; fi",
"test:format": "npx prettier --check .",
"test:lint": "npx eslint .",
"test:jasmine": "npx jasmine",
"test:jasmine-browser": "npx jasmine-browser-runner runSpecs --config ./spec/support/jasmine-browser.js",
"test": "npm run build && npm run fix-cli-permissions && npm run test:format && npm run test:lint && npm run test:jasmine && npm run test:jasmine-browser",
"build": "rm -rf dist; npx prettier . --write; rollup -c; rm -rf ./dist/scripts/expose-data.d.ts ./dist/cli.d.ts",
"refresh-downstream": "npx tsx scripts/refresh-downstream.ts",
"refresh-static": "npx tsx scripts/refresh-static.ts",
"update-data-file": "npx tsx scripts/update-data-file.ts; npx prettier ./src/data/data.js --write",
"update-data-dependencies": "npm i @mdn/browser-compat-data@latest web-features@latest -D",
"check-data-changes": "git diff --name-only | grep -q '^src/data/data.js$' && echo 'changes-available=TRUE' || echo 'changes-available=FALSE'"
},
"license": "Apache-2.0",
"devDependencies": {
"@mdn/browser-compat-data": "^7.2.5",
"@rollup/plugin-terser": "^0.4.4",
"@rollup/plugin-typescript": "^12.1.3",
"@types/node": "^22.15.17",
"eslint-plugin-new-with-error": "^5.0.0",
"jasmine": "^5.8.0",
"jasmine-browser-runner": "^3.0.0",
"jasmine-spec-reporter": "^7.0.0",
"prettier": "^3.5.3",
"rollup": "^4.44.0",
"tslib": "^2.8.1",
"typescript": "^5.7.2",
"typescript-eslint": "^8.35.0",
"web-features": "^3.14.0"
},
"repository": {
"type": "git",
"url": "git+https://github.com/web-platform-dx/baseline-browser-mapping.git"
}
}
+26 -42
View File
@@ -179,6 +179,25 @@ volumes:
With any authentication method enabled, Shelfmark supports multi-user management with admin/user roles. Users can have per-user settings for download destinations, email recipients, and notification preferences. Non-admin users only see their own downloads and can submit book requests for admin review. Admins can configure request policies per source to control whether users can download directly, must submit a request, or are blocked entirely.
## Project Scope
Shelfmark is a manual search and download tool, the entry point to your book library, not a library manager. It finds books, downloads them, and sends them to a configured destination. That's the full scope.
Shelfmark intentionally does not:
- **Track or manage your library** - it doesn't know or care what you already own
- **Integrate with library software** - what happens after delivery is up to your library tool
- **Monitor authors, series, or new releases** - there is no background automation
- **Queue future downloads** - if a book isn't available now, Shelfmark won't watch for it
These are non-goals, not missing features.
## Contributing
Shelfmark's core feature set is complete. Development focuses on stability, bug fixes, quality-of-life improvements, and refining the search experience. Contributions in these areas are welcome, please file issues or submit pull requests on GitHub.
Feature requests that fall outside the project scope (library integration, automation, collection management) will be closed. If you're unsure whether something fits, open a discussion first.
## Health Monitoring
The application exposes a health endpoint at `/api/health` (no authentication required). Add a health check to your compose:
@@ -217,55 +236,20 @@ make restart # Restart container
The frontend dev server proxies to the backend on port 8084.
### Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Web Interface │
│ (React + TypeScript + Vite) │
├─────────────────────────────────────────────────────────────┤
│ Flask Backend │
│ (REST API + WebSocket) │
├───────────────────┬─────────────────────┬───────────────────┤
│ Metadata Providers│ Download Queue │ Cloudflare │
│ │ & Orchestrator │ Bypass │
├───────────────────┼─────────────────────┼───────────────────┤
│ • Hardcover │ • Task scheduling │ • Internal │
│ • Open Library │ • Progress tracking │ • External │
│ │ • Retry logic │ (FlareSolverr) │
├───────────────────┴─────────────────────┴───────────────────┤
│ Release Sources │
├─────────────────────────────────────────────────────────────┤
│ • Direct Download (Web Sources → Mirrors → Fallbacks) │
├─────────────────────────────────────────────────────────────┤
│ Network Layer │
├─────────────────────────────────────────────────────────────┤
│ • Auto DNS rotation • Mirror failover • Resume support │
└─────────────────────────────────────────────────────────────┘
```
The backend uses a plugin architecture. Metadata providers and release sources register via decorators and are automatically discovered.
## Contributing
Shelfmark's core feature set is now largely complete. Development going forward will focus on stability, bug fixes, and maintenance rather than major new features. Contributions in these areas are welcome - please file issues or submit pull requests on GitHub.
## License
MIT License - see [LICENSE](LICENSE) for details.
## ⚠️ Disclaimers
## ⚠️ Disclaimer
### Copyright Notice
Shelfmark is a search interface that displays results from external metadata providers and sources. It does not host, store, or distribute any content. The developers are not responsible for how the tool is used or what is accessed through it.
This tool can access various sources including those that might contain copyrighted material. Users are responsible for:
- Ensuring they have the right to download requested materials
- Respecting copyright laws and intellectual property rights
- Using the tool in compliance with their local regulations
Users are solely responsible for:
- Ensuring they have the legal right to download any material they access
- Complying with copyright laws and intellectual property rights in their jurisdiction
- Understanding and accepting the terms of any sources they configure
### Library Integration
Downloads are written atomically (via intermediate `.crdownload` files) to prevent partial files from being ingested. However, if your library tool (CWA, Booklore, Calibre) is actively scanning or importing, there's a small chance of race conditions. If you experience database errors or import failures, try pausing your library's auto-import during bulk downloads.
Use of this tool is entirely at your own risk.
## Support
+2 -1
View File
@@ -1,8 +1,9 @@
flask>=3.1.0,<3.1.3 # Temporary: Flask 3.1.3 breaks flask-socketio (github.com/miguelgrinberg/Flask-SocketIO/pull/2153)
flask
flask-cors
flask-socketio
python-socketio
requests[socks]
defusedxml
beautifulsoup4
tqdm
dnspython
+3 -2
View File
@@ -222,6 +222,7 @@ def generate_env_docs() -> str:
"""Generate markdown documentation for all environment variables."""
# Import settings modules to ensure all settings are registered
import shelfmark.config.settings # noqa: F401
import shelfmark.config.security # noqa: F401
import shelfmark.release_sources.irc.settings # noqa: F401
import shelfmark.release_sources.prowlarr.settings # noqa: F401
import shelfmark.metadata_providers.hardcover # noqa: F401
@@ -310,7 +311,7 @@ def generate_env_docs() -> str:
def _generate_tab_docs(tab, group_prefix: Optional[str] = None) -> List[str]:
"""Generate documentation for a single settings tab."""
from shelfmark.core.settings_registry import ActionButton, HeadingField
from shelfmark.core.settings_registry import ActionButton, CustomComponentField, HeadingField
lines = []
@@ -327,7 +328,7 @@ def _generate_tab_docs(tab, group_prefix: Optional[str] = None) -> List[str]:
env_fields = []
for field in tab.fields:
# Skip non-value fields
if isinstance(field, (ActionButton, HeadingField)):
if isinstance(field, (ActionButton, CustomComponentField, HeadingField)):
continue
# Skip fields that don't support ENV vars
+3 -1
View File
@@ -11,6 +11,7 @@ from shelfmark.bypass import BypassCancelledException
from shelfmark.core.config import config
from shelfmark.core.logger import setup_logger
from shelfmark.core.utils import normalize_http_url
from shelfmark.download.network import get_ssl_verify
if TYPE_CHECKING:
from shelfmark.download import network
@@ -46,7 +47,8 @@ def _fetch_via_bypasser(target_url: str) -> Optional[str]:
f"{bypasser_url}{bypasser_path}",
headers={"Content-Type": "application/json"},
json={"cmd": "request.get", "url": target_url, "maxTimeout": bypasser_timeout},
timeout=(CONNECT_TIMEOUT, read_timeout)
timeout=(CONNECT_TIMEOUT, read_timeout),
verify=get_ssl_verify(bypasser_url),
)
response.raise_for_status()
result = response.json()
+2 -2
View File
@@ -23,7 +23,7 @@ from shelfmark.config.settings import RECORDING_DIR
from shelfmark.core.config import config as app_config
from shelfmark.core.logger import setup_logger
from shelfmark.download import network
from shelfmark.download.network import get_proxies
from shelfmark.download.network import get_proxies, get_ssl_verify
logger = setup_logger(__name__)
@@ -931,7 +931,7 @@ def _try_with_cached_cookies(url: str, hostname: str) -> Optional[str]:
headers['User-Agent'] = stored_ua
logger.debug(f"Trying request with cached cookies: {url}")
response = requests.get(url, cookies=cookies, headers=headers, proxies=get_proxies(url), timeout=(5, 10))
response = requests.get(url, cookies=cookies, headers=headers, proxies=get_proxies(url), timeout=(5, 10), verify=get_ssl_verify(url))
if response.status_code == 200:
logger.debug("Cached cookies worked, skipped Chrome bypass")
return response.text
+2
View File
@@ -115,6 +115,8 @@ FLASK_PORT = int(os.getenv("FLASK_PORT", "8084"))
SESSION_COOKIE_SECURE_ENV = os.getenv("SESSION_COOKIE_SECURE", "false")
SESSION_COOKIE_NAME = "shelfmark_session"
CWA_DB_PATH = _resolve_cwa_db_path()
HIDE_LOCAL_AUTH = string_to_bool(os.getenv("HIDE_LOCAL_AUTH", "false"))
OIDC_AUTO_REDIRECT = string_to_bool(os.getenv("OIDC_AUTO_REDIRECT", "false"))
# =============================================================================
+12
View File
@@ -79,6 +79,18 @@ def migrate_security_settings(
logger.info("Removed deprecated USE_CWA_AUTH setting (AUTH_METHOD already exists)")
migrated_security = True
# Backfill AUTH_METHOD for configs that have builtin credentials but
# were never migrated from USE_CWA_AUTH (e.g. dev builds that predated
# the AUTH_METHOD field).
if "AUTH_METHOD" not in config:
if config.get("BUILTIN_USERNAME") and config.get("BUILTIN_PASSWORD_HASH"):
config["AUTH_METHOD"] = "builtin"
migrated_security = True
logger.info(
"Backfilled AUTH_METHOD='builtin' from legacy "
"BUILTIN_USERNAME/BUILTIN_PASSWORD_HASH credentials"
)
if "RESTRICT_SETTINGS_TO_ADMIN" not in users_config:
legacy_restrict = _pick_legacy_settings_restriction(config)
if legacy_restrict is not None:
+60 -21
View File
@@ -18,6 +18,7 @@ from shelfmark.core.settings_registry import (
CheckboxField,
ActionButton,
TagListField,
CustomComponentField,
)
from shelfmark.core.user_db import sync_builtin_admin_user
@@ -28,12 +29,8 @@ def _auth_condition(auth_method: str) -> dict[str, str]:
return {"field": "AUTH_METHOD", "value": auth_method}
def _ui_field(factory: Callable[..., Any], **kwargs: Any) -> Any:
return factory(env_supported=False, **kwargs)
def _auth_ui_field(factory: Callable[..., Any], auth_method: str, **kwargs: Any) -> Any:
return _ui_field(factory, show_when=_auth_condition(auth_method), **kwargs)
def _auth_field(factory: Callable[..., Any], auth_method: str, **kwargs: Any) -> Any:
return factory(show_when=_auth_condition(auth_method), **kwargs)
def _migrate_security_settings() -> None:
@@ -59,9 +56,10 @@ def _on_save_security(values: Dict[str, Any]) -> Dict[str, Any]:
return on_save_security(values)
def _test_oidc_connection() -> Dict[str, Any]:
def _test_oidc_connection(current_values: Dict[str, Any] = None) -> Dict[str, Any]:
return test_oidc_connection(
load_security_config=lambda: load_config_file("security"),
current_values=current_values or {},
logger=logger,
)
@@ -78,31 +76,52 @@ def security_settings():
{"label": "Local", "value": "builtin"},
{"label": "Proxy Authentication", "value": "proxy"},
{"label": "OIDC (OpenID Connect)", "value": "oidc"},
{"label": "Calibre-Web Database", "value": "cwa"},
]
if cwa_db_available:
auth_method_options.append({"label": "Calibre-Web Database", "value": "cwa"})
auth_method_description = "Select the authentication method for accessing Shelfmark."
if not cwa_db_available:
auth_method_description += " Calibre-Web database option requires mounting your Calibre-Web app.db to /auth/app.db."
fields = [
SelectField(
key="AUTH_METHOD",
label="Authentication Method",
description=auth_method_description,
description="Select the authentication method for accessing Shelfmark.",
options=auth_method_options,
default="none",
env_supported=False,
),
CustomComponentField(
key="builtin_admin_requirement",
component="oidc_admin_hint",
label=(
"Local authentication is inactive until a local admin account with a "
"password is created."
),
show_when=_auth_condition("builtin"),
),
CustomComponentField(
key="oidc_admin_requirement",
component="oidc_admin_hint",
label="A local admin account is required before OIDC can be enabled.",
show_when=_auth_condition("oidc"),
),
*([] if cwa_db_available else [
CustomComponentField(
key="cwa_db_missing",
component="oidc_admin_hint",
label=(
"Calibre-Web database not detected. Mount your app.db to "
"/auth/app.db to enable this method. Authentication will fall "
"back to none until the database is available."
),
show_when=_auth_condition("cwa"),
),
]),
ActionButton(
key="open_users_tab",
label="Go to Users",
description="Configure local users and admin access in the Users tab.",
style="primary",
show_when=_auth_condition("builtin"),
show_when={"field": "AUTH_METHOD", "value": ["builtin", "oidc"]},
),
_auth_ui_field(
_auth_field(
TextField,
"proxy",
key="PROXY_AUTH_USER_HEADER",
@@ -111,7 +130,7 @@ def security_settings():
placeholder="e.g. X-Auth-User",
default="X-Auth-User",
),
_auth_ui_field(
_auth_field(
TextField,
"proxy",
key="PROXY_AUTH_LOGOUT_URL",
@@ -120,7 +139,7 @@ def security_settings():
placeholder="https://myauth.example.com/logout",
default="",
),
_auth_ui_field(
_auth_field(
TextField,
"proxy",
key="PROXY_AUTH_ADMIN_GROUP_HEADER",
@@ -129,7 +148,7 @@ def security_settings():
placeholder="e.g. X-Auth-Groups",
default="X-Auth-Groups",
),
_auth_ui_field(
_auth_field(
TextField,
"proxy",
key="PROXY_AUTH_ADMIN_GROUP_NAME",
@@ -140,6 +159,16 @@ def security_settings():
),
]
fields.append(
CustomComponentField(
key="oidc_callback_url",
component="settings_label",
label="Callback URL",
description="{origin}/api/auth/oidc/callback",
show_when=_auth_condition("oidc"),
)
)
oidc_specs = [
(
TextField,
@@ -228,7 +257,7 @@ def security_settings():
},
),
]
fields.extend(_auth_ui_field(factory, "oidc", **spec) for factory, spec in oidc_specs)
fields.extend(_auth_field(factory, "oidc", **spec) for factory, spec in oidc_specs)
fields.append(
ActionButton(
key="test_oidc",
@@ -239,6 +268,16 @@ def security_settings():
show_when=_auth_condition("oidc"),
)
)
fields.append(
CustomComponentField(
key="oidc_env_info",
component="oidc_env_info",
label="Environment-Only Options",
description="These options can only be set via environment variables because changing them through the UI could lock you out of the application.",
wrap_in_field_wrapper=True,
show_when=_auth_condition("oidc"),
)
)
return fields
+6 -3
View File
@@ -5,9 +5,10 @@ from typing import Any, Callable
from shelfmark.core.utils import normalize_http_url
from shelfmark.core.user_db import UserDB
from shelfmark.download.network import get_ssl_verify
_OIDC_LOCKOUT_MESSAGE = "Create a local admin account first (Users tab) before enabling OIDC. This ensures you can still log in with a password if SSO is unavailable."
_OIDC_LOCKOUT_MESSAGE = "A local admin account with a password is required before enabling OIDC. Use the 'Go to Users' button above to create one. This ensures you can still sign in if your identity provider is unavailable."
def _has_local_password_admin() -> bool:
@@ -47,17 +48,19 @@ def on_save_security(
def test_oidc_connection(
*,
load_security_config: Callable[[], dict[str, Any]],
current_values: dict[str, Any] | None = None,
logger: Any,
) -> dict[str, Any]:
"""Fetch and validate the configured OIDC discovery document."""
import requests
try:
discovery_url = load_security_config().get("OIDC_DISCOVERY_URL", "")
# Prefer the current (unsaved) form value over the saved config
discovery_url = (current_values or {}).get("OIDC_DISCOVERY_URL") or load_security_config().get("OIDC_DISCOVERY_URL", "")
if not discovery_url:
return {"success": False, "message": "Discovery URL is not configured."}
response = requests.get(discovery_url, timeout=10)
response = requests.get(discovery_url, timeout=10, verify=get_ssl_verify(discovery_url))
response.raise_for_status()
document = response.json()
+21 -6
View File
@@ -419,6 +419,7 @@ def search_mode_settings():
},
],
default="direct",
user_overridable=True,
),
SelectField(
key="AA_DEFAULT_SORT",
@@ -441,6 +442,7 @@ def search_mode_settings():
options=_get_metadata_provider_options, # Callable - evaluated lazily to avoid circular imports
default="openlibrary",
show_when={"field": "SEARCH_MODE", "value": "universal"},
user_overridable=True,
),
SelectField(
key="METADATA_PROVIDER_AUDIOBOOK",
@@ -449,6 +451,7 @@ def search_mode_settings():
options=_get_metadata_provider_options_with_none, # Callable - includes "Use main provider" option
default="",
show_when={"field": "SEARCH_MODE", "value": "universal"},
user_overridable=True,
),
SelectField(
key="DEFAULT_RELEASE_SOURCE",
@@ -457,6 +460,7 @@ def search_mode_settings():
options=_get_release_source_options, # Callable - evaluated lazily to avoid circular imports
default="direct_download",
show_when={"field": "SEARCH_MODE", "value": "universal"},
user_overridable=True,
),
]
@@ -473,6 +477,17 @@ def network_settings():
tor_overrides_network = tor_enabled # Only override when Tor is actually active
return [
SelectField(
key="CERTIFICATE_VALIDATION",
label="Certificate Validation",
description="Controls SSL/TLS certificate verification for outbound connections. Disable for self-signed certificates on internal services (e.g. OIDC providers, Prowlarr).",
options=[
{"value": "enabled", "label": "Enabled (Recommended)"},
{"value": "disabled_local", "label": "Disabled for Local Addresses"},
{"value": "disabled", "label": "Disabled"},
],
default="enabled",
),
SelectField(
key="CUSTOM_DNS",
label="DNS Provider",
@@ -791,7 +806,7 @@ def download_settings():
{
"value": "rename",
"label": "Rename Only",
"description": "Rename files using a template"
"description": "Rename single-file downloads; multi-file keeps original names."
},
{
"value": "organize",
@@ -809,7 +824,7 @@ def download_settings():
TextField(
key="TEMPLATE_RENAME",
label="Naming Template",
description="Variables: {Author}, {Title}, {Year}, {User}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\\'); use Organize for folders.",
description="Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension). Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\\'); use Organize for folders. Applies to single-file downloads.",
default="{Author} - {Title} ({Year})",
placeholder="{Author} - {Title} ({Year})",
show_when=[
@@ -821,7 +836,7 @@ def download_settings():
TextField(
key="TEMPLATE_ORGANIZE",
label="Path Template",
description="Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.",
description="Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension). Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.",
default="{Author}/{Title} ({Year})",
placeholder="{Author}/{Series/}{Title} ({Year})",
show_when=[
@@ -1046,7 +1061,7 @@ def download_settings():
description="Choose how downloaded audiobook files are named and organized.",
options=[
{"value": "none", "label": "None", "description": "Keep original filename from source"},
{"value": "rename", "label": "Rename Only", "description": "Rename files using a template"},
{"value": "rename", "label": "Rename Only", "description": "Rename single-file downloads; multi-file keeps original names."},
{"value": "organize", "label": "Rename and Organize", "description": "Create folders and rename files using a template. Recommended for Audiobookshelf. Do not use with ingest folders."},
],
default="rename",
@@ -1056,7 +1071,7 @@ def download_settings():
TextField(
key="TEMPLATE_AUDIOBOOK_RENAME",
label="Naming Template",
description="Variables: {Author}, {Title}, {Year}, {User}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\\'); use Organize for folders.",
description="Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension), {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\\'); use Organize for folders. Applies to single-file downloads.",
default="{Author} - {Title}",
placeholder="{Author} - {Title}{ - Part }{PartNumber}",
show_when={"field": "FILE_ORGANIZATION_AUDIOBOOK", "value": "rename"},
@@ -1066,7 +1081,7 @@ def download_settings():
TextField(
key="TEMPLATE_AUDIOBOOK_ORGANIZE",
label="Path Template",
description="Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.",
description="Use / to create folders. Variables: {Author}, {Title}, {Year}, {User}, {OriginalName} (source filename without extension), {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.",
default="{Author}/{Title}",
placeholder="{Author}/{Series/}{Title}{ - Part }{PartNumber}",
show_when={"field": "FILE_ORGANIZATION_AUDIOBOOK", "value": "organize"},
+70
View File
@@ -5,6 +5,8 @@ The actual user management is handled by a custom frontend component
that talks to /api/admin/users endpoints.
"""
from typing import Any
from shelfmark.core.settings_registry import (
CheckboxField,
CustomComponentField,
@@ -56,6 +58,11 @@ _SELF_SETTINGS_SECTION_OPTIONS = [
"label": "Delivery Preferences",
"description": "Show personal delivery output and destination settings.",
},
{
"value": "search",
"label": "Search Preferences",
"description": "Show personal search mode and provider settings.",
},
{
"value": "notifications",
"label": "Notifications",
@@ -64,6 +71,13 @@ _SELF_SETTINGS_SECTION_OPTIONS = [
]
_SELF_SETTINGS_SECTION_VALUES = {option["value"] for option in _SELF_SETTINGS_SECTION_OPTIONS}
_SELF_SETTINGS_SECTION_DEFAULTS = [option["value"] for option in _SELF_SETTINGS_SECTION_OPTIONS]
_SEARCH_MODE_VALUES = {"direct", "universal"}
_SEARCH_PREFERENCE_PROVIDER_KEYS = {"METADATA_PROVIDER", "METADATA_PROVIDER_AUDIOBOOK"}
_SEARCH_PREFERENCE_VALIDATABLE_KEYS = {
"SEARCH_MODE",
"DEFAULT_RELEASE_SOURCE",
*_SEARCH_PREFERENCE_PROVIDER_KEYS,
}
_USERS_HEADING_DESCRIPTION_BY_AUTH_MODE = {
"builtin": (
@@ -147,6 +161,50 @@ def _get_request_policy_rule_columns():
]
def validate_search_preference_value(key: str, value: Any) -> tuple[Any, str | None]:
"""Validate and normalize a search preference value for user overrides."""
if key not in _SEARCH_PREFERENCE_VALIDATABLE_KEYS:
return value, None
if value is None:
return None, None
normalized_value = str(value).strip()
if key == "SEARCH_MODE":
normalized_mode = normalized_value.lower()
if normalized_mode not in _SEARCH_MODE_VALUES:
return value, "SEARCH_MODE must be 'direct' or 'universal'"
return normalized_mode, None
if key in _SEARCH_PREFERENCE_PROVIDER_KEYS:
if normalized_value == "":
return "", None
from shelfmark.metadata_providers import is_provider_registered
if not is_provider_registered(normalized_value):
return (
value,
f"{key} must be a valid metadata provider name or empty",
)
return normalized_value, None
if key == "DEFAULT_RELEASE_SOURCE":
if normalized_value == "":
return "", None
from shelfmark.release_sources import list_available_sources
valid_sources = {source["name"] for source in list_available_sources()}
if normalized_value not in valid_sources:
return (
value,
"DEFAULT_RELEASE_SOURCE must be a valid release source name or empty",
)
return normalized_value, None
return value, None
def _on_save_users(values):
"""Validate users/request-policy settings before persistence."""
if "VISIBLE_SELF_SETTINGS_SECTIONS" in values:
@@ -207,6 +265,18 @@ def _on_save_users(values):
}
values["REQUEST_POLICY_RULES"] = normalized_rules
for key in _SEARCH_PREFERENCE_VALIDATABLE_KEYS:
if key not in values:
continue
normalized_value, validation_error = validate_search_preference_value(key, values[key])
if validation_error:
return {
"error": True,
"message": validation_error,
"values": values,
}
values[key] = normalized_value
return {"error": False, "values": values}
+1 -1
View File
@@ -1,5 +1,5 @@
"""Core module - shared models, queue, and utilities."""
from shelfmark.core.models import BookInfo, QueueItem, SearchFilters, QueueStatus
from shelfmark.core.models import QueueItem, SearchFilters, QueueStatus
from shelfmark.core.queue import BookQueue, book_queue
from shelfmark.core.logger import setup_logger
+481 -267
View File
@@ -2,12 +2,26 @@
from __future__ import annotations
from typing import Any, Callable
from typing import Any, Callable, NamedTuple
from flask import Flask, jsonify, request, session
from shelfmark.core.activity_service import ActivityService
from shelfmark.core.activity_view_state_service import (
ADMIN_VIEWER_SCOPE,
NOAUTH_VIEWER_SCOPE,
ActivityViewStateService,
user_viewer_scope,
)
from shelfmark.core.download_history_service import ACTIVE_DOWNLOAD_STATUS, DownloadHistoryService, VALID_TERMINAL_STATUSES
from shelfmark.core.logger import setup_logger
from shelfmark.core.models import ACTIVE_QUEUE_STATUSES, QueueStatus, TERMINAL_QUEUE_STATUSES
from shelfmark.core.request_validation import RequestStatus
from shelfmark.core.request_helpers import (
emit_ws_event,
extract_release_source_id,
normalize_positive_int,
populate_request_usernames,
)
from shelfmark.core.user_db import UserDB
logger = setup_logger(__name__)
@@ -22,7 +36,11 @@ def _require_authenticated(resolve_auth_mode: Callable[[], str]):
return None
def _resolve_db_user_id(require_in_auth_mode: bool = True):
def _resolve_db_user_id(
require_in_auth_mode: bool = True,
*,
user_db: UserDB | None = None,
):
raw_db_user_id = session.get("db_user_id")
if raw_db_user_id is None:
if not require_in_auth_mode:
@@ -37,8 +55,10 @@ def _resolve_db_user_id(require_in_auth_mode: bool = True):
403,
)
try:
return int(raw_db_user_id), None
parsed_db_user_id = int(raw_db_user_id)
except (TypeError, ValueError):
if not require_in_auth_mode:
return None, None
return None, (
jsonify(
{
@@ -49,30 +69,118 @@ def _resolve_db_user_id(require_in_auth_mode: bool = True):
403,
)
if parsed_db_user_id < 1:
if not require_in_auth_mode:
return None, None
return None, (
jsonify(
{
"error": "User identity unavailable for activity workflow",
"code": "user_identity_unavailable",
}
),
403,
)
def _emit_activity_event(ws_manager: Any | None, *, room: str, payload: dict[str, Any]) -> None:
if ws_manager is None:
return
try:
socketio = getattr(ws_manager, "socketio", None)
is_enabled = getattr(ws_manager, "is_enabled", None)
if socketio is None or not callable(is_enabled) or not is_enabled():
return
socketio.emit("activity_update", payload, to=room)
except Exception as exc:
logger.warning("Failed to emit activity_update event: %s", exc)
if user_db is not None:
try:
db_user = user_db.get_user(user_id=parsed_db_user_id)
except Exception as exc:
logger.warning("Failed to validate activity db identity %s: %s", parsed_db_user_id, exc)
db_user = None
if db_user is None:
if not require_in_auth_mode:
return None, None
return None, (
jsonify(
{
"error": "User identity unavailable for activity workflow",
"code": "user_identity_unavailable",
}
),
403,
)
return parsed_db_user_id, None
class _ActorContext(NamedTuple):
db_user_id: int | None
is_no_auth: bool
is_admin: bool
owner_scope: int | None
viewer_scope: str
def _resolve_activity_actor(
*,
user_db: UserDB,
resolve_auth_mode: Callable[[], str],
) -> tuple[_ActorContext | None, Any | None]:
"""Resolve acting user identity for activity mutations.
Returns (actor, error_response). On success actor is non-None.
"""
if resolve_auth_mode() == "none":
return _ActorContext(
db_user_id=None,
is_no_auth=True,
is_admin=True,
owner_scope=None,
viewer_scope=NOAUTH_VIEWER_SCOPE,
), None
db_user_id, db_gate = _resolve_db_user_id(user_db=user_db)
if db_user_id is None:
return None, db_gate
is_admin = bool(session.get("is_admin"))
viewer_scope = ADMIN_VIEWER_SCOPE if is_admin else user_viewer_scope(db_user_id)
return _ActorContext(
db_user_id=db_user_id,
is_no_auth=False,
is_admin=is_admin,
owner_scope=None if is_admin else db_user_id,
viewer_scope=viewer_scope,
), None
def _activity_ws_room(actor: _ActorContext) -> str:
"""Resolve the WebSocket room for activity events."""
if actor.is_no_auth or actor.is_admin:
return "admins"
if actor.db_user_id is not None:
return f"user_{actor.db_user_id}"
return "admins"
def _check_item_ownership(actor: _ActorContext, row: dict[str, Any]) -> Any | None:
"""Return a 403 response if the actor doesn't own the item, else None."""
if actor.is_admin:
return None
owner_user_id = normalize_positive_int(row.get("user_id"))
if owner_user_id != actor.db_user_id:
return jsonify({"error": "Forbidden"}), 403
return None
def _check_terminal_download(row: dict[str, Any]) -> Any | None:
final_status = str(row.get("final_status") or "").strip().lower()
if final_status not in VALID_TERMINAL_STATUSES:
return jsonify({"error": "Only terminal downloads can be dismissed"}), 409
return None
def _check_terminal_request(row: dict[str, Any]) -> Any | None:
if _request_terminal_status(row) is None:
return jsonify({"error": "Only terminal requests can be dismissed"}), 409
return None
def _list_visible_requests(user_db: UserDB, *, is_admin: bool, db_user_id: int | None) -> list[dict[str, Any]]:
if is_admin:
request_rows = user_db.list_requests()
user_cache: dict[int, str] = {}
for row in request_rows:
requester_id = row["user_id"]
if requester_id not in user_cache:
requester = user_db.get_user(user_id=requester_id)
user_cache[requester_id] = requester.get("username", "") if requester else ""
row["username"] = user_cache[requester_id]
populate_request_usernames(request_rows, user_db)
return request_rows
if db_user_id is None:
@@ -80,112 +188,84 @@ def _list_visible_requests(user_db: UserDB, *, is_admin: bool, db_user_id: int |
return user_db.list_requests(user_id=db_user_id)
def _parse_download_item_key(item_key: str) -> str | None:
if not isinstance(item_key, str) or not item_key.startswith("download:"):
def _parse_item_key(item_key: Any, prefix: str) -> str | None:
"""Extract the value after 'prefix:' from an item_key string."""
if not isinstance(item_key, str) or not item_key.startswith(f"{prefix}:"):
return None
task_id = item_key.split(":", 1)[1].strip()
return task_id or None
value = item_key.split(":", 1)[1].strip()
return value or None
def _parse_request_item_key(item_key: str) -> int | None:
if not isinstance(item_key, str) or not item_key.startswith("request:"):
return None
raw_id = item_key.split(":", 1)[1].strip()
try:
parsed = int(raw_id)
except (TypeError, ValueError):
return None
return parsed if parsed > 0 else None
_ALL_BUCKET_KEYS = (*ACTIVE_QUEUE_STATUSES, *TERMINAL_QUEUE_STATUSES)
def _task_id_from_download_item_key(item_key: str) -> str | None:
task_id = _parse_download_item_key(item_key)
if task_id is None:
return None
return task_id
def _merge_terminal_snapshot_backfill(
def _build_download_status_from_db(
*,
status: dict[str, dict[str, Any]],
terminal_rows: list[dict[str, Any]],
) -> None:
existing_task_ids: set[str] = set()
for bucket_key in ("queued", "resolving", "locating", "downloading", "complete", "error", "cancelled"):
bucket = status.get(bucket_key)
db_rows: list[dict[str, Any]],
queue_status: dict[str, dict[str, Any]],
) -> dict[str, dict[str, Any]]:
"""Build the download status dict from DB rows, overlaying live queue data.
Active DB rows are matched against the queue for live progress.
Terminal DB rows go directly into their final bucket.
Stale active rows (no queue entry) are treated as interrupted errors.
"""
status: dict[str, dict[str, Any]] = {key: {} for key in _ALL_BUCKET_KEYS}
# Index queue items by task_id for fast lookup: task_id -> (bucket_key, payload)
queue_index: dict[str, tuple[str, dict[str, Any]]] = {}
for bucket_key in _ALL_BUCKET_KEYS:
bucket = queue_status.get(bucket_key)
if not isinstance(bucket, dict):
continue
existing_task_ids.update(str(task_id) for task_id in bucket.keys())
for task_id, payload in bucket.items():
queue_index[str(task_id)] = (bucket_key, payload)
for row in terminal_rows:
item_key = row.get("item_key")
if not isinstance(item_key, str):
continue
task_id = _task_id_from_download_item_key(item_key)
if not task_id or task_id in existing_task_ids:
for row in db_rows:
task_id = str(row.get("task_id") or "").strip()
if not task_id:
continue
final_status = row.get("final_status")
if final_status not in {"complete", "error", "cancelled"}:
continue
snapshot = row.get("snapshot")
if not isinstance(snapshot, dict):
continue
raw_download = snapshot.get("download")
if not isinstance(raw_download, dict):
continue
if final_status == ACTIVE_DOWNLOAD_STATUS:
queue_entry = queue_index.pop(task_id, None)
if queue_entry is not None:
bucket_key, queue_payload = queue_entry
status[bucket_key][task_id] = queue_payload
else:
# Stale active row — no queue entry means it was interrupted
download_payload = DownloadHistoryService.to_download_payload(row)
download_payload["status_message"] = "Interrupted"
status[QueueStatus.ERROR][task_id] = download_payload
elif final_status in VALID_TERMINAL_STATUSES:
download_payload = DownloadHistoryService.to_download_payload(row)
# For complete/cancelled the saved status_message is a stale
# progress string (e.g. "Fetching download sources") — clear it
# so the frontend only shows its own status label. Error rows
# keep theirs since the message describes the failure.
if final_status in ("complete", "cancelled"):
download_payload["status_message"] = None
status[final_status][task_id] = download_payload
download_payload = dict(raw_download)
if not isinstance(download_payload.get("id"), str):
download_payload["id"] = task_id
if final_status not in status or not isinstance(status.get(final_status), dict):
status[final_status] = {}
status[final_status][task_id] = download_payload
existing_task_ids.add(task_id)
def _collect_active_download_item_keys(status: dict[str, dict[str, Any]]) -> set[str]:
active_keys: set[str] = set()
for bucket_key in ("queued", "resolving", "locating", "downloading"):
bucket = status.get(bucket_key)
if not isinstance(bucket, dict):
continue
for task_id in bucket.keys():
normalized_task_id = str(task_id).strip()
if not normalized_task_id:
continue
active_keys.add(f"download:{normalized_task_id}")
return active_keys
def _extract_request_source_id(row: dict[str, Any]) -> str | None:
release_data = row.get("release_data")
if not isinstance(release_data, dict):
return None
source_id = release_data.get("source_id")
if not isinstance(source_id, str):
return None
normalized = source_id.strip()
return normalized or None
return status
def _request_terminal_status(row: dict[str, Any]) -> str | None:
request_status = row.get("status")
if request_status == "pending":
if request_status == RequestStatus.PENDING:
return None
if request_status == "rejected":
return "rejected"
if request_status == "cancelled":
return "cancelled"
if request_status != "fulfilled":
if request_status == RequestStatus.REJECTED:
return RequestStatus.REJECTED
if request_status == RequestStatus.CANCELLED:
return RequestStatus.CANCELLED
if request_status != RequestStatus.FULFILLED:
return None
delivery_state = str(row.get("delivery_state") or "").strip().lower()
if delivery_state in {"error", "cancelled"}:
if delivery_state in {QueueStatus.ERROR, QueueStatus.CANCELLED}:
return delivery_state
return "complete"
return QueueStatus.COMPLETE
def _minimal_request_snapshot(request_row: dict[str, Any], request_id: int) -> dict[str, Any]:
@@ -207,7 +287,7 @@ def _minimal_request_snapshot(request_row: dict[str, Any], request_id: int) -> d
"note": request_row.get("note"),
"admin_note": request_row.get("admin_note"),
"created_at": request_row.get("created_at"),
"updated_at": request_row.get("updated_at"),
"updated_at": request_row.get("reviewed_at") or request_row.get("created_at"),
}
username = request_row.get("username")
if isinstance(username, str):
@@ -215,57 +295,38 @@ def _minimal_request_snapshot(request_row: dict[str, Any], request_id: int) -> d
return {"kind": "request", "request": minimal_request}
def _get_existing_activity_log_id_for_item(
def _request_history_entry(
request_row: dict[str, Any],
*,
activity_service: ActivityService,
user_db: UserDB,
item_type: str,
item_key: str,
) -> int | None:
if item_type not in {"request", "download"}:
return None
if not isinstance(item_key, str) or not item_key.strip():
return None
existing_log_id = activity_service.get_latest_activity_log_id(
item_type=item_type,
item_key=item_key,
)
if existing_log_id is not None or item_type != "request":
return existing_log_id
request_id = _parse_request_item_key(item_key)
dismissed_at: str | None,
) -> dict[str, Any] | None:
request_id = normalize_positive_int(request_row.get("id"))
if request_id is None:
return None
row = user_db.get_request(request_id)
if row is None:
return None
final_status = _request_terminal_status(row)
if final_status is None:
return None
source_id = _extract_request_source_id(row)
payload = activity_service.record_terminal_snapshot(
user_id=row.get("user_id"),
item_type="request",
item_key=item_key,
origin="request",
final_status=final_status,
snapshot=_minimal_request_snapshot(row, request_id),
request_id=request_id,
source_id=source_id,
)
return int(payload["id"])
final_status = _request_terminal_status(request_row)
item_key = f"request:{request_id}"
return {
"id": item_key,
"user_id": request_row.get("user_id"),
"item_type": "request",
"item_key": item_key,
"dismissed_at": dismissed_at,
"snapshot": _minimal_request_snapshot(request_row, request_id),
"origin": "request",
"final_status": final_status,
"terminal_at": request_row.get("reviewed_at") or request_row.get("created_at"),
"request_id": request_id,
"source_id": extract_release_source_id(request_row.get("release_data")),
}
def register_activity_routes(
app: Flask,
user_db: UserDB,
*,
activity_service: ActivityService,
activity_view_state_service: ActivityViewStateService,
download_history_service: DownloadHistoryService,
resolve_auth_mode: Callable[[], str],
resolve_status_scope: Callable[[], tuple[bool, int | None, bool]],
queue_status: Callable[..., dict[str, dict[str, Any]]],
sync_request_delivery_states: Callable[..., list[dict[str, Any]]],
emit_request_updates: Callable[[list[dict[str, Any]]], None],
@@ -279,58 +340,65 @@ def register_activity_routes(
if auth_gate is not None:
return auth_gate
is_admin, db_user_id, can_access_status = resolve_status_scope()
if not can_access_status:
return (
jsonify(
{
"error": "User identity unavailable for activity workflow",
"code": "user_identity_unavailable",
}
),
403,
)
actor, actor_error = _resolve_activity_actor(
user_db=user_db,
resolve_auth_mode=resolve_auth_mode,
)
if actor_error is not None:
return actor_error
hidden_rows = activity_view_state_service.list_hidden(viewer_scope=actor.viewer_scope)
hidden_item_keys = {str(row.get("item_key") or "").strip() for row in hidden_rows}
dismissed_entries = [
{
"item_type": str(row.get("item_type") or "").strip().lower(),
"item_key": str(row.get("item_key") or "").strip(),
}
for row in hidden_rows
if str(row.get("item_type") or "").strip().lower() in {"download", "request"}
and str(row.get("item_key") or "").strip()
]
live_queue = queue_status(user_id=actor.owner_scope)
db_rows = download_history_service.list_recent(
user_id=actor.owner_scope,
limit=200,
)
visible_db_rows = [
row
for row in db_rows
if f"download:{str(row.get('task_id') or '').strip()}" not in hidden_item_keys
]
status = _build_download_status_from_db(
db_rows=visible_db_rows,
queue_status=live_queue,
)
viewer_db_user_id, _ = _resolve_db_user_id(require_in_auth_mode=False)
scoped_user_id = None if is_admin else db_user_id
status = queue_status(user_id=scoped_user_id)
updated_requests = sync_request_delivery_states(
user_db,
queue_status=status,
user_id=scoped_user_id,
user_id=actor.owner_scope,
)
emit_request_updates(updated_requests)
request_rows = _list_visible_requests(user_db, is_admin=is_admin, db_user_id=db_user_id)
if not is_admin and db_user_id is not None:
try:
terminal_rows = activity_service.get_undismissed_terminal_downloads(db_user_id, limit=200)
_merge_terminal_snapshot_backfill(status=status, terminal_rows=terminal_rows)
except Exception as exc:
logger.warning("Failed to merge terminal snapshot backfill rows: %s", exc)
if viewer_db_user_id is not None:
active_download_keys = _collect_active_download_item_keys(status)
if active_download_keys:
try:
activity_service.clear_dismissals_for_item_keys(
user_id=viewer_db_user_id,
item_type="download",
item_keys=active_download_keys,
)
except Exception as exc:
logger.warning("Failed to clear stale download dismissals for active tasks: %s", exc)
dismissed: list[dict[str, str]] = []
# Admins can view unscoped queue status, but dismissals remain per-viewer.
if viewer_db_user_id is not None:
dismissed = activity_service.get_dismissal_set(viewer_db_user_id)
request_rows = _list_visible_requests(
user_db,
is_admin=actor.is_admin,
db_user_id=actor.db_user_id,
)
visible_request_rows: list[dict[str, Any]] = []
for row in request_rows:
request_id = normalize_positive_int(row.get("id"))
if request_id is None:
continue
if f"request:{request_id}" in hidden_item_keys:
continue
visible_request_rows.append(row)
return jsonify(
{
"status": status,
"requests": request_rows,
"dismissed": dismissed,
"requests": visible_request_rows,
"dismissed": dismissed_entries,
}
)
@@ -340,49 +408,83 @@ def register_activity_routes(
if auth_gate is not None:
return auth_gate
db_user_id, db_gate = _resolve_db_user_id()
if db_gate is not None or db_user_id is None:
return db_gate
actor, actor_error = _resolve_activity_actor(
user_db=user_db,
resolve_auth_mode=resolve_auth_mode,
)
if actor_error is not None:
return actor_error
data = request.get_json(silent=True)
if not isinstance(data, dict):
return jsonify({"error": "Invalid payload"}), 400
activity_log_id = data.get("activity_log_id")
if activity_log_id is None:
try:
activity_log_id = _get_existing_activity_log_id_for_item(
activity_service=activity_service,
user_db=user_db,
item_type=data.get("item_type"),
item_key=data.get("item_key"),
)
except Exception as exc:
logger.warning("Failed to resolve activity snapshot id for dismiss payload: %s", exc)
activity_log_id = None
item_type = str(data.get("item_type") or "").strip().lower()
item_key = data.get("item_key")
try:
dismissal = activity_service.dismiss_item(
user_id=db_user_id,
item_type=data.get("item_type"),
item_key=data.get("item_key"),
activity_log_id=activity_log_id,
dismissal_item: dict[str, str] | None = None
if item_type == "download":
task_id = _parse_item_key(item_key, "download")
if task_id is None:
return jsonify({"error": "item_key must be in the format download:<task_id>"}), 400
existing = download_history_service.get_by_task_id(task_id)
if existing is None:
return jsonify({"error": "Download not found"}), 404
ownership_gate = _check_item_ownership(actor, existing)
if ownership_gate is not None:
return ownership_gate
terminal_gate = _check_terminal_download(existing)
if terminal_gate is not None:
return terminal_gate
activity_view_state_service.dismiss(
viewer_scope=actor.viewer_scope,
item_type="download",
item_key=f"download:{task_id}",
)
except ValueError as exc:
return jsonify({"error": str(exc)}), 400
dismissal_item = {"item_type": "download", "item_key": f"download:{task_id}"}
_emit_activity_event(
elif item_type == "request":
request_id = normalize_positive_int(_parse_item_key(item_key, "request"))
if request_id is None:
return jsonify({"error": "item_key must be in the format request:<id>"}), 400
request_row = user_db.get_request(request_id)
if request_row is None:
return jsonify({"error": "Request not found"}), 404
ownership_gate = _check_item_ownership(actor, request_row)
if ownership_gate is not None:
return ownership_gate
terminal_gate = _check_terminal_request(request_row)
if terminal_gate is not None:
return terminal_gate
activity_view_state_service.dismiss(
viewer_scope=actor.viewer_scope,
item_type="request",
item_key=f"request:{request_id}",
)
dismissal_item = {"item_type": "request", "item_key": f"request:{request_id}"}
else:
return jsonify({"error": "item_type must be one of: download, request"}), 400
room = _activity_ws_room(actor)
emit_ws_event(
ws_manager,
room=f"user_{db_user_id}",
event_name="activity_update",
room=room,
payload={
"kind": "dismiss",
"user_id": db_user_id,
"item_type": dismissal["item_type"],
"item_key": dismissal["item_key"],
"item_type": dismissal_item["item_type"],
"item_key": dismissal_item["item_key"],
},
)
return jsonify({"status": "dismissed", "item": dismissal})
return jsonify({"status": "dismissed", "item": dismissal_item})
@app.route("/api/activity/dismiss-many", methods=["POST"])
def api_activity_dismiss_many():
@@ -390,9 +492,12 @@ def register_activity_routes(
if auth_gate is not None:
return auth_gate
db_user_id, db_gate = _resolve_db_user_id()
if db_gate is not None or db_user_id is None:
return db_gate
actor, actor_error = _resolve_activity_actor(
user_db=user_db,
resolve_auth_mode=resolve_auth_mode,
)
if actor_error is not None:
return actor_error
data = request.get_json(silent=True)
if not isinstance(data, dict):
@@ -401,43 +506,75 @@ def register_activity_routes(
if not isinstance(items, list):
return jsonify({"error": "items must be an array"}), 400
normalized_items: list[dict[str, Any]] = []
dismissal_items: list[dict[str, str]] = []
missing_item_keys: list[str] = []
for item in items:
if not isinstance(item, dict):
return jsonify({"error": "items must contain objects"}), 400
activity_log_id = item.get("activity_log_id")
if activity_log_id is None:
try:
activity_log_id = _get_existing_activity_log_id_for_item(
activity_service=activity_service,
user_db=user_db,
item_type=item.get("item_type"),
item_key=item.get("item_key"),
)
except Exception as exc:
logger.warning("Failed to resolve activity snapshot id for dismiss-many item: %s", exc)
activity_log_id = None
item_type = str(item.get("item_type") or "").strip().lower()
item_key = item.get("item_key")
normalized_payload = {
"item_type": item.get("item_type"),
"item_key": item.get("item_key"),
}
if activity_log_id is not None:
normalized_payload["activity_log_id"] = activity_log_id
normalized_items.append(normalized_payload)
if item_type == "download":
task_id = _parse_item_key(item_key, "download")
if task_id is None:
return jsonify({"error": "download item_key must be in the format download:<task_id>"}), 400
existing = download_history_service.get_by_task_id(task_id)
if existing is None:
missing_item_keys.append(f"download:{task_id}")
continue
ownership_gate = _check_item_ownership(actor, existing)
if ownership_gate is not None:
return ownership_gate
terminal_gate = _check_terminal_download(existing)
if terminal_gate is not None:
return terminal_gate
dismissal_items.append({"item_type": "download", "item_key": f"download:{task_id}"})
continue
try:
dismissed_count = activity_service.dismiss_many(user_id=db_user_id, items=normalized_items)
except ValueError as exc:
return jsonify({"error": str(exc)}), 400
if item_type == "request":
request_id = normalize_positive_int(_parse_item_key(item_key, "request"))
if request_id is None:
return jsonify({"error": "request item_key must be in the format request:<id>"}), 400
request_row = user_db.get_request(request_id)
if request_row is None:
missing_item_keys.append(f"request:{request_id}")
continue
ownership_gate = _check_item_ownership(actor, request_row)
if ownership_gate is not None:
return ownership_gate
terminal_gate = _check_terminal_request(request_row)
if terminal_gate is not None:
return terminal_gate
dismissal_items.append({"item_type": "request", "item_key": f"request:{request_id}"})
continue
_emit_activity_event(
return jsonify({"error": "item_type must be one of: download, request"}), 400
if missing_item_keys:
return (
jsonify(
{
"error": "One or more activity items were not found",
"missing_item_keys": missing_item_keys,
}
),
404,
)
dismissed_count = activity_view_state_service.dismiss_many(
viewer_scope=actor.viewer_scope,
items=dismissal_items,
)
room = _activity_ws_room(actor)
emit_ws_event(
ws_manager,
room=f"user_{db_user_id}",
event_name="activity_update",
room=room,
payload={
"kind": "dismiss_many",
"user_id": db_user_id,
"count": dismissed_count,
},
)
@@ -450,18 +587,88 @@ def register_activity_routes(
if auth_gate is not None:
return auth_gate
db_user_id, db_gate = _resolve_db_user_id()
if db_gate is not None or db_user_id is None:
return db_gate
actor, actor_error = _resolve_activity_actor(
user_db=user_db,
resolve_auth_mode=resolve_auth_mode,
)
if actor_error is not None:
return actor_error
limit = request.args.get("limit", type=int, default=50) or 50
offset = request.args.get("offset", type=int, default=0) or 0
limit = request.args.get("limit", type=int, default=50)
offset = request.args.get("offset", type=int, default=0)
if limit is None:
limit = 50
if offset is None:
offset = 0
if limit < 1:
return jsonify({"error": "limit must be a positive integer"}), 400
if offset < 0:
return jsonify({"error": "offset must be a non-negative integer"}), 400
try:
history = activity_service.get_history(db_user_id, limit=limit, offset=offset)
except ValueError as exc:
return jsonify({"error": str(exc)}), 400
return jsonify(history)
history_rows = activity_view_state_service.list_history(
viewer_scope=actor.viewer_scope,
limit=limit,
offset=offset,
)
payload: list[dict[str, Any]] = []
for history_row in history_rows:
item_type = str(history_row.get("item_type") or "").strip().lower()
item_key = str(history_row.get("item_key") or "").strip()
dismissed_at = history_row.get("dismissed_at")
if not isinstance(dismissed_at, str) or not dismissed_at.strip():
raise RuntimeError(f"Activity history state missing dismissed_at for {item_key}")
if item_type == "download":
task_id = _parse_item_key(item_key, "download")
if task_id is None:
raise RuntimeError(f"Invalid activity history item_key: {item_key}")
download_row = download_history_service.get_by_task_id(task_id)
if download_row is None:
raise RuntimeError(f"Download history row not found for {item_key}")
if not actor.is_admin:
owner_user_id = normalize_positive_int(download_row.get("user_id"))
if owner_user_id != actor.db_user_id:
raise RuntimeError(f"Viewer state out of scope for {item_key}")
payload.append(
DownloadHistoryService.to_history_row(
download_row,
dismissed_at=dismissed_at,
)
)
continue
if item_type == "request":
request_id = normalize_positive_int(_parse_item_key(item_key, "request"))
if request_id is None:
raise RuntimeError(f"Invalid activity history item_key: {item_key}")
request_row = user_db.get_request(request_id)
if request_row is None:
raise RuntimeError(f"Request row not found for {item_key}")
if not actor.is_admin:
owner_user_id = normalize_positive_int(request_row.get("user_id"))
if owner_user_id != actor.db_user_id:
raise RuntimeError(f"Viewer state out of scope for {item_key}")
populate_request_usernames([request_row], user_db)
entry = _request_history_entry(
request_row,
dismissed_at=dismissed_at,
)
if entry is None:
raise RuntimeError(f"Failed to build request history entry for {item_key}")
payload.append(entry)
continue
raise RuntimeError(f"Unknown activity history item_type: {item_type}")
return jsonify(payload)
@app.route("/api/activity/history", methods=["DELETE"])
def api_activity_history_clear():
@@ -469,18 +676,25 @@ def register_activity_routes(
if auth_gate is not None:
return auth_gate
db_user_id, db_gate = _resolve_db_user_id()
if db_gate is not None or db_user_id is None:
return db_gate
actor, actor_error = _resolve_activity_actor(
user_db=user_db,
resolve_auth_mode=resolve_auth_mode,
)
if actor_error is not None:
return actor_error
deleted_count = activity_service.clear_history(db_user_id)
_emit_activity_event(
cleared_count = activity_view_state_service.clear_history(
viewer_scope=actor.viewer_scope,
)
room = _activity_ws_room(actor)
emit_ws_event(
ws_manager,
room=f"user_{db_user_id}",
event_name="activity_update",
room=room,
payload={
"kind": "history_cleared",
"user_id": db_user_id,
"count": deleted_count,
"count": cleared_count,
},
)
return jsonify({"status": "cleared", "deleted_count": deleted_count})
return jsonify({"status": "cleared", "cleared_count": cleared_count})
-618
View File
@@ -1,618 +0,0 @@
"""Persistence helpers for Activity dismissals and terminal snapshots."""
from __future__ import annotations
from datetime import datetime, timezone
import json
import sqlite3
from typing import Any, Iterable
VALID_ITEM_TYPES = frozenset({"download", "request"})
VALID_ORIGINS = frozenset({"direct", "request", "requested"})
VALID_FINAL_STATUSES = frozenset({"complete", "error", "cancelled", "rejected"})
def _now_timestamp() -> str:
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def _normalize_item_type(item_type: Any) -> str:
if not isinstance(item_type, str):
raise ValueError("item_type must be a string")
normalized = item_type.strip().lower()
if normalized not in VALID_ITEM_TYPES:
raise ValueError("item_type must be one of: download, request")
return normalized
def _normalize_item_key(item_key: Any) -> str:
if not isinstance(item_key, str):
raise ValueError("item_key must be a string")
normalized = item_key.strip()
if not normalized:
raise ValueError("item_key must not be empty")
return normalized
def _normalize_origin(origin: Any) -> str:
if not isinstance(origin, str):
raise ValueError("origin must be a string")
normalized = origin.strip().lower()
if normalized not in VALID_ORIGINS:
raise ValueError("origin must be one of: direct, request, requested")
return normalized
def _normalize_final_status(final_status: Any) -> str:
if not isinstance(final_status, str):
raise ValueError("final_status must be a string")
normalized = final_status.strip().lower()
if normalized not in VALID_FINAL_STATUSES:
raise ValueError("final_status must be one of: complete, error, cancelled, rejected")
return normalized
def build_item_key(item_type: str, raw_id: Any) -> str:
"""Build a stable item key used by dismiss/history APIs."""
normalized_type = _normalize_item_type(item_type)
if normalized_type == "request":
try:
request_id = int(raw_id)
except (TypeError, ValueError) as exc:
raise ValueError("request item IDs must be integers") from exc
if request_id < 1:
raise ValueError("request item IDs must be positive integers")
return f"request:{request_id}"
if not isinstance(raw_id, str):
raise ValueError("download item IDs must be strings")
task_id = raw_id.strip()
if not task_id:
raise ValueError("download item IDs must not be empty")
return f"download:{task_id}"
def build_request_item_key(request_id: int) -> str:
"""Build a request item key."""
return build_item_key("request", request_id)
def build_download_item_key(task_id: str) -> str:
"""Build a download item key."""
return build_item_key("download", task_id)
def _parse_request_id_from_item_key(item_key: Any) -> int | None:
if not isinstance(item_key, str) or not item_key.startswith("request:"):
return None
raw_value = item_key.split(":", 1)[1].strip()
try:
parsed = int(raw_value)
except (TypeError, ValueError):
return None
return parsed if parsed > 0 else None
def _request_final_status(request_status: Any, delivery_state: Any) -> str | None:
status = str(request_status or "").strip().lower()
if status == "pending":
return None
if status == "rejected":
return "rejected"
if status == "cancelled":
return "cancelled"
if status != "fulfilled":
return None
delivery = str(delivery_state or "").strip().lower()
if delivery in {"error", "cancelled"}:
return delivery
return "complete"
class ActivityService:
"""Service for per-user activity dismissals and terminal history snapshots."""
def __init__(self, db_path: str):
self._db_path = db_path
def _connect(self) -> sqlite3.Connection:
conn = sqlite3.connect(self._db_path)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA foreign_keys = ON")
return conn
@staticmethod
def _coerce_positive_int(value: Any, field: str) -> int:
try:
parsed = int(value)
except (TypeError, ValueError) as exc:
raise ValueError(f"{field} must be an integer") from exc
if parsed < 1:
raise ValueError(f"{field} must be a positive integer")
return parsed
@staticmethod
def _row_to_dict(row: sqlite3.Row | None) -> dict[str, Any] | None:
return dict(row) if row is not None else None
@staticmethod
def _parse_json_column(value: Any) -> Any:
if not isinstance(value, str):
return None
try:
return json.loads(value)
except (ValueError, TypeError):
return None
def _build_legacy_request_snapshot(
self,
conn: sqlite3.Connection,
request_id: int,
) -> tuple[dict[str, Any] | None, str | None]:
request_row = conn.execute(
"""
SELECT
id,
user_id,
status,
delivery_state,
request_level,
book_data,
release_data,
note,
admin_note,
created_at,
reviewed_at
FROM download_requests
WHERE id = ?
""",
(request_id,),
).fetchone()
if request_row is None:
return None, None
row_dict = dict(request_row)
book_data = self._parse_json_column(row_dict.get("book_data"))
release_data = self._parse_json_column(row_dict.get("release_data"))
if not isinstance(book_data, dict):
book_data = {}
if not isinstance(release_data, dict):
release_data = {}
snapshot = {
"kind": "request",
"request": {
"id": int(row_dict["id"]),
"user_id": row_dict.get("user_id"),
"status": row_dict.get("status"),
"delivery_state": row_dict.get("delivery_state"),
"request_level": row_dict.get("request_level"),
"book_data": book_data,
"release_data": release_data,
"note": row_dict.get("note"),
"admin_note": row_dict.get("admin_note"),
"created_at": row_dict.get("created_at"),
"updated_at": row_dict.get("reviewed_at") or row_dict.get("created_at"),
},
}
final_status = _request_final_status(row_dict.get("status"), row_dict.get("delivery_state"))
return snapshot, final_status
def record_terminal_snapshot(
self,
*,
user_id: int | None,
item_type: str,
item_key: str,
origin: str,
final_status: str,
snapshot: dict[str, Any],
request_id: int | None = None,
source_id: str | None = None,
terminal_at: str | None = None,
) -> dict[str, Any]:
"""Record a durable terminal-state snapshot for an activity item."""
normalized_item_type = _normalize_item_type(item_type)
normalized_item_key = _normalize_item_key(item_key)
normalized_origin = _normalize_origin(origin)
normalized_final_status = _normalize_final_status(final_status)
if not isinstance(snapshot, dict):
raise ValueError("snapshot must be an object")
if user_id is not None:
user_id = self._coerce_positive_int(user_id, "user_id")
if request_id is not None:
request_id = self._coerce_positive_int(request_id, "request_id")
if source_id is not None and not isinstance(source_id, str):
raise ValueError("source_id must be a string when provided")
if source_id is not None:
source_id = source_id.strip() or None
effective_terminal_at = terminal_at if isinstance(terminal_at, str) and terminal_at.strip() else _now_timestamp()
serialized_snapshot = json.dumps(snapshot, separators=(",", ":"), ensure_ascii=False)
conn = self._connect()
try:
cursor = conn.execute(
"""
INSERT INTO activity_log (
user_id,
item_type,
item_key,
request_id,
source_id,
origin,
final_status,
snapshot_json,
terminal_at
)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
user_id,
normalized_item_type,
normalized_item_key,
request_id,
source_id,
normalized_origin,
normalized_final_status,
serialized_snapshot,
effective_terminal_at,
),
)
snapshot_id = int(cursor.lastrowid)
conn.commit()
row = conn.execute(
"SELECT * FROM activity_log WHERE id = ?",
(snapshot_id,),
).fetchone()
payload = self._row_to_dict(row)
if payload is None:
raise ValueError("Failed to read back recorded activity snapshot")
return payload
finally:
conn.close()
def get_latest_activity_log_id(self, *, item_type: str, item_key: str) -> int | None:
"""Get the newest snapshot ID for an item key."""
normalized_item_type = _normalize_item_type(item_type)
normalized_item_key = _normalize_item_key(item_key)
conn = self._connect()
try:
row = conn.execute(
"""
SELECT id
FROM activity_log
WHERE item_type = ? AND item_key = ?
ORDER BY terminal_at DESC, id DESC
LIMIT 1
""",
(normalized_item_type, normalized_item_key),
).fetchone()
if row is None:
return None
return int(row["id"])
finally:
conn.close()
def dismiss_item(
self,
*,
user_id: int,
item_type: str,
item_key: str,
activity_log_id: int | None = None,
) -> dict[str, Any]:
"""Dismiss an item for a specific user (upsert)."""
normalized_user_id = self._coerce_positive_int(user_id, "user_id")
normalized_item_type = _normalize_item_type(item_type)
normalized_item_key = _normalize_item_key(item_key)
normalized_log_id = (
self._coerce_positive_int(activity_log_id, "activity_log_id")
if activity_log_id is not None
else self.get_latest_activity_log_id(
item_type=normalized_item_type,
item_key=normalized_item_key,
)
)
conn = self._connect()
try:
conn.execute(
"""
INSERT INTO activity_dismissals (
user_id,
item_type,
item_key,
activity_log_id,
dismissed_at
)
VALUES (?, ?, ?, ?, ?)
ON CONFLICT(user_id, item_type, item_key)
DO UPDATE SET
activity_log_id = excluded.activity_log_id,
dismissed_at = excluded.dismissed_at
""",
(
normalized_user_id,
normalized_item_type,
normalized_item_key,
normalized_log_id,
_now_timestamp(),
),
)
conn.commit()
row = conn.execute(
"""
SELECT *
FROM activity_dismissals
WHERE user_id = ? AND item_type = ? AND item_key = ?
""",
(normalized_user_id, normalized_item_type, normalized_item_key),
).fetchone()
payload = self._row_to_dict(row)
if payload is None:
raise ValueError("Failed to read back dismissal row")
return payload
finally:
conn.close()
def dismiss_many(self, *, user_id: int, items: Iterable[dict[str, Any]]) -> int:
"""Dismiss many items for one user."""
normalized_user_id = self._coerce_positive_int(user_id, "user_id")
normalized_items: list[tuple[str, str, int | None]] = []
for item in items:
if not isinstance(item, dict):
raise ValueError("items must contain objects")
normalized_item_type = _normalize_item_type(item.get("item_type"))
normalized_item_key = _normalize_item_key(item.get("item_key"))
raw_log_id = item.get("activity_log_id")
normalized_log_id = (
self._coerce_positive_int(raw_log_id, "activity_log_id")
if raw_log_id is not None
else self.get_latest_activity_log_id(
item_type=normalized_item_type,
item_key=normalized_item_key,
)
)
normalized_items.append((normalized_item_type, normalized_item_key, normalized_log_id))
if not normalized_items:
return 0
conn = self._connect()
try:
timestamp = _now_timestamp()
for item_type, item_key, activity_log_id in normalized_items:
conn.execute(
"""
INSERT INTO activity_dismissals (
user_id,
item_type,
item_key,
activity_log_id,
dismissed_at
)
VALUES (?, ?, ?, ?, ?)
ON CONFLICT(user_id, item_type, item_key)
DO UPDATE SET
activity_log_id = excluded.activity_log_id,
dismissed_at = excluded.dismissed_at
""",
(
normalized_user_id,
item_type,
item_key,
activity_log_id,
timestamp,
),
)
conn.commit()
return len(normalized_items)
finally:
conn.close()
def get_dismissal_set(self, user_id: int) -> list[dict[str, str]]:
"""Return dismissed item keys for one user."""
normalized_user_id = self._coerce_positive_int(user_id, "user_id")
conn = self._connect()
try:
rows = conn.execute(
"""
SELECT item_type, item_key
FROM activity_dismissals
WHERE user_id = ?
ORDER BY dismissed_at DESC, id DESC
""",
(normalized_user_id,),
).fetchall()
return [
{
"item_type": str(row["item_type"]),
"item_key": str(row["item_key"]),
}
for row in rows
]
finally:
conn.close()
def clear_dismissals_for_item_keys(
self,
*,
user_id: int,
item_type: str,
item_keys: Iterable[str],
) -> int:
"""Clear dismissals for one user + item type + item keys."""
normalized_user_id = self._coerce_positive_int(user_id, "user_id")
normalized_item_type = _normalize_item_type(item_type)
normalized_keys = {
_normalize_item_key(item_key)
for item_key in item_keys
if isinstance(item_key, str) and item_key.strip()
}
if not normalized_keys:
return 0
conn = self._connect()
try:
cursor = conn.executemany(
"""
DELETE FROM activity_dismissals
WHERE user_id = ? AND item_type = ? AND item_key = ?
""",
(
(normalized_user_id, normalized_item_type, item_key)
for item_key in normalized_keys
),
)
conn.commit()
return int(cursor.rowcount or 0)
finally:
conn.close()
def get_history(self, user_id: int, *, limit: int = 50, offset: int = 0) -> list[dict[str, Any]]:
"""Return paged dismissal history for one user."""
normalized_user_id = self._coerce_positive_int(user_id, "user_id")
normalized_limit = max(1, min(int(limit), 200))
normalized_offset = max(0, int(offset))
conn = self._connect()
try:
rows = conn.execute(
"""
SELECT
d.id,
d.user_id,
d.item_type,
d.item_key,
d.activity_log_id,
d.dismissed_at,
l.snapshot_json,
l.origin,
l.final_status,
l.terminal_at,
l.request_id,
l.source_id
FROM activity_dismissals d
LEFT JOIN activity_log l ON l.id = d.activity_log_id
WHERE d.user_id = ?
ORDER BY d.dismissed_at DESC, d.id DESC
LIMIT ? OFFSET ?
""",
(normalized_user_id, normalized_limit, normalized_offset),
).fetchall()
payload: list[dict[str, Any]] = []
for row in rows:
row_dict = dict(row)
raw_snapshot_json = row_dict.pop("snapshot_json", None)
snapshot_payload = None
if isinstance(raw_snapshot_json, str):
try:
snapshot_payload = json.loads(raw_snapshot_json)
except (ValueError, TypeError):
snapshot_payload = None
if snapshot_payload is None and row_dict.get("item_type") == "request":
request_id = row_dict.get("request_id")
if request_id is None:
request_id = _parse_request_id_from_item_key(row_dict.get("item_key"))
try:
normalized_request_id = int(request_id) if request_id is not None else None
except (TypeError, ValueError):
normalized_request_id = None
if normalized_request_id and normalized_request_id > 0:
fallback_snapshot, fallback_final_status = self._build_legacy_request_snapshot(
conn,
normalized_request_id,
)
if fallback_snapshot is not None:
snapshot_payload = fallback_snapshot
if not row_dict.get("origin"):
row_dict["origin"] = "request"
if not row_dict.get("final_status") and fallback_final_status is not None:
row_dict["final_status"] = fallback_final_status
row_dict["snapshot"] = snapshot_payload
payload.append(row_dict)
return payload
finally:
conn.close()
def get_undismissed_terminal_downloads(self, user_id: int, *, limit: int = 200) -> list[dict[str, Any]]:
"""Return latest undismissed terminal download snapshots for one user."""
normalized_user_id = self._coerce_positive_int(user_id, "user_id")
normalized_limit = max(1, min(int(limit), 500))
conn = self._connect()
try:
rows = conn.execute(
"""
SELECT
l.id,
l.user_id,
l.item_type,
l.item_key,
l.request_id,
l.source_id,
l.origin,
l.final_status,
l.snapshot_json,
l.terminal_at
FROM activity_log l
LEFT JOIN activity_dismissals d
ON d.user_id = ?
AND d.item_type = l.item_type
AND d.item_key = l.item_key
WHERE l.user_id = ?
AND l.item_type = 'download'
AND l.final_status IN ('complete', 'error', 'cancelled')
AND d.id IS NULL
ORDER BY l.terminal_at DESC, l.id DESC
LIMIT ?
""",
(normalized_user_id, normalized_user_id, normalized_limit * 2),
).fetchall()
payload: list[dict[str, Any]] = []
seen_item_keys: set[str] = set()
for row in rows:
row_dict = dict(row)
item_key = str(row_dict.get("item_key") or "")
if not item_key or item_key in seen_item_keys:
continue
seen_item_keys.add(item_key)
raw_snapshot_json = row_dict.pop("snapshot_json", None)
snapshot_payload = None
if isinstance(raw_snapshot_json, str):
try:
snapshot_payload = json.loads(raw_snapshot_json)
except (ValueError, TypeError):
snapshot_payload = None
row_dict["snapshot"] = snapshot_payload
payload.append(row_dict)
if len(payload) >= normalized_limit:
break
return payload
finally:
conn.close()
def clear_history(self, user_id: int) -> int:
"""Delete all dismissals for a user and return deleted row count."""
normalized_user_id = self._coerce_positive_int(user_id, "user_id")
conn = self._connect()
try:
cursor = conn.execute(
"DELETE FROM activity_dismissals WHERE user_id = ?",
(normalized_user_id,),
)
conn.commit()
return int(cursor.rowcount or 0)
finally:
conn.close()
@@ -0,0 +1,310 @@
"""Persistence helpers for per-viewer activity visibility state."""
from __future__ import annotations
import sqlite3
import threading
from typing import Any
from shelfmark.core.request_helpers import now_utc_iso
VALID_ACTIVITY_ITEM_TYPES = frozenset({"download", "request"})
ADMIN_VIEWER_SCOPE = "admin:shared"
NOAUTH_VIEWER_SCOPE = "noauth:shared"
USER_VIEWER_SCOPE_PREFIX = "user:"
def user_viewer_scope(user_id: int) -> str:
if not isinstance(user_id, int) or user_id < 1:
raise ValueError("user_id must be a positive integer")
return f"{USER_VIEWER_SCOPE_PREFIX}{user_id}"
def normalize_viewer_scope(viewer_scope: Any) -> str:
if not isinstance(viewer_scope, str) or not viewer_scope.strip():
raise ValueError("viewer_scope must be a non-empty string")
normalized = viewer_scope.strip()
if normalized in {ADMIN_VIEWER_SCOPE, NOAUTH_VIEWER_SCOPE}:
return normalized
if not normalized.startswith(USER_VIEWER_SCOPE_PREFIX):
raise ValueError(
"viewer_scope must be one of: admin:shared, noauth:shared, or user:<id>"
)
raw_user_id = normalized[len(USER_VIEWER_SCOPE_PREFIX):].strip()
try:
parsed_user_id = int(raw_user_id)
except (TypeError, ValueError) as exc:
raise ValueError("viewer_scope user id must be a positive integer") from exc
return user_viewer_scope(parsed_user_id)
def _normalize_item_type(item_type: Any) -> str:
if not isinstance(item_type, str) or not item_type.strip():
raise ValueError("item_type must be a non-empty string")
normalized = item_type.strip().lower()
if normalized not in VALID_ACTIVITY_ITEM_TYPES:
raise ValueError("item_type must be one of: download, request")
return normalized
def _normalize_item_key(item_key: Any, *, item_type: str) -> str:
if not isinstance(item_key, str) or not item_key.strip():
raise ValueError("item_key must be a non-empty string")
normalized = item_key.strip()
expected_prefix = f"{item_type}:"
if not normalized.startswith(expected_prefix):
raise ValueError(f"item_key must be in the format {expected_prefix}<id>")
if not normalized.split(":", 1)[1].strip():
raise ValueError(f"item_key must be in the format {expected_prefix}<id>")
return normalized
class ActivityViewStateService:
"""Service for per-viewer activity dismissal and history visibility."""
def __init__(self, db_path: str):
self._db_path = db_path
self._lock = threading.Lock()
def _connect(self) -> sqlite3.Connection:
conn = sqlite3.connect(self._db_path)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA foreign_keys = ON")
return conn
def list_hidden(
self,
*,
viewer_scope: str,
limit: int | None = None,
) -> list[dict[str, Any]]:
normalized_scope = normalize_viewer_scope(viewer_scope)
normalized_limit = None if limit is None else max(1, int(limit))
query = """
SELECT item_type, item_key, dismissed_at, cleared_at
FROM activity_view_state
WHERE viewer_scope = ?
AND dismissed_at IS NOT NULL
ORDER BY COALESCE(cleared_at, dismissed_at) DESC, id DESC
"""
params: list[Any] = [normalized_scope]
if normalized_limit is not None:
query += "\nLIMIT ?"
params.append(normalized_limit)
conn = self._connect()
try:
rows = conn.execute(query, params).fetchall()
return [dict(row) for row in rows]
finally:
conn.close()
def list_history(
self,
*,
viewer_scope: str,
limit: int = 50,
offset: int = 0,
) -> list[dict[str, Any]]:
normalized_scope = normalize_viewer_scope(viewer_scope)
normalized_limit = max(1, min(int(limit), 5000))
normalized_offset = max(0, int(offset))
conn = self._connect()
try:
rows = conn.execute(
"""
SELECT item_type, item_key, dismissed_at
FROM activity_view_state
WHERE viewer_scope = ?
AND dismissed_at IS NOT NULL
AND cleared_at IS NULL
ORDER BY dismissed_at DESC, id DESC
LIMIT ? OFFSET ?
""",
(normalized_scope, normalized_limit, normalized_offset),
).fetchall()
return [dict(row) for row in rows]
finally:
conn.close()
def dismiss(
self,
*,
viewer_scope: str,
item_type: str,
item_key: str,
) -> int:
normalized_scope = normalize_viewer_scope(viewer_scope)
normalized_type = _normalize_item_type(item_type)
normalized_key = _normalize_item_key(item_key, item_type=normalized_type)
dismissed_at = now_utc_iso()
with self._lock:
conn = self._connect()
try:
cursor = conn.execute(
"""
INSERT INTO activity_view_state (
viewer_scope,
item_type,
item_key,
dismissed_at,
cleared_at
)
VALUES (?, ?, ?, ?, NULL)
ON CONFLICT(viewer_scope, item_type, item_key) DO UPDATE SET
dismissed_at = excluded.dismissed_at,
cleared_at = NULL
""",
(normalized_scope, normalized_type, normalized_key, dismissed_at),
)
conn.commit()
rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
return max(rowcount, 0)
finally:
conn.close()
def dismiss_many(
self,
*,
viewer_scope: str,
items: list[dict[str, str]],
) -> int:
normalized_scope = normalize_viewer_scope(viewer_scope)
if not items:
return 0
seen: set[tuple[str, str]] = set()
normalized_items: list[tuple[str, str]] = []
for item in items:
normalized_type = _normalize_item_type(item.get("item_type"))
normalized_key = _normalize_item_key(item.get("item_key"), item_type=normalized_type)
marker = (normalized_type, normalized_key)
if marker in seen:
continue
seen.add(marker)
normalized_items.append(marker)
if not normalized_items:
return 0
dismissed_at = now_utc_iso()
with self._lock:
conn = self._connect()
try:
total = 0
for normalized_type, normalized_key in normalized_items:
cursor = conn.execute(
"""
INSERT INTO activity_view_state (
viewer_scope,
item_type,
item_key,
dismissed_at,
cleared_at
)
VALUES (?, ?, ?, ?, NULL)
ON CONFLICT(viewer_scope, item_type, item_key) DO UPDATE SET
dismissed_at = excluded.dismissed_at,
cleared_at = NULL
""",
(normalized_scope, normalized_type, normalized_key, dismissed_at),
)
rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
total += max(rowcount, 0)
conn.commit()
return total
finally:
conn.close()
def clear_history(self, *, viewer_scope: str) -> int:
normalized_scope = normalize_viewer_scope(viewer_scope)
cleared_at = now_utc_iso()
with self._lock:
conn = self._connect()
try:
cursor = conn.execute(
"""
UPDATE activity_view_state
SET cleared_at = ?
WHERE viewer_scope = ?
AND dismissed_at IS NOT NULL
AND cleared_at IS NULL
""",
(cleared_at, normalized_scope),
)
conn.commit()
rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
return max(rowcount, 0)
finally:
conn.close()
def clear_item_for_all_viewers(self, *, item_type: str, item_key: str) -> int:
normalized_type = _normalize_item_type(item_type)
normalized_key = _normalize_item_key(item_key, item_type=normalized_type)
with self._lock:
conn = self._connect()
try:
cursor = conn.execute(
"""
DELETE FROM activity_view_state
WHERE item_type = ? AND item_key = ?
""",
(normalized_type, normalized_key),
)
conn.commit()
rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
return max(rowcount, 0)
finally:
conn.close()
def delete_viewer_scope(self, *, viewer_scope: str) -> int:
normalized_scope = normalize_viewer_scope(viewer_scope)
with self._lock:
conn = self._connect()
try:
cursor = conn.execute(
"DELETE FROM activity_view_state WHERE viewer_scope = ?",
(normalized_scope,),
)
conn.commit()
rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
return max(rowcount, 0)
finally:
conn.close()
def delete_items(self, *, item_type: str, item_keys: list[str]) -> int:
normalized_type = _normalize_item_type(item_type)
normalized_keys = [
_normalize_item_key(item_key, item_type=normalized_type)
for item_key in item_keys
]
if not normalized_keys:
return 0
placeholders = ",".join("?" for _ in normalized_keys)
with self._lock:
conn = self._connect()
try:
cursor = conn.execute(
f"""
DELETE FROM activity_view_state
WHERE item_type = ? AND item_key IN ({placeholders})
""",
(normalized_type, *normalized_keys),
)
conn.commit()
rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
return max(rowcount, 0)
finally:
conn.close()
+35 -54
View File
@@ -9,7 +9,7 @@ import os
import sqlite3
from typing import Any
from flask import Flask, jsonify, request, session
from flask import Flask, g, jsonify, request, session
from werkzeug.security import generate_password_hash
from shelfmark.config.booklore_settings import (
@@ -26,8 +26,8 @@ from shelfmark.core.auth_modes import (
AUTH_SOURCE_CWA,
AUTH_SOURCE_OIDC,
AUTH_SOURCE_PROXY,
determine_auth_mode,
has_local_password_admin,
is_user_active_for_auth_mode,
load_active_auth_mode,
normalize_auth_source,
)
from shelfmark.core.cwa_user_sync import sync_cwa_users_from_rows
@@ -65,37 +65,6 @@ def _get_user_edit_capabilities(
}
def _get_auth_mode():
"""Get current auth mode from config."""
try:
config = load_config_file("security")
return determine_auth_mode(
config,
CWA_DB_PATH,
has_local_admin=has_local_password_admin(),
)
except Exception:
return "none"
def _require_admin(f):
"""Decorator to require admin session for admin routes.
In no-auth mode, everyone has access (is_admin defaults True).
In auth-required modes, requires an authenticated session with admin role.
"""
@wraps(f)
def decorated(*args, **kwargs):
auth_mode = _get_auth_mode()
if auth_mode != "none":
if "user_id" not in session:
return jsonify({"error": "Authentication required"}), 401
if not session.get("is_admin", False):
return jsonify({"error": "Admin access required"}), 403
return f(*args, **kwargs)
return decorated
def _sanitize_user(user: dict) -> dict:
"""Remove sensitive fields from user dict before returning to client."""
sanitized = dict(user)
@@ -116,14 +85,6 @@ def _oidc_role_management_message(security_config: dict[str, Any]) -> str:
)
def _is_user_active(user: dict[str, Any], auth_method: str) -> bool:
"""Determine whether a user can authenticate in the current auth mode."""
source = normalize_auth_source(user.get("auth_source"), user.get("oidc_subject"))
if source == AUTH_SOURCE_BUILTIN:
return auth_method in (AUTH_SOURCE_BUILTIN, AUTH_SOURCE_OIDC)
return source == auth_method
def _serialize_user(
user: dict[str, Any],
auth_method: str,
@@ -135,7 +96,7 @@ def _serialize_user(
payload.get("auth_source"),
payload.get("oidc_subject"),
)
payload["is_active"] = _is_user_active(payload, auth_method)
payload["is_active"] = is_user_active_for_auth_mode(payload, auth_method)
payload["edit_capabilities"] = _get_user_edit_capabilities(
payload,
security_config=security_config,
@@ -143,6 +104,8 @@ def _serialize_user(
return payload
def _sync_all_cwa_users(user_db: UserDB) -> dict[str, int]:
"""Sync all users from the Calibre-Web database into users.db."""
if not CWA_DB_PATH or not CWA_DB_PATH.exists():
@@ -164,12 +127,31 @@ def _sync_all_cwa_users(user_db: UserDB) -> dict[str, int]:
def register_admin_routes(app: Flask, user_db: UserDB) -> None:
"""Register admin user management routes on the Flask app."""
def _require_admin(f):
"""Decorator to require admin session for admin routes.
In no-auth mode, everyone has access (is_admin defaults True).
In auth-required modes, requires an authenticated session with admin role.
Caches the resolved auth_mode in ``g.auth_mode`` for the request.
"""
@wraps(f)
def decorated(*args, **kwargs):
auth_mode = load_active_auth_mode(CWA_DB_PATH, user_db=user_db)
g.auth_mode = auth_mode
if auth_mode != "none":
if "user_id" not in session:
return jsonify({"error": "Authentication required"}), 401
if not session.get("is_admin", False):
return jsonify({"error": "Admin access required"}), 403
return f(*args, **kwargs)
return decorated
@app.route("/api/admin/users", methods=["GET"])
@_require_admin
def admin_list_users():
"""List all users."""
users = user_db.list_users()
auth_mode = _get_auth_mode()
auth_mode = g.auth_mode
security_config = load_config_file("security")
return jsonify([
_serialize_user(u, auth_mode, security_config=security_config)
@@ -181,7 +163,7 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
def admin_create_user():
"""Create a new user with password authentication."""
data = request.get_json() or {}
auth_mode = _get_auth_mode()
auth_mode = g.auth_mode
username = (data.get("username") or "").strip()
password = data.get("password", "")
@@ -206,7 +188,8 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
return jsonify({"error": "Role must be 'admin' or 'user'"}), 400
# First user is always admin
if not user_db.list_users():
existing_users = user_db.list_users()
if not existing_users:
role = "admin"
# Check if username already exists
@@ -233,7 +216,7 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
return jsonify(
_serialize_user(
user,
_get_auth_mode(),
g.auth_mode,
security_config=load_config_file("security"),
)
), 201
@@ -248,7 +231,7 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
result = _serialize_user(
user,
_get_auth_mode(),
g.auth_mode,
security_config=load_config_file("security"),
)
result["settings"] = user_db.get_user_settings(user_id)
@@ -356,14 +339,14 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
# Ensure runtime reads see updated per-user overrides immediately.
try:
from shelfmark.core.config import config as app_config
app_config.refresh()
app_config.refresh(force=True)
except Exception:
pass
updated = user_db.get_user(user_id=user_id)
result = _serialize_user(
updated,
_get_auth_mode(),
g.auth_mode,
security_config=security_config,
)
result["settings"] = user_db.get_user_settings(user_id)
@@ -374,8 +357,7 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
@_require_admin
def admin_sync_cwa_users():
"""Manually sync users from Calibre-Web into users.db."""
auth_mode = _get_auth_mode()
if auth_mode != AUTH_SOURCE_CWA:
if g.auth_mode != AUTH_SOURCE_CWA:
return jsonify({
"error": "CWA sync is only available when CWA authentication is enabled",
}), 400
@@ -419,12 +401,11 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
if not user:
return jsonify({"error": "User not found"}), 404
auth_mode = _get_auth_mode()
auth_source = normalize_auth_source(
user.get("auth_source"),
user.get("oidc_subject"),
)
if auth_source == AUTH_SOURCE_CWA and auth_source == auth_mode:
if auth_source == AUTH_SOURCE_CWA and auth_source == g.auth_mode:
return jsonify({
"error": f"Cannot delete active {auth_source.upper()} users",
"message": f"{auth_source.upper()} users are automatically re-provisioned on login.",
+28
View File
@@ -9,6 +9,7 @@ from shelfmark.config.notifications_settings import (
is_valid_notification_url,
normalize_notification_routes,
)
from shelfmark.config.users_settings import validate_search_preference_value
from shelfmark.core.settings_registry import load_config_file
from shelfmark.core.user_settings_overrides import (
build_user_preferences_payload as _build_user_preferences_payload,
@@ -68,6 +69,19 @@ def validate_user_settings(settings: dict[str, Any]) -> tuple[dict[str, Any], li
valid[key] = normalized_routes
continue
normalized_search_value, search_validation_error = validate_search_preference_value(key, value)
if search_validation_error:
errors.append(search_validation_error)
continue
if key in {
"SEARCH_MODE",
"METADATA_PROVIDER",
"METADATA_PROVIDER_AUDIOBOOK",
"DEFAULT_RELEASE_SOURCE",
}:
valid[key] = normalized_search_value
continue
valid[key] = value
return valid, errors
@@ -136,6 +150,20 @@ def register_admin_settings_routes(
return jsonify(payload)
@app.route("/api/admin/users/<int:user_id>/search-preferences", methods=["GET"])
@require_admin
def admin_get_search_preferences(user_id):
user = user_db.get_user(user_id=user_id)
if not user:
return jsonify({"error": "User not found"}), 404
try:
payload = _build_user_preferences_payload(user_db, user_id, "search_mode")
except ValueError:
return jsonify({"error": "Search mode settings tab not found"}), 500
return jsonify(payload)
@app.route("/api/admin/users/<int:user_id>/notification-preferences", methods=["GET"])
@require_admin
def admin_get_notification_preferences(user_id):
+28 -4
View File
@@ -28,10 +28,7 @@ def has_local_password_admin(user_db: Any | None = None) -> bool:
db = UserDB(os.path.join(config_root, "users.db"))
db.initialize()
return any(
user.get("password_hash") and user.get("role") == "admin"
for user in db.list_users()
)
return db.has_admin_with_password()
except Exception:
return False
@@ -78,6 +75,33 @@ def determine_auth_mode(
return "none"
def load_active_auth_mode(
cwa_db_path: Any | None,
*,
user_db: Any | None = None,
) -> str:
"""Resolve active auth mode using current security config and runtime prerequisites."""
try:
from shelfmark.core.settings_registry import load_config_file
security_config = load_config_file("security")
return determine_auth_mode(
security_config,
cwa_db_path,
has_local_admin=has_local_password_admin(user_db),
)
except Exception:
return "none"
def is_user_active_for_auth_mode(user: Mapping[str, Any], auth_mode: str) -> bool:
"""Return whether a user can authenticate under the current auth mode."""
source = normalize_auth_source(user.get("auth_source"), user.get("oidc_subject"))
if source == AUTH_SOURCE_BUILTIN:
return auth_mode in (AUTH_SOURCE_BUILTIN, AUTH_SOURCE_OIDC)
return source == auth_mode
def is_settings_or_onboarding_path(path: str) -> bool:
"""Return True when request path targets protected admin settings routes."""
return path.startswith("/api/settings") or path.startswith("/api/onboarding")
+8
View File
@@ -62,6 +62,14 @@ class CacheService:
return True
return False
def invalidate_prefix(self, prefix: str) -> int:
"""Remove all cache entries whose keys start with prefix."""
with self._lock:
matching_keys = [key for key in self._cache if key.startswith(prefix)]
for key in matching_keys:
del self._cache[key]
return len(matching_keys)
def clear(self) -> None:
"""Clear all cache entries."""
with self._lock:
+13 -1
View File
@@ -2,6 +2,7 @@
import os
import sqlite3
import time
from threading import Lock
from typing import Any, Dict, Optional
@@ -68,6 +69,7 @@ class Config:
self._user_db_load_attempted = False
self._initialized = True
self._loaded = False
self._last_refresh_time: float = 0.0
def _ensure_loaded(self) -> None:
"""Ensure settings are loaded from the registry."""
@@ -117,13 +119,22 @@ class Config:
self._loaded = True
def refresh(self) -> None:
def refresh(self, force: bool = False) -> None:
"""
Refresh all cached settings from config files.
Call this after settings are updated via the UI to ensure
the config singleton reflects the new values.
Multiple calls within a short window (50 ms) are coalesced to
avoid redundant disk I/O when several helpers each call refresh()
during the same request. Pass ``force=True`` to bypass the guard
(e.g. after a settings write).
"""
now = time.monotonic()
if not force and (now - self._last_refresh_time) < 0.05:
return
with self._cache_lock:
self._loaded = False
self._load_settings()
@@ -131,6 +142,7 @@ class Config:
self._user_settings_cache.clear()
self._user_db = None
self._user_db_load_attempted = False
self._last_refresh_time = time.monotonic()
def _get_user_db(self):
"""Get or initialize a UserDB handle if available."""
+306
View File
@@ -0,0 +1,306 @@
"""Persistence helpers for canonical download activity rows."""
from __future__ import annotations
import os
import sqlite3
import threading
from datetime import datetime, timezone
from typing import Any
from shelfmark.core.logger import setup_logger
from shelfmark.core.models import TERMINAL_QUEUE_STATUSES
from shelfmark.core.request_helpers import normalize_optional_positive_int, normalize_optional_text, now_utc_iso
logger = setup_logger(__name__)
VALID_TERMINAL_STATUSES = frozenset(s.value for s in TERMINAL_QUEUE_STATUSES)
ACTIVE_DOWNLOAD_STATUS = "active"
VALID_ORIGINS = frozenset({"direct", "requested"})
def _normalize_task_id(task_id: Any) -> str:
normalized = normalize_optional_text(task_id)
if normalized is None:
raise ValueError("task_id must be a non-empty string")
return normalized
def _normalize_origin(origin: Any) -> str:
normalized = normalize_optional_text(origin)
if normalized is None:
return "direct"
lowered = normalized.lower()
if lowered not in VALID_ORIGINS:
raise ValueError("origin must be one of: direct, requested")
return lowered
def _normalize_final_status(final_status: Any) -> str:
normalized = normalize_optional_text(final_status)
if normalized is None:
raise ValueError("final_status must be a non-empty string")
lowered = normalized.lower()
if lowered not in VALID_TERMINAL_STATUSES:
raise ValueError("final_status must be one of: complete, error, cancelled")
return lowered
def _normalize_limit(value: Any, *, default: int, minimum: int, maximum: int) -> int:
if value is None:
return default
try:
parsed = int(value)
except (TypeError, ValueError) as exc:
raise ValueError("limit must be an integer") from exc
if parsed < minimum:
return minimum
if parsed > maximum:
return maximum
return parsed
class DownloadHistoryService:
"""Service for persisted canonical download activity rows."""
def __init__(self, db_path: str):
self._db_path = db_path
self._lock = threading.Lock()
def _connect(self) -> sqlite3.Connection:
conn = sqlite3.connect(self._db_path)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA foreign_keys = ON")
return conn
@staticmethod
def _row_to_dict(row: sqlite3.Row | None) -> dict[str, Any] | None:
return dict(row) if row is not None else None
@staticmethod
def _to_item_key(task_id: str) -> str:
return f"download:{task_id}"
@staticmethod
def _resolve_existing_download_path(value: Any) -> str | None:
normalized = normalize_optional_text(value)
if normalized is None:
return None
return normalized if os.path.exists(normalized) else None
@staticmethod
def to_download_payload(row: dict[str, Any]) -> dict[str, Any]:
return {
"id": row.get("task_id"),
"title": row.get("title"),
"author": row.get("author"),
"format": row.get("format"),
"size": row.get("size"),
"preview": row.get("preview"),
"content_type": row.get("content_type"),
"source": row.get("source"),
"source_display_name": row.get("source_display_name"),
"status_message": row.get("status_message"),
"download_path": DownloadHistoryService._resolve_existing_download_path(row.get("download_path")),
"added_time": DownloadHistoryService._iso_to_epoch(row.get("queued_at")),
"user_id": row.get("user_id"),
"username": row.get("username"),
"request_id": row.get("request_id"),
}
@staticmethod
def _iso_to_epoch(value: Any) -> float | None:
if not isinstance(value, str) or not value.strip():
return None
normalized = value.strip().replace("Z", "+00:00")
try:
parsed = datetime.fromisoformat(normalized)
except ValueError:
return None
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=timezone.utc)
return parsed.timestamp()
@classmethod
def to_history_row(cls, row: dict[str, Any], *, dismissed_at: str) -> dict[str, Any]:
task_id = str(row.get("task_id") or "").strip()
item_key = cls._to_item_key(task_id)
download_payload = cls.to_download_payload(row)
# Clear stale progress messages for non-error terminal states.
if row.get("final_status") in ("complete", "cancelled"):
download_payload["status_message"] = None
return {
"id": item_key,
"user_id": row.get("user_id"),
"item_type": "download",
"item_key": item_key,
"dismissed_at": dismissed_at,
"snapshot": {
"kind": "download",
"download": download_payload,
},
"origin": row.get("origin"),
"final_status": row.get("final_status"),
"terminal_at": row.get("terminal_at"),
"request_id": row.get("request_id"),
"source_id": task_id or None,
}
def record_download(
self,
*,
task_id: str,
user_id: int | None,
username: str | None,
request_id: int | None,
source: str,
source_display_name: str | None,
title: str,
author: str | None,
format: str | None,
size: str | None,
preview: str | None,
content_type: str | None,
origin: str,
) -> None:
"""Record a download at queue time with final_status='active'.
On first queue: inserts a new row.
On retry (row already exists): resets the row back to 'active'
so the normal finalize path works when the retry completes.
"""
normalized_task_id = _normalize_task_id(task_id)
normalized_user_id = normalize_optional_positive_int(user_id, "user_id")
normalized_request_id = normalize_optional_positive_int(request_id, "request_id")
normalized_source = normalize_optional_text(source)
if normalized_source is None:
raise ValueError("source must be a non-empty string")
normalized_title = normalize_optional_text(title)
if normalized_title is None:
raise ValueError("title must be a non-empty string")
normalized_origin = _normalize_origin(origin)
recorded_at = now_utc_iso()
with self._lock:
conn = self._connect()
try:
conn.execute(
"""
INSERT INTO download_history (
task_id, user_id, username, request_id,
source, source_display_name,
title, author, format, size, preview, content_type,
origin, final_status,
status_message, download_path,
queued_at, terminal_at
)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'active', NULL, NULL, ?, ?)
ON CONFLICT(task_id) DO UPDATE SET
final_status = 'active',
status_message = NULL,
download_path = NULL,
terminal_at = ?
""",
(
normalized_task_id,
normalized_user_id,
normalize_optional_text(username),
normalized_request_id,
normalized_source,
normalize_optional_text(source_display_name),
normalized_title,
normalize_optional_text(author),
normalize_optional_text(format),
normalize_optional_text(size),
normalize_optional_text(preview),
normalize_optional_text(content_type),
normalized_origin,
recorded_at,
recorded_at,
recorded_at,
),
)
conn.commit()
finally:
conn.close()
def finalize_download(
self,
*,
task_id: str,
final_status: str,
status_message: str | None = None,
download_path: str | None = None,
) -> None:
"""Update an existing download row to its terminal state."""
normalized_task_id = _normalize_task_id(task_id)
normalized_final_status = _normalize_final_status(final_status)
normalized_status_message = normalize_optional_text(status_message)
normalized_download_path = normalize_optional_text(download_path)
effective_terminal_at = now_utc_iso()
with self._lock:
conn = self._connect()
try:
cursor = conn.execute(
"""
UPDATE download_history
SET final_status = ?,
status_message = ?,
download_path = ?,
terminal_at = ?
WHERE task_id = ? AND final_status = 'active'
""",
(
normalized_final_status,
normalized_status_message,
normalized_download_path,
effective_terminal_at,
normalized_task_id,
),
)
rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
if rowcount < 1:
logger.warning(
"finalize_download: no active row found for task_id=%s (may have been missed at queue time)",
normalized_task_id,
)
conn.commit()
finally:
conn.close()
def get_by_task_id(self, task_id: str) -> dict[str, Any] | None:
normalized_task_id = _normalize_task_id(task_id)
conn = self._connect()
try:
row = conn.execute(
"SELECT * FROM download_history WHERE task_id = ?",
(normalized_task_id,),
).fetchone()
return self._row_to_dict(row)
finally:
conn.close()
def list_recent(
self,
*,
user_id: int | None,
limit: int = 200,
) -> list[dict[str, Any]]:
normalized_user_id = normalize_optional_positive_int(user_id, "user_id")
normalized_limit = _normalize_limit(limit, default=200, minimum=1, maximum=1000)
query = "SELECT * FROM download_history"
params: list[Any] = []
if normalized_user_id is not None:
query += " WHERE user_id = ?"
params.append(normalized_user_id)
query += " ORDER BY terminal_at DESC, id DESC LIMIT ?"
params.append(normalized_limit)
conn = self._connect()
try:
rows = conn.execute(query, params).fetchall()
return [dict(row) for row in rows]
finally:
conn.close()
+2
View File
@@ -11,6 +11,7 @@ from typing import Any, Dict, Optional, Tuple
import requests
from shelfmark.core.logger import setup_logger
from shelfmark.download.network import get_ssl_verify
logger = setup_logger(__name__)
@@ -482,6 +483,7 @@ class ImageCacheService:
timeout=(5, 10),
headers=FETCH_HEADERS,
stream=True,
verify=get_ssl_verify(url),
)
response.raise_for_status()
+3 -1
View File
@@ -20,7 +20,9 @@ def _get_config():
# Default mirror lists (hardcoded fallbacks)
DEFAULT_AA_MIRRORS = [
"https://annas-archive.gl",
"https://annas-archive.li",
"https://annas-archive.pk",
"https://annas-archive.vg",
"https://annas-archive.gd",
]
DEFAULT_LIBGEN_MIRRORS = [
+12 -51
View File
@@ -38,12 +38,19 @@ class QueueStatus(str, Enum):
LOCATING = "locating"
DOWNLOADING = "downloading"
COMPLETE = "complete"
AVAILABLE = "available"
ERROR = "error"
DONE = "done"
CANCELLED = "cancelled"
TERMINAL_QUEUE_STATUSES: frozenset[QueueStatus] = frozenset({
QueueStatus.COMPLETE, QueueStatus.ERROR, QueueStatus.CANCELLED,
})
ACTIVE_QUEUE_STATUSES: frozenset[QueueStatus] = frozenset({
QueueStatus.QUEUED, QueueStatus.RESOLVING, QueueStatus.LOCATING, QueueStatus.DOWNLOADING,
})
class SearchMode(str, Enum):
DIRECT = "direct"
UNIVERSAL = "universal"
@@ -107,6 +114,9 @@ class DownloadTask:
status: QueueStatus = QueueStatus.QUEUED
status_message: Optional[str] = None
download_path: Optional[str] = None
last_error_message: Optional[str] = None
last_error_type: Optional[str] = None
staged_path: Optional[str] = None
def __lt__(self, other):
"""Compare tasks for priority queue (lower priority number = higher precedence)."""
@@ -121,55 +131,6 @@ class DownloadTask:
return build_filename(self.title, self.author, self.year, self.format)
@dataclass
class BookInfo:
"""Data class representing book information."""
id: str
title: str
preview: Optional[str] = None
author: Optional[str] = None
publisher: Optional[str] = None
year: Optional[str] = None
language: Optional[str] = None
content: Optional[str] = None
format: Optional[str] = None
size: Optional[str] = None
info: Optional[Dict[str, List[str]]] = None
description: Optional[str] = None
download_urls: List[str] = field(default_factory=list)
download_path: Optional[str] = None
priority: int = 0
progress: Optional[float] = None
status_message: Optional[str] = None # Detailed status message for UI display
added_time: Optional[float] = None # Timestamp when added to queue
source: str = "direct_download" # Release source handler to use for downloads
source_url: Optional[str] = None # Link to source page (e.g., Anna's Archive)
def get_filename(self, fallback_url: Optional[str] = None) -> str:
"""Build sanitized filename: 'Author - Title (Year).format'
Resolves format from self.format, download_urls, or fallback_url.
Args:
fallback_url: URL to extract format from if not already known
Returns:
Sanitized filename safe for filesystem use
"""
# Resolve format if needed
if not self.format:
urls = [self.download_urls[0]] if self.download_urls else []
if fallback_url:
urls.append(fallback_url)
for url in urls:
ext = url.split(".")[-1].lower()
if ext and len(ext) <= 5 and ext.isalnum():
self.format = ext
break
return build_filename(self.title, self.author, self.year, self.format)
@dataclass
class SearchFilters:
"""Filters for book search queries."""
+1
View File
@@ -14,6 +14,7 @@ logger = setup_logger(__name__)
# e.g., "SeriesPosition" must match before "Series"
KNOWN_TOKENS = [
'seriesposition',
'originalname',
'partnumber',
'subtitle',
'author',
+258 -25
View File
@@ -2,10 +2,14 @@
from __future__ import annotations
import logging
import threading
from concurrent.futures import ThreadPoolExecutor
from contextlib import contextmanager
from dataclasses import dataclass
from enum import Enum
from typing import Any, Iterable
from typing import Any, Iterable, Iterator
from urllib.parse import urlsplit
try:
import apprise
@@ -25,6 +29,7 @@ _APPRISE_APP_DESC = "Shelfmark notifications"
_APPRISE_LOGO_URL = (
"https://raw.githubusercontent.com/calibrain/shelfmark/main/src/frontend/public/logo.png"
)
_APPRISE_LOGGER_NAME = "apprise"
class NotificationEvent(str, Enum):
@@ -69,6 +74,13 @@ def _normalize_urls(value: Any) -> list[str]:
seen: set[str] = set()
for raw_url in raw_values:
url = str(raw_url or "").strip()
if not url:
continue
# Strip invisible/non-ASCII characters that can sneak in via copy-paste
# (zero-width spaces, smart quotes, non-breaking spaces, etc.).
# These pass Apprise URL validation but cause UnicodeEncodeError when
# requests tries to latin-1 encode credentials for Basic Auth headers.
url = url.encode("ascii", errors="ignore").decode("ascii").strip()
if not url:
continue
if url in seen:
@@ -78,6 +90,113 @@ def _normalize_urls(value: Any) -> list[str]:
return normalized
def _extract_url_schemes(urls: Iterable[str]) -> list[str]:
schemes: list[str] = []
seen: set[str] = set()
for raw_url in urls:
scheme = urlsplit(str(raw_url or "")).scheme.lower()
if not scheme or scheme in seen:
continue
seen.add(scheme)
schemes.append(scheme)
return schemes
class _AppriseLogCapture(logging.Handler):
def __init__(self, *, thread_id: int):
super().__init__(level=logging.INFO)
self.records: list[tuple[int, str, str, str]] = []
self._thread_id = thread_id
def emit(self, record: logging.LogRecord) -> None:
if record.thread != self._thread_id:
return
message = record.getMessage()
if message:
exception_summary = ""
if record.exc_info and record.exc_info[0]:
exc_type = getattr(record.exc_info[0], "__name__", "Exception")
exc = record.exc_info[1]
exception_summary = f"{exc_type}: {exc}"
elif record.exc_text:
exception_summary = str(record.exc_text).strip()
self.records.append((record.levelno, record.name, str(message), exception_summary))
@contextmanager
def _capture_apprise_logs(*, min_level: int = logging.INFO) -> Iterator[list[tuple[int, str, str, str]]]:
apprise_logger = logging.getLogger(_APPRISE_LOGGER_NAME)
previous_level = apprise_logger.level
handler = _AppriseLogCapture(thread_id=threading.get_ident())
apprise_logger.addHandler(handler)
if previous_level == logging.NOTSET or previous_level > min_level:
apprise_logger.setLevel(min_level)
try:
yield handler.records
finally:
apprise_logger.removeHandler(handler)
apprise_logger.setLevel(previous_level)
def _log_apprise_records(records: Iterable[tuple[int, str, str, str]]) -> None:
seen: set[tuple[int, str, str, str]] = set()
for level, source, raw_message, raw_exception_summary in records:
message = str(raw_message or "").strip()
source_name = str(source or "").strip() or _APPRISE_LOGGER_NAME
exception_summary = str(raw_exception_summary or "").strip()
key = (int(level), source_name, message, exception_summary)
if not message or key in seen:
continue
seen.add(key)
full_message = message if not exception_summary else f"{message} ({exception_summary})"
if level >= logging.ERROR:
logger.error("Apprise source [%s]: %s", source_name, full_message)
elif level >= logging.WARNING:
logger.warning("Apprise source [%s]: %s", source_name, full_message)
else:
logger.info("Apprise source [%s]: %s", source_name, full_message)
def _log_apprise_exception_debug(*, action: str, scheme: str, exc: Exception) -> None:
logger.debug(
"Apprise %s raised %s for scheme '%s': %s",
action,
type(exc).__name__,
scheme,
exc,
exc_info=True,
)
def _build_apprise_warning_detail(
records: Iterable[tuple[int, str, str, str]],
*,
scheme: str,
) -> str | None:
for level, source, raw_message, raw_exception_summary in records:
if level < logging.WARNING:
continue
message = str(raw_message or "").strip()
if not message:
continue
source_name = str(source or "").strip()
exception_summary = str(raw_exception_summary or "").strip()
full_message = message if not exception_summary else f"{message} ({exception_summary})"
if source_name and source_name != _APPRISE_LOGGER_NAME:
return f"{scheme}: {source_name}: {full_message}"
return f"{scheme}: {full_message}"
return None
def _normalize_routes(value: Any) -> list[dict[str, str]]:
if not isinstance(value, list):
return []
@@ -220,6 +339,31 @@ def _render_message(context: NotificationContext) -> tuple[str, str]:
return "Download Failed", f'Failed to download "{title}" by {author}.{error_line}'
def _plugin_label(plugin: Any, fallback_scheme: str) -> str:
"""Build a human-readable label from a validated Apprise plugin.
Combines the URL scheme with the plugin's service name (app_id) and
privacy-safe URL for richer diagnostics, e.g.
``"slack (Slack - slack://TokenA/To...n/To...n/)"``
"""
parts: list[str] = [fallback_scheme]
app_id = getattr(plugin, "app_id", None)
if app_id and str(app_id) != fallback_scheme:
privacy_url: str | None = None
try:
privacy_url = plugin.url(privacy=True)
except Exception:
pass
suffix = str(app_id)
if privacy_url:
suffix = f"{suffix} - {privacy_url}"
parts.append(f"({suffix})")
return " ".join(parts)
def _dispatch_to_apprise(
urls: Iterable[str],
*,
@@ -228,45 +372,134 @@ def _dispatch_to_apprise(
notify_type: Any,
) -> dict[str, Any]:
normalized_urls = _normalize_urls(list(urls))
url_schemes = _extract_url_schemes(normalized_urls)
if not normalized_urls:
return {"success": False, "message": "No notification URLs configured"}
if apprise is None:
return {"success": False, "message": "Apprise is not installed"}
apobj = _create_apprise_client()
if apobj is None:
return {"success": False, "message": "Apprise is not installed"}
valid_urls = 0
invalid_urls = 0
for url in normalized_urls:
try:
added = bool(apobj.add(url))
except Exception:
added = False
if added:
valid_urls += 1
else:
invalid_urls += 1
delivered_urls = 0
failed_delivery_urls = 0
failure_details: list[str] = []
for url in normalized_urls:
scheme = urlsplit(url).scheme or "unknown"
apobj = _create_apprise_client()
if apobj is None:
return {"success": False, "message": "Apprise is not installed"}
registration_failure_detail: str | None = None
with _capture_apprise_logs(min_level=logging.INFO) as apprise_records:
try:
plugin = apprise.Apprise.instantiate(url, asset=getattr(apobj, "asset", None))
except Exception as exc:
logger.warning(
"Failed to register notification route URL for scheme '%s': %s",
scheme,
exc,
)
_log_apprise_exception_debug(
action="route registration",
scheme=scheme,
exc=exc,
)
registration_failure_detail = (
f"{scheme}: route registration failed ({type(exc).__name__}: {exc})"
)
failure_details.append(registration_failure_detail)
plugin = None
if plugin is None:
invalid_urls += 1
logger.warning("Apprise rejected notification route URL for scheme '%s'", scheme)
_log_apprise_records(apprise_records)
warning_detail = _build_apprise_warning_detail(apprise_records, scheme=scheme)
if warning_detail:
failure_details.append(warning_detail)
elif registration_failure_detail is None:
failure_details.append(f"{scheme}: route URL rejected by Apprise")
continue
plugin_label = _plugin_label(plugin, scheme)
apobj.add(plugin)
valid_urls += 1
try:
delivered = bool(apobj.notify(title=title, body=body, notify_type=notify_type))
except Exception as exc:
_log_apprise_records(apprise_records)
failed_delivery_urls += 1
logger.warning(
"Apprise notify raised %s for %s: %s",
type(exc).__name__,
plugin_label,
exc,
)
_log_apprise_exception_debug(action="notify", scheme=scheme, exc=exc)
warning_detail = _build_apprise_warning_detail(apprise_records, scheme=scheme)
if warning_detail:
failure_details.append(warning_detail)
else:
failure_details.append(
f"{scheme}: notify raised {type(exc).__name__}: {exc}"
)
continue
_log_apprise_records(apprise_records)
if delivered:
delivered_urls += 1
logger.debug("Notification delivered via %s", plugin_label)
continue
failed_delivery_urls += 1
logger.warning("Apprise notify returned False for %s", plugin_label)
warning_detail = _build_apprise_warning_detail(apprise_records, scheme=scheme)
if warning_detail:
failure_details.append(warning_detail)
else:
failure_details.append(f"{scheme}: delivery failed")
scheme_summary = ", ".join(url_schemes) if url_schemes else "unknown"
if valid_urls == 0:
return {
logger.warning(
"No valid Apprise notification routes after registration for scheme(s): %s",
scheme_summary,
)
result: dict[str, Any] = {
"success": False,
"message": "No valid notification URLs configured",
}
if failure_details:
result["details"] = failure_details
return result
try:
delivered = bool(apobj.notify(title=title, body=body, notify_type=notify_type))
except Exception as exc:
return {"success": False, "message": f"Notification send failed: {type(exc).__name__}: {exc}"}
if delivered_urls == 0:
logger.warning(
(
"Apprise notify returned False for scheme(s): %s "
"(valid_urls=%s invalid_urls=%s failed_deliveries=%s)"
),
scheme_summary,
valid_urls,
invalid_urls,
failed_delivery_urls,
)
result = {"success": False, "message": "Notification delivery failed"}
if failure_details:
result["details"] = failure_details
return result
if not delivered:
return {"success": False, "message": "Notification delivery failed"}
message = f"Notification sent to {valid_urls} URL(s)"
if invalid_urls:
message += f" ({invalid_urls} invalid URL(s) skipped)"
return {"success": True, "message": message}
message = f"Notification sent to {delivered_urls} URL(s)"
failed_urls = invalid_urls + failed_delivery_urls
if failed_urls:
message += f" ({failed_urls} URL(s) failed)"
result = {"success": True, "message": message}
if failure_details:
result["details"] = failure_details
return result
def _create_apprise_client() -> Any:
+41 -28
View File
@@ -5,6 +5,7 @@ Business logic remains in oidc_auth.py.
"""
from typing import Any
from urllib.parse import quote
from authlib.jose.errors import InvalidClaimError
from authlib.integrations.flask_client import OAuth
@@ -18,6 +19,7 @@ from shelfmark.core.oidc_auth import (
)
from shelfmark.core.settings_registry import load_config_file
from shelfmark.core.user_db import UserDB
from shelfmark.download.network import get_ssl_verify
logger = setup_logger(__name__)
oauth = OAuth()
@@ -37,16 +39,22 @@ def _normalize_claims(raw_claims: Any) -> dict[str, Any]:
return {}
def _is_email_verified(claims: dict[str, Any]) -> bool:
"""Normalize provider-specific email_verified values into a strict boolean."""
value = claims.get("email_verified", False)
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.strip().lower() == "true"
def _has_username_or_email(claims: dict[str, Any]) -> bool:
"""Return True when claims include a usable username or email."""
for key in ("preferred_username", "email"):
value = claims.get(key)
if isinstance(value, str) and value.strip():
return True
return False
def _login_error_url(message: str) -> str:
"""Build a login URL (with script_root) that includes an OIDC error message."""
script_root = request.script_root.rstrip("/")
login_url = f"{script_root}/login" if script_root else "/login"
return f"{login_url}?oidc_error={quote(message)}"
def _get_oidc_client() -> tuple[Any, dict[str, Any]]:
"""Register and return an OIDC client from the current security config."""
config = load_config_file("security")
@@ -73,6 +81,11 @@ def _get_oidc_client() -> tuple[Any, dict[str, Any]]:
if admin_group and use_admin_group and group_claim and group_claim not in scopes:
scopes.append(group_claim)
def _ssl_compliance_fix(session, **kwargs):
"""Set session.verify based on the Certificate Validation setting."""
session.verify = get_ssl_verify(discovery_url)
return session
oauth._clients.pop("shelfmark_idp", None)
oauth.register(
name="shelfmark_idp",
@@ -83,6 +96,7 @@ def _get_oidc_client() -> tuple[Any, dict[str, Any]]:
"scope": " ".join(scopes),
"code_challenge_method": "S256",
},
compliance_fix=_ssl_compliance_fix,
overwrite=True,
)
@@ -117,7 +131,7 @@ def register_oidc_routes(app: Flask, user_db: UserDB) -> None:
error = request.args.get("error")
if error:
logger.warning(f"OIDC callback error from IdP: {error}")
return jsonify({"error": "Authentication failed"}), 400
return redirect(_login_error_url("Authentication failed"))
client, config = _get_oidc_client()
try:
@@ -141,32 +155,31 @@ def register_oidc_routes(app: Flask, user_db: UserDB) -> None:
provider_issuer or "<unknown>",
)
if claim_name == "iss":
return (
jsonify(
{
"error": (
"OIDC issuer validation failed. Verify your discovery URL and IdP issuer/"
"external URL configuration."
)
}
),
400,
msg = (
"OIDC issuer validation failed. Verify your discovery URL and IdP issuer/"
"external URL configuration."
)
return redirect(_login_error_url(msg))
return jsonify({"error": f"OIDC token claim validation failed: {claim_name}"}), 400
return redirect(_login_error_url(f"OIDC token claim validation failed: {claim_name}"))
claims = _normalize_claims(token.get("userinfo"))
# If userinfo isn't present in token payload, request it explicitly.
if not claims:
# If userinfo is missing or claims are too sparse, request it explicitly.
if not claims or not _has_username_or_email(claims):
fetched_claims: dict[str, Any] = {}
try:
claims = _normalize_claims(client.userinfo(token=token))
fetched_claims = _normalize_claims(client.userinfo(token=token))
except TypeError:
claims = _normalize_claims(client.userinfo())
fetched_claims = _normalize_claims(client.userinfo())
except Exception as e:
logger.error(f"Failed to fetch OIDC userinfo: {e}")
if fetched_claims:
claims = {**claims, **fetched_claims}
if not claims:
raise ValueError("OIDC authentication failed: missing user claims")
msg = "OIDC authentication failed: missing user claims"
logger.error(msg)
return redirect(_login_error_url(msg))
group_claim = config.get("OIDC_GROUP_CLAIM", "groups")
admin_group = config.get("OIDC_ADMIN_GROUP", "")
@@ -180,7 +193,7 @@ def register_oidc_routes(app: Flask, user_db: UserDB) -> None:
if admin_group and use_admin_group:
is_admin = admin_group in groups
allow_email_link = bool(user_info.get("email")) and _is_email_verified(claims)
allow_email_link = bool(user_info.get("email"))
user = provision_oidc_user(
user_db,
user_info,
@@ -192,7 +205,7 @@ def register_oidc_routes(app: Flask, user_db: UserDB) -> None:
logger.warning(
f"OIDC login rejected: auto-provision disabled for {user_info['username']}"
)
return jsonify({"error": "Account not found. Contact your administrator."}), 403
return redirect(_login_error_url("Account not found. Contact your administrator."))
session["user_id"] = user["username"]
session["is_admin"] = user.get("role") == "admin"
@@ -204,7 +217,7 @@ def register_oidc_routes(app: Flask, user_db: UserDB) -> None:
except ValueError as e:
logger.error(f"OIDC callback error: {e}")
return jsonify({"error": str(e)}), 400
return redirect(_login_error_url(str(e)))
except Exception as e:
logger.error(f"OIDC callback error: {e}")
return jsonify({"error": "Authentication failed"}), 500
return redirect(_login_error_url("Authentication failed"))
+86 -70
View File
@@ -8,7 +8,10 @@ from threading import Lock, Event
from typing import Dict, List, Optional, Tuple, Any, Callable
from shelfmark.core.config import config as app_config
from shelfmark.core.models import QueueStatus, QueueItem, DownloadTask
from shelfmark.core.logger import setup_logger
from shelfmark.core.models import QueueStatus, QueueItem, DownloadTask, TERMINAL_QUEUE_STATUSES
logger = setup_logger(__name__)
class BookQueue:
@@ -25,6 +28,7 @@ class BookQueue:
self._terminal_status_hook: Optional[
Callable[[str, QueueStatus, DownloadTask], None]
] = None
self._queue_hook: Optional[Callable[[str, DownloadTask], None]] = None
@property
def _status_timeout(self) -> timedelta:
@@ -33,11 +37,12 @@ class BookQueue:
def add(self, task: DownloadTask) -> bool:
"""Add a download task to the queue. Returns False if already exists."""
hook: Optional[Callable[[str, DownloadTask], None]] = None
with self._lock:
task_id = task.task_id
# Don't add if already exists and not in error/done state
if task_id in self._status and self._status[task_id] not in [QueueStatus.ERROR, QueueStatus.DONE, QueueStatus.CANCELLED]:
# Don't add if already exists and not in error/cancelled state
if task_id in self._status and self._status[task_id] not in [QueueStatus.ERROR, QueueStatus.CANCELLED]:
return False
# Ensure added_time is set
@@ -48,7 +53,14 @@ class BookQueue:
self._queue.put(queue_item)
self._task_data[task_id] = task
self._update_status(task_id, QueueStatus.QUEUED)
return True
hook = self._queue_hook
if hook is not None:
try:
hook(task_id, task)
except Exception as exc:
logger.warning("Queue hook failed while adding task %s: %s", task_id, exc)
return True
def get_next(self) -> Optional[Tuple[str, Event]]:
"""Get next task ID from queue with cancellation flag."""
@@ -77,6 +89,11 @@ class BookQueue:
with self._lock:
return self._task_data.get(task_id)
def get_task_status(self, task_id: str) -> Optional[QueueStatus]:
"""Get queue status for a task id."""
with self._lock:
return self._status.get(task_id)
def _update_status(self, book_id: str, status: QueueStatus) -> None:
"""Internal method to update status and timestamp."""
self._status[book_id] = status
@@ -90,6 +107,14 @@ class BookQueue:
with self._lock:
self._terminal_status_hook = hook
def set_queue_hook(
self,
hook: Optional[Callable[[str, DownloadTask], None]],
) -> None:
"""Register a callback invoked when a task is added to the queue."""
with self._lock:
self._queue_hook = hook
def update_status(self, book_id: str, status: QueueStatus) -> None:
"""Update status of a book in the queue."""
hook: Optional[Callable[[str, QueueStatus, DownloadTask], None]] = None
@@ -98,15 +123,8 @@ class BookQueue:
previous_status = self._status.get(book_id)
self._update_status(book_id, status)
terminal_statuses = {
QueueStatus.COMPLETE,
QueueStatus.AVAILABLE,
QueueStatus.ERROR,
QueueStatus.DONE,
QueueStatus.CANCELLED,
}
if (
status in terminal_statuses
status in TERMINAL_QUEUE_STATUSES
and previous_status != status
and self._terminal_status_hook is not None
):
@@ -116,7 +134,7 @@ class BookQueue:
hook_task = current_task
# Clean up active download tracking when finished
if status in [QueueStatus.COMPLETE, QueueStatus.AVAILABLE, QueueStatus.ERROR, QueueStatus.DONE, QueueStatus.CANCELLED]:
if status in TERMINAL_QUEUE_STATUSES:
self._active_downloads.pop(book_id, None)
self._cancel_flags.pop(book_id, None)
@@ -145,8 +163,8 @@ class BookQueue:
"""Get current queue status grouped by status.
Args:
user_id: If provided, only return tasks belonging to this user
(plus legacy tasks with no user_id). If None, return all.
user_id: If provided, only return tasks belonging to this user.
If None, return all.
"""
self.refresh()
with self._lock:
@@ -154,7 +172,7 @@ class BookQueue:
for task_id, status in self._status.items():
if task_id in self._task_data:
task = self._task_data[task_id]
if user_id is not None and task.user_id is not None and task.user_id != user_id:
if user_id is not None and task.user_id != user_id:
continue
result[status][task_id] = task
return result
@@ -191,29 +209,20 @@ class BookQueue:
return sorted(queue_items, key=lambda x: (x['priority'], x['added_time']))
def cancel_download(self, task_id: str) -> bool:
"""Cancel a download or clear a completed/errored item."""
"""Cancel an active or queued download."""
with self._lock:
current_status = self._status.get(task_id)
# Allow cancellation during any active state
if current_status in [QueueStatus.RESOLVING, QueueStatus.LOCATING, QueueStatus.DOWNLOADING]:
# Signal active download to stop
if task_id in self._cancel_flags:
self._cancel_flags[task_id].set()
if current_status in [QueueStatus.COMPLETE, QueueStatus.DONE, QueueStatus.AVAILABLE, QueueStatus.ERROR, QueueStatus.CANCELLED]:
# Clear completed/errored/cancelled items from tracking
self._status.pop(task_id, None)
self._status_timestamps.pop(task_id, None)
self._task_data.pop(task_id, None)
self._cancel_flags.pop(task_id, None)
self._active_downloads.pop(task_id, None)
return True
elif current_status not in [QueueStatus.QUEUED]:
# Not in a cancellable state
return False
if current_status in [QueueStatus.RESOLVING, QueueStatus.LOCATING, QueueStatus.DOWNLOADING, QueueStatus.QUEUED]:
self.update_status(task_id, QueueStatus.CANCELLED)
return True
return False
self.update_status(task_id, QueueStatus.CANCELLED)
return True
def set_priority(self, task_id: str, new_priority: int) -> bool:
"""Change the priority of a queued task (lower = higher priority)."""
@@ -247,6 +256,51 @@ class BookQueue:
return found
def enqueue_existing(self, task_id: str, *, priority: Optional[int] = None) -> bool:
"""Requeue an existing task regardless of current status.
This is used for retries where task metadata should be preserved.
"""
hook: Optional[Callable[[str, DownloadTask], None]] = None
hook_task: Optional[DownloadTask] = None
with self._lock:
task = self._task_data.get(task_id)
if task is None:
return False
if priority is not None:
task.priority = priority
# Ensure task doesn't appear active while waiting for retry.
self._active_downloads.pop(task_id, None)
self._cancel_flags.pop(task_id, None)
# De-duplicate queue entries for this task id.
temp_items: list[QueueItem] = []
while not self._queue.empty():
try:
item = self._queue.get_nowait()
except queue.Empty:
break
if item.book_id != task_id:
temp_items.append(item)
for item in temp_items:
self._queue.put(item)
queue_item = QueueItem(task_id, task.priority, time.time())
self._queue.put(queue_item)
self._update_status(task_id, QueueStatus.QUEUED)
hook = self._queue_hook
hook_task = task
if hook is not None and hook_task is not None:
try:
hook(task_id, hook_task)
except Exception as exc:
logger.warning("Queue hook failed while requeueing task %s: %s", task_id, exc)
return True
def reorder_queue(self, task_priorities: Dict[str, int]) -> bool:
"""Bulk reorder queue by mapping task_id to new priority."""
with self._lock:
@@ -285,43 +339,9 @@ class BookQueue:
return True
return any(status == QueueStatus.QUEUED for status in self._status.values())
def clear_completed(self, user_id: Optional[int] = None) -> int:
"""Remove terminal tasks from tracking, optionally scoped to one user.
Args:
user_id: If provided, only clear tasks belonging to this user,
plus legacy tasks with no user_id. If None, clear all.
"""
terminal_statuses = {QueueStatus.COMPLETE, QueueStatus.DONE, QueueStatus.AVAILABLE, QueueStatus.ERROR, QueueStatus.CANCELLED}
with self._lock:
to_remove: list[str] = []
for task_id, status in self._status.items():
if status not in terminal_statuses:
continue
if user_id is None:
to_remove.append(task_id)
continue
task = self._task_data.get(task_id)
if task is None:
# Without task ownership metadata we cannot safely scope removal.
continue
if task.user_id is None or task.user_id == user_id:
to_remove.append(task_id)
for task_id in to_remove:
self._status.pop(task_id, None)
self._status_timestamps.pop(task_id, None)
self._task_data.pop(task_id, None)
self._cancel_flags.pop(task_id, None)
self._active_downloads.pop(task_id, None)
return len(to_remove)
def refresh(self) -> None:
"""Remove any tasks that are done downloading or have stale status."""
terminal_statuses = {QueueStatus.COMPLETE, QueueStatus.DONE, QueueStatus.ERROR, QueueStatus.AVAILABLE, QueueStatus.CANCELLED}
terminal_statuses = TERMINAL_QUEUE_STATUSES
with self._lock:
current_time = datetime.now()
to_remove = []
@@ -335,10 +355,6 @@ class BookQueue:
if task.download_path and not Path(task.download_path).exists():
task.download_path = None
# Mark available downloads as done if file is gone
if status == QueueStatus.AVAILABLE and not task.download_path:
self._update_status(task_id, QueueStatus.DONE)
# Check for stale status entries
last_update = self._status_timestamps.get(task_id)
if last_update and (current_time - last_update) > self._status_timeout:
+129
View File
@@ -0,0 +1,129 @@
"""Shared request-related helper functions used by routes and services."""
from __future__ import annotations
from datetime import datetime, timezone
from typing import Any
from shelfmark.core.logger import setup_logger
from shelfmark.core.settings_registry import load_config_file
_logger = setup_logger(__name__)
def now_utc_iso() -> str:
"""Return the current UTC time as a seconds-precision ISO 8601 string."""
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def emit_ws_event(
ws_manager: Any,
*,
event_name: str,
payload: dict[str, Any],
room: str,
) -> None:
"""Emit a WebSocket event via the shared manager, swallowing failures."""
if ws_manager is None:
return
try:
socketio = getattr(ws_manager, "socketio", None)
is_enabled = getattr(ws_manager, "is_enabled", None)
if socketio is None or not callable(is_enabled) or not is_enabled():
return
socketio.emit(event_name, payload, to=room)
except Exception as exc:
_logger.warning("Failed to emit WebSocket event '%s' to room '%s': %s", event_name, room, exc)
def load_users_request_policy_settings() -> dict[str, Any]:
"""Load global request-policy settings from the users config file."""
return load_config_file("users")
def coerce_bool(value: Any, default: bool = False) -> bool:
"""Coerce arbitrary values into booleans with string-friendly semantics."""
if isinstance(value, bool):
return value
if value is None:
return default
if isinstance(value, str):
normalized = value.strip().lower()
if normalized in {"1", "true", "yes", "on"}:
return True
if normalized in {"0", "false", "no", "off", ""}:
return False
return bool(value)
def get_session_db_user_id(session_obj: Any) -> int | None:
"""Extract and coerce `db_user_id` from a Flask session to ``int | None``."""
raw = session_obj.get("db_user_id") if session_obj is not None else None
try:
return int(raw) if raw is not None else None
except (TypeError, ValueError):
return None
def coerce_int(value: Any, default: int) -> int:
"""Best-effort integer coercion with fallback to default."""
try:
return int(value)
except (TypeError, ValueError):
return default
def normalize_optional_text(value: Any) -> str | None:
"""Return a trimmed string or None for empty/non-string input."""
if not isinstance(value, str):
return None
normalized = value.strip()
return normalized or None
def normalize_positive_int(value: Any) -> int | None:
"""Parse *value* as a positive integer, returning ``None`` on failure."""
try:
parsed = int(value)
except (TypeError, ValueError):
return None
return parsed if parsed > 0 else None
def normalize_optional_positive_int(value: Any, field_name: str = "value") -> int | None:
"""Parse *value* as a positive integer or ``None``.
Raises ``ValueError`` when *value* is present but not a valid
positive integer.
"""
if value is None:
return None
try:
parsed = int(value)
except (TypeError, ValueError) as exc:
raise ValueError(f"{field_name} must be a positive integer when provided") from exc
if parsed < 1:
raise ValueError(f"{field_name} must be a positive integer when provided")
return parsed
def populate_request_usernames(rows: list[dict[str, Any]], user_db: Any) -> None:
"""Add 'username' to each request row by looking up user_id."""
cache: dict[int, str] = {}
for row in rows:
requester_id = row["user_id"]
if requester_id not in cache:
requester = user_db.get_user(user_id=requester_id)
cache[requester_id] = requester.get("username", "") if requester else ""
row["username"] = cache[requester_id]
def extract_release_source_id(release_data: Any) -> str | None:
"""Extract and normalize release_data.source_id."""
if not isinstance(release_data, dict):
return None
source_id = release_data.get("source_id")
if not isinstance(source_id, str):
return None
normalized = source_id.strip()
return normalized or None
+24 -2
View File
@@ -43,6 +43,21 @@ def cap_mode(mode: PolicyMode, ceiling: PolicyMode) -> PolicyMode:
return mode
def _source_results_are_releases(source: Any) -> bool:
normalized_source = normalize_source(source)
if normalized_source in {"", "*"}:
return False
from shelfmark.release_sources import source_results_are_releases
return source_results_are_releases(normalized_source)
def _normalize_release_result_mode(source: Any, mode: PolicyMode) -> PolicyMode:
"""Concrete release browse results cannot fall back to request_book semantics."""
if mode == PolicyMode.REQUEST_BOOK and _source_results_are_releases(source):
return PolicyMode.REQUEST_RELEASE
return mode
REQUEST_POLICY_KEYS = frozenset(
{
"REQUESTS_ENABLED",
@@ -320,6 +335,10 @@ def resolve_policy_mode(
The content-type default acts as a ceiling — matrix rules can only
match or restrict further, never upgrade beyond the default.
Concrete-release browse exception:
- sources whose browse results are already concrete releases normalize
request_book to request_release.
"""
effective = merge_request_policy_settings(global_settings, user_settings)
@@ -346,6 +365,9 @@ def resolve_policy_mode(
for candidate_source, candidate_content_type in candidates:
for rule_source, rule_content_type, rule_mode in rules:
if rule_source == candidate_source and rule_content_type == candidate_content_type:
return cap_mode(rule_mode, ceiling)
return _normalize_release_result_mode(
normalized_source,
cap_mode(rule_mode, ceiling),
)
return ceiling
return _normalize_release_result_mode(normalized_source, ceiling)
+159 -201
View File
@@ -17,6 +17,7 @@ from shelfmark.core.request_policy import (
parse_policy_mode,
resolve_policy_mode,
)
from shelfmark.core.request_validation import RequestStatus
from shelfmark.core.requests_service import (
RequestServiceError,
cancel_request,
@@ -24,46 +25,26 @@ from shelfmark.core.requests_service import (
fulfil_request,
reject_request,
)
from shelfmark.core.activity_service import ActivityService, build_request_item_key
from shelfmark.core.notifications import (
NotificationContext,
NotificationEvent,
notify_admin,
notify_user,
)
from shelfmark.core.settings_registry import load_config_file
from shelfmark.core.request_helpers import (
coerce_bool,
coerce_int,
emit_ws_event,
load_users_request_policy_settings,
normalize_optional_text,
normalize_positive_int,
populate_request_usernames,
)
from shelfmark.core.user_db import UserDB
logger = setup_logger(__name__)
def _load_users_request_policy_settings() -> dict[str, Any]:
"""Load global request-policy settings from users config."""
return load_config_file("users")
def _as_bool(value: Any, default: bool = False) -> bool:
if isinstance(value, bool):
return value
if value is None:
return default
if isinstance(value, str):
normalized = value.strip().lower()
if normalized in {"1", "true", "yes", "on"}:
return True
if normalized in {"0", "false", "no", "off", ""}:
return False
return bool(value)
def _as_int(value: Any, default: int) -> int:
try:
parsed = int(value)
except (TypeError, ValueError):
return default
return parsed
def _error_response(
message: str,
status_code: int,
@@ -110,111 +91,127 @@ def _require_db_user_id() -> tuple[int | None, Any | None]:
)
def _require_admin_user_id() -> tuple[int | None, Any | None]:
if not session.get("is_admin", False):
return None, (jsonify({"error": "Admin access required"}), 403)
raw_admin_id = session.get("db_user_id")
if raw_admin_id is None:
return None, (jsonify({"error": "Admin user identity unavailable"}), 403)
try:
return int(raw_admin_id), None
except (TypeError, ValueError):
return None, (jsonify({"error": "Admin user identity unavailable"}), 403)
def _resolve_effective_policy(
user_db: UserDB,
*,
db_user_id: int | None,
) -> tuple[dict[str, Any], dict[str, Any], dict[str, Any], bool]:
global_settings = _load_users_request_policy_settings()
global_settings = load_users_request_policy_settings()
user_settings = user_db.get_user_settings(db_user_id) if db_user_id is not None else {}
effective = merge_request_policy_settings(global_settings, user_settings)
requests_enabled = _as_bool(effective.get("REQUESTS_ENABLED"), False)
requests_enabled = coerce_bool(effective.get("REQUESTS_ENABLED"), False)
return global_settings, user_settings, effective, requests_enabled
def _emit_request_event(
ws_manager: Any,
*,
event_name: str,
payload: dict[str, Any],
room: str,
) -> None:
if ws_manager is None:
return
try:
socketio = getattr(ws_manager, "socketio", None)
is_enabled = getattr(ws_manager, "is_enabled", None)
if socketio is None or not callable(is_enabled) or not is_enabled():
return
socketio.emit(event_name, payload, to=room)
except Exception as exc:
logger.warning(f"Failed to emit WebSocket event '{event_name}' to room '{room}': {exc}")
def _extract_release_source_id(release_data: Any) -> str | None:
if not isinstance(release_data, dict):
return None
source_id = release_data.get("source_id")
if not isinstance(source_id, str):
return None
normalized = source_id.strip()
return normalized or None
def _record_terminal_request_snapshot(
activity_service: ActivityService | None,
*,
request_row: dict[str, Any],
) -> None:
if activity_service is None:
return
request_status = request_row.get("status")
if request_status not in {"rejected", "cancelled"}:
return
raw_request_id = request_row.get("id")
try:
request_id = int(raw_request_id)
except (TypeError, ValueError):
return
if request_id < 1:
return
raw_user_id = request_row.get("user_id")
try:
user_id = int(raw_user_id)
except (TypeError, ValueError):
user_id = None
source_id = _extract_release_source_id(request_row.get("release_data"))
try:
activity_service.record_terminal_snapshot(
user_id=user_id,
item_type="request",
item_key=build_request_item_key(request_id),
origin="request",
final_status=request_status,
snapshot={"kind": "request", "request": request_row},
request_id=request_id,
source_id=source_id,
)
except Exception as exc:
logger.warning("Failed to record terminal request snapshot for request %s: %s", request_id, exc)
def _normalize_optional_text(value: Any) -> str | None:
if not isinstance(value, str):
return None
normalized = value.strip()
return normalized or None
def _resolve_title_from_book_data(book_data: Any) -> str:
if isinstance(book_data, dict):
title = _normalize_optional_text(book_data.get("title"))
title = normalize_optional_text(book_data.get("title"))
if title is not None:
return title
return "Unknown title"
def _normalize_optional_source_id(value: Any) -> str | None:
"""Normalize source identifiers while allowing integer provider ids."""
if isinstance(value, bool) or value is None:
return None
if isinstance(value, int):
value = str(value)
return normalize_optional_text(value)
def _build_release_result_data_from_book_data(
*,
source: str,
book_data: dict[str, Any],
content_type: str,
) -> dict[str, Any]:
"""Build release-level payload fields for sources whose browse results are releases."""
source_id = _normalize_optional_source_id(book_data.get("provider_id")) or _normalize_optional_source_id(
book_data.get("id")
)
payload: dict[str, Any] = {
"source": source,
"source_id": source_id,
"title": book_data.get("title"),
"author": book_data.get("author"),
"year": book_data.get("year"),
"format": book_data.get("format"),
"size": book_data.get("size"),
"preview": book_data.get("preview"),
"content_type": content_type,
"source_url": book_data.get("source_url"),
"search_mode": "direct",
}
return {key: value for key, value in payload.items() if value is not None}
def _source_results_are_releases(source: str) -> bool:
normalized_source = normalize_source(source)
if normalized_source in {"", "*"}:
return False
from shelfmark.release_sources import source_results_are_releases
return source_results_are_releases(normalized_source)
def _normalize_release_result_request_payload(
*,
source: str,
request_level: Any,
book_data: Any,
release_data: Any,
content_type: str,
) -> tuple[Any, Any]:
"""Concrete-release browse results are always handled as release-level requests."""
if not _source_results_are_releases(source):
return request_level, release_data
normalized_release_data = release_data
if normalized_release_data is None and isinstance(book_data, dict):
normalized_release_data = _build_release_result_data_from_book_data(
source=source,
book_data=book_data,
content_type=content_type,
)
elif isinstance(normalized_release_data, dict):
normalized_release_data = dict(normalized_release_data)
if isinstance(normalized_release_data, dict):
normalized_release_data["source"] = source
if normalized_release_data.get("content_type") is None:
normalized_release_data["content_type"] = content_type
normalized_source_id = _normalize_optional_source_id(normalized_release_data.get("source_id"))
if normalized_source_id is not None:
normalized_release_data["source_id"] = normalized_source_id
elif isinstance(book_data, dict):
fallback_source_id = _normalize_optional_source_id(book_data.get("provider_id")) or _normalize_optional_source_id(
book_data.get("id")
)
if fallback_source_id is not None:
normalized_release_data["source_id"] = fallback_source_id
return "release", normalized_release_data
def _resolve_request_title(request_row: dict[str, Any]) -> str:
return _resolve_title_from_book_data(request_row.get("book_data"))
def _format_user_label(username: str | None, user_id: int | None = None) -> str:
normalized_username = _normalize_optional_text(username)
normalized_username = normalize_optional_text(username)
if normalized_username is not None:
return normalized_username
if user_id is not None and user_id > 0:
@@ -222,30 +219,23 @@ def _format_user_label(username: str | None, user_id: int | None = None) -> str:
return "unknown user"
def _resolve_request_username(
user_db: UserDB,
*,
request_row: dict[str, Any],
fallback_username: str | None = None,
) -> str | None:
normalized_fallback = _normalize_optional_text(fallback_username)
raw_user_id = request_row.get("user_id")
try:
request_user_id = int(raw_user_id)
except (TypeError, ValueError):
return normalized_fallback
requester = user_db.get_user(user_id=request_user_id)
if not isinstance(requester, dict):
return normalized_fallback
return _normalize_optional_text(requester.get("username")) or normalized_fallback
def _format_requester_label(user_db: UserDB, request_row: dict[str, Any]) -> str:
"""Resolve a display label for the user who created a request."""
user_id = normalize_positive_int(request_row.get("user_id"))
if user_id is not None:
requester = user_db.get_user(user_id=user_id)
if isinstance(requester, dict):
username = normalize_optional_text(requester.get("username"))
if username is not None:
return username
return _format_user_label(None, user_id)
def _resolve_request_source_and_format(request_row: dict[str, Any]) -> tuple[str, str | None]:
release_data = request_row.get("release_data")
if isinstance(release_data, dict):
source = normalize_source(release_data.get("source") or request_row.get("source_hint"))
release_format = _normalize_optional_text(
release_format = normalize_optional_text(
release_data.get("format")
or release_data.get("filetype")
or release_data.get("extension")
@@ -254,13 +244,6 @@ def _resolve_request_source_and_format(request_row: dict[str, Any]) -> tuple[str
return normalize_source(request_row.get("source_hint")), None
def _resolve_request_user_id(request_row: dict[str, Any]) -> int | None:
raw_user_id = request_row.get("user_id")
try:
user_id = int(raw_user_id)
except (TypeError, ValueError):
return None
return user_id if user_id > 0 else None
def _notify_admin_for_request_event(
@@ -268,7 +251,6 @@ def _notify_admin_for_request_event(
*,
event: NotificationEvent,
request_row: dict[str, Any],
fallback_username: str | None = None,
) -> None:
book_data = request_row.get("book_data")
if not isinstance(book_data, dict):
@@ -279,21 +261,17 @@ def _notify_admin_for_request_event(
event=event,
title=str(book_data.get("title") or "Unknown title"),
author=str(book_data.get("author") or "Unknown author"),
username=_resolve_request_username(
user_db,
request_row=request_row,
fallback_username=fallback_username,
),
username=_format_requester_label(user_db, request_row),
content_type=normalize_content_type(
request_row.get("content_type") or book_data.get("content_type")
),
format=release_format,
source=source,
admin_note=_normalize_optional_text(request_row.get("admin_note")),
admin_note=normalize_optional_text(request_row.get("admin_note")),
error_message=None,
)
owner_user_id = _resolve_request_user_id(request_row)
owner_user_id = normalize_positive_int(request_row.get("user_id"))
try:
notify_admin(event, context)
except Exception as exc:
@@ -321,7 +299,6 @@ def register_request_routes(
*,
resolve_auth_mode: Callable[[], str],
queue_release: Callable[..., tuple[bool, str | None]],
activity_service: ActivityService | None = None,
ws_manager: Any | None = None,
) -> None:
"""Register request policy and request lifecycle routes."""
@@ -355,6 +332,7 @@ def register_request_routes(
default_audio_mode = parse_policy_mode(effective.get("REQUEST_POLICY_DEFAULT_AUDIOBOOK"))
source_capabilities = get_source_content_type_capabilities()
from shelfmark.release_sources import source_results_are_releases
source_modes = []
for source_name in sorted(source_capabilities):
supported_types = sorted(
@@ -374,6 +352,7 @@ def register_request_routes(
{
"source": source_name,
"supported_content_types": supported_types,
"browse_results_are_releases": source_results_are_releases(source_name),
"modes": modes,
}
)
@@ -382,7 +361,7 @@ def register_request_routes(
{
"requests_enabled": requests_enabled,
"is_admin": is_admin,
"allow_notes": _as_bool(effective.get("REQUESTS_ALLOW_NOTES"), default=True),
"allow_notes": coerce_bool(effective.get("REQUESTS_ALLOW_NOTES"), default=True),
"defaults": {
"ebook": (
default_ebook_mode.value
@@ -409,7 +388,7 @@ def register_request_routes(
db_user_id, db_gate = _require_db_user_id()
if db_gate is not None or db_user_id is None:
return db_gate
actor_username = _normalize_optional_text(session.get("user_id"))
actor_username = normalize_optional_text(session.get("user_id"))
actor_label = _format_user_label(actor_username, db_user_id)
data = request.get_json(silent=True)
@@ -436,6 +415,13 @@ def register_request_routes(
or data.get("content_type")
or book_data.get("content_type")
)
request_level, release_data = _normalize_release_result_request_payload(
source=source,
request_level=request_level,
book_data=book_data,
release_data=release_data,
content_type=content_type,
)
global_settings, user_settings, effective, requests_enabled = _resolve_effective_policy(
user_db,
@@ -453,7 +439,7 @@ def register_request_routes(
code="requests_unavailable",
)
max_pending = _as_int(
max_pending = coerce_int(
effective.get("MAX_PENDING_REQUESTS_PER_USER"),
default=20,
)
@@ -461,7 +447,7 @@ def register_request_routes(
max_pending = 1
if max_pending > 1000:
max_pending = 1000
allow_notes = _as_bool(effective.get("REQUESTS_ALLOW_NOTES"), default=True)
allow_notes = coerce_bool(effective.get("REQUESTS_ALLOW_NOTES"), default=True)
note_value = data.get("note") if allow_notes else None
resolved_mode = resolve_policy_mode(
@@ -535,13 +521,13 @@ def register_request_routes(
event_payload["title"],
actor_label,
)
_emit_request_event(
emit_ws_event(
ws_manager,
event_name="new_request",
payload=event_payload,
room="admins",
)
_emit_request_event(
emit_ws_event(
ws_manager,
event_name="request_update",
payload=event_payload,
@@ -552,7 +538,6 @@ def register_request_routes(
user_db,
event=NotificationEvent.REQUEST_CREATED,
request_row=created,
fallback_username=actor_username,
)
return jsonify(created), 201
@@ -601,27 +586,25 @@ def register_request_routes(
except RequestServiceError as exc:
return _error_response(str(exc), exc.status_code, code=exc.code)
_record_terminal_request_snapshot(activity_service, request_row=updated)
event_payload = {
"request_id": updated["id"],
"status": updated["status"],
"title": _resolve_request_title(updated),
}
actor_label = _format_user_label(_normalize_optional_text(session.get("user_id")), db_user_id)
actor_label = _format_user_label(normalize_optional_text(session.get("user_id")), db_user_id)
logger.info(
"Request cancelled #%s for '%s' by %s",
updated["id"],
event_payload["title"],
actor_label,
)
_emit_request_event(
emit_ws_event(
ws_manager,
event_name="request_update",
payload=event_payload,
room=f"user_{db_user_id}",
)
_emit_request_event(
emit_ws_event(
ws_manager,
event_name="request_update",
payload=event_payload,
@@ -647,13 +630,7 @@ def register_request_routes(
except ValueError as exc:
return jsonify({"error": str(exc)}), 400
user_cache: dict[int, str] = {}
for row in rows:
requester_id = row["user_id"]
if requester_id not in user_cache:
requester = user_db.get_user(user_id=requester_id)
user_cache[requester_id] = requester.get("username", "") if requester else ""
row["username"] = user_cache[requester_id]
populate_request_usernames(rows, user_db)
return jsonify(rows)
@@ -667,11 +644,11 @@ def register_request_routes(
by_status = {
status: len(user_db.list_requests(status=status))
for status in ("pending", "fulfilled", "rejected", "cancelled")
for status in RequestStatus
}
return jsonify(
{
"pending": by_status["pending"],
"pending": by_status[RequestStatus.PENDING],
"total": sum(by_status.values()),
"by_status": by_status,
}
@@ -682,16 +659,10 @@ def register_request_routes(
auth_gate = _require_request_endpoints_available(resolve_auth_mode)
if auth_gate is not None:
return auth_gate
if not session.get("is_admin", False):
return jsonify({"error": "Admin access required"}), 403
raw_admin_id = session.get("db_user_id")
if raw_admin_id is None:
return jsonify({"error": "Admin user identity unavailable"}), 403
try:
admin_user_id = int(raw_admin_id)
except (TypeError, ValueError):
return jsonify({"error": "Admin user identity unavailable"}), 403
admin_user_id, admin_gate = _require_admin_user_id()
if admin_gate is not None:
return admin_gate
data = request.get_json(silent=True) or {}
if not isinstance(data, dict):
@@ -705,6 +676,7 @@ def register_request_routes(
queue_release=queue_release,
release_data=data.get("release_data"),
admin_note=data.get("admin_note"),
manual_approval=data.get("manual_approval", False),
)
except RequestServiceError as exc:
return _error_response(str(exc), exc.status_code, code=exc.code)
@@ -714,11 +686,8 @@ def register_request_routes(
"status": updated["status"],
"title": _resolve_request_title(updated),
}
admin_label = _format_user_label(_normalize_optional_text(session.get("user_id")), admin_user_id)
requester_label = _format_user_label(
_resolve_request_username(user_db, request_row=updated),
_resolve_request_user_id(updated),
)
admin_label = _format_user_label(normalize_optional_text(session.get("user_id")), admin_user_id)
requester_label = _format_requester_label(user_db, updated)
logger.info(
"Request fulfilled #%s for '%s' by %s (requested by %s)",
updated["id"],
@@ -726,13 +695,13 @@ def register_request_routes(
admin_label,
requester_label,
)
_emit_request_event(
emit_ws_event(
ws_manager,
event_name="request_update",
payload=event_payload,
room=f"user_{updated['user_id']}",
)
_emit_request_event(
emit_ws_event(
ws_manager,
event_name="request_update",
payload=event_payload,
@@ -752,16 +721,10 @@ def register_request_routes(
auth_gate = _require_request_endpoints_available(resolve_auth_mode)
if auth_gate is not None:
return auth_gate
if not session.get("is_admin", False):
return jsonify({"error": "Admin access required"}), 403
raw_admin_id = session.get("db_user_id")
if raw_admin_id is None:
return jsonify({"error": "Admin user identity unavailable"}), 403
try:
admin_user_id = int(raw_admin_id)
except (TypeError, ValueError):
return jsonify({"error": "Admin user identity unavailable"}), 403
admin_user_id, admin_gate = _require_admin_user_id()
if admin_gate is not None:
return admin_gate
data = request.get_json(silent=True) or {}
if not isinstance(data, dict):
@@ -777,18 +740,13 @@ def register_request_routes(
except RequestServiceError as exc:
return _error_response(str(exc), exc.status_code, code=exc.code)
_record_terminal_request_snapshot(activity_service, request_row=updated)
event_payload = {
"request_id": updated["id"],
"status": updated["status"],
"title": _resolve_request_title(updated),
}
admin_label = _format_user_label(_normalize_optional_text(session.get("user_id")), admin_user_id)
requester_label = _format_user_label(
_resolve_request_username(user_db, request_row=updated),
_resolve_request_user_id(updated),
)
admin_label = _format_user_label(normalize_optional_text(session.get("user_id")), admin_user_id)
requester_label = _format_requester_label(user_db, updated)
logger.info(
"Request rejected #%s for '%s' by %s (requested by %s)",
updated["id"],
@@ -796,13 +754,13 @@ def register_request_routes(
admin_label,
requester_label,
)
_emit_request_event(
emit_ws_event(
ws_manager,
event_name="request_update",
payload=event_payload,
room=f"user_{updated['user_id']}",
)
_emit_request_event(
emit_ws_event(
ws_manager,
event_name="request_update",
payload=event_payload,
+84
View File
@@ -0,0 +1,84 @@
"""Shared request validation and normalization helpers."""
from __future__ import annotations
from enum import Enum
from typing import Any
from shelfmark.core.models import QueueStatus
from shelfmark.core.request_policy import parse_policy_mode
class RequestStatus(str, Enum):
"""Enum for request lifecycle statuses."""
PENDING = "pending"
FULFILLED = "fulfilled"
REJECTED = "rejected"
CANCELLED = "cancelled"
DELIVERY_STATE_NONE = "none"
VALID_REQUEST_STATUSES = frozenset(RequestStatus)
TERMINAL_REQUEST_STATUSES = frozenset({
RequestStatus.FULFILLED, RequestStatus.REJECTED, RequestStatus.CANCELLED,
})
VALID_REQUEST_LEVELS = frozenset({"book", "release"})
VALID_DELIVERY_STATES = frozenset({DELIVERY_STATE_NONE} | set(QueueStatus))
def normalize_request_status(status: Any) -> str:
"""Validate and normalize request status values."""
if not isinstance(status, str):
raise ValueError(f"Invalid request status: {status}")
normalized = status.strip().lower()
if normalized not in VALID_REQUEST_STATUSES:
raise ValueError(f"Invalid request status: {status}")
return normalized
def normalize_policy_mode(mode: Any) -> str:
"""Validate and normalize policy mode values."""
parsed = parse_policy_mode(mode)
if parsed is None:
raise ValueError(f"Invalid policy_mode: {mode}")
return parsed.value
def normalize_request_level(request_level: Any) -> str:
"""Validate and normalize request level values."""
if not isinstance(request_level, str):
raise ValueError(f"Invalid request_level: {request_level}")
normalized = request_level.strip().lower()
if normalized not in VALID_REQUEST_LEVELS:
raise ValueError(f"Invalid request_level: {request_level}")
return normalized
def normalize_delivery_state(state: Any) -> str:
"""Validate and normalize delivery-state values."""
if not isinstance(state, str):
raise ValueError(f"Invalid delivery_state: {state}")
normalized = state.strip().lower()
if normalized not in VALID_DELIVERY_STATES:
raise ValueError(f"Invalid delivery_state: {state}")
return normalized
def validate_request_level_payload(request_level: Any, release_data: Any) -> str:
"""Validate request_level and release_data shape coupling."""
normalized_level = normalize_request_level(request_level)
if normalized_level == "release" and release_data is None:
raise ValueError("request_level=release requires non-null release_data")
if normalized_level == "book" and release_data is not None:
raise ValueError("request_level=book requires null release_data")
return normalized_level
def validate_status_transition(current_status: Any, new_status: Any) -> tuple[str, str]:
"""Validate request status transitions and terminal immutability."""
current = normalize_request_status(current_status)
new = normalize_request_status(new_status)
if current in TERMINAL_REQUEST_STATUSES and new != current:
raise ValueError("Terminal request statuses are immutable")
return current, new
+131 -210
View File
@@ -6,25 +6,20 @@ from datetime import datetime, timezone
import json
from typing import Any, Callable, TYPE_CHECKING
from shelfmark.core.request_policy import normalize_content_type, parse_policy_mode
VALID_REQUEST_STATUSES = frozenset({"pending", "fulfilled", "rejected", "cancelled"})
TERMINAL_REQUEST_STATUSES = frozenset({"fulfilled", "rejected", "cancelled"})
VALID_REQUEST_LEVELS = frozenset({"book", "release"})
VALID_DELIVERY_STATES = frozenset(
{
"none",
"unknown",
"queued",
"resolving",
"locating",
"downloading",
"complete",
"error",
"cancelled",
}
from shelfmark.core.request_policy import normalize_content_type
from shelfmark.core.models import QueueStatus
from shelfmark.core.request_validation import (
DELIVERY_STATE_NONE,
RequestStatus,
normalize_policy_mode,
normalize_request_level,
normalize_request_status,
validate_request_level_payload,
validate_status_transition,
)
from shelfmark.core.request_helpers import extract_release_source_id, normalize_positive_int
MAX_REQUEST_NOTE_LENGTH = 1000
MAX_REQUEST_JSON_BLOB_BYTES = 10 * 1024
@@ -48,63 +43,6 @@ class RequestServiceError(ValueError):
self.code = code
def normalize_request_status(status: Any) -> str:
"""Validate and normalize request status values."""
if not isinstance(status, str):
raise ValueError(f"Invalid request status: {status}")
normalized = status.strip().lower()
if normalized not in VALID_REQUEST_STATUSES:
raise ValueError(f"Invalid request status: {status}")
return normalized
def normalize_policy_mode(mode: Any) -> str:
"""Validate and normalize policy mode values."""
parsed = parse_policy_mode(mode)
if parsed is None:
raise ValueError(f"Invalid policy_mode: {mode}")
return parsed.value
def normalize_request_level(request_level: Any) -> str:
"""Validate and normalize request level values."""
if not isinstance(request_level, str):
raise ValueError(f"Invalid request_level: {request_level}")
normalized = request_level.strip().lower()
if normalized not in VALID_REQUEST_LEVELS:
raise ValueError(f"Invalid request_level: {request_level}")
return normalized
def normalize_delivery_state(state: Any) -> str:
"""Validate and normalize delivery-state values."""
if not isinstance(state, str):
raise ValueError(f"Invalid delivery_state: {state}")
normalized = state.strip().lower()
if normalized not in VALID_DELIVERY_STATES:
raise ValueError(f"Invalid delivery_state: {state}")
return normalized
def validate_request_level_payload(request_level: Any, release_data: Any) -> str:
"""Validate request_level and release_data shape coupling."""
normalized_level = normalize_request_level(request_level)
if normalized_level == "release" and release_data is None:
raise ValueError("request_level=release requires non-null release_data")
if normalized_level == "book" and release_data is not None:
raise ValueError("request_level=book requires null release_data")
return normalized_level
def validate_status_transition(current_status: Any, new_status: Any) -> tuple[str, str]:
"""Validate request status transitions and terminal immutability."""
current = normalize_request_status(current_status)
new = normalize_request_status(new_status)
if current in TERMINAL_REQUEST_STATUSES and new != current:
raise ValueError("Terminal request statuses are immutable")
return current, new
def _normalize_match_text(value: Any) -> str:
if not isinstance(value, str):
return ""
@@ -166,7 +104,7 @@ def _find_duplicate_pending_request(
author: str,
content_type: str,
) -> dict[str, Any] | None:
pending_rows = user_db.list_requests(user_id=user_id, status="pending")
pending_rows = user_db.list_requests(user_id=user_id, status=RequestStatus.PENDING)
for row in pending_rows:
row_book_data = row.get("book_data") or {}
if not isinstance(row_book_data, dict):
@@ -186,22 +124,12 @@ def _now_timestamp() -> str:
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def _extract_release_source_id(release_data: Any) -> str | None:
if not isinstance(release_data, dict):
def _normalize_admin_note(admin_note: Any) -> str | None:
if admin_note is None:
return None
source_id = release_data.get("source_id")
if not isinstance(source_id, str):
return None
normalized = source_id.strip()
return normalized or None
def _existing_delivery_state(request_row: dict[str, Any]) -> str:
raw_state = request_row.get("delivery_state")
if not isinstance(raw_state, str):
return "none"
normalized = raw_state.strip().lower()
return normalized if normalized in VALID_DELIVERY_STATES else "none"
if not isinstance(admin_note, str):
raise RequestServiceError("admin_note must be a string", status_code=400)
return admin_note.strip() or None
def sync_delivery_states_from_queue_status(
@@ -211,30 +139,48 @@ def sync_delivery_states_from_queue_status(
user_id: int | None = None,
) -> list[dict[str, Any]]:
"""Persist delivery-state transitions for fulfilled requests based on queue status."""
source_delivery_states: dict[str, str] = {}
for status_key in ("queued", "resolving", "locating", "downloading", "complete", "error", "cancelled"):
fulfilled_rows = user_db.list_requests(user_id=user_id, status=RequestStatus.FULFILLED)
if not fulfilled_rows:
return []
unique_request_ids_by_source: dict[str, int] = {}
ambiguous_source_ids: set[str] = set()
for row in fulfilled_rows:
source_id = extract_release_source_id(row.get("release_data"))
if source_id is None:
continue
if source_id in unique_request_ids_by_source:
ambiguous_source_ids.add(source_id)
continue
unique_request_ids_by_source[source_id] = int(row["id"])
for source_id in ambiguous_source_ids:
unique_request_ids_by_source.pop(source_id, None)
request_delivery_states: dict[int, str] = {}
for status_key in QueueStatus:
status_bucket = queue_status.get(status_key)
if not isinstance(status_bucket, dict):
continue
for source_id in status_bucket:
source_delivery_states[source_id] = status_key
for source_id, task_payload in status_bucket.items():
request_id = None
if isinstance(task_payload, dict):
request_id = normalize_positive_int(task_payload.get("request_id"))
if request_id is None:
request_id = unique_request_ids_by_source.get(str(source_id).strip())
if request_id is None:
continue
request_delivery_states[request_id] = status_key
if not source_delivery_states:
if not request_delivery_states:
return []
fulfilled_rows = user_db.list_requests(user_id=user_id, status="fulfilled")
updated: list[dict[str, Any]] = []
for row in fulfilled_rows:
source_id = _extract_release_source_id(row.get("release_data"))
if source_id is None:
continue
delivery_state = source_delivery_states.get(source_id)
delivery_state = request_delivery_states.get(int(row["id"]))
if delivery_state is None:
continue
if _existing_delivery_state(row) == delivery_state:
if row.get("delivery_state", DELIVERY_STATE_NONE) == delivery_state:
continue
updated.append(
@@ -335,6 +281,15 @@ def ensure_request_access(
return request_row
def _require_pending(request_row: dict[str, Any]) -> None:
if request_row["status"] != RequestStatus.PENDING:
raise RequestServiceError(
"Request is already in a terminal state",
status_code=409,
code="stale_transition",
)
def cancel_request(
user_db: "UserDB",
*,
@@ -348,18 +303,13 @@ def cancel_request(
actor_user_id=actor_user_id,
is_admin=False,
)
if request_row["status"] != "pending":
raise RequestServiceError(
"Request is already in a terminal state",
status_code=409,
code="stale_transition",
)
_require_pending(request_row)
try:
return user_db.update_request(
request_id,
expected_current_status="pending",
status="cancelled",
expected_current_status=RequestStatus.PENDING,
status=RequestStatus.CANCELLED,
)
except ValueError as exc:
raise RequestServiceError(str(exc), status_code=409, code="stale_transition") from exc
@@ -379,24 +329,15 @@ def reject_request(
actor_user_id=admin_user_id,
is_admin=True,
)
if request_row["status"] != "pending":
raise RequestServiceError(
"Request is already in a terminal state",
status_code=409,
code="stale_transition",
)
_require_pending(request_row)
normalized_admin_note = None
if admin_note is not None:
if not isinstance(admin_note, str):
raise RequestServiceError("admin_note must be a string", status_code=400)
normalized_admin_note = admin_note.strip() or None
normalized_admin_note = _normalize_admin_note(admin_note)
try:
return user_db.update_request(
request_id,
expected_current_status="pending",
status="rejected",
expected_current_status=RequestStatus.PENDING,
status=RequestStatus.REJECTED,
admin_note=normalized_admin_note,
reviewed_by=admin_user_id,
reviewed_at=_now_timestamp(),
@@ -413,6 +354,7 @@ def fulfil_request(
queue_release: Callable[..., tuple[bool, str | None]],
release_data: Any = None,
admin_note: Any = None,
manual_approval: Any = False,
) -> dict[str, Any]:
"""Fulfil a pending request and queue the release under requesting-user identity."""
request_row = ensure_request_access(
@@ -421,31 +363,37 @@ def fulfil_request(
actor_user_id=admin_user_id,
is_admin=True,
)
if request_row["status"] != "pending":
raise RequestServiceError(
"Request is already in a terminal state",
status_code=409,
code="stale_transition",
)
_require_pending(request_row)
normalized_admin_note = None
if admin_note is not None:
if not isinstance(admin_note, str):
raise RequestServiceError("admin_note must be a string", status_code=400)
normalized_admin_note = admin_note.strip() or None
normalized_admin_note = _normalize_admin_note(admin_note)
if not isinstance(manual_approval, bool):
raise RequestServiceError("manual_approval must be a boolean", status_code=400)
selected_release_data = release_data if release_data is not None else request_row.get("release_data")
if selected_release_data is not None and not isinstance(selected_release_data, dict):
raise RequestServiceError("release_data must be an object", status_code=400)
if request_row["request_level"] == "book" and selected_release_data is None:
if selected_release_data is None and manual_approval:
try:
return user_db.update_request(
request_id,
expected_current_status=RequestStatus.PENDING,
status=RequestStatus.FULFILLED,
release_data=None,
delivery_state=QueueStatus.COMPLETE,
delivery_updated_at=_now_timestamp(),
last_failure_reason=None,
admin_note=normalized_admin_note,
reviewed_by=admin_user_id,
reviewed_at=_now_timestamp(),
)
except ValueError as exc:
raise RequestServiceError(str(exc), status_code=409, code="stale_transition") from exc
if selected_release_data is None:
raise RequestServiceError(
"release_data is required to fulfil book-level requests",
status_code=400,
)
if request_row["request_level"] == "release" and selected_release_data is None:
raise RequestServiceError(
"release_data is required to fulfil release-level requests",
"release_data is required to fulfil requests",
status_code=400,
)
@@ -455,29 +403,14 @@ def fulfil_request(
if requester is None:
raise RequestServiceError("Requesting user not found", status_code=404)
queued_release_data = dict(selected_release_data)
queued_release_data["_request_id"] = request_id
success, error = queue_release(
queued_release_data,
0,
user_id=request_row["user_id"],
username=requester.get("username"),
)
if not success:
raise RequestServiceError(
error or "Failed to queue release",
status_code=409,
code="queue_failed",
)
original_release_data = request_row.get("release_data")
try:
return user_db.update_request(
claimed_request = user_db.update_request(
request_id,
expected_current_status="pending",
status="fulfilled",
expected_current_status=RequestStatus.PENDING,
status=RequestStatus.FULFILLED,
release_data=selected_release_data,
delivery_state="queued",
delivery_state=QueueStatus.QUEUED,
delivery_updated_at=_now_timestamp(),
last_failure_reason=None,
admin_note=normalized_admin_note,
@@ -487,6 +420,37 @@ def fulfil_request(
except ValueError as exc:
raise RequestServiceError(str(exc), status_code=409, code="stale_transition") from exc
queued_release_data = dict(selected_release_data)
queued_release_data["_request_id"] = request_id
try:
success, error = queue_release(
queued_release_data,
0,
user_id=request_row["user_id"],
username=requester.get("username"),
)
except Exception:
user_db.rollback_request_fulfilment(
request_id,
release_data=original_release_data,
last_failure_reason="Queue dispatch raised an exception",
)
raise
if not success:
user_db.rollback_request_fulfilment(
request_id,
release_data=original_release_data,
last_failure_reason=error,
)
raise RequestServiceError(
error or "Failed to queue release",
status_code=409,
code="queue_failed",
)
return claimed_request
def reopen_failed_request(
user_db: "UserDB",
@@ -495,50 +459,7 @@ def reopen_failed_request(
failure_reason: str | None = None,
) -> dict[str, Any] | None:
"""Reopen a failed fulfilled request so admins can re-approve with a new release."""
normalized_failure_reason = None
if isinstance(failure_reason, str):
normalized_failure_reason = failure_reason.strip() or None
with user_db._lock:
conn = user_db._connect()
try:
current_row = conn.execute(
"SELECT * FROM download_requests WHERE id = ?",
(request_id,),
).fetchone()
current_request = user_db._parse_request_row(current_row)
if current_request is None:
return None
if current_request.get("status") != "fulfilled":
return None
current_delivery_state = _existing_delivery_state(current_request)
# Terminal hook callbacks can run before delivery-state sync persists "error".
# Allow reopening fulfilled requests unless they are already complete.
if current_delivery_state == "complete":
return None
if current_delivery_state not in {"error", "cancelled"} and normalized_failure_reason is None:
return None
conn.execute(
"""
UPDATE download_requests
SET status = 'pending',
delivery_state = 'none',
delivery_updated_at = NULL,
release_data = NULL,
last_failure_reason = ?,
reviewed_by = NULL,
reviewed_at = NULL
WHERE id = ?
""",
(normalized_failure_reason, request_id),
)
updated_row = conn.execute(
"SELECT * FROM download_requests WHERE id = ?",
(request_id,),
).fetchone()
conn.commit()
return user_db._parse_request_row(updated_row)
finally:
conn.close()
return user_db.reopen_failed_request(
request_id,
failure_reason=failure_reason,
)
+5
View File
@@ -6,6 +6,7 @@ from typing import List, Optional
MANUAL_QUERY_MAX_LEN = 256
from shelfmark.core.config import config
from shelfmark.core.models import SearchFilters
from shelfmark.metadata_providers import (
BookMetadata,
group_languages_by_localized_title,
@@ -37,6 +38,7 @@ class ReleaseSearchPlan:
grouped_title_variants: List[ReleaseSearchVariant]
manual_query: Optional[str] = None
indexers: Optional[List[str]] = None # Indexer names for Prowlarr (overrides settings)
source_filters: Optional[SearchFilters] = None
@property
def primary_query(self) -> str:
@@ -88,6 +90,7 @@ def build_release_search_plan(
languages: Optional[List[str]] = None,
manual_query: Optional[str] = None,
indexers: Optional[List[str]] = None,
source_filters: Optional[SearchFilters] = None,
) -> ReleaseSearchPlan:
resolved_languages = _normalize_languages(languages)
@@ -109,6 +112,7 @@ def build_release_search_plan(
grouped_title_variants=[variant],
manual_query=resolved_manual_query,
indexers=indexers,
source_filters=source_filters,
)
isbn_candidates: List[str] = []
@@ -165,4 +169,5 @@ def build_release_search_plan(
grouped_title_variants=grouped_variants,
manual_query=None,
indexers=indexers,
source_filters=source_filters,
)
+42 -43
View File
@@ -3,7 +3,7 @@
from functools import wraps
from typing import Any, Callable, Mapping
from flask import Flask, jsonify, request, session
from flask import Flask, g, jsonify, request, session
from werkzeug.security import generate_password_hash
from shelfmark.config.env import CWA_DB_PATH
@@ -16,8 +16,8 @@ from shelfmark.core.auth_modes import (
AUTH_SOURCE_CWA,
AUTH_SOURCE_OIDC,
AUTH_SOURCE_PROXY,
determine_auth_mode,
has_local_password_admin,
is_user_active_for_auth_mode,
load_active_auth_mode,
normalize_auth_source,
)
from shelfmark.core.logger import setup_logger
@@ -33,42 +33,16 @@ logger = setup_logger(__name__)
MIN_PASSWORD_LENGTH = 4
_VISIBLE_SELF_SETTINGS_SECTIONS_KEY = "VISIBLE_SELF_SETTINGS_SECTIONS"
_SELF_SETTINGS_SECTION_DELIVERY = "delivery"
_SELF_SETTINGS_SECTION_SEARCH = "search"
_SELF_SETTINGS_SECTION_NOTIFICATIONS = "notifications"
_VALID_SELF_SETTINGS_SECTIONS = (
_SELF_SETTINGS_SECTION_DELIVERY,
_SELF_SETTINGS_SECTION_SEARCH,
_SELF_SETTINGS_SECTION_NOTIFICATIONS,
)
_DEFAULT_VISIBLE_SELF_SETTINGS_SECTIONS = list(_VALID_SELF_SETTINGS_SECTIONS)
def _get_auth_mode() -> str:
"""Get current auth mode from config."""
try:
config = load_config_file("security")
return determine_auth_mode(
config,
CWA_DB_PATH,
has_local_admin=has_local_password_admin(),
)
except Exception:
return "none"
def _require_authenticated_user(f: Callable[..., Any]) -> Callable[..., Any]:
"""Decorator requiring an authenticated session linked to a local user row."""
@wraps(f)
def decorated(*args, **kwargs):
auth_mode = _get_auth_mode()
if auth_mode != "none" and "user_id" not in session:
return jsonify({"error": "Authentication required"}), 401
if "db_user_id" not in session:
return jsonify({"error": "Authenticated session is missing local user context"}), 403
return f(*args, **kwargs)
return decorated
def _get_current_user(user_db: UserDB) -> tuple[int | None, dict[str, Any] | None, tuple[Any, int] | None]:
raw_user_id = session.get("db_user_id")
try:
@@ -82,13 +56,6 @@ def _get_current_user(user_db: UserDB) -> tuple[int | None, dict[str, Any] | Non
return user_id, user, None
def _is_user_active(user: Mapping[str, Any], auth_method: str) -> bool:
source = normalize_auth_source(user.get("auth_source"), user.get("oidc_subject"))
if source == AUTH_SOURCE_BUILTIN:
return auth_method in (AUTH_SOURCE_BUILTIN, AUTH_SOURCE_OIDC)
return source == auth_method
def _get_self_edit_capabilities(user: Mapping[str, Any]) -> dict[str, Any]:
auth_source = normalize_auth_source(
user.get("auth_source"),
@@ -111,7 +78,7 @@ def _serialize_self_user(user: Mapping[str, Any], auth_mode: str) -> dict[str, A
payload.get("auth_source"),
payload.get("oidc_subject"),
)
payload["is_active"] = _is_user_active(payload, auth_mode)
payload["is_active"] = is_user_active_for_auth_mode(payload, auth_mode)
payload["edit_capabilities"] = _get_self_edit_capabilities(payload)
return payload
@@ -155,6 +122,11 @@ def _get_allowed_self_settings_keys(visible_sections: list[str]) -> set[str]:
key for key, _field in _get_ordered_user_overridable_fields("downloads")
}
if _SELF_SETTINGS_SECTION_SEARCH in visible_sections_set:
allowed_keys |= {
key for key, _field in _get_ordered_user_overridable_fields("search_mode")
}
if _SELF_SETTINGS_SECTION_NOTIFICATIONS in visible_sections_set:
allowed_keys |= {
key for key, _field in _get_ordered_user_overridable_fields("notifications")
@@ -166,6 +138,22 @@ def _get_allowed_self_settings_keys(visible_sections: list[str]) -> set[str]:
def register_self_user_routes(app: Flask, user_db: UserDB) -> None:
"""Register self-service user endpoints."""
def _require_authenticated_user(f: Callable[..., Any]) -> Callable[..., Any]:
"""Decorator requiring an authenticated session linked to a local user row.
Caches the resolved auth_mode in ``g.auth_mode`` for the request.
"""
@wraps(f)
def decorated(*args, **kwargs):
auth_mode = load_active_auth_mode(CWA_DB_PATH, user_db=user_db)
g.auth_mode = auth_mode
if auth_mode != "none" and "user_id" not in session:
return jsonify({"error": "Authentication required"}), 401
if "db_user_id" not in session:
return jsonify({"error": "Authenticated session is missing local user context"}), 403
return f(*args, **kwargs)
return decorated
@app.route("/api/users/me/edit-context", methods=["GET"])
@_require_authenticated_user
def users_me_edit_context():
@@ -173,8 +161,7 @@ def register_self_user_routes(app: Flask, user_db: UserDB) -> None:
if user_error:
return user_error
auth_mode = _get_auth_mode()
serialized_user = _serialize_self_user(user, auth_mode)
serialized_user = _serialize_self_user(user, g.auth_mode)
serialized_user["settings"] = user_db.get_user_settings(user_id)
visible_self_settings_sections = _get_visible_self_settings_sections()
@@ -188,6 +175,16 @@ def register_self_user_routes(app: Flask, user_db: UserDB) -> None:
logger.warning(f"Failed to build user delivery preferences for user_id={user_id}: {exc}")
delivery_preferences = None
search_preferences = None
if _SELF_SETTINGS_SECTION_SEARCH in visible_self_settings_sections:
try:
search_preferences = _build_user_preferences_payload(user_db, user_id, "search_mode")
except ValueError:
return jsonify({"error": "Search mode settings tab not found"}), 500
except Exception as exc:
logger.warning(f"Failed to build user search preferences for user_id={user_id}: {exc}")
search_preferences = None
notification_preferences = None
if _SELF_SETTINGS_SECTION_NOTIFICATIONS in visible_self_settings_sections:
try:
@@ -200,6 +197,7 @@ def register_self_user_routes(app: Flask, user_db: UserDB) -> None:
user_overridable_keys = sorted(
set(delivery_preferences.get("keys", []) if delivery_preferences else [])
| set(search_preferences.get("keys", []) if search_preferences else [])
| set(notification_preferences.get("keys", []) if notification_preferences else [])
)
@@ -207,6 +205,7 @@ def register_self_user_routes(app: Flask, user_db: UserDB) -> None:
{
"user": serialized_user,
"deliveryPreferences": delivery_preferences,
"searchPreferences": search_preferences,
"notificationPreferences": notification_preferences,
"userOverridableKeys": user_overridable_keys,
"visibleUserSettingsSections": visible_self_settings_sections,
@@ -340,7 +339,7 @@ def register_self_user_routes(app: Flask, user_db: UserDB) -> None:
try:
from shelfmark.core.config import config as app_config
app_config.refresh()
app_config.refresh(force=True)
except Exception:
pass
@@ -348,7 +347,7 @@ def register_self_user_routes(app: Flask, user_db: UserDB) -> None:
if not updated:
return jsonify({"error": "User not found"}), 404
result = _serialize_self_user(updated, _get_auth_mode())
result = _serialize_self_user(updated, g.auth_mode)
result["settings"] = user_db.get_user_settings(user_id)
logger.info(f"User {user_id} updated their own account")
return jsonify(result)
+70 -14
View File
@@ -486,22 +486,20 @@ def sync_env_to_config() -> None:
def migrate_mirror_settings() -> None:
"""
Migrate legacy AA mirror config into the new editable mirror list setting.
Sync AA mirror list when code defaults change between versions.
Legacy:
- AA_ADDITIONAL_URLS: comma-separated extra URLs appended to defaults
On startup, compares a hash of DEFAULT_AA_MIRRORS against the hash stored
in the config file. If they differ (i.e., an update shipped new defaults),
the config is overwritten with the new defaults. If they match, the user's
customizations are left untouched.
New:
- AA_MIRROR_URLS: full ordered list of available mirrors (used for Auto mode and for Settings options)
Also handles legacy migration from AA_ADDITIONAL_URLS.
"""
mirrors_config = load_config_file("mirrors")
import hashlib
from shelfmark.core.mirrors import DEFAULT_AA_MIRRORS
from shelfmark.core.utils import normalize_http_url
raw_list = mirrors_config.get("AA_MIRROR_URLS")
raw_additional = mirrors_config.get("AA_ADDITIONAL_URLS", "")
def _normalize_list(values: list[str]) -> list[str]:
out: list[str] = []
for item in values:
@@ -512,12 +510,42 @@ def migrate_mirror_settings() -> None:
out.append(norm)
return out
def _hash_mirrors(mirrors: list[str]) -> str:
return hashlib.sha256(",".join(mirrors).encode()).hexdigest()
normalized_defaults = _normalize_list(DEFAULT_AA_MIRRORS)
current_defaults_hash = _hash_mirrors(normalized_defaults)
mirrors_config = load_config_file("mirrors")
stored_hash = mirrors_config.get("_AA_MIRRORS_DEFAULTS_HASH")
raw_list = mirrors_config.get("AA_MIRROR_URLS")
raw_additional = mirrors_config.get("AA_ADDITIONAL_URLS", "")
def _save_mirrors(values: dict[str, Any]) -> None:
merged = dict(mirrors_config)
merged.update(values)
save_config_file("mirrors", merged)
mirrors_config.update(values)
# Defaults changed since last startup — push new mirrors to config
if stored_hash != current_defaults_hash:
_save_mirrors({
"AA_MIRROR_URLS": normalized_defaults,
"_AA_MIRRORS_DEFAULTS_HASH": current_defaults_hash,
})
return
# --- Legacy migration (only runs if hash already matches / first time) ---
# If already a proper list, just ensure it's non-empty.
if isinstance(raw_list, list):
normalized = _normalize_list([str(v) for v in raw_list])
if normalized:
return
save_config_file("mirrors", {"AA_MIRROR_URLS": _normalize_list(DEFAULT_AA_MIRRORS)})
_save_mirrors({
"AA_MIRROR_URLS": normalized_defaults,
"_AA_MIRRORS_DEFAULTS_HASH": current_defaults_hash,
})
return
# If saved as a string, convert to list.
@@ -525,17 +553,33 @@ def migrate_mirror_settings() -> None:
parts = [p.strip() for p in raw_list.split(",") if p.strip()]
normalized = _normalize_list(parts)
if normalized:
save_config_file("mirrors", {"AA_MIRROR_URLS": normalized})
_save_mirrors({
"AA_MIRROR_URLS": normalized,
"_AA_MIRRORS_DEFAULTS_HASH": current_defaults_hash,
})
return
save_config_file("mirrors", {"AA_MIRROR_URLS": _normalize_list(DEFAULT_AA_MIRRORS)})
_save_mirrors({
"AA_MIRROR_URLS": normalized_defaults,
"_AA_MIRRORS_DEFAULTS_HASH": current_defaults_hash,
})
return
# If there's legacy additional mirrors, seed the full list so the UI reflects reality.
# If there's legacy additional mirrors, seed the full list.
if isinstance(raw_additional, str) and raw_additional.strip():
additional_parts = [p.strip() for p in raw_additional.split(",") if p.strip()]
combined = _normalize_list(DEFAULT_AA_MIRRORS + additional_parts)
if combined:
save_config_file("mirrors", {"AA_MIRROR_URLS": combined})
_save_mirrors({
"AA_MIRROR_URLS": combined,
"_AA_MIRRORS_DEFAULTS_HASH": current_defaults_hash,
})
return
# No config at all yet — write defaults
_save_mirrors({
"AA_MIRROR_URLS": normalized_defaults,
"_AA_MIRRORS_DEFAULTS_HASH": current_defaults_hash,
})
def migrate_legacy_settings() -> None:
@@ -1092,6 +1136,18 @@ def update_settings(tab_name: str, values: Dict[str, Any]) -> Dict[str, Any]:
):
_apply_dns_settings(config_obj)
# Apply certificate validation changes live (network tab)
if (
config_obj is not None
and tab_name == "network"
and "CERTIFICATE_VALIDATION" in values_to_save
):
try:
from shelfmark.download.network import _apply_ssl_warning_suppression
_apply_ssl_warning_suppression()
except Exception as e:
logger.warning(f"Failed to apply certificate validation setting: {e}")
# Apply AA mirror settings changes live (mirrors tab)
aa_keys = {"AA_BASE_URL", "AA_MIRROR_URLS", "AA_ADDITIONAL_URLS"}
if (
+204 -99
View File
@@ -7,8 +7,13 @@ import threading
from typing import Any, Dict, List, Optional
from shelfmark.core.auth_modes import AUTH_SOURCE_BUILTIN, AUTH_SOURCE_SET
from shelfmark.core.activity_view_state_service import user_viewer_scope
from shelfmark.core.logger import setup_logger
from shelfmark.core.requests_service import (
from shelfmark.core.request_helpers import normalize_optional_positive_int
from shelfmark.core.models import QueueStatus
from shelfmark.core.request_validation import (
DELIVERY_STATE_NONE,
RequestStatus,
normalize_delivery_state,
normalize_policy_mode,
normalize_request_level,
@@ -62,38 +67,51 @@ ON download_requests (user_id, status, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_download_requests_status_created_at
ON download_requests (status, created_at DESC);
CREATE TABLE IF NOT EXISTS activity_log (
CREATE TABLE IF NOT EXISTS download_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER REFERENCES users(id) ON DELETE CASCADE,
item_type TEXT NOT NULL,
item_key TEXT NOT NULL,
task_id TEXT UNIQUE NOT NULL,
user_id INTEGER REFERENCES users(id) ON DELETE SET NULL,
username TEXT,
request_id INTEGER,
source_id TEXT,
origin TEXT NOT NULL,
source TEXT NOT NULL,
source_display_name TEXT,
title TEXT NOT NULL,
author TEXT,
format TEXT,
size TEXT,
preview TEXT,
content_type TEXT,
origin TEXT NOT NULL DEFAULT 'direct',
final_status TEXT NOT NULL,
snapshot_json TEXT NOT NULL,
terminal_at TIMESTAMP NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
status_message TEXT,
download_path TEXT,
queued_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
terminal_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_activity_log_user_terminal
ON activity_log (user_id, terminal_at DESC);
CREATE INDEX IF NOT EXISTS idx_download_history_user_status
ON download_history (user_id, final_status, terminal_at DESC);
CREATE INDEX IF NOT EXISTS idx_activity_log_lookup
ON activity_log (user_id, item_type, item_key, id DESC);
CREATE INDEX IF NOT EXISTS idx_download_history_recent
ON download_history (user_id, terminal_at DESC, id DESC);
CREATE TABLE IF NOT EXISTS activity_dismissals (
CREATE TABLE IF NOT EXISTS activity_view_state (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
viewer_scope TEXT NOT NULL,
item_type TEXT NOT NULL,
item_key TEXT NOT NULL,
activity_log_id INTEGER REFERENCES activity_log(id) ON DELETE SET NULL,
dismissed_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(user_id, item_type, item_key)
dismissed_at TIMESTAMP,
cleared_at TIMESTAMP,
UNIQUE(viewer_scope, item_type, item_key)
);
CREATE INDEX IF NOT EXISTS idx_activity_dismissals_user_dismissed_at
ON activity_dismissals (user_id, dismissed_at DESC);
CREATE INDEX IF NOT EXISTS idx_activity_view_state_history
ON activity_view_state (viewer_scope, dismissed_at DESC, id DESC)
WHERE dismissed_at IS NOT NULL AND cleared_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_activity_view_state_hidden
ON activity_view_state (viewer_scope, item_type, item_key)
WHERE dismissed_at IS NOT NULL;
"""
@@ -119,6 +137,14 @@ def sync_builtin_admin_user(
existing = user_db.get_user(username=normalized_username)
if existing:
existing_auth_source = str(existing.get("auth_source") or AUTH_SOURCE_BUILTIN).strip().lower()
if existing_auth_source != AUTH_SOURCE_BUILTIN:
logger.warning(
"Skipped builtin admin sync for username '%s' because it belongs to auth_source='%s'",
normalized_username,
existing_auth_source,
)
return
updates: dict[str, Any] = {}
if existing.get("password_hash") != normalized_hash:
updates["password_hash"] = normalized_hash
@@ -163,7 +189,7 @@ class UserDB:
conn.executescript(_CREATE_TABLES_SQL)
self._migrate_auth_source_column(conn)
self._migrate_request_delivery_columns(conn)
self._migrate_activity_tables(conn)
self._migrate_download_history_queued_at(conn)
conn.commit()
# WAL mode must be changed outside an open transaction.
conn.execute("PRAGMA journal_mode=WAL")
@@ -203,18 +229,11 @@ class UserDB:
if "last_failure_reason" not in column_names:
conn.execute("ALTER TABLE download_requests ADD COLUMN last_failure_reason TEXT")
conn.execute(
"""
UPDATE download_requests
SET delivery_state = 'unknown'
WHERE status = 'fulfilled' AND (delivery_state IS NULL OR TRIM(delivery_state) = '' OR delivery_state = 'none')
"""
)
conn.execute(
"""
UPDATE download_requests
SET delivery_state = 'none'
WHERE status != 'fulfilled' AND (delivery_state IS NULL OR TRIM(delivery_state) = '')
WHERE delivery_state IS NULL OR TRIM(delivery_state) = '' OR delivery_state IN ('unknown', 'available', 'done')
"""
)
conn.execute(
@@ -224,57 +243,16 @@ class UserDB:
WHERE delivery_state != 'none' AND delivery_updated_at IS NULL
"""
)
conn.execute(
"""
UPDATE download_requests
SET delivery_state = 'complete'
WHERE delivery_state = 'cleared'
"""
)
def _migrate_activity_tables(self, conn: sqlite3.Connection) -> None:
"""Ensure activity log and dismissal tables exist with current columns/indexes."""
conn.executescript(
"""
CREATE TABLE IF NOT EXISTS activity_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER REFERENCES users(id) ON DELETE CASCADE,
item_type TEXT NOT NULL,
item_key TEXT NOT NULL,
request_id INTEGER,
source_id TEXT,
origin TEXT NOT NULL,
final_status TEXT NOT NULL,
snapshot_json TEXT NOT NULL,
terminal_at TIMESTAMP NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_activity_log_user_terminal
ON activity_log (user_id, terminal_at DESC);
CREATE INDEX IF NOT EXISTS idx_activity_log_lookup
ON activity_log (user_id, item_type, item_key, id DESC);
CREATE TABLE IF NOT EXISTS activity_dismissals (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
item_type TEXT NOT NULL,
item_key TEXT NOT NULL,
activity_log_id INTEGER REFERENCES activity_log(id) ON DELETE SET NULL,
dismissed_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(user_id, item_type, item_key)
);
CREATE INDEX IF NOT EXISTS idx_activity_dismissals_user_dismissed_at
ON activity_dismissals (user_id, dismissed_at DESC);
"""
)
dismissal_columns = conn.execute("PRAGMA table_info(activity_dismissals)").fetchall()
dismissal_column_names = {str(col["name"]) for col in dismissal_columns}
if "activity_log_id" not in dismissal_column_names:
conn.execute("ALTER TABLE activity_dismissals ADD COLUMN activity_log_id INTEGER")
def _migrate_download_history_queued_at(self, conn: sqlite3.Connection) -> None:
"""Ensure download_history.queued_at exists for queue-time recording."""
columns = conn.execute("PRAGMA table_info(download_history)").fetchall()
column_names = {str(col["name"]) for col in columns}
if "queued_at" not in column_names:
conn.execute("ALTER TABLE download_history ADD COLUMN queued_at TIMESTAMP")
conn.execute(
"UPDATE download_history SET queued_at = CURRENT_TIMESTAMP WHERE queued_at IS NULL"
)
def create_user(
self,
@@ -380,6 +358,26 @@ class UserDB:
with self._lock:
conn = self._connect()
try:
request_rows = conn.execute(
"SELECT id FROM download_requests WHERE user_id = ?",
(user_id,),
).fetchall()
request_item_keys = [f"request:{row['id']}" for row in request_rows]
if request_item_keys:
placeholders = ",".join("?" for _ in request_item_keys)
conn.execute(
f"""
DELETE FROM activity_view_state
WHERE item_type = 'request'
AND item_key IN ({placeholders})
""",
request_item_keys,
)
conn.execute(
"DELETE FROM activity_view_state WHERE viewer_scope = ?",
(user_viewer_scope(user_id),),
)
conn.execute("UPDATE download_requests SET reviewed_by = NULL WHERE reviewed_by = ?", (user_id,))
conn.execute("DELETE FROM users WHERE id = ?", (user_id,))
conn.commit()
finally:
@@ -394,6 +392,19 @@ class UserDB:
finally:
conn.close()
def has_admin_with_password(self) -> bool:
"""Return True when at least one admin user with a password hash exists."""
conn = self._connect()
try:
row = conn.execute(
"SELECT 1 FROM users WHERE role = 'admin'"
" AND password_hash IS NOT NULL AND password_hash != ''"
" LIMIT 1",
).fetchone()
return row is not None
finally:
conn.close()
def get_user_settings(self, user_id: int) -> Dict[str, Any]:
"""Get per-user settings. Returns empty dict if none set."""
conn = self._connect()
@@ -468,13 +479,13 @@ class UserDB:
policy_mode: str,
book_data: Dict[str, Any],
release_data: Optional[Dict[str, Any]] = None,
status: str = "pending",
status: str = RequestStatus.PENDING,
source_hint: Optional[str] = None,
note: Optional[str] = None,
admin_note: Optional[str] = None,
reviewed_by: Optional[int] = None,
reviewed_at: Optional[str] = None,
delivery_state: str = "none",
delivery_state: str = DELIVERY_STATE_NONE,
delivery_updated_at: Optional[str] = None,
) -> Dict[str, Any]:
"""Create a download request row and return the created record."""
@@ -679,24 +690,8 @@ class UserDB:
if "content_type" in updates and not updates["content_type"]:
raise ValueError("content_type is required")
candidate_request_level = updates.get("request_level", current["request_level"])
candidate_release_data = (
updates["release_data"] if "release_data" in updates else current["release_data"]
)
candidate_status = updates.get("status", current["status"])
normalized_request_level = normalize_request_level(candidate_request_level)
normalized_candidate_status = normalize_request_status(candidate_status)
if normalized_request_level == "release" and candidate_release_data is None:
raise ValueError("request_level=release requires non-null release_data")
if (
normalized_request_level == "book"
and candidate_release_data is not None
and normalized_candidate_status != "fulfilled"
):
raise ValueError("request_level=book requires null release_data")
if "request_level" in updates:
updates["request_level"] = normalized_request_level
updates["request_level"] = normalize_request_level(updates["request_level"])
if "book_data" in updates:
if not isinstance(updates["book_data"], dict):
@@ -730,6 +725,116 @@ class UserDB:
finally:
conn.close()
def reopen_failed_request(
self,
request_id: int,
*,
failure_reason: Optional[str] = None,
) -> Optional[Dict[str, Any]]:
"""Reopen a failed fulfilled request so admins can re-approve it."""
normalized_failure_reason = None
if isinstance(failure_reason, str):
normalized_failure_reason = failure_reason.strip() or None
with self._lock:
conn = self._connect()
try:
current_row = conn.execute(
"SELECT * FROM download_requests WHERE id = ?",
(request_id,),
).fetchone()
current_request = self._parse_request_row(current_row)
if current_request is None:
return None
if current_request.get("status") != RequestStatus.FULFILLED:
return None
current_delivery_state = current_request.get("delivery_state", DELIVERY_STATE_NONE)
# Terminal hook callbacks can run before delivery-state sync persists "error".
# Allow reopening fulfilled requests unless they are already complete.
if current_delivery_state == QueueStatus.COMPLETE:
return None
if (
current_delivery_state not in {QueueStatus.ERROR, QueueStatus.CANCELLED}
and normalized_failure_reason is None
):
return None
conn.execute(
"""
UPDATE download_requests
SET status = 'pending',
delivery_state = 'none',
delivery_updated_at = NULL,
release_data = NULL,
last_failure_reason = ?,
reviewed_by = NULL,
reviewed_at = NULL
WHERE id = ?
""",
(normalized_failure_reason, request_id),
)
updated_row = conn.execute(
"SELECT * FROM download_requests WHERE id = ?",
(request_id,),
).fetchone()
conn.commit()
return self._parse_request_row(updated_row)
finally:
conn.close()
def rollback_request_fulfilment(
self,
request_id: int,
*,
release_data: Optional[Dict[str, Any]],
last_failure_reason: Optional[str] = None,
) -> Dict[str, Any]:
"""Restore a request to pending after fulfilment claimed it but queueing failed."""
with self._lock:
conn = self._connect()
try:
row = conn.execute(
"SELECT * FROM download_requests WHERE id = ?",
(request_id,),
).fetchone()
current = self._parse_request_row(row)
if current is None:
raise ValueError(f"Request {request_id} not found")
conn.execute(
"""
UPDATE download_requests
SET status = 'pending',
release_data = ?,
admin_note = NULL,
reviewed_by = NULL,
reviewed_at = NULL,
delivery_state = 'none',
delivery_updated_at = NULL,
last_failure_reason = ?
WHERE id = ?
""",
(
self._serialize_json(release_data, "release_data"),
last_failure_reason,
request_id,
),
)
updated_row = conn.execute(
"SELECT * FROM download_requests WHERE id = ?",
(request_id,),
).fetchone()
conn.commit()
parsed = self._parse_request_row(updated_row)
if parsed is None:
raise ValueError(f"Request {request_id} not found after rollback")
return parsed
finally:
conn.close()
def count_pending_requests(self) -> int:
"""Count all pending requests."""
conn = self._connect()
+25
View File
@@ -1,8 +1,11 @@
"""Shared utility functions for the Shelfmark."""
import base64
import importlib
import os
import re
from threading import Lock
from types import ModuleType
from pathlib import Path
from typing import Optional
from urllib.parse import urlparse
@@ -54,6 +57,28 @@ def normalize_http_url(
return normalized
_xmlrpc_patch_lock = Lock()
_xmlrpc_patch_applied = False
def get_hardened_xmlrpc_client() -> ModuleType:
"""Return ``xmlrpc.client`` after best-effort defusedxml monkey patching."""
global _xmlrpc_patch_applied
if not _xmlrpc_patch_applied:
with _xmlrpc_patch_lock:
if not _xmlrpc_patch_applied:
try:
from defusedxml.xmlrpc import monkey_patch
monkey_patch()
_xmlrpc_patch_applied = True
except Exception:
# Keep runtime behavior unchanged if defusedxml is unavailable.
_xmlrpc_patch_applied = False
return importlib.import_module("xmlrpc.client")
def normalize_base_path(value: Optional[str]) -> str:
"""Normalize a URL base path for reverse proxy subpath deployments."""
if not isinstance(value, str):
+2 -1
View File
@@ -18,6 +18,7 @@ from urllib.parse import urlparse
import requests
from shelfmark.core.config import config
from shelfmark.download.network import get_ssl_verify
from shelfmark.core.logger import setup_logger
from shelfmark.core.utils import normalize_http_url
from shelfmark.download.clients import (
@@ -110,7 +111,7 @@ class DelugeClient(DownloadClient):
"params": list(params),
}
response = self._session.post(self._rpc_url, json=payload, timeout=timeout)
response = self._session.post(self._rpc_url, json=payload, timeout=timeout, verify=get_ssl_verify(self._rpc_url))
response.raise_for_status()
data = response.json()
+3 -1
View File
@@ -12,6 +12,7 @@ import requests
from shelfmark.core.config import config
from shelfmark.core.logger import setup_logger
from shelfmark.core.utils import normalize_http_url
from shelfmark.download.network import get_ssl_verify
from shelfmark.download.clients import (
DownloadClient,
DownloadStatus,
@@ -79,6 +80,7 @@ class NZBGetClient(DownloadClient):
headers={"Content-Type": "application/json"},
auth=(self.username, self.password),
timeout=30,
verify=get_ssl_verify(rpc_url),
)
response.raise_for_status()
@@ -135,7 +137,7 @@ class NZBGetClient(DownloadClient):
try:
# Fetch NZB content from the URL (handles Prowlarr proxy redirects)
logger.debug(f"Fetching NZB from: {url}")
response = requests.get(url, timeout=30)
response = requests.get(url, timeout=30, verify=get_ssl_verify(url))
response.raise_for_status()
nzb_content = base64.b64encode(response.content).decode('ascii')
+2 -24
View File
@@ -1,13 +1,13 @@
"""qBittorrent download client for Prowlarr integration."""
import time
from pathlib import Path
from types import SimpleNamespace
from typing import Optional, Tuple
from shelfmark.core.config import config
from shelfmark.core.logger import setup_logger
from shelfmark.core.utils import normalize_http_url
from shelfmark.download.network import get_ssl_verify
from shelfmark.download.clients import (
DownloadClient,
DownloadStatus,
@@ -137,6 +137,7 @@ class QBittorrentClient(DownloadClient):
host=self._base_url,
username=config.get("QBITTORRENT_USERNAME", ""),
password=config.get("QBITTORRENT_PASSWORD", ""),
VERIFY_WEBUI_CERTIFICATE=get_ssl_verify(self._base_url),
)
self._category = config.get("QBITTORRENT_CATEGORY", "books")
self._download_dir = config.get("QBITTORRENT_DOWNLOAD_DIR", "")
@@ -503,37 +504,14 @@ class QBittorrentClient(DownloadClient):
Centralizes the logic shared by `get_status()` and `get_download_path()`:
- accept `content_path` only when it's not equal to `save_path`
- when the torrent is complete and both `content_path` and `save_path` are present,
prefer a path rooted at `save_path` to avoid races where qBittorrent briefly reports
a temp/incomplete `content_path` and then moves the payload
- otherwise derive via properties+files
- finally fall back to `save_path + name`
"""
torrent_progress = getattr(torrent, "progress", 0.0)
try:
progress = float(torrent_progress)
except (TypeError, ValueError):
progress = 0.0
# Prefer content_path, but treat content_path == save_path as invalid.
content_path = getattr(torrent, "content_path", "")
save_path = getattr(torrent, "save_path", "")
if content_path and (not save_path or str(content_path) != str(save_path)):
# When using a temp/incomplete directory, qBittorrent can briefly keep reporting
# `content_path` under that temp path right at completion, then move the files
# into `save_path`. Returning the temp path can race with that move.
if save_path and progress >= 1.0:
# Use the basename of content_path under save_path (works for single-file
# torrents and multi-file torrents where content_path is a top-level dir).
try:
content_basename = str(Path(str(content_path)).name)
except Exception:
content_basename = ""
rooted = self._build_path(str(save_path), content_basename)
if rooted:
return rooted
return str(content_path)
download_id = getattr(torrent, "hash", "")
+28 -12
View File
@@ -4,12 +4,14 @@ rTorrent download client for Prowlarr integration.
Uses xmlrpc to communicate with rTorrent's RPC interface.
"""
from typing import Optional, Tuple
import ssl
from typing import Any, Optional, Tuple
from urllib.parse import urlparse
from shelfmark.core.config import config
from shelfmark.core.logger import setup_logger
from shelfmark.core.utils import normalize_http_url
from shelfmark.core.utils import normalize_http_url, get_hardened_xmlrpc_client
from shelfmark.download.network import get_ssl_verify
from shelfmark.download.clients import (
DownloadClient,
DownloadStatus,
@@ -22,6 +24,21 @@ from shelfmark.download.clients.torrent_utils import (
logger = setup_logger(__name__)
def _create_rtorrent_server_proxy(url: str) -> Any:
"""Create an XML-RPC ServerProxy honoring certificate validation mode."""
xmlrpc_client = get_hardened_xmlrpc_client()
verify = get_ssl_verify(url)
if url.startswith("https://") and not verify:
ssl_context = ssl.create_default_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE
transport = xmlrpc_client.SafeTransport(context=ssl_context)
return xmlrpc_client.ServerProxy(url, transport=transport)
return xmlrpc_client.ServerProxy(url)
@register_client("torrent")
class RTorrentClient(DownloadClient):
"""rTorrent download client using xmlrpc."""
@@ -31,8 +48,6 @@ class RTorrentClient(DownloadClient):
def __init__(self):
"""Initialize rTorrent client with settings from config."""
from xmlrpc.client import ServerProxy
raw_url = config.get("RTORRENT_URL", "")
if not raw_url:
raise ValueError("RTORRENT_URL is required")
@@ -50,7 +65,7 @@ class RTorrentClient(DownloadClient):
f"{parsed.scheme}://{username}:{password}@{parsed.netloc}{parsed.path}"
)
self._rpc = ServerProxy(self._base_url)
self._rpc = _create_rtorrent_server_proxy(self._base_url)
self._download_dir = config.get("RTORRENT_DOWNLOAD_DIR", "")
self._label = config.get("RTORRENT_LABEL", "")
@@ -105,7 +120,7 @@ class RTorrentClient(DownloadClient):
download_dir = self._download_dir or self._get_download_dir()
if download_dir:
logger.debug(f"Setting rTorrent download directory: {download_dir}")
commands.append(f"d.directory_base.set={download_dir}")
commands.append(f"d.directory.set={download_dir}")
if torrent_info.torrent_data:
logger.debug(f"Adding torrent data directly to rTorrent for: {name} with commands: {commands} with data size: {len(torrent_info.torrent_data)}")
@@ -141,10 +156,9 @@ class RTorrentClient(DownloadClient):
try:
# rtorrent is somehow case sensitive and requires uppercase hashes for look
download_id = download_id.upper()
torrent_list = self._rpc.d.multicall.filtered(
all_torrents = self._rpc.d.multicall2(
"",
"",
"default",
f"equal={{d.hash=,cat={download_id}}}",
"d.hash=",
"d.state=",
"d.completed_bytes=",
@@ -154,6 +168,7 @@ class RTorrentClient(DownloadClient):
"d.custom1=",
"d.complete=",
)
torrent_list = [t for t in all_torrents if t and t[0] == download_id]
logger.debug(f"Fetched torrent status from rTorrent for: {download_id} - {torrent_list}")
if not torrent_list:
logger.warning(f"Torrent not found in rTorrent: {download_id}")
@@ -312,12 +327,13 @@ class RTorrentClient(DownloadClient):
try:
# rTorrent is case sensitive for hashes; use uppercase as in get_status()
download_hash = download_id.upper()
details = self._rpc.d.multicall.filtered(
all_torrents = self._rpc.d.multicall2(
"",
"default",
f"equal={{d.hash=,cat={download_hash}}}",
"",
"d.hash=",
"d.base_path=",
)
details = [t[1:] for t in all_torrents if t and t[0] == download_hash]
if not details:
return None
path = details[0][0]
+4 -3
View File
@@ -12,6 +12,7 @@ import requests
from shelfmark.core.config import config
from shelfmark.core.logger import setup_logger
from shelfmark.core.utils import normalize_http_url
from shelfmark.download.network import get_ssl_verify
from shelfmark.download.clients import (
DownloadClient,
DownloadStatus,
@@ -148,7 +149,7 @@ class SABnzbdClient(DownloadClient):
if params:
request_params.update(params)
response = requests.get(api_url, params=request_params, timeout=30)
response = requests.get(api_url, params=request_params, timeout=30, verify=get_ssl_verify(api_url))
response.raise_for_status()
result = response.json()
@@ -177,7 +178,7 @@ class SABnzbdClient(DownloadClient):
}
files = {"name": (filename, nzb_content, "application/x-nzb")}
response = requests.post(api_url, params=request_params, files=files, timeout=30)
response = requests.post(api_url, params=request_params, files=files, timeout=30, verify=get_ssl_verify(api_url))
response.raise_for_status()
result = response.json()
@@ -190,7 +191,7 @@ class SABnzbdClient(DownloadClient):
def _fetch_nzb_content(self, url: str) -> bytes:
"""Fetch NZB content, including Prowlarr auth headers when appropriate."""
headers = self._get_prowlarr_headers(url)
response = requests.get(url, timeout=30, headers=headers)
response = requests.get(url, timeout=30, headers=headers, verify=get_ssl_verify(url))
response.raise_for_status()
return response.content
+61 -9
View File
@@ -1,5 +1,6 @@
"""Shared download client settings registration."""
from contextlib import contextmanager
from typing import Any, Dict, Optional
from shelfmark.core.settings_registry import (
@@ -11,11 +12,40 @@ from shelfmark.core.settings_registry import (
SelectField,
TagListField,
)
from shelfmark.core.utils import normalize_http_url
from shelfmark.core.utils import normalize_http_url, get_hardened_xmlrpc_client
from shelfmark.download.network import get_ssl_verify
# ==================== Test Connection Callbacks ====================
@contextmanager
def _transmission_session_verify_override(url: str):
"""Ensure transmission-rpc constructor uses the configured TLS verify mode."""
verify = get_ssl_verify(url)
if verify:
yield
return
try:
import transmission_rpc.client as transmission_rpc_client
except Exception:
yield
return
original_session_factory = transmission_rpc_client.requests.Session
def _session_factory(*args: Any, **kwargs: Any) -> Any:
session = original_session_factory(*args, **kwargs)
session.verify = False
return session
transmission_rpc_client.requests.Session = _session_factory
try:
yield
finally:
transmission_rpc_client.requests.Session = original_session_factory
def _test_qbittorrent_connection(current_values: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
"""Test the qBittorrent connection using current form values."""
from shelfmark.core.config import config
@@ -36,7 +66,7 @@ def _test_qbittorrent_connection(current_values: Optional[Dict[str, Any]] = None
if not url:
return {"success": False, "message": "qBittorrent URL is invalid"}
client = Client(host=url, username=username, password=password)
client = Client(host=url, username=username, password=password, VERIFY_WEBUI_CERTIFICATE=get_ssl_verify(url))
client.auth_log_in()
api_version = client.app.web_api_version
return {"success": True, "message": f"Connected to qBittorrent (API v{api_version})"}
@@ -81,17 +111,25 @@ def _test_transmission_connection(current_values: Optional[Dict[str, Any]] = Non
"protocol": protocol,
}
try:
client = Client(**client_kwargs)
with _transmission_session_verify_override(url):
client = Client(**client_kwargs)
except TypeError as e:
if "protocol" not in str(e):
raise
client_kwargs.pop("protocol", None)
client = Client(**client_kwargs)
with _transmission_session_verify_override(url):
client = Client(**client_kwargs)
if protocol == "https" and hasattr(client, "protocol"):
try:
setattr(client, "protocol", protocol)
except Exception:
pass
# Keep session verify aligned for subsequent calls beyond constructor bootstrap.
http_session = getattr(client, "_http_session", None)
if http_session is not None:
http_session.verify = get_ssl_verify(url)
session = client.get_session()
version = session.version
return {"success": True, "message": f"Connected to Transmission {version}"}
@@ -151,7 +189,7 @@ def _test_deluge_connection(current_values: Optional[Dict[str, Any]] = None) ->
def rpc_call(session: requests.Session, rpc_id: int, method: str, *params: Any) -> Any:
payload = {"id": rpc_id, "method": method, "params": list(params)}
resp = session.post(rpc_url, json=payload, timeout=15)
resp = session.post(rpc_url, json=payload, timeout=15, verify=get_ssl_verify(rpc_url))
resp.raise_for_status()
data = resp.json()
if data.get("error"):
@@ -214,8 +252,8 @@ def _test_deluge_connection(current_values: Optional[Dict[str, Any]] = None) ->
def _test_rtorrent_connection(current_values: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
"""Test the rTorrent connection using current form values."""
from shelfmark.core.config import config
import ssl
from urllib.parse import urlparse
from xmlrpc.client import ServerProxy
current_values = current_values or {}
@@ -231,12 +269,26 @@ def _test_rtorrent_connection(current_values: Optional[Dict[str, Any]] = None) -
return {"success": False, "message": "rTorrent URL is invalid"}
try:
xmlrpc_client = get_hardened_xmlrpc_client()
# Add HTTP auth to URL if credentials provided
if username and password:
parsed = urlparse(url)
url = f"{parsed.scheme}://{username}:{password}@{parsed.netloc}{parsed.path}"
rpc = ServerProxy(url.rstrip("/"))
rpc_url = url.rstrip("/")
verify = get_ssl_verify(rpc_url)
if rpc_url.startswith("https://") and not verify:
ssl_context = ssl.create_default_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE
rpc = xmlrpc_client.ServerProxy(
rpc_url,
transport=xmlrpc_client.SafeTransport(context=ssl_context),
)
else:
rpc = xmlrpc_client.ServerProxy(rpc_url)
version = rpc.system.client_version()
return {"success": True, "message": f"Connected to rTorrent {version}"}
except Exception as e:
@@ -264,7 +316,7 @@ def _test_nzbget_connection(current_values: Optional[Dict[str, Any]] = None) ->
try:
rpc_url = f"{url.rstrip('/')}/jsonrpc"
payload = {"jsonrpc": "2.0", "method": "status", "params": [], "id": 1}
response = requests.post(rpc_url, json=payload, auth=(username, password), timeout=30)
response = requests.post(rpc_url, json=payload, auth=(username, password), timeout=30, verify=get_ssl_verify(rpc_url))
response.raise_for_status()
result = response.json()
if "error" in result and result["error"]:
@@ -301,7 +353,7 @@ def _test_sabnzbd_connection(current_values: Optional[Dict[str, Any]] = None) ->
try:
api_url = f"{url.rstrip('/')}/api"
params = {"apikey": api_key, "mode": "version", "output": "json"}
response = requests.get(api_url, params=params, timeout=30)
response = requests.get(api_url, params=params, timeout=30, verify=get_ssl_verify(api_url))
response.raise_for_status()
result = response.json()
version = result.get("version", "unknown")
+3 -2
View File
@@ -11,6 +11,7 @@ import requests
from shelfmark.core.config import config
from shelfmark.core.logger import setup_logger
from shelfmark.download.network import get_ssl_verify
logger = setup_logger(__name__)
@@ -88,7 +89,7 @@ def extract_torrent_info(
# Use allow_redirects=False to handle magnet link redirects manually
# Some indexers redirect download URLs to magnet links
resp = requests.get(url, timeout=30, allow_redirects=False, headers=headers)
resp = requests.get(url, timeout=30, allow_redirects=False, headers=headers, verify=get_ssl_verify(url))
# Check if this is a redirect to a magnet link
if resp.status_code in (301, 302, 303, 307, 308):
@@ -103,7 +104,7 @@ def extract_torrent_info(
)
# Not a magnet redirect, follow it manually
logger.debug(f"Following redirect to: {redirect_url[:80]}...")
resp = requests.get(redirect_url, timeout=30, headers=headers)
resp = requests.get(redirect_url, timeout=30, headers=headers, verify=get_ssl_verify(redirect_url))
resp.raise_for_status()
torrent_data = resp.content
+52 -3
View File
@@ -4,12 +4,14 @@ Transmission download client for Prowlarr integration.
Uses the transmission-rpc library to communicate with Transmission's RPC API.
"""
from typing import Optional, Tuple
from contextlib import contextmanager
from typing import Any, Iterator, Optional, Tuple
from shelfmark.core.config import config
from shelfmark.core.logger import setup_logger
from shelfmark.core.utils import normalize_http_url
from shelfmark.download.network import get_ssl_verify
from shelfmark.download.clients import (
DownloadClient,
DownloadStatus,
@@ -23,6 +25,50 @@ from shelfmark.download.clients.torrent_utils import (
logger = setup_logger(__name__)
@contextmanager
def _transmission_session_verify_override(url: str) -> Iterator[None]:
"""Temporarily override transmission-rpc's session factory when verify is disabled.
transmission-rpc performs an RPC call inside Client.__init__, so verify must be
set before the client is constructed.
"""
verify = get_ssl_verify(url)
if verify:
yield
return
try:
import transmission_rpc.client as transmission_rpc_client
except Exception:
# If internals differ, gracefully fall back to default behavior.
yield
return
original_session_factory = transmission_rpc_client.requests.Session
def _session_factory(*args: Any, **kwargs: Any) -> Any:
session = original_session_factory(*args, **kwargs)
session.verify = False
return session
transmission_rpc_client.requests.Session = _session_factory
try:
yield
finally:
transmission_rpc_client.requests.Session = original_session_factory
def _apply_transmission_ssl_verify(client: Any, url: str) -> None:
"""Apply global certificate validation policy to transmission-rpc client."""
session = getattr(client, "_http_session", None)
if session is None:
return
try:
session.verify = get_ssl_verify(url)
except Exception as e:
logger.debug("Unable to apply Transmission TLS verify setting: %s", e)
@register_client("torrent")
class TransmissionClient(DownloadClient):
"""Transmission download client using transmission-rpc library."""
@@ -57,19 +103,22 @@ class TransmissionClient(DownloadClient):
"protocol": protocol,
}
try:
self._client = Client(**client_kwargs)
with _transmission_session_verify_override(url):
self._client = Client(**client_kwargs)
except TypeError as e:
# Older transmission-rpc versions may not accept protocol as a kwarg.
if "protocol" not in str(e):
raise
client_kwargs.pop("protocol", None)
self._client = Client(**client_kwargs)
with _transmission_session_verify_override(url):
self._client = Client(**client_kwargs)
# Some versions expose protocol as an attribute rather than kwarg.
if protocol == "https" and hasattr(self._client, "protocol"):
try:
setattr(self._client, "protocol", protocol)
except Exception:
pass
_apply_transmission_ssl_verify(self._client, url)
self._category = config.get("TRANSMISSION_CATEGORY", "books")
self._download_dir = config.get("TRANSMISSION_DOWNLOAD_DIR", "")
+91 -32
View File
@@ -52,6 +52,20 @@ def _call_and_capture(func: Callable[..., T], args: tuple[Any, ...], kwargs: dic
return False, exc
def _must_avoid_gevent_threadpool(func: Callable[..., Any]) -> bool:
"""Return True when `func` is unsafe to execute inside gevent's threadpool."""
if not _use_gevent_threadpool() or not _gevent_monkey:
return False
# gevent.subprocess requires child watchers on the default event loop.
# Executing patched subprocess functions in a worker thread can raise:
# "TypeError: child watchers are only available on the default loop".
if _gevent_monkey.is_object_patched("subprocess", "run") and func is subprocess.run:
return True
return False
def run_blocking_io(func: Callable[..., T], *args: Any, **kwargs: Any) -> T:
"""Run blocking I/O in a native thread when under gevent.
@@ -60,6 +74,9 @@ def run_blocking_io(func: Callable[..., T], *args: Any, **kwargs: Any) -> T:
collision retries, EXDEV for cross-device moves). Capture and re-raise in the
caller to avoid noisy, misleading tracebacks.
"""
if _must_avoid_gevent_threadpool(func):
return func(*args, **kwargs)
if _use_gevent_threadpool():
ok, result = _get_io_threadpool().apply(_call_and_capture, (func, args, kwargs))
if ok:
@@ -200,6 +217,28 @@ def _perform_nfs_fallback(source: Path, dest: Path, is_move: bool) -> None:
raise
def _is_enoent_error(error: Exception) -> bool:
return isinstance(error, FileNotFoundError) or (
isinstance(error, OSError) and error.errno == errno.ENOENT
)
def _can_use_partial_copy_after_enoent(
temp_path: Optional[Path],
expected_size: int,
action: str,
) -> bool:
"""Recover when copy2 writes bytes but fails while copying source metadata."""
if not temp_path or not run_blocking_io(temp_path.exists):
return False
try:
_verify_transfer_size(temp_path, expected_size, action)
return True
except Exception:
return False
def _claim_destination(path: Path) -> bool:
"""Atomically claim a destination path by creating a placeholder file.
@@ -224,10 +263,12 @@ def _hardlink_not_supported(error: OSError) -> bool:
return err in {
errno.EXDEV,
errno.EMLINK,
errno.EIO,
errno.EPERM,
errno.EACCES,
getattr(errno, "ENOTSUP", errno.EPERM),
getattr(errno, "EOPNOTSUPP", errno.EPERM),
getattr(errno, "ENOSYS", errno.EPERM),
errno.EINVAL,
}
@@ -248,36 +289,33 @@ def _publish_temp_file(temp_path: Path, dest_path: Path) -> bool:
Returns True on success, False if the destination already exists.
"""
try:
run_blocking_io(os.link, str(temp_path), str(dest_path))
run_blocking_io(temp_path.unlink, missing_ok=True)
return True
except FileExistsError:
claimed = _claim_destination(dest_path)
if not claimed:
return False
except OSError as e:
try:
# Publish by renaming the fully-written temp file into place. This gives
# watchers an IN_MOVED_TO-style event on the final path instead of relying
# on hardlink support in the destination filesystem.
run_blocking_io(os.replace, str(temp_path), str(dest_path))
# Best-effort nudge for watchers that only react to close-write on the
# final filename rather than rename/move events.
try:
fd = run_blocking_io(os.open, str(dest_path), os.O_WRONLY)
run_blocking_io(os.close, fd)
except OSError:
pass
return True
except Exception as e:
if _is_permission_error(e):
log_transfer_permission_context(
"publish_hardlink",
"publish_replace",
source=temp_path,
dest=dest_path,
error=e,
)
if _hardlink_not_supported(e):
logger.debug(
"Hardlink publish unsupported; falling back to claim+replace: %s -> %s (%s)",
temp_path,
dest_path,
e,
)
claimed = _claim_destination(dest_path)
if not claimed:
return False
try:
run_blocking_io(os.replace, str(temp_path), str(dest_path))
except Exception:
run_blocking_io(dest_path.unlink, missing_ok=True)
raise
return True
run_blocking_io(dest_path.unlink, missing_ok=True)
raise
@@ -359,6 +397,16 @@ def atomic_move(source_path: Path, dest_path: Path, max_attempts: int = 100) ->
copy_error,
)
_perform_nfs_fallback(source_path, temp_path, is_move=False)
elif _is_enoent_error(copy_error) and _can_use_partial_copy_after_enoent(
temp_path,
expected_size,
"move",
):
logger.warning(
"Source vanished during move-copy metadata step; preserving copied data: %s -> %s",
source_path,
temp_path,
)
else:
raise
@@ -449,14 +497,15 @@ def atomic_hardlink(source_path: Path, dest_path: Path, max_attempts: int = 100)
except FileExistsError:
continue
except OSError as e:
if _is_permission_error(e) or e.errno in (errno.EXDEV, errno.EMLINK):
if _is_permission_error(e):
log_transfer_permission_context(
"atomic_hardlink",
source=source_path,
dest=try_path,
error=e,
)
permission_error = _is_permission_error(e)
if permission_error:
log_transfer_permission_context(
"atomic_hardlink",
source=source_path,
dest=try_path,
error=e,
)
if permission_error or _hardlink_not_supported(e):
logger.debug(
"Hardlink failed (%s), falling back to copy: %s -> %s",
e,
@@ -472,7 +521,7 @@ def atomic_hardlink(source_path: Path, dest_path: Path, max_attempts: int = 100)
def atomic_copy(source_path: Path, dest_path: Path, max_attempts: int = 100) -> Path:
"""Copy a file with atomic collision detection.
Uses a temp file in the destination directory and publishes it atomically,
Uses a temp file in the destination directory and publishes it via rename,
avoiding partial files on failure.
Args:
@@ -525,6 +574,16 @@ def atomic_copy(source_path: Path, dest_path: Path, max_attempts: int = 100) ->
fallback_error,
)
raise e from fallback_error
elif _is_enoent_error(e) and _can_use_partial_copy_after_enoent(
temp_path,
expected_size,
"copy",
):
logger.warning(
"Source vanished during copy2 metadata step; preserving copied data: %s -> %s",
source_path,
temp_path,
)
else:
raise
+4 -3
View File
@@ -11,7 +11,7 @@ import requests
from tqdm import tqdm
from shelfmark.download import network
from shelfmark.download.network import get_proxies
from shelfmark.download.network import get_proxies, get_ssl_verify
from shelfmark.core.config import config as app_config
from shelfmark.core.logger import setup_logger
@@ -261,6 +261,7 @@ def html_get_page(
cookies=cookies,
headers=headers,
allow_redirects=allow_redirects,
verify=get_ssl_verify(current_url),
)
if is_aa_url and response.is_redirect:
@@ -403,7 +404,7 @@ def download_url(
logger.info(f"Downloading: {current_url} (attempt {attempt + 1}/{MAX_DOWNLOAD_RETRIES})")
# Try with CF cookies/UA if available
cookies = _apply_cf_bypass(current_url, headers)
response = requests.get(current_url, stream=True, proxies=get_proxies(current_url), timeout=REQUEST_TIMEOUT, cookies=cookies, headers=headers)
response = requests.get(current_url, stream=True, proxies=get_proxies(current_url), timeout=REQUEST_TIMEOUT, cookies=cookies, headers=headers, verify=get_ssl_verify(current_url))
response.raise_for_status()
if status_callback:
@@ -514,7 +515,7 @@ def _try_resume(
cookies = _apply_cf_bypass(url, resume_headers)
response = requests.get(
url, stream=True, proxies=get_proxies(url), timeout=REQUEST_TIMEOUT,
headers=resume_headers, cookies=cookies
headers=resume_headers, cookies=cookies, verify=get_ssl_verify(url)
)
# Check resume support
+57 -2
View File
@@ -90,6 +90,59 @@ def get_proxies(url: str = "") -> dict:
return {}
def get_ssl_verify(url: str = "") -> bool:
"""Return the ``verify`` value for outbound requests based on the
CERTIFICATE_VALIDATION setting.
- ``enabled`` → always ``True``
- ``disabled_local`` → ``False`` for local/private addresses, ``True`` otherwise
- ``disabled`` → always ``False``
"""
mode = app_config.get("CERTIFICATE_VALIDATION", "enabled")
if mode == "disabled":
return False
if mode == "disabled_local" and url:
try:
parsed = urllib.parse.urlparse(url)
hostname = parsed.hostname or ""
if hostname and _is_local_address(hostname):
return False
except Exception:
pass
return True
_ssl_warnings_suppressed = False
def _apply_ssl_warning_suppression() -> None:
"""Suppress or restore urllib3 InsecureRequestWarning based on the
CERTIFICATE_VALIDATION setting.
Called once at init and again whenever the setting changes via the UI.
Only modifies warning filters when the mode is not 'enabled', so the
default case is a complete no-op (zero behavioural change for users who
never touch the setting).
"""
global _ssl_warnings_suppressed # noqa: PLW0603
import urllib3
mode = app_config.get("CERTIFICATE_VALIDATION", "enabled")
if mode in ("disabled", "disabled_local"):
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
_ssl_warnings_suppressed = True
logger.debug("SSL warnings suppressed (certificate validation: %s)", mode)
elif _ssl_warnings_suppressed:
import warnings
warnings.simplefilter("default", urllib3.exceptions.InsecureRequestWarning)
_ssl_warnings_suppressed = False
logger.debug("SSL warnings restored (certificate validation: enabled)")
# DNS state - authoritative values managed by this module
# Other modules should use get_dns_config() to read these
CUSTOM_DNS: List[str] = []
@@ -418,7 +471,8 @@ class DoHResolver:
self.base_url,
params=params,
proxies=get_proxies(self.base_url),
timeout=10 # Increased from 5s to handle slow network conditions
timeout=10, # Increased from 5s to handle slow network conditions
verify=get_ssl_verify(self.base_url),
)
response.raise_for_status()
@@ -940,7 +994,7 @@ def _initialize_aa_state() -> None:
logger.debug(f"AA_BASE_URL: auto, checking available urls {_aa_urls}")
for i, url in enumerate(_aa_urls):
try:
response = requests.get(url, proxies=get_proxies(url), timeout=3)
response = requests.get(url, proxies=get_proxies(url), timeout=3, verify=get_ssl_verify(url))
if response.status_code == 200:
_current_aa_url_index = i
_aa_base_url = url
@@ -1036,6 +1090,7 @@ def init(force: bool = False) -> None:
try:
init_dns(force=force)
init_aa(force=force)
_apply_ssl_warning_suppression()
# Only set flag AFTER work completes successfully
_initialized = True
except Exception:
+182 -166
View File
@@ -16,14 +16,17 @@ from typing import Any, Dict, List, Optional, Tuple
from shelfmark.core.config import config
from shelfmark.core.logger import setup_logger
from shelfmark.core.models import BookInfo, DownloadTask, QueueStatus, SearchFilters, SearchMode
from shelfmark.core.models import DownloadTask, QueueStatus, SearchMode
from shelfmark.core.queue import book_queue
from shelfmark.core.utils import transform_cover_url, is_audiobook as check_audiobook
from shelfmark.config import env as env_config
from shelfmark.download.fs import run_blocking_io
from shelfmark.download.postprocess.pipeline import is_torrent_source, safe_cleanup_path
from shelfmark.download.postprocess.router import post_process_download
from shelfmark.release_sources import direct_download, get_handler, get_source_display_name
from shelfmark.release_sources.direct_download import SearchUnavailable
from shelfmark.release_sources import (
get_handler,
get_source_display_name,
)
logger = setup_logger(__name__)
@@ -56,26 +59,6 @@ _last_activity: Dict[str, float] = {}
_last_status_event: Dict[str, Tuple[str, Optional[str]]] = {}
STALL_TIMEOUT = 300 # 5 minutes without progress/status update = stalled
def search_books(query: str, filters: SearchFilters) -> List[Dict[str, Any]]:
"""Search for books matching the query."""
try:
books = direct_download.search_books(query, filters)
return [_book_info_to_dict(book) for book in books]
except SearchUnavailable:
raise
except Exception as e:
logger.error_trace(f"Error searching books: {e}")
raise
def get_book_info(book_id: str) -> Optional[Dict[str, Any]]:
"""Get detailed information for a specific book."""
try:
book = direct_download.get_book_info(book_id)
return _book_info_to_dict(book)
except Exception as e:
logger.error_trace(f"Error getting book info: {e}")
raise
def _is_plain_email_address(value: str) -> bool:
parsed = parseaddr(value or "")[1]
return bool(parsed) and "@" in parsed and parsed == value
@@ -96,76 +79,17 @@ def _resolve_email_destination(
return None, "Configured email recipient is invalid"
return None, None
def queue_book(
book_id: str,
priority: int = 0,
source: str = "direct_download",
user_id: Optional[int] = None,
username: Optional[str] = None,
) -> Tuple[bool, Optional[str]]:
"""Add a book to the download queue. Returns (success, error_message)."""
try:
book_info = direct_download.get_book_info(book_id, fetch_download_count=False)
if not book_info:
error_msg = f"Could not fetch book info for {book_id}"
logger.warning(error_msg)
return False, error_msg
books_output_mode = str(
config.get("BOOKS_OUTPUT_MODE", "folder", user_id=user_id) or "folder"
).strip().lower()
is_audiobook = check_audiobook(book_info.content)
# Capture output mode at queue time so tasks aren't affected if settings change later.
output_mode = "folder" if is_audiobook else books_output_mode
output_args: Dict[str, Any] = {}
if output_mode == "email" and not is_audiobook:
email_to, email_error = _resolve_email_destination(user_id=user_id)
if email_error:
return False, email_error
if email_to:
output_args = {"to": email_to}
# Create a source-agnostic download task
task = DownloadTask(
task_id=book_id,
source=source,
title=book_info.title,
author=book_info.author,
format=book_info.format,
size=book_info.size,
preview=book_info.preview,
content_type=book_info.content,
search_mode=SearchMode.DIRECT,
output_mode=output_mode,
output_args=output_args,
priority=priority,
user_id=user_id,
username=username,
)
if not book_queue.add(task):
logger.info(f"Book already in queue: {book_info.title}")
return False, "Book is already in the download queue"
logger.info(f"Book queued with priority {priority}: {book_info.title}")
# Broadcast status update via WebSocket
if ws_manager:
ws_manager.broadcast_status_update(queue_status())
return True, None
except SearchUnavailable as e:
error_msg = f"Search service unavailable: {e}"
logger.warning(error_msg)
return False, error_msg
except Exception as e:
error_msg = f"Error queueing book: {e}"
logger.error_trace(error_msg)
return False, error_msg
def _parse_release_search_mode(value: Any) -> SearchMode:
if isinstance(value, SearchMode):
return value
if value is None:
return SearchMode.UNIVERSAL
if isinstance(value, str):
try:
return SearchMode(value.strip().lower())
except ValueError as exc:
raise ValueError(f"Invalid search_mode: {value}") from exc
raise ValueError(f"Invalid search_mode: {value}")
def queue_release(
@@ -176,12 +100,13 @@ def queue_release(
) -> Tuple[bool, Optional[str]]:
"""Add a release to the download queue. Returns (success, error_message)."""
try:
source = release_data.get('source', 'direct_download')
source = release_data['source']
extra = release_data.get('extra', {})
raw_request_id = release_data.get('_request_id')
request_id: Optional[int] = None
if isinstance(raw_request_id, int) and raw_request_id > 0:
request_id = raw_request_id
search_mode = _parse_release_search_mode(release_data.get("search_mode"))
# Get author, year, preview, and content_type from top-level (preferred) or extra (fallback)
author = release_data.get('author') or extra.get('author')
@@ -234,7 +159,7 @@ def queue_release(
series_name=series_name,
series_position=series_position,
subtitle=subtitle,
search_mode=SearchMode.UNIVERSAL,
search_mode=search_mode,
output_mode=output_mode,
output_args=output_args,
priority=priority,
@@ -256,8 +181,7 @@ def queue_release(
return True, None
except ValueError as e:
# Handler not found for this source
error_msg = f"Unknown release source: {e}"
error_msg = str(e)
logger.warning(error_msg)
return False, error_msg
except KeyError as e:
@@ -306,20 +230,6 @@ def get_book_data(task_id: str) -> Tuple[Optional[bytes], Optional[DownloadTask]
task.download_path = None
return None, task
def _book_info_to_dict(book: BookInfo) -> Dict[str, Any]:
"""Convert BookInfo to dict, transforming cover URLs for caching."""
result = {
key: value for key, value in book.__dict__.items()
if value is not None
}
# Transform external preview URLs to local proxy URLs
if result.get('preview'):
result['preview'] = transform_cover_url(result['preview'], book.id)
return result
def _task_to_dict(task: DownloadTask) -> Dict[str, Any]:
"""Convert DownloadTask to dict for frontend, transforming cover URLs."""
# Transform external preview URLs to local proxy URLs
@@ -347,6 +257,36 @@ def _task_to_dict(task: DownloadTask) -> Dict[str, Any]:
}
def _clear_task_error_state(task: DownloadTask) -> None:
task.last_error_message = None
task.last_error_type = None
def _capture_task_error(
task: DownloadTask,
*,
message: Optional[str] = None,
exc_type: Optional[str] = None,
) -> None:
if isinstance(message, str):
normalized = message.strip()
if normalized:
task.last_error_message = normalized
book_queue.update_status_message(task.task_id, normalized)
if isinstance(exc_type, str):
normalized_type = exc_type.strip()
if normalized_type:
task.last_error_type = normalized_type
def _format_download_exception_message(exc: Exception) -> str:
if isinstance(exc, PermissionError) and "/cwa-book-ingest" in str(exc):
return "Destination misconfigured. Go to Settings → Downloads to update."
if isinstance(exc, PermissionError):
return f"Permission denied: {exc}"
return f"Download failed: {type(exc).__name__}"
def _download_task(task_id: str, cancel_flag: Event) -> Optional[str]:
"""Download a task via appropriate handler, then post-process to ingest."""
try:
@@ -372,25 +312,57 @@ def _download_task(task_id: str, cancel_flag: Event) -> Optional[str]:
update_download_progress(task_id, progress)
def status_callback(status: str, message: Optional[str] = None) -> None:
status_key = status.lower()
if status_key == "error":
_capture_task_error(
task,
message=message or "Download failed",
exc_type="StatusCallbackError",
)
return
# Don't propagate terminal statuses to the queue here. Output modules
# call status_callback("complete") before returning the download path,
# but _process_single_download needs to set download_path on the task
# first so the terminal hook captures it for history persistence.
if status_key in ("complete", "cancelled"):
if message is not None:
book_queue.update_status_message(task_id, message)
return
update_download_status(task_id, status, message)
# Get the download handler based on the task's source
handler = get_handler(task.source)
temp_path = handler.download(
task,
cancel_flag,
progress_callback,
status_callback
)
temp_file: Optional[Path] = None
# Handler returns temp path - orchestrator handles post-processing
if not temp_path:
return None
if task.staged_path:
staged_file = Path(task.staged_path)
if run_blocking_io(staged_file.exists):
temp_file = staged_file
logger.info("Task %s: reusing staged file for retry: %s", task_id, staged_file)
else:
task.staged_path = None
temp_file = Path(temp_path)
if not run_blocking_io(temp_file.exists):
logger.error(f"Handler returned non-existent path: {temp_path}")
return None
if temp_file is None:
temp_path = handler.download(
task,
cancel_flag,
progress_callback,
status_callback,
)
# Handler returns temp path - orchestrator handles post-processing
if not temp_path:
return None
temp_file = Path(temp_path)
if not run_blocking_io(temp_file.exists):
logger.error(f"Handler returned non-existent path: {temp_path}")
_capture_task_error(
task,
message=f"Download file missing: {temp_path}",
exc_type="MissingDownloadPath",
)
return None
# Check cancellation before post-processing
if cancel_flag.is_set():
@@ -401,9 +373,17 @@ def _download_task(task_id: str, cancel_flag: Event) -> Optional[str]:
logger.info("Task %s: download finished; starting post-processing", task_id)
logger.debug("Task %s: post-processing input path: %s", task_id, temp_file)
task.staged_path = str(temp_file)
preserve_source_on_failure = True
# Post-processing: output routing + file processing pipeline
result = post_process_download(temp_file, task, cancel_flag, status_callback)
result = post_process_download(
temp_file,
task,
cancel_flag,
status_callback,
preserve_source_on_failure=preserve_source_on_failure,
)
if cancel_flag.is_set():
logger.info("Task %s: post-processing cancelled", task_id)
@@ -412,12 +392,22 @@ def _download_task(task_id: str, cancel_flag: Event) -> Optional[str]:
logger.debug("Task %s: post-processing result: %s", task_id, result)
else:
logger.warning("Task %s: post-processing failed", task_id)
if not task.last_error_message:
_capture_task_error(
task,
message="Download failed",
exc_type="UnknownFailure",
)
try:
handler.post_process_cleanup(task, success=bool(result))
except Exception as e:
logger.warning("Post-processing cleanup hook failed for %s: %s", task_id, e)
if result:
task.staged_path = None
_clear_task_error_state(task)
return result
except Exception as e:
@@ -425,21 +415,13 @@ def _download_task(task_id: str, cancel_flag: Event) -> Optional[str]:
logger.info("Task %s: cancelled during error handling", task_id)
else:
logger.error_trace("Task %s: error downloading: %s", task_id, e)
# Update task status so user sees the failure
task = book_queue.get_task(task_id)
if task:
book_queue.update_status(task_id, QueueStatus.ERROR)
# Check for known misconfiguration from earlier versions
if isinstance(e, PermissionError) and "/cwa-book-ingest" in str(e):
book_queue.update_status_message(
task_id,
"Destination misconfigured. Go to Settings → Downloads to update."
)
else:
if isinstance(e, PermissionError):
book_queue.update_status_message(task_id, f"Permission denied: {e}")
else:
book_queue.update_status_message(task_id, f"Download failed: {type(e).__name__}")
_capture_task_error(
task,
message=_format_download_exception_message(e),
exc_type=type(e).__name__,
)
return None
@@ -483,22 +465,10 @@ def update_download_progress(book_id: str, progress: float) -> None:
def update_download_status(book_id: str, status: str, message: Optional[str] = None) -> None:
"""Update download status with optional message for UI display."""
# Map string status to QueueStatus enum
status_map = {
'queued': QueueStatus.QUEUED,
'resolving': QueueStatus.RESOLVING,
'locating': QueueStatus.LOCATING,
'downloading': QueueStatus.DOWNLOADING,
'complete': QueueStatus.COMPLETE,
'available': QueueStatus.AVAILABLE,
'error': QueueStatus.ERROR,
'done': QueueStatus.DONE,
'cancelled': QueueStatus.CANCELLED,
}
status_key = status.lower()
queue_status_enum = status_map.get(status_key)
if not queue_status_enum:
try:
queue_status_enum = QueueStatus(status_key)
except ValueError:
return
# Always update activity timestamp (used by stall detection) even if the status
@@ -531,6 +501,38 @@ def cancel_download(book_id: str) -> bool:
return result
def retry_download(book_id: str) -> Tuple[bool, Optional[str]]:
"""Retry a failed or cancelled download.
Request-linked downloads can only be retried when cancelled (errors
reopen the request for admin re-approval instead).
"""
task = book_queue.get_task(book_id)
if task is None:
return False, "Download not found"
status = book_queue.get_task_status(book_id)
if status not in (QueueStatus.ERROR, QueueStatus.CANCELLED):
return False, "Download is not in an error or cancelled state"
if task.request_id and status != QueueStatus.CANCELLED:
return False, "Request-linked downloads must be retried from requests"
task.last_error_message = None
task.last_error_type = None
task.priority = -10
if not book_queue.enqueue_existing(book_id, priority=-10):
return False, "Failed to requeue download"
book_queue.update_status_message(book_id, "Retrying now")
if ws_manager:
ws_manager.broadcast_status_update(queue_status())
return True, None
def set_book_priority(book_id: str, priority: int) -> bool:
"""Set priority for a queued book (lower = higher priority)."""
return book_queue.set_priority(book_id, priority)
@@ -547,10 +549,6 @@ def get_active_downloads() -> List[str]:
"""Get list of currently active downloads."""
return book_queue.get_active_downloads()
def clear_completed(user_id: Optional[int] = None) -> int:
"""Clear completed downloads from tracking (optionally user-scoped)."""
return book_queue.clear_completed(user_id=user_id)
def _cleanup_progress_tracking(task_id: str) -> None:
"""Clean up progress tracking data for a completed/cancelled download."""
with _progress_lock:
@@ -560,6 +558,24 @@ def _cleanup_progress_tracking(task_id: str) -> None:
_last_status_event.pop(task_id, None)
def _finalize_download_failure(task_id: str) -> None:
task = book_queue.get_task(task_id)
if not task:
return
message = task.last_error_message or task.status_message or ""
normalized_message = message.strip()
if not normalized_message:
normalized_message = (
f"Download failed: {task.last_error_type}"
if task.last_error_type
else "Download failed"
)
book_queue.update_status_message(task_id, normalized_message)
book_queue.update_status(task_id, QueueStatus.ERROR)
def _process_single_download(task_id: str, cancel_flag: Event) -> None:
"""Process a single download job."""
try:
@@ -579,12 +595,9 @@ def _process_single_download(task_id: str, cancel_flag: Event) -> None:
if download_path:
book_queue.update_download_path(task_id, download_path)
# Only update status if not already set (e.g., by archive extraction callback)
task = book_queue.get_task(task_id)
if not task or task.status != QueueStatus.COMPLETE:
book_queue.update_status(task_id, QueueStatus.COMPLETE)
book_queue.update_status(task_id, QueueStatus.COMPLETE)
else:
book_queue.update_status(task_id, QueueStatus.ERROR)
_finalize_download_failure(task_id)
# Broadcast final status (completed or error)
if ws_manager:
@@ -596,11 +609,14 @@ def _process_single_download(task_id: str, cancel_flag: Event) -> None:
if not cancel_flag.is_set():
logger.error_trace(f"Error in download processing: {e}")
book_queue.update_status(task_id, QueueStatus.ERROR)
# Set error message if not already set by handler
task = book_queue.get_task(task_id)
if task and not task.status_message:
book_queue.update_status_message(task_id, f"Download failed: {type(e).__name__}: {str(e)}")
if task:
_capture_task_error(
task,
message=f"Download failed: {type(e).__name__}: {str(e)}",
exc_type=type(e).__name__,
)
_finalize_download_failure(task_id)
else:
logger.info(f"Download cancelled: {task_id}")
book_queue.update_status(task_id, QueueStatus.CANCELLED)
+1 -1
View File
@@ -8,7 +8,7 @@ from typing import Callable, Optional
from shelfmark.core.models import DownloadTask
StatusCallback = Callable[[str, Optional[str]], None]
OutputHandler = Callable[[Path, DownloadTask, Event, StatusCallback], Optional[str]]
OutputHandler = Callable[[Path, DownloadTask, Event, StatusCallback, bool], Optional[str]]
@dataclass(frozen=True)
+19 -3
View File
@@ -13,7 +13,7 @@ from shelfmark.core.logger import setup_logger
from shelfmark.core.models import DownloadTask
from shelfmark.core.utils import is_audiobook as check_audiobook
from shelfmark.download.outputs import register_output
from shelfmark.download.staging import STAGE_MOVE, STAGE_NONE, build_staging_dir, get_staging_dir
from shelfmark.download.staging import STAGE_COPY, STAGE_MOVE, STAGE_NONE, build_staging_dir, get_staging_dir
logger = setup_logger(__name__)
@@ -241,6 +241,7 @@ def _post_process_booklore(
task: DownloadTask,
cancel_flag: Event,
status_callback,
preserve_source_on_failure: bool = False,
) -> Optional[str]:
from shelfmark.download.postprocess.pipeline import (
CustomScriptContext,
@@ -249,6 +250,7 @@ def _post_process_booklore(
is_managed_workspace_path,
maybe_run_custom_script,
prepare_output_files,
safe_cleanup_path,
)
if cancel_flag.is_set():
@@ -267,7 +269,9 @@ def _post_process_booklore(
status_callback("resolving", "Preparing Booklore upload")
stage_action = STAGE_MOVE if is_managed_workspace_path(temp_file) else STAGE_NONE
stage_action = STAGE_NONE
if is_managed_workspace_path(temp_file):
stage_action = STAGE_COPY if preserve_source_on_failure else STAGE_MOVE
staging_dir = build_staging_dir("booklore", task.task_id) if stage_action != STAGE_NONE else get_staging_dir()
output_plan = OutputPlan(
@@ -283,12 +287,14 @@ def _post_process_booklore(
BOOKLORE_OUTPUT_MODE,
status_callback,
output_plan=output_plan,
preserve_source_on_failure=preserve_source_on_failure,
)
if not prepared:
return None
logger.debug("Task %s: prepared %d file(s) for Booklore upload", task.task_id, len(prepared.files))
success = False
try:
unsupported_files = [
file_path
@@ -359,6 +365,7 @@ def _post_process_booklore(
if len(prepared.files) > 1:
message = f"Uploaded to Booklore ({len(prepared.files)} files)"
status_callback("complete", message)
success = True
return f"booklore://{task.task_id}"
except BookloreError as e:
@@ -376,6 +383,8 @@ def _post_process_booklore(
task,
prepared.cleanup_paths,
)
if preserve_source_on_failure and success:
safe_cleanup_path(temp_file, task)
@register_output(BOOKLORE_OUTPUT_MODE, supports_task=_supports_booklore, priority=10)
@@ -384,5 +393,12 @@ def process_booklore_output(
task: DownloadTask,
cancel_flag: Event,
status_callback,
preserve_source_on_failure: bool = False,
) -> Optional[str]:
return _post_process_booklore(temp_file, task, cancel_flag, status_callback)
return _post_process_booklore(
temp_file,
task,
cancel_flag,
status_callback,
preserve_source_on_failure=preserve_source_on_failure,
)
+19 -3
View File
@@ -15,7 +15,7 @@ from shelfmark.core.logger import setup_logger
from shelfmark.core.models import DownloadTask
from shelfmark.core.utils import is_audiobook as check_audiobook
from shelfmark.download.outputs import register_output
from shelfmark.download.staging import STAGE_MOVE, STAGE_NONE, build_staging_dir, get_staging_dir
from shelfmark.download.staging import STAGE_COPY, STAGE_MOVE, STAGE_NONE, build_staging_dir, get_staging_dir
logger = setup_logger(__name__)
@@ -268,6 +268,7 @@ def _post_process_email(
task: DownloadTask,
cancel_flag: Event,
status_callback,
preserve_source_on_failure: bool = False,
) -> Optional[str]:
from shelfmark.download.postprocess.pipeline import (
CustomScriptContext,
@@ -276,6 +277,7 @@ def _post_process_email(
is_managed_workspace_path,
maybe_run_custom_script,
prepare_output_files,
safe_cleanup_path,
)
if cancel_flag.is_set():
@@ -304,7 +306,9 @@ def _post_process_email(
status_callback("resolving", "Preparing email")
stage_action = STAGE_MOVE if is_managed_workspace_path(temp_file) else STAGE_NONE
stage_action = STAGE_NONE
if is_managed_workspace_path(temp_file):
stage_action = STAGE_COPY if preserve_source_on_failure else STAGE_MOVE
staging_dir = build_staging_dir("email", task.task_id) if stage_action != STAGE_NONE else get_staging_dir()
output_plan = OutputPlan(
@@ -320,10 +324,12 @@ def _post_process_email(
EMAIL_OUTPUT_MODE,
status_callback,
output_plan=output_plan,
preserve_source_on_failure=preserve_source_on_failure,
)
if not prepared:
return None
success = False
try:
limit_mb_raw = core_config.config.get("EMAIL_ATTACHMENT_SIZE_LIMIT_MB", 25)
try:
@@ -399,6 +405,7 @@ def _post_process_email(
return None
status_callback("complete", f"Sent to {label}")
success = True
return f"email://{task.task_id}"
except EmailOutputError as exc:
@@ -416,6 +423,8 @@ def _post_process_email(
task,
prepared.cleanup_paths,
)
if preserve_source_on_failure and success:
safe_cleanup_path(temp_file, task)
@register_output(EMAIL_OUTPUT_MODE, supports_task=_supports_email, priority=10)
@@ -424,5 +433,12 @@ def process_email_output(
task: DownloadTask,
cancel_flag: Event,
status_callback,
preserve_source_on_failure: bool = False,
) -> Optional[str]:
return _post_process_email(temp_file, task, cancel_flag, status_callback)
return _post_process_email(
temp_file,
task,
cancel_flag,
status_callback,
preserve_source_on_failure=preserve_source_on_failure,
)
+10 -7
View File
@@ -88,6 +88,7 @@ def process_folder_output(
task: DownloadTask,
cancel_flag: Event,
status_callback,
preserve_source_on_failure: bool = False,
) -> Optional[str]:
"""Post-process download to the configured folder destination."""
from shelfmark.download.postprocess.pipeline import (
@@ -122,6 +123,7 @@ def process_folder_output(
output_mode=plan.output_mode,
status_callback=status_callback,
destination=plan.destination,
preserve_source_on_failure=preserve_source_on_failure,
)
if not prepared:
return None
@@ -143,7 +145,7 @@ def process_folder_output(
# For external usenet downloads, always copy from the client path.
# "Move" is implemented as a client-side cleanup after import.
preserve_source = is_usenet
preserve_source = is_usenet or preserve_source_on_failure
copy_for_label = is_torrent or preserve_source or prepared.output_plan.stage_action != STAGE_NONE
@@ -227,12 +229,13 @@ def process_folder_output(
)
if not maybe_run_custom_script(script_context, status_callback=status_callback, steps=steps):
cleanup_output_staging(
prepared.output_plan,
prepared.working_path,
task,
prepared.cleanup_paths,
)
if not preserve_source_on_failure:
cleanup_output_staging(
prepared.output_plan,
prepared.working_path,
task,
prepared.cleanup_paths,
)
return None
cleanup_output_staging(
@@ -6,12 +6,12 @@ from pathlib import Path
from shelfmark.core.logger import setup_logger
from shelfmark.core.models import DownloadTask
from shelfmark.core.utils import (
get_aa_content_type_dir,
get_destination,
is_audiobook as check_audiobook,
)
from shelfmark.download.fs import run_blocking_io
from shelfmark.download.permissions_debug import log_path_permission_context
from shelfmark.release_sources import get_source
logger = setup_logger("shelfmark.download.postprocess.pipeline")
@@ -63,9 +63,12 @@ def get_final_destination(task: DownloadTask) -> Path:
is_audiobook = check_audiobook(task.content_type)
if task.source == "direct_download" and not is_audiobook:
override = get_aa_content_type_dir(task.content_type)
if override:
return override
try:
override = get_source(task.source).get_destination_override(task)
except ValueError:
override = None
if override:
return override
return get_destination(is_audiobook, user_id=task.user_id, username=task.username)
+8 -3
View File
@@ -43,6 +43,7 @@ def prepare_output_files(
status_callback,
destination: Optional[Path] = None,
output_plan: Optional[OutputPlan] = None,
preserve_source_on_failure: bool = False,
) -> Optional[PreparedFiles]:
if output_plan is None:
output_plan = build_output_plan(
@@ -59,19 +60,23 @@ def prepare_output_files(
status_callback("resolving", step_label)
working_path = stage_path(working_path, output_plan.staging_dir, output_plan.stage_action)
can_delete_source_archives = output_plan.stage_action != STAGE_NONE or is_managed_workspace_path(working_path)
can_delete_source_archives = output_plan.stage_action != STAGE_NONE or is_managed_workspace_path(
working_path
)
cleanup_archives = can_delete_source_archives and not preserve_source_on_failure
files, rejected_files, cleanup_paths, error = collect_staged_files(
working_path=working_path,
task=task,
allow_archive_extraction=output_plan.allow_archive_extraction,
status_callback=status_callback,
cleanup_archives=can_delete_source_archives,
cleanup_archives=cleanup_archives,
)
if error:
status_callback("error", error)
cleanup_output_staging(output_plan, working_path, task, cleanup_paths)
if not preserve_source_on_failure:
cleanup_output_staging(output_plan, working_path, task, cleanup_paths)
return None
if output_plan.stage_action == STAGE_NONE and is_managed_workspace_path(working_path):
+15 -2
View File
@@ -26,6 +26,7 @@ def post_process_download(
task: DownloadTask,
cancel_flag: Event,
status_callback,
preserve_source_on_failure: bool = False,
) -> Optional[str]:
"""Post-process download using the selected output handler."""
@@ -44,9 +45,21 @@ def post_process_download(
output_handler = resolve_output_handler(task)
if output_handler:
logger.info("Task %s: using output mode %s", task.task_id, output_handler.mode)
return output_handler.handler(temp_file, task, cancel_flag, status_callback)
return output_handler.handler(
temp_file,
task,
cancel_flag,
status_callback,
preserve_source_on_failure,
)
from shelfmark.download.outputs.folder import process_folder_output
logger.info("Task %s: using output mode folder", task.task_id)
return process_folder_output(temp_file, task, cancel_flag, status_callback)
return process_folder_output(
temp_file,
task,
cancel_flag,
status_callback,
preserve_source_on_failure,
)
+15 -5
View File
@@ -57,6 +57,14 @@ def build_metadata_dict(task: DownloadTask) -> dict:
}
def build_file_metadata(task: DownloadTask, source_file: Path, part_number: Optional[str] = None) -> dict:
metadata = build_metadata_dict(task)
metadata["OriginalName"] = source_file.stem
if part_number is not None:
metadata["PartNumber"] = part_number
return metadata
def resolve_hardlink_source(
temp_file: Path,
task: DownloadTask,
@@ -157,16 +165,16 @@ def transfer_book_files(
if organization_mode == "organize":
template = get_template(is_audiobook, "organize")
metadata = build_metadata_dict(task)
if len(book_files) == 1:
source_file = book_files[0]
ext = source_file.suffix.lstrip(".") or task.format or ""
file_metadata = build_file_metadata(task, source_file)
dest_path = run_blocking_io(
build_library_path,
str(destination),
template,
metadata,
file_metadata,
extension=ext or None,
)
run_blocking_io(dest_path.parent.mkdir, parents=True, exist_ok=True)
@@ -188,7 +196,7 @@ def transfer_book_files(
for source_file, part_number in files_with_parts:
ext = source_file.suffix.lstrip(".") or task.format or ""
file_metadata = {**metadata, "PartNumber": part_number}
file_metadata = build_file_metadata(task, source_file, part_number=part_number)
dest_path = run_blocking_io(
build_library_path,
str(destination),
@@ -218,7 +226,7 @@ def transfer_book_files(
task.format = book_file.suffix.lower().lstrip(".")
template = get_template(is_audiobook, "rename")
metadata = build_metadata_dict(task)
metadata = build_file_metadata(task, book_file)
extension = book_file.suffix.lstrip(".") or task.format or ""
filename = parse_naming_template(template, metadata, allow_path_separators=False)
@@ -311,7 +319,9 @@ def transfer_file_to_library(
use_hardlink: bool,
) -> Optional[str]:
extension = source_path.suffix.lstrip(".") or task.format
dest_path = run_blocking_io(build_library_path, library_base, template, metadata, extension)
template_metadata = dict(metadata)
template_metadata.setdefault("OriginalName", source_path.stem)
dest_path = run_blocking_io(build_library_path, library_base, template, template_metadata, extension)
run_blocking_io(dest_path.parent.mkdir, parents=True, exist_ok=True)
is_torrent = is_torrent_source(source_path, task)
+783 -343
View File
File diff suppressed because it is too large Load Diff
+150 -16
View File
@@ -35,6 +35,14 @@ SORT_LABELS: Dict[SortOrder, str] = {
}
@dataclass
class MetadataCapability:
"""Declarative provider capability consumed by shared UI code."""
key: str
field_key: Optional[str] = None
sort: Optional[SortOrder] = None
@dataclass
class TextSearchField:
"""Text input search field."""
@@ -42,6 +50,8 @@ class TextSearchField:
label: str # Display label in UI
placeholder: str = "" # Placeholder text
description: str = "" # Help text
suggestions_endpoint: Optional[str] = None # Remote suggestions endpoint for typeahead
suggestions_min_query_length: int = 2 # Minimum chars before requesting suggestions
@dataclass
@@ -75,8 +85,39 @@ class CheckboxSearchField:
default: bool = False
@dataclass
class DynamicSelectSearchField:
"""Single-choice dropdown field with options loaded from an API endpoint."""
key: str
label: str
options_endpoint: str
placeholder: str = ""
description: str = ""
# Type alias for all search field types
SearchField = Union[TextSearchField, NumberSearchField, SelectSearchField, CheckboxSearchField]
SearchField = Union[
TextSearchField,
NumberSearchField,
SelectSearchField,
CheckboxSearchField,
DynamicSelectSearchField,
]
def serialize_metadata_capability(capability: MetadataCapability) -> Dict[str, Any]:
"""Serialize a provider capability for API responses."""
result: Dict[str, Any] = {
"key": capability.key,
}
if capability.field_key:
result["field_key"] = capability.field_key
if capability.sort:
result["sort"] = capability.sort.value
return result
def serialize_search_field(search_field: SearchField) -> Dict[str, Any]:
@@ -94,10 +135,16 @@ def serialize_search_field(search_field: SearchField) -> Dict[str, Any]:
result["min"] = search_field.min_value
result["max"] = search_field.max_value
result["step"] = search_field.step
elif isinstance(search_field, TextSearchField):
if search_field.suggestions_endpoint:
result["suggestions_endpoint"] = search_field.suggestions_endpoint
result["suggestions_min_query_length"] = search_field.suggestions_min_query_length
elif isinstance(search_field, SelectSearchField):
result["options"] = search_field.options
elif isinstance(search_field, CheckboxSearchField):
result["default"] = search_field.default
elif isinstance(search_field, DynamicSelectSearchField):
result["options_endpoint"] = search_field.options_endpoint
return result
@@ -151,6 +198,7 @@ class BookMetadata:
display_fields: List[DisplayField] = field(default_factory=list)
# Series info (if book is part of a series)
series_id: Optional[str] = None # Provider-specific series ID
series_name: Optional[str] = None # Name of the series
series_position: Optional[float] = None # This book's position (e.g., 3, 1.5 for novellas)
series_count: Optional[int] = None # Total books in the series
@@ -262,6 +310,8 @@ class SearchResult:
page: int = 1
total_found: int = 0 # Total matching results (if known)
has_more: bool = False # True if more results available
source_url: Optional[str] = None # External URL for the result set (e.g. Hardcover list page)
source_title: Optional[str] = None # Display title for the result set (e.g. list name)
class MetadataProvider(ABC):
@@ -276,12 +326,14 @@ class MetadataProvider(ABC):
requires_auth: True if API key/authentication is required
supported_sorts: List of SortOrder values this provider supports
search_fields: List of provider-specific search fields
capabilities: Declarative capabilities exposed to shared UI code
"""
name: str
display_name: str
requires_auth: bool
supported_sorts: List[SortOrder] = [SortOrder.RELEVANCE]
search_fields: List[SearchField] = []
capabilities: List[MetadataCapability] = []
@abstractmethod
def search(self, options: MetadataSearchOptions) -> List[BookMetadata]:
@@ -315,6 +367,44 @@ class MetadataProvider(ABC):
has_more=has_more
)
def get_search_field_options(
self,
field_key: str,
query: Optional[str] = None,
) -> List[Dict[str, str]]:
"""Get dynamic options for a provider-specific search field."""
return []
def get_book_targets(self, book_id: str) -> List[Dict[str, Any]]:
"""Get provider-managed list or status targets for a specific book."""
raise NotImplementedError(f"{self.display_name} does not support book targets")
def get_book_targets_batch(self, book_ids: List[str]) -> Dict[str, List[Dict[str, Any]]]:
"""Get provider-managed targets for multiple books.
Returns a dict mapping each book_id to its list of target options.
Default implementation calls get_book_targets per book.
"""
results: Dict[str, List[Dict[str, Any]]] = {}
for book_id in book_ids:
try:
results[book_id] = self.get_book_targets(book_id)
except (NotImplementedError, ValueError):
results[book_id] = []
return results
def set_book_target_state(
self,
book_id: str,
target: str,
selected: bool,
) -> Dict[str, Any]:
"""Set whether a book belongs to a provider-managed list or shelf.
Returns a dict with at least ``{"changed": bool}``.
"""
raise NotImplementedError(f"{self.display_name} does not support book targets")
# Provider registry
_PROVIDERS: Dict[str, Type[MetadataProvider]] = {}
@@ -393,7 +483,10 @@ def get_enabled_providers() -> List[str]:
return [name for name in _PROVIDERS if is_provider_enabled(name)]
def get_configured_provider(content_type: str = "ebook") -> Optional[MetadataProvider]:
def get_configured_provider(
content_type: str = "ebook",
user_id: Optional[int] = None,
) -> Optional[MetadataProvider]:
"""Get the currently configured metadata provider for the content type."""
from shelfmark.core.config import config as app_config
@@ -402,11 +495,11 @@ def get_configured_provider(content_type: str = "ebook") -> Optional[MetadataPro
# For audiobooks, try audiobook-specific provider first, then fall back to main provider
if content_type == "audiobook":
metadata_provider = app_config.get("METADATA_PROVIDER_AUDIOBOOK", "")
metadata_provider = app_config.get("METADATA_PROVIDER_AUDIOBOOK", "", user_id=user_id)
if not metadata_provider:
metadata_provider = app_config.get("METADATA_PROVIDER", "")
metadata_provider = app_config.get("METADATA_PROVIDER", "", user_id=user_id)
else:
metadata_provider = app_config.get("METADATA_PROVIDER", "")
metadata_provider = app_config.get("METADATA_PROVIDER", "", user_id=user_id)
if not metadata_provider:
return None
@@ -422,17 +515,35 @@ def get_configured_provider(content_type: str = "ebook") -> Optional[MetadataPro
return get_provider(metadata_provider, **kwargs)
def _get_configured_provider_name() -> str:
"""Get the currently configured metadata provider name from config."""
def get_configured_provider_name(
content_type: str = "ebook",
user_id: Optional[int] = None,
fallback_to_main: bool = True,
) -> str:
"""Get the configured metadata provider name for a content type."""
from shelfmark.core.config import config as app_config
app_config.refresh()
return app_config.get("METADATA_PROVIDER", "")
if content_type == "audiobook":
audiobook_provider = app_config.get(
"METADATA_PROVIDER_AUDIOBOOK",
"",
user_id=user_id,
)
if audiobook_provider or not fallback_to_main:
return audiobook_provider
return app_config.get("METADATA_PROVIDER", "", user_id=user_id)
def get_provider_sort_options(provider_name: Optional[str] = None) -> List[Dict[str, str]]:
def get_provider_sort_options(
provider_name: Optional[str] = None,
user_id: Optional[int] = None,
) -> List[Dict[str, str]]:
"""Get sort options for a metadata provider as {value, label} dicts."""
if provider_name is None:
provider_name = _get_configured_provider_name()
provider_name = get_configured_provider_name(user_id=user_id)
if provider_name and provider_name in _PROVIDERS:
provider_class = _PROVIDERS[provider_name]
@@ -446,10 +557,13 @@ def get_provider_sort_options(provider_name: Optional[str] = None) -> List[Dict[
]
def get_provider_search_fields(provider_name: Optional[str] = None) -> List[Dict[str, Any]]:
def get_provider_search_fields(
provider_name: Optional[str] = None,
user_id: Optional[int] = None,
) -> List[Dict[str, Any]]:
"""Get search fields for a metadata provider as serialized dicts."""
if provider_name is None:
provider_name = _get_configured_provider_name()
provider_name = get_configured_provider_name(user_id=user_id)
if provider_name and provider_name in _PROVIDERS:
provider_class = _PROVIDERS[provider_name]
@@ -460,19 +574,39 @@ def get_provider_search_fields(provider_name: Optional[str] = None) -> List[Dict
return [serialize_search_field(f) for f in fields]
def get_provider_default_sort(provider_name: Optional[str] = None) -> str:
def get_provider_capabilities(
provider_name: Optional[str] = None,
user_id: Optional[int] = None,
) -> List[Dict[str, Any]]:
"""Get declarative capabilities for a metadata provider."""
if provider_name is None:
provider_name = get_configured_provider_name(user_id=user_id)
if provider_name and provider_name in _PROVIDERS:
provider_class = _PROVIDERS[provider_name]
capabilities = getattr(provider_class, "capabilities", [])
else:
capabilities = []
return [serialize_metadata_capability(capability) for capability in capabilities]
def get_provider_default_sort(
provider_name: Optional[str] = None,
user_id: Optional[int] = None,
) -> str:
"""Get the default sort order for a metadata provider."""
from shelfmark.core.config import config as app_config
if provider_name is None:
provider_name = _get_configured_provider_name()
provider_name = get_configured_provider_name(user_id=user_id)
if not provider_name:
return "relevance"
# Look up provider-specific default sort setting
setting_key = f"{provider_name.upper()}_DEFAULT_SORT"
return app_config.get(setting_key, "relevance")
return app_config.get(setting_key, "relevance", user_id=user_id)
def sync_metadata_provider_selection() -> None:
@@ -502,7 +636,7 @@ def sync_metadata_provider_selection() -> None:
general_config = load_config_file("general")
general_config["METADATA_PROVIDER"] = new_provider
save_config_file("general", general_config)
app_config.refresh()
app_config.refresh(force=True)
# Import provider implementations to trigger registration
+2 -1
View File
@@ -20,6 +20,7 @@ from shelfmark.core.settings_registry import (
HeadingField,
)
from shelfmark.core.config import config as app_config
from shelfmark.download.network import get_ssl_verify
from shelfmark.metadata_providers import (
BookMetadata,
DisplayField,
@@ -233,7 +234,7 @@ class GoogleBooksProvider(MetadataProvider):
url = f"{GOOGLE_BOOKS_BASE_URL}{endpoint}"
try:
response = self.session.get(url, params=params, timeout=15)
response = self.session.get(url, params=params, timeout=15, verify=get_ssl_verify(url))
response.raise_for_status()
return response.json()
File diff suppressed because it is too large Load Diff
+11 -5
View File
@@ -10,6 +10,7 @@ import requests
from shelfmark.core.cache import cacheable
from shelfmark.core.logger import setup_logger
from shelfmark.download.network import get_ssl_verify
from shelfmark.core.settings_registry import (
register_settings,
CheckboxField,
@@ -188,7 +189,8 @@ class OpenLibraryProvider(MetadataProvider):
response = self.session.get(
f"{OPENLIBRARY_BASE_URL}/search.json",
params=params,
timeout=15
timeout=15,
verify=get_ssl_verify(OPENLIBRARY_BASE_URL),
)
response.raise_for_status()
data = response.json()
@@ -229,7 +231,8 @@ class OpenLibraryProvider(MetadataProvider):
try:
response = self.session.get(
f"{OPENLIBRARY_BASE_URL}/works/{book_id}.json",
timeout=15
timeout=15,
verify=get_ssl_verify(OPENLIBRARY_BASE_URL),
)
response.raise_for_status()
work = response.json()
@@ -261,7 +264,8 @@ class OpenLibraryProvider(MetadataProvider):
# First try the ISBN API which returns edition data
response = self.session.get(
f"{OPENLIBRARY_BASE_URL}/isbn/{clean_isbn}.json",
timeout=15
timeout=15,
verify=get_ssl_verify(OPENLIBRARY_BASE_URL),
)
response.raise_for_status()
edition = response.json()
@@ -485,7 +489,8 @@ class OpenLibraryProvider(MetadataProvider):
try:
response = self.session.get(
f"{OPENLIBRARY_BASE_URL}{author_key}.json",
timeout=10
timeout=10,
verify=get_ssl_verify(OPENLIBRARY_BASE_URL),
)
response.raise_for_status()
author = response.json()
@@ -504,7 +509,8 @@ def _test_openlibrary_connection() -> Dict[str, Any]:
response = provider.session.get(
f"{OPENLIBRARY_BASE_URL}/search.json",
params={"q": "test", "limit": 1},
timeout=10
timeout=10,
verify=get_ssl_verify(OPENLIBRARY_BASE_URL),
)
response.raise_for_status()
data = response.json()
+106
View File
@@ -3,6 +3,7 @@
from abc import ABC, abstractmethod
from dataclasses import dataclass, field, asdict
from enum import Enum
from pathlib import Path
from threading import Event
from typing import List, Optional, Dict, Type, Callable, Literal, Any, TYPE_CHECKING
@@ -21,6 +22,35 @@ class ReleaseProtocol(str, Enum):
DCC = "dcc" # IRC DCC
class SourceUnavailableError(Exception):
"""Raised when a source is configured but currently unreachable."""
@dataclass
class BrowseRecord:
"""Source-native browse/search record used before normalization to Release."""
id: str
title: str
source: str
preview: Optional[str] = None
author: Optional[str] = None
publisher: Optional[str] = None
year: Optional[str] = None
language: Optional[str] = None
content: Optional[str] = None
format: Optional[str] = None
size: Optional[str] = None
info: Optional[Dict[str, List[str]]] = None
description: Optional[str] = None
download_urls: List[str] = field(default_factory=list)
download_path: Optional[str] = None
priority: int = 0
progress: Optional[float] = None
status_message: Optional[str] = None
added_time: Optional[float] = None
source_url: Optional[str] = None
@dataclass
class Release:
"""A downloadable release - all sources return this same structure."""
@@ -113,6 +143,13 @@ class LeadingCellConfig:
uppercase: bool = False # Force uppercase for badge text
@dataclass
class SortOption:
"""A sort option that appears in the sort dropdown without being tied to a column."""
label: str # Display label in the sort dropdown
sort_key: str # Field to sort by on the Release object
@dataclass
class SourceActionButton:
"""Action button configuration for a release source."""
@@ -131,6 +168,7 @@ class ReleaseColumnConfig:
default_indexers: Optional[List[str]] = None # For Prowlarr: indexers selected in settings (pre-selected in filter)
cache_ttl_seconds: Optional[int] = None # How long to cache results (default: 5 min)
supported_filters: Optional[List[str]] = None # Which filters this source supports: ["format", "language", "indexer"]
extra_sort_options: Optional[List[SortOption]] = None # Additional sort options not tied to a column
action_button: Optional[SourceActionButton] = None # Custom action button (replaces default expand search)
@@ -191,6 +229,13 @@ def serialize_column_config(config: ReleaseColumnConfig) -> Dict[str, Any]:
if config.supported_filters is not None:
result["supported_filters"] = config.supported_filters
# Include extra sort options (sort entries not tied to a column)
if config.extra_sort_options:
result["extra_sort_options"] = [
{"label": opt.label, "sort_key": opt.sort_key}
for opt in config.extra_sort_options
]
# Include action button if specified (replaces default expand search)
if config.action_button is not None:
result["action_button"] = {
@@ -266,6 +311,23 @@ class ReleaseSource(ABC):
"""Get column configuration for release list UI. Override for custom columns."""
return _default_column_config()
def get_record(
self,
record_id: str,
*,
fetch_download_count: bool = True,
) -> Optional[BrowseRecord]:
"""Resolve a source-native record for browse flows."""
raise NotImplementedError(f"{self.display_name} does not support record lookup")
def search_results_are_releases(self) -> bool:
"""Whether source-native browse results already represent concrete releases."""
return False
def get_destination_override(self, task: DownloadTask) -> Optional[Path]:
"""Return a source-specific destination override for a queued download."""
return None
class DownloadHandler(ABC):
"""Interface for executing downloads.
@@ -349,6 +411,7 @@ def list_available_sources() -> List[dict]:
"display_name": instance.display_name,
"enabled": instance.is_available(),
"supported_content_types": getattr(instance, 'supported_content_types', ["ebook", "audiobook"]),
"browse_results_are_releases": instance.search_results_are_releases(),
"can_be_default": getattr(instance, 'can_be_default', True),
})
return result
@@ -361,6 +424,49 @@ def get_source_display_name(name: str) -> str:
return name.replace('_', ' ').title()
def browse_record_to_book_metadata(
record: BrowseRecord,
*,
title_override: Optional[str] = None,
author_override: Optional[str] = None,
) -> BookMetadata:
"""Convert a source-native browse record into generic book metadata."""
resolved_title = title_override or str(record.title or "").strip() or "Unknown title"
resolved_author = author_override or str(record.author or "").strip()
authors = [part.strip() for part in resolved_author.split(",") if part.strip()]
publish_year = None
if isinstance(record.year, int):
publish_year = record.year
elif isinstance(record.year, str):
normalized_year = record.year.strip()
if normalized_year.isdigit():
publish_year = int(normalized_year)
return BookMetadata(
provider=record.source,
provider_id=record.id,
provider_display_name=get_source_display_name(record.source),
title=resolved_title,
search_title=resolved_title,
search_author=resolved_author or None,
authors=authors,
cover_url=record.preview,
description=record.description,
publisher=record.publisher,
publish_year=publish_year,
language=record.language,
source_url=record.source_url,
)
def source_results_are_releases(name: str) -> bool:
"""Whether a source's browse/search results already map to concrete releases."""
if name not in _SOURCES:
return False
return _SOURCES[name]().search_results_are_releases()
# Import source implementations to trigger registration
# These must be imported AFTER the base classes and registry are defined
from shelfmark.release_sources import direct_download # noqa: F401, E402
+92 -54
View File
@@ -17,15 +17,17 @@ from shelfmark.download import http as downloader
from shelfmark.download import network
from shelfmark.config.env import DEBUG_SKIP_SOURCES, TMP_DIR
from shelfmark.core.config import config
from shelfmark.core.utils import CONTENT_TYPES
from shelfmark.core.utils import CONTENT_TYPES, get_aa_content_type_dir, is_audiobook as check_audiobook
from shelfmark.core.logger import setup_logger
from shelfmark.core.models import BookInfo, SearchFilters, DownloadTask
from shelfmark.core.models import SearchFilters, DownloadTask, build_filename
from shelfmark.metadata_providers import BookMetadata, group_languages_by_localized_title
from shelfmark.release_sources import (
BrowseRecord,
Release,
ReleaseProtocol,
ReleaseSource,
DownloadHandler,
SourceUnavailableError,
register_source,
register_handler,
ReleaseColumnConfig,
@@ -136,11 +138,11 @@ def _normalize_size(size_str: str) -> str:
return _SIZE_UNIT_PATTERN.sub(lambda m: m.group(1).upper(), size_str.strip())
class SearchUnavailable(Exception):
class SearchUnavailable(SourceUnavailableError):
"""Raised when Anna's Archive cannot be reached via any mirror/DNS."""
def search_books(query: str, filters: SearchFilters) -> List[BookInfo]:
def search_books(query: str, filters: SearchFilters) -> List[BrowseRecord]:
"""Search for books matching the query.
Args:
@@ -148,7 +150,7 @@ def search_books(query: str, filters: SearchFilters) -> List[BookInfo]:
filters: Search filters (language, format, content type, etc.)
Returns:
List[BookInfo]: List of matching books
List[BrowseRecord]: List of matching books
Raises:
SearchUnavailable: If Anna's Archive cannot be reached
@@ -164,8 +166,8 @@ def search_books(query: str, filters: SearchFilters) -> List[BookInfo]:
filters_query = ""
for value in filters.lang or config.BOOK_LANGUAGE:
if value != "all":
for value in filters.lang if filters.lang else config.BOOK_LANGUAGE or []:
if value and value != "all":
filters_query += f"&lang={quote(value)}"
if filters.sort and filters.sort != "relevance":
@@ -232,7 +234,7 @@ def search_books(query: str, filters: SearchFilters) -> List[BookInfo]:
return books
def get_book_info(book_id: str, fetch_download_count: bool = True) -> BookInfo:
def get_book_info(book_id: str, fetch_download_count: bool = True) -> BrowseRecord:
"""Get detailed information for a specific book.
Args:
@@ -241,22 +243,22 @@ def get_book_info(book_id: str, fetch_download_count: bool = True) -> BookInfo:
Only needed for display in DetailsModal, not for downloads.
Returns:
BookInfo: Detailed book information including download URLs
BrowseRecord: Detailed book information including download URLs
"""
url = f"{network.get_aa_base_url()}/md5/{book_id}"
selector = network.AAMirrorSelector()
html = downloader.html_get_page(url, selector=selector, allow_bypasser_fallback=False)
if not html:
raise Exception(f"Failed to fetch book info for ID: {book_id}")
raise SearchUnavailable("Unable to reach download source. Network restricted or mirrors are blocked.")
soup = BeautifulSoup(html, "html.parser")
return _parse_book_info_page(soup, book_id, fetch_download_count)
def _parse_search_result_row(row: Tag) -> Optional[BookInfo]:
"""Parse a single search result row into a BookInfo object."""
def _parse_search_result_row(row: Tag) -> Optional[BrowseRecord]:
"""Parse a single search result row into a browse record."""
try:
if row.text.strip().lower().startswith("your ad here"):
return None
@@ -264,10 +266,11 @@ def _parse_search_result_row(row: Tag) -> Optional[BookInfo]:
preview_img = cells[0].find("img")
preview = preview_img["src"] if preview_img else None
return BookInfo(
return BrowseRecord(
id=row.find_all("a")[0]["href"].split("/")[-1],
preview=preview,
title=cells[1].find("span").next,
source="direct_download",
preview=preview,
author=cells[2].find("span").next,
publisher=cells[3].find("span").next,
year=cells[4].find("span").next,
@@ -281,8 +284,8 @@ def _parse_search_result_row(row: Tag) -> Optional[BookInfo]:
return None
def _parse_book_info_page(soup: BeautifulSoup, book_id: str, fetch_download_count: bool = True) -> BookInfo:
"""Parse the book info page HTML into a BookInfo object."""
def _parse_book_info_page(soup: BeautifulSoup, book_id: str, fetch_download_count: bool = True) -> BrowseRecord:
"""Parse the book info page HTML into a browse record."""
data = soup.select_one("body > main > div:nth-of-type(1)")
if not data:
@@ -379,10 +382,11 @@ def _parse_book_info_page(soup: BeautifulSoup, book_id: str, fetch_download_coun
# Extract basic information
description = _extract_book_description(soup)
book_info = BookInfo(
book_info = BrowseRecord(
id=book_id,
preview=preview,
title=book_title,
source="direct_download",
preview=preview,
content=content,
publisher=(_find_in_divs(divs, "icon-[mdi--company]", is_class=True) or [""])[0],
author=(_find_in_divs(divs, "icon-[mdi--user-edit]", is_class=True) or [""])[0],
@@ -538,7 +542,7 @@ def _group_urls_by_source(urls: List[str], urls_by_source: Dict[str, List[str]])
urls_by_source.setdefault(source_type, []).append(url)
def _fetch_aa_page_urls(book_info: BookInfo, urls_by_source: Dict[str, List[str]]) -> None:
def _fetch_aa_page_urls(book_info: BrowseRecord, urls_by_source: Dict[str, List[str]]) -> None:
"""Fetch and parse AA page, populating urls_by_source dict.
Groups existing book_info.download_urls by source type. If book_info
@@ -557,7 +561,7 @@ def _fetch_aa_page_urls(book_info: BookInfo, urls_by_source: Dict[str, List[str]
def _get_urls_for_source(
source_id: str,
book_info: BookInfo,
book_info: BrowseRecord,
selector: network.AAMirrorSelector,
cancel_flag: Optional[Event],
status_callback: Optional[Callable[[str, Optional[str]], None]],
@@ -608,7 +612,7 @@ def _get_urls_for_source(
def _try_download_url(
url: str,
source_id: str,
book_info: BookInfo,
book_info: BrowseRecord,
book_path: Path,
progress_callback: Optional[Callable[[float], None]],
cancel_flag: Optional[Event],
@@ -704,6 +708,7 @@ def _extract_libgen_download_url(link: str, cancel_flag: Optional[Event] = None)
timeout=(5, 10),
allow_redirects=True,
proxies=network.get_proxies(link),
verify=network.get_ssl_verify(link),
)
if response.status_code != 200:
@@ -746,7 +751,7 @@ def _extract_libgen_download_url(link: str, cancel_flag: Optional[Event] = None)
def _download_book(
book_info: BookInfo,
book_info: BrowseRecord,
book_path: Path,
progress_callback: Optional[Callable[[float], None]] = None,
cancel_flag: Optional[Event] = None,
@@ -1045,33 +1050,32 @@ def _extract_countdown_seconds(soup: BeautifulSoup, html_str: str) -> int:
return 0
def _book_info_to_release(book_info: BookInfo) -> Release:
"""Convert a BookInfo object to a Release object.
def _browse_record_to_release(record: BrowseRecord) -> Release:
"""Convert a browse record to a Release object.
This bridges the existing BookInfo model (which combines metadata + release info)
to the new Release model (release info only).
This bridges the direct source's browse data to the generic release model.
"""
return Release(
source="direct_download",
source_id=book_info.id,
title=book_info.title,
format=book_info.format,
language=book_info.language, # Top-level language for filtering
size=book_info.size,
download_url=book_info.download_urls[0] if book_info.download_urls else None,
info_url=f"{network.get_aa_base_url()}/md5/{book_info.id}",
source=record.source,
source_id=record.id,
title=record.title,
format=record.format,
language=record.language, # Top-level language for filtering
size=record.size,
download_url=record.download_urls[0] if record.download_urls else None,
info_url=f"{network.get_aa_base_url()}/md5/{record.id}",
protocol=ReleaseProtocol.HTTP,
indexer="Direct Download",
content_type=book_info.content, # Preserve content type from source
content_type=record.content, # Preserve content type from source
extra={
"author": book_info.author,
"publisher": book_info.publisher,
"year": book_info.year,
"language": book_info.language,
"preview": book_info.preview,
"description": book_info.description,
"download_urls": book_info.download_urls,
"info": book_info.info,
"author": record.author,
"publisher": record.publisher,
"year": record.year,
"language": record.language,
"preview": record.preview,
"description": record.description,
"download_urls": record.download_urls,
"info": record.info,
}
)
@@ -1139,6 +1143,25 @@ class DirectDownloadSource(ReleaseSource):
supported_filters=["format", "language"], # AA has reliable language metadata
)
def get_record(
self,
record_id: str,
*,
fetch_download_count: bool = True,
) -> Optional[BrowseRecord]:
"""Resolve a direct-download record for direct-mode info/download flows."""
return get_book_info(record_id, fetch_download_count=fetch_download_count)
def search_results_are_releases(self) -> bool:
"""Direct search results already represent concrete downloadable releases."""
return True
def get_destination_override(self, task: DownloadTask) -> Optional[Path]:
"""Apply Anna's Archive content-type routing when configured."""
if check_audiobook(task.content_type):
return None
return get_aa_content_type_dir(task.content_type)
def search(
self,
book: BookMetadata,
@@ -1163,6 +1186,15 @@ class DirectDownloadSource(ReleaseSource):
# Reset search type tracking
self._last_search_type = "title_author"
if plan.source_filters is not None:
query = plan.manual_query or ""
logger.debug(f"Searching direct_download: source_query='{query}', langs={lang_filter}")
filters = plan.source_filters or SearchFilters()
filters.lang = lang_filter if lang_filter is not None else (filters.lang or [])
results = search_books(query, filters)
self._last_search_type = "manual" if query else "title_author"
return [_browse_record_to_release(record) for record in results]
# ISBN search first (unless expand_search requested)
if plan.manual_query:
expand_search = True
@@ -1172,14 +1204,13 @@ class DirectDownloadSource(ReleaseSource):
if isbn:
logger.debug(f"Searching direct_download: isbn='{isbn}', langs={lang_filter}")
filters = SearchFilters(isbn=[isbn])
if lang_filter:
filters.lang = lang_filter
filters.lang = lang_filter if lang_filter is not None else []
try:
results = search_books(isbn, filters)
if results:
logger.info(f"Found {len(results)} releases via ISBN")
self._last_search_type = "isbn"
return [_book_info_to_release(bi) for bi in results]
return [_browse_record_to_release(record) for record in results]
logger.debug("No ISBN results, falling back to title+author")
except SearchUnavailable:
raise
@@ -1192,7 +1223,7 @@ class DirectDownloadSource(ReleaseSource):
# Execute searches with deduplication
seen_ids: set = set()
all_results: List[BookInfo] = []
all_results: List[BrowseRecord] = []
for title, langs in searches:
query = f"{title} {author}".strip()
@@ -1200,7 +1231,7 @@ class DirectDownloadSource(ReleaseSource):
continue
logger.debug(f"Searching direct_download: title_author='{query}', langs={langs}")
filters = SearchFilters(lang=langs) if langs else SearchFilters()
filters = SearchFilters(lang=langs if langs is not None else [])
try:
for bi in search_books(query, filters):
if bi.id not in seen_ids:
@@ -1212,7 +1243,7 @@ class DirectDownloadSource(ReleaseSource):
logger.error(f"Search error: {e}")
logger.info(f"Found {len(all_results)} releases via title+author")
return [_book_info_to_release(bi) for bi in all_results]
return [_browse_record_to_release(record) for record in all_results]
def is_available(self) -> bool:
"""Direct download is always available."""
@@ -1258,13 +1289,15 @@ class DirectDownloadHandler(DownloadHandler):
status_callback("cancelled", "Cancelled")
return None
# Create BookInfo from task data - NO AA page fetch here
# Create browse record from task data - NO AA page fetch here
# AA page is fetched lazily by _fetch_aa_page_urls only when
# we actually reach an AA slow source in the priority order
book_info = BookInfo(
book_info = BrowseRecord(
id=task.task_id,
title=task.title,
source="direct_download",
author=task.author,
year=task.year,
format=task.format,
size=task.size,
preview=task.preview,
@@ -1288,13 +1321,13 @@ class DirectDownloadHandler(DownloadHandler):
def _execute_download(
self,
book_info: BookInfo,
book_info: BrowseRecord,
cancel_flag: Event,
progress_callback: Callable[[float], None],
status_callback: Callable[[str, Optional[str]], None]
) -> Optional[str]:
"""
Internal method to execute the download with fetched BookInfo.
Internal method to execute the download with fetched browse record.
This contains the core download logic: cascade through sources,
handle bypass, move to final location.
@@ -1308,7 +1341,12 @@ class DirectDownloadHandler(DownloadHandler):
if file_org == "none":
book_name = f"{book_info.id}.{book_info.format or 'bin'}"
else:
book_name = book_info.get_filename()
book_name = build_filename(
book_info.title,
book_info.author,
book_info.year,
book_info.format,
)
book_path = TMP_DIR / book_name
# Check cancellation before download
@@ -6,6 +6,7 @@ import requests
from shelfmark.core.logger import setup_logger
from shelfmark.core.utils import normalize_http_url
from shelfmark.download.network import get_ssl_verify
from shelfmark.release_sources.prowlarr.torznab import parse_torznab_xml
logger = setup_logger(__name__)
@@ -42,6 +43,7 @@ class ProwlarrClient:
params=params,
json=json_data,
timeout=self.timeout,
verify=get_ssl_verify(url),
)
if not response.ok:
@@ -193,6 +195,7 @@ class ProwlarrClient:
# Override the session default JSON accept header.
"Accept": "application/rss+xml, application/xml;q=0.9, */*;q=0.8"
},
verify=get_ssl_verify(url),
)
if not response.ok:
try:
@@ -22,6 +22,7 @@ from shelfmark.release_sources import (
ColumnColorHint,
LeadingCellConfig,
LeadingCellType,
SortOption,
)
from shelfmark.release_sources.prowlarr.api import ProwlarrClient
from shelfmark.core.utils import normalize_http_url
@@ -467,6 +468,9 @@ class ProwlarrSource(ReleaseSource):
sort_key="size_bytes",
),
],
extra_sort_options=[
SortOption(label="Peers", sort_key="seeders"),
],
grid_template="minmax(0,2fr) minmax(140px,1fr) 50px 50px 90px 80px",
leading_cell=LeadingCellConfig(type=LeadingCellType.NONE), # No leading cell for Prowlarr
available_indexers=available_indexers,
@@ -8,7 +8,9 @@ isn't available via Prowlarr's JSON search endpoint.
from __future__ import annotations
from typing import Any, Dict, List, Optional
from xml.etree import ElementTree as ET
from defusedxml import ElementTree as DefusedElementTree
from defusedxml.common import DefusedXmlException
def _local_name(tag: str) -> str:
@@ -67,8 +69,8 @@ def parse_torznab_xml(xml_text: str) -> List[Dict[str, Any]]:
return []
try:
root = ET.fromstring(xml_text)
except ET.ParseError:
root = DefusedElementTree.fromstring(xml_text)
except (DefusedElementTree.ParseError, DefusedXmlException):
return []
items = root.findall(".//item")
@@ -168,4 +170,3 @@ def parse_torznab_xml(xml_text: str) -> List[Dict[str, Any]]:
})
return results
+8 -5
View File
@@ -20,23 +20,26 @@
<link rel="apple-touch-icon" href="logo.png" />
<title>Shelfmark</title>
<script>
// Apply theme immediately before first paint to prevent flash
// Apply theme immediately before first paint to prevent flash.
// CSS variables aren't available until the stylesheet loads, so set
// the background color directly on <html> to avoid a white flash.
(function() {
const savedTheme = localStorage.getItem('preferred-theme') || 'auto';
let theme = savedTheme;
if (savedTheme === 'auto') {
theme = window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light';
}
document.documentElement.setAttribute('data-theme', theme);
document.documentElement.style.backgroundColor = theme === 'dark' ? '#121212' : '#f8f8f8';
// Add class to prevent transitions on initial load
document.documentElement.classList.add('preload');
})();
</script>
</head>
<body style="background: var(--bg); color: var(--text);">
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
+585 -954
View File
File diff suppressed because it is too large Load Diff
+4 -4
View File
@@ -9,23 +9,23 @@
"preview": "vite preview",
"typecheck": "tsc --noEmit",
"test:unit": "npm run test:unit:build && node --experimental-specifier-resolution=node --test ../../.local/frontend-test-dist/tests/**/*.node.test.js",
"test:unit:build": "tsc -p tsconfig.tests.json"
"test:unit:build": "rm -rf ../../.local/frontend-test-dist && tsc -p tsconfig.tests.json"
},
"dependencies": {
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-router-dom": "^6.30.2",
"react-router-dom": "^6.30.3",
"socket.io-client": "^4.7.5"
},
"devDependencies": {
"@types/node": "^24.10.0",
"@types/react": "^18.3.3",
"@types/react-dom": "^18.3.0",
"@vitejs/plugin-react": "^4.3.1",
"@vitejs/plugin-react": "^5.1.4",
"autoprefixer": "^10.4.16",
"postcss": "^8.4.32",
"tailwindcss": "^3.4.0",
"typescript": "^5.5.3",
"vite": "^5.4.0"
"vite": "^7.3.1"
}
}
+1093 -209
View File
File diff suppressed because it is too large Load Diff
+106 -167
View File
@@ -1,11 +1,15 @@
import { ReactNode, KeyboardEvent } from 'react';
import { AdvancedFilterState, Language, MetadataSearchField } from '../types';
import { ReactNode } from 'react';
import {
AdvancedFilterState,
ContentType,
Language,
MetadataProviderSummary,
SearchMode,
} from '../types';
import { normalizeLanguageSelection } from '../utils/languageFilters';
import { useSearchMode } from '../contexts/SearchModeContext';
import { LanguageMultiSelect } from './LanguageMultiSelect';
import { DropdownList } from './DropdownList';
import { CONTENT_OPTIONS } from '../data/filterOptions';
import { SearchFieldRenderer } from './shared';
const FORMAT_TYPES = ['pdf', 'epub', 'mobi', 'azw3', 'fb2', 'djvu', 'cbz', 'cbr', 'zip', 'rar'] as const;
@@ -13,42 +17,41 @@ interface AdvancedFiltersProps {
visible: boolean;
bookLanguages: Language[];
defaultLanguage: string[];
supportedFormats: string[];
filters: AdvancedFilterState;
onFiltersChange: (updates: Partial<AdvancedFilterState>) => void;
formClassName?: string;
renderWrapper?: (form: ReactNode) => ReactNode;
// Universal mode props
metadataSearchFields?: MetadataSearchField[];
searchFieldValues?: Record<string, string | number | boolean>;
onSearchFieldChange?: (key: string, value: string | number | boolean) => void;
// Submit handler for Enter key
onSubmit?: () => void;
searchMode: SearchMode;
onSearchModeChange: (mode: SearchMode) => void;
metadataProviders?: MetadataProviderSummary[];
activeMetadataProvider?: string | null;
onMetadataProviderChange?: (provider: string) => void;
contentType?: ContentType;
isAdmin?: boolean;
}
const SEARCH_MODE_OPTIONS = [
{ value: 'direct', label: 'Direct', description: 'Search web sources for books and download directly. Works out of the box.' },
{ value: 'universal', label: 'Universal', description: 'Metadata-based search with downloads from all sources. Book and Audiobook support.' },
];
export const AdvancedFilters = ({
visible,
bookLanguages,
defaultLanguage,
supportedFormats,
filters,
onFiltersChange,
formClassName,
renderWrapper,
metadataSearchFields = [],
searchFieldValues = {},
onSearchFieldChange,
onSubmit,
searchMode,
onSearchModeChange,
metadataProviders = [],
activeMetadataProvider,
onMetadataProviderChange,
contentType = 'ebook',
isAdmin = false,
}: AdvancedFiltersProps) => {
const { searchMode } = useSearchMode();
const { isbn, author, title, lang, content, formats } = filters;
const handleKeyDown = (e: KeyboardEvent<HTMLInputElement>) => {
if (e.key === 'Enter' && onSubmit) {
e.preventDefault();
onSubmit();
}
};
const { lang, content, formats } = filters;
const handleLangChange = (next: string[]) => {
const normalized = normalizeLanguageSelection(next);
@@ -68,149 +71,85 @@ export const AdvancedFilters = ({
const formatOptions = FORMAT_TYPES.map(format => ({
value: format,
label: format.toUpperCase(),
disabled: !supportedFormats.includes(format),
}));
const providerOptions = metadataProviders.map((provider) => {
const details: string[] = [];
if (!provider.enabled) details.push('Disabled in Settings');
if (provider.enabled && !provider.available) details.push('Not configured');
if (provider.requires_auth) details.push('API key required');
return {
value: provider.name,
label: provider.display_name,
description: details.length > 0 ? details.join(' • ') : undefined,
disabled: !provider.enabled || !provider.available,
};
});
if (!visible) return null;
// Universal search mode: render dynamic provider fields
if (searchMode === 'universal') {
// If no fields defined for this provider, don't show the section
if (metadataSearchFields.length === 0) return null;
const wrapperClassName = formClassName
? 'px-2'
: 'px-2 lg:ml-[calc(3rem+1rem)] lg:w-[calc(50vw+4rem)]';
const universalForm = (
<form
id="search-filters"
className={
formClassName ??
'grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4 px-2 lg:ml-[calc(3rem+1rem)] lg:w-[50vw]'
}
>
{metadataSearchFields.map((field) => (
<div key={field.key}>
{field.type !== 'CheckboxSearchField' && (
<label htmlFor={`${field.key}-input`} className="block text-sm mb-1 opacity-80">
{field.label}
</label>
)}
<SearchFieldRenderer
field={field}
value={searchFieldValues[field.key] ?? (field.type === 'CheckboxSearchField' ? false : '')}
onChange={(value) => onSearchFieldChange?.(field.key, value)}
onSubmit={onSubmit}
const settingsForm = (
<div className={wrapperClassName}>
{isAdmin && (
<>
<div className="grid grid-cols-1 md:grid-cols-2 gap-4 mb-4">
<DropdownList
label="Search Mode"
options={SEARCH_MODE_OPTIONS}
value={searchMode}
onChange={(value) => {
const next = Array.isArray(value) ? value[0] ?? 'direct' : value;
onSearchModeChange(next === 'universal' ? 'universal' : 'direct');
}}
placeholder="Choose a mode"
widthClassName="w-full"
/>
{field.description && (
<p className="text-xs mt-1 opacity-60">{field.description}</p>
{searchMode === 'universal' && (
<DropdownList
label={contentType === 'audiobook' ? 'Audiobook Metadata Provider' : 'Book Metadata Provider'}
options={providerOptions}
value={activeMetadataProvider ?? ''}
onChange={(value) => {
const next = Array.isArray(value) ? value[0] ?? '' : value;
onMetadataProviderChange?.(next);
}}
placeholder="Choose a provider"
widthClassName="w-full"
/>
)}
</div>
))}
</form>
);
</>
)}
const wrappedUniversalForm = renderWrapper ? (
renderWrapper(universalForm)
) : (
<div className="w-full border-b pt-6 pb-4 mb-4" style={{ borderColor: 'var(--border-muted)' }}>
<div className="w-full px-4 sm:px-6 lg:px-8">{universalForm}</div>
</div>
);
return wrappedUniversalForm;
}
// Direct download mode: render existing hardcoded filters
const form = (
<form
id="search-filters"
className={
formClassName ??
'grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4 px-2 lg:ml-[calc(3rem+1rem)] lg:w-[50vw]'
}
>
<div>
<label htmlFor="isbn-input" className="block text-sm mb-1 opacity-80">
ISBN
</label>
<input
id="isbn-input"
type="text"
placeholder="ISBN"
autoComplete="off"
enterKeyHint="search"
className="w-full px-3 py-2 text-sm rounded-lg border"
style={{
background: 'var(--bg-soft)',
color: 'var(--text)',
borderColor: 'var(--border-muted)',
}}
value={isbn}
onChange={e => {
onFiltersChange({ isbn: e.target.value });
}}
onKeyDown={handleKeyDown}
{searchMode === 'direct' && (
<div className="space-y-4">
<form
id="search-filters"
className={
formClassName ??
'grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4'
}
>
<LanguageMultiSelect
options={bookLanguages}
value={lang}
onChange={handleLangChange}
defaultLanguageCodes={defaultLanguage}
label="Language"
/>
</div>
<div>
<label htmlFor="author-input" className="block text-sm mb-1 opacity-80">
Author
</label>
<input
id="author-input"
type="text"
placeholder="Author"
autoComplete="off"
enterKeyHint="search"
className="w-full px-3 py-2 text-sm rounded-lg border"
style={{
background: 'var(--bg-soft)',
color: 'var(--text)',
borderColor: 'var(--border-muted)',
}}
value={author}
onChange={e => {
onFiltersChange({ author: e.target.value });
}}
onKeyDown={handleKeyDown}
<DropdownList
label="Content"
options={CONTENT_OPTIONS}
value={content}
onChange={handleContentChange}
placeholder="All"
/>
</div>
<div>
<label htmlFor="title-input" className="block text-sm mb-1 opacity-80">
Title
</label>
<input
id="title-input"
type="text"
placeholder="Title"
autoComplete="off"
enterKeyHint="search"
className="w-full px-3 py-2 text-sm rounded-lg border"
style={{
background: 'var(--bg-soft)',
color: 'var(--text)',
borderColor: 'var(--border-muted)',
}}
value={title}
onChange={e => {
onFiltersChange({ title: e.target.value });
}}
onKeyDown={handleKeyDown}
/>
</div>
<LanguageMultiSelect
options={bookLanguages}
value={lang}
onChange={handleLangChange}
defaultLanguageCodes={defaultLanguage}
label="Language"
/>
<DropdownList
label="Content"
options={CONTENT_OPTIONS}
value={content}
onChange={handleContentChange}
placeholder="All"
/>
<div>
<DropdownList
label="Formats"
placeholder="Any"
@@ -221,17 +160,17 @@ export const AdvancedFilters = ({
showCheckboxes
keepOpenOnSelect
/>
</div>
</form>
);
const wrappedForm = renderWrapper ? (
renderWrapper(form)
) : (
<div className="w-full border-b pt-6 pb-4 mb-4" style={{ borderColor: 'var(--border-muted)' }}>
<div className="w-full px-4 sm:px-6 lg:px-8">{form}</div>
</form>
</div>
)}
</div>
);
return wrappedForm;
return renderWrapper ? (
renderWrapper(settingsForm)
) : (
<div className="w-full border-b pt-6 pb-4 mb-4" style={{ borderColor: 'var(--border-muted)' }}>
<div className="w-full px-4 sm:px-6 lg:px-8">{settingsForm}</div>
</div>
);
};
@@ -69,6 +69,11 @@ export const BookDownloadButton = ({
const displayText = isQueuing ? 'Queuing...' : buttonState.text;
const showCircularProgress = buttonState.state === 'downloading' && buttonState.progress !== undefined;
const showSpinner = (isInProgress && !showCircularProgress) || isQueuing;
const isRequestAction = buttonState.state === 'download' && buttonState.text === 'Request';
const iconVariantActionIconPath = isRequestAction
? 'M12 4.5v15m7.5-7.5h-15'
: 'M3 16.5v2.25A2.25 2.25 0 0 0 5.25 21h13.5A2.25 2.25 0 0 0 21 18.75V16.5M16.5 12 12 16.5m0 0L7.5 12m4.5 4.5V3';
const primaryActionIconPath = isRequestAction ? 'M12 4.5v15m7.5-7.5h-15' : 'M12 4v12m0 0l-4-4m4 4 4-4M6 20h12';
const primaryStateClasses =
isCompleted
@@ -198,10 +203,10 @@ export const BookDownloadButton = ({
return (
<>
<svg className={`${iconSizes.mobile} sm:hidden`} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M3 16.5v2.25A2.25 2.25 0 0 0 5.25 21h13.5A2.25 2.25 0 0 0 21 18.75V16.5M16.5 12 12 16.5m0 0L7.5 12m4.5 4.5V3" />
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d={iconVariantActionIconPath} />
</svg>
<svg className={`${iconSizes.desktop} hidden sm:block`} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M3 16.5v2.25A2.25 2.25 0 0 0 5.25 21h13.5A2.25 2.25 0 0 0 21 18.75V16.5M16.5 12 12 16.5m0 0L7.5 12m4.5 4.5V3" />
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d={iconVariantActionIconPath} />
</svg>
</>
);
@@ -221,7 +226,7 @@ export const BookDownloadButton = ({
>
{variant === 'primary' && showIcon && !isCompleted && !hasError && !showCircularProgress && !showSpinner && (
<svg className={primaryIconSizes[size]} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 4v12m0 0l-4-4m4 4 4-4M6 20h12" />
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d={primaryActionIconPath} />
</svg>
)}
@@ -0,0 +1,248 @@
import { useCallback, useEffect, useMemo, useState } from 'react';
import { DropdownList, type DropdownListOption } from './DropdownList';
import {
setBookTargetState,
type BookTargetOption,
} from '../services/api';
import { loadBookTargets } from '../utils/bookTargetLoader';
import { emitBookTargetChange, onBookTargetChange } from '../utils/bookTargetEvents';
interface BookTargetDropdownProps {
provider: string;
bookId: string;
onShowToast?: (message: string, type: 'success' | 'error' | 'info') => void;
widthClassName?: string;
variant?: 'default' | 'pill' | 'icon';
align?: 'left' | 'right' | 'auto';
className?: string;
onOpenChange?: (isOpen: boolean) => void;
}
const stripCountSuffix = (label: string): string => {
return label.replace(/\s+\(\d+\)\s*$/, '');
};
const BookmarkIcon = ({ className = 'h-4 w-4' }: { className?: string }) => (
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth="1.5"
stroke="currentColor"
aria-hidden="true"
className={`${className} flex-shrink-0`}
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M17.593 3.322c1.1.128 1.907 1.077 1.907 2.185V21L12 17.25 4.5 21V5.507c0-1.108.806-2.057 1.907-2.185a48.507 48.507 0 0 1 11.186 0Z"
/>
</svg>
);
const renderSummary = (selectedOptions: DropdownListOption[]) => {
const count = selectedOptions.length;
return (
<span className="inline-flex items-center gap-1.5 whitespace-nowrap">
<BookmarkIcon />
<span>Hardcover Lists{count > 0 ? ` (${count})` : ''}</span>
</span>
);
};
const updateOptionChecked = (
prev: BookTargetOption[],
target: string,
checked: boolean,
): BookTargetOption[] =>
prev.map((option) =>
option.value === target ? { ...option, checked } : option,
);
export const BookTargetDropdown = ({
provider,
bookId,
onShowToast,
widthClassName = 'w-full sm:w-56',
variant = 'default',
align = 'auto',
className,
onOpenChange,
}: BookTargetDropdownProps) => {
const [options, setOptions] = useState<BookTargetOption[]>([]);
const [isLoading, setIsLoading] = useState(true);
const [loadError, setLoadError] = useState<string | null>(null);
const [pendingTargets, setPendingTargets] = useState<Set<string>>(new Set());
useEffect(() => {
let isMounted = true;
const run = async () => {
try {
const loaded = await loadBookTargets(provider, bookId);
if (!isMounted) return;
setOptions(loaded);
setLoadError(null);
} catch (error) {
if (!isMounted) return;
const message = error instanceof Error ? error.message : 'Failed to load Hardcover lists';
setOptions([]);
setLoadError(message);
} finally {
if (isMounted) {
setIsLoading(false);
}
}
};
setLoadError(null);
setPendingTargets(new Set());
setIsLoading(true);
void run();
return () => {
isMounted = false;
};
}, [provider, bookId]);
// Sync from changes made by other BookTargetDropdown instances for the same book
useEffect(() => {
return onBookTargetChange((event) => {
if (event.provider !== provider || event.bookId !== bookId) return;
setOptions((prev) => updateOptionChecked(prev, event.target, event.selected));
});
}, [provider, bookId]);
const selectedValues = useMemo(
() => options.filter((option) => option.checked).map((option) => option.value),
[options],
);
const dropdownOptions = useMemo<DropdownListOption[]>(() => {
if (isLoading) {
return [{ value: '__loading', label: 'Loading…', disabled: true }];
}
if (loadError) {
return [{ value: '__error', label: loadError, disabled: true }];
}
if (options.length === 0) {
return [{ value: '__empty', label: 'No writable Hardcover targets', disabled: true }];
}
return options.map((option) => ({
value: option.value,
label: option.label,
description: option.description,
disabled: !option.writable || pendingTargets.has(option.value),
}));
}, [isLoading, loadError, options, pendingTargets]);
const handleChange = useCallback((nextValue: string[] | string) => {
if (!Array.isArray(nextValue)) {
return;
}
const nextSelected = new Set(nextValue);
const currentSelected = new Set(selectedValues);
const toggledTarget =
nextValue.find((value) => !currentSelected.has(value))
?? selectedValues.find((value) => !nextSelected.has(value));
if (!toggledTarget || pendingTargets.has(toggledTarget)) {
return;
}
const selected = nextSelected.has(toggledTarget);
const toggledOption = options.find((option) => option.value === toggledTarget);
if (!toggledOption) {
return;
}
setPendingTargets((prev) => new Set(prev).add(toggledTarget));
setOptions((prev) => updateOptionChecked(prev, toggledTarget, selected));
void (async () => {
try {
const result = await setBookTargetState(provider, bookId, toggledTarget, selected);
setOptions((prev) => updateOptionChecked(prev, toggledTarget, result.selected));
if (result.changed) {
emitBookTargetChange({
provider,
bookId,
target: toggledTarget,
selected: result.selected,
});
const label = stripCountSuffix(toggledOption.label);
onShowToast?.(
`${result.selected ? 'Added to' : 'Removed from'} ${label}`,
'success',
);
}
} catch (error) {
setOptions((prev) => updateOptionChecked(prev, toggledTarget, !selected));
const message = error instanceof Error ? error.message : 'Failed to update Hardcover list';
onShowToast?.(message, 'error');
} finally {
setPendingTargets((prev) => {
const nextPending = new Set(prev);
nextPending.delete(toggledTarget);
return nextPending;
});
}
})();
}, [bookId, onShowToast, options, pendingTargets, provider, selectedValues]);
const customTrigger = variant === 'pill'
? ({ toggle }: { isOpen: boolean; toggle: () => void }) => {
const count = selectedValues.length;
return (
<button
type="button"
onClick={toggle}
className={`inline-flex items-center gap-1 px-2 py-1 text-xs font-medium rounded-full transition-colors text-emerald-600 dark:text-emerald-400 bg-emerald-50 dark:bg-emerald-900/20 hover:bg-emerald-100 dark:hover:bg-emerald-900/40 focus:outline-none`}
>
<BookmarkIcon className="w-3 h-3" />
Hardcover Lists{count > 0 ? ` (${count})` : ''}
</button>
);
}
: variant === 'icon'
? ({ toggle }: { isOpen: boolean; toggle: () => void }) => {
const count = selectedValues.length;
return (
<button
type="button"
onClick={(e) => { e.stopPropagation(); toggle(); }}
className={`flex items-center justify-center rounded-full transition-colors duration-200 focus:outline-none ${className ?? 'p-1.5 sm:p-2 text-gray-600 dark:text-gray-200 hover-action'}`}
aria-label="Hardcover Lists"
title={count > 0 ? `On ${count} Hardcover list${count > 1 ? 's' : ''}` : 'Hardcover Lists'}
>
<BookmarkIcon className={`w-4 h-4 sm:w-5 sm:h-5 ${count > 0 ? 'fill-current' : ''}`} />
</button>
);
}
: undefined;
return (
<DropdownList
options={dropdownOptions}
value={selectedValues}
onChange={handleChange}
placeholder={isLoading ? 'Loading…' : 'Lists & Want to Read'}
widthClassName={variant !== 'default' ? 'w-auto' : widthClassName}
buttonClassName={variant !== 'default' ? '' : 'py-1.5 leading-none'}
panelClassName={variant !== 'default' ? 'w-56' : undefined}
align={align}
multiple
showCheckboxes
keepOpenOnSelect
summaryFormatter={(selectedOptions) => renderSummary(selectedOptions)}
renderTrigger={customTrigger}
onOpenChange={onOpenChange}
/>
);
};
+48 -20
View File
@@ -2,17 +2,28 @@ import { useState, useEffect, useCallback } from 'react';
import { createPortal } from 'react-dom';
import { Book, ButtonStateInfo, isMetadataBook } from '../types';
import { isUserCancelledError } from '../utils/errors';
import { BookTargetDropdown } from './BookTargetDropdown';
import { bookSupportsTargets } from '../utils/bookTargetLoader';
interface DetailsModalProps {
book: Book | null;
onClose: () => void;
onDownload: (book: Book) => Promise<void>;
onFindDownloads?: (book: Book) => void; // For Universal mode
onSearchSeries?: (seriesName: string) => void; // Callback to search for series
onSearchSeries?: (seriesName: string, seriesId?: string) => void; // Callback to search for series
buttonState: ButtonStateInfo;
onShowToast?: (message: string, type: 'success' | 'error' | 'info') => void;
}
export const DetailsModal = ({ book, onClose, onDownload, onFindDownloads, onSearchSeries, buttonState }: DetailsModalProps) => {
export const DetailsModal = ({
book,
onClose,
onDownload,
onFindDownloads,
onSearchSeries,
buttonState,
onShowToast,
}: DetailsModalProps) => {
const [isQueuing, setIsQueuing] = useState(false);
const [isClosing, setIsClosing] = useState(false);
@@ -56,6 +67,8 @@ export const DetailsModal = ({ book, onClose, onDownload, onFindDownloads, onSea
}
}, [book]);
const hasBookTargets = Boolean(book && isMetadataBook(book) && bookSupportsTargets(book));
if (!book && !isClosing) return null;
if (!book) return null;
@@ -78,7 +91,10 @@ export const DetailsModal = ({ book, onClose, onDownload, onFindDownloads, onSea
// Determine if this is a metadata book (Universal mode) vs a release (Direct Download)
const isMetadata = isMetadataBook(book);
const metadataActionText =
isMetadata && buttonState.state === 'download' && buttonState.text === 'Get'
? 'Find Downloads'
: buttonState.text;
const publisherInfo = { label: 'Publisher', value: book.publisher || '-' };
// Build metadata grid based on mode
@@ -280,7 +296,7 @@ export const DetailsModal = ({ book, onClose, onDownload, onFindDownloads, onSea
<button
type="button"
onClick={() => {
onSearchSeries(book.series_name!);
onSearchSeries(book.series_name!, book.series_id);
handleClose();
}}
className="inline-flex items-center gap-1 px-2 py-1 text-xs font-medium text-emerald-600 dark:text-emerald-400 bg-emerald-50 dark:bg-emerald-900/20 rounded-full hover:bg-emerald-100 dark:hover:bg-emerald-900/40 transition-colors flex-shrink-0"
@@ -316,14 +332,14 @@ export const DetailsModal = ({ book, onClose, onDownload, onFindDownloads, onSea
className="border-t border-[var(--border-muted)] bg-[var(--bg)] sm:bg-[var(--bg-soft)] px-5 py-4"
style={{ paddingBottom: 'calc(1rem + env(safe-area-inset-bottom))' }}
>
<div className="flex items-center justify-between gap-4">
<div className="flex flex-col gap-3 sm:flex-row sm:items-center sm:justify-between">
{/* Source link - shown for both Universal and Direct Download modes */}
{book.source_url && (
<a
href={book.source_url}
target="_blank"
rel="noopener noreferrer"
className="inline-flex items-center gap-1.5 rounded-full border border-[var(--border-muted)] bg-[var(--bg)] px-3 py-2 text-xs font-medium text-gray-600 transition-colors hover:border-gray-400 hover:text-gray-900 dark:text-gray-400 dark:hover:border-gray-500 dark:hover:text-gray-200"
className="inline-flex items-center gap-1.5 text-xs font-medium text-gray-600 transition-colors hover:text-gray-900 dark:text-gray-400 dark:hover:text-gray-200"
>
View on {isMetadata ? providerDisplay : "Source"}
<svg className="h-3 w-3" fill="none" stroke="currentColor" viewBox="0 0 24 24">
@@ -336,20 +352,32 @@ export const DetailsModal = ({ book, onClose, onDownload, onFindDownloads, onSea
</svg>
</a>
)}
{/* Action button - Find Downloads (Universal) or Download (Direct) */}
<button
onClick={isMetadata ? () => onFindDownloads?.(book) : handleDownload}
disabled={!isMetadata && buttonState.state !== 'download'}
className={`ml-auto rounded-full px-6 py-2.5 text-sm font-medium text-white transition-colors focus:outline-none focus:ring-2 focus:ring-offset-2 disabled:opacity-50 disabled:cursor-not-allowed ${
isMetadata
? 'bg-emerald-600 hover:bg-emerald-700 focus:ring-emerald-500'
: buttonState.state === 'blocked'
? 'bg-gray-500 focus:ring-gray-400'
: 'bg-sky-700 hover:bg-sky-800 focus:ring-sky-500'
}`}
>
{isMetadata ? 'Find Downloads' : buttonState.text}
</button>
<div className="flex w-full flex-col gap-2 sm:ml-auto sm:w-auto sm:flex-row sm:items-center">
{hasBookTargets && book.provider_id && (
<BookTargetDropdown
provider={book.provider!}
bookId={book.provider_id}
onShowToast={onShowToast}
widthClassName="w-full sm:w-56"
/>
)}
{/* Action button - mirrors search result action state/flow */}
<button
onClick={isMetadata ? () => onFindDownloads?.(book) : handleDownload}
disabled={isMetadata ? buttonState.state === 'blocked' : buttonState.state !== 'download'}
className={`rounded-lg px-5 py-2 text-sm font-medium text-white transition-colors disabled:opacity-50 disabled:cursor-not-allowed ${
isMetadata
? buttonState.state === 'blocked'
? 'bg-gray-500'
: 'bg-emerald-600 hover:bg-emerald-700'
: buttonState.state === 'blocked'
? 'bg-gray-500'
: 'bg-sky-700 hover:bg-sky-800'
}`}
>
{isMetadata ? metadataActionText : buttonState.text}
</button>
</div>
</div>
</footer>
</div>
@@ -1,362 +0,0 @@
import { useEffect } from 'react';
import { StatusData, Book } from '../types';
import { withBasePath } from '../utils/basePath';
interface DownloadsSidebarProps {
isOpen: boolean;
onClose: () => void;
status: StatusData;
onClearCompleted: () => void;
onCancel: (id: string) => void;
}
const STATUS_STYLES: Record<string, { bg: string; text: string; label: string; waveColor: string }> = {
queued: { bg: 'bg-amber-500/20', text: 'text-amber-700 dark:text-amber-300', label: 'Queued', waveColor: 'rgba(217, 119, 6, 0.3)' },
resolving: { bg: 'bg-indigo-500/20', text: 'text-indigo-700 dark:text-indigo-300', label: 'Resolving', waveColor: 'rgba(79, 70, 229, 0.3)' },
downloading: { bg: 'bg-sky-500/20', text: 'text-sky-700 dark:text-sky-300', label: 'Downloading', waveColor: 'rgba(2, 132, 199, 0.3)' },
locating: { bg: 'bg-teal-500/20', text: 'text-teal-700 dark:text-teal-300', label: 'Locating files', waveColor: 'rgba(13, 148, 136, 0.3)' },
complete: { bg: 'bg-green-500/20', text: 'text-green-700 dark:text-green-300', label: 'Complete', waveColor: '' },
error: { bg: 'bg-red-500/20', text: 'text-red-700 dark:text-red-300', label: 'Error', waveColor: '' },
cancelled: { bg: 'bg-gray-500/20', text: 'text-gray-700 dark:text-gray-300', label: 'Cancelled', waveColor: '' },
};
// Book thumbnail component with fallback
const BookThumbnail = ({ preview, title }: { preview?: string; title?: string }) => {
if (!preview) {
return (
<div
className="w-16 h-24 rounded-tl bg-gray-200 dark:bg-gray-700 flex items-center justify-center text-[8px] font-medium text-gray-500 dark:text-gray-400"
style={{ aspectRatio: '2/3' }}
>
No Cover
</div>
);
}
return (
<img
src={preview}
alt={title || 'Book cover'}
className="w-16 h-24 object-cover rounded-tl shadow-sm"
style={{ aspectRatio: '2/3' }}
onError={(e) => {
// Replace with placeholder on error
const target = e.target as HTMLImageElement;
const placeholder = document.createElement('div');
placeholder.className = 'w-16 h-24 rounded-tl bg-gray-200 dark:bg-gray-700 flex items-center justify-center text-[8px] font-medium text-gray-500 dark:text-gray-400';
placeholder.style.aspectRatio = '2/3';
placeholder.textContent = 'No Cover';
target.replaceWith(placeholder);
}}
/>
);
};
// Helper to get progress percentage based on status
const getStatusProgress = (statusName: string, bookProgress?: number): number => {
switch (statusName) {
case 'queued':
return 5;
case 'resolving':
return 15;
case 'downloading':
// Map actual progress (0-100) to 20-100 range
if (typeof bookProgress === 'number') {
return 20 + (bookProgress * 0.8);
}
return 20;
case 'locating':
return 90;
case 'complete':
case 'error':
return 100;
default:
return 0;
}
};
// Helper to get progress bar color based on status
const getProgressBarColor = (statusName: string): string => {
if (statusName === 'complete') return 'bg-green-600';
if (statusName === 'error') return 'bg-red-600';
if (statusName === 'queued') return 'bg-amber-600';
if (statusName === 'resolving') return 'bg-indigo-600';
if (statusName === 'downloading') return 'bg-sky-600';
if (statusName === 'locating') return 'bg-teal-600';
return 'bg-sky-600';
};
export const DownloadsSidebar = ({
isOpen,
onClose,
status,
onClearCompleted,
onCancel,
}: DownloadsSidebarProps) => {
// Handle ESC key to close sidebar
useEffect(() => {
if (!isOpen) return; // Only listen when sidebar is open
const handleEscape = (e: KeyboardEvent) => {
if (e.key === 'Escape') {
onClose();
}
};
document.addEventListener('keydown', handleEscape);
return () => document.removeEventListener('keydown', handleEscape);
}, [isOpen, onClose]);
// Collect all download items from different status sections
const allDownloadItems: Array<{ book: Book; status: string }> = [];
const statusTypes = ['downloading', 'locating', 'resolving', 'queued', 'error', 'complete', 'cancelled'];
statusTypes.forEach((statusName) => {
const items = (status as any)[statusName];
if (items && Object.keys(items).length > 0) {
Object.values(items).forEach((book: any) => {
allDownloadItems.push({ book, status: statusName });
});
}
});
// Sort by added_time descending (newest first)
allDownloadItems.sort((a, b) => (b.book.added_time || 0) - (a.book.added_time || 0));
const renderDownloadItem = (item: { book: Book; status: string }) => {
const { book, status: statusName } = item;
const statusStyle = STATUS_STYLES[statusName] || {
bg: 'bg-gray-500/10',
text: 'text-gray-600',
label: statusName.charAt(0).toUpperCase() + statusName.slice(1),
};
const isInProgress = ['queued', 'resolving', 'locating', 'downloading'].includes(statusName);
const isQueued = statusName === 'queued';
const isActive = statusName === 'resolving' || statusName === 'locating' || statusName === 'downloading';
const isCompleted = statusName === 'complete';
const hasError = statusName === 'error';
// Get progress information
const progress = getStatusProgress(statusName, book.progress);
const progressBarColor = getProgressBarColor(statusName);
// Format progress text - use status_message from backend if available
let progressText = book.status_message || statusStyle.label;
if (statusName === 'downloading' && !book.status_message && book.progress && book.size) {
// Fallback: calculate size progress only if backend didn't provide a message
const sizeValue = parseFloat(book.size.replace(/[^\d.]/g, ''));
const sizeUnit = book.size.replace(/[\d.\s]/g, '');
const downloadedSize = (book.progress / 100) * sizeValue;
progressText = `${downloadedSize.toFixed(1)}${sizeUnit} / ${book.size}`;
} else if (isCompleted) {
progressText = book.status_message || 'Complete';
} else if (hasError) {
progressText = book.status_message || 'Failed';
}
return (
<div
key={book.id}
className="relative rounded-lg border hover:shadow-md transition-shadow overflow-hidden"
style={{ borderColor: 'var(--border-muted)', background: 'var(--bg-soft)' }}
>
{/* Action Button - top right corner */}
<button
type="button"
onClick={(e) => {
e.stopPropagation();
onCancel(book.id);
}}
className={`absolute top-1 right-1 z-10 flex h-8 w-8 items-center justify-center rounded-full transition-colors ${
isActive || isQueued
? 'text-red-600 dark:text-red-400 hover:bg-red-100 dark:hover:bg-red-900/30'
: 'text-gray-500 hover:text-red-600 hover:bg-red-100 dark:hover:bg-red-900/30'
}`}
title={isActive ? 'Stop download' : isQueued ? 'Remove from queue' : 'Clear from list'}
aria-label={isActive ? 'Stop download' : isQueued ? 'Remove from queue' : 'Clear from list'}
>
{isActive ? (
<svg className="w-4 h-4" viewBox="0 0 24 24" fill="currentColor" aria-hidden="true">
<rect x="6" y="6" width="12" height="12" rx="2" />
</svg>
) : (
<svg className="w-4 h-4" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2.5" aria-hidden="true">
<path strokeLinecap="round" strokeLinejoin="round" d="M6 18L18 6M6 6l12 12" />
</svg>
)}
</button>
{/* Main content area */}
<div className="flex gap-2">
{/* Book Thumbnail - left side */}
<div className="flex-shrink-0">
<BookThumbnail preview={book.preview} title={book.title} />
</div>
{/* Book Info - right side */}
<div className="flex-1 min-w-0 flex flex-col pl-1.5 pr-3 pt-2 pb-2">
{/* Title & Author - with safe area for cancel/clear button */}
<div className="pr-6">
<h3 className="font-semibold text-sm truncate" title={book.title}>
{isCompleted && book.download_path ? (
<a
href={withBasePath(`/api/localdownload?id=${encodeURIComponent(book.id)}`)}
className="text-sky-600 hover:underline"
>
{book.title || 'Unknown Title'}
</a>
) : (
book.title || 'Unknown Title'
)}
</h3>
<p className="text-xs opacity-70 truncate" title={book.author}>
{book.author || 'Unknown Author'}
</p>
</div>
{/* Format, Size, Source */}
<div className="text-xs opacity-70 mt-1">
{book.format && <span className="uppercase">{book.format}</span>}
{book.format && book.size && <span> • </span>}
{book.size && <span>{book.size}</span>}
{book.source_display_name && (
<>
<span> • </span>
<span>{book.source_display_name}</span>
</>
)}
{book.username && (
<>
<span> • </span>
<span>{book.username}</span>
</>
)}
</div>
{/* Status Badge */}
<div className="flex justify-end mt-auto pt-1">
<span
className={`relative px-2 py-0.5 rounded-lg text-xs font-medium ${statusStyle.bg} ${statusStyle.text}`}
>
{/* Wave animation overlay for in-progress states */}
{isInProgress && statusStyle.waveColor && (
<span
key={statusName}
className="absolute inset-0 rounded-lg"
style={{
background: `linear-gradient(90deg, transparent 0%, ${statusStyle.waveColor} 50%, transparent 100%)`,
backgroundSize: '200% 100%',
animation: 'wave 2s linear infinite',
}}
/>
)}
<span className="relative">{progressText}</span>
</span>
</div>
</div>
</div>
{/* Progress Bar - at bottom */}
<div className="h-1.5 bg-gray-200 dark:bg-gray-700 overflow-hidden relative">
<div
className={`h-full ${progressBarColor} transition-all duration-300 relative overflow-hidden`}
style={{ width: `${Math.min(100, Math.max(0, progress))}%` }}
>
{/* Animated wave effect for in-progress states */}
{isInProgress && progress < 100 && (
<div
className="absolute inset-0 opacity-30"
style={{
background: 'linear-gradient(90deg, transparent 0%, rgba(255, 255, 255, 0.5) 50%, transparent 100%)',
backgroundSize: '200% 100%',
animation: 'wave 2s ease-in-out infinite',
}}
/>
)}
</div>
</div>
</div>
);
};
return (
<>
{/* Backdrop */}
<div
className={`fixed inset-0 bg-black/50 z-40 transition-opacity duration-300 ${
isOpen ? 'opacity-100' : 'opacity-0 pointer-events-none'
}`}
onClick={onClose}
/>
{/* Sidebar */}
<div
className={`fixed top-0 right-0 h-full w-full sm:w-96 z-50 flex flex-col shadow-2xl transition-transform duration-300 ${
isOpen ? 'translate-x-0' : 'translate-x-full'
}`}
style={{ background: 'var(--bg)' }}
>
{/* Header */}
<div
className="flex items-center justify-between p-4 border-b"
style={{ paddingTop: 'calc(1rem + env(safe-area-inset-top))', borderColor: 'var(--border-muted)' }}
>
<h2 className="text-lg font-semibold">
Downloads{allDownloadItems.length > 0 && ` (${allDownloadItems.length})`}
</h2>
<button
type="button"
onClick={onClose}
className="flex h-10 w-10 items-center justify-center rounded-full hover-action transition-colors"
aria-label="Close sidebar"
>
<svg
className="w-5 h-5"
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth="2"
stroke="currentColor"
>
<path strokeLinecap="round" strokeLinejoin="round" d="M6 18L18 6M6 6l12 12" />
</svg>
</button>
</div>
{/* Queue Items */}
<div
className="flex-1 overflow-y-auto p-4 space-y-3"
style={{ paddingBottom: 'calc(1rem + env(safe-area-inset-bottom))' }}
>
{allDownloadItems.length > 0 ? (
allDownloadItems.map((item) => renderDownloadItem(item))
) : (
<div className="text-center text-sm opacity-70 mt-8">
No downloads in queue
</div>
)}
</div>
{/* Footer */}
<div
className="p-3 border-t flex items-center justify-center"
style={{
borderColor: 'var(--border-muted)',
paddingBottom: 'calc(0.75rem + env(safe-area-inset-bottom))',
}}
>
<button
type="button"
onClick={onClearCompleted}
className="text-sm text-gray-500 dark:text-gray-400 hover:text-gray-700 dark:hover:text-gray-200 transition-colors"
>
Clear Completed
</button>
</div>
</div>
</>
);
};
+49 -16
View File
@@ -6,7 +6,7 @@ function getScrollableAncestor(element: HTMLElement | null): HTMLElement | null
while (current) {
const style = getComputedStyle(current);
const overflowY = style.overflowY;
if (overflowY === 'auto' || overflowY === 'scroll') {
if (overflowY === 'auto' || overflowY === 'scroll' || overflowY === 'hidden') {
return current;
}
current = current.parentElement;
@@ -41,7 +41,7 @@ interface DropdownProps {
label?: string;
summary?: ReactNode;
children: (helpers: { close: () => void }) => ReactNode;
align?: 'left' | 'right';
align?: 'left' | 'right' | 'auto';
widthClassName?: string;
buttonClassName?: string;
panelClassName?: string;
@@ -49,6 +49,8 @@ interface DropdownProps {
renderTrigger?: (props: { isOpen: boolean; toggle: () => void }) => ReactNode;
/** Disable max-height and overflow scrolling (for panels with nested dropdowns) */
noScrollLimit?: boolean;
triggerChrome?: 'default' | 'minimal';
onOpenChange?: (isOpen: boolean) => void;
}
export const Dropdown = ({
@@ -62,18 +64,28 @@ export const Dropdown = ({
disabled = false,
renderTrigger,
noScrollLimit = false,
triggerChrome = 'default',
onOpenChange,
}: DropdownProps) => {
const [isOpen, setIsOpen] = useState(false);
const containerRef = useRef<HTMLDivElement>(null);
const panelRef = useRef<HTMLDivElement>(null);
const [panelDirection, setPanelDirection] = useState<'down' | 'up'>('down');
const [resolvedAlign, setResolvedAlign] = useState<'left' | 'right'>(align === 'right' ? 'right' : 'left');
const toggleOpen = () => {
if (disabled) return;
setIsOpen(prev => !prev);
setIsOpen(prev => {
const next = !prev;
onOpenChange?.(next);
return next;
});
};
const close = () => setIsOpen(false);
const close = () => {
setIsOpen(false);
onOpenChange?.(false);
};
useEffect(() => {
if (!isOpen) return;
@@ -122,7 +134,24 @@ export const Dropdown = ({
const shouldOpenUp = spaceBelow < panelHeight && spaceAbove >= panelHeight;
setPanelDirection(shouldOpenUp ? 'up' : 'down');
}, []);
// Auto horizontal alignment: check if panel overflows viewport right/left
if (align === 'auto') {
const panelWidth = panelRef.current.offsetWidth || panelRef.current.scrollWidth;
const overflowsRight = rect.left + panelWidth > window.innerWidth - 8;
const overflowsLeft = rect.right - panelWidth < 8;
if (overflowsRight && !overflowsLeft) {
setResolvedAlign('right');
} else if (overflowsLeft && !overflowsRight) {
setResolvedAlign('left');
} else {
setResolvedAlign('left');
}
} else {
setResolvedAlign(align === 'right' ? 'right' : 'left');
}
}, [align]);
useLayoutEffect(() => {
if (!isOpen) return;
@@ -155,23 +184,28 @@ export const Dropdown = ({
type="button"
onClick={toggleOpen}
disabled={disabled}
className={`w-full px-3 py-2 text-sm border flex items-center justify-between text-left focus:outline-none focus-visible:outline-none focus-visible:ring-0 focus-visible:ring-offset-0 transition-[border-radius] duration-150 ${buttonClassName}`}
className={`w-full px-3 py-2 text-sm border flex items-center justify-between gap-2 text-left focus:outline-none focus-visible:outline-none focus-visible:ring-0 focus-visible:ring-offset-0 transition-[border-radius] duration-150 ${buttonClassName}`}
style={{
background: 'var(--bg-soft)',
background: triggerChrome === 'minimal' ? 'transparent' : 'var(--bg-soft)',
color: 'var(--text)',
borderColor: 'var(--border-muted)',
borderColor: triggerChrome === 'minimal' ? 'transparent' : 'var(--border-muted)',
borderWidth: triggerChrome === 'minimal' ? 0 : undefined,
borderRadius: isOpen
? panelDirection === 'down'
? '0.5rem 0.5rem 0 0'
: '0 0 0.5rem 0.5rem'
: '0.5rem',
? triggerChrome === 'minimal'
? '0'
: panelDirection === 'down'
? '0.5rem 0.5rem 0 0'
: '0 0 0.5rem 0.5rem'
: triggerChrome === 'minimal'
? '0'
: '0.5rem',
}}
>
<span className="truncate">
<span className="min-w-0 flex-1 truncate">
{summary ?? <span className="opacity-60">Select an option</span>}
</span>
<svg
className={`w-4 h-4 transition-transform ${isOpen ? 'rotate-180' : ''}`}
className={`h-4 w-4 flex-shrink-0 transition-transform ${isOpen ? 'rotate-180' : ''}`}
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
@@ -185,7 +219,7 @@ export const Dropdown = ({
{isOpen && (
<div
ref={panelRef}
className={`absolute ${align === 'right' ? 'right-0' : 'left-0'} ${
className={`absolute ${resolvedAlign === 'right' ? 'right-0' : 'left-0'} ${
panelDirection === 'down'
? renderTrigger ? 'mt-2' : ''
: renderTrigger ? 'bottom-full mb-2' : 'bottom-full'
@@ -211,4 +245,3 @@ export const Dropdown = ({
</div>
);
};
+13 -2
View File
@@ -17,11 +17,15 @@ interface DropdownListProps {
showCheckboxes?: boolean;
value: string[] | string | null | undefined;
onChange: (value: string[] | string) => void;
align?: 'left' | 'right';
align?: 'left' | 'right' | 'auto';
widthClassName?: string;
buttonClassName?: string;
panelClassName?: string;
summaryFormatter?: (selected: DropdownListOption[], placeholder: string) => ReactNode;
keepOpenOnSelect?: boolean;
triggerChrome?: 'default' | 'minimal';
renderTrigger?: (props: { isOpen: boolean; toggle: () => void }) => ReactNode;
onOpenChange?: (isOpen: boolean) => void;
}
export const DropdownList = ({
@@ -35,8 +39,12 @@ export const DropdownList = ({
align,
widthClassName,
buttonClassName,
panelClassName,
summaryFormatter,
keepOpenOnSelect,
triggerChrome = 'default',
renderTrigger,
onOpenChange,
}: DropdownListProps) => {
const selectedValues = normalizeValue(value, multiple);
const selectedOptions = options.filter(opt => selectedValues.includes(opt.value));
@@ -102,6 +110,10 @@ export const DropdownList = ({
align={align}
widthClassName={widthClassName}
buttonClassName={buttonClassName}
panelClassName={panelClassName}
triggerChrome={triggerChrome}
renderTrigger={renderTrigger}
onOpenChange={onOpenChange}
>
{({ close }) => (
<div role="listbox" aria-multiselectable={multiple}>
@@ -159,4 +171,3 @@ const normalizeValue = (value: string[] | string | null | undefined, multiple: b
return [];
};
+184 -9
View File
@@ -1,7 +1,10 @@
import { useState, useEffect, useRef, forwardRef, useImperativeHandle } from 'react';
import { useState, useEffect, useRef, useCallback, useMemo, forwardRef, useImperativeHandle } from 'react';
import { SearchBar, SearchBarHandle } from './SearchBar';
import { ContentType } from '../types';
import { DropdownList } from './DropdownList';
import { getAdminUsers } from '../services/api';
import { ContentType, ActingAsUserSelection, MetadataSearchField, QueryTargetOption } from '../types';
import { ActivityStatusCounts, getActivityBadgeState } from '../utils/activityBadge';
import { formatActingAsUserName } from '../utils/actingAsUser';
import { withBasePath } from '../utils/basePath';
export interface HeaderHandle {
@@ -14,8 +17,9 @@ interface HeaderProps {
debug?: boolean;
logoUrl?: string;
showSearch?: boolean;
searchInput?: string;
onSearchChange?: (value: string) => void;
searchInput?: string | number | boolean;
searchInputLabel?: string;
onSearchChange?: (value: string | number | boolean, label?: string) => void;
onSearch?: () => void;
onAdvancedToggle?: () => void;
isLoading?: boolean;
@@ -29,11 +33,18 @@ interface HeaderProps {
isAuthenticated?: boolean;
username?: string | null;
displayName?: string | null;
actingAsUser?: ActingAsUserSelection | null;
onActingAsUserChange?: (user: ActingAsUserSelection | null) => void;
onLogout?: () => void;
onShowToast?: (message: string, type: 'success' | 'error' | 'info', persistent?: boolean) => string;
onRemoveToast?: (id: string) => void;
contentType?: ContentType;
onContentTypeChange?: (type: ContentType) => void;
allowedContentTypes?: ContentType[];
queryTargets?: QueryTargetOption[];
activeQueryTarget?: string;
onQueryTargetChange?: (target: string) => void;
activeQueryField?: MetadataSearchField | null;
}
export const Header = forwardRef<HeaderHandle, HeaderProps>(({
@@ -43,6 +54,7 @@ export const Header = forwardRef<HeaderHandle, HeaderProps>(({
logoUrl,
showSearch = false,
searchInput = '',
searchInputLabel,
onSearchChange,
onSearch,
onAdvancedToggle,
@@ -57,11 +69,18 @@ export const Header = forwardRef<HeaderHandle, HeaderProps>(({
isAuthenticated = false,
username,
displayName,
actingAsUser = null,
onActingAsUserChange,
onLogout,
onShowToast,
onRemoveToast,
contentType = 'ebook',
onContentTypeChange,
allowedContentTypes,
queryTargets = [],
activeQueryTarget = 'general',
onQueryTargetChange,
activeQueryField = null,
}, ref) => {
const activityBadge = getActivityBadgeState(statusCounts, isAdmin);
const settingsEnabled = canAccessSettings ?? isAdmin;
@@ -76,6 +95,59 @@ export const Header = forwardRef<HeaderHandle, HeaderProps>(({
const [isClosing, setIsClosing] = useState(false);
const [shouldAnimateIn, setShouldAnimateIn] = useState(false);
const dropdownRef = useRef<HTMLDivElement>(null);
const [adminUsers, setAdminUsers] = useState<ActingAsUserSelection[]>([]);
const [isAdminUsersLoading, setIsAdminUsersLoading] = useState(false);
const [adminUsersError, setAdminUsersError] = useState<string | null>(null);
const [hasLoadedAdminUsers, setHasLoadedAdminUsers] = useState(false);
const loadAdminUsers = useCallback(async () => {
if (!isAdmin) {
return;
}
setIsAdminUsersLoading(true);
setAdminUsersError(null);
try {
const users = await getAdminUsers();
const filteredUsers = users.filter((user) => {
if (username && user.username === username) {
return false;
}
return true;
});
setAdminUsers(
filteredUsers.map((user) => ({
id: user.id,
username: user.username,
displayName: user.display_name,
}))
);
setHasLoadedAdminUsers(true);
} catch (error) {
console.error('Failed to load admin users:', error);
setAdminUsersError('Failed to load users');
} finally {
setIsAdminUsersLoading(false);
}
}, [isAdmin, username]);
const actingAsOptions = useMemo(
() => [
{ value: '', label: 'Myself' },
...adminUsers.map((user) => {
const displayLabel = formatActingAsUserName(user);
return {
value: String(user.id),
label: displayLabel,
description: displayLabel !== user.username ? `@${user.username}` : undefined,
};
}),
],
[adminUsers]
);
const selectedActingAsValue = actingAsUser ? String(actingAsUser.id) : '';
const dropdownPanelWidthClass = 'w-48';
useEffect(() => {
const saved = localStorage.getItem('preferred-theme') || 'auto';
@@ -98,6 +170,39 @@ export const Header = forwardRef<HeaderHandle, HeaderProps>(({
return () => mq.removeEventListener('change', handler);
}, []);
useEffect(() => {
if (isAdmin) {
return;
}
setAdminUsers([]);
setAdminUsersError(null);
setIsAdminUsersLoading(false);
setHasLoadedAdminUsers(false);
}, [isAdmin]);
useEffect(() => {
if (!onActingAsUserChange || !actingAsUser) {
return;
}
if (username && actingAsUser.username === username) {
onActingAsUserChange(null);
return;
}
if (hasLoadedAdminUsers && !isAdminUsersLoading) {
const stillAvailable = adminUsers.some((user) => user.id === actingAsUser.id);
if (!stillAvailable) {
onActingAsUserChange(null);
}
}
}, [
onActingAsUserChange,
actingAsUser,
username,
hasLoadedAdminUsers,
isAdminUsersLoading,
adminUsers,
]);
// Helper function to close dropdown with animation
const closeDropdown = () => {
setIsClosing(true);
@@ -150,6 +255,9 @@ export const Header = forwardRef<HeaderHandle, HeaderProps>(({
if (isDropdownOpen) {
closeDropdown();
} else {
if (isAdmin && !hasLoadedAdminUsers && !isAdminUsersLoading) {
void loadAdminUsers();
}
setShouldAnimateIn(true);
setIsDropdownOpen(true);
// Reset animation flag after animation completes
@@ -161,8 +269,26 @@ export const Header = forwardRef<HeaderHandle, HeaderProps>(({
onSearch?.();
};
const handleSearchChange = (value: string) => {
onSearchChange?.(value);
const handleSearchChange = (value: string | number | boolean, label?: string) => {
onSearchChange?.(value, label);
};
const handleActingAsChange = (nextValue: string[] | string) => {
if (Array.isArray(nextValue)) {
return;
}
if (nextValue === '') {
onActingAsUserChange?.(null);
return;
}
const selectedUser = adminUsers.find((user) => String(user.id) === nextValue);
if (!selectedUser) {
return;
}
onActingAsUserChange?.(selectedUser);
};
// Determine if we should show icons only (both URLs configured)
@@ -266,12 +392,18 @@ export const Header = forwardRef<HeaderHandle, HeaderProps>(({
d="M3.75 6.75h16.5M3.75 12h16.5m-16.5 5.25h16.5"
/>
</svg>
{actingAsUser && (
<span
className="absolute top-1 right-1 h-2 w-2 rounded-full bg-sky-500 border border-[var(--bg)]"
title={`Downloading as ${formatActingAsUserName(actingAsUser)}`}
/>
)}
</button>
{/* Dropdown Menu */}
{(isDropdownOpen || isClosing) && (
<div
className={`absolute right-0 mt-2 w-48 rounded-lg shadow-lg border z-50 ${
className={`absolute right-0 mt-2 ${dropdownPanelWidthClass} rounded-lg shadow-lg border z-50 ${
isClosing ? 'animate-fade-out-up' : shouldAnimateIn ? 'animate-fade-in-down' : ''
}`}
style={{
@@ -441,6 +573,44 @@ export const Header = forwardRef<HeaderHandle, HeaderProps>(({
</div>
</div>
)}
{isAdmin && onActingAsUserChange && (
<div
className="border-t px-4 py-3 space-y-2"
style={{ borderColor: 'var(--border-muted)' }}
>
<div className="text-xs font-medium uppercase tracking-wide opacity-70">
Download as
</div>
<div className={isAdminUsersLoading ? 'pointer-events-none opacity-60' : ''}>
<DropdownList
options={actingAsOptions}
value={selectedActingAsValue}
onChange={handleActingAsChange}
placeholder="Myself"
widthClassName="w-full"
buttonClassName="rounded-lg text-sm"
/>
</div>
{isAdminUsersLoading && (
<div className="text-xs opacity-70">Loading users...</div>
)}
{adminUsersError && (
<div className="flex items-center justify-between gap-3">
<div className="text-xs text-red-600 dark:text-red-400">
{adminUsersError}
</div>
<button
type="button"
onClick={() => void loadAdminUsers()}
className="text-xs font-medium text-sky-600 hover:text-sky-700 dark:text-sky-400 dark:hover:text-sky-300"
>
Retry
</button>
</div>
)}
</div>
)}
</div>
</div>
)}
@@ -485,15 +655,20 @@ export const Header = forwardRef<HeaderHandle, HeaderProps>(({
)}
<SearchBar
ref={searchBarRef}
className="flex-1 lg:flex-initial"
inputClassName="lg:w-[50vw]"
className="flex-1 lg:w-[calc(50vw+5rem)] lg:flex-none"
value={searchInput}
valueLabel={searchInputLabel}
onChange={handleSearchChange}
onSubmit={handleHeaderSearch}
onAdvancedToggle={onAdvancedToggle}
isLoading={isLoading}
contentType={contentType}
onContentTypeChange={onContentTypeChange}
allowedContentTypes={allowedContentTypes}
queryTargets={queryTargets}
activeQueryTarget={activeQueryTarget}
onQueryTargetChange={onQueryTargetChange}
activeQueryField={activeQueryField}
/>
</div>
</div>
+37 -17
View File
@@ -1,4 +1,5 @@
import { FormEvent, KeyboardEvent, useEffect, useRef, useState } from 'react';
import { useSearchParams } from 'react-router-dom';
import { LoginCredentials } from '../types';
import { withBasePath } from '../utils/basePath';
@@ -9,6 +10,8 @@ interface LoginFormProps {
autoFocus?: boolean;
authMode?: string;
oidcButtonLabel?: string | null;
hideLocalAuth?: boolean;
oidcAutoRedirect?: boolean;
}
const EyeIcon = () => (
@@ -219,9 +222,13 @@ export const LoginForm = ({
autoFocus = true,
authMode,
oidcButtonLabel,
hideLocalAuth = false,
oidcAutoRedirect = false,
}: LoginFormProps) => {
const isOidc = authMode === 'oidc';
const [showPasswordLogin, setShowPasswordLogin] = useState(false);
const [searchParams] = useSearchParams();
const oidcError = searchParams.get('oidc_error');
// Auto-expand password form if there's an error (likely from a password attempt)
useEffect(() => {
@@ -230,6 +237,13 @@ export const LoginForm = ({
}
}, [error, isOidc]);
// Auto-redirect to OIDC provider when enabled and no errors present
useEffect(() => {
if (oidcAutoRedirect && isOidc && !error && !oidcError) {
window.location.href = withBasePath('/api/auth/oidc/login');
}
}, [oidcAutoRedirect, isOidc, error, oidcError]);
const handleSubmit = (e: FormEvent<HTMLFormElement>) => {
e.preventDefault();
const formData = new FormData(e.currentTarget);
@@ -245,11 +259,13 @@ export const LoginForm = ({
}
};
const displayError = oidcError || error;
return (
<div>
{error && (
{displayError && (
<div className="mb-4 p-3 rounded-lg text-sm bg-red-600 text-white">
{error}
{displayError}
</div>
)}
@@ -262,22 +278,26 @@ export const LoginForm = ({
{oidcButtonLabel || 'Sign in with OIDC'}
</a>
<div className="flex items-center mt-5 mb-2">
<div className="flex-1 border-t" style={{ borderColor: 'var(--border-color)' }} />
<button
type="button"
onClick={() => setShowPasswordLogin((prev) => !prev)}
className="px-3 text-sm opacity-60 hover:opacity-100 transition-opacity"
>
{showPasswordLogin ? 'Hide' : 'Use password'}
</button>
<div className="flex-1 border-t" style={{ borderColor: 'var(--border-color)' }} />
</div>
{!hideLocalAuth && (
<>
<div className="flex items-center mt-5 mb-2">
<div className="flex-1 border-t" style={{ borderColor: 'var(--border-color)' }} />
<button
type="button"
onClick={() => setShowPasswordLogin((prev) => !prev)}
className="px-3 text-sm opacity-60 hover:opacity-100 transition-opacity"
>
{showPasswordLogin ? 'Hide' : 'Use password'}
</button>
<div className="flex-1 border-t" style={{ borderColor: 'var(--border-color)' }} />
</div>
{showPasswordLogin && (
<div className="pt-2">
<PasswordLoginForm onSubmit={handleSubmit} isLoading={isLoading} autoFocus={true} />
</div>
{showPasswordLogin && (
<div className="pt-2">
<PasswordLoginForm onSubmit={handleSubmit} isLoading={isLoading} autoFocus={true} />
</div>
)}
</>
)}
</>
) : (

Some files were not shown because too many files have changed in this diff Show More