Compare commits

..
11 Commits
Author SHA1 Message Date
Alex f84fb082ad Fix: AA mirror behavior (#589)
- Refreshed available AA URLs
- Fixed potential redirect from AA itself causing mirror cache errors
- Added fully customizable mirror list in UI
- Segmented rotation behavior to Auto mode only

Fixes #588
2026-02-06 10:04:31 +00:00
Alex b10458a48b Patch: Migrate bypasser to pure CDP + Misc fixes (#575)
Bypasser:
- Refactored internal bypasser logic to use SeleniumBase Pure CDP mode,
removed chromedriver dependencies and UC code.
- Added dedicated threading for internal bypasser functions, fixes any
potential asyncio CPU spike behavior
- Fixed WebGL issue with Chromium 144. Reverted 1.0.3 hotfix and updated
to latest Chromium

Misc: 
- Added M4A color mapping
- Fix frontend language filtering with multi-language releases
- Added "days" age for usenet/torrent releases
- Improved entrypoint chown efficiency
- Added `ONBOARDING` env variable, default true
2026-02-02 20:32:19 +00:00
Andy Kelk f6dba959c9 Fix: Base path resolution timing issue for subpath deployments (#572)
When deployed under a URL prefix (e.g., /shelfmark), images loaded by
React were not respecting the base path, causing 404 errors. The logo
would incorrectly load from /logo.png instead of /shelfmark/logo.png.

The root cause was that the BASE_PATH constant was being initialized at
module load time, before the DOM was fully parsed. This meant
document.querySelector('base') returned null, causing BASE_PATH to
default to '/' regardless of the actual base tag value.

Changed to lazy initialization pattern where the base path is resolved
on first access, ensuring the DOM and base tag are ready.

Fixes [#571](https://github.com/calibrain/shelfmark/issues/571)
2026-02-02 17:26:06 +00:00
Alex e5ccabe1ef Patch: Various additions (#564)
- Added rich Prowlarr search results for whitelisted indexers
- Added torznab query for whitelisted indexers
- Added flags for all Prowlarr indexers
- Added completed external download retry mechanism and "locating" state
- Added client side preference storage of Book/Audiobook search
preference
- Fixed reverse proxy base URL in edge cases
- Added gevent locking for I/O operations, keeps healthcheck alive on
intensive processing operations
- Added M4A supported audiobook option
- Improved file transfer counting and logging with hardlink fallback
warnings
- Fixed proxy auth header for REMOTE_USER scenario
- Dependency tweak for internal bypasser
2026-01-31 12:53:11 +00:00
Marcel Meier 86082c999c Enhance naming templates with arbitrary prefix/suffix support (#560)
As described in https://github.com/calibrain/shelfmark/issues/559
I would like the option to use prefix/suffix text as part of my file
handling.

I appreciate every feedack
2026-01-30 13:53:05 +00:00
Webysther Sperandio 301b2e5456 Update readme (#557)
Confirmed working with automatic importing in calibre and calibre-web.
2026-01-29 19:39:05 +00:00
Ryan Dawes 4fde128fc7 Feature: Add indexer flags to results (#539)
Display Prowlarr indexer flags by rendering them as distinct,
color-coded badges.

- [New] TAGS Render Type: Added support for a TAGS column type that
renders a list of strings as distinct badges.
- Updated  `ReleaseCell` to handle the TAGS type:
  - Desktop: Renders distinct badges side-by-side.
- Mobile: Renders as a comma-separated text list (e.g., "FREELEECH,
DOUBLE UPLOAD").
- Styling: Added dynamic colors for common flags:
  - Freeleech → Green
  - Double Upload → Blue
  - VIP → Amber
  - Sticky → Yellow
- Prowlarr Source: Updated the "Flags" column to use the new TAGS render
type, enable uppercase styling, and show on mobile devices.

Desktop screenshot:
<img width="2042" height="110" alt="CleanShot 2026-01-25 at 21 32 24@2x"
src="https://github.com/user-attachments/assets/d135b1d6-176c-4cb9-afa8-fbbab0bcbc06"
/>

Mobile screenshot:
<img width="567" height="51" alt="image"
src="https://github.com/user-attachments/assets/b5b38a05-2466-4b6f-b4c4-ab6cce745408"
/>
2026-01-29 19:38:28 +00:00
Patrick VeverkaandCopilot d050417e01 fix newer versions of rtorrent (#549)
Closes https://github.com/calibrain/shelfmark/issues/534

This pull request enhances the rTorrent client testing and
implementation by adding more robust checks for directory paths and
improving how the base path is retrieved. The main focus is on verifying
and obtaining the correct download and base directories for torrents.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-29 18:41:19 +00:00
KhakisandClaude Opus 4.5 0a7785a333 Docs: Subpath reverse proxy configuration (#542)
## Summary

This PR updates the reverse proxy documentation with comprehensive
configuration examples for subpath deployments, addressing several
issues discovered when running Shelfmark behind nginx at a subpath like
`/shelfmark/`.

## Changes

- **Root path setup**: Added complete nginx server block example
- **Subpath setup without auth**: Complete nginx configuration with all
necessary workarounds
- **Subpath setup with Authelia**: Full example including Authelia
snippets and Shelfmark proxy auth settings
- **Known issues section**: Documents the frontend bugs that require
workarounds

## Issues Addressed

The current documentation's simple example doesn't work for subpath
deployments because:

1. **Socket.IO connects to root**: Frontend connects to `/socket.io/`
instead of `/shelfmark/socket.io/`
2. **API calls use root path**: Cover images request `/api/` instead of
`/shelfmark/api/`
3. **Logo uses root path**: Requested from `/logo.png` instead of
`/shelfmark/logo.png`
4. **Socket.IO backend path**: Always at `/socket.io/` regardless of
`URL_BASE` setting

## Testing

Tested with:
- Nginx reverse proxy
- Authelia authentication proxy
- `URL_BASE=/shelfmark/` configuration
- WebSocket connections working
- Cover images loading
- Proxy authentication with admin group restrictions

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 20:03:35 +00:00
Alex 10bfaec793 Fix: Broken bypasser dependencies (#540) 2026-01-26 08:02:26 +00:00
Alex 1f093de763 Fix: Bypass activation on non-fast downloads (#536) 2026-01-25 20:22:07 +00:00
63 changed files with 3606 additions and 1052 deletions
+4 -4
View File
@@ -130,10 +130,11 @@ RUN apt-get update && \
xvfb \
# For screen recording
ffmpeg \
# --- Chromium ---
# --- Chromium (unpinned - uses latest from Debian repos) ---
# Chrome 144+ requires --enable-unsafe-swiftshader for WebGL in Docker.
# This flag is set in internal_bypasser.py _get_browser_args()
chromium \
# --- ChromeDriver ---
chromium-driver \
chromium-common \
# For tkinter (pyautogui)
python3-tk \
# For RAR extraction
@@ -151,7 +152,6 @@ RUN --mount=type=cache,target=/root/.cache/pip \
# Grant read/execute permissions to others
RUN chmod -R o+rx /usr/bin/chromium && \
chmod -R o+rx /usr/bin/chromedriver && \
chmod -R o+rwx /usr/local/lib/python3.10/site-packages/seleniumbase/drivers/
# Default command to run the application entrypoint script
+39
View File
@@ -0,0 +1,39 @@
# Bypass testing - switch between dev build and v1.0.1
# Usage:
# Test dev build: docker compose -f docker-compose.bypass-test.yml up shelfmark-dev
# Test v1.0.1: docker compose -f docker-compose.bypass-test.yml up shelfmark-stable
# Pull latest dev: docker compose -f docker-compose.bypass-test.yml build shelfmark-dev
# Pull v1.0.1: docker compose -f docker-compose.bypass-test.yml pull shelfmark-stable
services:
# Dev image from registry
shelfmark-dev:
image: ghcr.io/calibrain/shelfmark:dev
container_name: shelfmark-bypass-dev
environment:
PUID: 1000
PGID: 1000
DEBUG: true
ports:
- 8084:8084
volumes:
- ./.local/bypass-test/config-dev:/config
- ./.local/bypass-test/books:/books
- ./.local/bypass-test/log-dev:/var/log/shelfmark
- ./.local/bypass-test/tmp:/tmp/shelfmark
# Stable v1.0.1 for comparison
shelfmark-stable:
image: ghcr.io/calibrain/shelfmark:1.0.1
container_name: shelfmark-bypass-stable
environment:
PUID: 1000
PGID: 1000
DEBUG: true
ports:
- 8085:8084
volumes:
- ./.local/bypass-test/config-stable:/config
- ./.local/bypass-test/books:/books
- ./.local/bypass-test/log-stable:/var/log/shelfmark
- ./.local/bypass-test/tmp:/tmp/shelfmark
+34 -16
View File
@@ -36,12 +36,13 @@ These environment variables are used at startup before the settings system loads
| `CONFIG_DIR` | Directory for storing configuration files and plugin settings. | string (path) | `/config` |
| `LOG_ROOT` | Root directory for log files. | string (path) | `/var/log/` |
| `TMP_DIR` | Staging directory for downloads before moving to destination. | string (path) | `/tmp/shelfmark` |
| `ENABLE_LOGGING` | Enable file logging to LOG_ROOT/shelfmark/shelfmark.log. | boolean | `true` |
| `ENABLE_LOGGING` | Enable file logging under LOG_ROOT/shelfmark/ (including shelfmark.log and startup logs). | boolean | `true` |
| `FLASK_HOST` | Host address for the Flask web server. | string | `0.0.0.0` |
| `FLASK_PORT` | Port number for the Flask web server. | number | `8084` |
| `SESSION_COOKIE_SECURE` | Enable secure cookies (requires HTTPS). | boolean | `false` |
| `CWA_DB_PATH` | Path to the Calibre-Web database for authentication integration. | string (path) | `/auth/app.db` |
| `DOCKERMODE` | Indicates the application is running inside a Docker container. | boolean | `false` |
| `ONBOARDING` | Show the onboarding wizard on first run. Set to false to skip (useful for ephemeral storage). | boolean | `true` |
<details>
<summary>Detailed descriptions</summary>
@@ -69,7 +70,7 @@ Staging directory for downloads before moving to destination.
#### `ENABLE_LOGGING`
Enable file logging to LOG_ROOT/shelfmark/shelfmark.log.
Enable file logging under LOG_ROOT/shelfmark/ (including shelfmark.log and startup logs).
- **Type:** boolean
- **Default:** `true`
@@ -109,6 +110,13 @@ Indicates the application is running inside a Docker container.
- **Type:** boolean
- **Default:** `false`
#### `ONBOARDING`
Show the onboarding wizard on first run. Set to false to skip (useful for ephemeral storage).
- **Type:** boolean
- **Default:** `true`
</details>
## General
@@ -243,8 +251,8 @@ The release source tab to open by default in the release modal.
| `BOOKS_OUTPUT_MODE` | Choose where completed book files are sent. | string (choice) | `folder` |
| `INGEST_DIR` | Directory where downloaded files are saved. | string | `/books` |
| `FILE_ORGANIZATION` | Choose how downloaded book files are named and organized. | string (choice) | `rename` |
| `TEMPLATE_RENAME` | Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Rename templates are filename-only (no '/' or '\'); use Organize for folders. | string | `{Author} - {Title} ({Year})` |
| `TEMPLATE_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle} | string | `{Author}/{Title} ({Year})` |
| `TEMPLATE_RENAME` | Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders. | string | `{Author} - {Title} ({Year})` |
| `TEMPLATE_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. | string | `{Author}/{Title} ({Year})` |
| `HARDLINK_TORRENTS` | Create hardlinks instead of copying. Preserves seeding but archives won't be extracted. Don't use if destination is a library ingest folder. | boolean | `false` |
| `BOOKLORE_HOST` | Base URL of your Booklore instance | string | _none_ |
| `BOOKLORE_USERNAME` | Booklore account username | string | _none_ |
@@ -253,8 +261,8 @@ The release source tab to open by default in the release modal.
| `BOOKLORE_PATH_ID` | Booklore library path for uploads. | string (choice) | _none_ |
| `DESTINATION_AUDIOBOOK` | Leave empty to use Books destination. | string | _none_ |
| `FILE_ORGANIZATION_AUDIOBOOK` | Choose how downloaded audiobook files are named and organized. | string (choice) | `rename` |
| `TEMPLATE_AUDIOBOOK_RENAME` | Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Rename templates are filename-only (no '/' or '\'); use Organize for folders. | string | `{Author} - {Title}` |
| `TEMPLATE_AUDIOBOOK_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber} | string | `{Author}/{Title}` |
| `TEMPLATE_AUDIOBOOK_RENAME` | Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders. | string | `{Author} - {Title}` |
| `TEMPLATE_AUDIOBOOK_ORGANIZE` | Use / to create folders. Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. | string | `{Author}/{Title}` |
| `HARDLINK_TORRENTS_AUDIOBOOK` | Create hardlinks instead of copying. Preserves seeding but archives won't be extracted. Don't use if destination is a library ingest folder. | boolean | `true` |
| `AUTO_OPEN_DOWNLOADS_SIDEBAR` | Automatically open the downloads sidebar when a new download is queued. | boolean | `false` |
| `DOWNLOAD_TO_BROWSER` | Automatically download completed files to your browser. | boolean | `false` |
@@ -298,7 +306,7 @@ Choose how downloaded book files are named and organized.
**Naming Template**
Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Rename templates are filename-only (no '/' or '\'); use Organize for folders.
Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders.
- **Type:** string
- **Default:** `{Author} - {Title} ({Year})`
@@ -307,7 +315,7 @@ Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}
**Path Template**
Use / to create folders. Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}
Use / to create folders. Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.
- **Type:** string
- **Default:** `{Author}/{Title} ({Year})`
@@ -394,7 +402,7 @@ Choose how downloaded audiobook files are named and organized.
**Naming Template**
Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Rename templates are filename-only (no '/' or '\'); use Organize for folders.
Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\'); use Organize for folders.
- **Type:** string
- **Default:** `{Author} - {Title}`
@@ -403,7 +411,7 @@ Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {P
**Path Template**
Use / to create folders. Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}
Use / to create folders. Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.
- **Type:** string
- **Default:** `{Author}/{Title}`
@@ -1130,8 +1138,9 @@ Timeout for external bypasser requests in milliseconds.
| Variable | Description | Type | Default |
|----------|-------------|------|---------|
| `AA_BASE_URL` | Select 'Auto' to probe mirrors on startup, or choose a specific mirror. | string (choice) | `auto` |
| `AA_ADDITIONAL_URLS` | Comma-separated list of custom mirror URLs. | string | _none_ |
| `AA_BASE_URL` | Select 'Auto' to try mirrors from your list on startup and fall back on failures. Choosing a specific mirror locks Shelfmark to that mirror (no fallback). | string (choice) | `auto` |
| `AA_MIRROR_URLS` | Editable list of AA mirrors. Used to populate the Primary Mirror dropdown and the order used when Auto is selected. Type a URL and press Enter to add. Order matters for auto-rotation | string | `https://annas-archive.gl,https://annas-archive.li` |
| `AA_ADDITIONAL_URLS` | Deprecated. Use Mirrors instead. This is kept for backwards compatibility with existing installs and environment variables. | string | _none_ |
| `LIBGEN_ADDITIONAL_URLS` | Comma-separated list of custom LibGen mirrors to add to the defaults. | string | _none_ |
| `ZLIB_PRIMARY_URL` | Z-Library mirror to use for downloads. | string (choice) | `https://z-lib.fm` |
| `ZLIB_ADDITIONAL_URLS` | Comma-separated list of custom Z-Library mirror URLs. | string | _none_ |
@@ -1145,17 +1154,26 @@ Timeout for external bypasser requests in milliseconds.
**Primary Mirror**
Select 'Auto' to probe mirrors on startup, or choose a specific mirror.
Select 'Auto' to try mirrors from your list on startup and fall back on failures. Choosing a specific mirror locks Shelfmark to that mirror (no fallback).
- **Type:** string (choice)
- **Default:** `auto`
- **Options:** `auto` (Auto (Recommended)), `https://annas-archive.se` (annas-archive.se), `https://annas-archive.li` (annas-archive.li), `https://annas-archive.pm` (annas-archive.pm), `https://annas-archive.in` (annas-archive.in)
- **Options:** `auto` (Auto (Recommended)), `https://annas-archive.gl` (annas-archive.gl), `https://annas-archive.li` (annas-archive.li)
#### `AA_MIRROR_URLS`
**Mirrors**
Editable list of AA mirrors. Used to populate the Primary Mirror dropdown and the order used when Auto is selected. Type a URL and press Enter to add. Order matters for auto-rotation
- **Type:** string
- **Default:** `https://annas-archive.gl,https://annas-archive.li`
#### `AA_ADDITIONAL_URLS`
**Additional Mirrors**
**Additional Mirrors (Legacy)**
Comma-separated list of custom mirror URLs.
Deprecated. Use Mirrors instead. This is kept for backwards compatibility with existing installs and environment variables.
- **Type:** string
- **Default:** _none_
+136 -24
View File
@@ -1,35 +1,147 @@
# Reverse Proxy & Subpath Hosting
Shelfmark can run behind a reverse proxy at the root path (recommended) or
under a subpath like `/shelfmark`.
Shelfmark can run behind a reverse proxy at the root path (recommended) or under a subpath like `/shelfmark`.
## Subpath setup
## Root path setup (Recommended)
1) Set the base path in Shelfmark:
- UI: Settings → Advanced → Base Path
- Env var: `URL_BASE=/shelfmark`
If you can serve Shelfmark at the root path (`https://shelfmark.example.com/`), leave `URL_BASE` empty. This is the simplest option and avoids extra subpath configuration.
2) Configure your reverse proxy to forward the subpath to Shelfmark and
**strip the prefix** before sending to the backend. The proxy must also allow
WebSocket upgrades for Socket.IO.
```nginx
server {
listen 443 ssl;
server_name shelfmark.example.com;
Example (Nginx-style):
```
location /shelfmark/ {
proxy_pass http://shelfmark:8084/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
location / {
proxy_pass http://shelfmark:8084;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
```
Notes:
- Use a trailing slash on the `location` and `proxy_pass` to ensure the
`/shelfmark` prefix is removed.
- Health checks still work at `/api/health` without the subpath.
## Subpath setup
## Root path setup
Running Shelfmark under a subpath like `/shelfmark` is supported without extra rewrite rules.
If you can serve Shelfmark at the root path (`https://shelfmark.example.com/`),
leave `URL_BASE` empty. This is the simplest option.
### 1. Set the base path in Shelfmark
- **UI**: Settings → Advanced → Base Path → `/shelfmark/`
- **Environment variable**: `URL_BASE=/shelfmark/`
### 2. Configure your reverse proxy
All Shelfmark paths (UI, API, assets, Socket.IO) are served under the base path. A single location block is enough.
---
### Without Authentication Proxy
**Complete Nginx configuration for subpath deployment:**
```nginx
location /shelfmark/ {
proxy_pass http://shelfmark:8084/shelfmark/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
proxy_send_timeout 86400;
proxy_buffering off;
}
```
---
### With Authentication Proxy (Authelia, Authentik, etc.)
Shelfmark supports Proxy Authentication. When enabled, Shelfmark trusts the authenticated user from headers set by your auth proxy.
#### Shelfmark Settings
Configure in Settings → Security:
| Setting | Value |
|---------|-------|
| Authentication Method | Proxy Authentication |
| Proxy Auth User Header | `Remote-User` |
| Proxy Auth Logout URL | `https://auth.example.com/logout` |
| Proxy Auth Admin Group Header | `Remote-Groups` |
| Proxy Auth Admin Group Name | `admins` (or your admin group) |
#### Nginx Configuration with Authelia
This example uses Authelia snippets. Adapt for your auth proxy.
**Authelia auth request snippet** (`/etc/nginx/snippets/authelia-authrequest.conf`):
```nginx
location /authelia {
internal;
proxy_pass http://authelia:9091/api/authz/auth-request;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header Host $host;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
proxy_set_header X-Original-Method $request_method;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
```
**Authelia location snippet** (`/etc/nginx/snippets/authelia-location.conf`):
```nginx
auth_request /authelia;
auth_request_set $target_url $scheme://$http_host$request_uri;
auth_request_set $user $upstream_http_remote_user;
auth_request_set $groups $upstream_http_remote_groups;
auth_request_set $name $upstream_http_remote_name;
auth_request_set $email $upstream_http_remote_email;
proxy_set_header Remote-User $user;
proxy_set_header Remote-Groups $groups;
proxy_set_header Remote-Name $name;
proxy_set_header Remote-Email $email;
error_page 401 =302 https://auth.example.com/?rd=$target_url;
```
**Complete Nginx configuration with Authelia:**
```nginx
# Include Authelia auth endpoint in your server block
include /etc/nginx/snippets/authelia-authrequest.conf;
# Main shelfmark location
location /shelfmark/ {
include /etc/nginx/snippets/authelia-location.conf;
proxy_pass http://shelfmark:8084/shelfmark/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
proxy_send_timeout 86400;
proxy_buffering off;
}
```
---
## Health checks
Health checks work at `/shelfmark/api/health` when using a subpath configuration.
+18 -12
View File
@@ -148,6 +148,7 @@ test_write() {
make_writable() {
folder=$1
did_full_chown=0
set +e
test_write $folder
is_writable=$?
@@ -158,31 +159,36 @@ make_writable() {
echo "Folder $folder is not writable, changing ownership"
change_ownership $folder
chmod -R g+r,g+w $folder || echo "Failed to change group permissions for ${folder}, continuing..."
did_full_chown=1
fi
# Fix any misowned subdirectories/files (e.g., from previous runs as root)
if [ -d "$folder" ]; then
misowned_count=$(find "$folder" -mindepth 1 \( ! -user "$RUN_UID" -o ! -group "$RUN_GID" \) 2>/dev/null | wc -l)
if [ "$misowned_count" -gt 0 ]; then
echo "Fixing ownership of $misowned_count files/directories in $folder"
find "$folder" -mindepth 1 \( ! -user "$RUN_UID" -o ! -group "$RUN_GID" \) \
-exec chown "$RUN_UID:$RUN_GID" {} \; 2>/dev/null || true
fi
if [ "$did_full_chown" -eq 0 ] && [ -d "$folder" ]; then
echo "Checking for misowned files/directories in $folder"
find "$folder" -mindepth 1 \( ! -user "$RUN_UID" -o ! -group "$RUN_GID" \) \
-exec chown "$RUN_UID:$RUN_GID" {} + 2>/dev/null || true
fi
test_write $folder || echo "Failed to test write to ${folder}, continuing..."
}
fix_misowned() {
folder=$1
mkdir -p $folder
echo "Checking for misowned files/directories in $folder"
find "$folder" \( ! -user "$RUN_UID" -o ! -group "$RUN_GID" \) \
-exec chown "$RUN_UID:$RUN_GID" {} + 2>/dev/null || true
}
# Ensure proper ownership of application directories
change_ownership() {
folder=$1
mkdir -p $folder
echo "Changing ownership of $folder to $USERNAME:$RUN_GID"
chown -R "${RUN_UID}" "${folder}" || echo "Failed to change user ownership for ${folder}, continuing..."
chown -R ":${RUN_GID}" "${folder}" || echo "Failed to change group ownership for ${folder}, continuing..."
chown -R "${RUN_UID}:${RUN_GID}" "${folder}" || echo "Failed to change ownership for ${folder}, continuing..."
}
change_ownership /app
change_ownership /var/log/shelfmark
change_ownership /tmp/shelfmark
fix_misowned /app
fix_misowned /var/log/shelfmark
fix_misowned /tmp/shelfmark
# SeleniumBase (internal bypasser) writes a patched chromedriver binary (uc_driver)
# into its own drivers directory. Some NAS/docker setups can apply restrictive ACLs
+6 -1
View File
@@ -6,7 +6,12 @@ Formerly *Calibre Web Automated Book Downloader (CWABD)*
Shelfmark is a unified web interface for searching and aggregating books and audiobook downloads from multiple sources - all in one place. Works out of the box with popular web sources, no configuration required. Add metadata providers, additional release sources, and download clients to create a single hub for building your digital library.
**Fully standalone** - no external dependencies required. Works great alongside library tools like [Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated), [Booklore](https://github.com/booklore-app/booklore) or [Audiobookshelf](https://github.com/advplyr/audiobookshelf) for automatic import.
**Fully standalone** - no external dependencies required. Works great alongside the following library tools, with support for automatic imports:
- [Calibre](https://calibre-ebook.com/)
- [Calibre-Web](https://github.com/janeczku/calibre-web)
- [Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated)
- [Booklore](https://github.com/booklore-app/booklore)
- [Audiobookshelf](https://github.com/advplyr/audiobookshelf)
## ✨ Features
+1 -1
View File
@@ -1,4 +1,4 @@
pyvirtualdisplay
pyautogui
seleniumbase>=4.45.6
seleniumbase==4.45.10
python-xlib
+6
View File
@@ -178,6 +178,12 @@ def _generate_bootstrap_env_docs() -> List[str]:
"type": "boolean",
"default": "false",
},
{
"name": "ONBOARDING",
"description": "Show the onboarding wizard on first run. Set to false to skip (useful for ephemeral storage).",
"type": "boolean",
"default": "true",
},
]
lines = [
+20 -2
View File
@@ -434,6 +434,10 @@ def test_rtorrent():
version = client.system.library_version()
print(f" Connected to rTorrent {version}")
# default download directory test
default_dir = client.directory.default()
print(f" Default download directory: {default_dir}")
# Get torrent list
torrents = client.download_list()
print(f" Active torrents: {len(torrents)}")
@@ -458,10 +462,13 @@ def test_rtorrent():
torrent_id = "3B245504CF5F11BBDBE1201CEA6A6BF45AEE1BC0" # rtorrent uses uppercase hashes
print(f" Added test torrent: {torrent_id}")
torrents = client.download_list()
print(f" Active torrents: {len(torrents)}")
torrent_list = client.d.multicall.filtered(
"",
"default",
f"equal=d.hash=,cat={torrent_id}",
f"equal={{d.hash=,cat={torrent_id}}}"
"d.hash=",
"d.state=",
"d.completed_bytes=",
@@ -472,11 +479,22 @@ def test_rtorrent():
"d.complete=",
)
torrent = torrent_list[0]
if not torrent:
print(" ERROR: Could not find added torrent in list")
return False
# let's test the base path call
details = client.d.multicall.filtered(
"",
"default",
f"equal=d.hash=,cat={torrent_id}",
"d.base_path=",
)
base_path = details[0][0] if details else None
print(f" Base path: {base_path}")
client.d.erase(torrent_id)
print(" Removed test torrent")
File diff suppressed because it is too large Load Diff
+8
View File
@@ -133,6 +133,14 @@ TOR_VARIANT_AVAILABLE = shutil.which("tor") is not None
USING_TOR = string_to_bool(os.getenv("USING_TOR", "false"))
# =============================================================================
# Onboarding
# =============================================================================
# Set to false to skip the onboarding wizard entirely (useful for ephemeral storage)
ONBOARDING = string_to_bool(os.getenv("ONBOARDING", "true"))
# =============================================================================
# Debug/development settings
# =============================================================================
+72 -10
View File
@@ -125,6 +125,7 @@ from shelfmark.core.settings_registry import (
CheckboxField,
SelectField,
MultiSelectField,
TagListField,
OrderableListField,
TableField,
HeadingField,
@@ -178,6 +179,7 @@ _FORMAT_OPTIONS = [
_AUDIOBOOK_FORMAT_OPTIONS = [
{"value": "m4b", "label": "M4B"},
{"value": "mp3", "label": "MP3"},
{"value": "m4a", "label": "M4A"},
{"value": "zip", "label": "ZIP"},
{"value": "rar", "label": "RAR"},
]
@@ -224,16 +226,30 @@ _LANGUAGE_OPTIONS = [{"value": lang["code"], "label": lang["language"]} for lang
def _get_aa_base_url_options():
"""Build AA URL options dynamically, including additional mirrors from config."""
from shelfmark.core.mirrors import DEFAULT_AA_MIRRORS, get_aa_mirrors
from shelfmark.core.config import config
from shelfmark.core.utils import normalize_http_url
options = [{"value": "auto", "label": "Auto (Recommended)"}]
# Get all mirrors (defaults + custom)
all_mirrors = get_aa_mirrors()
# If AA_BASE_URL is configured to a custom mirror that isn't present in the
# defaults/additional list, include it so the UI can display the active value.
configured_url = normalize_http_url(
config.get("AA_BASE_URL", "auto"),
default_scheme="https",
allow_special=("auto",),
)
if configured_url and configured_url != "auto" and configured_url not in all_mirrors:
all_mirrors = [configured_url] + all_mirrors
for url in all_mirrors:
domain = url.replace("https://", "").replace("http://", "")
is_custom = url not in DEFAULT_AA_MIRRORS
label = f"{domain} (custom)" if is_custom else domain
if configured_url and url == configured_url and is_custom:
label = f"{domain} (configured)"
options.append({"value": url, "label": label})
return options
@@ -683,7 +699,7 @@ def download_settings():
TextField(
key="TEMPLATE_RENAME",
label="Naming Template",
description="Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Rename templates are filename-only (no '/' or '\\'); use Organize for folders.",
description="Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\\'); use Organize for folders.",
default="{Author} - {Title} ({Year})",
placeholder="{Author} - {Title} ({Year})",
show_when=[
@@ -695,7 +711,7 @@ def download_settings():
TextField(
key="TEMPLATE_ORGANIZE",
label="Path Template",
description="Use / to create folders. Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}",
description="Use / to create folders. Variables: {Author}, {Title}, {Year}. Universal adds: {Series}, {SeriesPosition}, {Subtitle}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.",
default="{Author}/{Title} ({Year})",
placeholder="{Author}/{Series/}{Title} ({Year})",
show_when=[
@@ -799,7 +815,7 @@ def download_settings():
TextField(
key="TEMPLATE_AUDIOBOOK_RENAME",
label="Naming Template",
description="Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Rename templates are filename-only (no '/' or '\\'); use Organize for folders.",
description="Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty. Rename templates are filename-only (no '/' or '\\'); use Organize for folders.",
default="{Author} - {Title}",
placeholder="{Author} - {Title}{ - Part }{PartNumber}",
show_when={"field": "FILE_ORGANIZATION_AUDIOBOOK", "value": "rename"},
@@ -809,7 +825,7 @@ def download_settings():
TextField(
key="TEMPLATE_AUDIOBOOK_ORGANIZE",
label="Path Template",
description="Use / to create folders. Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}",
description="Use / to create folders. Variables: {Author}, {Title}, {Year}, {Series}, {SeriesPosition}, {Subtitle}, {PartNumber}. Use arbitrary prefix/suffix: {Vol. SeriesPosition - } outputs 'Vol. 2 - ' when set, nothing when empty.",
default="{Author}/{Title}",
placeholder="{Author}/{Series/}{Title}{ - Part }{PartNumber}",
show_when={"field": "FILE_ORGANIZATION_AUDIOBOOK", "value": "organize"},
@@ -1102,30 +1118,76 @@ def cloudflare_bypass_settings():
),
]
def _on_save_mirrors(values: Dict[str, Any]) -> Dict[str, Any]:
"""Normalize mirror list settings before persisting."""
from shelfmark.core.logger import setup_logger
from shelfmark.core.mirrors import DEFAULT_AA_MIRRORS
from shelfmark.core.utils import normalize_http_url
logger = setup_logger(__name__)
raw_urls = values.get("AA_MIRROR_URLS")
if raw_urls is None:
return {"error": False, "values": values}
if isinstance(raw_urls, str):
parts = [p.strip() for p in raw_urls.split(",") if p.strip()]
elif isinstance(raw_urls, list):
parts = [str(p).strip() for p in raw_urls if str(p).strip()]
else:
parts = []
normalized: list[str] = []
for url in parts:
if url.lower() == "auto":
continue
norm = normalize_http_url(url, default_scheme="https")
if norm and norm not in normalized:
normalized.append(norm)
if not normalized:
logger.warning("AA_MIRROR_URLS saved empty/invalid; falling back to defaults")
normalized = [normalize_http_url(url, default_scheme="https") for url in DEFAULT_AA_MIRRORS]
normalized = [url for url in normalized if url]
values["AA_MIRROR_URLS"] = normalized
return {"error": False, "values": values}
# Register the on_save handler for this tab
register_on_save("mirrors", _on_save_mirrors)
@register_settings("mirrors", "Mirrors", icon="globe", order=23, group="direct_download")
def mirror_settings():
"""Configure download source mirrors."""
from shelfmark.core.mirrors import DEFAULT_ZLIB_MIRRORS, DEFAULT_WELIB_MIRRORS
from shelfmark.core.mirrors import DEFAULT_AA_MIRRORS, DEFAULT_ZLIB_MIRRORS, DEFAULT_WELIB_MIRRORS
return [
# === PRIMARY SOURCE ===
HeadingField(
key="aa_mirrors_heading",
title="Primary Source",
description="Primary mirror with auto-probe on startup. Additional mirrors used as fallback.",
title="Anna's Archive",
description="Choose a primary mirror, or use Auto to try mirrors from your list below. The mirror list controls which options appear in the dropdown and the order used in Auto mode.",
),
SelectField(
key="AA_BASE_URL",
label="Primary Mirror",
description="Select 'Auto' to probe mirrors on startup, or choose a specific mirror.",
description="Select 'Auto' to try mirrors from your list on startup and fall back on failures. Choosing a specific mirror locks Shelfmark to that mirror (no fallback).",
options=_get_aa_base_url_options,
default="auto",
),
TagListField(
key="AA_MIRROR_URLS",
label="Mirrors",
description="Editable list of AA mirrors. Used to populate the Primary Mirror dropdown and the order used when Auto is selected. Type a URL and press Enter to add. Order matters for auto-rotation",
placeholder="https://annas-archive.gl",
default=DEFAULT_AA_MIRRORS,
),
TextField(
key="AA_ADDITIONAL_URLS",
label="Additional Mirrors",
description="Comma-separated list of custom mirror URLs.",
label="Additional Mirrors (Legacy)",
description="Deprecated. Use Mirrors instead. This is kept for backwards compatibility with existing installs and environment variables.",
show_when={"field": "AA_ADDITIONAL_URLS", "notEmpty": True},
),
# === LIBGEN ===
+33 -10
View File
@@ -19,10 +19,8 @@ def _get_config():
# Default mirror lists (hardcoded fallbacks)
DEFAULT_AA_MIRRORS = [
"https://annas-archive.se",
"https://annas-archive.gl",
"https://annas-archive.li",
"https://annas-archive.pm",
"https://annas-archive.in",
]
DEFAULT_LIBGEN_MIRRORS = [
@@ -52,22 +50,47 @@ def _normalize_mirror_url(url: str) -> str:
def get_aa_mirrors() -> List[str]:
"""
Get Anna's Archive mirrors from config + defaults.
Get Anna's Archive mirrors.
Returns:
List of AA mirror URLs, starting with defaults then custom additions.
Ordered list of AA mirror URLs.
If AA_MIRROR_URLS is configured, it is treated as the full list.
Otherwise, defaults are used and AA_ADDITIONAL_URLS (legacy) is appended.
Notes:
- The list is used to populate the AA mirror dropdown in Settings.
- When AA_BASE_URL is set to 'auto', mirrors are tried in the order listed.
"""
mirrors = [_normalize_mirror_url(url) for url in DEFAULT_AA_MIRRORS]
mirrors = [url for url in mirrors if url]
config = _get_config()
additional = config.get("AA_ADDITIONAL_URLS", "")
if additional:
for url in additional.split(","):
mirrors: list[str] = []
configured_list = config.get("AA_MIRROR_URLS", None)
if isinstance(configured_list, list):
for url in configured_list:
normalized = _normalize_mirror_url(str(url))
if normalized and normalized not in mirrors:
mirrors.append(normalized)
elif isinstance(configured_list, str) and configured_list.strip():
# Allow comma-separated env/manual configs.
for url in configured_list.split(","):
normalized = _normalize_mirror_url(url)
if normalized and normalized not in mirrors:
mirrors.append(normalized)
if not mirrors:
mirrors = [_normalize_mirror_url(url) for url in DEFAULT_AA_MIRRORS]
mirrors = [url for url in mirrors if url]
# Backwards-compatible append-only behavior for legacy configs/env.
additional = config.get("AA_ADDITIONAL_URLS", "")
if additional:
for url in additional.split(","):
normalized = _normalize_mirror_url(url)
if normalized and normalized not in mirrors:
mirrors.append(normalized)
return mirrors
+1
View File
@@ -35,6 +35,7 @@ class QueueStatus(str, Enum):
"""Enum for possible book queue statuses."""
QUEUED = "queued"
RESOLVING = "resolving"
LOCATING = "locating"
DOWNLOADING = "downloading"
COMPLETE = "complete"
AVAILABLE = "available"
+38 -28
View File
@@ -10,11 +10,12 @@ from shelfmark.core.logger import setup_logger
logger = setup_logger(__name__)
TOKEN_PATTERN = re.compile(
r'\{([- ._/\[(]*)' # prefix: space, dash, dot, underscore, slash, brackets
r'([A-Za-z]+)' # token name
r'([- ._/\])]*)\}' # suffix: space, dash, dot, underscore, slash, brackets
)
# Known variable tokens, sorted longest-first to avoid partial matches
# e.g., "SeriesPosition" must match before "Series"
KNOWN_TOKENS = ['seriesposition', 'partnumber', 'subtitle', 'author', 'series', 'title', 'year']
# Match any {...} block for template parsing
BRACE_PATTERN = re.compile(r'\{([^}]+)\}')
# Characters that are invalid in filenames on various filesystems
INVALID_CHARS = re.compile(r'[\\/:*?"<>|]')
@@ -88,37 +89,46 @@ def parse_naming_template(
# Normalize metadata keys to lowercase for case-insensitive matching
normalized = {k.lower(): v for k, v in metadata.items()}
def replace_token(match: re.Match) -> str:
prefix = match.group(1)
token_name = match.group(2).lower()
suffix = match.group(3)
def replace_block(match: re.Match) -> str:
content = match.group(1)
content_lower = content.lower()
# Get the value for this token
value = normalized.get(token_name)
# Find which known token appears in this block (longest first)
for token in KNOWN_TOKENS:
idx = content_lower.find(token)
if idx != -1:
prefix = content[:idx]
suffix = content[idx + len(token):]
# Special handling for series position
if token_name == 'seriesposition':
value = format_series_position(value)
# Get the value for this token
value = normalized.get(token)
# Convert to string
if value is None:
value = ""
else:
value = str(value).strip()
# Special handling for series position
if token == 'seriesposition':
value = format_series_position(value)
# If value is empty, return empty string (no prefix/suffix)
if not value:
return ""
# Convert to string
if value is None:
value = ""
else:
value = str(value).strip()
if not allow_path_separators:
value = value.replace("/", "_")
# Sanitize the value
value = sanitize_filename(value)
# If value is empty, return empty string (no prefix/suffix)
if not value:
return ""
return f"{prefix}{value}{suffix}"
if not allow_path_separators:
value = value.replace("/", "_")
# Sanitize the value
value = sanitize_filename(value)
return f"{prefix}{value}{suffix}"
# No known token found → return original block unchanged
return match.group(0)
# Replace all tokens
result = TOKEN_PATTERN.sub(replace_token, template)
result = BRACE_PATTERN.sub(replace_block, template)
# Clean up any double slashes that might result from empty tokens
result = re.sub(r'/+', '/', result)
+6
View File
@@ -34,6 +34,12 @@ def _get_config_dir() -> Path:
def is_onboarding_complete() -> bool:
"""Check if onboarding has been completed."""
from shelfmark.config.env import ONBOARDING
# If onboarding is disabled via env var, treat as complete
if not ONBOARDING:
return True
config_file = _get_config_dir() / "settings.json"
if not config_file.exists():
return False
+1 -1
View File
@@ -154,7 +154,7 @@ class BookQueue:
current_status = self._status.get(task_id)
# Allow cancellation during any active state
if current_status in [QueueStatus.RESOLVING, QueueStatus.DOWNLOADING]:
if current_status in [QueueStatus.RESOLVING, QueueStatus.LOCATING, QueueStatus.DOWNLOADING]:
# Signal active download to stop
if task_id in self._cancel_flags:
self._cancel_flags[task_id].set()
+4
View File
@@ -36,6 +36,7 @@ class ReleaseSearchPlan:
title_variants: List[ReleaseSearchVariant]
grouped_title_variants: List[ReleaseSearchVariant]
manual_query: Optional[str] = None
indexers: Optional[List[str]] = None # Indexer names for Prowlarr (overrides settings)
@property
def primary_query(self) -> str:
@@ -86,6 +87,7 @@ def build_release_search_plan(
book: BookMetadata,
languages: Optional[List[str]] = None,
manual_query: Optional[str] = None,
indexers: Optional[List[str]] = None,
) -> ReleaseSearchPlan:
resolved_languages = _normalize_languages(languages)
@@ -106,6 +108,7 @@ def build_release_search_plan(
title_variants=[variant],
grouped_title_variants=[variant],
manual_query=resolved_manual_query,
indexers=indexers,
)
isbn_candidates: List[str] = []
@@ -161,4 +164,5 @@ def build_release_search_plan(
title_variants=title_variants,
grouped_title_variants=grouped_variants,
manual_query=None,
indexers=indexers,
)
+115 -1
View File
@@ -83,6 +83,13 @@ class MultiSelectField(FieldBase):
variant: str = "pills" # "pills" (default) or "dropdown" for checkbox dropdown style
@dataclass
class TagListField(FieldBase):
"""Editable list of free-form string values (tag/chip input)."""
placeholder: str = ""
default: List[str] = field(default_factory=list)
@dataclass
class OrderableListField(FieldBase):
# Options can be a list or a callable that returns a list (for lazy evaluation)
@@ -145,7 +152,19 @@ class HeadingField:
# Type alias for all field types
SettingsField = Union[TextField, PasswordField, NumberField, CheckboxField, SelectField, MultiSelectField, OrderableListField, ActionButton, HeadingField]
SettingsField = Union[
TextField,
PasswordField,
NumberField,
CheckboxField,
SelectField,
MultiSelectField,
TagListField,
OrderableListField,
TableField,
ActionButton,
HeadingField,
]
@dataclass
@@ -398,6 +417,61 @@ def sync_env_to_config() -> None:
logger.debug(f"Synced {len(values_to_sync)} ENV values to {tab.name} config: {list(values_to_sync.keys())}")
migrate_legacy_settings()
migrate_mirror_settings()
def migrate_mirror_settings() -> None:
"""
Migrate legacy AA mirror config into the new editable mirror list setting.
Legacy:
- AA_ADDITIONAL_URLS: comma-separated extra URLs appended to defaults
New:
- AA_MIRROR_URLS: full ordered list of available mirrors (used for Auto mode and for Settings options)
"""
mirrors_config = load_config_file("mirrors")
from shelfmark.core.mirrors import DEFAULT_AA_MIRRORS
from shelfmark.core.utils import normalize_http_url
raw_list = mirrors_config.get("AA_MIRROR_URLS")
raw_additional = mirrors_config.get("AA_ADDITIONAL_URLS", "")
def _normalize_list(values: list[str]) -> list[str]:
out: list[str] = []
for item in values:
if str(item).strip().lower() == "auto":
continue
norm = normalize_http_url(str(item), default_scheme="https")
if norm and norm not in out:
out.append(norm)
return out
# If already a proper list, just ensure it's non-empty.
if isinstance(raw_list, list):
normalized = _normalize_list([str(v) for v in raw_list])
if normalized:
return
save_config_file("mirrors", {"AA_MIRROR_URLS": _normalize_list(DEFAULT_AA_MIRRORS)})
return
# If saved as a string, convert to list.
if isinstance(raw_list, str) and raw_list.strip():
parts = [p.strip() for p in raw_list.split(",") if p.strip()]
normalized = _normalize_list(parts)
if normalized:
save_config_file("mirrors", {"AA_MIRROR_URLS": normalized})
return
save_config_file("mirrors", {"AA_MIRROR_URLS": _normalize_list(DEFAULT_AA_MIRRORS)})
return
# If there's legacy additional mirrors, seed the full list so the UI reflects reality.
if isinstance(raw_additional, str) and raw_additional.strip():
additional_parts = [p.strip() for p in raw_additional.split(",") if p.strip()]
combined = _normalize_list(DEFAULT_AA_MIRRORS + additional_parts)
if combined:
save_config_file("mirrors", {"AA_MIRROR_URLS": combined})
def migrate_legacy_settings() -> None:
@@ -542,6 +616,8 @@ def _parse_env_value(value: str, field: SettingsField) -> Any:
return field.default
elif isinstance(field, MultiSelectField):
return [v.strip() for v in value.split(',') if v.strip()]
elif isinstance(field, TagListField):
return [v.strip() for v in value.split(',') if v.strip()]
elif isinstance(field, OrderableListField):
# Parse JSON array: [{"id": "...", "enabled": true}, ...]
try:
@@ -643,6 +719,8 @@ def serialize_field(field: SettingsField, tab_name: str, include_value: bool = T
options = field.options() if callable(field.options) else field.options
result["options"] = options
result["variant"] = field.variant
elif isinstance(field, TagListField):
result["placeholder"] = field.placeholder
elif isinstance(field, OrderableListField):
# Support callable options for lazy evaluation (avoids circular imports)
options = field.options() if callable(field.options) else field.options
@@ -674,6 +752,16 @@ def serialize_field(field: SettingsField, tab_name: str, include_value: bool = T
value = [v.strip() for v in value.split(",") if v.strip()]
else:
value = []
elif isinstance(field, TagListField):
if value is None:
value = []
elif isinstance(value, list):
value = [str(v) for v in value]
elif isinstance(value, str):
# Support legacy/manual configs where lists were saved as comma-separated strings.
value = [v.strip() for v in value.split(",") if v.strip()]
else:
value = []
elif isinstance(field, TableField):
if value is None:
value = []
@@ -801,6 +889,23 @@ def _apply_dns_settings(config) -> None:
except Exception as e:
logger.warning(f"Failed to apply DNS settings: {e}")
def _apply_aa_mirror_settings(config) -> None:
"""
Apply AA mirror settings changes to the network module.
This ensures AA_BASE_URL / AA_ADDITIONAL_URLS changes take effect immediately
without requiring a container restart.
"""
try:
from shelfmark.download import network
# Reload AA mirror list and configured base URL from refreshed config.
network.init_aa(force=True)
except ImportError:
pass # Network module not available
except Exception as e:
logger.warning(f"Failed to apply AA mirror settings: {e}")
def update_settings(tab_name: str, values: Dict[str, Any]) -> Dict[str, Any]:
tab = get_settings_tab(tab_name)
@@ -885,6 +990,15 @@ def update_settings(tab_name: str, values: Dict[str, Any]) -> Dict[str, Any]:
):
_apply_dns_settings(config_obj)
# Apply AA mirror settings changes live (mirrors tab)
aa_keys = {"AA_BASE_URL", "AA_MIRROR_URLS", "AA_ADDITIONAL_URLS"}
if (
config_obj is not None
and tab_name == "mirrors"
and aa_keys.intersection(values_to_save.keys())
):
_apply_aa_mirror_settings(config_obj)
# Sync metadata provider selection when a provider's enabled state changes
tab = get_settings_tab(tab_name)
if tab and tab.group == "metadata_providers":
+5
View File
@@ -191,6 +191,11 @@ def transform_cover_url(cover_url: Optional[str], cache_id: str) -> Optional[str
if not is_covers_cache_enabled():
return cover_url
from shelfmark.core.config import config as app_config
# Encode the original URL and create a proxy URL
encoded_url = base64.urlsafe_b64encode(cover_url.encode()).decode()
base_path = normalize_base_path(app_config.get("URL_BASE", ""))
if base_path:
return f"{base_path}/api/covers/{cache_id}?url={encoded_url}"
return f"/api/covers/{cache_id}?url={encoded_url}"
+40 -5
View File
@@ -10,12 +10,46 @@ import shutil
import subprocess
import time
from pathlib import Path
from typing import Any, Callable, Optional, TypeVar
from shelfmark.core.logger import setup_logger
from shelfmark.download.permissions_debug import log_transfer_permission_context
logger = setup_logger(__name__)
try:
from gevent import monkey as _gevent_monkey
from gevent.threadpool import ThreadPool as _GeventThreadPool
except Exception:
_gevent_monkey = None
_GeventThreadPool = None
T = TypeVar("T")
_IO_THREADPOOL: Optional["_GeventThreadPool"] = None
def _use_gevent_threadpool() -> bool:
return bool(
_gevent_monkey
and _GeventThreadPool
and _gevent_monkey.is_module_patched("threading")
)
def _get_io_threadpool() -> "_GeventThreadPool":
global _IO_THREADPOOL
if _IO_THREADPOOL is None:
pool_size = max(2, min(8, os.cpu_count() or 2))
_IO_THREADPOOL = _GeventThreadPool(pool_size)
return _IO_THREADPOOL
def run_blocking_io(func: Callable[..., T], *args: Any, **kwargs: Any) -> T:
"""Run blocking I/O in a native thread when under gevent."""
if _use_gevent_threadpool():
return _get_io_threadpool().apply(func, args, kwds=kwargs)
return func(*args, **kwargs)
_VERIFY_IO_WAIT_SECONDS = 3.0
@@ -96,11 +130,12 @@ def _is_permission_error(e: Exception) -> bool:
def _system_op(op: str, source: Path, dest: Path) -> None:
"""Execute system command (mv or cp) as final fallback."""
logger.warning("Attempting system %s as final fallback: %s -> %s", op, source, dest)
subprocess.run(
run_blocking_io(
subprocess.run,
[op, "-f", str(source), str(dest)],
check=True,
capture_output=True,
text=True
text=True,
)
@@ -110,7 +145,7 @@ def _perform_nfs_fallback(source: Path, dest: Path, is_move: bool) -> None:
try:
# Fallback 1: copy content only
shutil.copyfile(str(source), str(dest))
run_blocking_io(shutil.copyfile, str(source), str(dest))
_verify_transfer_size(dest, expected_size, "copy")
if is_move:
@@ -225,7 +260,7 @@ def atomic_move(source_path: Path, dest_path: Path, max_attempts: int = 100) ->
temp_path = try_path.parent / f".{try_path.name}.tmp"
try:
try:
shutil.copy2(str(source_path), str(temp_path))
run_blocking_io(shutil.copy2, str(source_path), str(temp_path))
except (PermissionError, OSError) as copy_error:
if _is_permission_error(copy_error):
logger.debug(
@@ -365,7 +400,7 @@ def atomic_copy(source_path: Path, dest_path: Path, max_attempts: int = 100) ->
temp_path = try_path.parent / f".{try_path.name}.tmp"
try:
try:
shutil.copy2(str(source_path), str(temp_path))
run_blocking_io(shutil.copy2, str(source_path), str(temp_path))
except (PermissionError, OSError) as e:
# Handle NFS permission errors immediately here
if _is_permission_error(e):
+94 -10
View File
@@ -3,9 +3,9 @@
import random
import time
from io import BytesIO
from threading import Event
from threading import Event, Thread
from typing import Callable, Optional
from urllib.parse import urlparse
from urllib.parse import urlparse, urljoin
import requests
from tqdm import tqdm
@@ -200,22 +200,106 @@ def html_get_page(
if use_bypasser_now and _is_cf_bypass_enabled():
logger.debug(f"GET (bypasser): {current_url}")
if status_callback:
status_callback("resolving", "Bypassing protection")
status_callback("resolving", "Bypassing protection...")
heartbeat_stop = Event()
heartbeat_thread: Optional[Thread] = None
if status_callback:
def _heartbeat() -> None:
# Keep the download "alive" during long bypass operations so the orchestrator
# doesn't flag it as stalled.
while not heartbeat_stop.wait(timeout=30):
if cancel_flag and cancel_flag.is_set():
return
try:
status_callback("resolving", "Bypassing protection...")
except Exception:
return
heartbeat_thread = Thread(target=_heartbeat, daemon=True, name="BypassHeartbeat")
heartbeat_thread.start()
try:
result = get_bypassed_page(current_url, selector, cancel_flag)
return result or ""
except Exception as e:
logger.warning(f"Bypasser error: {type(e).__name__}: {e}")
return ""
finally:
heartbeat_stop.set()
if heartbeat_thread:
heartbeat_thread.join(timeout=1)
logger.debug(f"GET: {current_url}")
# Try with CF cookies/UA if available (from previous bypass)
headers = {}
cookies = _apply_cf_bypass(current_url, headers)
response = requests.get(current_url, proxies=get_proxies(current_url), timeout=REQUEST_TIMEOUT, cookies=cookies, headers=headers)
response.raise_for_status()
time.sleep(1)
return response.text
# Use a browser-like UA by default (AA can behave differently for python-requests UA).
headers = {"User-Agent": DOWNLOAD_HEADERS["User-Agent"]}
# AA mirrors sometimes redirect to other (seized/dead) mirror domains. If we let
# requests follow those redirects, the request fails on DNS and we rotate away
# from an otherwise working mirror. Handle AA redirects manually instead.
is_aa_url = network.should_rotate_dns_for_url(current_url)
allow_redirects = not is_aa_url
redirects_followed = 0
while True:
# Try with CF cookies/UA if available (from previous bypass)
cookies = _apply_cf_bypass(current_url, headers)
response = requests.get(
current_url,
proxies=get_proxies(current_url),
timeout=REQUEST_TIMEOUT,
cookies=cookies,
headers=headers,
allow_redirects=allow_redirects,
)
if is_aa_url and response.is_redirect:
location = response.headers.get("Location", "")
if not location:
raise requests.exceptions.TooManyRedirects(f"Redirect with no Location header: {current_url}")
redirect_url = urljoin(current_url, location)
current_host = urlparse(current_url).hostname or ""
redirect_host = urlparse(redirect_url).hostname or ""
# If an AA mirror redirects to a different hostname, treat that as a mirror
# failure and rotate rather than following the redirect (auto mode only).
if current_host and redirect_host and current_host != redirect_host:
if not network.is_aa_auto_mode():
logger.warning(
"AA mirror locked to %s but redirected to %s: %s",
current_host,
redirect_host,
current_url,
)
return ""
new_url = _try_rotation(original_url, current_url, selector)
if new_url:
current_url = new_url
# Reset per-request state for the new host.
headers = {"User-Agent": DOWNLOAD_HEADERS["User-Agent"]}
is_aa_url = network.should_rotate_dns_for_url(current_url)
allow_redirects = not is_aa_url
redirects_followed = 0
continue
logger.warning(
"AA redirect from %s to %s but mirrors exhausted: %s",
current_host,
redirect_host,
current_url,
)
return ""
# Same-host redirect (relative or absolute) - follow manually.
redirects_followed += 1
if redirects_followed > 5:
raise requests.exceptions.TooManyRedirects(f"Too many redirects for {current_url}")
current_url = redirect_url
continue
response.raise_for_status()
time.sleep(1)
return response.text
except Exception as e:
status = _get_status_code(e)
+35 -3
View File
@@ -738,11 +738,22 @@ def rotate_dns_and_reset_aa() -> bool:
if not configured_url:
configured_url = "auto"
if configured_url == "auto" or _aa_base_url in _aa_urls:
if configured_url == "auto":
# Auto mode always resets to the first mirror to restart the cascade
_current_aa_url_index = 0
_aa_base_url = _aa_urls[0] if _aa_urls else "https://annas-archive.se"
_aa_base_url = _aa_urls[0] if _aa_urls else "https://annas-archive.gl"
logger.info(f"After DNS switch, resetting AA URL to: {_aa_base_url}")
_save_state(aa_url=_aa_base_url)
else:
# Keep the user's configured primary mirror (if it exists in the list),
# otherwise keep the configured URL as-is (custom/env).
if configured_url in _aa_urls:
_current_aa_url_index = _aa_urls.index(configured_url)
else:
_current_aa_url_index = 0
_aa_base_url = configured_url
logger.info(f"After DNS switch, keeping configured AA URL: {_aa_base_url}")
_save_state(aa_url=_aa_base_url)
return True
def set_dns_provider(provider: str, manual_servers: list[str] | None = None, use_doh: bool | None = None) -> bool:
@@ -916,6 +927,11 @@ def _initialize_aa_state() -> None:
if not configured_url:
configured_url = "auto"
# If AA_BASE_URL is pinned to a custom URL that's not in the mirror list, we still
# want to treat it as the active base (and rewrite known mirror links to it).
if configured_url != "auto" and configured_url not in _aa_urls:
_aa_urls = [configured_url] + _aa_urls
if configured_url == "auto":
if state.get('aa_base_url') and state['aa_base_url'] in _aa_urls:
_current_aa_url_index = _aa_urls.index(state['aa_base_url'])
@@ -1031,6 +1047,17 @@ def get_aa_base_url():
_ensure_initialized()
return _aa_base_url
def is_aa_auto_mode() -> bool:
"""Return True when AA_BASE_URL is set to 'auto' (mirror failover enabled)."""
configured_url = normalize_http_url(
app_config.get("AA_BASE_URL", "auto"),
default_scheme="https",
allow_special=("auto",),
)
if not configured_url:
configured_url = "auto"
return configured_url == "auto"
def get_available_aa_urls():
"""Get list of configured AA URLs (copy)."""
_ensure_initialized()
@@ -1082,12 +1109,17 @@ class AAMirrorSelector:
Returns (new_base, action) where action is 'mirror', 'dns', or 'exhausted'.
"""
self.attempts_this_dns += 1
if self.attempts_this_dns >= len(self.aa_urls):
max_attempts = len(self.aa_urls) if is_aa_auto_mode() else 1
if self.attempts_this_dns >= max_attempts:
if allow_dns and rotate_dns_and_reset_aa():
self._ensure_fresh_state(reset_attempts=True)
return self.current_base, "dns"
return None, "exhausted"
if not is_aa_auto_mode():
# Mirror is explicitly configured; do not fail over to other mirrors.
return None, "exhausted"
next_index = (self._index + 1) % len(self.aa_urls)
set_aa_url_index(next_index)
self._ensure_fresh_state(reset_attempts=False)
+29 -12
View File
@@ -50,6 +50,8 @@ _progress_lock = Lock()
# Stall detection - track last activity time per download
_last_activity: Dict[str, float] = {}
# De-duplicate status updates (keep-alive updates shouldn't spam clients)
_last_status_event: Dict[str, Tuple[str, Optional[str]]] = {}
STALL_TIMEOUT = 300 # 5 minutes without progress/status update = stalled
def search_books(query: str, filters: SearchFilters) -> List[Dict[str, Any]]:
@@ -390,6 +392,7 @@ def update_download_status(book_id: str, status: str, message: Optional[str] = N
status_map = {
'queued': QueueStatus.QUEUED,
'resolving': QueueStatus.RESOLVING,
'locating': QueueStatus.LOCATING,
'downloading': QueueStatus.DOWNLOADING,
'complete': QueueStatus.COMPLETE,
'available': QueueStatus.AVAILABLE,
@@ -398,21 +401,29 @@ def update_download_status(book_id: str, status: str, message: Optional[str] = N
'cancelled': QueueStatus.CANCELLED,
}
queue_status_enum = status_map.get(status.lower())
if queue_status_enum:
book_queue.update_status(book_id, queue_status_enum)
status_key = status.lower()
queue_status_enum = status_map.get(status_key)
if not queue_status_enum:
return
# Track activity for stall detection
with _progress_lock:
_last_activity[book_id] = time.time()
# Always update activity timestamp (used by stall detection) even if the status
# event is a duplicate keep-alive update.
with _progress_lock:
_last_activity[book_id] = time.time()
status_event = (status_key, message)
if _last_status_event.get(book_id) == status_event:
return
_last_status_event[book_id] = status_event
# Update status message if provided (empty string clears the message)
if message is not None:
book_queue.update_status_message(book_id, message)
book_queue.update_status(book_id, queue_status_enum)
# Broadcast status update via WebSocket
if ws_manager:
ws_manager.broadcast_status_update(queue_status())
# Update status message if provided (empty string clears the message)
if message is not None:
book_queue.update_status_message(book_id, message)
# Broadcast status update via WebSocket
if ws_manager:
ws_manager.broadcast_status_update(queue_status())
def cancel_download(book_id: str) -> bool:
"""Cancel a download."""
@@ -450,6 +461,7 @@ def _cleanup_progress_tracking(task_id: str) -> None:
_progress_last_broadcast.pop(task_id, None)
_progress_last_broadcast.pop(f"{task_id}_progress", None)
_last_activity.pop(task_id, None)
_last_status_event.pop(task_id, None)
def _process_single_download(task_id: str, cancel_flag: Event) -> None:
@@ -508,12 +520,14 @@ def concurrent_download_loop() -> None:
with ThreadPoolExecutor(max_workers=max_workers, thread_name_prefix="Download") as executor:
active_futures: Dict[Future, str] = {} # Track active download futures
stalled_tasks: set[str] = set() # Track tasks already cancelled due to stall
while True:
# Clean up completed futures
completed_futures = [f for f in active_futures if f.done()]
for future in completed_futures:
task_id = active_futures.pop(future)
stalled_tasks.discard(task_id)
try:
future.result() # This will raise any exceptions from the worker
except Exception as e:
@@ -523,11 +537,14 @@ def concurrent_download_loop() -> None:
current_time = time.time()
with _progress_lock:
for future, task_id in list(active_futures.items()):
if task_id in stalled_tasks:
continue
last_active = _last_activity.get(task_id, current_time)
if current_time - last_active > STALL_TIMEOUT:
logger.warning(f"Download stalled for {task_id}, cancelling")
book_queue.cancel_download(task_id)
book_queue.update_status_message(task_id, f"Download stalled (no activity for {STALL_TIMEOUT}s)")
stalled_tasks.add(task_id)
# Start new downloads if we have capacity
while len(active_futures) < max_workers:
+16 -4
View File
@@ -30,7 +30,12 @@ def _resolve_custom_script_target(target_path: Path, destination: Path, path_mod
except ValueError:
if target_path.is_absolute():
return Path(target_path.name)
return target_path
return target_path
def _format_op_counts(op_counts: dict[str, int]) -> str:
parts = [f"{op}={count}" for op, count in op_counts.items() if count]
return ", ".join(parts) if parts else "none"
@dataclass(frozen=True)
@@ -255,7 +260,7 @@ def process_folder_output(
record_step(steps, "cleanup_staging", path=str(prepared.working_path))
log_plan_steps(task.task_id, steps)
final_paths, error = transfer_book_files(
final_paths, error, op_counts = transfer_book_files(
prepared.files,
destination=plan.destination,
task=task,
@@ -271,12 +276,19 @@ def process_folder_output(
return None
logger.info(
"Task %s: transferred %d file(s) to %s (%s)",
"Task %s: transferred %d file(s) to %s (ops: %s)",
task.task_id,
len(final_paths),
plan.destination,
op_label.lower(),
_format_op_counts(op_counts),
)
if use_hardlink and op_counts.get("copy", 0):
logger.warning(
"Task %s: hardlink requested but %d of %d file(s) copied (fallback)",
task.task_id,
op_counts.get("copy", 0),
len(final_paths),
)
# Run custom script once per successful task, after transfer.
if core_config.config.CUSTOM_SCRIPT:
+34 -13
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
import os
from pathlib import Path
from typing import List, Optional, Tuple
from typing import Dict, List, Optional, Tuple
import shelfmark.core.config as core_config
from shelfmark.core.logger import setup_logger
@@ -142,15 +142,16 @@ def transfer_book_files(
is_torrent: bool,
preserve_source: bool = False,
organization_mode: Optional[str] = None,
) -> Tuple[List[Path], Optional[str]]:
) -> Tuple[List[Path], Optional[str], Dict[str, int]]:
if not book_files:
return [], "No book files found"
return [], "No book files found", {"hardlink": 0, "copy": 0, "move": 0}
is_audiobook = check_audiobook(task.content_type)
organization_mode = organization_mode or get_file_organization(is_audiobook)
max_attempts = _max_attempts_for_batch(len(book_files))
final_paths: List[Path] = []
op_counts: Dict[str, int] = {"hardlink": 0, "copy": 0, "move": 0}
if organization_mode == "organize":
template = get_template(is_audiobook, "organize")
@@ -171,6 +172,7 @@ def transfer_book_files(
max_attempts=max_attempts,
)
final_paths.append(final_path)
op_counts[op] = op_counts.get(op, 0) + 1
logger.debug(f"{op.capitalize()} to destination: {final_path.name}")
else:
zero_pad_width = max(len(str(len(book_files))), 2)
@@ -191,9 +193,10 @@ def transfer_book_files(
max_attempts=max_attempts,
)
final_paths.append(final_path)
op_counts[op] = op_counts.get(op, 0) + 1
logger.debug(f"{op.capitalize()} to destination: {final_path.name}")
return final_paths, None
return final_paths, None, op_counts
for book_file in book_files:
if len(book_files) == 1 and organization_mode != "none":
@@ -223,9 +226,10 @@ def transfer_book_files(
max_attempts=max_attempts,
)
final_paths.append(final_path)
op_counts[op] = op_counts.get(op, 0) + 1
logger.debug(f"{op.capitalize()} to destination: {final_path.name}")
return final_paths, None
return final_paths, None, op_counts
def process_directory(
@@ -257,7 +261,7 @@ def process_directory(
if use_hardlink is None:
use_hardlink = should_hardlink(task)
final_paths, error = transfer_book_files(
final_paths, error, _op_counts = transfer_book_files(
book_files,
destination=ingest_dir,
task=task,
@@ -305,6 +309,12 @@ def transfer_file_to_library(
max_attempts=_max_attempts_for_batch(1),
)
logger.info(f"Library {op}: {final_path}")
if use_hardlink and op != "hardlink":
logger.warning(
"Library hardlink requested but %s used instead for %s",
op,
final_path,
)
if use_hardlink and temp_file and not is_torrent_source(temp_file, task):
safe_cleanup_path(temp_file, task)
@@ -344,6 +354,7 @@ def transfer_directory_to_library(
is_torrent = is_torrent_source(source_dir, task)
transferred_paths: List[Path] = []
op_counts: Dict[str, int] = {"hardlink": 0, "copy": 0, "move": 0}
max_attempts = _max_attempts_for_batch(len(source_files))
if len(source_files) == 1:
@@ -359,6 +370,7 @@ def transfer_directory_to_library(
)
logger.debug(f"Library {op}: {source_file.name} -> {final_path}")
transferred_paths.append(final_path)
op_counts[op] = op_counts.get(op, 0) + 1
else:
zero_pad_width = max(len(str(len(source_files))), 2)
files_with_parts = assign_part_numbers(source_files, zero_pad_width)
@@ -378,14 +390,23 @@ def transfer_directory_to_library(
)
logger.debug(f"Library {op}: {source_file.name} -> {final_path}")
transferred_paths.append(final_path)
op_counts[op] = op_counts.get(op, 0) + 1
if use_hardlink:
operation = "hardlinks"
elif is_torrent:
operation = "copies"
else:
operation = "files"
logger.info(f"Created {len(transferred_paths)} library {operation} in {base_library_path.parent}")
op_summary = ", ".join(
f"{op}={count}" for op, count in op_counts.items() if count
) or "none"
logger.info(
"Created %d library file(s) in %s (ops: %s)",
len(transferred_paths),
base_library_path.parent,
op_summary,
)
if use_hardlink and op_counts.get("copy", 0):
logger.warning(
"Library hardlink requested but %d of %d file(s) copied (fallback)",
op_counts.get("copy", 0),
len(transferred_paths),
)
if use_hardlink and temp_file and not is_torrent_source(temp_file, task):
safe_cleanup_path(temp_file, task)
+2 -1
View File
@@ -7,6 +7,7 @@ from typing import List, Optional
from shelfmark.config import env as env_config
from shelfmark.core.logger import setup_logger
from shelfmark.core.models import DownloadTask
from shelfmark.download.fs import run_blocking_io
from shelfmark.download.staging import STAGE_NONE
from .types import OutputPlan
@@ -59,7 +60,7 @@ def safe_cleanup_path(path: Optional[Path], task: DownloadTask) -> None:
try:
if path.is_dir():
shutil.rmtree(path, ignore_errors=True)
run_blocking_io(shutil.rmtree, path, ignore_errors=True)
elif path.exists():
path.unlink(missing_ok=True)
except (OSError, PermissionError) as exc:
+5 -4
View File
@@ -7,6 +7,7 @@ from typing import Literal
from shelfmark.config import env as env_config
from shelfmark.core.logger import setup_logger
from shelfmark.download.fs import run_blocking_io
logger = setup_logger(__name__)
@@ -67,17 +68,17 @@ def stage_path(source: Path, staging_dir: Path, action: StageAction) -> Path:
staged_path = staging_dir / f"{source.name}_{counter}"
counter += 1
if action == STAGE_COPY:
shutil.copytree(str(source), str(staged_path))
run_blocking_io(shutil.copytree, str(source), str(staged_path))
else:
shutil.move(str(source), str(staged_path))
run_blocking_io(shutil.move, str(source), str(staged_path))
else:
while staged_path.exists():
staged_path = staging_dir / f"{source.stem}_{counter}{source.suffix}"
counter += 1
if action == STAGE_COPY:
shutil.copy2(str(source), str(staged_path))
run_blocking_io(shutil.copy2, str(source), str(staged_path))
else:
shutil.move(str(source), str(staged_path))
run_blocking_io(shutil.move, str(source), str(staged_path))
staged_kind = "directory" if source.is_dir() else "file"
logger.debug("Staged %s via %s: %s -> %s", staged_kind, action, source, staged_path)
+26 -4
View File
@@ -51,6 +51,7 @@ if BASE_PATH:
async_mode = 'gevent'
# Initialize Flask-SocketIO with reverse proxy support
socketio_path = f"{BASE_PATH}/socket.io" if BASE_PATH else "/socket.io"
socketio = SocketIO(
app,
cors_allowed_origins="*",
@@ -58,7 +59,7 @@ socketio = SocketIO(
logger=False,
engineio_logger=False,
# Reverse proxy / Traefik compatibility settings
path='/socket.io',
path=socketio_path,
ping_timeout=60, # Time to wait for pong response
ping_interval=25, # Send ping every 25 seconds
# Allow both websocket and polling for better compatibility
@@ -268,12 +269,29 @@ def proxy_auth_middleware():
from shelfmark.core.settings_registry import load_config_file
def get_proxy_header(header_name: str) -> str | None:
"""Resolve proxy auth values from headers with WSGI env fallbacks."""
value = request.headers.get(header_name)
if value:
return value
env_key = f"HTTP_{header_name.upper().replace('-', '_')}"
value = request.environ.get(env_key)
if value:
return value
# Some proxies set authenticated username in REMOTE_USER (not as a header).
if header_name.lower().replace("_", "-") == "remote-user":
return request.environ.get("REMOTE_USER")
return None
try:
security_config = load_config_file("security")
user_header = security_config.get("PROXY_AUTH_USER_HEADER", "X-Auth-User")
# Extract username from proxy header
username = request.headers.get(user_header)
username = get_proxy_header(user_header)
if not username:
if request.path.startswith('/api/auth/'):
@@ -291,7 +309,7 @@ def proxy_auth_middleware():
admin_group_name = security_config.get("PROXY_AUTH_ADMIN_GROUP_NAME", "admins")
# Extract groups from proxy header (can be comma or pipe separated)
groups_header = request.headers.get(admin_group_header, "")
groups_header = get_proxy_header(admin_group_header) or ""
user_groups_delimiter = "," if "," in groups_header else "|"
user_groups = [g.strip() for g in groups_header.split(user_groups_delimiter) if g.strip()]
@@ -1423,6 +1441,10 @@ def api_releases() -> Union[Response, Tuple[Response, int]]:
manual_query = request.args.get('manual_query', '').strip()
# Accept indexer names for Prowlarr filtering (comma-separated)
indexers_param = request.args.get('indexers', '').strip()
indexers = [idx.strip() for idx in indexers_param.split(',') if idx.strip()] if indexers_param else None
if not provider or not book_id:
return jsonify({"error": "Parameters 'provider' and 'book_id' are required"}), 400
@@ -1463,7 +1485,7 @@ def api_releases() -> Union[Response, Tuple[Response, int]]:
from shelfmark.core.search_plan import build_release_search_plan
plan = build_release_search_plan(book, languages=languages, manual_query=manual_query)
plan = build_release_search_plan(book, languages=languages, manual_query=manual_query, indexers=indexers)
if plan.manual_query:
planned_query = plan.manual_query
+15 -1
View File
@@ -58,9 +58,13 @@ class ColumnRenderType(str, Enum):
"""How the frontend should render the column value."""
TEXT = "text" # Plain text
BADGE = "badge" # Colored badge (format, language)
TAGS = "tags" # List of colored badges
SIZE = "size" # File size formatting
NUMBER = "number" # Numeric value
PEERS = "peers" # Peers display: "S/L" with color based on seeder count
INDEXER_PROTOCOL = "indexer_protocol" # Text + colored dot for torrent/usenet
FLAG_ICON = "flag_icon" # Icon with tooltip (VIP, freeleech, etc.)
FORMAT_CONTENT_TYPE = "format_content_type" # Content type icon + format badge
class ColumnAlign(str, Enum):
@@ -123,8 +127,10 @@ class ReleaseColumnConfig:
grid_template: str = "minmax(0,2fr) 60px 80px 80px" # CSS grid-template-columns
leading_cell: Optional[LeadingCellConfig] = None # Defaults to thumbnail mode if None
online_servers: Optional[List[str]] = None # For IRC: list of currently online server nicks
available_indexers: Optional[List[str]] = None # For Prowlarr: list of all enabled indexer names
default_indexers: Optional[List[str]] = None # For Prowlarr: indexers selected in settings (pre-selected in filter)
cache_ttl_seconds: Optional[int] = None # How long to cache results (default: 5 min)
supported_filters: Optional[List[str]] = None # Which filters this source supports: ["format", "language"]
supported_filters: Optional[List[str]] = None # Which filters this source supports: ["format", "language", "indexer"]
action_button: Optional[SourceActionButton] = None # Custom action button (replaces default expand search)
@@ -169,6 +175,14 @@ def serialize_column_config(config: ReleaseColumnConfig) -> Dict[str, Any]:
if config.online_servers is not None:
result["online_servers"] = config.online_servers
# Include available_indexers if provided (e.g., for Prowlarr source)
if config.available_indexers is not None:
result["available_indexers"] = config.available_indexers
# Include default_indexers if provided (indexers selected in settings, for pre-selection)
if config.default_indexers is not None:
result["default_indexers"] = config.default_indexers
# Include cache TTL if specified (sources can request longer caching)
if config.cache_ttl_seconds is not None:
result["cache_ttl_seconds"] = config.cache_ttl_seconds
@@ -1056,6 +1056,7 @@ def _book_info_to_release(book_info: BookInfo) -> Release:
source_id=book_info.id,
title=book_info.title,
format=book_info.format,
language=book_info.language, # Top-level language for filtering
size=book_info.size,
download_url=book_info.download_urls[0] if book_info.download_urls else None,
info_url=f"{network.get_aa_base_url()}/md5/{book_info.id}",
+100
View File
@@ -6,6 +6,7 @@ import requests
from shelfmark.core.logger import setup_logger
from shelfmark.core.utils import normalize_http_url
from shelfmark.release_sources.prowlarr.torznab import parse_torznab_xml
logger = setup_logger(__name__)
@@ -90,6 +91,44 @@ class ProwlarrClient:
logger.error(f"Failed to get indexers: {e}")
return []
def get_enabled_indexers_detailed(self) -> List[Dict[str, Any]]:
"""
Get enabled indexers, including implementation metadata.
Note: Prowlarr indexer "name" is user-configurable; prefer
"implementation"/"implementationName" for stable identification.
"""
indexers = self.get_indexers()
return [idx for idx in indexers if idx.get("enable", False)]
def get_enriched_indexer_ids(self, *, restrict_to: Optional[List[int]] = None) -> List[int]:
"""
Return enabled indexer IDs that should use Torznab for richer metadata.
Args:
restrict_to: Optional list of candidate indexer IDs to consider.
"""
enriched_ids: List[int] = []
for idx in self.get_enabled_indexers_detailed():
idx_id = idx.get("id")
if idx_id is None:
continue
try:
idx_id_int = int(idx_id)
except (TypeError, ValueError):
continue
if restrict_to is not None and idx_id_int not in restrict_to:
continue
impl = str(idx.get("implementation") or idx.get("implementationName") or idx.get("definitionName") or "")
# Currently only MyAnonamouse provides consistently rich Torznab metadata.
if impl.strip().lower() == "myanonamouse":
enriched_ids.append(idx_id_int)
return enriched_ids
def get_enabled_indexers(self) -> List[Dict[str, Any]]:
"""Get enabled indexers with book capability info."""
indexers = self.get_indexers()
@@ -112,6 +151,67 @@ class ProwlarrClient:
return result
def torznab_search(
self,
*,
indexer_id: int,
query: str,
categories: Optional[List[int]] = None,
search_type: str = "book",
limit: int = 100,
offset: int = 0,
) -> List[Dict[str, Any]]:
"""
Search a specific indexer via Prowlarr's Torznab/Newznab endpoint.
This returns richer fields (e.g., author/booktitle, torznab tags like
FreeLeech) than the JSON /api/v1/search endpoint.
"""
if not query:
return []
endpoint = f"/api/v1/indexer/{int(indexer_id)}/newznab"
url = self.base_url + endpoint
params: Dict[str, Any] = {
"t": search_type,
"q": query,
"limit": limit,
"offset": offset,
}
if categories:
params["cat"] = ",".join(str(c) for c in categories)
logger.debug(f"Prowlarr API: GET {url} (torznab)")
try:
response = self._session.get(
url=url,
params=params,
timeout=self.timeout,
headers={
# Override the session default JSON accept header.
"Accept": "application/rss+xml, application/xml;q=0.9, */*;q=0.8"
},
)
if not response.ok:
try:
error_body = response.text[:500]
logger.error(f"Prowlarr Torznab error response: {error_body}")
except Exception:
pass
response.raise_for_status()
results = parse_torznab_xml(response.text)
# Ensure indexerId is always set (Prowlarr includes it, but be defensive).
for r in results:
if r.get("indexerId") is None:
r["indexerId"] = int(indexer_id)
return results
except Exception as e:
logger.error(f"Prowlarr Torznab search failed for indexer {indexer_id}: {e}")
return []
def _has_book_categories(self, categories: List[Dict[str, Any]]) -> bool:
"""Check if any category or subcategory is in the book range (7000-7999)."""
for cat in categories:
@@ -310,7 +310,17 @@ class RTorrentClient(DownloadClient):
this corresponds to `d.get_base_path()`.
"""
try:
base_path = self._rpc.d.get_base_path(download_id)
return base_path if base_path else None
# rTorrent is case sensitive for hashes; use uppercase as in get_status()
download_hash = download_id.upper()
details = self._rpc.d.multicall.filtered(
"",
"default",
f"equal={{d.hash=,cat={download_hash}}}",
"d.base_path=",
)
if not details:
return None
path = details[0][0]
return path if path else None
except Exception:
return None
+235 -184
View File
@@ -1,6 +1,7 @@
"""Prowlarr download handler - executes downloads via torrent/usenet clients."""
import shutil
import time
from pathlib import Path
from threading import Event
from typing import Callable, Optional
@@ -23,6 +24,9 @@ logger = setup_logger(__name__)
# How often to poll the download client for status (seconds)
POLL_INTERVAL = 2
# How long to wait for completed files to appear (seconds)
COMPLETED_PATH_RETRY_INTERVAL = 5
COMPLETED_PATH_MAX_ATTEMPTS = 12 # 12 attempts * 5s = 60s grace period
def _diagnose_path_issue(path: str) -> str:
@@ -195,6 +199,212 @@ class ProwlarrHandler(DownloadHandler):
f"Failed to remove download {download_id} from {client.name} after {reason}: {e}"
)
def _handle_cancelled_download(
self,
client: DownloadClient,
download_id: str,
protocol: str,
status_callback: Callable[[str, Optional[str]], None],
) -> None:
if protocol == "usenet":
logger.info(f"Download cancelled, removing from {client.name}: {download_id}")
try:
self._delete_local_download_data(client, download_id)
self._remove_usenet_download(client, download_id, delete_files=True, archive=True)
except Exception as e:
logger.warning(
f"Failed to remove download {download_id} from {client.name} after cancellation: {e}"
)
else:
logger.info(
f"Download cancelled for protocol={protocol}; leaving in {client.name}: {download_id}"
)
status_callback("cancelled", "Cancelled")
def _resolve_download_path_once(
self,
client: DownloadClient,
download_id: str,
*,
log_details: bool,
) -> tuple[Optional[Path], Optional[str]]:
"""Resolve and validate the completed download path once."""
try:
raw_path = client.get_download_path(download_id)
except Exception as e:
message = (
f"Could not locate completed download in {client.name} (path not returned). "
f"Check volume mappings and category settings."
)
if log_details:
logger.error(
f"Failed to resolve download path for {client.name} {download_id}: {e}"
)
else:
logger.debug(
f"Failed to resolve download path for {client.name} {download_id}: {e}"
)
return None, message
if not raw_path:
message = (
f"Could not locate completed download in {client.name} (path not returned). "
f"Check volume mappings and category settings."
)
if log_details:
logger.error(f"Download client returned empty path for {client.name} {download_id}")
else:
logger.debug(f"Download client returned empty path for {client.name} {download_id}")
return None, message
from shelfmark.core.path_mappings import (
get_client_host_identifier,
parse_remote_path_mappings,
remap_remote_to_local_with_match,
)
source_path_obj = Path(raw_path)
host = get_client_host_identifier(client) or ""
mapping_value = config.get("PROWLARR_REMOTE_PATH_MAPPINGS", [])
mappings = parse_remote_path_mappings(mapping_value)
if log_details:
logger.debug(
"Attempting path remap: client=%s, host=%s, path=%s, mappings=%s",
client.name,
host,
source_path_obj,
[(m.host, m.remote_path, m.local_path) for m in mappings],
)
remapped, matched_mapping = remap_remote_to_local_with_match(
mappings=mappings,
host=host,
remote_path=source_path_obj,
)
if log_details:
logger.debug(
"Remap result: %s -> %s (exists=%s, changed=%s, matched=%s)",
source_path_obj,
remapped,
remapped.exists(),
remapped != source_path_obj,
matched_mapping,
)
if matched_mapping:
if remapped.exists():
logger.info(
"Remapped download path for %s (%s): %s -> %s",
client.name,
download_id,
source_path_obj,
remapped,
)
return remapped, None
message = (
f"Remapped path '{remapped}' does not exist. "
f"Check your Docker volume mounts match the Local Path in Settings > Advanced > Remote Path Mappings."
)
if log_details:
logger.error(
f"Download path does not exist after remapping: {raw_path} -> {remapped}. "
f"Client: {client.name}, ID: {download_id}."
)
else:
logger.debug(
f"Download path does not exist after remapping: {raw_path} -> {remapped}. "
f"Client: {client.name}, ID: {download_id}."
)
return None, message
if mappings:
if source_path_obj.exists():
logger.info(
"No remote path mapping matched for %s (%s); using client path: %s",
client.name,
download_id,
source_path_obj,
)
return source_path_obj, None
hint = _diagnose_path_issue(raw_path)
message = f"{hint} No remote path mapping matched for client '{client.name}'."
if log_details:
logger.error(
f"Download path does not exist and no remote path mapping matched for {client.name} "
f"({download_id}): {raw_path}. {hint}"
)
else:
logger.debug(
f"Download path does not exist and no remote path mapping matched for {client.name} "
f"({download_id}): {raw_path}. {hint}"
)
return None, message
if not source_path_obj.exists():
hint = _diagnose_path_issue(raw_path)
message = hint
if log_details:
logger.error(
f"Download path does not exist: {raw_path}. "
f"Client: {client.name}, ID: {download_id}. {hint}"
)
else:
logger.debug(
f"Download path does not exist: {raw_path}. "
f"Client: {client.name}, ID: {download_id}. {hint}"
)
return None, message
return source_path_obj, None
def _wait_for_completed_path(
self,
client: DownloadClient,
download_id: str,
*,
cancel_flag: Optional[Event],
status_callback: Callable[[str, Optional[str]], None],
) -> tuple[Optional[Path], Optional[str]]:
"""Wait briefly for completed files to appear on disk."""
last_error: Optional[str] = None
for attempt in range(1, COMPLETED_PATH_MAX_ATTEMPTS + 1):
if cancel_flag and cancel_flag.is_set():
return None, last_error
log_details = attempt == COMPLETED_PATH_MAX_ATTEMPTS
resolved_path, error = self._resolve_download_path_once(
client,
download_id,
log_details=log_details,
)
if resolved_path:
return resolved_path, None
last_error = error
if attempt < COMPLETED_PATH_MAX_ATTEMPTS:
status_callback("locating", "Waiting for completed files...")
logger.debug(
"Completed files not available yet for %s (%s) (attempt %d/%d)",
client.name,
download_id,
attempt,
COMPLETED_PATH_MAX_ATTEMPTS,
)
if cancel_flag:
if cancel_flag.wait(timeout=COMPLETED_PATH_RETRY_INTERVAL):
return None, last_error
else:
time.sleep(COMPLETED_PATH_RETRY_INTERVAL)
return None, last_error
def _build_progress_message(self, status) -> str:
"""Build a progress message from download status."""
msg = f"{status.progress:.0f}%"
@@ -265,86 +475,22 @@ class ProwlarrHandler(DownloadHandler):
logger.info("Existing download is complete, copying file directly")
status_callback("resolving", "Found existing download, copying to library")
source_path = client.get_download_path(download_id)
if not source_path:
logger.error(
f"Could not get path for existing download. "
f"Client: {client.name}, ID: {download_id}. "
f"The download may have been moved or deleted."
)
source_path_obj, path_error = self._wait_for_completed_path(
client=client,
download_id=download_id,
cancel_flag=cancel_flag,
status_callback=status_callback,
)
if not source_path_obj:
if cancel_flag.is_set():
return None
status_callback(
"error",
f"Could not locate existing download in {client.name}. "
f"Check that the file still exists."
path_error
or f"Could not locate existing download in {client.name}. Check that the file still exists.",
)
return None
from shelfmark.core.path_mappings import (
get_client_host_identifier,
parse_remote_path_mappings,
remap_remote_to_local_with_match,
)
source_path_obj = Path(source_path)
host = get_client_host_identifier(client) or ""
mapping_value = config.get("PROWLARR_REMOTE_PATH_MAPPINGS", [])
mappings = parse_remote_path_mappings(mapping_value)
remapped, matched_mapping = remap_remote_to_local_with_match(
mappings=mappings,
host=host,
remote_path=source_path_obj,
)
if matched_mapping:
if remapped.exists():
logger.info(
"Remapped existing download path for %s (%s): %s -> %s",
client.name,
download_id,
source_path_obj,
remapped,
)
source_path_obj = remapped
else:
logger.error(
f"Download path does not exist after remapping: {source_path} -> {remapped}. "
f"Client: {client.name}, ID: {download_id}. "
f"Check that the local path in your mapping is mounted correctly."
)
status_callback(
"error",
f"Remapped path '{remapped}' does not exist. "
f"Check your Docker volume mounts match the Local Path in Settings > Advanced > Remote Path Mappings.",
)
return None
elif mappings:
if source_path_obj.exists():
logger.info(
"No remote path mapping matched for %s (%s); using client path: %s",
client.name,
download_id,
source_path_obj,
)
else:
hint = _diagnose_path_issue(source_path)
logger.error(
f"Download path does not exist and no remote path mapping matched for {client.name} "
f"({download_id}): {source_path}. {hint}"
)
status_callback(
"error",
f"{hint} No remote path mapping matched for client '{client.name}'.",
)
return None
elif not source_path_obj.exists():
hint = _diagnose_path_issue(source_path)
logger.error(
f"Download path does not exist: {source_path}. "
f"Client: {client.name}, ID: {download_id}. {hint}"
)
status_callback("error", hint)
return None
result = self._handle_completed_file(
source_path=source_path_obj,
protocol=protocol,
@@ -495,122 +641,27 @@ class ProwlarrHandler(DownloadHandler):
# Handle cancellation
if cancel_flag.is_set():
if protocol == "usenet":
logger.info(f"Download cancelled, removing from {client.name}: {download_id}")
try:
self._delete_local_download_data(client, download_id)
self._remove_usenet_download(client, download_id, delete_files=True, archive=True)
except Exception as e:
logger.warning(
f"Failed to remove download {download_id} from {client.name} after cancellation: {e}"
)
else:
logger.info(
f"Download cancelled for protocol={protocol}; leaving in {client.name}: {download_id}"
)
status_callback("cancelled", "Cancelled")
self._handle_cancelled_download(client, download_id, protocol, status_callback)
return None
# Handle completed file
source_path = client.get_download_path(download_id)
if not source_path:
logger.error(
f"Download client returned empty path for completed download. "
f"Client: {client.name}, ID: {download_id}. "
f"Check that the download client's completion folder is accessible to Shelfmark."
)
# Handle completed file (wait briefly for files to appear)
source_path_obj, path_error = self._wait_for_completed_path(
client=client,
download_id=download_id,
cancel_flag=cancel_flag,
status_callback=status_callback,
)
if not source_path_obj:
if cancel_flag.is_set():
self._handle_cancelled_download(client, download_id, protocol, status_callback)
return None
status_callback(
"error",
f"Could not locate completed download in {client.name} (path not returned). "
f"Check volume mappings and category settings."
path_error
or f"Could not locate completed download in {client.name} (path not returned). Check volume mappings and category settings.",
)
return None
# Apply remote path mappings (client path -> shelfmark container path)
from shelfmark.core.path_mappings import (
get_client_host_identifier,
parse_remote_path_mappings,
remap_remote_to_local_with_match,
)
source_path_obj = Path(source_path)
host = get_client_host_identifier(client) or ""
mapping_value = config.get("PROWLARR_REMOTE_PATH_MAPPINGS", [])
mappings = parse_remote_path_mappings(mapping_value)
logger.debug(
"Attempting path remap: client=%s, host=%s, path=%s, mappings=%s",
client.name,
host,
source_path_obj,
[(m.host, m.remote_path, m.local_path) for m in mappings],
)
remapped, matched_mapping = remap_remote_to_local_with_match(
mappings=mappings,
host=host,
remote_path=source_path_obj,
)
logger.debug(
"Remap result: %s -> %s (exists=%s, changed=%s, matched=%s)",
source_path_obj,
remapped,
remapped.exists(),
remapped != source_path_obj,
matched_mapping,
)
if matched_mapping:
if remapped.exists():
logger.info(
"Remapped download path for %s (%s): %s -> %s",
client.name,
download_id,
source_path_obj,
remapped,
)
source_path_obj = remapped
else:
logger.error(
f"Download path does not exist after remapping: {source_path} -> {remapped}. "
f"Client: {client.name}, ID: {download_id}. "
f"Check that the local path in your mapping is mounted correctly."
)
status_callback(
"error",
f"Remapped path '{remapped}' does not exist. "
f"Check your Docker volume mounts match the Local Path in Settings > Advanced > Remote Path Mappings.",
)
return None
elif mappings:
if source_path_obj.exists():
logger.info(
"No remote path mapping matched for %s (%s); using client path: %s",
client.name,
download_id,
source_path_obj,
)
else:
hint = _diagnose_path_issue(source_path)
logger.error(
f"Download path does not exist and no remote path mapping matched for {client.name} "
f"({download_id}): {source_path}. {hint}"
)
status_callback(
"error",
f"{hint} No remote path mapping matched for client '{client.name}'.",
)
return None
elif not source_path_obj.exists():
hint = _diagnose_path_issue(source_path)
logger.error(
f"Download path does not exist: {source_path}. "
f"Client: {client.name}, ID: {download_id}. {hint}"
)
status_callback("error", hint)
return None
result = self._handle_completed_file(
source_path=source_path_obj,
protocol=protocol,
+352 -69
View File
@@ -59,6 +59,51 @@ AUDIOBOOK_FORMATS = ["m4b", "mp3", "m4a", "flac", "ogg", "wma", "aac", "wav", "o
# Combined list for format detection (audiobook formats first for priority)
ALL_BOOK_FORMATS = AUDIOBOOK_FORMATS + EBOOK_FORMATS
# Map 3-char MAM language codes to 2-char ISO codes used by frontend color maps
MAM_LANGUAGE_MAP = {
"eng": "en",
"ita": "it",
"spa": "es",
"fra": "fr",
"fre": "fr",
"ger": "de",
"deu": "de",
"por": "pt",
"rus": "ru",
"jpn": "ja",
"jap": "ja",
"chi": "zh",
"zho": "zh",
"dut": "nl",
"nld": "nl",
"swe": "sv",
"nor": "no",
"dan": "da",
"fin": "fi",
"pol": "pl",
"cze": "cs",
"ces": "cs",
"hun": "hu",
"kor": "ko",
"ara": "ar",
"heb": "he",
"tur": "tr",
"gre": "el",
"ell": "el",
"hin": "hi",
"tha": "th",
"vie": "vi",
"ind": "id",
"ukr": "uk",
"rom": "ro",
"ron": "ro",
"bul": "bg",
"cat": "ca",
"hrv": "hr",
"slv": "sl",
"srp": "sr",
}
# Backend safeguard: cap total Prowlarr search time per request.
PROWLARR_SEARCH_TIMEOUT_SECONDS = 120.0
@@ -83,33 +128,79 @@ def _extract_format(title: str) -> Optional[str]:
return None
def _extract_language(title: str) -> Optional[str]:
"""Extract language code from release title (e.g., [German] -> 'de')."""
title_lower = title.lower()
def _extract_mam_language(raw_title: str) -> Optional[str]:
"""
Extract the language code from MyAnonamouse titles.
# Common language names and their codes
languages = {
"english": "en", "eng": "en", "[en]": "en", "(en)": "en",
"german": "de", "deutsch": "de", "[de]": "de", "(de)": "de", "ger": "de",
"french": "fr", "français": "fr", "[fr]": "fr", "(fr)": "fr", "fra": "fr",
"spanish": "es", "español": "es", "[es]": "es", "(es)": "es", "spa": "es",
"italian": "it", "italiano": "it", "[it]": "it", "(it)": "it", "ita": "it",
"portuguese": "pt", "[pt]": "pt", "(pt)": "pt", "por": "pt",
"dutch": "nl", "nederlands": "nl", "[nl]": "nl", "(nl)": "nl", "nld": "nl",
"russian": "ru", "[ru]": "ru", "(ru)": "ru", "rus": "ru",
"polish": "pl", "polski": "pl", "[pl]": "pl", "(pl)": "pl", "pol": "pl",
"chinese": "zh", "[zh]": "zh", "(zh)": "zh", "chi": "zh",
"japanese": "ja", "[ja]": "ja", "(ja)": "ja", "jpn": "ja",
"korean": "ko", "[ko]": "ko", "(ko)": "ko", "kor": "ko",
}
Prowlarr's MAM parser appends a structured bracket segment like:
[ENG / EPUB MOBI PDF]
for lang_pattern, lang_code in languages.items():
if lang_pattern in title_lower:
return lang_code
The language code appears before the "/" - we extract it and map to
the 2-char ISO code used by the frontend color maps.
"""
if not raw_title:
return None
for bracket in re.findall(r"\[([^\]]+)\]", raw_title):
if "/" not in bracket:
continue
before_slash, _ = bracket.split("/", 1)
# Extract the language token (should be a 3-char code like ENG, ITA, etc.)
tokens = re.findall(r"[A-Za-z]+", before_slash.strip())
for token in tokens:
lang_code = token.lower()
if lang_code in MAM_LANGUAGE_MAP:
return MAM_LANGUAGE_MAP[lang_code]
return None
def _extract_mam_formats(raw_title: str) -> List[str]:
"""
Extract a list of formats from MyAnonamouse titles.
Prowlarr's MAM parser appends a structured bracket segment like:
[ENG / EPUB MOBI PDF]
We only trust this structured segment (and do not attempt generic title
heuristics for other indexers).
"""
if not raw_title:
return []
format_set = set(ALL_BOOK_FORMATS)
for bracket in re.findall(r"\[([^\]]+)\]", raw_title):
if "/" not in bracket:
continue
_, after_slash = bracket.split("/", 1)
tokens = re.findall(r"[A-Za-z0-9]+", after_slash)
formats: List[str] = []
for token in tokens:
fmt = token.lower()
if fmt in format_set and fmt not in formats:
formats.append(fmt)
if formats:
return formats
return []
def _formats_display(formats: List[str]) -> Optional[str]:
if not formats:
return None
if len(formats) == 1:
return formats[0]
if len(formats) == 2:
return f"{formats[0]}, {formats[1]}"
# Show first two formats + count of others to prevent overflow
return f"{formats[0]}, {formats[1]} +{len(formats) - 2}"
# Prowlarr category IDs for content type detection
# See: https://wiki.servarr.com/prowlarr/cardigann-yml-definition#categories
AUDIOBOOK_CATEGORY_IDS = {3000, 3030} # 3000 = Audio, 3030 = Audio/Audiobook
@@ -144,9 +235,15 @@ def _detect_content_type_from_categories(categories: list, fallback: str = "book
return "other"
def _prowlarr_result_to_release(result: dict, search_content_type: str = "ebook") -> Release:
def _prowlarr_result_to_release(
result: dict,
search_content_type: str = "ebook",
*,
enable_format_detection: bool = False,
) -> Release:
"""Convert a Prowlarr API result to a Release object."""
title = result.get("title", "Unknown")
raw_title = result.get("title", "Unknown")
title = raw_title
size_bytes = result.get("size")
indexer = result.get("indexer", "Unknown")
protocol = get_protocol(result)
@@ -154,6 +251,27 @@ def _prowlarr_result_to_release(result: dict, search_content_type: str = "ebook"
leechers = result.get("leechers")
categories = result.get("categories", [])
is_torrent = protocol == ReleaseProtocol.TORRENT
raw_indexer_flags = result.get("indexerFlags") or []
indexer_flags: List[str] = []
seen_flags: set[str] = set()
def add_indexer_flag(flag: object) -> None:
if flag is None:
return
flag_str = str(flag).strip()
if not flag_str:
return
lowered = flag_str.lower()
if lowered in seen_flags:
return
seen_flags.add(lowered)
indexer_flags.append(flag_str)
if isinstance(raw_indexer_flags, list):
for flag in raw_indexer_flags:
add_indexer_flag(flag)
elif isinstance(raw_indexer_flags, str):
add_indexer_flag(raw_indexer_flags)
# Format peers display string: "seeders / leechers"
peers_display = (
@@ -162,22 +280,50 @@ def _prowlarr_result_to_release(result: dict, search_content_type: str = "ebook"
else None
)
# For format detection, prefer fileName over title (often cleaner)
file_name = result.get("fileName", "")
format_detected = _extract_format(file_name) if file_name else _extract_format(title)
format_detected: Optional[str] = None
formats: List[str] = []
formats_display: Optional[str] = None
language_detected: Optional[str] = None
if enable_format_detection:
book_title = str(result.get("bookTitle") or "").strip()
if book_title:
title = book_title
formats = _extract_mam_formats(str(raw_title or ""))
format_detected = formats[0] if formats else None
formats_display = _formats_display(formats)
language_detected = _extract_mam_language(str(raw_title or ""))
# Build the source_id from GUID or generate from indexer + title
source_id = result.get("guid") or f"{indexer}:{hash(title)}"
source_id = result.get("guid") or f"{indexer}:{hash(raw_title)}"
# Cache the raw Prowlarr result so handler can look it up by source_id
cache_release(source_id, result)
# Derive common indicators from torznab/newznab attrs when present.
download_volume_factor = result.get("downloadVolumeFactor")
is_freeleech = False
try:
if download_volume_factor is not None and float(download_volume_factor) == 0.0:
is_freeleech = True
except (TypeError, ValueError):
pass
if any(flag.lower() in {"freeleech", "fl"} for flag in indexer_flags):
is_freeleech = True
is_vip = "[vip]" in str(raw_title).lower()
if is_vip:
add_indexer_flag("VIP")
if is_freeleech:
add_indexer_flag("FreeLeech")
return Release(
source="prowlarr",
source_id=source_id,
title=title,
format=format_detected,
language=_extract_language(title),
language=language_detected,
size=_parse_size(size_bytes),
size_bytes=size_bytes,
download_url=get_preferred_download_url(result),
@@ -199,6 +345,20 @@ def _prowlarr_result_to_release(result: dict, search_content_type: str = "ebook"
"indexer_id": result.get("indexerId"),
"files": result.get("files"),
"grabs": result.get("grabs"),
"author": result.get("author"),
"book_title": result.get("bookTitle"),
"indexer_flags": indexer_flags,
"vip": is_vip,
"freeleech": is_freeleech,
"download_volume_factor": result.get("downloadVolumeFactor"),
"upload_volume_factor": result.get("uploadVolumeFactor"),
"minimum_ratio": result.get("minimumRatio"),
"minimum_seed_time": result.get("minimumSeedTime"),
"info_hash": result.get("infoHash"),
"formats": formats if formats else None,
"formats_display": formats_display,
# Raw torznab attributes for rich tooltips (enriched indexers)
"torznab_attrs": result.get("torznabAttrs"),
},
)
@@ -216,48 +376,85 @@ class ProwlarrSource(ReleaseSource):
def get_column_config(self) -> ReleaseColumnConfig:
"""Column configuration for Prowlarr releases."""
# Fetch available indexers from Prowlarr
available_indexers: Optional[List[str]] = None
default_indexers: Optional[List[str]] = None
client = self._get_client()
if client:
try:
enabled_indexers = client.get_enabled_indexers_detailed()
# Get user-selected indexer IDs if configured
selected_ids = self._get_selected_indexer_ids()
all_indexer_names = []
selected_indexer_names = []
for idx in enabled_indexers:
idx_id = idx.get("id")
idx_name = idx.get("name")
if not idx_name:
continue
# Add to all indexers list
all_indexer_names.append(idx_name)
# If user has selected specific indexers, track those separately
if selected_ids is not None:
try:
if int(idx_id) in selected_ids:
selected_indexer_names.append(idx_name)
except (TypeError, ValueError):
pass
available_indexers = sorted(all_indexer_names) if all_indexer_names else None
# Only set default_indexers if user has selected specific ones
default_indexers = sorted(selected_indexer_names) if selected_indexer_names else None
except Exception as e:
logger.warning(f"Failed to fetch indexer list for column config: {e}")
return ReleaseColumnConfig(
columns=[
ColumnSchema(
key="indexer",
label="Indexer",
render_type=ColumnRenderType.TEXT,
render_type=ColumnRenderType.INDEXER_PROTOCOL,
align=ColumnAlign.LEFT,
width="minmax(80px, 1fr)",
hide_mobile=True,
width="minmax(140px, 1fr)",
hide_mobile=False,
sortable=True,
),
ColumnSchema(
key="protocol",
label="Type",
render_type=ColumnRenderType.BADGE,
key="extra.indexer_flags",
label="Flags",
render_type=ColumnRenderType.TAGS,
align=ColumnAlign.CENTER,
width="60px",
width="50px",
hide_mobile=False,
color_hint=ColumnColorHint(type="map", value="download_type"),
color_hint=ColumnColorHint(type="map", value="flags"),
fallback="",
uppercase=True,
),
ColumnSchema(
key="peers",
label="Peers",
render_type=ColumnRenderType.PEERS,
key="language",
label="Lang",
render_type=ColumnRenderType.BADGE,
align=ColumnAlign.CENTER,
width="70px",
width="50px",
hide_mobile=True,
fallback="-",
sortable=True,
sort_key="seeders",
color_hint=ColumnColorHint(type="map", value="language"),
uppercase=True,
fallback="",
),
ColumnSchema(
key="content_type",
label="Type",
render_type=ColumnRenderType.BADGE,
key="extra.formats_display",
label="Format",
render_type=ColumnRenderType.FORMAT_CONTENT_TYPE,
align=ColumnAlign.CENTER,
width="90px",
hide_mobile=False,
color_hint=ColumnColorHint(type="map", value="content_type"),
color_hint=ColumnColorHint(type="map", value="format"),
uppercase=True,
fallback="-",
fallback="",
),
ColumnSchema(
key="size",
@@ -270,9 +467,11 @@ class ProwlarrSource(ReleaseSource):
sort_key="size_bytes",
),
],
grid_template="minmax(0,2fr) minmax(80px,1fr) 60px 70px 90px 80px",
grid_template="minmax(0,2fr) minmax(140px,1fr) 50px 50px 90px 80px",
leading_cell=LeadingCellConfig(type=LeadingCellType.NONE), # No leading cell for Prowlarr
supported_filters=["language"], # Enables multi-language query expansion; Prowlarr language metadata is unreliable
available_indexers=available_indexers,
default_indexers=default_indexers,
supported_filters=["language", "indexer"], # Enables multi-language query expansion and indexer filtering
)
def _get_client(self) -> Optional[ProwlarrClient]:
@@ -313,6 +512,39 @@ class ProwlarrSource(ReleaseSource):
logger.warning(f"Invalid PROWLARR_INDEXERS format: {selected} ({e})")
return None
def _resolve_indexer_ids_from_names(
self, client: ProwlarrClient, names: List[str]
) -> Optional[List[int]]:
"""
Convert indexer names to IDs by looking up enabled indexers.
Returns None if no names could be resolved.
"""
if not names:
return None
try:
enabled_indexers = client.get_enabled_indexers_detailed()
name_to_id = {
idx.get("name"): idx.get("id")
for idx in enabled_indexers
if idx.get("name") and idx.get("id") is not None
}
ids = []
for name in names:
idx_id = name_to_id.get(name)
if idx_id is not None:
try:
ids.append(int(idx_id))
except (TypeError, ValueError):
pass
return ids if ids else None
except Exception as e:
logger.warning(f"Failed to resolve indexer names to IDs: {e}")
return None
def search(
self,
book: BookMetadata,
@@ -336,8 +568,12 @@ class ProwlarrSource(ReleaseSource):
logger.warning("No search query available for book")
return []
# Get selected indexer IDs from config (None means search all)
indexer_ids = self._get_selected_indexer_ids()
# Get indexer IDs: prefer plan.indexers (from filter), else use settings
if plan.indexers:
indexer_ids = self._resolve_indexer_ids_from_names(client, plan.indexers)
logger.debug(f"Using filter-specified indexers: {plan.indexers} -> IDs {indexer_ids}")
else:
indexer_ids = self._get_selected_indexer_ids()
# Get search categories based on content type
# Audiobooks use 3030 (Audio/Audiobook), ebooks use 7000 (Books)
@@ -370,37 +606,61 @@ class ProwlarrSource(ReleaseSource):
f"Searching Prowlarr: {query_type} ({len(queries)} variants), {indexer_desc}, categories={categories}"
)
# Identify indexers that should be enriched via Torznab/Newznab.
enriched_indexer_ids = client.get_enriched_indexer_ids(restrict_to=indexer_ids)
non_enriched_indexer_ids: Optional[List[int]] = None
if indexer_ids:
non_enriched_indexer_ids = [i for i in indexer_ids if i not in enriched_indexer_ids]
def search_indexers(query: str, cats: Optional[List[int]]) -> List[dict]:
"""Search indexers with given categories, collecting results."""
results = []
if indexer_ids:
# Prefer a single request for all selected indexers to reduce latency.
try:
raw = client.search(query=query, indexer_ids=indexer_ids, categories=cats)
if raw:
results.extend(raw)
return results
except Exception as e:
logger.warning(
f"Search failed for selected indexers {indexer_ids}: {e}. Falling back to per-indexer search."
)
# Fallback: search specific indexers one at a time
for indexer_id in indexer_ids:
# Search standard indexers via JSON endpoint.
if indexer_ids:
if non_enriched_indexer_ids:
# Prefer a single request for selected indexers to reduce latency.
try:
raw = client.search(query=query, indexer_ids=[indexer_id], categories=cats)
raw = client.search(query=query, indexer_ids=non_enriched_indexer_ids, categories=cats)
if raw:
results.extend(raw)
except Exception as e:
logger.warning(f"Search failed for indexer {indexer_id}: {e}")
logger.warning(
f"Search failed for selected indexers {non_enriched_indexer_ids}: {e}. Falling back to per-indexer search."
)
for indexer_id in non_enriched_indexer_ids:
try:
raw = client.search(query=query, indexer_ids=[indexer_id], categories=cats)
if raw:
results.extend(raw)
except Exception as e:
logger.warning(f"Search failed for indexer {indexer_id}: {e}")
else:
# Search all enabled indexers at once
# Search all enabled indexers at once, then remove enriched results (re-fetched via Torznab).
try:
raw = client.search(query=query, indexer_ids=None, categories=cats)
if raw:
if enriched_indexer_ids:
raw = [r for r in raw if r.get("indexerId") not in enriched_indexer_ids]
results.extend(raw)
except Exception as e:
logger.warning(f"Search failed for all indexers: {e}")
# Search enriched indexers via Torznab/Newznab for richer metadata.
for indexer_id in enriched_indexer_ids:
raw = client.torznab_search(indexer_id=indexer_id, query=query, categories=cats, search_type="book")
if raw:
results.extend(raw)
else:
# Fallback to JSON search for enriched indexers if Torznab fails.
try:
raw_fallback = client.search(query=query, indexer_ids=[indexer_id], categories=cats)
if raw_fallback:
results.extend(raw_fallback)
except Exception as e:
logger.warning(f"Fallback search failed for enriched indexer {indexer_id}: {e}")
return results
try:
@@ -442,7 +702,30 @@ class ProwlarrSource(ReleaseSource):
seen_keys.add(key)
all_results.append(r)
results = [_prowlarr_result_to_release(r, content_type) for r in all_results]
enriched_indexer_ids_set = set(enriched_indexer_ids)
results: List[Release] = []
enriched_source_ids: set[str] = set()
for r in all_results:
idx_id = r.get("indexerId")
try:
idx_id_int = int(idx_id) if idx_id is not None else None
except (TypeError, ValueError):
idx_id_int = None
is_enriched = bool(idx_id_int is not None and idx_id_int in enriched_indexer_ids_set)
release = _prowlarr_result_to_release(
r,
content_type,
enable_format_detection=is_enriched,
)
results.append(release)
if is_enriched:
enriched_source_ids.add(release.source_id)
# Sort results: enriched indexers first, then others
results.sort(key=lambda r: (0 if r.source_id in enriched_source_ids else 1))
if results:
torrent_count = sum(1 for r in results if r.protocol == ReleaseProtocol.TORRENT)
@@ -0,0 +1,171 @@
"""
Torznab/Newznab (RSS/XML) helpers for Prowlarr.
Used to fetch richer metadata from specific indexers (e.g., MyAnonamouse) that
isn't available via Prowlarr's JSON search endpoint.
"""
from __future__ import annotations
from typing import Any, Dict, List, Optional
from xml.etree import ElementTree as ET
def _local_name(tag: str) -> str:
"""Return tag name without namespace."""
if tag.startswith("{"):
return tag.split("}", 1)[1]
return tag
def _coerce_int(value: Optional[str]) -> Optional[int]:
if value is None:
return None
value = value.strip()
if not value:
return None
try:
return int(value)
except ValueError:
return None
def _coerce_float(value: Optional[str]) -> Optional[float]:
if value is None:
return None
value = value.strip()
if not value:
return None
try:
return float(value)
except ValueError:
return None
def _strip_author_from_title(title: str, author: Optional[str]) -> str:
"""
Prowlarr's MyAnonamouse parser appends " by {author}" into the title while
also emitting author/booktitle fields. Shelfmark's UI shows author
separately, so strip the duplicated " by author" segment when present.
"""
if not title or not author:
return title
needle = f" by {author}"
if needle in title:
return title.replace(needle, "", 1).strip()
return title
def parse_torznab_xml(xml_text: str) -> List[Dict[str, Any]]:
"""
Parse a Torznab/Newznab XML response into a list of dicts that roughly match
Prowlarr's JSON search results shape.
"""
if not xml_text or not xml_text.strip():
return []
try:
root = ET.fromstring(xml_text)
except ET.ParseError:
return []
items = root.findall(".//item")
results: List[Dict[str, Any]] = []
for item in items:
title = (item.findtext("title") or "").strip()
guid = (item.findtext("guid") or "").strip() or None
download_url = (item.findtext("link") or "").strip() or None
info_url = (item.findtext("comments") or "").strip() or None
pub_date = (item.findtext("pubDate") or "").strip() or None
size = _coerce_int(item.findtext("size"))
enclosure = item.find("enclosure")
enclosure_type = enclosure.get("type") if enclosure is not None else None
enclosure_url = enclosure.get("url") if enclosure is not None else None
protocol: Optional[str] = None
if enclosure_type == "application/x-bittorrent":
protocol = "torrent"
elif enclosure_type == "application/x-nzb":
protocol = "usenet"
if not download_url and enclosure_url:
download_url = enclosure_url.strip() or None
prowlarr_indexer_el = item.find("prowlarrindexer")
indexer_id = _coerce_int(prowlarr_indexer_el.get("id")) if prowlarr_indexer_el is not None else None
indexer_name = (prowlarr_indexer_el.text or "").strip() if prowlarr_indexer_el is not None else ""
categories: List[int] = []
for cat_el in item.findall("category"):
cat_id = _coerce_int(cat_el.text)
if cat_id is not None:
categories.append(cat_id)
# Collect torznab/newznab attr elements (namespaced).
attrs: Dict[str, str] = {}
tags: List[str] = []
for el in item.iter():
if _local_name(el.tag) != "attr":
continue
name = (el.get("name") or "").strip()
value = (el.get("value") or "").strip()
if not name:
continue
if name == "tag" and value:
tags.append(value)
continue
if value:
attrs[name] = value
seeders = _coerce_int(attrs.get("seeders"))
peers = _coerce_int(attrs.get("peers"))
leechers: Optional[int] = None
if peers is not None and seeders is not None and peers >= seeders:
leechers = peers - seeders
author = attrs.get("author") or None
book_title = attrs.get("booktitle") or None
info_hash = attrs.get("infohash") or None
download_volume_factor = _coerce_float(attrs.get("downloadvolumefactor"))
upload_volume_factor = _coerce_float(attrs.get("uploadvolumefactor"))
minimum_ratio = _coerce_float(attrs.get("minimumratio"))
minimum_seed_time = _coerce_int(attrs.get("minimumseedtime"))
cleaned_title = _strip_author_from_title(title, author)
results.append({
"title": cleaned_title or title,
"guid": guid or info_url or download_url or f"{indexer_id}:{title}",
"size": size,
"protocol": protocol or "unknown",
"downloadUrl": download_url,
"infoUrl": info_url,
"publishDate": pub_date,
"indexer": indexer_name or None,
"indexerId": indexer_id,
"categories": categories,
"seeders": seeders,
"leechers": leechers,
"files": _coerce_int(attrs.get("files")),
"grabs": _coerce_int(attrs.get("grabs")),
"infoHash": info_hash,
"indexerFlags": tags,
# Optional richer fields (not available via JSON search)
"author": author,
"bookTitle": book_title,
"downloadVolumeFactor": download_volume_factor,
"uploadVolumeFactor": upload_volume_factor,
"minimumRatio": minimum_ratio,
"minimumSeedTime": minimum_seed_time,
# Pass through all torznab attributes for tooltip display
"torznabAttrs": attrs,
})
return results
+27 -3
View File
@@ -33,6 +33,20 @@ import { withBasePath } from './utils/basePath';
import { SearchModeProvider } from './contexts/SearchModeContext';
import './styles.css';
const CONTENT_TYPE_STORAGE_KEY = 'preferred-content-type';
const getInitialContentType = (): ContentType => {
try {
const saved = localStorage.getItem(CONTENT_TYPE_STORAGE_KEY);
if (saved === 'ebook' || saved === 'audiobook') {
return saved;
}
} catch {
// localStorage may be unavailable in private browsing
}
return 'ebook';
};
function App() {
const { toasts, showToast, removeToast } = useToast();
@@ -73,7 +87,15 @@ function App() {
});
// Content type state (ebook vs audiobook) - defined before useSearch since it's passed to it
const [contentType, setContentType] = useState<ContentType>('ebook');
const [contentType, setContentType] = useState<ContentType>(() => getInitialContentType());
useEffect(() => {
try {
localStorage.setItem(CONTENT_TYPE_STORAGE_KEY, contentType);
} catch {
// localStorage may be unavailable in private browsing
}
}, [contentType]);
// Search state and handlers
const {
@@ -143,6 +165,7 @@ function App() {
const ongoing = [
currentStatus.queued,
currentStatus.resolving,
currentStatus.locating,
currentStatus.downloading,
].reduce((sum, status) => sum + (status ? Object.keys(status).length : 0), 0);
@@ -506,6 +529,7 @@ function App() {
: [bookLanguages[0]?.code || 'en'];
const searchMode = config?.search_mode || 'direct';
const logoUrl = withBasePath('/logo.png');
// Handle "View Series" - trigger search with series field and series order sort
const handleSearchSeries = useCallback((seriesName: string) => {
@@ -537,7 +561,7 @@ function App() {
calibreWebUrl={config?.calibre_web_url || ''}
audiobookLibraryUrl={config?.audiobook_library_url || ''}
debug={config?.debug || false}
logoUrl="/logo.png"
logoUrl={logoUrl}
showSearch={!isInitialState}
searchInput={searchInput}
onSearchChange={setSearchInput}
@@ -604,7 +628,7 @@ function App() {
bookLanguages={bookLanguages}
defaultLanguage={defaultLanguageCodes}
supportedFormats={config?.supported_formats || DEFAULT_SUPPORTED_FORMATS}
logoUrl="/logo.png"
logoUrl={logoUrl}
searchInput={searchInput}
onSearchInputChange={setSearchInput}
showAdvanced={showAdvanced}
@@ -63,7 +63,7 @@ export const BookDownloadButton = ({
const isCompleted = buttonState.state === 'complete';
const hasError = buttonState.state === 'error';
const isInProgress = ['queued', 'resolving', 'downloading'].includes(buttonState.state);
const isInProgress = ['queued', 'resolving', 'locating', 'downloading'].includes(buttonState.state);
const isDisabled = buttonState.state !== 'download' || isQueuing || isCompleted;
const displayText = isQueuing ? 'Queuing...' : buttonState.text;
const showCircularProgress = buttonState.state === 'downloading' && buttonState.progress !== undefined;
@@ -205,4 +205,3 @@ export const BookDownloadButton = ({
</button>
);
};
@@ -56,7 +56,7 @@ export const BookGetButton = ({
// Determine states based on buttonState
const isCompleted = buttonState?.state === 'complete';
const hasError = buttonState?.state === 'error';
const isInProgress = buttonState && ['queued', 'resolving', 'downloading'].includes(buttonState.state);
const isInProgress = buttonState && ['queued', 'resolving', 'locating', 'downloading'].includes(buttonState.state);
const showCircularProgress = buttonState?.state === 'downloading' && buttonState.progress !== undefined;
const showSpinner = (isInProgress && !showCircularProgress) || isLoading;
@@ -104,6 +104,7 @@ export const BookGetButton = ({
if (hasError) return 'Failed';
if (isLoading) return 'Loading';
if (buttonState?.state === 'downloading') return 'Downloading';
if (buttonState?.state === 'locating') return 'Locating files';
if (buttonState?.state === 'resolving') return 'Resolving';
if (buttonState?.state === 'queued') return 'Queued';
return 'Get';
@@ -14,6 +14,7 @@ const STATUS_STYLES: Record<string, { bg: string; text: string; label: string; w
queued: { bg: 'bg-amber-500/20', text: 'text-amber-700 dark:text-amber-300', label: 'Queued', waveColor: 'rgba(217, 119, 6, 0.3)' },
resolving: { bg: 'bg-indigo-500/20', text: 'text-indigo-700 dark:text-indigo-300', label: 'Resolving', waveColor: 'rgba(79, 70, 229, 0.3)' },
downloading: { bg: 'bg-sky-500/20', text: 'text-sky-700 dark:text-sky-300', label: 'Downloading', waveColor: 'rgba(2, 132, 199, 0.3)' },
locating: { bg: 'bg-teal-500/20', text: 'text-teal-700 dark:text-teal-300', label: 'Locating files', waveColor: 'rgba(13, 148, 136, 0.3)' },
complete: { bg: 'bg-green-500/20', text: 'text-green-700 dark:text-green-300', label: 'Complete', waveColor: '' },
error: { bg: 'bg-red-500/20', text: 'text-red-700 dark:text-red-300', label: 'Error', waveColor: '' },
cancelled: { bg: 'bg-gray-500/20', text: 'text-gray-700 dark:text-gray-300', label: 'Cancelled', waveColor: '' },
@@ -77,6 +78,8 @@ const getStatusProgress = (statusName: string, bookProgress?: number): number =>
return 20 + (bookProgress * 0.8);
}
return 20;
case 'locating':
return 90;
case 'complete':
case 'error':
return 100;
@@ -92,6 +95,7 @@ const getProgressBarColor = (statusName: string): string => {
if (statusName === 'queued') return 'bg-amber-600';
if (statusName === 'resolving') return 'bg-indigo-600';
if (statusName === 'downloading') return 'bg-sky-600';
if (statusName === 'locating') return 'bg-teal-600';
return 'bg-sky-600';
};
@@ -120,7 +124,7 @@ export const DownloadsSidebar = ({
// Collect all download items from different status sections
const allDownloadItems: Array<{ book: Book; status: string }> = [];
const statusTypes = ['downloading', 'resolving', 'queued', 'error', 'complete', 'cancelled'];
const statusTypes = ['downloading', 'locating', 'resolving', 'queued', 'error', 'complete', 'cancelled'];
statusTypes.forEach((statusName) => {
const items = (status as any)[statusName];
@@ -142,9 +146,9 @@ export const DownloadsSidebar = ({
label: statusName.charAt(0).toUpperCase() + statusName.slice(1),
};
const isInProgress = ['queued', 'resolving', 'downloading'].includes(statusName);
const isInProgress = ['queued', 'resolving', 'locating', 'downloading'].includes(statusName);
const isQueued = statusName === 'queued';
const isActive = statusName === 'resolving' || statusName === 'downloading';
const isActive = statusName === 'resolving' || statusName === 'locating' || statusName === 'downloading';
const isCompleted = statusName === 'complete';
const hasError = statusName === 'error';
@@ -13,6 +13,7 @@ import {
CheckboxFieldConfig,
SelectFieldConfig,
MultiSelectFieldConfig,
TagListFieldConfig,
HeadingFieldConfig,
ActionButtonConfig,
ActionResult,
@@ -25,6 +26,7 @@ import {
CheckboxField,
SelectField,
MultiSelectField,
TagListField,
HeadingField,
ActionButton,
} from './settings/fields';
@@ -137,6 +139,15 @@ const renderField = (
disabled={isDisabled}
/>
);
case 'TagListField':
return (
<TagListField
field={field as TagListFieldConfig}
value={(value as string[]) ?? []}
onChange={(v) => onChange(v)}
disabled={isDisabled}
/>
);
case 'ActionButton':
return (
<ActionButton
+361 -6
View File
@@ -1,6 +1,7 @@
import { ColumnSchema, Release } from '../types';
import { getColorStyleFromHint } from '../utils/colorMaps';
import { getColorStyleFromHint, getProtocolDotColor, getFormatColor } from '../utils/colorMaps';
import { getNestedValue } from '../utils/objectHelpers';
import { Tooltip } from './shared/Tooltip';
interface ReleaseCellProps {
column: ColumnSchema;
@@ -37,12 +38,48 @@ export const ReleaseCell = ({ column, release, compact = false, onlineServers }:
return <span>{displayValue}</span>;
}
const colorStyle = getColorStyleFromHint(value, column.color_hint);
// Build rich tooltip content for formats
let tooltipContent: React.ReactNode = null;
if (column.key === 'extra.formats_display') {
const formats = (release.extra as Record<string, unknown> | undefined)?.formats;
if (Array.isArray(formats) && formats.length > 1) {
tooltipContent = (
<div className="flex flex-wrap gap-1.5">
{formats.map((fmt) => {
const fmtColor = getFormatColor(String(fmt));
return (
<span
key={String(fmt)}
className={`${fmtColor.bg} ${fmtColor.text} text-[10px] sm:text-[11px] font-semibold px-1.5 sm:px-2 py-0.5 rounded-lg tracking-wide`}
>
{String(fmt).toUpperCase()}
</span>
);
})}
</div>
);
}
}
const badge = (
<span
className={`${colorStyle.bg} ${colorStyle.text} text-[10px] sm:text-[11px] font-semibold px-1.5 sm:px-2 py-0.5 rounded-lg tracking-wide`}
>
{displayValue}
</span>
);
return (
<div className={`flex items-center ${alignClass}`}>
{value !== column.fallback ? (
<span className={`${colorStyle.bg} ${colorStyle.text} text-[10px] sm:text-[11px] font-semibold px-1.5 sm:px-2 py-0.5 rounded-lg tracking-wide`}>
{displayValue}
</span>
tooltipContent ? (
<Tooltip content={tooltipContent} position="top">
{badge}
</Tooltip>
) : (
badge
)
) : (
<span className="text-[10px] sm:text-xs text-gray-500 dark:text-gray-400">{column.fallback}</span>
)}
@@ -50,15 +87,175 @@ export const ReleaseCell = ({ column, release, compact = false, onlineServers }:
);
}
case 'size':
case 'tags': {
const tags = Array.isArray(rawValue)
? rawValue.map((tag) => String(tag)).filter((tag) => tag.trim())
: rawValue !== undefined && rawValue !== null && String(rawValue).trim()
? [String(rawValue)]
: [];
const isFlags = column.color_hint?.type === 'map' && column.color_hint.value === 'flags';
const normalizeFlagLabel = (tag: string): string => {
const normalized = tag.trim().toLowerCase();
if (!normalized) return tag;
switch (normalized) {
case 'freeleech':
case 'free leech':
case 'fl':
return 'FL';
case 'double upload':
case 'doubleupload':
case 'du':
return 'DU';
case 'vip':
return 'VIP';
case 'internal':
case 'int':
return 'INT';
default:
return tag;
}
};
if (compact) {
if (tags.length === 0) {
return column.fallback ? <span>{column.fallback}</span> : null;
}
const displayTags = tags.map((tag) => {
const normalized = isFlags ? normalizeFlagLabel(tag) : tag;
return column.uppercase ? normalized.toUpperCase() : normalized;
});
return <span>{displayTags.join(', ')}</span>;
}
if (!tags.length) {
if (!column.fallback) {
return <div className={`flex items-center ${alignClass}`} />;
}
return (
<div className={`flex items-center ${alignClass}`}>
<span className="text-[10px] sm:text-xs text-gray-500 dark:text-gray-400">{column.fallback}</span>
</div>
);
}
return (
<div className={`flex flex-wrap items-center gap-1.5 ${alignClass}`}>
{tags.map((tag, idx) => {
const normalized = isFlags ? normalizeFlagLabel(tag) : tag;
const displayTag = column.uppercase ? normalized.toUpperCase() : normalized;
const colorStyle = getColorStyleFromHint(tag, column.color_hint);
return (
<span
key={`${tag}-${idx}`}
className={`${colorStyle.bg} ${colorStyle.text} text-[10px] sm:text-[11px] font-semibold px-1.5 sm:px-2 py-0.5 rounded-lg tracking-wide whitespace-nowrap`}
>
{displayTag}
</span>
);
})}
</div>
);
}
case 'size': {
// Build tooltip from extra metadata (torznab attrs, publish date, etc.)
const extra = release.extra as Record<string, unknown> | undefined;
const torznabAttrs = extra?.torznab_attrs as Record<string, string> | undefined;
const publishDate = extra?.publish_date as string | undefined;
// Helper to format age in days
const formatRelativeTime = (dateStr: string): string | null => {
try {
const date = new Date(dateStr);
if (isNaN(date.getTime())) return null;
const now = new Date();
const diffMs = now.getTime() - date.getTime();
const diffDays = Math.floor(diffMs / (1000 * 60 * 60 * 24));
if (diffDays === 0) return 'Today';
if (diffDays === 1) return '1 day';
return `${diffDays} days`;
} catch {
return null;
}
};
const rows: Array<{ label: string; value: string }> = [];
// Add publish date first if available
if (publishDate) {
const relativeTime = formatRelativeTime(publishDate);
if (relativeTime) {
rows.push({ label: 'Added', value: relativeTime });
}
}
// Add torznab attributes if available (MAM, etc.)
if (torznabAttrs && Object.keys(torznabAttrs).length > 0) {
const displayAttrs: Array<{ key: string; label: string }> = [
{ key: 'description', label: 'Description' },
{ key: 'year', label: 'Year' },
{ key: 'genre', label: 'Genre' },
{ key: 'narrator', label: 'Narrator' },
{ key: 'bitrate', label: 'Bitrate' },
{ key: 'samplerate', label: 'Sample Rate' },
{ key: 'runtime', label: 'Runtime' },
{ key: 'pages', label: 'Pages' },
{ key: 'publisher', label: 'Publisher' },
{ key: 'language', label: 'Language' },
];
for (const attr of displayAttrs) {
const val = torznabAttrs[attr.key];
if (val && val.trim()) {
rows.push({ label: attr.label, value: val.trim() });
}
}
}
// Add files and grabs from extra
const files = extra?.files as number | undefined;
const grabs = extra?.grabs as number | undefined;
if (files !== undefined && files !== null) {
rows.push({ label: 'Files', value: String(files) });
}
if (grabs !== undefined && grabs !== null) {
rows.push({ label: 'Grabs', value: String(grabs) });
}
let sizeTooltipContent: React.ReactNode = null;
if (rows.length > 0) {
sizeTooltipContent = (
<div className="flex flex-col gap-1 max-w-xs">
{rows.map((row) => (
<div key={row.label} className="flex gap-2">
<span className="text-gray-400 dark:text-gray-500 flex-shrink-0">{row.label}:</span>
<span className="truncate">{row.value}</span>
</div>
))}
</div>
);
}
if (compact) {
return <span>{displayValue}</span>;
}
const sizeText = <span>{displayValue}</span>;
return (
<div className={`flex items-center ${alignClass} text-xs text-gray-600 dark:text-gray-300`}>
{displayValue}
{sizeTooltipContent ? (
<Tooltip content={sizeTooltipContent} position="left">
{sizeText}
</Tooltip>
) : (
sizeText
)}
</div>
);
}
case 'peers': {
// Peers display: "S/L" string with badge colored by seeder count
@@ -101,6 +298,164 @@ export const ReleaseCell = ({ column, release, compact = false, onlineServers }:
);
}
case 'indexer_protocol': {
// Indexer name with colored dot indicating protocol (torrent/usenet) and peers count
const protocol = release.protocol as string | undefined;
const dotColor = getProtocolDotColor(protocol);
const protocolLabel = protocol === 'torrent' ? 'Torrent' : protocol === 'nzb' ? 'Usenet' : protocol || 'Unknown';
const peers = release.peers;
if (compact) {
return (
<span className="inline-flex items-center gap-1">
<span
className={`w-1.5 h-1.5 rounded-full flex-shrink-0 ${dotColor}`}
title={protocolLabel}
/>
{displayValue}
{peers && <span className="text-gray-400">({peers})</span>}
</span>
);
}
return (
<div className={`flex items-center ${alignClass} text-xs text-gray-600 dark:text-gray-300 truncate gap-1.5`}>
<span
className={`w-2 h-2 rounded-full flex-shrink-0 ${dotColor}`}
title={protocolLabel}
/>
<span className="truncate">{displayValue}</span>
{peers && <span className="text-gray-400 dark:text-gray-500 flex-shrink-0">{peers}</span>}
</div>
);
}
case 'flag_icon': {
// Colored badge showing FL, VIP, or both
if (!value || value === column.fallback) {
if (compact) return null;
return <div className={`flex items-center ${alignClass}`} />;
}
const flagColor = getColorStyleFromHint(value, { type: 'map', value: 'flags' });
if (compact) {
return <span className={`${flagColor.text} font-medium`}>{value}</span>;
}
return (
<div className={`flex items-center ${alignClass}`}>
<span className={`${flagColor.bg} ${flagColor.text} text-[10px] sm:text-[11px] font-semibold px-1.5 sm:px-2 py-0.5 rounded-lg tracking-wide whitespace-nowrap`}>
{value}
</span>
</div>
);
}
case 'format_content_type': {
// Content type icon + format badge combined
// Shows primary format as badge with colored dots for additional formats
const contentType = release.content_type as string | undefined;
const isAudiobook = contentType === 'audiobook';
const formats = (release.extra as Record<string, unknown> | undefined)?.formats as string[] | undefined;
const primaryFormat = formats?.[0] || null;
const additionalFormats = formats?.slice(1) || [];
// Use blue for book, violet for audiobook when no format specified
const noFormatStyle = isAudiobook
? { bg: 'bg-violet-500/20', text: 'text-violet-600 dark:text-violet-400' }
: { bg: 'bg-blue-500/20', text: 'text-blue-600 dark:text-blue-400' };
const colorStyle = primaryFormat ? getFormatColor(primaryFormat) : noFormatStyle;
// Build rich tooltip content for formats (if multiple)
let tooltipContent: React.ReactNode = null;
if (formats && formats.length > 1) {
tooltipContent = (
<div className="flex flex-wrap gap-1.5">
{formats.map((fmt) => {
const fmtColor = getFormatColor(String(fmt));
return (
<span
key={String(fmt)}
className={`${fmtColor.bg} ${fmtColor.text} text-[10px] sm:text-[11px] font-semibold px-1.5 sm:px-2 py-0.5 rounded-lg tracking-wide`}
>
{String(fmt).toUpperCase()}
</span>
);
})}
</div>
);
}
// Icon sized to match visual height of format text badges
const icon = isAudiobook ? (
// Headphones icon for audiobook
<svg className="w-4 h-4 flex-shrink-0" fill="none" stroke="currentColor" viewBox="0 0 24 24" strokeWidth={2}>
<path strokeLinecap="round" strokeLinejoin="round" d="M19.114 5.636a9 9 0 0 1 0 12.728M16.463 8.288a5.25 5.25 0 0 1 0 7.424M6.75 8.25l4.72-4.72a.75.75 0 0 1 1.28.53v15.88a.75.75 0 0 1-1.28.53l-4.72-4.72H4.51c-.88 0-1.704-.507-1.938-1.354A9.009 9.009 0 0 1 2.25 12c0-.83.112-1.633.322-2.396C2.806 8.756 3.63 8.25 4.51 8.25H6.75Z" />
</svg>
) : (
// Book icon for ebook
<svg className="w-4 h-4 flex-shrink-0" fill="none" stroke="currentColor" viewBox="0 0 24 24" strokeWidth={2}>
<path strokeLinecap="round" strokeLinejoin="round" d="M12 6.042A8.967 8.967 0 0 0 6 3.75c-1.052 0-2.062.18-3 .512v14.25A8.987 8.987 0 0 1 6 18c2.305 0 4.408.867 6 2.292m0-14.25a8.966 8.966 0 0 1 6-2.292c1.052 0 2.062.18 3 .512v14.25A8.987 8.987 0 0 0 18 18a8.967 8.967 0 0 0-6 2.292m0-14.25v14.25" />
</svg>
);
if (compact) {
if (!primaryFormat) {
return <span className="inline-flex items-center text-gray-500" title={isAudiobook ? 'Audiobook' : 'Book'}>{icon}</span>;
}
// Simple text tooltip for compact mode
const compactTooltip = formats && formats.length > 1
? formats.map((fmt) => String(fmt).toUpperCase()).join(', ')
: undefined;
return (
<span className={column.uppercase ? 'uppercase' : ''} title={compactTooltip}>
{primaryFormat.toUpperCase()}
{additionalFormats.length > 0 && ` +${additionalFormats.length}`}
</span>
);
}
// No format - just show icon with same width as format badges
if (!primaryFormat) {
return (
<div className="flex items-center justify-start" title={isAudiobook ? 'Audiobook' : 'Book'}>
<span className={`${colorStyle.bg} ${colorStyle.text} text-[10px] sm:text-[11px] font-semibold py-0.5 rounded-lg inline-flex items-center justify-center w-[3.25rem]`}>
{icon}
</span>
</div>
);
}
// Format badge - left-aligned so primary format stays in place, +N appears to right
const formatBadge = (
<span className="inline-flex items-center gap-1">
<span
className={`${colorStyle.bg} ${colorStyle.text} text-[10px] sm:text-[11px] font-semibold py-0.5 rounded-lg tracking-wide whitespace-nowrap w-[3.25rem] text-center`}
>
{column.uppercase ? primaryFormat.toUpperCase() : primaryFormat}
</span>
{additionalFormats.length > 0 && (
<span
className="bg-gray-500/20 text-gray-700 dark:text-gray-300 text-[10px] sm:text-[11px] font-semibold px-1.5 sm:px-2 py-0.5 rounded-lg tracking-wide"
>
+{additionalFormats.length}
</span>
)}
</span>
);
return (
<div className="flex items-center justify-start">
{tooltipContent ? (
<Tooltip content={tooltipContent} position="top">
{formatBadge}
</Tooltip>
) : (
formatBadge
)}
</div>
);
}
case 'number':
if (compact) {
return <span>{displayValue}</span>;
+251 -142
View File
@@ -9,7 +9,7 @@ import { ReleaseCell } from './ReleaseCell';
import { getColorStyleFromHint } from '../utils/colorMaps';
import { getNestedValue } from '../utils/objectHelpers';
import { LanguageMultiSelect } from './LanguageMultiSelect';
import { LANGUAGE_OPTION_ALL, LANGUAGE_OPTION_DEFAULT, getLanguageFilterValues, releaseLanguageMatchesFilter } from '../utils/languageFilters';
import { LANGUAGE_OPTION_ALL, LANGUAGE_OPTION_DEFAULT, getLanguageFilterValues, releaseLanguageMatchesFilter, buildLanguageNormalizer } from '../utils/languageFilters';
// Module-level cache for release search results
// Key format: `${provider}:${provider_id}:${source}:${contentType}`
@@ -426,12 +426,38 @@ const ReleaseRow = ({
{/* Plugin-provided info line (format, size, indexer, seeders, etc.) */}
{mobileColumns.length > 0 && (
<div className="flex items-center gap-1.5 mt-1 text-[10px] text-gray-500 dark:text-gray-400">
{mobileColumns.map((col, idx) => (
<span key={col.key} className="flex items-center gap-1.5">
{idx > 0 && <span className="text-gray-300 dark:text-gray-600">·</span>}
<ReleaseCell column={col} release={release} compact onlineServers={onlineServers} />
</span>
))}
{(() => {
// Pre-filter columns that will render content to avoid orphan dots
const columnsWithContent = mobileColumns.filter((col) => {
const rawValue = getNestedValue(release as unknown as Record<string, unknown>, col.key);
const value = rawValue !== undefined && rawValue !== null ? String(rawValue) : col.fallback;
if (col.render_type === 'flag_icon') {
// flag_icon returns null in compact mode when empty
if (!value || value === col.fallback) {
return false;
}
}
if (col.render_type === 'tags') {
// tags render null in compact mode when empty and no fallback
if (Array.isArray(rawValue)) {
return rawValue.some((tag) => String(tag).trim()) || Boolean(col.fallback);
}
if (!value || value === col.fallback) {
return Boolean(col.fallback);
}
}
return true;
});
return columnsWithContent.map((col, idx) => (
<span key={col.key} className="flex items-center gap-1.5">
{idx > 0 && <span className="text-gray-300 dark:text-gray-600">·</span>}
<ReleaseCell column={col} release={release} compact onlineServers={onlineServers} />
</span>
));
})()}
</div>
)}
</div>
@@ -622,6 +648,10 @@ export const ReleaseModal = ({
// A specific value means "show only that format"
const [formatFilter, setFormatFilter] = useState<string>('');
const [languageFilter, setLanguageFilter] = useState<string[]>([LANGUAGE_OPTION_DEFAULT]);
// Indexer filter - empty array means "show all", otherwise show only selected indexers
const [indexerFilter, setIndexerFilter] = useState<string[]>([]);
// Track which tabs have had indexer filter initialized (to avoid overriding user changes)
const indexerFilterInitializedRef = useRef<Set<string>>(new Set());
const [manualQuery, setManualQuery] = useState<string>('');
const [showManualQuery, setShowManualQuery] = useState<boolean>(false);
@@ -673,6 +703,8 @@ export const ReleaseModal = ({
setExpandedBySource({});
setFormatFilter('');
setLanguageFilter([LANGUAGE_OPTION_DEFAULT]);
setIndexerFilter([]);
indexerFilterInitializedRef.current = new Set();
setManualQuery('');
setShowManualQuery(false);
setSearchStatus(null);
@@ -739,6 +771,22 @@ export const ReleaseModal = ({
}
}, [loadingBySource, activeTab]);
// Initialize indexer filter from default_indexers when results first load for a tab
useEffect(() => {
const response = releasesBySource[activeTab];
if (!response?.column_config) return;
// Only initialize once per tab per book
if (indexerFilterInitializedRef.current.has(activeTab)) return;
const defaultIndexers = response.column_config.default_indexers;
if (defaultIndexers && defaultIndexers.length > 0) {
setIndexerFilter(defaultIndexers);
}
// Mark as initialized even if no default_indexers (to avoid re-checking)
indexerFilterInitializedRef.current.add(activeTab);
}, [releasesBySource, activeTab]);
// Check if description text overflows (needs "more" button)
useEffect(() => {
const el = descriptionRef.current;
@@ -881,9 +929,13 @@ export const ReleaseModal = ({
? undefined
: langCodes;
// Pass indexer filter only if the source supports it (empty array = search all)
const supportsIndexerFilter = releasesBySource[activeTab]?.column_config?.supported_filters?.includes('indexer');
const indexersParam = supportsIndexerFilter && indexerFilter.length > 0 ? indexerFilter : undefined;
// Fetch with expand_search=true (title+author search)
const expandedResponse = await getReleases(
provider, bookId, activeTab, book.title, book.author, true, languagesParam, contentType, manualQuery.trim() || undefined
provider, bookId, activeTab, book.title, book.author, true, languagesParam, contentType, manualQuery.trim() || undefined, indexersParam
);
// Merge with existing results, deduplicating by source_id
@@ -910,7 +962,7 @@ export const ReleaseModal = ({
} finally {
setLoadingBySource((prev) => ({ ...prev, [activeTab]: false }));
}
}, [activeTab, book, languageFilter, bookLanguages, defaultLanguages, contentType, manualQuery]);
}, [activeTab, book, languageFilter, bookLanguages, defaultLanguages, contentType, manualQuery, indexerFilter, releasesBySource]);
// Build list of tabs to show
// Only show enabled sources that support the current content type
@@ -992,11 +1044,36 @@ export const ReleaseModal = ({
return options;
}, [availableFormats]);
// Get available indexers for filter dropdown
// Prefer available_indexers from column config (all enabled Prowlarr indexers),
// fall back to unique indexers from results if not provided
const availableIndexers = useMemo(() => {
// Use column config's available_indexers if provided (e.g., from Prowlarr)
const configIndexers = releasesBySource[activeTab]?.column_config?.available_indexers;
if (configIndexers && configIndexers.length > 0) {
return configIndexers;
}
// Fall back to indexers found in results
const releases = releasesBySource[activeTab]?.releases || [];
const indexers = new Set<string>();
releases.forEach((r) => {
if (r.indexer) {
indexers.add(r.indexer);
}
});
return Array.from(indexers).sort();
}, [releasesBySource, activeTab]);
// Resolve language filter to actual language codes for filtering
const resolvedLanguageCodes = useMemo(() => {
return getLanguageFilterValues(languageFilter, bookLanguages, defaultLanguages);
}, [languageFilter, bookLanguages, defaultLanguages]);
// Build language normalizer for release filtering (handles both codes like "en" and names like "English")
const languageNormalizer = useMemo(() => {
return buildLanguageNormalizer(bookLanguages);
}, [bookLanguages]);
// Get column config from response or use default (moved before filteredReleases for sorting)
const columnConfig = useMemo((): ReleaseColumnConfig => {
const response = releasesBySource[activeTab];
@@ -1073,6 +1150,7 @@ export const ReleaseModal = ({
const filteredReleases = useMemo(() => {
const releases = releasesBySource[activeTab]?.releases || [];
const effectiveLower = effectiveFormats.map((f) => f.toLowerCase());
const supportsIndexerFilter = columnConfig.supported_filters?.includes('indexer');
// First, filter
let filtered = releases.filter((r) => {
@@ -1088,12 +1166,20 @@ export const ReleaseModal = ({
}
// Releases with no format pass through when no filter is set (show all)
// Language filtering
const releaseLang = r.extra?.language as string | undefined;
if (!releaseLanguageMatchesFilter(releaseLang, resolvedLanguageCodes ?? defaultLanguages)) {
// Language filtering - use r.language when provided by enriched indexers
// Releases with no language (null/undefined) always pass
const releaseLang = r.language as string | undefined;
if (!releaseLanguageMatchesFilter(releaseLang, resolvedLanguageCodes ?? defaultLanguages, languageNormalizer)) {
return false;
}
// Indexer filtering - empty array means show all
if (supportsIndexerFilter && indexerFilter.length > 0 && r.indexer) {
if (!indexerFilter.includes(r.indexer)) {
return false;
}
}
return true;
});
@@ -1103,7 +1189,7 @@ export const ReleaseModal = ({
}
return filtered;
}, [releasesBySource, activeTab, formatFilter, resolvedLanguageCodes, effectiveFormats, defaultLanguages, currentSort, sortableColumns]);
}, [releasesBySource, activeTab, formatFilter, resolvedLanguageCodes, effectiveFormats, defaultLanguages, languageNormalizer, indexerFilter, currentSort, sortableColumns, columnConfig]);
// Pre-compute display field lookups to avoid repeated .find() calls in JSX
const displayFields = useMemo(() => {
@@ -1137,6 +1223,9 @@ export const ReleaseModal = ({
progress: book.progress,
};
}
if (currentStatus.locating && currentStatus.locating[releaseId]) {
return { text: 'Locating files', state: 'locating' };
}
if (currentStatus.resolving && currentStatus.resolving[releaseId]) {
return { text: 'Resolving', state: 'resolving' };
}
@@ -1248,8 +1337,8 @@ export const ReleaseModal = ({
</div>
</header>
{/* Scrollable content */}
<div ref={scrollContainerRef} className="flex-1 min-h-0 overflow-y-auto">
{/* Scrollable content */}
<div ref={scrollContainerRef} className="flex-1 min-h-0 overflow-y-auto">
{/* Book summary - scrolls with content */}
<div ref={bookSummaryRef} className="flex gap-4 px-5 py-4 border-b border-[var(--border-muted)]">
{book.preview ? (
@@ -1391,18 +1480,17 @@ export const ReleaseModal = ({
}}
/>
{allTabs.map((tab) => (
<button
key={tab.name}
ref={(el) => { tabRefs.current[tab.name] = el; }}
onClick={() => setActiveTab(tab.name)}
className={`px-4 py-2.5 text-sm font-medium border-b-2 border-transparent transition-colors whitespace-nowrap ${
activeTab === tab.name
? 'text-emerald-600 dark:text-emerald-400'
: 'text-gray-600 dark:text-gray-400 hover:text-gray-900 dark:hover:text-gray-200'
<button
key={tab.name}
ref={(el) => { tabRefs.current[tab.name] = el; }}
onClick={() => setActiveTab(tab.name)}
className={`px-4 py-2.5 text-sm font-medium border-b-2 border-transparent transition-colors whitespace-nowrap ${activeTab === tab.name
? 'text-emerald-600 dark:text-emerald-400'
: 'text-gray-600 dark:text-gray-400 hover:text-gray-900 dark:hover:text-gray-200'
}`}
>
{tab.displayName}
</button>
>
{tab.displayName}
</button>
))}
</div>
</div>
@@ -1423,9 +1511,8 @@ export const ReleaseModal = ({
return next;
});
}}
className={`p-2.5 rounded-full transition-colors hover-surface text-gray-500 dark:text-gray-400 ${
manualQuery.trim() ? 'text-emerald-600 dark:text-emerald-400' : ''
}`}
className={`p-2.5 rounded-full transition-colors hover-surface text-gray-500 dark:text-gray-400 ${manualQuery.trim() ? 'text-emerald-600 dark:text-emerald-400' : ''
}`}
aria-label="Manual search query"
title="Manual query"
>
@@ -1444,9 +1531,8 @@ export const ReleaseModal = ({
<button
type="button"
onClick={toggle}
className={`relative p-2.5 rounded-full transition-colors hover-surface text-gray-500 dark:text-gray-400 ${
isOpen ? 'bg-[var(--hover-surface)]' : ''
}`}
className={`relative p-2.5 rounded-full transition-colors hover-surface text-gray-500 dark:text-gray-400 ${isOpen ? 'bg-[var(--hover-surface)]' : ''
}`}
aria-label="Sort releases"
>
<svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24" strokeWidth={1.5}>
@@ -1467,11 +1553,10 @@ export const ReleaseModal = ({
handleSortChange(null, null);
close();
}}
className={`w-full px-3 py-2 text-left text-sm flex items-center justify-between hover-surface rounded ${
!currentSort
className={`w-full px-3 py-2 text-left text-sm flex items-center justify-between hover-surface rounded ${!currentSort
? 'text-emerald-600 dark:text-emerald-400 font-medium'
: 'text-gray-700 dark:text-gray-300'
}`}
}`}
>
<span>Default</span>
{!currentSort && (
@@ -1493,11 +1578,10 @@ export const ReleaseModal = ({
// Don't close - allow toggling direction
if (!isSelected) close();
}}
className={`w-full px-3 py-2 text-left text-sm flex items-center justify-between hover-surface rounded ${
isSelected
className={`w-full px-3 py-2 text-left text-sm flex items-center justify-between hover-surface rounded ${isSelected
? 'text-emerald-600 dark:text-emerald-400 font-medium'
: 'text-gray-700 dark:text-gray-300'
}`}
}`}
>
<span>{col.label}</span>
{isSelected && direction && (
@@ -1520,16 +1604,28 @@ export const ReleaseModal = ({
{/* Filter funnel button - stays fixed */}
{/* Only show filter button if source supports at least one filter type */}
{((columnConfig.supported_filters?.includes('format') && availableFormats.length > 0) ||
(columnConfig.supported_filters?.includes('language') && bookLanguages.length > 0)) && (
(columnConfig.supported_filters?.includes('language') && bookLanguages.length > 0) ||
(columnConfig.supported_filters?.includes('indexer') && availableIndexers.length > 1)) && (
<Dropdown
align="right"
widthClassName="w-auto flex-shrink-0"
panelClassName="w-56"
noScrollLimit
renderTrigger={({ isOpen, toggle }) => {
// Active filter: format is set, or language is not just default
// Active filter: format is set, language is not default, or indexers differ from defaults
const hasLanguageFilter = !(languageFilter.length === 1 && languageFilter[0] === LANGUAGE_OPTION_DEFAULT);
const hasActiveFilter = formatFilter !== '' || hasLanguageFilter;
const supportsIndexerFilter = columnConfig.supported_filters?.includes('indexer');
// Check if indexer filter differs from defaults (only after initialization)
// Don't show dot while loading or before filter is initialized from defaults
const hasResults = releasesBySource[activeTab]?.releases !== undefined;
const isInitialized = indexerFilterInitializedRef.current.has(activeTab);
const defaultIndexers = columnConfig.default_indexers ?? [];
const indexersMatchDefault = (
indexerFilter.length === defaultIndexers.length &&
indexerFilter.every((idx) => defaultIndexers.includes(idx))
);
const hasIndexerFilter = supportsIndexerFilter && hasResults && isInitialized && !indexersMatchDefault;
const hasActiveFilter = formatFilter !== '' || hasLanguageFilter || hasIndexerFilter;
return (
<button
type="button"
@@ -1549,83 +1645,97 @@ export const ReleaseModal = ({
);
}}
>
{({ close }) => (
<div className="p-4 space-y-4">
{columnConfig.supported_filters?.includes('format') && availableFormats.length > 0 && (
<DropdownList
label="Format"
options={formatOptions}
value={formatFilter}
onChange={(val) => setFormatFilter(typeof val === 'string' ? val : val[0] ?? '')}
placeholder="All Formats"
/>
)}
{columnConfig.supported_filters?.includes('language') && (
<LanguageMultiSelect
label="Language"
options={bookLanguages}
value={languageFilter}
onChange={setLanguageFilter}
defaultLanguageCodes={defaultLanguages}
/>
)}
{/* Apply button - re-fetch with server-side filters/expansion (e.g. language-aware searches) */}
{(activeTab === 'direct_download' || activeTab === 'prowlarr') && (
<button
type="button"
onClick={async () => {
close();
if (!book?.provider || !book?.provider_id) return;
{({ close }) => (
<div className="p-4 space-y-4">
{columnConfig.supported_filters?.includes('format') && availableFormats.length > 0 && (
<DropdownList
label="Format"
options={formatOptions}
value={formatFilter}
onChange={(val) => setFormatFilter(typeof val === 'string' ? val : val[0] ?? '')}
placeholder="All Formats"
/>
)}
{columnConfig.supported_filters?.includes('language') && (
<LanguageMultiSelect
label="Language"
options={bookLanguages}
value={languageFilter}
onChange={setLanguageFilter}
defaultLanguageCodes={defaultLanguages}
/>
)}
{columnConfig.supported_filters?.includes('indexer') && availableIndexers.length > 1 && (
<DropdownList
label="Indexers"
options={availableIndexers.map((idx) => ({ value: idx, label: idx }))}
multiple
value={indexerFilter}
onChange={(val) => setIndexerFilter(Array.isArray(val) ? val : val ? [val] : [])}
placeholder="All Indexers"
/>
)}
{/* Apply button - re-fetch with server-side filters/expansion (e.g. language-aware searches) */}
{(activeTab === 'direct_download' || activeTab === 'prowlarr') && (
<button
type="button"
onClick={async () => {
close();
if (!book?.provider || !book?.provider_id) return;
const provider = book.provider;
const bookId = book.provider_id;
const provider = book.provider;
const bookId = book.provider_id;
// Clear cache and state
const key = getCacheKey(provider, bookId, activeTab, contentType);
releaseCache.delete(key);
cacheTimestamps.delete(key);
setExpandedBySource((prev) => {
const next = { ...prev };
delete next[activeTab];
return next;
});
setErrorBySource((prev) => {
const next = { ...prev };
delete next[activeTab];
return next;
});
// Clear cache and state
const key = getCacheKey(provider, bookId, activeTab, contentType);
releaseCache.delete(key);
cacheTimestamps.delete(key);
setExpandedBySource((prev) => {
const next = { ...prev };
delete next[activeTab];
return next;
});
setErrorBySource((prev) => {
const next = { ...prev };
delete next[activeTab];
return next;
});
// Fetch with language filter
setLoadingBySource((prev) => ({ ...prev, [activeTab]: true }));
try {
// Resolve language codes for the API call
const langCodes = getLanguageFilterValues(languageFilter, bookLanguages, defaultLanguages);
// Don't pass languages if "All" is selected or null
const languagesParam = (langCodes === null || langCodes?.includes(LANGUAGE_OPTION_ALL))
? undefined
: langCodes;
// Fetch with language filter
setLoadingBySource((prev) => ({ ...prev, [activeTab]: true }));
try {
// Resolve language codes for the API call
const langCodes = getLanguageFilterValues(languageFilter, bookLanguages, defaultLanguages);
// Don't pass languages if "All" is selected or null
const languagesParam = (langCodes === null || langCodes?.includes(LANGUAGE_OPTION_ALL))
? undefined
: langCodes;
const response = await getReleases(
provider, bookId, activeTab, book.title, book.author, false, languagesParam, contentType, manualQuery.trim() || undefined
);
setCachedReleases(provider, bookId, activeTab, contentType, response);
setReleasesBySource((prev) => ({ ...prev, [activeTab]: response }));
} catch (err) {
const message = err instanceof Error ? err.message : 'Failed to fetch releases';
setErrorBySource((prev) => ({ ...prev, [activeTab]: message }));
} finally {
setLoadingBySource((prev) => ({ ...prev, [activeTab]: false }));
}
}}
className="w-full px-3 py-2 text-sm font-medium text-white bg-emerald-600 hover:bg-emerald-700 rounded-lg transition-colors"
>
Apply
</button>
)}
</div>
)}
</Dropdown>
)}
// Pass indexer filter only if the source supports it (empty array = search all)
const supportsIndexerFilter = columnConfig.supported_filters?.includes('indexer');
const indexersParam = supportsIndexerFilter && indexerFilter.length > 0 ? indexerFilter : undefined;
const response = await getReleases(
provider, bookId, activeTab, book.title, book.author, false, languagesParam, contentType, manualQuery.trim() || undefined, indexersParam
);
setCachedReleases(provider, bookId, activeTab, contentType, response);
setReleasesBySource((prev) => ({ ...prev, [activeTab]: response }));
} catch (err) {
const message = err instanceof Error ? err.message : 'Failed to fetch releases';
setErrorBySource((prev) => ({ ...prev, [activeTab]: message }));
} finally {
setLoadingBySource((prev) => ({ ...prev, [activeTab]: false }));
}
}}
className="w-full px-3 py-2 text-sm font-medium text-white bg-emerald-600 hover:bg-emerald-700 rounded-lg transition-colors"
>
Apply
</button>
)}
</div>
)}
</Dropdown>
)}
</div>
</div>
)}
@@ -1691,11 +1801,10 @@ export const ReleaseModal = ({
<button
type="submit"
disabled={currentTabLoading || !manualQuery.trim()}
className={`px-3 py-2 text-sm font-medium text-white rounded-lg transition-colors ${
currentTabLoading || !manualQuery.trim()
className={`px-3 py-2 text-sm font-medium text-white rounded-lg transition-colors ${currentTabLoading || !manualQuery.trim()
? 'bg-emerald-600/60 cursor-not-allowed'
: 'bg-emerald-600 hover:bg-emerald-700'
}`}
}`}
>
{currentTabLoading ? 'Searching…' : 'Search'}
</button>
@@ -1731,16 +1840,16 @@ export const ReleaseModal = ({
releasesBySource[activeTab]?.search_info?.[activeTab]?.search_type ?? ''
)
)) && (
<div className="py-3 text-center">
<button
type="button"
onClick={handleExpandSearch}
className="px-3 py-1.5 text-sm text-gray-500 dark:text-gray-400 rounded-full hover-action transition-all duration-200"
>
{columnConfig.action_button?.label ?? 'Expand search'}
</button>
</div>
)}
<div className="py-3 text-center">
<button
type="button"
onClick={handleExpandSearch}
className="px-3 py-1.5 text-sm text-gray-500 dark:text-gray-400 rounded-full hover-action transition-all duration-200"
>
{columnConfig.action_button?.label ?? 'Expand search'}
</button>
</div>
)}
</>
) : (
<>
@@ -1768,22 +1877,22 @@ export const ReleaseModal = ({
releasesBySource[activeTab]?.search_info?.[activeTab]?.search_type ?? ''
)
)) && (
<div
className="py-3 text-center animate-pop-up will-change-transform"
style={{
animationDelay: `${filteredReleases.length * 30}ms`,
animationFillMode: 'both',
}}
>
<button
type="button"
onClick={handleExpandSearch}
className="px-3 py-1.5 text-sm text-gray-500 dark:text-gray-400 rounded-full hover-action transition-all duration-200"
<div
className="py-3 text-center animate-pop-up will-change-transform"
style={{
animationDelay: `${filteredReleases.length * 30}ms`,
animationFillMode: 'both',
}}
>
{columnConfig.action_button?.label ?? 'Expand search'}
</button>
</div>
)}
<button
type="button"
onClick={handleExpandSearch}
className="px-3 py-1.5 text-sm text-gray-500 dark:text-gray-400 rounded-full hover-action transition-all duration-200"
>
{columnConfig.action_button?.label ?? 'Expand search'}
</button>
</div>
)}
{/* Expanding search - show skeleton below existing results */}
{currentTabLoading && filteredReleases.length > 0 && (
<ReleaseSkeleton />
@@ -9,6 +9,7 @@ import {
CheckboxFieldConfig,
SelectFieldConfig,
MultiSelectFieldConfig,
TagListFieldConfig,
OrderableListFieldConfig,
OrderableListItem,
ActionButtonConfig,
@@ -24,6 +25,7 @@ import {
CheckboxField,
SelectField,
MultiSelectField,
TagListField,
OrderableListField,
ActionButton,
HeadingField,
@@ -198,6 +200,15 @@ const renderField = (
disabled={isDisabled}
/>
);
case 'TagListField':
return (
<TagListField
field={field as TagListFieldConfig}
value={(value as string[]) ?? []}
onChange={(v) => onChange(v)}
disabled={isDisabled}
/>
);
case 'OrderableListField':
return (
<OrderableListField
@@ -0,0 +1,149 @@
import { useMemo, useRef, useState } from 'react';
import { TagListFieldConfig } from '../../../types/settings';
interface TagListFieldProps {
field: TagListFieldConfig;
value: string[];
onChange: (value: string[]) => void;
disabled?: boolean;
}
function normalizeTag(raw: string): string {
let s = raw.trim();
if (!s) return '';
// Strip wrapping quotes from pasted env/JSON style strings.
if (
(s.startsWith('"') && s.endsWith('"')) ||
(s.startsWith("'") && s.endsWith("'"))
) {
s = s.slice(1, -1).trim();
}
// Avoid special sentinel values.
if (s.toLowerCase() === 'auto') return '';
// Basic URL normalization to keep UX friendly; backend also normalizes on save.
if (!s.includes('://') && !s.startsWith('/')) {
s = `https://${s}`;
}
s = s.replace(/\/+$/, '');
return s.trim();
}
export const TagListField = ({ field, value, onChange, disabled }: TagListFieldProps) => {
const isDisabled = disabled ?? false;
const inputRef = useRef<HTMLInputElement>(null);
const [draft, setDraft] = useState('');
const tags = useMemo(() => (value ?? []).map(String).filter((t) => t.trim() !== ''), [value]);
const addFromRaw = (raw: string) => {
if (isDisabled) return;
const parts = raw
.split(/[\n,]+/)
.map((p) => p.trim())
.filter(Boolean);
if (parts.length === 0) return;
const next = [...tags];
for (const part of parts) {
const normalized = normalizeTag(part);
if (!normalized) continue;
if (next.includes(normalized)) continue;
next.push(normalized);
}
if (next.length !== tags.length) {
onChange(next);
}
};
const removeAt = (idx: number) => {
if (isDisabled) return;
onChange(tags.filter((_, i) => i !== idx));
};
const commitDraft = () => {
const raw = draft;
if (!raw.trim()) return;
addFromRaw(raw);
setDraft('');
};
return (
<div
className={`w-full px-3 py-2 rounded-lg border border-[var(--border-muted)]
bg-[var(--bg-soft)] text-sm
${isDisabled ? 'opacity-60 cursor-not-allowed' : 'cursor-text'}`}
onClick={() => {
if (isDisabled) return;
inputRef.current?.focus();
}}
>
<div className="flex flex-wrap gap-2 items-center">
{tags.map((tag, idx) => (
<span
key={`${tag}-${idx}`}
className="inline-flex items-center gap-1 px-3 py-1.5 rounded-full
border border-[var(--border-muted)] bg-[var(--bg)]
max-w-full"
title={tag}
>
<span className="truncate max-w-[22rem]">{tag}</span>
{!isDisabled && (
<button
type="button"
onClick={(e) => {
e.stopPropagation();
removeAt(idx);
}}
className="p-0.5 rounded-full hover:bg-[var(--hover-surface)]"
aria-label={`Remove ${tag}`}
>
<svg
className="w-4 h-4"
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth={1.5}
stroke="currentColor"
>
<path strokeLinecap="round" strokeLinejoin="round" d="M6 18L18 6M6 6l12 12" />
</svg>
</button>
)}
</span>
))}
{!isDisabled && (
<input
ref={inputRef}
type="text"
value={draft}
onChange={(e) => setDraft(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') {
e.preventDefault();
commitDraft();
return;
}
if (e.key === 'Backspace' && !draft && tags.length > 0) {
removeAt(tags.length - 1);
}
}}
onBlur={() => commitDraft()}
placeholder={tags.length === 0 ? field.placeholder : ''}
className="flex-1 min-w-[12rem] bg-transparent outline-none py-1"
/>
)}
{isDisabled && tags.length === 0 && (
<span className="opacity-60">{field.placeholder || 'No values'}</span>
)}
</div>
</div>
);
};
@@ -4,6 +4,7 @@ export { NumberField } from './NumberField';
export { CheckboxField } from './CheckboxField';
export { SelectField } from './SelectField';
export { MultiSelectField } from './MultiSelectField';
export { TagListField } from './TagListField';
export { OrderableListField } from './OrderableListField';
export { ActionButton } from './ActionButton';
export { TableField } from './TableField';
@@ -0,0 +1,115 @@
import { useState, useRef, useEffect, ReactNode } from 'react';
import { createPortal } from 'react-dom';
interface TooltipProps {
content: ReactNode;
children: ReactNode;
position?: 'top' | 'bottom' | 'left' | 'right';
delay?: number;
className?: string;
}
export function Tooltip({
content,
children,
position = 'top',
delay = 200,
className = '',
}: TooltipProps) {
const [isVisible, setIsVisible] = useState(false);
const [coords, setCoords] = useState<{ top: number; left: number } | null>(null);
const triggerRef = useRef<HTMLDivElement>(null);
const timeoutRef = useRef<NodeJS.Timeout | null>(null);
const showTooltip = () => {
if (timeoutRef.current) clearTimeout(timeoutRef.current);
timeoutRef.current = setTimeout(() => {
if (triggerRef.current) {
const rect = triggerRef.current.getBoundingClientRect();
let top = 0;
let left = 0;
switch (position) {
case 'top':
top = rect.top - 8;
left = rect.left + rect.width / 2;
break;
case 'bottom':
top = rect.bottom + 8;
left = rect.left + rect.width / 2;
break;
case 'left':
top = rect.top + rect.height / 2;
left = rect.left - 8;
break;
case 'right':
top = rect.top + rect.height / 2;
left = rect.right + 8;
break;
}
setCoords({ top, left });
setIsVisible(true);
}
}, delay);
};
const hideTooltip = () => {
if (timeoutRef.current) {
clearTimeout(timeoutRef.current);
timeoutRef.current = null;
}
setIsVisible(false);
setCoords(null);
};
useEffect(() => {
return () => {
if (timeoutRef.current) clearTimeout(timeoutRef.current);
};
}, []);
if (!content) {
return <>{children}</>;
}
// Transform classes to center tooltip relative to the anchor point
const transformClass = {
top: '-translate-x-1/2 -translate-y-full',
bottom: '-translate-x-1/2',
left: '-translate-x-full -translate-y-1/2',
right: '-translate-y-1/2',
}[position];
return (
<>
<div
ref={triggerRef}
onMouseEnter={showTooltip}
onMouseLeave={hideTooltip}
className="inline-flex"
>
{children}
</div>
{isVisible && coords && createPortal(
<div
role="tooltip"
className={`fixed z-[9999] px-3 py-2 text-xs rounded-lg shadow-lg
pointer-events-none ${transformClass} ${className}`}
style={{
top: coords.top,
left: coords.left,
background: 'var(--bg-soft)',
color: 'var(--text)',
border: '1px solid var(--border-muted)',
}}
>
{content}
</div>,
document.body
)}
</>
);
}
export default Tooltip;
@@ -49,6 +49,9 @@ export function useDownloadTracking(currentStatus: StatusData): UseDownloadTrack
progress: book.progress,
};
}
if (currentStatus.locating && currentStatus.locating[bookId]) {
return { text: 'Locating files', state: 'locating' };
}
if (currentStatus.resolving && currentStatus.resolving[bookId]) {
return { text: 'Resolving', state: 'resolving' };
}
@@ -84,6 +87,10 @@ export function useDownloadTracking(currentStatus: StatusData): UseDownloadTrack
};
}
if (currentStatus.locating && currentStatus.locating[releaseId]) {
return { text: 'Locating files', state: 'locating' };
}
if (currentStatus.resolving && currentStatus.resolving[releaseId]) {
return { text: 'Resolving', state: 'resolving' };
}
+2 -2
View File
@@ -201,7 +201,7 @@ export function useSearch(options: UseSearchOptions): UseSearchReturn {
} finally {
setIsSearching(false);
}
}, [showToast, setIsAuthenticated, authRequired, navigate, searchFieldValues, handleSearchError]);
}, [showToast, setIsAuthenticated, authRequired, navigate, searchFieldValues, handleSearchError, contentType]);
const handleResetSearch = useCallback((config: AppConfig | null) => {
setBooks([]);
@@ -257,7 +257,7 @@ export function useSearch(options: UseSearchOptions): UseSearchReturn {
} finally {
setIsLoadingMore(false);
}
}, [currentPage, hasMore, isLoadingMore, handleSearchError]);
}, [currentPage, hasMore, isLoadingMore, handleSearchError, contentType]);
const handleSortChange = useCallback((value: string, config: AppConfig | null) => {
updateAdvancedFilters({ sort: value });
+4 -2
View File
@@ -1,5 +1,6 @@
import { LoginForm } from '../components/LoginForm';
import { LoginCredentials } from '../types';
import { withBasePath } from '../utils/basePath';
interface LoginPageProps {
onLogin: (credentials: LoginCredentials) => void;
@@ -8,6 +9,8 @@ interface LoginPageProps {
}
export const LoginPage = ({ onLogin, error, isLoading }: LoginPageProps) => {
const logoUrl = withBasePath('/logo.png');
return (
<div
className="min-h-screen flex items-center justify-center px-4 py-8"
@@ -15,7 +18,7 @@ export const LoginPage = ({ onLogin, error, isLoading }: LoginPageProps) => {
>
<div className="w-full max-w-md">
<div className="text-center mb-8">
<img src="/logo.png" alt="Logo" className="mx-auto mb-6 w-20 h-20" />
<img src={logoUrl} alt="Logo" className="mx-auto mb-6 w-20 h-20" />
<h1 className="text-2xl font-semibold">Sign in to continue</h1>
</div>
<div
@@ -33,4 +36,3 @@ export const LoginPage = ({ onLogin, error, isLoading }: LoginPageProps) => {
);
};
+5 -1
View File
@@ -339,7 +339,8 @@ export const getReleases = async (
expandSearch?: boolean,
languages?: string[],
contentType?: string,
manualQuery?: string
manualQuery?: string,
indexers?: string[]
): Promise<ReleasesResponse> => {
const params = new URLSearchParams({
provider,
@@ -366,6 +367,9 @@ export const getReleases = async (
if (manualQuery) {
params.set('manual_query', manualQuery);
}
if (indexers && indexers.length > 0) {
params.set('indexers', indexers.join(','));
}
// Let the backend control timeouts for release searches (can be long-running).
return fetchJSON<ReleasesResponse>(`${API_BASE}/releases?${params.toString()}`, {}, null);
};
+7 -3
View File
@@ -54,6 +54,7 @@ export interface Book {
export interface StatusData {
queued?: Record<string, Book>;
resolving?: Record<string, Book>;
locating?: Record<string, Book>;
downloading?: Record<string, Book>;
complete?: Record<string, Book>;
error?: Record<string, Book>;
@@ -65,7 +66,7 @@ export interface ActiveDownloadsResponse {
}
// Button states
export type ButtonState = 'download' | 'queued' | 'resolving' | 'downloading' | 'complete' | 'error';
export type ButtonState = 'download' | 'queued' | 'resolving' | 'locating' | 'downloading' | 'complete' | 'error';
export interface ButtonStateInfo {
text: string;
@@ -204,7 +205,7 @@ export interface ReleaseSource {
}
// Column schema types for plugin-driven release list UI
export type ColumnRenderType = 'text' | 'badge' | 'size' | 'number' | 'peers';
export type ColumnRenderType = 'text' | 'badge' | 'tags' | 'size' | 'number' | 'peers' | 'indexer_protocol' | 'flag_icon' | 'format_content_type';
export type ColumnAlign = 'left' | 'center' | 'right';
export interface ColumnColorHint {
@@ -246,8 +247,10 @@ export interface ReleaseColumnConfig {
grid_template: string; // CSS grid-template-columns for dynamic section
leading_cell?: LeadingCellConfig; // Defaults to thumbnail from extra.preview
online_servers?: string[]; // For IRC: list of currently online server nicks
available_indexers?: string[]; // For Prowlarr: list of all enabled indexer names
default_indexers?: string[]; // For Prowlarr: indexers selected in settings (pre-selected in filter)
cache_ttl_seconds?: number; // How long to cache results (default: 300 = 5 min)
supported_filters?: string[]; // Which filters this source supports: ["format", "language"]
supported_filters?: string[]; // Which filters this source supports: ["format", "language", "indexer"]
action_button?: SourceActionButton; // Custom action button (replaces default expand search)
}
@@ -266,6 +269,7 @@ export interface Release {
indexer?: string; // Display name for the source/indexer
seeders?: number; // For torrents
peers?: string; // For torrents: "seeders/leechers" display string
content_type?: string; // "ebook", "audiobook", or "book"
extra?: Record<string, unknown>; // Source-specific metadata
}
+8
View File
@@ -7,6 +7,7 @@ export type FieldType =
| 'CheckboxField'
| 'SelectField'
| 'MultiSelectField'
| 'TagListField'
| 'OrderableListField'
| 'TableField'
| 'ActionButton'
@@ -93,6 +94,12 @@ export interface MultiSelectFieldConfig extends BaseField {
variant?: 'pills' | 'dropdown'; // 'pills' (default) or 'dropdown' for checkbox dropdown style
}
export interface TagListFieldConfig extends BaseField {
type: 'TagListField';
value: string[];
placeholder?: string;
}
// OrderableListField types - generic drag-and-drop reorderable list
export interface OrderableListItem {
id: string;
@@ -163,6 +170,7 @@ export type SettingsField =
| CheckboxFieldConfig
| SelectFieldConfig
| MultiSelectFieldConfig
| TagListFieldConfig
| OrderableListFieldConfig
| TableFieldConfig
| ActionButtonConfig
+11 -4
View File
@@ -30,16 +30,23 @@ const resolveBasePath = (): string => {
}
};
const BASE_PATH = normalizeBasePath(resolveBasePath());
// Lazy initialization to ensure DOM is ready when base path is resolved
let _basePath: string | null = null;
export const getBasePath = (): string => BASE_PATH;
export const getBasePath = (): string => {
if (_basePath === null) {
_basePath = normalizeBasePath(resolveBasePath());
}
return _basePath;
};
export const withBasePath = (path: string): string => {
const basePath = getBasePath();
const normalizedPath = path.startsWith('/') ? path : `/${path}`;
if (BASE_PATH === '/') {
if (basePath === '/') {
return normalizedPath;
}
return `${BASE_PATH}${normalizedPath}`;
return `${basePath}${normalizedPath}`;
};
export const getApiBase = (): string => withBasePath('/api');
+40 -1
View File
@@ -5,6 +5,7 @@ interface ColorStyle {
}
const FORMAT_COLORS: Record<string, ColorStyle> = {
// Ebook formats
pdf: { bg: 'bg-red-500/20', text: 'text-red-700 dark:text-red-300' },
epub: { bg: 'bg-green-500/20', text: 'text-green-700 dark:text-green-300' },
mobi: { bg: 'bg-blue-500/20', text: 'text-blue-700 dark:text-blue-300' },
@@ -14,6 +15,11 @@ const FORMAT_COLORS: Record<string, ColorStyle> = {
fb2: { bg: 'bg-teal-500/20', text: 'text-teal-700 dark:text-teal-300' },
cbr: { bg: 'bg-yellow-500/20', text: 'text-yellow-700 dark:text-yellow-300' },
cbz: { bg: 'bg-amber-500/20', text: 'text-amber-700 dark:text-amber-300' },
// Audiobook formats
m4b: { bg: 'bg-violet-500/20', text: 'text-violet-700 dark:text-violet-300' },
m4a: { bg: 'bg-violet-500/20', text: 'text-violet-700 dark:text-violet-300' },
mp3: { bg: 'bg-rose-500/20', text: 'text-rose-700 dark:text-rose-300' },
flac: { bg: 'bg-indigo-500/20', text: 'text-indigo-700 dark:text-indigo-300' },
};
const LANGUAGE_COLORS: Record<string, ColorStyle> = {
@@ -51,15 +57,30 @@ const CONTENT_TYPE_COLORS: Record<string, ColorStyle> = {
audiobook: { bg: 'bg-violet-500/20', text: 'text-violet-700 dark:text-violet-300' },
};
const FLAG_COLORS: Record<string, ColorStyle> = {
fl: { bg: 'bg-green-500/20', text: 'text-green-700 dark:text-green-300' },
freeleech: { bg: 'bg-green-500/20', text: 'text-green-700 dark:text-green-300' },
'double upload': { bg: 'bg-blue-500/20', text: 'text-blue-700 dark:text-blue-300' },
vip: { bg: 'bg-amber-500/20', text: 'text-amber-700 dark:text-amber-300' },
'vip fl': { bg: 'bg-amber-500/20', text: 'text-amber-700 dark:text-amber-300' },
'fl vip': { bg: 'bg-amber-500/20', text: 'text-amber-700 dark:text-amber-300' },
sticky: { bg: 'bg-yellow-500/20', text: 'text-yellow-700 dark:text-yellow-300' },
};
const DEFAULT_FORMAT_COLOR: ColorStyle = { bg: 'bg-cyan-500/20', text: 'text-cyan-700 dark:text-cyan-300' };
const DEFAULT_LANGUAGE_COLOR: ColorStyle = { bg: 'bg-indigo-500/20', text: 'text-indigo-700 dark:text-indigo-300' };
const DEFAULT_DOWNLOAD_TYPE_COLOR: ColorStyle = { bg: 'bg-violet-500/20', text: 'text-violet-700 dark:text-violet-300' };
const DEFAULT_CONTENT_TYPE_COLOR: ColorStyle = { bg: 'bg-gray-500/20', text: 'text-gray-700 dark:text-gray-300' };
const DEFAULT_FLAG_COLOR: ColorStyle = { bg: 'bg-gray-500/20', text: 'text-gray-700 dark:text-gray-300' };
const FALLBACK_COLOR: ColorStyle = { bg: 'bg-gray-500/20', text: 'text-gray-700 dark:text-gray-300' };
export function getFormatColor(format?: string): ColorStyle {
if (!format || format === '-') return FALLBACK_COLOR;
return FORMAT_COLORS[format.toLowerCase()] || DEFAULT_FORMAT_COLOR;
const normalized = format.toLowerCase();
// Support display strings like "EPUB, MOBI +1" by using the first token for color mapping.
const match = normalized.match(/[a-z0-9]+/);
const key = match ? match[0] : normalized;
return FORMAT_COLORS[key] || DEFAULT_FORMAT_COLOR;
}
export function getLanguageColor(language?: string): ColorStyle {
@@ -72,11 +93,27 @@ export function getDownloadTypeColor(downloadType?: string): ColorStyle {
return DOWNLOAD_TYPE_COLORS[downloadType.toLowerCase()] || DEFAULT_DOWNLOAD_TYPE_COLOR;
}
// Returns just the dot color class for protocol indicators
export function getProtocolDotColor(protocol?: string): string {
if (!protocol) return 'bg-gray-400';
const p = protocol.toLowerCase();
if (p === 'torrent') return 'bg-orange-500';
if (p === 'nzb' || p === 'usenet') return 'bg-sky-500';
return 'bg-gray-400';
}
export function getContentTypeColor(contentType?: string): ColorStyle {
if (!contentType || contentType === '-') return FALLBACK_COLOR;
return CONTENT_TYPE_COLORS[contentType.toLowerCase()] || DEFAULT_CONTENT_TYPE_COLOR;
}
export function getFlagColor(flag?: string): ColorStyle {
if (!flag || flag === '-') return FALLBACK_COLOR;
const normalized = flag.trim().toLowerCase();
if (!normalized) return FALLBACK_COLOR;
return FLAG_COLORS[normalized] || DEFAULT_FLAG_COLOR;
}
/**
* Color hint type for dynamic column coloring.
*/
@@ -106,6 +143,8 @@ export function getColorStyleFromHint(value: string, colorHint?: ColumnColorHint
return getDownloadTypeColor(value);
case 'content_type':
return getContentTypeColor(value);
case 'flags':
return getFlagColor(value);
default:
return FALLBACK_COLOR;
}
+33 -1
View File
@@ -87,12 +87,33 @@ export const formatDefaultLanguageLabel = (
return `Default (${joined}${suffix})`;
};
/**
* Build a mapping from language names to codes for normalization.
* Handles both directions: "english" -> "en" and "en" -> "en"
*/
export const buildLanguageNormalizer = (languages: Language[]): Map<string, string> => {
const map = new Map<string, string>();
for (const lang of languages) {
const code = lang.code.toLowerCase();
map.set(code, code); // code -> code
map.set(lang.language.toLowerCase(), code); // name -> code
}
return map;
};
/**
* Check if ALL languages in a multi-language release match the selected filter.
* Multi-language releases use separators like comma, slash, plus, or ampersand
* (e.g., "English, Spanish", "English/Spanish", "English + Spanish", "English & Spanish").
*
* @param releaseLang - Language string from the release (can be code or full name)
* @param selectedCodes - Array of selected ISO language codes
* @param languageNormalizer - Optional map to normalize language names to codes
*/
export const releaseLanguageMatchesFilter = (
releaseLang: string | undefined,
selectedCodes: string[] | null,
languageNormalizer?: Map<string, string>,
): boolean => {
if (!releaseLang || !selectedCodes) {
return true;
@@ -100,7 +121,18 @@ export const releaseLanguageMatchesFilter = (
if (selectedCodes.includes(LANGUAGE_OPTION_ALL)) {
return true;
}
const releaseCodes = releaseLang.split(/[,/]/).map(l => l.trim().toLowerCase()).filter(Boolean);
// Split by common multi-language separators: comma, slash, plus, ampersand
const releaseParts = releaseLang.split(/[,/+&]/).map(l => l.trim().toLowerCase()).filter(Boolean);
// Normalize release language parts to codes (handles both "en" and "english")
const releaseCodes = releaseParts.map(part => {
if (languageNormalizer) {
return languageNormalizer.get(part) ?? part;
}
return part;
});
const selectedSet = new Set(selectedCodes.map(c => c.toLowerCase()));
return releaseCodes.every(code => selectedSet.has(code));
};
+91
View File
@@ -0,0 +1,91 @@
def test_bypass_tries_all_methods_before_abort(monkeypatch):
"""Regression test for issue #524: don't abort before cycling through bypass methods."""
import shelfmark.bypass.internal_bypasser as internal_bypasser
calls: list[str] = []
def _make_method(name: str):
def _method(_sb) -> bool:
calls.append(name)
return False
_method.__name__ = name
return _method
methods = [_make_method(f"m{i}") for i in range(6)]
monkeypatch.setattr(internal_bypasser, "BYPASS_METHODS", methods)
monkeypatch.setattr(internal_bypasser, "_is_bypassed", lambda _sb, escape_emojis=True: False)
monkeypatch.setattr(internal_bypasser, "_detect_challenge_type", lambda _sb: "ddos_guard")
monkeypatch.setattr(internal_bypasser.time, "sleep", lambda _seconds: None)
monkeypatch.setattr(internal_bypasser.random, "uniform", lambda _a, _b: 0)
assert internal_bypasser._bypass(object(), max_retries=10) is False
assert calls == [f"m{i}" for i in range(6)]
def test_extract_cookies_from_cdp_filters_and_stores_ua():
import time
import shelfmark.bypass.internal_bypasser as internal_bypasser
class FakeCookie:
def __init__(self, name, value, domain, path, expires, secure=True):
self.name = name
self.value = value
self.domain = domain
self.path = path
self.expires = expires
self.secure = secure
class FakeSb:
def get_all_cookies(self, requests_cookie_format=False):
assert requests_cookie_format is True
return [
FakeCookie("cf_clearance", "abc", "example.com", "/", int(time.time()) + 3600),
FakeCookie("sessionid", "zzz", "example.com", "/", int(time.time()) + 3600),
]
def get_user_agent(self):
return "TestUA/1.0"
internal_bypasser.clear_cf_cookies()
internal_bypasser._extract_cookies_from_cdp(FakeSb(), "https://www.example.com/path")
cookies = internal_bypasser.get_cf_cookies_for_domain("example.com")
assert cookies == {"cf_clearance": "abc"}
assert internal_bypasser.get_cf_user_agent_for_domain("example.com") == "TestUA/1.0"
def test_extract_cookies_from_cdp_normalizes_session_expiry():
import time
import shelfmark.bypass.internal_bypasser as internal_bypasser
class FakeCookie:
def __init__(self, name, value, domain, path, expires, secure=True):
self.name = name
self.value = value
self.domain = domain
self.path = path
self.expires = expires
self.secure = secure
class FakeSb:
def get_all_cookies(self, requests_cookie_format=False):
assert requests_cookie_format is True
return [
FakeCookie("cf_clearance", "abc", "example.com", "/", 0),
]
def get_user_agent(self):
return "TestUA/1.0"
internal_bypasser.clear_cf_cookies()
internal_bypasser._extract_cookies_from_cdp(FakeSb(), "https://example.com")
stored = internal_bypasser._cf_cookies.get("example.com", {})
assert stored["cf_clearance"]["expiry"] is None
assert internal_bypasser.get_cf_cookies_for_domain("example.com") == {"cf_clearance": "abc"}
# Verify fallback to "expires" key for expiry checks
internal_bypasser._cf_cookies["example.com"]["cf_clearance"]["expires"] = int(time.time()) - 10
assert internal_bypasser.get_cf_cookies_for_domain("example.com") == {}
@@ -0,0 +1,28 @@
def test_update_settings_mirrors_applies_aa_changes_live(monkeypatch):
# Ensure settings tabs are registered (mirrors tab lives here).
import shelfmark.config.settings # noqa: F401
from shelfmark.core.config import config as config_obj
from shelfmark.core.settings_registry import update_settings
monkeypatch.delenv("AA_BASE_URL", raising=False)
monkeypatch.delenv("AA_ADDITIONAL_URLS", raising=False)
# Avoid writing to disk and avoid forcing a full config refresh in this unit test.
monkeypatch.setattr("shelfmark.core.settings_registry.save_config_file", lambda _tab, _values: True)
monkeypatch.setattr(config_obj, "refresh", lambda: None)
called: dict[str, object] = {}
def fake_init_aa(*, force: bool = False) -> None:
called["force"] = force
import shelfmark.download.network as network
monkeypatch.setattr(network, "init_aa", fake_init_aa)
result = update_settings("mirrors", {"AA_BASE_URL": "https://annas-archive.li"})
assert result["success"] is True
assert called["force"] is True
@@ -0,0 +1,11 @@
def test_aa_base_url_options_include_configured_custom_url(monkeypatch):
import shelfmark.config.settings as settings
from shelfmark.core.config import config as config_obj
# Use a custom mirror not present in defaults/additional.
monkeypatch.setenv("AA_BASE_URL", "https://custom-aa.example")
config_obj.refresh()
options = settings._get_aa_base_url_options()
assert any(opt["value"] == "https://custom-aa.example" for opt in options)
+41
View File
@@ -0,0 +1,41 @@
from shelfmark.core import mirrors
class _DummyConfig:
def __init__(self, values: dict):
self._values = values
def get(self, key: str, default=None):
return self._values.get(key, default)
def test_get_aa_mirrors_prefers_full_configured_list(monkeypatch):
dummy = _DummyConfig(
{
"AA_MIRROR_URLS": ["annas-archive.gl/", "https://annas-archive.li"],
"AA_ADDITIONAL_URLS": "https://should-not-be-appended.example",
}
)
monkeypatch.setattr(mirrors, "_get_config", lambda: dummy)
assert mirrors.get_aa_mirrors() == [
"https://annas-archive.gl",
"https://annas-archive.li",
]
def test_get_aa_mirrors_falls_back_to_defaults_and_legacy_additional(monkeypatch):
dummy = _DummyConfig(
{
"AA_MIRROR_URLS": [],
"AA_ADDITIONAL_URLS": "extra.example, https://extra2.example/",
}
)
monkeypatch.setattr(mirrors, "_get_config", lambda: dummy)
aa = mirrors.get_aa_mirrors()
assert "https://annas-archive.gl" in aa
assert "https://annas-archive.li" in aa
assert "https://extra.example" in aa
assert "https://extra2.example" in aa
+147
View File
@@ -216,6 +216,153 @@ class TestParseNamingTemplate:
assert result == "Brandon Sanderson/The Way of Kings"
class TestArbitraryPrefixSuffix:
"""Tests for enhanced template syntax with arbitrary prefix/suffix text."""
def test_vol_prefix_with_value(self):
"""Test {Vol. SeriesPosition - } with a value."""
result = parse_naming_template(
"{Vol. SeriesPosition - }{Title}",
{"SeriesPosition": 2, "Title": "Book Title"}
)
assert result == "Vol. 2 - Book Title"
def test_vol_prefix_without_value(self):
"""Test {Vol. SeriesPosition - } without a value produces nothing."""
result = parse_naming_template(
"{Vol. SeriesPosition - }{Title}",
{"SeriesPosition": None, "Title": "Book Title"}
)
assert result == "Book Title"
def test_vol_prefix_empty_string(self):
"""Test {Vol. SeriesPosition - } with empty string produces nothing."""
result = parse_naming_template(
"{Vol. SeriesPosition - }{Title}",
{"SeriesPosition": "", "Title": "Book Title"}
)
assert result == "Book Title"
def test_book_x_of_series_pattern(self):
"""Test {Book SeriesPosition of the Series} pattern."""
result = parse_naming_template(
"{Book SeriesPosition of the Series}",
{"SeriesPosition": 2, "Series": "Stormlight"}
)
assert result == "Book 2 of the Series"
def test_case_insensitive_arbitrary_prefix(self):
"""Test case-insensitive token matching with arbitrary prefix."""
result = parse_naming_template(
"{vol. seriesposition - }{Title}",
{"SeriesPosition": 3, "Title": "Book"}
)
assert result == "vol. 3 - Book"
def test_arbitrary_prefix_with_part_number(self):
"""Test arbitrary prefix with PartNumber token."""
result = parse_naming_template(
"{Part PartNumber}",
{"PartNumber": "05"}
)
assert result == "Part 05"
def test_arbitrary_prefix_part_number_empty(self):
"""Test arbitrary prefix with empty PartNumber."""
result = parse_naming_template(
"{Title}{Part PartNumber}",
{"Title": "Book", "PartNumber": None}
)
assert result == "Book"
def test_no_variable_in_block_unchanged(self):
"""Test that blocks without known variables are left unchanged."""
result = parse_naming_template(
"{literal text}",
{"Title": "Book"}
)
assert result == "{literal text}"
def test_mixed_legacy_and_new_syntax(self):
"""Test mixed template with both legacy and new syntax."""
result = parse_naming_template(
"{Author}/{Vol. SeriesPosition - }{Title}",
{"Author": "Sanderson", "SeriesPosition": 1, "Title": "Mistborn"}
)
assert result == "Sanderson/Vol. 1 - Mistborn"
def test_mixed_with_empty_series_position(self):
"""Test mixed template when series position is empty."""
result = parse_naming_template(
"{Author}/{Vol. SeriesPosition - }{Title}",
{"Author": "Sanderson", "SeriesPosition": None, "Title": "Elantris"}
)
assert result == "Sanderson/Elantris"
def test_subtitle_with_arbitrary_prefix(self):
"""Test Subtitle token with arbitrary prefix text."""
result = parse_naming_template(
"{Title}{: Subtitle}",
{"Title": "Main", "Subtitle": "Secondary"}
)
assert result == "Main: Secondary"
def test_year_with_arbitrary_prefix_suffix(self):
"""Test Year with arbitrary text around it."""
result = parse_naming_template(
"{Title} {(Year)}",
{"Title": "Book", "Year": 2020}
)
assert result == "Book (2020)"
def test_year_empty_with_arbitrary_prefix_suffix(self):
"""Test Year with arbitrary text when Year is empty."""
result = parse_naming_template(
"{Title} {(Year)}",
{"Title": "Book", "Year": None}
)
# Note: trailing space gets cleaned up
assert result == "Book"
def test_series_position_longest_match(self):
"""Test that SeriesPosition matches before Series."""
result = parse_naming_template(
"{SeriesPosition - }{Series}",
{"SeriesPosition": 1, "Series": "Stormlight"}
)
assert result == "1 - Stormlight"
def test_arbitrary_prefix_float_position(self):
"""Test arbitrary prefix with float series position."""
result = parse_naming_template(
"{Vol. SeriesPosition - }{Title}",
{"SeriesPosition": 1.5, "Title": "Novella"}
)
assert result == "Vol. 1.5 - Novella"
def test_complex_template_with_arbitrary_prefixes(self):
"""Test complex template combining multiple arbitrary prefix patterns."""
template = "{Author}/{Series/}{Vol. SeriesPosition - }{Title}{: Subtitle} {(Year)}"
# All fields present
result = parse_naming_template(template, {
"Author": "Brandon Sanderson",
"Series": "Stormlight Archive",
"SeriesPosition": 1,
"Title": "The Way of Kings",
"Subtitle": "Epic Fantasy",
"Year": 2010
})
assert result == "Brandon Sanderson/Stormlight Archive/Vol. 1 - The Way of Kings: Epic Fantasy (2010)"
# Minimal fields
result = parse_naming_template(template, {
"Author": "Brandon Sanderson",
"Title": "Standalone Novel"
})
assert result == "Brandon Sanderson/Standalone Novel"
class TestBuildLibraryPath:
"""Tests for complete library path building."""
+142
View File
@@ -0,0 +1,142 @@
import requests
class _FakeResponse:
def __init__(self, status_code: int, *, headers: dict | None = None, text: str = "") -> None:
self.status_code = status_code
self.headers = headers or {}
self.text = text
@property
def is_redirect(self) -> bool: # requests.Response compatibility
return self.status_code in (301, 302, 303, 307, 308) and bool(self.headers.get("Location"))
def raise_for_status(self) -> None: # requests.Response compatibility
if self.status_code >= 400:
raise requests.exceptions.HTTPError(response=self)
class _DummySelector:
"""Minimal AA selector stub for unit testing http.html_get_page()."""
def __init__(self, bases: list[str]) -> None:
self._bases = bases
self._index = 0
self.current_base = bases[0]
self.attempts_this_dns = 0
def rewrite(self, url: str) -> str:
for base in self._bases:
if url.startswith(base):
return url.replace(base, self.current_base, 1)
return url
def next_mirror_or_rotate_dns(self, allow_dns: bool = True) -> tuple[str | None, str]:
self.attempts_this_dns += 1
self._index = (self._index + 1) % len(self._bases)
self.current_base = self._bases[self._index]
return self.current_base, "mirror"
def test_html_get_page_aa_cross_host_redirect_rotates_mirror(monkeypatch):
import shelfmark.download.http as http
# Avoid bypasser imports in unit tests.
monkeypatch.setattr(http, "_is_cf_bypass_enabled", lambda: False)
monkeypatch.setattr(http, "get_proxies", lambda _url: {})
monkeypatch.setattr(http.time, "sleep", lambda _s: None)
monkeypatch.setattr(http.network, "get_aa_base_url", lambda: "https://annas-archive.li")
monkeypatch.setattr(http.network, "is_aa_auto_mode", lambda: True)
calls: list[dict] = []
def fake_get(url: str, **kwargs):
calls.append({"url": url, "allow_redirects": kwargs.get("allow_redirects")})
if url.startswith("https://annas-archive.li/"):
return _FakeResponse(302, headers={"Location": "https://annas-archive.pm/search?q=test"})
if url.startswith("https://annas-archive.gl/"):
return _FakeResponse(200, text="OK")
raise AssertionError(f"Unexpected URL: {url}")
monkeypatch.setattr(http.requests, "get", fake_get)
selector = _DummySelector(["https://annas-archive.li", "https://annas-archive.gl"])
html = http.html_get_page(
"https://annas-archive.li/search?q=test",
selector=selector,
retry=2,
allow_bypasser_fallback=False,
)
assert html == "OK"
assert calls[0]["allow_redirects"] is False # AA redirects handled manually
assert calls[0]["url"].startswith("https://annas-archive.li/")
assert calls[1]["url"].startswith("https://annas-archive.gl/") # rotated away from redirect target
def test_html_get_page_aa_same_host_redirect_is_followed(monkeypatch):
import shelfmark.download.http as http
monkeypatch.setattr(http, "_is_cf_bypass_enabled", lambda: False)
monkeypatch.setattr(http, "get_proxies", lambda _url: {})
monkeypatch.setattr(http.time, "sleep", lambda _s: None)
monkeypatch.setattr(http.network, "get_aa_base_url", lambda: "https://annas-archive.li")
monkeypatch.setattr(http.network, "is_aa_auto_mode", lambda: True)
calls: list[dict] = []
def fake_get(url: str, **kwargs):
calls.append({"url": url, "allow_redirects": kwargs.get("allow_redirects")})
if url == "https://annas-archive.li/search?q=test":
return _FakeResponse(302, headers={"Location": "/search?q=test&page=1"})
if url == "https://annas-archive.li/search?q=test&page=1":
return _FakeResponse(200, text="OK2")
raise AssertionError(f"Unexpected URL: {url}")
monkeypatch.setattr(http.requests, "get", fake_get)
selector = _DummySelector(["https://annas-archive.li"])
html = http.html_get_page(
"https://annas-archive.li/search?q=test",
selector=selector,
retry=1,
allow_bypasser_fallback=False,
)
assert html == "OK2"
assert [c["url"] for c in calls] == [
"https://annas-archive.li/search?q=test",
"https://annas-archive.li/search?q=test&page=1",
]
assert all(c["allow_redirects"] is False for c in calls)
def test_html_get_page_locked_aa_does_not_fail_over_on_cross_host_redirect(monkeypatch):
import shelfmark.download.http as http
monkeypatch.setattr(http, "_is_cf_bypass_enabled", lambda: False)
monkeypatch.setattr(http, "get_proxies", lambda _url: {})
monkeypatch.setattr(http.time, "sleep", lambda _s: None)
monkeypatch.setattr(http.network, "get_aa_base_url", lambda: "https://annas-archive.li")
monkeypatch.setattr(http.network, "is_aa_auto_mode", lambda: False)
calls: list[str] = []
def fake_get(url: str, **kwargs):
calls.append(url)
if url.startswith("https://annas-archive.li/"):
return _FakeResponse(302, headers={"Location": "https://annas-archive.pm/search?q=test"})
raise AssertionError(f"Unexpected URL: {url}")
monkeypatch.setattr(http.requests, "get", fake_get)
selector = _DummySelector(["https://annas-archive.li", "https://annas-archive.gl"])
html = http.html_get_page(
"https://annas-archive.li/search?q=test",
selector=selector,
retry=2,
allow_bypasser_fallback=False,
)
assert html == ""
assert calls == ["https://annas-archive.li/search?q=test"]
@@ -0,0 +1,33 @@
from unittest.mock import MagicMock
def test_update_download_status_dedupes_identical_events(monkeypatch):
import shelfmark.download.orchestrator as orchestrator
book_id = "test-book-id"
# Ensure clean module-level state
orchestrator._last_activity.clear()
orchestrator._last_status_event.clear()
mock_queue = MagicMock()
monkeypatch.setattr(orchestrator, "book_queue", mock_queue)
monkeypatch.setattr(orchestrator, "queue_status", lambda: {})
mock_ws = MagicMock()
monkeypatch.setattr(orchestrator, "ws_manager", mock_ws)
times = iter([1.0, 2.0])
monkeypatch.setattr(orchestrator.time, "time", lambda: next(times))
orchestrator.update_download_status(book_id, "resolving", "Bypassing protection...")
orchestrator.update_download_status(book_id, "resolving", "Bypassing protection...")
# Status + message should only be applied/broadcast once.
assert mock_queue.update_status.call_count == 1
assert mock_queue.update_status_message.call_count == 1
assert mock_ws.broadcast_status_update.call_count == 1
# Activity timestamp should still be updated on the duplicate keep-alive call.
assert orchestrator._last_activity[book_id] == 2.0