ci: debounce dev image builds instead of building nightly (#1376)

Replace the nightly cron and its check-changes job with a debounce.
Every push to main starts dev-image-debounce.yml, which waits out the
60-minute wait timer on the dev-image-debounce environment, then
dispatches the Docker workflow only if main still points at its commit.
A burst of merges now publishes one dev image, an hour after the last
merge.

The Docker workflow keeps only its tag and workflow_dispatch triggers,
so its history holds real builds only. The debounce workflow deletes its
own finished runs, so no-op runs don't pile up either.

Requires the dev-image-debounce environment with a 60-minute wait
timer (Settings → Environments).

.dockerignore

@@ -1,6 +1,7 @@
 .git
 .github
 .vscode
+.local
 .mypy_cache
 README_images
 .gitignore

.github/ISSUE_TEMPLATE/bug_report.md

@@ -11,9 +11,9 @@ assignees: ''
 
 ## Steps To Reproduce
 <!-- Steps to reproduce the behavior -->
-1. 
-2. 
-3. 
+1.
+2.
+3.
 
 ## Expected Behavior
 <!-- A clear and concise description of what you expected to happen -->
@@ -34,4 +34,4 @@ When running in debug mode, a DEBUG button will appear in the interface. Please
 <!-- If applicable, please provide your full docker-compose (redacted from any secrets) -->
 
 ## Additional Context
-<!-- Add any other context about the problem here --> 
+<!-- Add any other context about the problem here -->

.github/dependabot.yml

@@ -0,0 +1,87 @@
+version: 2
+updates:
+  # Python dependencies
+  # Dependabot supports uv version updates, but GitHub currently lists uv
+  # security updates as "Not applicable"; daily checks keep uv.lock moving
+  # while repo-level Dependabot alerts/security updates cover supported ecosystems.
+  - package-ecosystem: "uv"
+    directory: "/"
+    schedule:
+      interval: "daily"
+      time: "05:00"
+      timezone: "Europe/London"
+    cooldown:
+      default-days: 3
+    open-pull-requests-limit: 10
+    groups:
+      python-deps:
+        patterns: ["*"]
+        update-types: ["minor", "patch"]
+
+  # Frontend npm dependencies
+  - package-ecosystem: "npm"
+    directory: "/src/frontend"
+    schedule:
+      interval: "weekly"
+    cooldown:
+      default-days: 3
+    open-pull-requests-limit: 10
+    groups:
+      npm-deps:
+        patterns: ["*"]
+        update-types: ["minor", "patch"]
+
+  # Dockerfile base image digests. When a tag stays the same, Dependabot titles
+  # can only show digest prefixes, so keep the group name explicit.
+  - package-ecosystem: "docker"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    cooldown:
+      default-days: 3
+    open-pull-requests-limit: 5
+    groups:
+      docker-base-image-digests:
+        # Exclude python from the group on purpose. Dependabot's Docker
+        # pre-release filter is bypassed for *grouped* updates
+        # (dependabot-core#9496), so a grouped python update proposes pre-release
+        # tags like python:3.15.0b2 as if they were a normal stable minor bump.
+        # node + uv stay grouped into a single digest PR.
+        patterns: ["*"]
+        exclude-patterns: ["python"]
+    ignore:
+      # Node.js: block major-version bumps so dependabot never proposes
+      # moving from one LTS line to a non-LTS "Current" release (e.g. 24 -> 25).
+      # Node LTS is even-numbered only; major bumps should be deliberate.
+      - dependency-name: "node"
+        update-types: ["version-update:semver-major"]
+
+      # Python: block minor/major bumps. Ungrouping python (above) is NOT enough
+      # to keep pre-releases out — dependabot-core#13815 rewrote the Docker
+      # pre-release heuristic to catch PEP 440 tags like 3.15.0a2 / 3.5.0b3, but
+      # the suffixed real tag still slipped through as PR #1169
+      # (python:3.14.6-slim -> python:3.15.0b3-slim). CPython spells
+      # pre-releases without a separator, so tag parsing reads 3.15.0b3 as an
+      # ordinary version that sorts above 3.14.6.
+      #
+      # A minor-version ignore blocks it regardless of spelling. Patch bumps
+      # (3.14.6 -> 3.14.7) and same-tag digest refreshes still land automatically.
+      # Moving the runtime to a new Python minor is a manual, deliberate change:
+      # bump the tag here and confirm C-extension wheels (greenlet/gevent) exist
+      # for it — a source build against a pre-release ABI boots an app that binds
+      # its port but never serves, which wedges e2e for the full 6h job limit.
+      - dependency-name: "python"
+        update-types:
+          ["version-update:semver-major", "version-update:semver-minor"]
+
+  # GitHub Actions
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    cooldown:
+      default-days: 3
+    open-pull-requests-limit: 5
+    groups:
+      gh-actions:
+        patterns: ["*"]

.github/workflows/build-and-publish-docker-image.yml

@@ -1,11 +1,13 @@
 name: Create and publish Docker images
 on:
   push:
-    branches:
-      - 'main'
     tags:
       - 'v*'
+  # Also dispatched on main by dev-image-debounce.yml, once main has been
+  # quiet for an hour, to publish the dev image.
   workflow_dispatch:
+permissions: read-all
+
 env:
   REGISTRY: ghcr.io
   IMAGE_NAME: ${{ github.repository_owner }}/shelfmark
@@ -27,20 +29,26 @@ jobs:
       - name: Get current date
         id: date
         run: echo "date=$(date +'%Y-%m-%d')" >> $GITHUB_OUTPUT
-      
+
       - name: Checkout repository
-        uses: actions/checkout@v4
-      
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+
       - name: Log in to the Container registry
-        uses: docker/login-action@v3
+        uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
         with:
           registry: ${{ env.REGISTRY }}
           username: ${{ github.actor }}
           password: ${{ secrets.GITHUB_TOKEN }}
-      
+
       - name: Extract metadata for ${{ matrix.target }} image
         id: meta
-        uses: docker/metadata-action@v5
+        uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
+        env:
+          # Annotate both the per-platform manifests and the multi-arch image
+          # index. The index level is what manifest-list consumers (Renovate's
+          # minimumReleaseAge soak check, provenance/SBOM tooling) read for the
+          # standard org.opencontainers.image.* annotations, including `created`.
+          DOCKER_METADATA_ANNOTATIONS_LEVELS: index,manifest
         with:
           images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}${{ matrix.image_name_suffix }}
           tags: |
@@ -50,13 +58,13 @@ jobs:
             type=semver,pattern={{major}}.{{minor}}
             type=sha
             type=ref,event=tag
-      
+
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
-      
+        uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
+
       - name: Build and push ${{ matrix.target }} Docker image
         id: push
-        uses: docker/build-push-action@v5
+        uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
         with:
           platforms: linux/amd64,linux/arm64
           context: .
@@ -67,10 +75,11 @@ jobs:
             RELEASE_VERSION=${{ github.ref_name }}
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
-      
+          annotations: ${{ steps.meta.outputs.annotations }}
+
       - name: Generate artifact attestation for ${{ matrix.target }} image
         if: github.event_name != 'pull_request'
-        uses: actions/attest-build-provenance@v2
+        uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2
         with:
           subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}${{ matrix.image_name_suffix }}
           subject-digest: ${{ steps.push.outputs.digest }}
@@ -89,14 +98,14 @@ jobs:
       LEGACY_NAME: calibre-web-automated-book-downloader
     steps:
       - name: Log in to registry
-        uses: docker/login-action@v3
+        uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
         with:
           registry: ${{ env.REGISTRY }}
           username: ${{ github.actor }}
           password: ${{ secrets.GITHUB_TOKEN }}
 
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
+        uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
 
       - name: Create legacy aliases
         run: |

.github/workflows/ci.yml

@@ -0,0 +1,158 @@
+name: CI
+
+on:
+  pull_request:
+    branches: [main]
+
+permissions:
+  contents: read
+
+jobs:
+  python-quality:
+    name: Python Quality
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+
+      - name: Install uv and Python
+        uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
+        with:
+          version: "0.11.3"
+          python-version: "3.14"
+          enable-cache: true
+
+      - name: Sync dependencies
+        run: make install-python-dev
+
+      - name: Lint
+        run: make python-lint
+
+      - name: Check formatting
+        run: make python-format
+
+      - name: Check dead code
+        run: make python-dead-code
+
+  python-typechecks:
+    name: Python Typechecks
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+
+      - name: Install uv and Python
+        uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
+        with:
+          version: "0.11.3"
+          python-version: "3.14"
+          enable-cache: true
+
+      - name: Sync dependencies
+        run: make install-python-dev
+
+      - name: Typecheck
+        run: make python-typecheck
+
+  python-tests:
+    name: Python Tests
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+
+      - name: Install uv and Python
+        uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
+        with:
+          version: "0.11.3"
+          python-version: "3.14"
+          enable-cache: true
+
+      - name: Sync dependencies
+        run: make install-python-dev
+
+      - name: Run tests
+        run: make python-test
+
+  docker-build-check:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
+
+      - name: Build shelfmark-lite image
+        uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
+        with:
+          context: .
+          target: shelfmark-lite
+          platforms: linux/amd64
+          push: false
+          build-args: |
+            BUILD_VERSION=pr-${{ github.sha }}
+            RELEASE_VERSION=pr-${{ github.event.pull_request.number }}
+
+  frontend-quality:
+    name: Frontend Quality
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+
+      - name: Set up Node
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: 24
+          cache: "npm"
+          cache-dependency-path: src/frontend/package-lock.json
+
+      - name: Install dependencies
+        run: make install-ci
+
+      - name: Lint
+        run: make frontend-lint
+
+      - name: Check formatting
+        run: make frontend-format
+
+  frontend-typechecks:
+    name: Frontend Typechecks
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+
+      - name: Set up Node
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: 24
+          cache: "npm"
+          cache-dependency-path: src/frontend/package-lock.json
+
+      - name: Install dependencies
+        run: make install-ci
+
+      - name: Typecheck
+        run: make frontend-typecheck
+
+  frontend-tests:
+    name: Frontend Tests
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+
+      - name: Set up Node
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: 24
+          cache: "npm"
+          cache-dependency-path: src/frontend/package-lock.json
+
+      - name: Install dependencies
+        run: make install-ci
+
+      - name: Unit tests
+        run: make frontend-test

.github/workflows/codeql.yml

@@ -0,0 +1,38 @@
+name: CodeQL
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+  schedule:
+    - cron: "0 6 * * 1" # Weekly on Monday at 6am UTC
+
+permissions:
+  contents: read
+
+jobs:
+  analyze:
+    runs-on: ubuntu-latest
+    permissions:
+      security-events: write
+    strategy:
+      fail-fast: false
+      matrix:
+        language: [python, javascript-typescript]
+    steps:
+      - name: Checkout
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+
+      - name: Initialize CodeQL
+        uses: github/codeql-action/init@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v3
+        with:
+          languages: ${{ matrix.language }}
+
+      - name: Autobuild
+        uses: github/codeql-action/autobuild@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v3
+
+      - name: Perform CodeQL Analysis
+        uses: github/codeql-action/analyze@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v3
+        with:
+          category: "/language:${{ matrix.language }}"

.github/workflows/dev-image-debounce.yml

@@ -0,0 +1,51 @@
+name: Debounce dev image
+# A burst of merges to main should publish one dev image, not one per commit.
+# Each push to main waits out the 60-minute wait timer on the
+# dev-image-debounce environment (Settings → Environments; waiting holds no
+# runner), then dispatches the Docker workflow only if main still points at
+# its commit. So only the last push of a burst builds, and the Docker
+# workflow's history holds real builds only.
+on:
+  push:
+    branches:
+      - main
+permissions: {}
+
+jobs:
+  debounce:
+    runs-on: ubuntu-latest
+    environment:
+      name: dev-image-debounce
+      deployment: false
+    permissions:
+      actions: write # dispatch the build, delete finished debounce runs
+      contents: read
+    env:
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      GH_REPO: ${{ github.repository }}
+    steps:
+      - name: Dispatch the dev build if main is still at this commit
+        env:
+          CURRENT_SHA: ${{ github.sha }}
+        run: |
+          HEAD_SHA=$(gh api "repos/${GH_REPO}/git/ref/heads/main" --jq '.object.sha')
+          echo "This run's commit: ${CURRENT_SHA}"
+          echo "main HEAD now:     ${HEAD_SHA}"
+          if [[ "$HEAD_SHA" == "$CURRENT_SHA" ]]; then
+            echo "No newer commits on main — dispatching the dev build."
+            gh workflow run build-and-publish-docker-image.yml --ref main
+          else
+            echo "main has moved on — the newer push's run will build it."
+          fi
+
+      # A finished debounce run is noise: it either did nothing or its build
+      # run is the record. Best effort, since runs from the same burst race to
+      # delete the same runs.
+      - name: Delete finished debounce runs
+        run: |
+          # On an HTTP error gh prints the error body to stdout, so bail out
+          # rather than loop over it.
+          ids=$(gh api "repos/${GH_REPO}/actions/workflows/dev-image-debounce.yml/runs?status=success&per_page=100" --jq '.workflow_runs[].id') || exit 0
+          for id in $ids; do
+            gh api -X DELETE "repos/${GH_REPO}/actions/runs/${id}" || true
+          done

.github/workflows/e2e-platform.yml

@@ -0,0 +1,135 @@
+name: E2E Platform
+
+# Hermetic end-to-end matrix: boots the app under test against mock
+# Anna's Archive / Cloudflare / bypasser / DNS / proxy / Tor / real torrent
+# clients and runs the cluster suite under each config profile.
+#
+# On a PR that touches relevant code, this runs a fast core subset *and* the heavy
+# `full` profile (real Chrome solving Cloudflare + DoH + real qBittorrent). The
+# `e2e-required` job aggregates them into ONE status check — make that check a
+# required status check in branch protection to block merges on any e2e failure
+# (see tests/e2e/platform/README.md "Gating PRs").
+
+on:
+  pull_request:
+  schedule:
+    - cron: "0 4 * * *"   # nightly full matrix
+  workflow_dispatch:
+
+concurrency:
+  group: e2e-platform-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  # Detect whether anything that affects the e2e platform changed. This lets the
+  # required check always report (never stuck "pending") while only spending CI on
+  # PRs that can actually break the e2e stack.
+  changes:
+    runs-on: ubuntu-latest
+    outputs:
+      relevant: ${{ steps.filter.outputs.relevant }}
+    steps:
+      - uses: actions/checkout@v7
+      - uses: dorny/paths-filter@v4.0.3
+        id: filter
+        with:
+          filters: |
+            relevant:
+              - 'shelfmark/**'
+              - 'entrypoint.sh'
+              - 'tor.sh'
+              - 'Dockerfile'
+              - 'tests/e2e/platform/**'
+              - '.github/workflows/e2e-platform.yml'
+
+  select-profiles:
+    needs: changes
+    if: needs.changes.outputs.relevant == 'true' || github.event_name != 'pull_request'
+    runs-on: ubuntu-latest
+    outputs:
+      profiles: ${{ steps.pick.outputs.profiles }}
+    steps:
+      - id: pick
+        run: |
+          if [ "${{ github.event_name }}" = "pull_request" ]; then
+            echo 'profiles=["baseline","bypasser-external","dns-blocked"]' >> "$GITHUB_OUTPUT"
+          else
+            echo 'profiles=["baseline","bypasser-external","bypasser-disabled","dns-manual","dns-blocked","dns-doh","proxy-http","proxy-socks","tor","client-transmission","client-deluge","client-qbittorrent-delayed"]' >> "$GITHUB_OUTPUT"
+          fi
+
+  e2e:
+    needs: select-profiles
+    runs-on: ubuntu-latest
+    # A wedged app under test must not burn GitHub's 6h max job limit. A healthy
+    # profile run finishes in ~3-5 min; anything past 25 is hung, not slow.
+    timeout-minutes: 25
+    strategy:
+      fail-fast: false
+      matrix:
+        profile: ${{ fromJSON(needs.select-profiles.outputs.profiles) }}
+    name: e2e (${{ matrix.profile }})
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v7
+      - name: Install uv and Python
+        uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
+        with:
+          python-version: "3.14"
+          enable-cache: true
+      - name: Sync dependencies
+        run: make install-python-dev
+      - name: Run e2e platform (${{ matrix.profile }})
+        run: tests/e2e/platform/run-e2e.sh env/${{ matrix.profile }}.env
+      - name: Dump shelfmark logs on failure
+        if: failure()
+        run: cat tests/e2e/platform/.state/shelfmark.${{ matrix.profile }}.log || true
+
+  # Heavy "everything real" job: real Chrome internal bypasser solving Cloudflare +
+  # DoH + real qBittorrent webseed download. Runs on relevant PRs and nightly.
+  e2e-full:
+    needs: changes
+    if: needs.changes.outputs.relevant == 'true' || github.event_name != 'pull_request'
+    runs-on: ubuntu-latest
+    # Real Chrome + qBittorrent is the slowest profile; still nowhere near 40 min.
+    timeout-minutes: 40
+    name: e2e (full — real Chrome + qBittorrent)
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v7
+      - name: Install uv and Python
+        uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
+        with:
+          python-version: "3.14"
+          enable-cache: true
+      - name: Sync dependencies
+        run: make install-python-dev
+      - name: Run full pipeline
+        run: tests/e2e/platform/run-e2e.sh env/full.env
+      - name: Dump logs on failure
+        if: failure()
+        run: |
+          cat tests/e2e/platform/.state/shelfmark.full.log || true
+          docker logs e2e-qbittorrent || true
+
+  # Single aggregated gate. ALWAYS runs (so a required check never hangs "pending"
+  # on unrelated PRs) and FAILS if any e2e job failed/was cancelled. Make THIS the
+  # required status check in branch protection.
+  e2e-required:
+    name: e2e required
+    needs: [e2e, e2e-full]
+    if: always()
+    runs-on: ubuntu-latest
+    steps:
+      - name: Aggregate e2e results
+        run: |
+          matrix='${{ needs.e2e.result }}'
+          full='${{ needs.e2e-full.result }}'
+          echo "e2e matrix=$matrix, e2e-full=$full"
+          # success or skipped (unrelated PR) is OK; failure/cancelled blocks.
+          for r in "$matrix" "$full"; do
+            if [ "$r" = "failure" ] || [ "$r" = "cancelled" ]; then
+              echo "::error::An e2e platform job did not pass — blocking."
+              exit 1
+            fi
+          done
+          echo "All e2e platform jobs passed (or were skipped as not relevant)."

.gitignore

@@ -74,6 +74,7 @@ pip-log.txt
 pip-delete-this-directory.txt
 
 # Unit test / coverage reports
+src/frontend/coverage/
 htmlcov/
 .tox/
 .nox/
@@ -165,6 +166,10 @@ ENV/
 env.bak/
 venv.bak/
 
+# ...but the e2e platform test profiles live in an env/ dir and must be tracked
+!tests/e2e/platform/env/
+!tests/e2e/platform/env/*.env
+
 # Spyder project settings
 .spyderproject
 .spyproject
@@ -231,5 +236,8 @@ pyrightconfig.json
 *.local.*
 AGENTS.md
 .claude/
+CLAUDE.md
+.nvmrc
 .playwright-mcp/
 frontend-dist/
+node_modules/

.pre-commit-config.yaml

@@ -0,0 +1,23 @@
+repos:
+  - repo: builtin
+    hooks:
+      - id: trailing-whitespace
+      - id: end-of-file-fixer
+      - id: check-added-large-files
+
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.15.10
+    hooks:
+      - id: ruff-check
+        args: [--fix]
+      - id: ruff-format
+
+  - repo: local
+    hooks:
+      - id: oxfmt
+        name: oxfmt
+        entry: npx --prefix src/frontend oxfmt --config src/frontend/.oxfmtrc.json
+        language: system
+        types_or: [javascript, jsx, ts, tsx, css, json]
+        files: ^src/frontend/
+        exclude: package-lock\.json

.vscode/launch.json

@@ -61,4 +61,4 @@
       ]
     }
   ]
-}
+}

Dockerfile

@@ -4,7 +4,7 @@ ARG BUILDPLATFORM
 ARG BUILDARCH
 
 # Frontend build stage.
-FROM --platform=$BUILDPLATFORM node:20-alpine AS frontend-builder
+FROM --platform=$BUILDPLATFORM node:24-alpine@sha256:ebfe2f90462722a7a4de65e91990e97fe0d401c70e0e762c5b53302f905ec1c1 AS frontend-builder
 
 # Helpful debug output to see what platforms BuildKit thinks it's using
 RUN echo "BUILDPLATFORM=$BUILDPLATFORM BUILDARCH=$BUILDARCH TARGETPLATFORM=$TARGETPLATFORM TARGETARCH=$TARGETARCH"
@@ -24,8 +24,14 @@ COPY src/frontend/ ./
 # Build the frontend
 RUN npm run build
 
+# uv is a build-time tool only, so it is mounted into the RUNs that need it rather
+# than copied into the image. A COPY here would land ~24 MB in a `base` layer that
+# every published image inherits, and a later `rm` cannot take it back out again --
+# a RUN adds a layer, it does not rewrite the one underneath.
+FROM ghcr.io/astral-sh/uv:0.12.16@sha256:adc68cd785ca65ea25c0611043b0a00b4ea3a22e1b54102fc084406d888082ee AS uv
+
 # Use python-slim as the base image
-FROM python:3.10-slim AS base
+FROM python:3.14.7-slim@sha256:cad9a2c871761c413caa6fdd6441c783451e740a48aaeba60ae62a8b53525ef6 AS base
 
 # Add build argument for version
 ARG BUILD_VERSION
@@ -39,13 +45,12 @@ SHELL ["/bin/bash", "-o", "pipefail", "-c"]
 # Consistent environment variables grouped together
 ENV DEBIAN_FRONTEND=noninteractive \
     DOCKERMODE=true \
+    UV_LINK_MODE=copy \
     PYTHONUNBUFFERED=1 \
     PYTHONDONTWRITEBYTECODE=1 \
     PYTHONIOENCODING=UTF-8 \
-    PIP_NO_CACHE_DIR=1 \
-    PIP_DISABLE_PIP_VERSION_CHECK=1 \
-    PIP_DEFAULT_TIMEOUT=100 \
     NAME=Shelfmark \
+    PATH=/app/.venv/bin:$PATH \
     PYTHONPATH=/app \
     # PUID/PGID will be handled by entrypoint script, but TZ/Locale are still needed
     LANG=en_US.UTF-8 \
@@ -56,9 +61,13 @@ ENV DEBIAN_FRONTEND=noninteractive \
 ENV FLASK_PORT=8084
 
 # Configure locale, timezone, and perform initial cleanup in a single layer
-# User/group creation is removed
 RUN apt-get update && \
     apt-get install -y --no-install-recommends \
+    # For building C-extensions (cffi, gevent, etc.)
+    gcc \
+    g++ \
+    libffi-dev \
+    python3-dev \
     # For locale
     locales tzdata \
     # For healthcheck
@@ -68,11 +77,16 @@ RUN apt-get update && \
     # For debug
     zip iputils-ping \
     # For user switching
-    sudo \
+    gosu \
     # --- Tor support (activated via USING_TOR=true) ---
     tor \
     supervisor \
-    iptables && \
+    iptables \
+    # --- WireGuard support (activated via USING_WIREGUARD=true) ---
+    wireguard-tools \
+    iproute2 \
+    procps \
+    ca-certificates && \
     # Configure iptables alternatives for tor.sh compatibility
     update-alternatives --set iptables /usr/sbin/iptables-legacy && \
     update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy && \
@@ -88,15 +102,30 @@ RUN apt-get update && \
     echo "LC_ALL=en_US.UTF-8" >> /etc/environment && \
     echo "LANG=en_US.UTF-8" > /etc/locale.conf
 
+# Create a fixed runtime user/group so hardened Docker/Kubernetes deployments
+# can start the container directly as a non-root user with a passwd entry.
+RUN groupadd -g 1000 shelfmark && \
+    useradd -u 1000 -g shelfmark -d /home/shelfmark -s /usr/sbin/nologin shelfmark && \
+    mkdir -p /home/shelfmark && \
+    chown 1000:1000 /home/shelfmark
+
 # Set working directory
 WORKDIR /app
 
-# Install Python dependencies using pip
-# Copying requirements files separately leverages build cache
-# Cache mount persists pip cache between builds for faster installs
-COPY requirements-base.txt requirements-shelfmark.txt ./
-RUN --mount=type=cache,target=/root/.cache/pip \
-    pip install -r requirements-base.txt
+# Install core Python dependencies first for better layer caching
+COPY pyproject.toml uv.lock ./
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=from=uv,source=/uv,target=/usr/local/bin/uv \
+    uv sync --locked --no-default-groups
+
+# Runtime dependencies are installed into /app/.venv during the build. Remove the
+# base image's system pip so stale installer CVEs do not ship in the final image.
+RUN rm -rf \
+        /usr/local/bin/pip \
+        /usr/local/bin/pip3 \
+        /usr/local/bin/pip3.* \
+        /usr/local/lib/python*/site-packages/pip \
+        /usr/local/lib/python*/site-packages/pip-*.dist-info
 
 # Copy application code *after* dependencies are installed
 COPY . .
@@ -104,19 +133,33 @@ COPY . .
 # Copy built frontend from frontend-builder stage
 COPY --from=frontend-builder /frontend/dist /app/frontend-dist
 
-# Final setup: permissions and directories in one layer
-# Only creating directories and setting executable bits.
-# Ownership will be handled by the entrypoint script.
-RUN mkdir -p /var/log/shelfmark /books && \
-    chmod +x /app/entrypoint.sh /app/tor.sh /app/genDebug.sh
+# Final setup: create image-owned runtime paths for the fixed non-root user.
+# Root/PUID mode still re-homes ownership at startup when needed.
+RUN mkdir -p \
+        /config \
+        /books \
+        /var/log/shelfmark \
+        /tmp/shelfmark/seleniumbase/downloaded_files \
+        /tmp/shelfmark/seleniumbase/archived_files && \
+    rm -rf /app/downloaded_files /app/archived_files && \
+    ln -s /tmp/shelfmark/seleniumbase/downloaded_files /app/downloaded_files && \
+    ln -s /tmp/shelfmark/seleniumbase/archived_files /app/archived_files && \
+    chown -R 1000:1000 /config /books /home/shelfmark /tmp/shelfmark /var/log/shelfmark && \
+    chmod -R a+rX /app && \
+    chmod +x /app/entrypoint.sh /app/tor.sh /app/wireguard.sh /app/genDebug.sh
 
 # Expose the application port
 EXPOSE ${FLASK_PORT}
 
 # Add healthcheck for container status
-# Uses /api/health which doesn't require authentication
-HEALTHCHECK --interval=60s --timeout=60s --start-period=60s --retries=3 \
-    CMD curl -s http://localhost:${FLASK_PORT}/api/health > /dev/null || exit 1
+# Uses /api/health which doesn't require authentication.
+# curl needs -f so an HTTP error status fails the probe instead of passing it:
+# plain `curl -s` exits 0 on a 500, which reported a broken app as healthy.
+# timeout stays well under interval so a hung probe cannot occupy a whole cycle.
+# --start-interval matches the daemon default (5s), made explicit so startup
+# probing does not depend on that default staying put.
+HEALTHCHECK --interval=30s --timeout=10s --start-period=90s --start-interval=5s --retries=3 \
+    CMD curl -fsS http://localhost:${FLASK_PORT}/api/health > /dev/null || exit 1
 
 # Use dumb-init as the entrypoint to handle signals properly
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
@@ -124,21 +167,39 @@ ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 FROM base AS shelfmark
 
-RUN apt-get update && \
-    apt-get install -y --no-install-recommends \
+# --- Chromium (PINNED to 149.0.7827.196) ---
+# Debian's chromium 150.0.7871.46-1~deb13u1 security update (trixie-security,
+# 2026-07-05) no longer opens the DevTools remote-debugging TCP port at all
+# (no listener, no DevToolsActivePort file, even with a custom --user-data-dir;
+# the RemoteDebuggingAllowed policy does not restore it). The SeleniumBase
+# Pure-CDP driver connects through that port (/json/version), so with 150 every
+# internal bypass dies with "Pure CDP browser startup failed" and all
+# CF-gated downloads fail. Install the last working version from
+# snapshot.debian.org until the bypasser can talk to Chromium >= 150 (e.g.
+# pipe-based DevTools / UC mode) or seleniumbase ships a fix.
+# Chrome 144+ requires --enable-unsafe-swiftshader for WebGL in Docker.
+# This flag is set in internal_bypasser.py _get_browser_args()
+ARG CHROMIUM_VERSION=149.0.7827.196-1~deb13u1
+ARG CHROMIUM_SNAPSHOT=20260704T000000Z
+
+RUN echo "deb [check-valid-until=no] https://snapshot.debian.org/archive/debian-security/${CHROMIUM_SNAPSHOT}/ trixie-security main" \
+        > /etc/apt/sources.list.d/chromium-pin-snapshot.list && \
+    apt-get update -o Acquire::Retries=5 && \
+    apt-get install -y --no-install-recommends -o Acquire::Retries=5 \
     # For dumb display
     xvfb \
     # For screen recording
     ffmpeg \
-    # --- Chromium (unpinned - uses latest from Debian repos) ---
-    # Chrome 144+ requires --enable-unsafe-swiftshader for WebGL in Docker.
-    # This flag is set in internal_bypasser.py _get_browser_args()
-    chromium \
-    chromium-common \
+    chromium=${CHROMIUM_VERSION} \
+    chromium-common=${CHROMIUM_VERSION} \
     # For tkinter (pyautogui)
     python3-tk \
     # For RAR extraction
     unrar-free && \
+    # Keep apt from "upgrading" chromium past the pin inside derived images
+    printf 'Package: chromium chromium-common\nPin: version %s\nPin-Priority: 1001\n' "${CHROMIUM_VERSION}" \
+        > /etc/apt/preferences.d/chromium-pin && \
+    rm /etc/apt/sources.list.d/chromium-pin-snapshot.list && \
     # Create symlink so rarfile library can find unrar
     ln -sf /usr/bin/unrar-free /usr/bin/unrar && \
     # Cleanup APT cache
@@ -146,13 +207,35 @@ RUN apt-get update && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/*
 
-# Install additional dependencies (requirements file already copied in base stage)
-RUN --mount=type=cache,target=/root/.cache/pip \
-    pip install -r requirements-shelfmark.txt
+# Install the browser automation stack used by the full image
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=from=uv,source=/uv,target=/usr/local/bin/uv \
+    uv sync --locked --no-default-groups --extra browser
+
+# Deterministically resolve the Xlib namespace collision.
+# pyautogui/mouseinfo pull the stale `python3-xlib` (0.15, 2014), while the
+# `--extra browser` set pulls `python-xlib` (0.33). Both packages install into
+# the same top-level `Xlib/` namespace, so whichever lands last wins. When the
+# 2014 build wins, `Xlib.X` is missing `FamilyServerInterpreted`, which the
+# SeleniumBase Pure-CDP driver requires at browser startup -> every bypass fails
+# with "module 'Xlib.X' has no attribute 'FamilyServerInterpreted'" and no
+# Cloudflare/DDoS-Guard protected download can complete. Drop the stale package
+# and force python-xlib 0.33 to own the namespace. pyautogui runs fine against
+# 0.33 (superset API).
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=from=uv,source=/uv,target=/usr/local/bin/uv \
+    uv pip uninstall --python /app/.venv/bin/python python3-xlib && \
+    uv pip install --python /app/.venv/bin/python --reinstall python-xlib==0.33 && \
+    /app/.venv/bin/python -c "import Xlib.X; assert hasattr(Xlib.X, 'FamilyServerInterpreted'), 'Xlib.X.FamilyServerInterpreted missing after fix'; print('Xlib namespace OK:', Xlib.__version__)"
+
+# Keep SeleniumBase's bundled driver cache writable for the fixed non-root user.
+RUN SELENIUMBASE_DRIVERS_DIR=$(/app/.venv/bin/python -c "import pathlib, seleniumbase; print(pathlib.Path(seleniumbase.__file__).resolve().parent / 'drivers')") && \
+    chown -R 1000:1000 "${SELENIUMBASE_DRIVERS_DIR}" && \
+    chmod -R u+rwX,go+rX "${SELENIUMBASE_DRIVERS_DIR}" && \
+    if [ -f "${SELENIUMBASE_DRIVERS_DIR}/uc_driver" ]; then chmod +x "${SELENIUMBASE_DRIVERS_DIR}/uc_driver"; fi
 
 # Grant read/execute permissions to others
-RUN chmod -R o+rx /usr/bin/chromium && \
-    chmod -R o+rwx /usr/local/lib/python3.10/site-packages/seleniumbase/drivers/
+RUN chmod -R o+rx /usr/bin/chromium
 
 # Default command to run the application entrypoint script
 CMD ["/app/entrypoint.sh"]

Makefile

@@ -1,4 +1,4 @@
-.PHONY: help install dev build preview typecheck clean up down docker-build refresh restart
+.PHONY: help install install-ci install-python-dev dev build preview frontend-typecheck frontend-lint frontend-format frontend-format-fix frontend-checks frontend-test clean up down docker-build refresh restart build-serve python-lint python-lint-fix python-format python-format-fix python-typecheck python-dead-code python-checks python-test python-test-cov e2e-platform e2e-platform-profile e2e-platform-matrix e2e-platform-full e2e-platform-build checks fix
 
 # Frontend directory
 FRONTEND_DIR := src/frontend
@@ -10,12 +10,38 @@ COMPOSE_FILE := docker-compose.dev.yml
 help:
 	@echo "Available targets:"
 	@echo ""
+	@echo "Quality:"
+	@echo "  checks     - Run ALL static analysis checks (frontend + Python)"
+	@echo "  fix        - Auto-fix lint + format issues (frontend + Python)"
+	@echo ""
 	@echo "Frontend:"
 	@echo "  install    - Install frontend dependencies"
 	@echo "  dev        - Start development server"
 	@echo "  build      - Build frontend for production"
+	@echo "  build-serve - Build and serve via Flask (test prod build without Docker)"
 	@echo "  preview    - Preview production build"
-	@echo "  typecheck  - Run TypeScript type checking"
+	@echo "  frontend-typecheck - Run TypeScript type checking"
+	@echo "  frontend-lint - Run Oxlint against frontend code"
+	@echo "  frontend-format - Check frontend formatting with Oxfmt"
+	@echo "  frontend-format-fix - Format frontend code with Oxfmt"
+	@echo "  frontend-checks - Run all frontend static analysis checks"
+	@echo "  frontend-test - Run frontend unit tests"
+	@echo ""
+	@echo "Python:"
+	@echo "  install-python-dev - Sync Python runtime + dev tooling with uv"
+	@echo "  python-lint - Run Ruff against Python code (backend + tests)"
+	@echo "  python-lint-fix - Run Ruff with safe auto-fixes"
+	@echo "  python-format - Check Python formatting with Ruff"
+	@echo "  python-format-fix - Format Python code with Ruff"
+	@echo "  python-typecheck - Run BasedPyright against backend + tests"
+	@echo "  python-dead-code - Run Vulture against backend code"
+	@echo "  python-checks - Run all Python static analysis checks"
+	@echo "  python-test - Run unit tests"
+	@echo "  python-test-cov - Run unit tests with coverage report"
+	@echo "  e2e-platform - Run e2e docker platform (baseline profile)"
+	@echo "  e2e-platform-profile PROFILE=<name> - Run e2e platform for one profile"
+	@echo "  e2e-platform-matrix - Run e2e platform across all config profiles"
+	@echo "  e2e-platform-full - Run heavy 'full' profile (real Chrome bypasser + DoH + real qBittorrent)"
 	@echo "  clean      - Remove node_modules and build artifacts"
 	@echo ""
 	@echo "Backend (Docker):"
@@ -30,6 +56,17 @@ install:
 	@echo "Installing frontend dependencies..."
 	cd $(FRONTEND_DIR) && npm install
 
+install-ci:
+	@echo "Installing frontend dependencies (CI, lockfile-strict)..."
+	cd $(FRONTEND_DIR) && npm ci
+
+# Install Python development dependencies
+install-python-dev:
+	@echo "Syncing Python runtime and dev tooling with uv..."
+	uv sync --locked --extra browser
+	@echo "Installing prek git hooks..."
+	uv run prek install
+
 # Start development server
 dev:
 	@echo "Starting development server..."
@@ -40,16 +77,112 @@ build:
 	@echo "Building frontend for production..."
 	cd $(FRONTEND_DIR) && npm run build
 
+# Build frontend and sync to frontend-dist for the running container to serve
+build-serve: build
+	@echo "Syncing build to frontend-dist..."
+	@mkdir -p frontend-dist
+	rsync -a --delete $(FRONTEND_DIR)/dist/ frontend-dist/
+	@echo "Done. Hit the Flask backend (port 8084) to test the production build."
+
 # Preview production build
 preview:
 	@echo "Previewing production build..."
 	cd $(FRONTEND_DIR) && npm run preview
 
 # Type checking
-typecheck:
+frontend-typecheck:
 	@echo "Running TypeScript type checking..."
 	cd $(FRONTEND_DIR) && npm run typecheck
 
+# Python linting (backend + tests)
+python-lint:
+	@echo "Running Ruff..."
+	uv run ruff check shelfmark tests
+
+python-lint-fix:
+	@echo "Running Ruff with safe auto-fixes..."
+	uv run ruff check shelfmark tests --fix
+
+python-format:
+	@echo "Checking Python formatting with Ruff..."
+	uv run ruff format --check shelfmark tests
+
+python-format-fix:
+	@echo "Formatting Python code with Ruff..."
+	uv run ruff format shelfmark tests
+
+python-typecheck:
+	@echo "Running BasedPyright..."
+	uv run basedpyright
+	@echo "Running BasedPyright against tests..."
+	uv run basedpyright tests --skipunannotated
+
+python-dead-code:
+	@echo "Running Vulture..."
+	uv run vulture shelfmark
+
+python-checks: python-lint python-format python-typecheck python-dead-code
+
+python-test:
+	@echo "Running tests..."
+	uv run pytest tests/ -x --tb=short -m "not integration and not e2e"
+
+python-test-cov:
+	@echo "Running tests with coverage..."
+	uv run pytest tests/ -x --tb=short -m "not integration and not e2e" --cov --cov-report=term-missing
+
+# E2E docker platform: hermetic stack (mock AA/Cloudflare/bypasser/DNS/proxy/Tor)
+# exercised across config profiles. See tests/e2e/platform/README.md.
+E2E_PLATFORM_DIR := tests/e2e/platform
+
+e2e-platform:
+	@echo "Running e2e platform (baseline profile)..."
+	cd $(E2E_PLATFORM_DIR) && ./run-e2e.sh env/baseline.env
+
+e2e-platform-profile:
+	@echo "Running e2e platform (profile=$(PROFILE))..."
+	cd $(E2E_PLATFORM_DIR) && ./run-e2e.sh env/$(PROFILE).env
+
+e2e-platform-matrix:
+	@echo "Running e2e platform matrix (all profiles)..."
+	cd $(E2E_PLATFORM_DIR) && ./run-matrix.sh
+
+e2e-platform-build:
+	@echo "Pre-building e2e platform images once (reused across profiles)..."
+	cd $(E2E_PLATFORM_DIR) && ./build-images.sh
+
+e2e-platform-full:
+	@echo "Running e2e platform FULL profile (real Chrome bypasser + DoH + real qBittorrent)..."
+	cd $(E2E_PLATFORM_DIR) && ./run-e2e.sh env/full.env
+
+# Frontend linting
+frontend-lint:
+	@echo "Running Oxlint..."
+	cd $(FRONTEND_DIR) && npm run lint
+
+# Frontend formatting
+frontend-format:
+	@echo "Checking frontend formatting with Oxfmt..."
+	cd $(FRONTEND_DIR) && npm run format:check
+
+frontend-format-fix:
+	@echo "Formatting frontend code with Oxfmt..."
+	cd $(FRONTEND_DIR) && npm run format
+
+# All frontend static analysis
+frontend-checks: frontend-lint frontend-format frontend-typecheck
+
+# Run frontend unit tests
+frontend-test:
+	@echo "Running frontend unit tests..."
+	cd $(FRONTEND_DIR) && npm run test:unit
+
+# All static analysis checks (frontend + Python)
+checks: frontend-checks python-checks
+
+# Auto-fix lint + format issues (frontend + Python)
+fix: python-lint-fix python-format-fix frontend-format-fix
+
 # Clean build artifacts and dependencies
 clean:
 	@echo "Cleaning build artifacts and dependencies..."

compose/docker-compose.lite.yml

@@ -1,6 +1,7 @@
 services:
   shelfmark-lite:
     image: ghcr.io/calibrain/shelfmark-lite:latest
+    container_name: shelfmark-lite
     environment:
       # EXT_BYPASSER_URL: http://flaresolverr:8191 #If using Flaresolverr
       PUID: 1000
@@ -12,4 +13,4 @@ services:
       - /path/to/books:/books # Default destination for book downloads
       - /path/to/config:/config # App configuration
       # Required for torrent / usenet - path must match your download client's volume exactly
-      # - /path/to/downloads:/path/to/downloads
+      # - /path/to/downloads:/path/to/downloads

compose/docker-compose.tor.yml

@@ -1,4 +1,4 @@
-# Routes all traffic through Tor - requires NET_ADMIN capability
+# Routes all traffic through Tor - requires root startup
 services:
   shelfmark-tor:
     image: ghcr.io/calibrain/shelfmark:latest

compose/docker-compose.wireguard.yml

@@ -0,0 +1,43 @@
+# Routes all traffic through a WireGuard tunnel - requires root startup.
+#
+# Mount your wg-quick config at /config/wg0.conf (read-only is fine). All
+# non-LAN egress is forced through the tunnel by an iptables kill-switch, so if
+# the tunnel drops, external traffic fails closed. LAN ranges (WebUI + internal
+# download clients like Prowlarr / qBittorrent) stay reachable off-tunnel.
+services:
+  shelfmark-wireguard:
+    image: ghcr.io/calibrain/shelfmark:latest
+    environment:
+      FLASK_PORT: 8084
+      # Quoted so it is passed as the literal string "true": entrypoint.sh compares
+      # $USING_WIREGUARD against "true", and some Compose implementations stringify
+      # a bare YAML boolean as "True", which would silently NOT enable WireGuard.
+      USING_WIREGUARD: "true"
+      # Path to the mounted wg-quick config (default shown).
+      WIREGUARD_CONFIG: /config/wg0.conf
+      # CIDRs kept OFF the tunnel so the WebUI and internal clients stay reachable.
+      LAN_NETWORK: 127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
+      PUID: 1000
+      PGID: 1000
+    cap_add:
+      - NET_ADMIN
+      - NET_RAW
+    # WireGuard needs the module/kernel routing; NET_ADMIN covers wg-quick.
+    sysctls:
+      - net.ipv4.conf.all.src_valid_mark=1
+      # Disable IPv6 in the container so the kill-switch can guarantee no IPv6
+      # leak path on kernels/containers without a usable ip6tables. wireguard.sh
+      # fails closed if IPv6 is neither kill-switched nor disabled. If your host
+      # DOES have a working ip6tables you may omit these (an ip6tables kill-switch
+      # is installed instead); or set WIREGUARD_ALLOW_IPV6_LEAK=true only if the
+      # container genuinely has no IPv6 connectivity.
+      - net.ipv6.conf.all.disable_ipv6=1
+      - net.ipv6.conf.default.disable_ipv6=1
+    ports:
+      - 8084:8084
+    restart: unless-stopped
+    volumes:
+      - /path/to/books:/books # Default destination for book downloads
+      - /path/to/config:/config # App configuration (put wg0.conf here)
+      # Required for torrent / usenet - path must match your download client's volume exactly
+      # - /path/to/downloads:/path/to/downloads

data/book-languages.json

@@ -1,72 +1,79 @@
 [
-    { "language": "English", "code": "en" },
-    { "language": "Chinese", "code": "zh" },
-    { "language": "Russian", "code": "ru" },
-    { "language": "Spanish", "code": "es" },
-    { "language": "French", "code": "fr" },
-    { "language": "German", "code": "de" },
-    { "language": "Italian", "code": "it" },
-    { "language": "Portuguese", "code": "pt" },
-    { "language": "Polish", "code": "pl" },
-    { "language": "Bulgarian", "code": "bg" },
-    { "language": "Dutch", "code": "nl" },
-    { "language": "Japanese", "code": "ja" },
-    { "language": "Arabic", "code": "ar" },
-    { "language": "Hebrew", "code": "he" },
-    { "language": "Turkish", "code": "tr" },
-    { "language": "Hungarian", "code": "hu" },
-    { "language": "Latin", "code": "la" },
-    { "language": "Czech", "code": "cs" },
-    { "language": "Korean", "code": "ko" },
-    { "language": "Ukrainian", "code": "uk" },
-    { "language": "Indonesian", "code": "id" },
-    { "language": "Romanian", "code": "ro" },
-    { "language": "Swedish", "code": "sv" },
-    { "language": "Greek", "code": "el" },
-    { "language": "Lithuanian", "code": "lt" },
-    { "language": "Bangla", "code": "bn" },
-    { "language": "Traditional Chinese", "code": "zh‑Hant" },
-    { "language": "Afrikaans", "code": "af" },
-    { "language": "Catalan", "code": "ca" },
-    { "language": "Danish", "code": "da" },
-    { "language": "Thai", "code": "th" },
-    { "language": "Hindi", "code": "hi" },
-    { "language": "Irish", "code": "ga" },
-    { "language": "Latvian", "code": "lv" },
+    { "language": "English", "code": "en", "aliases": ["eng"] },
+    { "language": "Chinese", "code": "zh", "aliases": ["chi", "zho"] },
+    { "language": "Russian", "code": "ru", "aliases": ["rus"] },
+    { "language": "Spanish", "code": "es", "aliases": ["spa"] },
+    { "language": "French", "code": "fr", "aliases": ["fra", "fre"] },
+    { "language": "German", "code": "de", "aliases": ["deu", "ger"] },
+    { "language": "Italian", "code": "it", "aliases": ["ita"] },
+    { "language": "Portuguese", "code": "pt", "aliases": ["por"] },
+    { "language": "Polish", "code": "pl", "aliases": ["pol"] },
+    { "language": "Bulgarian", "code": "bg", "aliases": ["bul"] },
+    { "language": "Dutch", "code": "nl", "aliases": ["dut", "nld"] },
+    { "language": "Japanese", "code": "ja", "aliases": ["jap", "jpn"] },
+    { "language": "Arabic", "code": "ar", "aliases": ["ara"] },
+    { "language": "Hebrew", "code": "he", "aliases": ["heb"] },
+    { "language": "Turkish", "code": "tr", "aliases": ["tur"] },
+    { "language": "Hungarian", "code": "hu", "aliases": ["hun"] },
+    { "language": "Latin", "code": "la", "aliases": ["lat"] },
+    { "language": "Czech", "code": "cs", "aliases": ["ces", "cze"] },
+    { "language": "Korean", "code": "ko", "aliases": ["kor"] },
+    { "language": "Ukrainian", "code": "uk", "aliases": ["ukr"] },
+    { "language": "Indonesian", "code": "id", "aliases": ["ind"] },
+    { "language": "Romanian", "code": "ro", "aliases": ["rom", "ron"] },
+    { "language": "Swedish", "code": "sv", "aliases": ["swe"] },
+    { "language": "Greek", "code": "el", "aliases": ["ell", "gre"] },
+    { "language": "Lithuanian", "code": "lt", "aliases": ["lit"] },
+    { "language": "Bangla", "code": "bn", "aliases": ["ben", "bengali"] },
+    { "language": "Traditional Chinese", "code": "zh-Hant", "aliases": ["zh‑Hant"] },
+    { "language": "Afrikaans", "code": "af", "aliases": ["afr"] },
+    { "language": "Catalan", "code": "ca", "aliases": ["cat"] },
+    { "language": "Danish", "code": "da", "aliases": ["dan"] },
+    { "language": "Thai", "code": "th", "aliases": ["tha"] },
+    { "language": "Hindi", "code": "hi", "aliases": ["hin"] },
+    { "language": "Irish", "code": "ga", "aliases": ["gle"] },
+    { "language": "Latvian", "code": "lv", "aliases": ["lav"] },
     { "language": "Tibetan", "code": "bo" },
-    { "language": "Kannada", "code": "kn" },
-    { "language": "Serbian", "code": "sr" },
-    { "language": "Persian", "code": "fa" },
-    { "language": "Croatian", "code": "hr" },
+    { "language": "Kannada", "code": "kn", "aliases": ["kan"] },
+    { "language": "Serbian", "code": "sr", "aliases": ["srp"] },
+    { "language": "Persian", "code": "fa", "aliases": ["farsi", "fas", "per"] },
+    { "language": "Croatian", "code": "hr", "aliases": ["hrv"] },
     { "language": "Slovak", "code": "sk" },
-    { "language": "Javanese", "code": "jv" },
-    { "language": "Vietnamese", "code": "vi" },
-    { "language": "Urdu", "code": "ur" },
-    { "language": "Finnish", "code": "fi" },
-    { "language": "Norwegian", "code": "no" },
+    { "language": "Javanese", "code": "jv", "aliases": ["jav"] },
+    { "language": "Vietnamese", "code": "vi", "aliases": ["vie"] },
+    { "language": "Urdu", "code": "ur", "aliases": ["urd"] },
+    { "language": "Finnish", "code": "fi", "aliases": ["fin"] },
+    { "language": "Norwegian", "code": "no", "aliases": ["nor"] },
     { "language": "Kinyarwanda", "code": "rw" },
-    { "language": "Tamil", "code": "ta" },
+    { "language": "Tamil", "code": "ta", "aliases": ["tam"] },
     { "language": "Belarusian", "code": "be" },
     { "language": "Kazakh", "code": "kk" },
     { "language": "Mongolian", "code": "mn" },
     { "language": "Georgian", "code": "ka" },
-    { "language": "Slovenian", "code": "sl" },
+    { "language": "Slovenian", "code": "sl", "aliases": ["slv"] },
     { "language": "Esperanto", "code": "eo" },
     { "language": "Galician", "code": "gl" },
-    { "language": "Marathi", "code": "mr" },
-    { "language": "Filipino", "code": "fil" },
-    { "language": "Gujarati", "code": "gu" },
-    { "language": "Malayalam", "code": "ml" },
+    { "language": "Marathi", "code": "mr", "aliases": ["mar"] },
+    { "language": "Filipino", "code": "fil", "aliases": ["tagalog", "tgl"] },
+    { "language": "Gujarati", "code": "gu", "aliases": ["guj"] },
+    { "language": "Malayalam", "code": "ml", "aliases": ["mal"] },
     { "language": "Kyrgyz", "code": "ky" },
     { "language": "Azerbaijani", "code": "az" },
     { "language": "Quechua", "code": "qu" },
     { "language": "Swahili", "code": "sw" },
     { "language": "Bashkir", "code": "ba" },
-    { "language": "Punjabi", "code": "pa" },
-    { "language": "Malay", "code": "ms" },
-    { "language": "Telugu", "code": "te" },
+    { "language": "Punjabi", "code": "pa", "aliases": ["pan"] },
+    { "language": "Malay", "code": "ms", "aliases": ["may", "msa"] },
+    { "language": "Telugu", "code": "te", "aliases": ["tel"] },
     { "language": "Albanian", "code": "sq" },
     { "language": "Uyghur", "code": "ug" },
     { "language": "Armenian", "code": "hy" },
-    { "language": "Shan", "code": "shn" }
-]
+    { "language": "Shan", "code": "shn" },
+    { "language": "Bosnian", "code": "bs", "aliases": ["bos"] },
+    { "language": "Burmese", "code": "my", "aliases": ["bur", "mya"] },
+    { "language": "Estonian", "code": "et", "aliases": ["est"] },
+    { "language": "Icelandic", "code": "is", "aliases": ["ice", "isl"] },
+    { "language": "Manx", "code": "gv", "aliases": ["glv"] },
+    { "language": "Scottish Gaelic", "code": "gd", "aliases": ["gla"] },
+    { "language": "Sanskrit", "code": "sa", "aliases": ["san"] }
+]

docker-compose.bypass-test.yml

@@ -1,39 +0,0 @@
-# Bypass testing - switch between dev build and v1.0.1
-# Usage:
-#   Test dev build:  docker compose -f docker-compose.bypass-test.yml up shelfmark-dev
-#   Test v1.0.1:     docker compose -f docker-compose.bypass-test.yml up shelfmark-stable
-#   Pull latest dev: docker compose -f docker-compose.bypass-test.yml build shelfmark-dev
-#   Pull v1.0.1:     docker compose -f docker-compose.bypass-test.yml pull shelfmark-stable
-
-services:
-  # Dev image from registry
-  shelfmark-dev:
-    image: ghcr.io/calibrain/shelfmark:dev
-    container_name: shelfmark-bypass-dev
-    environment:
-      PUID: 1000
-      PGID: 1000
-      DEBUG: true
-    ports:
-      - 8084:8084
-    volumes:
-      - ./.local/bypass-test/config-dev:/config
-      - ./.local/bypass-test/books:/books
-      - ./.local/bypass-test/log-dev:/var/log/shelfmark
-      - ./.local/bypass-test/tmp:/tmp/shelfmark
-
-  # Stable v1.0.1 for comparison
-  shelfmark-stable:
-    image: ghcr.io/calibrain/shelfmark:1.0.1
-    container_name: shelfmark-bypass-stable
-    environment:
-      PUID: 1000
-      PGID: 1000
-      DEBUG: true
-    ports:
-      - 8085:8084
-    volumes:
-      - ./.local/bypass-test/config-stable:/config
-      - ./.local/bypass-test/books:/books
-      - ./.local/bypass-test/log-stable:/var/log/shelfmark
-      - ./.local/bypass-test/tmp:/tmp/shelfmark

docker-compose.dev.wireguard.yml

@@ -0,0 +1,25 @@
+# Local development - WireGuard variant
+services:
+  shelfmark-wireguard-dev:
+    extends:
+      file: ./compose/docker-compose.wireguard.yml
+      service: shelfmark-wireguard
+    build:
+      context: .
+      dockerfile: Dockerfile
+      target: shelfmark
+    environment:
+      # Quoted so they are passed as the literal string "true" (entrypoint.sh and
+      # the app compare against "true"); a bare YAML boolean can be stringified as
+      # "True" by some Compose variants, silently disabling the feature.
+      DEBUG: "true"
+      USING_WIREGUARD: "true"
+      WIREGUARD_CONFIG: /config/wg0.conf
+    volumes:
+      - ./.local/config:/config
+      - ./.local/books:/books
+      - ./.local/log:/var/log/shelfmark
+      - ./.local/tmp:/tmp/shelfmark
+      # Place your wg-quick config at ./.local/config/wg0.conf
+      # Required for torrent / usenet - path must match your download client's volume exactly
+      # - /path/to/downloads:/path/to/downloads

docker-compose.dev.yml

@@ -12,11 +12,14 @@ services:
       - SYS_PTRACE
     environment:
       DEBUG: true
+      # HIDE_LOCAL_AUTH: true
+      OIDC_AUTO_REDIRECT: true
     volumes:
       - ./.local/config:/config
       - ./.local/books:/books
       - ./.local/log:/var/log/shelfmark
       - ./.local/tmp:/tmp/shelfmark
       - ./shelfmark:/app/shelfmark:ro
+      - ./frontend-dist:/app/frontend-dist:ro
       # Required for torrent / usenet - path must match your download client's volume exactly
       # - /path/to/downloads:/path/to/downloads

docker-compose.test-clients.yml

@@ -13,6 +13,7 @@
 #   - Transmission: http://localhost:9091 (admin / admin)
 #   - Deluge:       http://localhost:8112 (password: deluge)
 #   - NZBGet:       http://localhost:6789 (nzbget / tegbzn6789)
+#   - NZBHydra:     http://localhost:5076 (no auth by default)
 #   - SABnzbd:      http://localhost:8085 (complete setup wizard for API key)
 #   - rTorrent:     http://localhost:8000 (admin / admin - if auth enabled)
 #
@@ -35,6 +36,7 @@ services:
       #   - Transmission: http://transmission:9091
       #   - Deluge Web UI: http://deluge:8112
       #   - NZBGet: http://nzbget:6789
+      #   - NZBHydra: http://nzbhydra:5076
       #   - SABnzbd: http://sabnzbd:8080
       #   - rTorrent: http://rtorrent:80 (XMLRPC via HTTP) or rtorrent (port 5000 for SCGI)
     ports:
@@ -54,11 +56,13 @@ services:
       # Mount tests for running pytest in container
       - ./tests:/app/tests:ro
       - ./pyproject.toml:/app/pyproject.toml:ro
+      - ./uv.lock:/app/uv.lock:ro
       # Mount client configs for integration tests to read credentials
       - ./.local/test-clients/qbittorrent/config:/qbittorrent-config:ro
       - ./.local/test-clients/sabnzbd/config:/sabnzbd-config:ro
     depends_on:
       - nzbget
+      - nzbhydra
       - sabnzbd
       - qbittorrent
       - transmission
@@ -179,3 +183,18 @@ services:
       - "50000:50000"  # Incoming connections
       - "6881:6881/udp"
     restart: unless-stopped
+
+  nzbhydra:
+    image: lscr.io/linuxserver/nzbhydra2:latest
+    container_name: nzbhydra
+    environment:
+      - PUID=1000
+      - PGID=1000
+      - TZ=Europe/London
+    volumes:
+      - ./.local/test-clients/nzbhydra/config:/config
+      - ./.local/test-clients/downloads:/downloads
+    ports:
+      - 5076:5076
+    restart: unless-stopped
+

docs/api-access.md

@@ -0,0 +1,72 @@
+# API access with an API key
+
+Shelfmark's web interface is driven entirely by a JSON API under `/api/`. Set
+the `SHELFMARK_API_KEY` environment variable and scripts, dashboards and assistants can
+call the same API without a browser session. Browser logins keep working
+exactly as before: it is cookie **or** key.
+
+## Set the key
+
+```yaml
+environment:
+  SHELFMARK_API_KEY: "a-long-random-secret"
+```
+
+Generate something long and random (for example `openssl rand -base64 32`).
+A request carrying the key acts as an **admin**: the first admin user in
+Shelfmark's user database. Create an admin before relying on the key in any
+install that has none yet (for example an OIDC-only install). Without an
+admin user, the key still authenticates as an admin identity with no user
+row, and routes that need one (requests, activity) answer 403. To rotate,
+change the variable and restart. Unset it and the feature is off. When the
+instance runs with no authentication configured (`AUTH_METHOD=none`), the
+key is simply unnecessary.
+
+## Send the key
+
+Either header works, and both are checked, so the key can be sent in
+`X-Api-Key` behind a reverse proxy that sets its own `Authorization` header.
+
+```bash
+curl -s -H "Authorization: Bearer $SHELFMARK_API_KEY" https://shelfmark.example.com/api/downloads/active
+curl -s -H "X-Api-Key: $SHELFMARK_API_KEY" https://shelfmark.example.com/api/downloads/active
+```
+
+A request that carries the key is authenticated by the key alone. Session
+cookies are ignored and none are set. A bearer value that is not the configured
+key is ignored and the request continues with normal session authentication,
+so reverse proxies that forward their own tokens are unaffected; without a valid
+session such a request gets the usual `401 {"error": "Unauthorized"}`. A
+database error while resolving the admin returns
+`500 {"error": "Authentication error"}` — never anonymous access.
+`/api/auth/check` reflects the browser session only and ignores the key, so
+use `/api/status` to verify a key.
+
+## Examples
+
+Search, then look up releases, then queue one (the same calls the web UI makes):
+
+```bash
+curl -s -H "Authorization: Bearer $SHELFMARK_API_KEY" \
+  "https://shelfmark.example.com/api/metadata/search?query=dune%20frank%20herbert"
+# -> {"books":[{"provider":"hardcover","provider_id":"427363", ...}]}
+
+curl -s -H "Authorization: Bearer $SHELFMARK_API_KEY" \
+  "https://shelfmark.example.com/api/releases?provider=hardcover&book_id=427363&content_type=ebook"
+# -> {"releases":[{"source":"direct_download","source_id":"...", ...}], ...}
+
+curl -s -X POST -H "Authorization: Bearer $SHELFMARK_API_KEY" -H "Content-Type: application/json" \
+  -d @release.json https://shelfmark.example.com/api/releases/download
+# release.json = one object from "releases" (source and source_id are required)
+
+curl -s -H "Authorization: Bearer $SHELFMARK_API_KEY" https://shelfmark.example.com/api/status
+```
+
+## Security notes
+
+- The key is compared in constant time and is never logged.
+- Keyed requests never set cookies and ignore any cookie sent with them.
+- WebSocket (live activity) connections do not accept the key; poll `/api/status` instead.
+- The key is a root-equivalent credential: an admin can configure a custom
+  post-download script that the server executes, so treat it like a root
+  password and send it only over HTTPS.

docs/configuration.md

@@ -18,9 +18,9 @@ Prowlarr -> Download client saves to <client path>
 
 Key point: For torrent and usenet downloads, Shelfmark must see the same file path that your download client reports. The container path must match in both containers.
 
-## Direct Download Setup
+## Direct Download Volume Setup
 
-Direct downloads do not use an external download client. A simple two-folder setup is enough.
+If you plan to use Direct Download, it does not use an external download client. A simple two-folder setup is enough.
 
 Required volumes:
 
@@ -43,7 +43,12 @@ services:
 Notes:
 - Point `/books` to your library ingest folder (Calibre-Web, Booklore, Audiobookshelf, etc) for automatic import.
 - If you set Books Output Mode to Booklore (API), books are uploaded via API instead of written to `/books`. Audiobooks still use a destination folder.
-- Ensure `PUID`/`PGID` (or legacy `UID`/`GID`) match the owner of the host directories to avoid permission errors.
+- Ensure `PUID`/`PGID` (or legacy `UID`/`GID`) match the owner of the host directories.
+- For non-root mode, start the container as `1000:1000`.
+- On Kubernetes, set `runAsUser: 1000`, `runAsGroup: 1000`, and `runAsNonRoot: true` together.
+- `PUID`/`PGID` keep the default root startup flow.
+- In non-root mode, mounted paths must already be writable by `1000:1000`.
+- `USING_TOR=true` requires root startup.
 
 ## Torrent / Usenet Setup
 
@@ -86,6 +91,10 @@ Example:
 - Shelfmark can see the same files at `/downloads/books/...`
 - Add a mapping from Remote Path `/data/torrents` to Local Path `/downloads`
 
+If the files are copied or synced into Shelfmark on a delay, increase **Completed Path Wait (seconds)**
+in Settings -> Advanced. The default is 60 seconds; seedbox or remote-sync setups may need a value
+longer than the sync interval.
+
 ## File Processing Options
 
 ### Transfer Method (Torrent / Usenet Only)
@@ -113,6 +122,7 @@ Configure templates in Settings -> Downloads. Template syntax details are docume
 
 - "Download failed - file not found": Path mismatch between Shelfmark and the download client. Ensure container paths match or use Remote Path Mappings.
 - "Permission denied": `PUID`/`PGID` do not match the host directories. Ensure Shelfmark can read the client path and write to the destination.
+- "Permission denied" in non-root Docker/Kubernetes mode: ensure the mounted path is writable by UID/GID `1000:1000`, or switch back to root startup with `PUID`/`PGID`.
 - "Hardlinks not working" or "Files being copied instead": Source and destination are on different filesystems. Move the destination or accept copy fallback.
 - "Downloads work but library does not see them": Destination does not point to the library ingest folder. Check Settings -> Downloads -> Destination.
 - CIFS/SMB shares: Use the `nobrl` mount option to avoid database lock errors. Example: `//server/share /mnt/share cifs nobrl,... 0 0`

docs/custom-scripts.md

@@ -121,15 +121,26 @@ Example payload shape:
 }
 ```
 
-Example (bash + jq) (JSON payload must be enabled):
+Example (bash + python3) (JSON payload must be enabled):
 
 ```bash
 payload="$(cat)"
-mode="$(echo "$payload" | jq -r '.output.mode')"
-title="$(echo "$payload" | jq -r '.task.title')"
-final_paths="$(echo "$payload" | jq -r '.paths.final_paths[]')"
-echo "mode=$mode title=$title" >&2
-echo "$final_paths" >&2
+target="$1"
+PAYLOAD="$payload" TARGET="$target" python3 - <<'PY'
+import json
+import os
+import sys
+
+payload = json.loads(os.environ["PAYLOAD"])
+
+print(f"target={os.environ['TARGET']}", file=sys.stderr)
+print(
+    f"mode={payload['output']['mode']} title={payload['task']['title']}",
+    file=sys.stderr,
+)
+for path in payload["paths"]["final_paths"]:
+    print(path, file=sys.stderr)
+PY
 ```
 
 Example (Python) (works whether JSON payload is enabled or not):

docs/dev/release-sources-plugin-guide.md

@@ -276,6 +276,27 @@ class DownloadHandler(ABC):
         pass
 ```
 
+### Optional: Listing Files Before Download
+
+Some releases bundle several books (a whole-series torrent). Shelfmark inspects a
+release before queueing it so the user can review how it will be split into books.
+Override `list_files` when your source can enumerate a release's files without
+downloading it; the default returns `None`, which the UI reports as "can't inspect":
+
+```python
+from shelfmark.download.postprocess.packs import PackFile
+
+def list_files(self, release_data: dict[str, Any]) -> list[PackFile] | None:
+    """Return the release's files (release-relative paths + sizes), or None."""
+    torrent_bytes = ...  # e.g. fetch the .torrent, or scrape the indexer's detail page
+    return extract_file_list_from_torrent(torrent_bytes)  # from download.clients.torrent_utils
+```
+
+`release_data` is the same payload the frontend sends to `/api/releases/download`
+(`source_id`, `download_url`, `content_type`, `series_name`, ...). Built-in examples:
+Prowlarr parses the `.torrent` it already fetches (magnet-only releases return
+`None`), and AudiobookBay reads the file table off its detail page.
+
 ### Download Method Parameters
 
 | Parameter | Type | Description |

docs/index.md

@@ -1,3 +1,24 @@
 # Shelfmark Documentation
 
-TODO
+Shelfmark is a self-hosted interface for searching, requesting, and delivering books and audiobooks through the sources and services you choose to configure.
+
+Use the guides below to set up the app, connect your library tools, and understand the main configuration areas.
+
+## Getting Started
+
+- [Installation](installation.md)
+- [Directory and Volume Setup](configuration.md)
+- [Environment Variables](environment-variables.md)
+
+## Core Guides
+
+- [Users & Requests](users-and-requests.md)
+- [Reverse Proxy](reverse-proxy.md)
+- [OIDC](oidc.md)
+- [API Access](api-access.md)
+- [URL Search Parameters](url-search-parameters.md)
+- [Custom Scripts](custom-scripts.md)
+
+## Help
+
+- [Troubleshooting](troubleshooting.md)

docs/installation.md

@@ -1,3 +1,33 @@
 # Installation
 
-TODO
+Shelfmark is typically deployed with Docker Compose.
+
+## Quick Start
+
+1. Download the compose file from the repository:
+
+```bash
+curl -O https://raw.githubusercontent.com/calibrain/shelfmark/main/compose/docker-compose.yml
+```
+
+2. Start the service:
+
+```bash
+docker compose up -d
+```
+
+3. Open `http://localhost:8084`
+
+4. Configure the sources, metadata providers, and delivery settings you want to use
+
+## Next Steps
+
+- For volume and path setup, see [Directory and Volume Setup](configuration.md)
+- For environment-based setup, see [Environment Variables](environment-variables.md)
+- For authentication and user management, see [Users & Requests](users-and-requests.md) and [OIDC](oidc.md)
+
+## Notes
+
+- Universal search is the default mode for new installs
+- Direct Download is optional and must be enabled and configured before it can be used
+- Torrent and usenet setups require matching download paths between Shelfmark and your download client

docs/oidc.md

@@ -0,0 +1,65 @@
+# OpenID Connect (OIDC) Authentication
+
+## Callback URL
+
+```
+https://<your-shelfmark-domain>/api/auth/oidc/callback
+```
+
+With a subpath (`URL_BASE=/shelfmark/`):
+
+```
+https://<your-shelfmark-domain>/shelfmark/api/auth/oidc/callback
+```
+
+The callback URL is constructed from the incoming request, so your reverse proxy must forward `X-Forwarded-Proto` and `X-Forwarded-Host` correctly, including the external port when it is not the protocol default. PKCE (S256) is used automatically.
+
+## Settings
+
+Configure in **Settings → Security → Authentication Method → OIDC**.
+
+| Setting | Description | Default |
+|---------|-------------|---------|
+| Discovery URL | `/.well-known/openid-configuration` endpoint | — |
+| Client ID | OAuth2 client ID | — |
+| Client Secret | OAuth2 client secret | — |
+| Scopes | Scopes to request. The group claim is added automatically when admin group authorization is enabled | `openid email profile` |
+| Group Claim Name | Claim containing user groups | `groups` |
+| Admin Group Name | Group granted admin access. Leave empty for database-only roles | — |
+| Use Admin Group for Authorization | Toggle group-based admin detection | `true` |
+| Auto-Provision Users | Create accounts on first login | `true` |
+| Login Button Label | Custom text for the sign-in button | — |
+
+Use **Test Connection** to verify discovery, client configuration, and the provider's token signing keys (JWKS) before attempting login.
+
+> **Authentik users:** make sure your provider has a **Signing Key** selected (e.g. the default self-signed certificate). Without one, Authentik serves an empty JWKS document and every login fails with an OIDC callback error, even though the discovery document looks healthy.
+
+## Account Linking
+
+On login, Shelfmark matches the OIDC identity to a user account in this order:
+
+1. **OIDC subject** — a user who has logged in through this provider before.
+2. **Email** — a local account with the same (unique) email address. This only happens when the provider also asserts `email_verified: true` for the address; an unverified email would let anyone claim a local account by registering its address at the IdP.
+3. Otherwise, a new account is created when **Auto-Provision Users** is enabled (username conflicts get a numeric suffix), or the login is rejected with "Account not found" when it is disabled.
+
+If the `email_verified` claim is missing or `false`, email linking is silently skipped — a common surprise when the address was never verified at the identity provider (e.g. Keycloak's **Email verified** toggle on the user, or Authentik accounts created without email verification). Make sure the `email` scope is requested and the address is marked verified in your IdP.
+
+## Environment Variables
+
+These optional environment variables control login page behavior when OIDC is enabled.
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `HIDE_LOCAL_AUTH` | Hide the username/password login option, so only the OIDC button is shown | `false` |
+| `DISABLE_LOCAL_AUTH` | Disable username/password login and remove the local-admin prerequisite for OIDC. Implies `HIDE_LOCAL_AUTH`; with `AUTH_METHOD=builtin`, everyone is locked out until auth env vars are changed. | `false` |
+| `OIDC_AUTO_REDIRECT` | Automatically redirect to the OIDC provider instead of showing the login page | `false` |
+
+If `DISABLE_LOCAL_AUTH` and `OIDC_AUTO_REDIRECT` are both enabled, users are redirected straight to the OIDC provider. On failure they return to the login page with an error message but no password fallback.
+
+## Troubleshooting
+
+- **No token signing keys (empty JWKS)** — The provider's JWKS endpoint returned no keys, so ID tokens can't be verified. In Authentik this happens when the provider has no **Signing Key** selected; pick one (e.g. the default self-signed certificate) and try again.
+- **Issuer validation failed** — The issuer in the token doesn't match the discovery document. Check your provider's external URL / issuer configuration.
+- **Callback URL mismatch** — Reverse proxy isn't forwarding `X-Forwarded-Proto` or `X-Forwarded-Host`, so the constructed callback URL doesn't match what's registered in the provider.
+- **Account not found** — Auto-provision is disabled and the user hasn't been pre-created by an admin. If you pre-created the account with a matching email, see [Account Linking](#account-linking): the provider must send `email_verified: true` for linking to happen.
+- **Login created a duplicate account instead of linking to my local one** — Email linking requires a verified email; see [Account Linking](#account-linking). With `DEBUG=true`, the log notes when linking is skipped because the address isn't verified.

docs/reverse-proxy.md

@@ -6,6 +6,15 @@ Shelfmark can run behind a reverse proxy at the root path (recommended) or under
 
 If you can serve Shelfmark at the root path (`https://shelfmark.example.com/`), leave `URL_BASE` empty. This is the simplest option and avoids extra subpath configuration.
 
+Define this once in your Nginx `http` block so websocket upgrades are only sent when the client actually requests them:
+
+```nginx
+map $http_upgrade $connection_upgrade {
+    default upgrade;
+    ''      close;
+}
+```
+
 ```nginx
 server {
     listen 443 ssl;
@@ -14,12 +23,13 @@ server {
     location / {
         proxy_pass http://shelfmark:8084;
         proxy_http_version 1.1;
-        proxy_set_header Host $host;
+        proxy_set_header Host $http_host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header X-Forwarded-Proto $scheme;
+        proxy_set_header X-Forwarded-Host $http_host;
         proxy_set_header Upgrade $http_upgrade;
-        proxy_set_header Connection "upgrade";
+        proxy_set_header Connection $connection_upgrade;
     }
 }
 ```
@@ -47,13 +57,13 @@ All Shelfmark paths (UI, API, assets, Socket.IO) are served under the base path.
 location /shelfmark/ {
     proxy_pass http://shelfmark:8084/shelfmark/;
     proxy_http_version 1.1;
-    proxy_set_header Host $host;
+    proxy_set_header Host $http_host;
     proxy_set_header X-Real-IP $remote_addr;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header X-Forwarded-Proto $scheme;
-    proxy_set_header X-Forwarded-Host $host;
+    proxy_set_header X-Forwarded-Host $http_host;
     proxy_set_header Upgrade $http_upgrade;
-    proxy_set_header Connection "upgrade";
+    proxy_set_header Connection $connection_upgrade;
     proxy_read_timeout 86400;
     proxy_send_timeout 86400;
     proxy_buffering off;
@@ -77,6 +87,7 @@ Configure in Settings → Security:
 | Proxy Auth Logout URL | `https://auth.example.com/logout` |
 | Proxy Auth Admin Group Header | `Remote-Groups` |
 | Proxy Auth Admin Group Name | `admins` (or your admin group) |
+| Proxy Auth Default Role | `User` — first-time users are regular users; the very first account is still made admin. Only consulted when no admin group is set |
 
 #### Nginx Configuration with Authelia
 
@@ -127,13 +138,13 @@ location /shelfmark/ {
 
     proxy_pass http://shelfmark:8084/shelfmark/;
     proxy_http_version 1.1;
-    proxy_set_header Host $host;
+    proxy_set_header Host $http_host;
     proxy_set_header X-Real-IP $remote_addr;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header X-Forwarded-Proto $scheme;
-    proxy_set_header X-Forwarded-Host $host;
+    proxy_set_header X-Forwarded-Host $http_host;
     proxy_set_header Upgrade $http_upgrade;
-    proxy_set_header Connection "upgrade";
+    proxy_set_header Connection $connection_upgrade;
     proxy_read_timeout 86400;
     proxy_send_timeout 86400;
     proxy_buffering off;
@@ -142,6 +153,19 @@ location /shelfmark/ {
 
 ---
 
+## Troubleshooting false network errors
+
+If login, settings saves, or downloads appear to fail in the browser but the action still completes on the server, check your proxy headers first.
+
+- Do not force `Connection: upgrade` on every request. That can break normal `POST` and `PUT` responses while the backend still processes them.
+- If your proxy UI does not support conditional websocket headers, remove the forced websocket headers entirely and let Shelfmark fall back to polling.
+- Keep the standard forwarded headers: `Host`, `X-Forwarded-For`, `X-Forwarded-Proto`, and `X-Forwarded-Host` when using a subpath or OIDC.
+- Preserve the original port in `Host` and `X-Forwarded-Host` by using `$http_host` rather than `$host` when Shelfmark is exposed on a custom port.
+
+This is especially relevant for Nginx Proxy Manager or custom advanced config snippets that add websocket headers globally.
+
+---
+
 ## Health checks
 
 Health checks work at `/shelfmark/api/health` when using a subpath configuration.

docs/url-search-parameters.md

@@ -1,75 +1,112 @@
 # URL Search Parameters
 
-You can trigger searches directly via URL by adding query parameters. This enables bookmarking searches and sharing links.
+You can trigger searches directly via URL. This enables bookmarking searches and sharing links.
+
+Parameters live in the URL **hash** (`#…`), so they stay in the browser and are never sent to
+the server. Shelfmark also keeps the hash in sync as you search, so the address bar always
+holds a shareable link to what you're looking at.
 
 ## Basic Usage
 
 ```
-http://your-server:8084/?q=harry+potter
+http://your-server:8084/#q=harry+potter
 ```
 
+Older query-string links (`/?q=harry+potter`) still work: they're read once on load and
+rewritten to the hash form.
+
 ## Supported Parameters
 
 | Parameter | Description | Example |
 |-----------|-------------|---------|
-| `q` or `query` | Main search query | `/?q=dune` |
-| `author` | Filter by author name | `/?author=frank+herbert` |
-| `title` | Filter by book title | `/?title=foundation` |
-| `isbn` | Filter by ISBN | `/?isbn=978-0747532699` |
-| `lang` | Filter by language (ISO 639-1 code) | `/?lang=en` |
-| `format` | Filter by file format | `/?format=epub` |
-| `content` | Filter by content type | `/?content=fiction` |
-| `sort` | Sort order for results | `/?sort=newest` |
+| `q` or `query` | Main search query | `/#q=dune` |
+| `author` | Filter by author name | `/#author=frank+herbert` |
+| `title` | Filter by book title | `/#title=foundation` |
+| `isbn` | Filter by ISBN | `/#isbn=978-0747532699` |
+| `lang` | Filter by language (ISO 639-1 code) | `/#lang=en` |
+| `format` | Filter by file format | `/#format=epub` |
+| `content` | Filter by content type | `/#content=fiction` |
+| `content_type` | Select media type (`ebook`, `audiobook`, or `combined`) in Universal mode only | `/#q=dune&content_type=audiobook` |
+| `sort` | Sort order for results | `/#sort=newest` |
+| `search_by` | "Search By" target the query applies to (`general`, `author`, `title`, `isbn`, a metadata provider field like `series`, or `manual`) | `/#search_by=author&q=frank+herbert` |
 
 ## Multiple Values
 
 Some parameters support multiple values by repeating the parameter:
 
 ```
-/?lang=en&lang=de&lang=fr
-/?format=epub&format=mobi&format=azw3
+/#lang=en&lang=de&lang=fr
+/#format=epub&format=mobi&format=azw3
 ```
 
 ## Examples
 
 **Simple search:**
 ```
-/?q=lord+of+the+rings
+/#q=lord+of+the+rings
 ```
 
 **Search with author filter:**
 ```
-/?q=dune&author=frank+herbert
+/#q=dune&author=frank+herbert
 ```
 
 **Search with format and language:**
 ```
-/?q=harry+potter&format=epub&lang=en
+/#q=harry+potter&format=epub&lang=en
 ```
 
 **Author search with multiple formats:**
 ```
-/?author=stephen+king&format=epub&format=mobi
+/#author=stephen+king&format=epub&format=mobi
 ```
 
 **Search with sort order:**
 ```
-/?q=science+fiction&sort=newest
+/#q=science+fiction&sort=newest
+```
+
+**Universal search as audiobook:**
+```
+/#q=dune&content_type=audiobook
+```
+
+**Universal search forcing combined (ebook + audiobook):**
+```
+/#q=dune&content_type=combined
 ```
 
 ## Search Mode Behavior
 
-### Direct Download Mode (default)
+### Direct Mode
 
-All parameters are used to filter results from the direct download source.
+When Search Mode is set to Direct, all parameters are used to filter results from the configured direct source.
+`content_type` is ignored in Direct mode.
 
 ### Universal Mode
 
-Only `q` and `sort` are used. Other parameters (author, title, format, etc.) are silently ignored since metadata providers have their own search capabilities.
+`q`, `search_by`, `sort`, and `content_type` are used. Other parameters (author, title, format, etc.) are silently ignored since metadata providers have their own search capabilities — except when `search_by` names one of the provider's own search fields, in which case `q` is sent as that field's value.
+
+`content_type=combined` forces combined mode (search ebook and audiobook providers together), overriding the last-used preference. It is silently ignored if combined mode is unavailable (e.g. the combined selector is disabled in settings, or either content type is blocked by request policy).
+
+## Search By
+
+`search_by` picks which target the `q` value is applied to, matching the selector next to the
+search box. It can be deep-linked on its own (`/#search_by=manual`) to open the app in that
+mode with an empty query.
+
+`search_by=manual` fills the search box but does not auto-run: manual search opens the release
+browser from an explicit submit.
+
+A `search_by` naming a target that isn't available (wrong search mode, or a metadata provider
+that doesn't offer that field) is ignored, and the query falls back to a general search.
 
 ## Notes
 
-- URL parameters are read once on page load
-- The URL is not updated when you perform searches manually
+- URL parameters are read once on page load, and again if the hash is replaced in an open tab
+  (e.g. pasting a shared link into the address bar)
+- The hash is kept in sync with the search box, Search By target and filters as you search
 - Spaces should be encoded as `+` or `%20`
 - Invalid or unknown parameters are silently ignored
+- Your last-used Search By target is remembered in browser storage and used when a link
+  doesn't specify one

docs/users-and-requests.md

@@ -0,0 +1,107 @@
+# Users & Requests
+
+Configure in **Settings → Users & Requests**.
+
+## Authentication Methods
+
+Shelfmark supports four authentication methods, configured in **Settings → Security**.
+
+### Local
+
+You create user accounts directly in Shelfmark with a username and password. At least one local admin account must exist before this mode can be enabled.
+
+### Proxy Authentication
+
+Your reverse proxy handles authentication and passes the username to Shelfmark via a header (e.g. `Remote-User`). Accounts are created automatically on first sign-in. If a local user with the same username already exists, the proxy identity will be linked to that account rather than creating a duplicate. Admin status can optionally be derived from a groups header.
+
+### OIDC (OpenID Connect)
+
+Users sign in through your identity provider. Accounts are created automatically on first login (unless auto-provisioning is disabled, in which case you need to pre-create them). If a local user with a matching verified email already exists, the OIDC identity will be linked to that account on first sign-in. Admin status can optionally be derived from a group claim.
+
+A local admin account is required as a fallback. See [OIDC](oidc.md) for provider setup.
+
+### Calibre-Web Database
+
+User accounts are synced from your Calibre-Web `app.db`. If a local user with a matching email already exists, the CWA identity will be linked to that account. Roles are kept in sync with CWA. Users removed from CWA are cleaned up on the next sync.
+
+Requires mounting your Calibre-Web `app.db` to `/auth/app.db`.
+
+## Per-User Settings
+
+Admins can configure per-user settings by editing a user in the user management panel. Non-admin users can also edit their own settings through **My Account** (accessible from the user menu). Admins control which sections are visible in My Account via the **Visible Self-Settings Sections** option.
+
+There are four categories of per-user settings:
+
+### Delivery Preferences
+
+Override where a user's downloads are sent. Options depend on the global output mode configuration:
+
+- **Output mode** — Folder, Email (SMTP), or BookLore (API)
+- **Destination** — A custom folder path for this user's ebook downloads
+- **Audiobook destination** — A custom folder path for audiobook downloads
+- **BookLore library/path** — Per-user BookLore target (when using BookLore output mode)
+- **Email recipient** — Per-user email address (when using Email output mode)
+
+### Search Preferences
+
+Override how a user searches, on top of the global search defaults:
+
+- **Search mode** — Direct or Universal for this user
+- **Default book languages** — The languages a user's searches fall back to when they don't pick one themselves. Useful for a shared instance where readers want different languages.
+- **Metadata providers** — Book, audiobook, and combined-mode provider for this user
+- **Default release sources** — The release tab opened first for books and audiobooks
+
+### Notifications
+
+Users can configure personal notification routes, separate from the global notification settings. Each route targets a URL (e.g. an Apprise-compatible endpoint) and can be scoped to specific event types or all events.
+
+### Request Policy (admin-only)
+
+Admins can override the default ebook/audiobook modes and request rules for individual users. See [Per-User Overrides](#per-user-overrides) below.
+
+---
+
+## Requests
+
+The request system controls whether users can download directly or need admin approval first.
+
+### Policy Modes
+
+Each content type (ebook, audiobook) has a default mode that sets the baseline:
+
+| Mode | Behaviour |
+|------|-----------|
+| **Download** | Users download directly, no approval needed |
+| **Request Release** | Users pick a specific release, then submit it for admin approval |
+| **Request Book** | Users request the book itself — an admin picks the release and fulfils it |
+| **Blocked** | No downloads or requests allowed |
+
+### Settings
+
+| Setting | Description | Default |
+|---------|-------------|---------|
+| Enable Requests | Master toggle. When off, everyone downloads directly | Off |
+| Default Ebook Mode | Baseline mode for all ebook sources | Download |
+| Default Audiobook Mode | Baseline mode for all audiobook sources | Download |
+| Request Rules | Per-source overrides (see below) | None |
+| Max Pending Requests Per User | Open request limit per user | 20 |
+| Allow Notes on Requests | Let users attach a note when submitting | On |
+
+### Request Rules
+
+The rules matrix lets you override the mode for specific source + content type combinations. Rules can only be **equal to or more restrictive** than the content-type default — they cannot grant more access than the baseline.
+
+For example, if the default ebook mode is "Download", a rule can restrict a specific source to "Request Release" or "Blocked", but not the other way around. If no rule matches, the content-type default applies.
+
+### Per-User Overrides
+
+Admins can override the default ebook/audiobook modes and request rules for individual users. Per-user rules are overlaid on the global rules, not replacing them.
+
+### Request Lifecycle
+
+1. User submits a request (book or release level, depending on the resolved policy mode)
+2. Request appears in the admin request queue as **pending**
+3. Admin either **fulfils** (queues a download) or **rejects** the request
+4. For fulfilled requests, delivery state is tracked through the download pipeline
+5. If delivery fails, an admin can reopen the request to try a different release
+6. Users can cancel their own pending requests

entrypoint.sh

@@ -1,5 +1,7 @@
 #!/bin/bash
 
+set -e
+
 is_truthy() {
     case "${1,,}" in
         true|yes|1|y) return 0 ;;
@@ -11,6 +13,16 @@ ENABLE_LOGGING_VALUE="${ENABLE_LOGGING:-true}"
 LOG_PIPE_DIR=""
 LOG_PIPE=""
 TEE_PID=""
+FILE_LOGGING_ENABLED="false"
+CURRENT_UID=$(id -u)
+CURRENT_GID=$(id -g)
+RUN_AS_NON_ROOT="false"
+RUNTIME_TMP_DIR="${TMP_DIR:-/tmp/shelfmark}"
+DEFAULT_RUNTIME_HOME="${RUNTIME_TMP_DIR}/home"
+
+if [ "$CURRENT_UID" != "0" ]; then
+    RUN_AS_NON_ROOT="true"
+fi
 
 start_file_logging() {
     local logfile="$1"
@@ -43,30 +55,84 @@ stop_file_logging() {
 
 if is_truthy "$ENABLE_LOGGING_VALUE"; then
     LOG_DIR=${LOG_ROOT:-/var/log/}/shelfmark
-    mkdir -p "$LOG_DIR"
-    LOG_FILE="${LOG_DIR}/shelfmark_entrypoint.log"
+    if mkdir -p "$LOG_DIR" 2>/dev/null; then
+        LOG_FILE="${LOG_DIR}/shelfmark_entrypoint.log"
+        # Keep the previous entrypoint log instead of deleting all history on boot.
+        rotation_ok="true"
+        if [ -f "${LOG_FILE}.prev" ] && ! rm -f "${LOG_FILE}.prev"; then
+            echo "Warning: could not remove previous entrypoint log ${LOG_FILE}.prev, continuing without file logging" >&2
+            rotation_ok="false"
+        fi
+        if [ "$rotation_ok" = "true" ] && [ -f "$LOG_FILE" ] && ! mv "$LOG_FILE" "${LOG_FILE}.prev"; then
+            echo "Warning: could not rotate entrypoint log $LOG_FILE, continuing without file logging" >&2
+            rotation_ok="false"
+        fi
 
-    # Cleanup any existing files or folders in the log directory
-    rm -rf "$LOG_DIR"/*
+        if [ "$rotation_ok" = "true" ]; then
+            FILE_LOGGING_ENABLED="true"
+        else
+            ENABLE_LOGGING_VALUE="false"
+            export ENABLE_LOGGING="false"
+        fi
+    else
+        echo "Warning: could not create log directory $LOG_DIR, continuing without file logging" >&2
+        ENABLE_LOGGING_VALUE="false"
+        export ENABLE_LOGGING="false"
+    fi
 fi
 
-(
-    if [ "$USING_TOR" = "true" ]; then
-        ./tor.sh
-    fi
-)
+# Egress modes are mutually exclusive. Check this BEFORE starting either one so
+# we never run tor.sh and then abort, leaving a half-configured network stack.
+if [ "$USING_TOR" = "true" ] && [ "$USING_WIREGUARD" = "true" ]; then
+    echo "USING_TOR and USING_WIREGUARD are mutually exclusive; enable only one egress mode." >&2
+    exit 1
+fi
 
-if is_truthy "$ENABLE_LOGGING_VALUE"; then
+if [ "$USING_TOR" = "true" ]; then
+    if [ "$RUN_AS_NON_ROOT" = "true" ]; then
+        echo "USING_TOR=true requires the container to start as root." >&2
+        echo "Non-root mode skips the privileged filesystem and network setup Tor depends on." >&2
+        exit 1
+    fi
+    ./tor.sh
+fi
+
+if [ "$USING_WIREGUARD" = "true" ]; then
+    if [ "$RUN_AS_NON_ROOT" = "true" ]; then
+        echo "USING_WIREGUARD=true requires the container to start as root." >&2
+        echo "Non-root mode skips the privileged network setup WireGuard depends on." >&2
+        exit 1
+    fi
+    ./wireguard.sh
+fi
+
+if [ "$FILE_LOGGING_ENABLED" = "true" ]; then
     start_file_logging "$LOG_FILE"
 fi
 
 echo "Starting entrypoint script"
-if is_truthy "$ENABLE_LOGGING_VALUE"; then
+if [ "$FILE_LOGGING_ENABLED" = "true" ]; then
     echo "Log file: $LOG_FILE"
 else
     echo "File logging disabled (ENABLE_LOGGING=$ENABLE_LOGGING_VALUE)"
 fi
-set -e
+
+PYTHON_BIN="/app/.venv/bin/python"
+if [ ! -x "$PYTHON_BIN" ]; then
+    PYTHON_BIN="python3"
+fi
+
+# Defensive: some orchestrators (e.g. Unraid Dockhand templates) inject a default
+# PATH that drops the venv bin directory baked in by the Dockerfile. Prepend it
+# so subprocesses launched without an absolute path still resolve correctly.
+case ":${PATH}:" in
+    *":/app/.venv/bin:"*) ;;
+    *) export PATH="/app/.venv/bin:${PATH}" ;;
+esac
+GUNICORN_BIN="/app/.venv/bin/gunicorn"
+if [ ! -x "$GUNICORN_BIN" ]; then
+    GUNICORN_BIN="gunicorn"
+fi
 
 # Print build version
 echo "Build version: $BUILD_VERSION"
@@ -74,67 +140,133 @@ echo "Release version: $RELEASE_VERSION"
 
 # Configure timezone
 if [ "$TZ" ]; then
-    echo "Setting timezone to $TZ"
-    ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
+    if [ "$RUN_AS_NON_ROOT" = "true" ]; then
+        echo "TZ is set to $TZ (non-root mode leaves /etc/localtime unchanged)"
+    else
+        echo "Setting timezone to $TZ"
+        ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
+    fi
 fi
 
-# Determine user ID with proper precedence:
-# 1. PUID (LinuxServer.io standard - recommended)
-# 2. UID (legacy, for backward compatibility with existing installs)
-# 3. Default to 1000
-#
-# Note: $UID is a bash builtin that's always set. We use `printenv` to detect
-# if UID was explicitly set as an environment variable (e.g., via docker-compose).
-if [ -n "$PUID" ]; then
-    RUN_UID="$PUID"
-    echo "Using PUID=$RUN_UID"
-elif printenv UID >/dev/null 2>&1; then
-    RUN_UID="$(printenv UID)"
-    echo "Using UID=$RUN_UID (legacy - consider migrating to PUID)"
+if [ "$RUN_AS_NON_ROOT" = "true" ]; then
+    RUN_UID="$CURRENT_UID"
+    RUN_GID="$CURRENT_GID"
+    USERNAME=$(getent passwd "$RUN_UID" 2>/dev/null | cut -d: -f1 || true)
+    if [ -z "$USERNAME" ]; then
+        USERNAME="$RUN_UID"
+        echo "No passwd entry found for UID $RUN_UID; using numeric identity"
+    fi
+    TARGET_USER_SPEC="${RUN_UID}:${RUN_GID}"
 else
-    RUN_UID=1000
-    echo "Using default UID=$RUN_UID"
+    # Determine user ID with proper precedence:
+    # 1. PUID (LinuxServer.io standard - recommended)
+    # 2. UID (legacy, for backward compatibility with existing installs)
+    # 3. Default to 1000
+    #
+    # Note: $UID is a bash builtin that's always set. We use `printenv` to detect
+    # if UID was explicitly set as an environment variable (e.g., via docker-compose).
+    if [ -n "$PUID" ]; then
+        RUN_UID="$PUID"
+        echo "Using PUID=$RUN_UID"
+    elif printenv UID >/dev/null 2>&1; then
+        RUN_UID="$(printenv UID)"
+        echo "Using UID=$RUN_UID (legacy - consider migrating to PUID)"
+    else
+        RUN_UID=1000
+        echo "Using default UID=$RUN_UID"
+    fi
+
+    # Determine group ID with proper precedence:
+    # 1. PGID (LinuxServer.io standard - recommended)
+    # 2. GID (legacy, for backward compatibility with existing installs)
+    # 3. Default to 1000
+    if [ -n "$PGID" ]; then
+        RUN_GID="$PGID"
+        echo "Using PGID=$RUN_GID"
+    elif [ -n "$GID" ]; then
+        RUN_GID="$GID"
+        echo "Using GID=$RUN_GID (legacy - consider migrating to PGID)"
+    else
+        RUN_GID=1000
+        echo "Using default GID=$RUN_GID"
+    fi
+
+    if ! getent group "$RUN_GID" >/dev/null; then
+        echo "Adding group $RUN_GID with name appuser"
+        groupadd -g "$RUN_GID" appuser
+    fi
+
+    # Create user if it doesn't exist for this UID yet.
+    if ! getent passwd "$RUN_UID" >/dev/null; then
+        echo "Adding user $RUN_UID with name appuser"
+        useradd -u "$RUN_UID" -g "$RUN_GID" -d "$DEFAULT_RUNTIME_HOME" -s /sbin/nologin appuser
+    fi
+
+    # Get username for the UID (whether we just created it or it existed)
+    USERNAME=$(getent passwd "$RUN_UID" | cut -d: -f1)
+    if [ -z "$USERNAME" ]; then
+        USERNAME="$RUN_UID"
+    fi
+    TARGET_USER_SPEC="${RUN_UID}:${RUN_GID}"
 fi
 
-# Determine group ID with proper precedence:
-# 1. PGID (LinuxServer.io standard - recommended)
-# 2. GID (legacy, for backward compatibility with existing installs)
-# 3. Default to 1000
-if [ -n "$PGID" ]; then
-    RUN_GID="$PGID"
-    echo "Using PGID=$RUN_GID"
-elif [ -n "$GID" ]; then
-    RUN_GID="$GID"
-    echo "Using GID=$RUN_GID (legacy - consider migrating to PGID)"
-else
-    RUN_GID=1000
-    echo "Using default GID=$RUN_GID"
-fi
+# Avoid unnecessary gosu hops when we're already running as the target user.
+# Some nested LXC setups spin on root-to-root gosu invocations.
+needs_user_switch() {
+    local current_uid
+    local current_gid
 
-if ! getent group "$RUN_GID" >/dev/null; then
-    echo "Adding group $RUN_GID with name appuser"
-    groupadd -g "$RUN_GID" appuser
-fi
+    current_uid=$(id -u)
+    current_gid=$(id -g)
 
-# Create user if it doesn't exist
-if ! id -u "$RUN_UID" >/dev/null 2>&1; then
-    echo "Adding user $RUN_UID with name appuser"
-    useradd -u "$RUN_UID" -g "$RUN_GID" -d /app -s /sbin/nologin appuser
-fi
+    [ "$current_uid" != "$RUN_UID" ] || [ "$current_gid" != "$RUN_GID" ]
+}
 
-# Get username for the UID (whether we just created it or it existed)
-USERNAME=$(getent passwd "$RUN_UID" | cut -d: -f1)
-echo "Username for UID $RUN_UID is $USERNAME"
+run_as_target_user() {
+    if needs_user_switch; then
+        gosu "$TARGET_USER_SPEC" "$@"
+        return $?
+    fi
+
+    "$@"
+}
+
+exec_as_target_user() {
+    if needs_user_switch; then
+        exec gosu "$TARGET_USER_SPEC" "$@"
+    fi
+
+    exec "$@"
+}
 
 test_write() {
-    folder=$1
-    test_file=$folder/shelfmark_TEST_WRITE
-    mkdir -p $folder
-    (
-        echo 0123456789_TEST | sudo -E -u "$USERNAME" HOME=/app tee $test_file > /dev/null
-    )
-    FILE_CONTENT=$(cat $test_file || echo "")
-    rm -f $test_file
+    local folder=$1
+    local test_file="$folder/shelfmark_TEST_WRITE"
+    local FILE_CONTENT
+    local result
+    local result_text
+
+    if ! mkdir -p "$folder"; then
+        echo "Failed to create directory for write test: $folder"
+        return 1
+    fi
+
+    # This is a probe: a failure here is expected (e.g. a fresh root-owned bind
+    # mount) and is recovered by the caller via change_ownership + re-probe. Hide
+    # the shell's "Permission denied"/"Read-only file system" stderr so a handled
+    # probe miss doesn't masquerade as a real boot failure in the logs.
+    if ! run_as_target_user sh -c 'echo 0123456789_TEST 2>/dev/null > "$1"' _ "$test_file"; then
+        echo "Failed to write test file in $folder as $USERNAME"
+        return 1
+    fi
+
+    FILE_CONTENT=$(cat "$test_file" 2>/dev/null || echo "")
+    # A folder can be writable but not deletable (e.g. a Synology share without
+    # "Delete subfolders and files"). That is not a boot failure - the app writes
+    # files in place on such shares - so don't let a failed cleanup print an
+    # alarming error or fail the probe.
+    run_as_target_user rm -f "$test_file" 2>/dev/null || \
+        echo "Note: could not remove test file in $folder (folder is writable but not deletable)"
     [ "$FILE_CONTENT" = "0123456789_TEST" ]
     result=$?
     if [ $result -eq 0 ]; then
@@ -147,34 +279,43 @@ test_write() {
 }
 
 make_writable() {
-    folder=$1
-    did_full_chown=0
+    local folder="$1"
+    local mode="${2:-tree}"
+    local did_full_chown=0
+    local is_writable
     set +e
-    test_write $folder
+    test_write "$folder"
     is_writable=$?
     set -e
     if [ $is_writable -eq 0 ]; then
         echo "Folder $folder is writable, no need to change ownership"
     else
-        echo "Folder $folder is not writable, changing ownership"
-        change_ownership $folder
-        chmod -R g+r,g+w $folder || echo "Failed to change group permissions for ${folder}, continuing..."
+        if [ "$mode" = "root" ]; then
+            echo "Folder $folder is not writable, fixing top-level ownership and permissions"
+            mkdir -p "$folder"
+            chown "${RUN_UID}:${RUN_GID}" "$folder" || echo "Failed to change ownership for ${folder}, continuing..."
+            chmod u+rwx "$folder" || echo "Failed to change owner permissions for ${folder}, continuing..."
+        else
+            echo "Folder $folder is not writable, changing ownership"
+            change_ownership "$folder"
+            chmod -R g+r,g+w "$folder" || echo "Failed to change group permissions for ${folder}, continuing..."
+        fi
         did_full_chown=1
     fi
     # Fix any misowned subdirectories/files (e.g., from previous runs as root)
-    if [ "$did_full_chown" -eq 0 ] && [ -d "$folder" ]; then
+    if [ "$mode" = "tree" ] && [ "$did_full_chown" -eq 0 ] && [ -d "$folder" ]; then
         echo "Checking for misowned files/directories in $folder"
         # Stay on the same filesystem to avoid traversing mounted subpaths
         # (for example read-only bind mounts under /app in dev setups).
         find "$folder" -xdev -mindepth 1 \( ! -user "$RUN_UID" -o ! -group "$RUN_GID" \) \
             -exec chown "$RUN_UID:$RUN_GID" {} + 2>/dev/null || true
     fi
-    test_write $folder || echo "Failed to test write to ${folder}, continuing..."
+    test_write "$folder" || echo "Failed to test write to ${folder}, continuing..."
 }
 
 fix_misowned() {
-    folder=$1
-    mkdir -p $folder
+    local folder="$1"
+    mkdir -p "$folder"
     echo "Checking for misowned files/directories in $folder"
     # Stay on the same filesystem to avoid traversing mounted subpaths
     # (for example read-only bind mounts under /app in dev setups).
@@ -184,89 +325,178 @@ fix_misowned() {
 
 # Ensure proper ownership of application directories
 change_ownership() {
-  folder=$1
-  mkdir -p $folder
+  local folder="$1"
+  mkdir -p "$folder"
   echo "Changing ownership of $folder to $USERNAME:$RUN_GID"
   chown -R "${RUN_UID}:${RUN_GID}" "${folder}" || echo "Failed to change ownership for ${folder}, continuing..."
 }
 
-fix_misowned /app
-fix_misowned /var/log/shelfmark
-fix_misowned /tmp/shelfmark
+require_writable_dir() {
+    local folder="$1"
+    local label="${2:-Directory}"
 
-# SeleniumBase (internal bypasser) writes a patched chromedriver binary (uc_driver)
-# into its own drivers directory. Some NAS/docker setups can apply restrictive ACLs
-# to extracted image layers that block non-root writes; ensure the runtime UID owns it.
-if [ "${USING_EXTERNAL_BYPASSER}" != "true" ]; then
-    set +e
-    SELENIUMBASE_DRIVERS_DIR=$(python3 -c "import pathlib, seleniumbase; print(pathlib.Path(seleniumbase.__file__).resolve().parent / 'drivers')" 2>/dev/null)
-    set -e
+    if ! mkdir -p "$folder"; then
+        echo "Failed to create ${label} directory: $folder"
+        exit 1
+    fi
 
-    if [ -n "$SELENIUMBASE_DRIVERS_DIR" ] && [ -d "$SELENIUMBASE_DRIVERS_DIR" ]; then
-        change_ownership "$SELENIUMBASE_DRIVERS_DIR"
+    if ! test_write "$folder"; then
+        echo "${label} directory is not writable in non-root mode: $folder"
+        echo "Prepare ownership outside the container (for example with a pre-owned volume or Kubernetes fsGroup)."
+        exit 1
+    fi
+}
 
-        # If the driver already exists, ensure it's executable for the runtime user.
-        if [ -f "${SELENIUMBASE_DRIVERS_DIR}/uc_driver" ]; then
-            chmod +x "${SELENIUMBASE_DRIVERS_DIR}/uc_driver" || echo "Failed to chmod uc_driver, continuing..."
+fail_unwritable_config_dir() {
+    local folder="$1"
+    local owner
+
+    owner=$(stat -c '%u:%g' "$folder" 2>/dev/null || echo "unknown")
+
+    echo ""
+    echo "========================================================"
+    echo "ERROR: Config directory is not writable!"
+    echo ""
+    echo "Config directory: $folder"
+    echo "Current owner: $owner"
+    echo "Configured runtime identity: ${RUN_UID}:${RUN_GID}"
+    echo ""
+    echo "To fix this permanently, run on your HOST machine:"
+    echo "  chown -R $RUN_UID:$RUN_GID /path/to/config"
+    echo "========================================================"
+    echo ""
+    exit 1
+}
+
+resolve_runtime_home() {
+    local runtime_home
+
+    runtime_home=$(getent passwd "$RUN_UID" 2>/dev/null | cut -d: -f6 || true)
+    case "$runtime_home" in
+        ""|/|/app|/nonexistent)
+            runtime_home="$DEFAULT_RUNTIME_HOME"
+            ;;
+    esac
+
+    printf '%s\n' "$runtime_home"
+}
+
+ensure_tree_writable() {
+    local folder="$1"
+
+    make_writable "$folder"
+    if [ -d "$folder" ]; then
+        chmod -R u+rwX,g+rwX "$folder" || echo "Failed to relax permissions for ${folder}, continuing..."
+    fi
+}
+
+ensure_symlinked_dir() {
+    local link_path="$1"
+    local target_path="$2"
+
+    ensure_tree_writable "$target_path"
+
+    if [ -L "$link_path" ]; then
+        local current_target
+        current_target=$(readlink "$link_path" 2>/dev/null || echo "")
+        if [ "$current_target" = "$target_path" ]; then
+            echo "$link_path already points to $target_path"
+            return 0
+        fi
+        echo "Replacing symlink $link_path -> $current_target with $target_path"
+        rm -f "$link_path" || echo "Failed to replace symlink ${link_path}, continuing..."
+    elif [ -d "$link_path" ]; then
+        echo "Moving existing scratch files from $link_path to $target_path"
+        find "$link_path" -xdev -mindepth 1 -maxdepth 1 -exec mv -t "$target_path" {} + 2>/dev/null || true
+        ensure_tree_writable "$target_path"
+
+        if ! rmdir "$link_path" 2>/dev/null; then
+            echo "Could not replace $link_path with symlink, leaving existing directory in place"
+            ensure_tree_writable "$link_path"
+            return 0
+        fi
+    elif [ -e "$link_path" ]; then
+        echo "$link_path exists and is not a directory, leaving it in place"
+        return 0
+    fi
+
+    if [ ! -e "$link_path" ]; then
+        ln -s "$target_path" "$link_path" || echo "Failed to create symlink ${link_path}, continuing..."
+    fi
+}
+
+if [ "$RUN_AS_NON_ROOT" = "true" ]; then
+    require_writable_dir /tmp/shelfmark "Temporary"
+
+    if [ "${USING_EXTERNAL_BYPASSER}" != "true" ]; then
+        require_writable_dir /tmp/shelfmark/seleniumbase/downloaded_files "SeleniumBase downloads"
+        require_writable_dir /tmp/shelfmark/seleniumbase/archived_files "SeleniumBase archive"
+    fi
+
+    require_writable_dir "${CONFIG_DIR:-/config}" "Config"
+else
+    fix_misowned /var/log/shelfmark
+    fix_misowned /tmp/shelfmark
+
+    # Keep SeleniumBase on its default /app-based paths, but redirect the scratch
+    # directories into /tmp so bypasser startup doesn't depend on image-layer writes.
+    if [ "${USING_EXTERNAL_BYPASSER}" != "true" ]; then
+        ensure_symlinked_dir /app/downloaded_files /tmp/shelfmark/seleniumbase/downloaded_files
+        ensure_symlinked_dir /app/archived_files /tmp/shelfmark/seleniumbase/archived_files
+
+        # Keep SeleniumBase's bundled drivers directory writable as well for
+        # compatibility with legacy UC code paths that still probe bundled assets.
+        set +e
+        SELENIUMBASE_DRIVERS_DIR=$("$PYTHON_BIN" -c "import pathlib, seleniumbase; print(pathlib.Path(seleniumbase.__file__).resolve().parent / 'drivers')" 2>/dev/null)
+        set -e
+
+        if [ -n "$SELENIUMBASE_DRIVERS_DIR" ] && [ -d "$SELENIUMBASE_DRIVERS_DIR" ]; then
+            change_ownership "$SELENIUMBASE_DRIVERS_DIR"
+
+            # If the legacy driver already exists, ensure it's executable for the runtime user.
+            if [ -f "${SELENIUMBASE_DRIVERS_DIR}/uc_driver" ]; then
+                chmod +x "${SELENIUMBASE_DRIVERS_DIR}/uc_driver" || echo "Failed to chmod uc_driver, continuing..."
+            fi
         fi
     fi
-fi
 
-# Test write to all folders
-make_writable ${CONFIG_DIR:-/config}
-make_writable ${INGEST_DIR:-/books}
+    # Config is Shelfmark-owned state, so it keeps the thorough repair path.
+    make_writable "${CONFIG_DIR:-/config}" tree
 
-# Fix permissions on directories configured in settings
-echo "Checking for additional configured directories..."
-if [ -f /app/scripts/fix_permissions.py ]; then
-    configured_dirs=$(python3 /app/scripts/fix_permissions.py 2>/dev/null || echo "")
-    if [ -n "$configured_dirs" ]; then
-        echo "$configured_dirs" | while read -r dir; do
-            if [ -n "$dir" ] && [ -d "$dir" ]; then
-                echo "Checking configured directory: $dir"
-                make_writable "$dir"
-            fi
-        done
+    # Refuse to continue if the config directory is still not writable after repair.
+    CONFIG_PATH=${CONFIG_DIR:-/config}
+    set +e
+    test_write "$CONFIG_PATH" >/dev/null 2>&1
+    config_ok=$?
+    set -e
+
+    if [ $config_ok -ne 0 ]; then
+        fail_unwritable_config_dir "$CONFIG_PATH"
     fi
-fi
 
-# Fallback to root if config dir is still not writable (common on NAS/Unraid after upgrade from v0.4.0)
-CONFIG_PATH=${CONFIG_DIR:-/config}
-set +e
-test_write "$CONFIG_PATH" >/dev/null 2>&1
-config_ok=$?
-set -e
-
-if [ $config_ok -ne 0 ] && [ "$RUN_UID" != "0" ]; then
-    config_owner=$(stat -c '%u' "$CONFIG_PATH" 2>/dev/null || echo "unknown")
-    if [ "$config_owner" = "0" ]; then
-        echo ""
-        echo "========================================================"
-        echo "WARNING: Permission issue detected!"
-        echo ""
-        echo "Config directory is owned by root but PUID=$RUN_UID."
-        echo "This typically happens after upgrading from v0.4.0 where"
-        echo "PUID/PGID settings were not respected."
-        echo ""
-        echo "Falling back to running as root to prevent data loss."
-        echo ""
-        echo "To fix this permanently, run on your HOST machine:"
-        echo "  chown -R $RUN_UID:$RUN_GID /path/to/config"
-        echo ""
-        echo "Then restart the container."
-        echo "========================================================"
-        echo ""
-        RUN_UID=0
-        RUN_GID=0
-        USERNAME=root
-    fi
+    # The ingest/destination library (default /books) is user data and may be a
+    # bind mount owned by another uid; downloads fail with "Destination not
+    # writable" if the runtime user can't write there. Fix the top-level dir only
+    # (root mode) so we don't recursively chown a potentially huge library.
+    make_writable "${INGEST_DIR:-/books}" root
 fi
 
 # Always run Gunicorn (even when DEBUG=true) to ensure Socket.IO WebSocket
 # upgrades work reliably on customer machines.
 # Map app LOG_LEVEL (often DEBUG/INFO/...) to gunicorn's --log-level (lowercase).
-gunicorn_loglevel=$([ "$DEBUG" = "true" ] && echo debug || echo "${LOG_LEVEL:-info}" | tr '[:upper:]' '[:lower:]')
-command="gunicorn --log-level ${gunicorn_loglevel} --access-logfile - --error-logfile - --worker-class geventwebsocket.gunicorn.workers.GeventWebSocketWorker --workers 1 -t 300 -b ${FLASK_HOST:-0.0.0.0}:${FLASK_PORT:-8084} shelfmark.main:app"
+# Gunicorn rejects anything outside its own list, so normalize and fall back to
+# info rather than letting a typo stop the container from booting.
+if [ "$DEBUG" = "true" ]; then
+    gunicorn_loglevel=debug
+else
+    gunicorn_loglevel=$(echo "${LOG_LEVEL:-info}" | tr '[:upper:]' '[:lower:]')
+    [ "$gunicorn_loglevel" = "warn" ] && gunicorn_loglevel=warning
+    case "$gunicorn_loglevel" in
+        debug|info|warning|error|critical) ;;
+        *) gunicorn_loglevel=info ;;
+    esac
+fi
+command="${GUNICORN_BIN} --log-level ${gunicorn_loglevel} --access-logfile - --error-logfile - --worker-class geventwebsocket.gunicorn.workers.GeventWebSocketWorker --workers 1 -t 300 -b ${FLASK_HOST:-0.0.0.0}:${FLASK_PORT:-8084} shelfmark.main:app"
 
 # If DEBUG and not using an external bypass
 if [ "$DEBUG" = "true" ] && [ "$USING_EXTERNAL_BYPASSER" != "true" ]; then
@@ -274,7 +504,7 @@ if [ "$DEBUG" = "true" ] && [ "$USING_EXTERNAL_BYPASSER" != "true" ]; then
     set -x
     echo "vvvvvvvvvvvv DEBUG MODE vvvvvvvvvvvv"
     echo "Starting Xvfb for debugging"
-    python3 -c "from pyvirtualdisplay import Display; Display(visible=False, size=(1440,1880)).start()"
+    "$PYTHON_BIN" -c "from pyvirtualdisplay import Display; Display(visible=False, size=(1440,1880)).start()"
     id
     free -h
     uname -a
@@ -294,7 +524,7 @@ if [ "$DEBUG" = "true" ] && [ "$USING_EXTERNAL_BYPASSER" != "true" ]; then
             --enable-logging --v=1 --log-level=0 \
             --log-file=/tmp/chrome_entrypoint_test.log \
             --crash-dumps-dir=/tmp/chrome_crash_dumps \
-            < /dev/null 
+            < /dev/null
     EXIT_CODE=$?
     echo "Chrome exit code: $EXIT_CODE"
     ls -lh /tmp/chrome_entrypoint_test.log
@@ -334,7 +564,24 @@ else
     exit 1
 fi
 
-echo "Running command: '$command' as '$USERNAME' (debug=$is_debug)"
+RUNTIME_HOME=$(resolve_runtime_home)
+if [ "$RUN_AS_NON_ROOT" = "true" ]; then
+    require_writable_dir "$RUNTIME_HOME" "Home"
+else
+    mkdir -p "$RUNTIME_HOME"
+    make_writable "$RUNTIME_HOME" tree
+fi
+
+if [ "$RUN_AS_NON_ROOT" = "true" ]; then
+    echo "Startup mode: non-root"
+elif [ "$RUN_UID" = "0" ] && [ "$RUN_GID" = "0" ]; then
+    echo "Startup mode: root"
+else
+    echo "Startup mode: root bootstrap with privilege drop"
+fi
+echo "Runtime identity: $USERNAME (${RUN_UID}:${RUN_GID})"
+
+echo "Running command: '$command' as '$USERNAME' (debug=${DEBUG:-false})"
 
 # Set umask for file permissions (default: 0022 = files 644, dirs 755)
 UMASK_VALUE=${UMASK:-0022}
@@ -342,4 +589,4 @@ echo "Setting umask to $UMASK_VALUE"
 umask $UMASK_VALUE
 
 stop_file_logging
-exec sudo -E -u "$USERNAME" HOME=/app $command
+exec_as_target_user env HOME="$RUNTIME_HOME" $command

genDebug.sh

@@ -199,4 +199,3 @@ else
   echo "Failed to create debug archive"
   exit 1
 fi
-

node_modules/.bin/baseline-browser-mapping

@@ -1 +0,0 @@
-../baseline-browser-mapping/dist/cli.js

node_modules/.package-lock.json

@@ -1,17 +0,0 @@
-{
-  "name": "shelfmark",
-  "lockfileVersion": 3,
-  "requires": true,
-  "packages": {
-    "node_modules/baseline-browser-mapping": {
-      "version": "2.9.19",
-      "resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.9.19.tgz",
-      "integrity": "sha512-ipDqC8FrAl/76p2SSWKSI+H9tFwm7vYqXQrItCuiVPt26Km0jS+NzSsBWAaBusvSbQcfJG+JitdMm+wZAgTYqg==",
-      "dev": true,
-      "license": "Apache-2.0",
-      "bin": {
-        "baseline-browser-mapping": "dist/cli.js"
-      }
-    }
-  }
-}

node_modules/baseline-browser-mapping/LICENSE.txt

@@ -1,201 +0,0 @@
-                                 Apache License
-                           Version 2.0, January 2004
-                        http://www.apache.org/licenses/
-
-   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
-   1. Definitions.
-
-      "License" shall mean the terms and conditions for use, reproduction,
-      and distribution as defined by Sections 1 through 9 of this document.
-
-      "Licensor" shall mean the copyright owner or entity authorized by
-      the copyright owner that is granting the License.
-
-      "Legal Entity" shall mean the union of the acting entity and all
-      other entities that control, are controlled by, or are under common
-      control with that entity. For the purposes of this definition,
-      "control" means (i) the power, direct or indirect, to cause the
-      direction or management of such entity, whether by contract or
-      otherwise, or (ii) ownership of fifty percent (50%) or more of the
-      outstanding shares, or (iii) beneficial ownership of such entity.
-
-      "You" (or "Your") shall mean an individual or Legal Entity
-      exercising permissions granted by this License.
-
-      "Source" form shall mean the preferred form for making modifications,
-      including but not limited to software source code, documentation
-      source, and configuration files.
-
-      "Object" form shall mean any form resulting from mechanical
-      transformation or translation of a Source form, including but
-      not limited to compiled object code, generated documentation,
-      and conversions to other media types.
-
-      "Work" shall mean the work of authorship, whether in Source or
-      Object form, made available under the License, as indicated by a
-      copyright notice that is included in or attached to the work
-      (an example is provided in the Appendix below).
-
-      "Derivative Works" shall mean any work, whether in Source or Object
-      form, that is based on (or derived from) the Work and for which the
-      editorial revisions, annotations, elaborations, or other modifications
-      represent, as a whole, an original work of authorship. For the purposes
-      of this License, Derivative Works shall not include works that remain
-      separable from, or merely link (or bind by name) to the interfaces of,
-      the Work and Derivative Works thereof.
-
-      "Contribution" shall mean any work of authorship, including
-      the original version of the Work and any modifications or additions
-      to that Work or Derivative Works thereof, that is intentionally
-      submitted to Licensor for inclusion in the Work by the copyright owner
-      or by an individual or Legal Entity authorized to submit on behalf of
-      the copyright owner. For the purposes of this definition, "submitted"
-      means any form of electronic, verbal, or written communication sent
-      to the Licensor or its representatives, including but not limited to
-      communication on electronic mailing lists, source code control systems,
-      and issue tracking systems that are managed by, or on behalf of, the
-      Licensor for the purpose of discussing and improving the Work, but
-      excluding communication that is conspicuously marked or otherwise
-      designated in writing by the copyright owner as "Not a Contribution."
-
-      "Contributor" shall mean Licensor and any individual or Legal Entity
-      on behalf of whom a Contribution has been received by Licensor and
-      subsequently incorporated within the Work.
-
-   2. Grant of Copyright License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      copyright license to reproduce, prepare Derivative Works of,
-      publicly display, publicly perform, sublicense, and distribute the
-      Work and such Derivative Works in Source or Object form.
-
-   3. Grant of Patent License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      (except as stated in this section) patent license to make, have made,
-      use, offer to sell, sell, import, and otherwise transfer the Work,
-      where such license applies only to those patent claims licensable
-      by such Contributor that are necessarily infringed by their
-      Contribution(s) alone or by combination of their Contribution(s)
-      with the Work to which such Contribution(s) was submitted. If You
-      institute patent litigation against any entity (including a
-      cross-claim or counterclaim in a lawsuit) alleging that the Work
-      or a Contribution incorporated within the Work constitutes direct
-      or contributory patent infringement, then any patent licenses
-      granted to You under this License for that Work shall terminate
-      as of the date such litigation is filed.
-
-   4. Redistribution. You may reproduce and distribute copies of the
-      Work or Derivative Works thereof in any medium, with or without
-      modifications, and in Source or Object form, provided that You
-      meet the following conditions:
-
-      (a) You must give any other recipients of the Work or
-          Derivative Works a copy of this License; and
-
-      (b) You must cause any modified files to carry prominent notices
-          stating that You changed the files; and
-
-      (c) You must retain, in the Source form of any Derivative Works
-          that You distribute, all copyright, patent, trademark, and
-          attribution notices from the Source form of the Work,
-          excluding those notices that do not pertain to any part of
-          the Derivative Works; and
-
-      (d) If the Work includes a "NOTICE" text file as part of its
-          distribution, then any Derivative Works that You distribute must
-          include a readable copy of the attribution notices contained
-          within such NOTICE file, excluding those notices that do not
-          pertain to any part of the Derivative Works, in at least one
-          of the following places: within a NOTICE text file distributed
-          as part of the Derivative Works; within the Source form or
-          documentation, if provided along with the Derivative Works; or,
-          within a display generated by the Derivative Works, if and
-          wherever such third-party notices normally appear. The contents
-          of the NOTICE file are for informational purposes only and
-          do not modify the License. You may add Your own attribution
-          notices within Derivative Works that You distribute, alongside
-          or as an addendum to the NOTICE text from the Work, provided
-          that such additional attribution notices cannot be construed
-          as modifying the License.
-
-      You may add Your own copyright statement to Your modifications and
-      may provide additional or different license terms and conditions
-      for use, reproduction, or distribution of Your modifications, or
-      for any such Derivative Works as a whole, provided Your use,
-      reproduction, and distribution of the Work otherwise complies with
-      the conditions stated in this License.
-
-   5. Submission of Contributions. Unless You explicitly state otherwise,
-      any Contribution intentionally submitted for inclusion in the Work
-      by You to the Licensor shall be under the terms and conditions of
-      this License, without any additional terms or conditions.
-      Notwithstanding the above, nothing herein shall supersede or modify
-      the terms of any separate license agreement you may have executed
-      with Licensor regarding such Contributions.
-
-   6. Trademarks. This License does not grant permission to use the trade
-      names, trademarks, service marks, or product names of the Licensor,
-      except as required for reasonable and customary use in describing the
-      origin of the Work and reproducing the content of the NOTICE file.
-
-   7. Disclaimer of Warranty. Unless required by applicable law or
-      agreed to in writing, Licensor provides the Work (and each
-      Contributor provides its Contributions) on an "AS IS" BASIS,
-      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-      implied, including, without limitation, any warranties or conditions
-      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
-      PARTICULAR PURPOSE. You are solely responsible for determining the
-      appropriateness of using or redistributing the Work and assume any
-      risks associated with Your exercise of permissions under this License.
-
-   8. Limitation of Liability. In no event and under no legal theory,
-      whether in tort (including negligence), contract, or otherwise,
-      unless required by applicable law (such as deliberate and grossly
-      negligent acts) or agreed to in writing, shall any Contributor be
-      liable to You for damages, including any direct, indirect, special,
-      incidental, or consequential damages of any character arising as a
-      result of this License or out of the use or inability to use the
-      Work (including but not limited to damages for loss of goodwill,
-      work stoppage, computer failure or malfunction, or any and all
-      other commercial damages or losses), even if such Contributor
-      has been advised of the possibility of such damages.
-
-   9. Accepting Warranty or Additional Liability. While redistributing
-      the Work or Derivative Works thereof, You may choose to offer,
-      and charge a fee for, acceptance of support, warranty, indemnity,
-      or other liability obligations and/or rights consistent with this
-      License. However, in accepting such obligations, You may act only
-      on Your own behalf and on Your sole responsibility, not on behalf
-      of any other Contributor, and only if You agree to indemnify,
-      defend, and hold each Contributor harmless for any liability
-      incurred by, or claims asserted against, such Contributor by reason
-      of your accepting any such warranty or additional liability.
-
-   END OF TERMS AND CONDITIONS
-
-   APPENDIX: How to apply the Apache License to your work.
-
-      To apply the Apache License to your work, attach the following
-      boilerplate notice, with the fields enclosed by brackets "[]"
-      replaced with your own identifying information. (Don't include
-      the brackets!)  The text should be enclosed in the appropriate
-      comment syntax for the file format. We also recommend that a
-      file or class name and description of purpose be included on the
-      same "printed page" as the copyright notice for easier
-      identification within third-party archives.
-
-   Copyright [yyyy] [name of copyright owner]
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.

node_modules/baseline-browser-mapping/README.md

@@ -1,463 +0,0 @@
-# [`baseline-browser-mapping`](https://github.com/web-platform-dx/web-features/packages/baseline-browser-mapping)
-
-By the [W3C WebDX Community Group](https://www.w3.org/community/webdx/) and contributors.
-
-`baseline-browser-mapping` provides:
-
-- An `Array` of browsers compatible with Baseline Widely available and Baseline year feature sets via the [`getCompatibleVersions()` function](#get-baseline-widely-available-browser-versions-or-baseline-year-browser-versions).
-- An `Array`, `Object` or `CSV` as a string describing the Baseline feature set support of all browser versions included in the module's data set via the [`getAllVersions()` function](#get-data-for-all-browser-versions).
-
-You can use `baseline-browser-mapping` to help you determine minimum browser version support for your chosen Baseline feature set; or to analyse the level of support for different Baseline feature sets in your site's traffic by joining the data with your analytics data.
-
-## Install for local development
-
-To install the package, run:
-
-`npm install --save-dev baseline-browser-mapping`
-
-`baseline-browser-mapping` depends on `web-features` and `@mdn/browser-compat-data` for core browser version selection, but the data is pre-packaged and minified. This package checks for updates to those modules and the supported [downstream browsers](#downstream-browsers) on a daily basis and is updated frequently. Consider adding a script to your `package.json` to update `baseline-browser-mapping` and using it as part of your build process to ensure your data is as up to date as possible:
-
-```javascript
-"scripts": [
-  "refresh-baseline-browser-mapping": "npm i --save-dev baseline-browser-mapping@latest"
-]
-```
-
-The minimum supported NodeJS version for `baseline-browser-mapping` is v8 in alignment with `browserslist`. For NodeJS versions earlier than v13.2, the [`require('baseline-browser-mapping')`](https://nodejs.org/api/modules.html#requireid) syntax should be used to import the module.
-
-## Keeping `baseline-browser-mapping` up to date
-
-If you are only using this module to generate minimum browser versions for Baseline Widely available or Baseline year feature sets, you don't need to update this module frequently, as the backward looking data is reasonably stable.
-
-However, if you are targeting Newly available, using the [`getAllVersions()`](#get-data-for-all-browser-versions) function or heavily relying on the data for downstream browsers, you should update this module more frequently. If you target a feature cut off date within the last two months and your installed version of `baseline-browser-mapping` has data that is more than 2 months old, you will receive a console warning advising you to update to the latest version when you call `getCompatibleVersions()` or `getAllVersions()`.
-
-If you want to suppress these warnings you can use the `suppressWarnings: true` option in the configuration object passed to `getCompatibleVersions()` or `getAllVersions()`. Alternatively, you can use the `BASELINE_BROWSER_MAPPING_IGNORE_OLD_DATA=true` environment variable when running your build process. This module also respects the `BROWSERSLIST_IGNORE_OLD_DATA=true` environment variable. Environment variables can also be provided in a `.env` file from Node 20 onwards; however, this module does not load .env files automatically to avoid conflicts with other libraries with different requirements. You will need to use `process.loadEnvFile()` or a library like `dotenv` to load .env files before `baseline-browser-mapping` is called.
-
-If you want to ensure [reproducible builds](https://www.wikiwand.com/en/articles/Reproducible_builds), we strongly recommend using the `widelyAvailableOnDate` option to fix the Widely available date on a per build basis to ensure dependent tools provide the same output and you do not produce data staleness warnings. If you are using [`browserslist`](https://github.com/browserslist/browserslist) to target Baseline Widely available, consider automatically updating your `browserslist` configuration in `package.json` or `.browserslistrc` to `baseline widely available on {YYYY-MM-DD}` as part of your build process to ensure the same or sufficiently similar list of minimum browsers is reproduced for historical builds.
-
-## Importing `baseline-browser-mapping`
-
-This module exposes two functions: `getCompatibleVersions()` and `getAllVersions()`, both which can be imported directly from `baseline-browser-mapping`:
-
-```javascript
-import {
-  getCompatibleVersions,
-  getAllVersions,
-} from "baseline-browser-mapping";
-```
-
-If you want to load the script and data directly in a web page without hosting it yourself, consider using a CDN:
-
-```html
-<script type="module">
-  import {
-    getCompatibleVersions,
-    getAllVersions,
-  } from "https://cdn.jsdelivr.net/npm/baseline-browser-mapping";
-</script>
-```
-
-## Get Baseline Widely available browser versions or Baseline year browser versions
-
-To get the current list of minimum browser versions compatible with Baseline Widely available features from the core browser set, call the `getCompatibleVersions()` function:
-
-```javascript
-getCompatibleVersions();
-```
-
-Executed on 7th March 2025, the above code returns the following browser versions:
-
-```javascript
-[
-  { browser: "chrome", version: "105", release_date: "2022-09-02" },
-  {
-    browser: "chrome_android",
-    version: "105",
-    release_date: "2022-09-02",
-  },
-  { browser: "edge", version: "105", release_date: "2022-09-02" },
-  { browser: "firefox", version: "104", release_date: "2022-08-23" },
-  {
-    browser: "firefox_android",
-    version: "104",
-    release_date: "2022-08-23",
-  },
-  { browser: "safari", version: "15.6", release_date: "2022-09-02" },
-  {
-    browser: "safari_ios",
-    version: "15.6",
-    release_date: "2022-09-02",
-  },
-];
-```
-
-> [!NOTE]
-> The minimum versions of each browser are not necessarily the final release before the Widely available cutoff date of `TODAY - 30 MONTHS`. Some earlier versions will have supported the full Widely available feature set.
-
-### `getCompatibleVersions()` configuration options
-
-`getCompatibleVersions()` accepts an `Object` as an argument with configuration options. The defaults are as follows:
-
-```javascript
-{
-  targetYear: undefined,
-  widelyAvailableOnDate: undefined,
-  includeDownstreamBrowsers: false,
-  listAllCompatibleVersions: false,
-  suppressWarnings: false
-}
-```
-
-#### `targetYear`
-
-The `targetYear` option returns the minimum browser versions compatible with all **Baseline Newly available** features at the end of the specified calendar year. For example, calling:
-
-```javascript
-getCompatibleVersions({
-  targetYear: 2020,
-});
-```
-
-Returns the following versions:
-
-```javascript
-[
-  { browser: "chrome", version: "87", release_date: "2020-11-19" },
-  {
-    browser: "chrome_android",
-    version: "87",
-    release_date: "2020-11-19",
-  },
-  { browser: "edge", version: "87", release_date: "2020-11-19" },
-  { browser: "firefox", version: "83", release_date: "2020-11-17" },
-  {
-    browser: "firefox_android",
-    version: "83",
-    release_date: "2020-11-17",
-  },
-  { browser: "safari", version: "14", release_date: "2020-09-16" },
-  { browser: "safari_ios", version: "14", release_date: "2020-09-16" },
-];
-```
-
-> [!NOTE]
-> The minimum version of each browser is not necessarily the final version released in that calendar year. In the above example, Firefox 84 was the final version released in 2020; however Firefox 83 supported all of the features that were interoperable at the end of 2020.
-> [!WARNING]
-> You cannot use `targetYear` and `widelyAavailableDate` together. Please only use one of these options at a time.
-
-#### `widelyAvailableOnDate`
-
-The `widelyAvailableOnDate` option returns the minimum versions compatible with Baseline Widely available on a specified date in the format `YYYY-MM-DD`:
-
-```javascript
-getCompatibleVersions({
-  widelyAvailableOnDate: `2023-04-05`,
-});
-```
-
-> [!TIP]
-> This option is useful if you provide a versioned library that targets Baseline Widely available on each version's release date and you need to provide a statement on minimum supported browser versions in your documentation.
-
-#### `includeDownstreamBrowsers`
-
-Setting `includeDownstreamBrowsers` to `true` will include browsers outside of the Baseline core browser set where it is possible to map those browsers to an upstream Chromium or Gecko version:
-
-```javascript
-getCompatibleVersions({
-  includeDownstreamBrowsers: true,
-});
-```
-
-For more information on downstream browsers, see [the section on downstream browsers](#downstream-browsers) below.
-
-#### `includeKaiOS`
-
-KaiOS is an operating system and app framework based on the Gecko engine from Firefox. KaiOS is based on the Gecko engine and feature support can be derived from the upstream Gecko version that each KaiOS version implements. However KaiOS requires other considerations beyond feature compatibility to ensure a good user experience as it runs on device types that do not have either mouse and keyboard or touch screen input in the way that all the other browsers supported by this module do.
-
-```javascript
-getCompatibleVersions({
-  includeDownstreamBrowsers: true,
-  includeKaiOS: true,
-});
-```
-
-> [!NOTE]
-> Including KaiOS requires you to include all downstream browsers using the `includeDownstreamBrowsers` option.
-
-#### `listAllCompatibleVersions`
-
-Setting `listAllCompatibleVersions` to true will include the minimum versions of each compatible browser, and all the subsequent versions:
-
-```javascript
-getCompatibleVersions({
-  listAllCompatibleVersions: true,
-});
-```
-
-#### `suppressWarnings`
-
-Setting `suppressWarnings` to `true` will suppress the console warning about old data:
-
-```javascript
-getCompatibleVersions({
-  suppressWarnings: true,
-});
-```
-
-## Get data for all browser versions
-
-You may want to obtain data on all the browser versions available in this module for use in an analytics solution or dashboard. To get details of each browser version's level of Baseline support, call the `getAllVersions()` function:
-
-```javascript
-import { getAllVersions } from "baseline-browser-mapping";
-
-getAllVersions();
-```
-
-By default, this function returns an `Array` of `Objects` and excludes downstream browsers:
-
-```javascript
-[
-  ...
-  {
-    browser: "firefox_android", // Browser name
-    version: "125", // Browser version
-    release_date: "2024-04-16", // Release date
-    year: 2023, // Baseline year feature set the version supports
-    wa_compatible: true // Whether the browser version supports Widely available
-  },
-  ...
-]
-```
-
-For browser versions in `@mdn/browser-compat-data` that were released before Baseline can be defined, i.e. Baseline 2015, the `year` property is always the string: `"pre_baseline"`.
-
-### Understanding which browsers support Newly available features
-
-You may want to understand which recent browser versions support all Newly available features. You can replace the `wa_compatible` property with a `supports` property using the `useSupport` option:
-
-```javascript
-getAllVersions({
-  useSupports: true,
-});
-```
-
-The `supports` property is optional and has two possible values:
-
-- `widely` for browser versions that support all Widely available features.
-- `newly` for browser versions that support all Newly available features.
-
-Browser versions that do not support Widely or Newly available will not include the `support` property in the `array` or `object` outputs, and in the CSV output, the `support` column will contain an empty string. Browser versions that support all Newly available features also support all Widely available features.
-
-### `getAllVersions()` Configuration options
-
-`getAllVersions()` accepts an `Object` as an argument with configuration options. The defaults are as follows:
-
-```javascript
-{
-  includeDownstreamBrowsers: false,
-  outputFormat: "array",
-  suppressWarnings: false
-}
-```
-
-#### `includeDownstreamBrowsers` (in `getAllVersions()` output)
-
-As with `getCompatibleVersions()`, you can set `includeDownstreamBrowsers` to `true` to include the Chromium and Gecko downstream browsers [listed below](#list-of-downstream-browsers).
-
-```javascript
-getAllVersions({
-  includeDownstreamBrowsers: true,
-});
-```
-
-Downstream browsers include the same properties as core browsers, as well as the `engine`they use and `engine_version`, for example:
-
-```javascript
-[
-  ...
-  {
-    browser: "samsunginternet_android",
-    version: "27.0",
-    release_date: "2024-11-06",
-    engine: "Blink",
-    engine_version: "125",
-    year: 2023,
-    supports: "widely"
-  },
-  ...
-]
-```
-
-#### `includeKaiOS` (in `getAllVersions()` output)
-
-As with `getCompatibleVersions()` you can include KaiOS in your output. The same requirement to have `includeDownstreamBrowsers: true` applies.
-
-```javascript
-getAllVersions({
-  includeDownstreamBrowsers: true,
-  includeKaiOS: true,
-});
-```
-
-#### `suppressWarnings` (in `getAllVersions()` output)
-
-As with `getCompatibleVersions()`, you can set `suppressWarnings` to `true` to suppress the console warning about old data:
-
-```javascript
-getAllVersions({
-  suppressWarnings: true,
-});
-```
-
-#### `outputFormat`
-
-By default, this function returns an `Array` of `Objects` which can be manipulated in Javascript or output to JSON.
-
-To return an `Object` that nests keys , set `outputFormat` to `object`:
-
-```javascript
-getAllVersions({
-  outputFormat: "object",
-});
-```
-
-In thise case, `getAllVersions()` returns a nested object with the browser [IDs listed below](#list-of-downstream-browsers) as keys, and versions as keys within them:
-
-```javascript
-{
-  "chrome": {
-    "53": {
-      "year": 2016,
-      "release_date": "2016-09-07"
-    },
-    ...
-}
-```
-
-Downstream browsers will include extra fields for `engine` and `engine_versions`
-
-```javascript
-{
-  ...
-  "webview_android": {
-    "53": {
-      "year": 2016,
-      "release_date": "2016-09-07",
-      "engine": "Blink",
-      "engine_version": "53"
-    },
-  ...
-}
-```
-
-To return a `String` in CSV format, set `outputFormat` to `csv`:
-
-```javascript
-getAllVersions({
-  outputFormat: "csv",
-});
-```
-
-`getAllVersions` returns a `String` with a header row and comma-separated values for each browser version that you can write to a file or pass to another service. Core browsers will have "NULL" as the value for their `engine` and `engine_version`:
-
-```csv
-"browser","version","year","supports","release_date","engine","engine_version"
-...
-"chrome","24","pre_baseline","","2013-01-10","NULL","NULL"
-...
-"chrome","53","2016","","2016-09-07","NULL","NULL"
-...
-"firefox","135","2024","widely","2025-02-04","NULL","NULL"
-"firefox","136","2024","newly","2025-03-04","NULL","NULL"
-...
-"ya_android","20.12","2020","year_only","2020-12-20","Blink","87"
-...
-```
-
-> [!NOTE]
-> The above example uses `"includeDownstreamBrowsers": true`
-
-### Static resources
-
-The outputs of `getAllVersions()` are available as JSON or CSV files generated on a daily basis and hosted on GitHub pages:
-
-- Core browsers only
-  - [Array](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions_array.json)
-  - [Object](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions_object.json)
-  - [CSV](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions.csv)
-- Core browsers only, with `supports` property
-  - [Array](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions_array_with_supports.json)
-  - [Object](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions_object_with_supports.json)
-  - [CSV](https://web-platform-dx.github.io/baseline-browser-mapping/all_versions_with_supports.csv)
-- Including downstream browsers
-  - [Array](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions_array.json)
-  - [Object](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions_object.json)
-  - [CSV](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions.csv)
-- Including downstream browsers with `supports` property
-  - [Array](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions_array_with_supports.json)
-  - [Object](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions_object_with_supports.json)
-  - [CSV](https://web-platform-dx.github.io/baseline-browser-mapping/with_downstream/all_versions_with_supports.csv)
-
-These files are updated on a daily basis.
-
-## CLI
-
-`baseline-browser-mapping` includes a command line interface that exposes the same data and options as the `getCompatibleVersions()` function. To learn more about using the CLI, run:
-
-```sh
-npx baseline-browser-mapping --help
-```
-
-## Downstream browsers
-
-### Limitations
-
-The browser versions in this module come from two different sources:
-
-- MDN's `browser-compat-data` module.
-- Parsed user agent strings provided by [useragents.io](https://useragents.io/)
-
-MDN `browser-compat-data` is an authoritative source of information for the browsers it contains. The release dates for the Baseline core browser set and the mapping of downstream browsers to Chromium versions should be considered accurate.
-
-Browser mappings from useragents.io are provided on a best effort basis. They assume that browser vendors are accurately stating the Chromium version they have implemented. The initial set of version mappings was derived from a bulk export in November 2024. This version was iterated over with a Regex match looking for a major Chrome version and a corresponding version of the browser in question, e.g.:
-
-`Mozilla/5.0 (Linux; U; Android 10; en-US; STK-L21 Build/HUAWEISTK-L21) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/100.0.4896.58 UCBrowser/13.8.2.1324 Mobile Safari/537.36`
-
-Shows UC Browser Mobile 13.8 implementing Chromium 100, and:
-
-`Mozilla/5.0 (Linux; arm_64; Android 11; Redmi Note 8 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.6613.123 YaBrowser/24.10.2.123.00 SA/3 Mobile Safari/537.36`
-
-Shows Yandex Browser Mobile 24.10 implementing Chromium 128. The Chromium version from this string is mapped to the corresponding Chrome version from MDN `browser-compat-data`.
-
-> [!NOTE]
-> Where possible, approximate release dates have been included based on useragents.io "first seen" data. useragents.io does not have "first seen" dates prior to June 2020. However, these browsers' Baseline compatibility is determined by their Chromium or Gecko version, so their release dates are more informative than critical.
-
-This data is updated on a daily basis using a [script](https://github.com/web-platform-dx/web-features/tree/main/scripts/refresh-downstream.ts) triggered by a GitHub [action](https://github.com/web-platform-dx/web-features/tree/main/.github/workflows/refresh_downstream.yml). Useragents.io provides a private API for this module which exposes the last 7 days of newly seen user agents for the currently tracked browsers. If a new major version of one of the tracked browsers is encountered with a Chromium version that meets or exceeds the previous latest version of that browser, it is added to the [src/data/downstream-browsers.json](src/data/downstream-browsers.json) file with the date it was first seen by useragents.io as its release date.
-
-KaiOS is an exception - its upstream version mappings are handled separately from the other browsers because they happen very infrequently.
-
-### List of downstream browsers
-
-| Browser               | ID                        | Core    | Source                    |
-| --------------------- | ------------------------- | ------- | ------------------------- |
-| Chrome                | `chrome`                  | `true`  | MDN `browser-compat-data` |
-| Chrome for Android    | `chrome_android`          | `true`  | MDN `browser-compat-data` |
-| Edge                  | `edge`                    | `true`  | MDN `browser-compat-data` |
-| Firefox               | `firefox`                 | `true`  | MDN `browser-compat-data` |
-| Firefox for Android   | `firefox_android`         | `true`  | MDN `browser-compat-data` |
-| Safari                | `safari`                  | `true`  | MDN `browser-compat-data` |
-| Safari on iOS         | `safari_ios`              | `true`  | MDN `browser-compat-data` |
-| Opera                 | `opera`                   | `false` | MDN `browser-compat-data` |
-| Opera Android         | `opera_android`           | `false` | MDN `browser-compat-data` |
-| Samsung Internet      | `samsunginternet_android` | `false` | MDN `browser-compat-data` |
-| WebView Android       | `webview_android`         | `false` | MDN `browser-compat-data` |
-| QQ Browser Mobile     | `qq_android`              | `false` | useragents.io             |
-| UC Browser Mobile     | `uc_android`              | `false` | useragents.io             |
-| Yandex Browser Mobile | `ya_android`              | `false` | useragents.io             |
-| KaiOS                 | `kai_os`                  | `false` | Manual                    |
-| Facebook for Android  | `facebook_android`        | `false` | useragents.io             |
-| Instagram for Android | `instagram_android`       | `false` | useragents.io             |
-
-> [!NOTE]
-> All the non-core browsers currently included implement Chromium or Gecko. Their inclusion in any of the above methods is based on the Baseline feature set supported by the Chromium or Gecko version they implement, not their release date.

node_modules/baseline-browser-mapping/package.json

@@ -1,64 +0,0 @@
-{
-  "name": "baseline-browser-mapping",
-  "main": "./dist/index.cjs",
-  "version": "2.9.19",
-  "description": "A library for obtaining browser versions with their maximum supported Baseline feature set and Widely Available status.",
-  "exports": {
-    ".": {
-      "require": "./dist/index.cjs",
-      "types": "./dist/index.d.ts",
-      "default": "./dist/index.js"
-    },
-    "./legacy": {
-      "require": "./dist/index.cjs",
-      "types": "./dist/index.d.ts"
-    }
-  },
-  "jsdelivr": "./dist/index.js",
-  "files": [
-    "dist/*",
-    "!dist/scripts/*",
-    "LICENSE.txt",
-    "README.md"
-  ],
-  "types": "./dist/index.d.ts",
-  "type": "module",
-  "bin": {
-    "baseline-browser-mapping": "dist/cli.js"
-  },
-  "scripts": {
-    "fix-cli-permissions": "output=$(npx baseline-browser-mapping 2>&1); path=$(printf '%s\n' \"$output\" | sed -n 's/^.*: \\(.*\\): Permission denied$/\\1/p; t; s/^\\(.*\\): Permission denied$/\\1/p'); if [ -n \"$path\" ]; then echo \"Permission denied for: $path\"; echo \"Removing $path ...\"; rm -rf \"$path\"; else echo \"$output\"; fi",
-    "test:format": "npx prettier --check .",
-    "test:lint": "npx eslint .",
-    "test:jasmine": "npx jasmine",
-    "test:jasmine-browser": "npx jasmine-browser-runner runSpecs --config ./spec/support/jasmine-browser.js",
-    "test": "npm run build && npm run fix-cli-permissions && npm run test:format && npm run test:lint && npm run test:jasmine && npm run test:jasmine-browser",
-    "build": "rm -rf dist; npx prettier . --write; rollup -c; rm -rf ./dist/scripts/expose-data.d.ts ./dist/cli.d.ts",
-    "refresh-downstream": "npx tsx scripts/refresh-downstream.ts",
-    "refresh-static": "npx tsx scripts/refresh-static.ts",
-    "update-data-file": "npx tsx scripts/update-data-file.ts; npx prettier ./src/data/data.js --write",
-    "update-data-dependencies": "npm i @mdn/browser-compat-data@latest web-features@latest -D",
-    "check-data-changes": "git diff --name-only | grep -q '^src/data/data.js$' && echo 'changes-available=TRUE' || echo 'changes-available=FALSE'"
-  },
-  "license": "Apache-2.0",
-  "devDependencies": {
-    "@mdn/browser-compat-data": "^7.2.5",
-    "@rollup/plugin-terser": "^0.4.4",
-    "@rollup/plugin-typescript": "^12.1.3",
-    "@types/node": "^22.15.17",
-    "eslint-plugin-new-with-error": "^5.0.0",
-    "jasmine": "^5.8.0",
-    "jasmine-browser-runner": "^3.0.0",
-    "jasmine-spec-reporter": "^7.0.0",
-    "prettier": "^3.5.3",
-    "rollup": "^4.44.0",
-    "tslib": "^2.8.1",
-    "typescript": "^5.7.2",
-    "typescript-eslint": "^8.35.0",
-    "web-features": "^3.14.0"
-  },
-  "repository": {
-    "type": "git",
-    "url": "git+https://github.com/web-platform-dx/baseline-browser-mapping.git"
-  }
-}

pyproject.toml

@@ -2,7 +2,50 @@
 name = "shelfmark"
 version = "0.1.0"
 description = "Shelfmark - Book Downloader"
-requires-python = ">=3.10"
+requires-python = ">=3.14"
+dependencies = [
+    "flask",
+    "flask-cors",
+    "flask-socketio",
+    "python-socketio",
+    "requests[socks]",
+    "defusedxml",
+    "beautifulsoup4",
+    "tqdm",
+    "dnspython",
+    "gunicorn",
+    "gevent",
+    "gevent-websocket",
+    "psutil",
+    "emoji",
+    "rarfile",
+    "qbittorrent-api>=2026.8.1",
+    "transmission-rpc",
+    "authlib>=1.8.0,<1.9",
+    "apprise>=1.13.1",
+    # HTTP/2 client for RFC 8484 DoH: quad9 rejects HTTP/1.1 outright (505), which
+    # requests cannot speak. See shelfmark/download/doh_wireformat.py.
+    "httpx[http2]>=0.28.1",
+]
+
+[project.optional-dependencies]
+browser = [
+    "pyvirtualdisplay",
+    "pyautogui",
+    "seleniumbase==4.54.9",
+    "python-xlib",
+]
+
+[dependency-groups]
+dev = [
+    "basedpyright>=1.40.1",
+    "prek",
+    "pytest",
+    "pytest-cov",
+    "pytest-xdist>=3.8.0",
+    "ruff==0.16.8",
+    "vulture>=2.14",
+]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]
@@ -12,6 +55,8 @@ python_functions = ["test_*"]
 addopts = [
     "-v",
     "--tb=short",
+    "-n",
+    "auto",
 ]
 markers = [
     "integration: marks tests that require running services (deselect with '-m \"not integration\"')",
@@ -19,8 +64,112 @@ markers = [
     "e2e: marks end-to-end tests that require the full application stack",
 ]
 
-[tool.mypy]
-python_version = "3.10"
-warn_return_any = true
-warn_unused_ignores = true
-ignore_missing_imports = true
+[tool.ruff]
+line-length = 100
+extend-exclude = [".local"]
+
+[tool.ruff.lint]
+select = [
+    "F", "I", "UP", "B", "C4", "SIM", "PTH", "RET", "PIE", "FURB", "PERF", "TRY",
+    "A", "DTZ", "N",
+    "BLE001",
+    "ANN001", "ANN002", "ANN003", "ANN201", "ANN202", "ANN204",
+    "E402",
+    "ERA001",
+    "E731",
+    "S101",
+    "S110",
+    "S105", "S108",
+    "S311", "S324",
+    "S607", "S608",
+    "G003", "G004",
+    "PGH003",
+    "PLC0414",
+    "PLR1714",
+    "PLW1510",
+    "PLW2901",
+    "PLW0108",
+    "PT028",
+    "PYI034",
+    "Q000",
+    "RUF005", "RUF012", "RUF013", "RUF059", "RUF100",
+    "TC001", "TC002", "TC003",
+]
+ignore = ["D", "EM", "FBT", "PLR2004", "UP035", "TRY003", "E501", "TD002", "S104", "S603"]
+
+[tool.ruff.lint.per-file-ignores]
+"scripts/**/*.py" = [
+    "BLE001",
+    "S",
+    "TRY",
+]
+"tests/**/*.py" = [
+    "ANN",
+    "BLE001",
+    "B010",
+    "B017",
+    "B028",
+    "DTZ",
+    "E402",
+    "E731",
+    "ERA001",
+    "FURB",
+    "G003",
+    "G004",
+    "PERF",
+    "PIE",
+    "PLC0414",
+    "PLW0108",
+    "PLW1510",
+    "PLW2901",
+    "PTH",
+    "PT028",
+    "PYI034",
+    "Q000",
+    "RET",
+    "RUF012",
+    "S",
+    "SIM",
+    "TC001",
+    "TC002",
+    "TC003",
+    "TRY",
+    "UP028",
+]
+
+[tool.basedpyright]
+include = ["shelfmark"]
+exclude = [".local", "tests", "**/__pycache__", "**/node_modules"]
+pythonVersion = "3.14"
+typeCheckingMode = "standard"
+
+[tool.vulture]
+paths = ["shelfmark"]
+exclude = [".local", "tests"]
+ignore_decorators = [
+    "@app.route",
+    "@app.before_request",
+    "@app.after_request",
+    "@app.errorhandler",
+    "@socketio.on",
+    "@register_provider",
+    "@register_provider_kwargs",
+    "@register_settings",
+    "@register_source",
+    "@register_handler",
+    "@register_client",
+    "@register_output",
+]
+min_confidence = 90
+sort_by_size = true
+
+[tool.coverage.run]
+source = ["shelfmark"]
+branch = true
+
+[tool.coverage.report]
+show_missing = true
+skip_empty = true
+
+[tool.uv]
+package = false

readme.md

@@ -1,30 +1,29 @@
-# 📚 Shelfmark: Book Downloader
-
-Formerly *Calibre Web Automated Book Downloader (CWABD)*
+# 📚 Shelfmark: Book Search & Request Tool
 
 <img src="src/frontend/public/logo.png" alt="Shelfmark" width="200">
 
-Shelfmark is a self-hosted web interface for searching and downloading books and audiobooks from multiple sources. Works out of the box with popular web sources, no configuration required. Add metadata providers, additional release sources, and download clients to build a single hub for your digital library. Supports multiple users with a built-in request system, so you can share your instance with others and let them browse and request books on their own.
+> [!NOTE]
+> Shelfmark is feature stable and maintained on a best-effort basis. Bug fixes, security updates, and small quality-of-life improvements are still shipped, and pull requests are reviewed — including new features. There is no roadmap for new features for now.
 
-**Fully standalone** - no external dependencies required. Works great alongside the following library tools, with support for automatic imports:
+Shelfmark is a self-hosted web interface for searching and requesting books and audiobooks across multiple sources. Bring your own sources, metadata providers, and download clients to build a single hub for your digital library. Supports multiple users with a built-in request system, so you can share your instance with others and let them browse and request books on their own.
+
+Works great alongside the following library tools, with support for automatic imports:
 - [Calibre](https://calibre-ebook.com/)
 - [Calibre-Web](https://github.com/janeczku/calibre-web)
 - [Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated)
-- [Booklore](https://github.com/booklore-app/booklore)
+- [Grimmory](https://github.com/grimmory-tools/grimmory)
 - [Audiobookshelf](https://github.com/advplyr/audiobookshelf)
 
 ## ✨ Features
 
-- **One-Stop Interface** - A clean, modern UI to search, browse, and download from multiple sources in one place
-- **Multiple Sources** - Popular archive websites, Torrent, Usenet, and IRC download support
+- **One-Stop Interface** - A clean, modern UI to search, browse, and download from multiple configured sources in one place
+- **Multiple Sources** - Configurable web, torrent, usenet, and IRC source support
 - **Audiobook Support** - Full audiobook search and download with dedicated processing
-- **Two Search Modes**:
-  - **Direct** - Search popular web sources
-  - **Universal** - Search metadata providers (Hardcover, Open Library) for richer book and audiobook discovery, with multi-source downloads
+- **Flexible Search** - Search metadata providers (Hardcover, Open Library, Google Books) for rich book and audiobook discovery, or query configured sources directly
 - **Multi-User & Requests** - Share your instance with others, let users browse and request books, and manage approvals with configurable notifications
 - **Authentication** - Built-in login, OIDC single sign-on, proxy auth, and Calibre-Web database support
 - **Real-Time Progress** - Unified download queue with live status updates across all sources
-- **Cloudflare Bypass** - Built-in bypasser for reliable access to protected sources
+- **Network Flexibility** - Configurable proxy support, DNS settings, and optional Cloudflare handling for protected sources
 
 ## 🖼️ Screenshots
 
@@ -45,6 +44,7 @@ Shelfmark is a self-hosted web interface for searching and downloading books and
 ### Prerequisites
 
 - Docker & Docker Compose
+- At least 2 GB of RAM available to the container when using the standard image — see [Memory Requirements](#memory-requirements)
 
 ### Installation
 
@@ -60,7 +60,7 @@ Shelfmark is a self-hosted web interface for searching and downloading books and
 
 3. Open `http://localhost:8084`
 
-That's it! Configure settings through the web interface as needed.
+Open the web interface, then configure the sources and settings you want to use.
 
 ### Volume Setup
 
@@ -68,27 +68,56 @@ That's it! Configure settings through the web interface as needed.
 volumes:
   - /your/config/path:/config # Config, database, and artwork cache directory
   - /your/download/path:/books # Downloaded books
-  - /client/path:/client/path # Optional: For Torrent/Usenet downloads, match your client directory exactly. 
+  - /client/path:/client/path # Optional: For Torrent/Usenet downloads, match your client directory exactly.
 ```
 
-> **Tip**: Point the download volume to your CWA or Booklore ingest folder for automatic import.
+> **Tip**: Point the download volume to your CWA or Grimmory ingest folder for automatic import.
 
 > **Note**: CIFS shares require `nobrl` mount option to avoid database lock errors.
 
+### Non-root container mode
+
+- Start the container as `1000:1000` with Docker `user: "1000:1000"` or `docker run --user 1000:1000`.
+- For Kubernetes, set `runAsUser: 1000`, `runAsGroup: 1000`, and `runAsNonRoot: true` together.
+- `PUID`/`PGID` keep the default root startup flow.
+- Mounted paths must already be writable by `1000:1000`.
+- `USING_TOR=true` requires root startup.
+
 ## ⚙️ Configuration
 
 ### Search Modes
 
-**Direct** (default)
-- Works out of the box, no setup required
-- Searches a huge library of books directly
-- Returns downloadable releases immediately
+**Direct**
+- Queries configured sources directly
 
-**Universal**
-- Cleaner search results via metadata providers (Hardcover is recommended)
+**Universal** (recommended)
+- Search via metadata providers (Hardcover, Open Library, Google Books) for richer results
 - Aggregates releases from multiple configured sources
-- Full Audiobook support
-- Requires manual setup (API keys, additional sources)
+- Full audiobook support
+
+### Hardcover API Key
+
+Hardcover powers metadata search in Universal mode. Create a token at
+[hardcover.app/account/api](https://hardcover.app/account/api) — current keys start with `hc_pat_`
+and are far shorter than the JWTs Hardcover issued before August 2026.
+
+Tick these seven scopes on the token screen:
+
+| Scope | Used for |
+|-------|----------|
+| `read:catalog` | Metadata search, plus book, edition, author and series lookups |
+| `read:library` | Your reading status and shelf counts |
+| `read:lists` | Your lists and the books on them |
+| `read:me:content` | Test Connection and the "Connected as" label |
+| `read:users` | Usernames shown alongside lists |
+| `write:library` | Setting a book's reading status from Shelfmark |
+| `write:lists` | Adding and removing books from lists, including auto-remove on download |
+
+The two `write:` scopes matter only if you set reading status from Shelfmark or leave
+**Auto-Remove from List on Download** enabled (it is on by default) — without them those actions
+fail silently. Everything else Hardcover offers (journal, goals, reviews, prompts, notifications,
+account) can stay unticked. The `all` scope works too, but it grants full account access including
+deletion, so prefer the list above.
 
 ### Environment Variables
 
@@ -99,20 +128,30 @@ Environment variables work for initial setup and Docker deployments. They serve
 | `FLASK_PORT` | Web interface port | `8084` |
 | `INGEST_DIR` | Book download directory | `/books` |
 | `TZ` | Container timezone | `UTC` |
-| `PUID` / `PGID` | Runtime user/group ID (also supports legacy `UID`/`GID`) | `1000` / `1000` |
-| `SEARCH_MODE` | `direct` or `universal` | `direct` |
-| `USING_TOR` | Enable Tor routing (requires `NET_ADMIN` capability) | `false` |
+| `PUID` / `PGID` | Runtime user/group for the default root-startup flow (also supports legacy `UID`/`GID`) | `1000` / `1000` |
+| `SEARCH_MODE` | `direct` or `universal` | `universal` |
+| `USING_TOR` | Enable Tor routing (requires root startup) | `false` |
+| `USING_WIREGUARD` | Enable WireGuard VPN egress with kill-switch (requires root startup) | `false` |
+| `WIREGUARD_CONFIG` | Path to the mounted wg-quick config | `/config/wg0.conf` |
+| `WIREGUARD_INTERFACE` | WireGuard interface name | `wg0` |
+| `LAN_NETWORK` | Comma-separated CIDRs kept off the tunnel so the WebUI / internal clients stay reachable | `127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16` |
+| `WIREGUARD_ENFORCE_DNS` | Pin the resolver (via `WIREGUARD_DNS`, else the config's `DNS =`) so DNS can't silently fall back to an off-tunnel path. Designed for a trusted LAN resolver kept reachable via `LAN_NETWORK` (query leaves over the LAN; download still egresses via the tunnel) — it does **not** force queries through the tunnel. Docker's embedded resolver (`127.0.0.11`) is preserved when present so container-name resolution keeps working; pin its upstream via the container's `dns:` list. Fails closed if no resolver is available or `/etc/resolv.conf` is not writable. | `true` |
+| `WIREGUARD_DNS` | Explicit resolver(s) to pin (comma/space separated). Use when the VPN's pushed DNS filters domains you need; point at a resolver reachable via the tunnel or an allowed LAN resolver. | _(unset; uses config `DNS =`)_ |
+| `WIREGUARD_DISABLE_IPV6` | Strip IPv6 from the tunnel config (many container kernels lack the ip6tables `raw` table wg-quick needs) and remove IPv6 as a leak surface. | `true` |
+| `WIREGUARD_ALLOW_IPV6_LEAK` | Escape hatch: continue even when an IPv6 kill-switch can't be installed AND IPv6 can't be disabled. Only set if the container has no IPv6 connectivity. | `false` |
+| `WIREGUARD_ALLOW_WEBUI_OFFTUNNEL` | Opt-in off-tunnel WebUI reachability. Default (`false`) keeps the kill-switch strictly fail-closed: the only off-tunnel egress is loopback, the tunnel device and the LAN allowlist. Set `true` only if a **non-LAN** client (e.g. a public reverse proxy on another segment) must reach the WebUI; it permits app-server **replies** (`--sport FLASK_PORT`, conntrack REPLY) off-tunnel — server replies only, never client-initiated egress. LAN clients never need it (covered by `LAN_NETWORK`). | `false` |
+| `WIREGUARD_STALE_AFTER` | Seconds since the last handshake before the healthcheck bounces the tunnel. | `180` |
 
 See the full [Environment Variables Reference](docs/environment-variables.md) for all available options.
 
 Some of the additional options available in Settings:
-- **Fast Download Key** - Use your paid account to skip Cloudflare challenges entirely and use faster, direct downloads
 - **Prowlarr** - Configure indexers and download clients to download books and audiobooks
-- **AudiobookBay** - Web scraping source for audiobook torrents (audiobooks only)
-- **IRC** - Add details for IRC book sources and download directly from the UI
-- **Library Link** - Add a link to your Calibre-Web or Booklore instance in the UI header
+- **Additional audiobook sources** - Configure additional sources for audiobook discovery
+- **Direct Download mirrors** - Supply your own Anna's Archive mirror URLs; Auto mode tries them in the order listed. The `annas-archive.is` domain does not currently work as a source — use `annas-archive.gl` instead (checked August 2026; mirror availability changes)
+- **IRC** - Add details for IRC book sources and download directly from the UI. Most networks serve audiobooks from the same channel as ebooks (on `irc.irchighway.net` that's `#ebooks`, while `#bookz` is effectively inactive), so leave the separate audiobook channel blank unless your network actually indexes one. IRC audiobooks usually arrive as ZIP/RAR archives — keep those enabled under Supported Audiobook Formats or the releases are filtered out of results
+- **Library Link** - Add a link to your Calibre-Web or Grimmory instance in the UI header
 - **File processing** - Customiseable download paths, file renaming and directory creation with template-based renaming
-- **Network Resilience** - Auto DNS rotation and mirror fallback when sources are unreachable. Custom proxy support (SOCK5 + HTTP/S), Tor routing.
+- **Network Settings** - Custom proxy support (SOCKS5 + HTTP/S) and configurable DNS
 - **Format & Language** - Filter downloads by preferred formats, languages and sorting order
 - **Metadata Providers** - Configure API keys for Hardcover, Open Library, etc.
 
@@ -123,34 +162,64 @@ Some of the additional options available in Settings:
 docker compose up -d
 ```
 
-The full-featured image with built-in Cloudflare bypass.
+The full-featured image with all network capabilities included.
 
-#### Enable Tor Routing
-Routes all traffic through Tor for enhanced privacy:
+#### Memory Requirements
+
+The standard image ships a real Chromium browser, which it launches to solve Cloudflare challenges for Direct Download. Chromium needs room to run:
+
+- **2 GB of RAM available to the container** is a safe minimum; 1 GB or less is where problems usually start
+- Only relevant if you use Direct Download. Prowlarr, IRC and audiobook sources don't start the browser
+
+When the container is starved of memory, Chromium fails to start and every Direct Download fails with unrelated-looking errors — repeated `403 detected; switching to bypasser` followed by `No download URL found`, and downloads that never complete. If you're seeing that, check the container's memory limit and the host's free memory before suspecting your ISP or DNS.
+
+If you can't spare the memory, use the [Lite](#lite) image with an external resolver (e.g. FlareSolverr) running elsewhere.
+
+#### Tor Routing
+Optional Tor support for network privacy:
 ```bash
 curl -O https://raw.githubusercontent.com/calibrain/shelfmark/main/compose/docker-compose.tor.yml
 docker compose -f docker-compose.tor.yml up -d
 ```
 
 **Notes:**
+- Requires root startup
 - Requires `NET_ADMIN` and `NET_RAW` capabilities
 - Timezone is auto-detected from Tor exit node
 - Custom DNS/proxy settings are ignored when Tor is active
 
-### Lite
-A smaller image without the built-in Cloudflare bypasser. Ideal for:
+#### WireGuard VPN Routing
+Optional WireGuard support to route all external egress through a VPN tunnel with a fail-closed kill-switch:
+```bash
+curl -O https://raw.githubusercontent.com/calibrain/shelfmark/main/compose/docker-compose.wireguard.yml
+# place your wg-quick config where the compose mounts /config, as wg0.conf
+docker compose -f docker-compose.wireguard.yml up -d
+```
 
-- **External bypassers** - Already running FlareSolverr or ByParr for other services
-- **Fast downloads** - Using fast download sources
-- **Alternative sources only** - Exclusively using Prowlarr, AudiobookBay, IRC, or other sources
-- **Audiobooks** - Using Shelfmark exclusively for audiobooks
+**Notes:**
+- Requires root startup
+- Requires `NET_ADMIN` and `NET_RAW` capabilities
+- Mount a standard wg-quick config at `WIREGUARD_CONFIG` (default `/config/wg0.conf`)
+- All non-LAN egress is forced through the tunnel; if the tunnel drops, external traffic **fails closed** while LAN ranges (WebUI, Prowlarr, qBittorrent) stay reachable
+- IPv4 and IPv6 both fail closed. On kernels without a usable `ip6tables`, disable IPv6 for the container (`sysctls: net.ipv6.conf.all.disable_ipv6=1`, as in the compose example) or the container refuses to start rather than risk an IPv6 leak
+- A supervised healthcheck bounces the tunnel if the handshake goes stale, and refreshes the endpoint allow rules so a roaming/rotated peer endpoint can reconnect
+- Mutually exclusive with `USING_TOR`
+- **DNS trust:** `WIREGUARD_DNS` must be a resolver you trust on a trusted network segment. When it is a LAN resolver (kept reachable off-tunnel by `LAN_NETWORK`), the query to that resolver leaves as plaintext UDP/53 on the LAN — the resolver is responsible for encrypting upstream. Two resolver paths exist: (1) when Docker's embedded resolver (`127.0.0.11`) is present it is **preserved** so container names (Prowlarr, qBittorrent) resolve — you MUST pin its upstream to a trusted resolver via the container's compose `dns:` list, since `WIREGUARD_DNS` cannot repoint the embedded resolver from inside the container; (2) otherwise `WIREGUARD_DNS`/the config `DNS =` line is written to `/etc/resolv.conf`. Setting `WIREGUARD_ENFORCE_DNS=false` is a **foot-gun**: with no embedded resolver present the container then uses its inherited resolver, which forwards to the Docker daemon's upstream **off-tunnel**, leaking your DNS. Leave enforcement on unless you have pinned the resolver another way.
+
+### Lite
+A lighter image without the built-in browser automation. Ideal for:
+
+- **External services** - Already running FlareSolverr or similar for other applications
+- **Alternative sources** - Using Prowlarr, IRC, or other configured sources
+- **Audiobooks** - Using Shelfmark primarily for audiobooks
+- **Constrained hosts** - No bundled browser, so it runs comfortably below the standard image's [memory requirements](#memory-requirements)
 
 ```bash
 curl -O https://raw.githubusercontent.com/calibrain/shelfmark/main/compose/docker-compose.lite.yml
 docker compose -f docker-compose.lite.yml up -d
 ```
 
-If you need Cloudflare bypass with the Lite image, configure an external resolver (FlareSolverr/ByParr) in Settings under the Cloudflare tab.
+If you need browser-based access with the Lite image, configure an external resolver in Settings.
 
 ## 🔐 Authentication
 
@@ -179,6 +248,29 @@ volumes:
 
 With any authentication method enabled, Shelfmark supports multi-user management with admin/user roles. Users can have per-user settings for download destinations, email recipients, and notification preferences. Non-admin users only see their own downloads and can submit book requests for admin review. Admins can configure request policies per source to control whether users can download directly, must submit a request, or are blocked entirely.
 
+See [API Access](docs/api-access.md) to call the API with a static key from scripts and integrations.
+
+## Project Scope
+
+Shelfmark is a manual search and download tool, the entry point to your book library, not a library manager. It finds books, downloads them, and sends them to a configured destination. That's the full scope.
+
+Shelfmark intentionally does not:
+
+- **Track or manage your library** - it doesn't know or care what you already own
+- **Integrate with library software** - what happens after delivery is up to your library tool
+- **Monitor authors, series, or new releases** - there is no background automation
+- **Queue future downloads** - if a book isn't available now, Shelfmark won't watch for it
+
+These are non-goals, not missing features.
+
+## Contributing
+
+Shelfmark's core feature set is complete.
+
+Pull requests are welcome and all of them get reviewed, new features included. If you want a feature, the fastest path is to send a PR for it rather than to file a request.
+
+Feature requests that fall outside the project scope (library integration, automation, collection management) will be closed, and PRs implementing them won't be merged. If you're unsure whether something fits, open a discussion first.
+
 ## Health Monitoring
 
 The application exposes a health endpoint at `/api/health` (no authentication required). Add a health check to your compose:
@@ -197,16 +289,24 @@ Logs are available via:
 - `docker logs <container-name>`
 - `/var/log/shelfmark/` inside the container (when `ENABLE_LOGGING=true`)
 
-Log level is configurable via Settings or `LOG_LEVEL` environment variable.
+Log level is configurable under Settings → Advanced or via the `LOG_LEVEL` environment
+variable (`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`; case-insensitive, defaults to
+`INFO`). The environment variable wins over the setting, and `DEBUG=true` forces `DEBUG`
+regardless of either. Changes take effect on restart.
 
 ## Development
 
 ```bash
+# Quality checks
+make checks              # Run ALL static analysis (frontend + Python)
+make python-checks       # Run Ruff, BasedPyright, and Vulture
+make install-python-dev  # Sync Python runtime + dev tools with uv
+
 # Frontend development
 make install     # Install dependencies
 make dev         # Start Vite dev server (localhost:5173)
 make build       # Production build
-make typecheck   # TypeScript checks
+make frontend-typecheck  # TypeScript checks
 
 # Backend (Docker)
 make up          # Start backend via docker-compose.dev.yml
@@ -217,55 +317,20 @@ make restart     # Restart container
 
 The frontend dev server proxies to the backend on port 8084.
 
-### Architecture
-
-```
-┌─────────────────────────────────────────────────────────────┐
-│                      Web Interface                          │
-│                 (React + TypeScript + Vite)                 │
-├─────────────────────────────────────────────────────────────┤
-│                      Flask Backend                          │
-│                   (REST API + WebSocket)                    │
-├───────────────────┬─────────────────────┬───────────────────┤
-│ Metadata Providers│   Download Queue    │  Cloudflare       │
-│                   │   & Orchestrator    │  Bypass           │
-├───────────────────┼─────────────────────┼───────────────────┤
-│ • Hardcover       │ • Task scheduling   │ • Internal        │
-│ • Open Library    │ • Progress tracking │ • External        │
-│                   │ • Retry logic       │   (FlareSolverr)  │
-├───────────────────┴─────────────────────┴───────────────────┤
-│                     Release Sources                         │
-├─────────────────────────────────────────────────────────────┤
-│ • Direct Download (Web Sources → Mirrors → Fallbacks)       │
-├─────────────────────────────────────────────────────────────┤
-│                     Network Layer                           │
-├─────────────────────────────────────────────────────────────┤
-│ • Auto DNS rotation  • Mirror failover  • Resume support    │
-└─────────────────────────────────────────────────────────────┘
-```
-
-The backend uses a plugin architecture. Metadata providers and release sources register via decorators and are automatically discovered.
-
-## Contributing
-
-Shelfmark's core feature set is now largely complete. Development going forward will focus on stability, bug fixes, and maintenance rather than major new features. Contributions in these areas are welcome - please file issues or submit pull requests on GitHub.
-
 ## License
 
 MIT License - see [LICENSE](LICENSE) for details.
 
-## ⚠️ Disclaimers
+## ⚠️ Disclaimer
 
-### Copyright Notice
+Shelfmark is a search interface that displays results from external metadata providers and sources. It does not host, store, or distribute any content. The developers are not responsible for how the tool is used or what is accessed through it.
 
-This tool can access various sources including those that might contain copyrighted material. Users are responsible for:
-- Ensuring they have the right to download requested materials
-- Respecting copyright laws and intellectual property rights
-- Using the tool in compliance with their local regulations
+Users are solely responsible for:
+- Ensuring they have the legal right to download any material they access
+- Complying with copyright laws and intellectual property rights in their jurisdiction
+- Understanding and accepting the terms of any sources they configure
 
-### Library Integration
-
-Downloads are written atomically (via intermediate `.crdownload` files) to prevent partial files from being ingested. However, if your library tool (CWA, Booklore, Calibre) is actively scanning or importing, there's a small chance of race conditions. If you experience database errors or import failures, try pausing your library's auto-import during bulk downloads.
+Use of this tool is entirely at your own risk.
 
 ## Support
 

requirements-base.txt

@@ -1,18 +0,0 @@
-flask>=3.1.0,<3.1.3  # Temporary: Flask 3.1.3 breaks flask-socketio (github.com/miguelgrinberg/Flask-SocketIO/pull/2153)
-flask-cors
-flask-socketio
-python-socketio
-requests[socks]
-beautifulsoup4
-tqdm
-dnspython
-gunicorn
-gevent
-gevent-websocket
-psutil
-emoji
-rarfile
-qbittorrent-api
-transmission-rpc
-authlib>=1.6.6,<1.7
-apprise>=1.9.0

requirements-shelfmark.txt

@@ -1,4 +0,0 @@
-pyvirtualdisplay
-pyautogui
-seleniumbase==4.45.10
-python-xlib

scripts/bypasser_permission_lab.sh

@@ -0,0 +1,246 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+LATEST_IMAGE="${LATEST_IMAGE:-ghcr.io/calibrain/shelfmark:latest}"
+LEGACY_IMAGE="${LEGACY_IMAGE:-ghcr.io/calibrain/shelfmark:v1.0.2}"
+WAIT_SECONDS="${WAIT_SECONDS:-5}"
+STARTUP_TIMEOUT_SECONDS="${STARTUP_TIMEOUT_SECONDS:-120}"
+
+require_cmd() {
+    command -v "$1" >/dev/null 2>&1 || {
+        echo "Missing required command: $1" >&2
+        exit 1
+    }
+}
+
+cleanup() {
+    local name="$1"
+    docker rm -f "$name" >/dev/null 2>&1 || true
+}
+
+wait_for_startup() {
+    local name="$1"
+    local elapsed=0
+
+    while [ "$elapsed" -lt "$STARTUP_TIMEOUT_SECONDS" ]; do
+        if ! docker inspect "$name" >/dev/null 2>&1; then
+            echo "Container $name no longer exists" >&2
+            return 1
+        fi
+
+        if [ "$(docker inspect -f '{{.State.Status}}' "$name" 2>/dev/null)" != "running" ]; then
+            echo "Container $name exited before startup completed" >&2
+            docker logs --tail 120 "$name" 2>&1 || true
+            return 1
+        fi
+
+        if docker exec "$name" sh -lc "getent passwd 1000 >/dev/null 2>&1 && ps -eo comm,args | awk '\$1 == \"gunicorn\" && index(\$0, \"shelfmark.main:app\") { found=1 } END { exit(found ? 0 : 1) }'" >/dev/null 2>&1; then
+            return 0
+        fi
+
+        sleep 1
+        elapsed=$((elapsed + 1))
+    done
+
+    echo "Timed out waiting for $name to finish startup" >&2
+    docker logs --tail 120 "$name" 2>&1 || true
+    return 1
+}
+
+start_container() {
+    local name="$1"
+    local image="$2"
+    local pre_entrypoint_script="${3:-}"
+
+    cleanup "$name"
+
+    if [ -n "$pre_entrypoint_script" ]; then
+        docker run -d \
+            --name "$name" \
+            --entrypoint sh \
+            -e PUID=1000 \
+            -e PGID=1000 \
+            -e TZ=UTC \
+            "$image" \
+            -lc "$pre_entrypoint_script
+exec /app/entrypoint.sh" >/dev/null
+    else
+        docker run -d \
+            --name "$name" \
+            -e PUID=1000 \
+            -e PGID=1000 \
+            -e TZ=UTC \
+            "$image" >/dev/null
+        sleep "$WAIT_SECONDS"
+    fi
+
+    wait_for_startup "$name"
+}
+
+run_probe() {
+    local name="$1"
+    local mode="${2:-default}"
+    docker exec -u 1000:1000 -e PROBE_MODE="$mode" "$name" sh -lc 'python3 - <<'"'"'PY'"'"'
+import asyncio
+import os
+import shelfmark.bypass.internal_bypasser as ib
+
+
+async def run_probe():
+    driver = None
+    probe_mode = os.environ.get("PROBE_MODE", "default")
+
+    if probe_mode == "proxy_auth" and hasattr(ib, "_get_proxy_string"):
+        ib._get_proxy_string = lambda _url: "user:pass@127.0.0.1:8888"
+
+    if hasattr(ib, "_create_cdp_browser"):
+        try:
+            driver = await ib._create_cdp_browser("https://example.com")
+            profile = getattr(getattr(driver, "config", None), "user_data_dir", "")
+            print(f"PROBE=OK mode={probe_mode} fn=_create_cdp_browser profile={profile}")
+        except Exception as e:
+            print(f"PROBE=ERR mode={probe_mode} fn=_create_cdp_browser type={type(e).__name__} msg={e}")
+        finally:
+            if driver and hasattr(ib, "_close_cdp_driver"):
+                await ib._close_cdp_driver(driver)
+        return
+
+    if hasattr(ib, "_create_driver"):
+        try:
+            driver = await ib._create_driver()
+            print(f"PROBE=OK mode={probe_mode} fn=_create_driver driver_type={type(driver).__name__}")
+        except Exception as e:
+            print(f"PROBE=ERR mode={probe_mode} fn=_create_driver type={type(e).__name__} msg={e}")
+        finally:
+            if driver and hasattr(ib, "_quit_driver"):
+                await ib._quit_driver(driver)
+        return
+
+    print(f"PROBE=ERR mode={probe_mode} fn=unknown type=RuntimeError msg=no supported startup function found")
+
+
+asyncio.run(run_probe())
+PY'
+}
+
+show_logs() {
+    local name="$1"
+    docker logs --tail 80 "$name" 2>&1 | tail -n 20
+}
+
+scenario_latest_baseline() {
+    local name="sb-lab-latest-baseline"
+    echo
+    echo "== latest baseline =="
+    start_container "$name" "$LATEST_IMAGE"
+    run_probe "$name"
+    cleanup "$name"
+}
+
+scenario_latest_drivers_readonly() {
+    local name="sb-lab-latest-drivers"
+    echo
+    echo "== latest drivers readonly =="
+    start_container "$name" "$LATEST_IMAGE" '
+        chown -R root:root /usr/local/lib/python3.10/site-packages/seleniumbase/drivers &&
+        chmod -R a-w /usr/local/lib/python3.10/site-packages/seleniumbase/drivers &&
+        ls -ld /usr/local/lib/python3.10/site-packages/seleniumbase/drivers
+    '
+    run_probe "$name"
+    cleanup "$name"
+}
+
+scenario_latest_proxy_auth_baseline() {
+    local name="sb-lab-latest-proxy-baseline"
+    echo
+    echo "== latest proxy auth baseline =="
+    start_container "$name" "$LATEST_IMAGE"
+    run_probe "$name" "proxy_auth"
+    cleanup "$name"
+}
+
+scenario_latest_downloads_readonly() {
+    local name="sb-lab-latest-downloads"
+    echo
+    echo "== latest downloaded_files readonly =="
+    start_container "$name" "$LATEST_IMAGE" '
+        mkdir -p /app/downloaded_files &&
+        touch /app/downloaded_files/pipfinding.lock /app/downloaded_files/proxy_dir.lock &&
+        chown -R root:root /app/downloaded_files &&
+        chmod -R a-w /app/downloaded_files &&
+        find /app/downloaded_files -maxdepth 2 -printf "%M %u:%g %p\n"
+    '
+    run_probe "$name"
+    show_logs "$name"
+    cleanup "$name"
+}
+
+scenario_latest_proxy_auth_downloads_readonly() {
+    local name="sb-lab-latest-proxy-downloads"
+    echo
+    echo "== latest proxy auth with readonly downloaded_files =="
+    start_container "$name" "$LATEST_IMAGE" '
+        mkdir -p /app/downloaded_files &&
+        touch /app/downloaded_files/pipfinding.lock /app/downloaded_files/proxy_dir.lock &&
+        chown 1000:1000 /app/downloaded_files/pipfinding.lock /app/downloaded_files/proxy_dir.lock &&
+        chmod 0666 /app/downloaded_files/pipfinding.lock /app/downloaded_files/proxy_dir.lock &&
+        chown root:root /app/downloaded_files &&
+        chmod 0555 /app/downloaded_files &&
+        ls -ld /app/downloaded_files &&
+        ls -la /app/downloaded_files
+    '
+    run_probe "$name" "proxy_auth"
+    show_logs "$name"
+    cleanup "$name"
+}
+
+scenario_latest_bind_mount_readonly() {
+    local name="sb-lab-latest-bind-ro"
+    local bind_dir
+    bind_dir="$(mktemp -d /tmp/sb-lab-bind.XXXXXX)"
+    echo
+    echo "== latest readonly bind mount for downloaded_files =="
+    chmod 0555 "$bind_dir"
+    cleanup "$name"
+    docker run -d \
+        --name "$name" \
+        -e PUID=1000 \
+        -e PGID=1000 \
+        -e TZ=UTC \
+        --mount "type=bind,src=${bind_dir},target=/app/downloaded_files,readonly" \
+        "$LATEST_IMAGE" >/dev/null
+    wait_for_startup "$name"
+    run_probe "$name"
+    show_logs "$name"
+    cleanup "$name"
+    rm -rf "$bind_dir"
+}
+
+scenario_legacy_drivers_readonly() {
+    local name="sb-lab-legacy-drivers"
+    echo
+    echo "== legacy drivers readonly =="
+    start_container "$name" "$LEGACY_IMAGE" '
+        chown -R root:root /usr/local/lib/python3.10/site-packages/seleniumbase/drivers &&
+        chmod -R a-w /usr/local/lib/python3.10/site-packages/seleniumbase/drivers &&
+        ls -ld /usr/local/lib/python3.10/site-packages/seleniumbase/drivers
+    '
+    run_probe "$name"
+    show_logs "$name"
+    cleanup "$name"
+}
+
+main() {
+    require_cmd docker
+
+    scenario_latest_baseline
+    scenario_latest_drivers_readonly
+    scenario_latest_proxy_auth_baseline
+    scenario_latest_downloads_readonly
+    scenario_latest_proxy_auth_downloads_readonly
+    scenario_latest_bind_mount_readonly
+    scenario_legacy_drivers_readonly
+}
+
+main "$@"

scripts/fix_permissions.py

@@ -1,92 +0,0 @@
-#!/usr/bin/env python3
-"""Fix permissions on all configured directories.
-
-This script is called by the entrypoint to ensure all user-configured
-directories have correct ownership. It reads directory paths from:
-- CONFIG_DIR environment variable
-- Config files in CONFIG_DIR/plugins/
-
-Outputs directory paths that need permission fixing (one per line).
-The entrypoint handles the actual chown operations.
-"""
-
-import json
-import os
-import sys
-from pathlib import Path
-
-
-def get_directories_from_config() -> set[str]:
-    """Extract all directory paths from config files."""
-    directories = set()
-
-    config_dir = Path(os.getenv("CONFIG_DIR", "/config"))
-    plugins_dir = config_dir / "plugins"
-
-    if not plugins_dir.exists():
-        return directories
-
-    # Keys that contain directory paths
-    directory_keys = {
-        # Main destinations
-        "DESTINATION",
-        "DESTINATION_AUDIOBOOK",
-        # Content type routing directories
-        "AA_CONTENT_TYPE_DIR_FICTION",
-        "AA_CONTENT_TYPE_DIR_NON_FICTION",
-        "AA_CONTENT_TYPE_DIR_UNKNOWN",
-        "AA_CONTENT_TYPE_DIR_MAGAZINE",
-        "AA_CONTENT_TYPE_DIR_COMIC",
-        "AA_CONTENT_TYPE_DIR_STANDARDS",
-        "AA_CONTENT_TYPE_DIR_MUSICAL_SCORE",
-        "AA_CONTENT_TYPE_DIR_OTHER",
-        # Legacy keys (in case of old configs)
-        "INGEST_DIR",
-        "INGEST_DIR_AUDIOBOOK",
-        "INGEST_DIR_BOOK_FICTION",
-        "INGEST_DIR_BOOK_NON_FICTION",
-        "INGEST_DIR_BOOK_UNKNOWN",
-        "INGEST_DIR_MAGAZINE",
-        "INGEST_DIR_COMIC_BOOK",
-        "INGEST_DIR_STANDARDS_DOCUMENT",
-        "INGEST_DIR_MUSICAL_SCORE",
-        "INGEST_DIR_OTHER",
-        "LIBRARY_PATH",
-        "LIBRARY_PATH_AUDIOBOOK",
-    }
-
-    # Read all JSON config files
-    for config_file in plugins_dir.glob("*.json"):
-        try:
-            with open(config_file, "r") as f:
-                config = json.load(f)
-
-            for key in directory_keys:
-                if key in config:
-                    value = config[key]
-                    if value and isinstance(value, str) and value.startswith("/"):
-                        directories.add(value)
-        except (json.JSONDecodeError, OSError):
-            continue
-
-    return directories
-
-
-def main():
-    """Output all configured directories that exist."""
-    directories = get_directories_from_config()
-
-    # Filter to directories that actually exist
-    existing = []
-    for dir_path in directories:
-        path = Path(dir_path)
-        if path.exists() and path.is_dir():
-            existing.append(dir_path)
-
-    # Output one directory per line
-    for dir_path in sorted(existing):
-        print(dir_path)
-
-
-if __name__ == "__main__":
-    main()

scripts/generate_env_docs.py

@@ -17,16 +17,15 @@ The generated documentation includes:
 
 import argparse
 import sys
-from datetime import datetime
 from pathlib import Path
-from typing import Any, Dict, List, Optional
+from typing import Any
 
 # Add project root to path
 project_root = Path(__file__).resolve().parent.parent
 sys.path.insert(0, str(project_root))
 
 
-def get_field_type_name(field) -> str:
+def get_field_type_name(field: Any) -> str:
     """Get a human-readable type name for a field."""
     from shelfmark.core.settings_registry import (
         CheckboxField,
@@ -35,54 +34,55 @@ def get_field_type_name(field) -> str:
         OrderableListField,
         PasswordField,
         SelectField,
+        TagListField,
         TextField,
     )
 
     if isinstance(field, CheckboxField):
         return "boolean"
-    elif isinstance(field, NumberField):
+    if isinstance(field, NumberField):
         return "number"
-    elif isinstance(field, SelectField):
+    if isinstance(field, SelectField):
         return "string (choice)"
-    elif isinstance(field, MultiSelectField):
+    if isinstance(field, MultiSelectField):
         return "string (comma-separated)"
-    elif isinstance(field, OrderableListField):
+    if isinstance(field, TagListField):
+        return "string (comma-separated)"
+    if isinstance(field, OrderableListField):
         return "JSON array"
-    elif isinstance(field, PasswordField):
+    if isinstance(field, PasswordField):
         return "string (secret)"
-    elif isinstance(field, TextField):
-        return "string"
-    else:
+    if isinstance(field, TextField):
         return "string"
+    return "string"
 
 
-def format_default_value(field) -> str:
+def format_default_value(field: Any) -> str:
     """Format the default value for display."""
     default = field.default
 
     if default is None:
         return "_none_"
-    elif isinstance(default, bool):
+    if isinstance(default, bool):
         return f"`{str(default).lower()}`"
-    elif isinstance(default, (int, float)):
+    if isinstance(default, (int, float)):
         return f"`{default}`"
-    elif isinstance(default, str):
+    if isinstance(default, str):
         if default == "":
             return "_empty string_"
         return f"`{default}`"
-    elif isinstance(default, list):
+    if isinstance(default, list):
         if not default:
             return "_empty list_"
         # For simple lists, show comma-separated values
         if all(isinstance(item, str) for item in default):
             return f"`{','.join(default)}`"
         # For complex lists (e.g., OrderableListField defaults), summarize
-        return f"_see UI for defaults_"
-    else:
-        return f"`{default}`"
+        return "_see UI for defaults_"
+    return f"`{default}`"
 
 
-def get_select_options(field) -> Optional[List[str]]:
+def get_select_options(field: Any) -> list[str] | None:
     """Get the available options for a SelectField.
 
     Returns options formatted as 'value (label)' or just 'value' if they match,
@@ -119,7 +119,7 @@ def get_select_options(field) -> Optional[List[str]]:
     return result
 
 
-def _generate_bootstrap_env_docs() -> List[str]:
+def _generate_bootstrap_env_docs() -> list[str]:
     """Generate documentation for bootstrap environment variables from env.py."""
     # These are environment variables defined in env.py that are used before
     # the settings registry is available
@@ -172,6 +172,30 @@ def _generate_bootstrap_env_docs() -> List[str]:
             "type": "string (path)",
             "default": "/auth/app.db",
         },
+        {
+            "name": "HIDE_LOCAL_AUTH",
+            "description": "Hide the username/password login form when OIDC is active.",
+            "type": "boolean",
+            "default": "false",
+        },
+        {
+            "name": "DISABLE_LOCAL_AUTH",
+            "description": "Disable username/password login and remove the local-admin prerequisite for OIDC. Implies HIDE_LOCAL_AUTH; with AUTH_METHOD=builtin, everyone is locked out until auth env vars are changed.",
+            "type": "boolean",
+            "default": "false",
+        },
+        {
+            "name": "SHELFMARK_API_KEY",
+            "description": "Optional static API key. When set, requests carrying it as 'Authorization: Bearer <key>' (or X-Api-Key) are authenticated as an admin; browser sessions keep working. Unset = off.",
+            "type": "string",
+            "default": "unset",
+        },
+        {
+            "name": "OIDC_AUTO_REDIRECT",
+            "description": "Automatically redirect to the OIDC provider instead of showing the login page.",
+            "type": "boolean",
+            "default": "false",
+        },
         {
             "name": "DOCKERMODE",
             "description": "Indicates the application is running inside a Docker container.",
@@ -189,14 +213,16 @@ def _generate_bootstrap_env_docs() -> List[str]:
     lines = [
         "## Bootstrap Configuration",
         "",
-        "These environment variables are used at startup before the settings system loads. They typically configure paths and server settings.",
+        "These environment variables are used at startup before the settings system loads. They typically configure paths, server settings, and authentication startup behavior.",
         "",
         "| Variable | Description | Type | Default |",
         "|----------|-------------|------|---------|",
     ]
 
-    for var in bootstrap_vars:
-        lines.append(f"| `{var['name']}` | {var['description']} | {var['type']} | `{var['default']}` |")
+    lines.extend(
+        f"| `{var['name']}` | {var['description']} | {var['type']} | `{var['default']}` |"
+        for var in bootstrap_vars
+    )
 
     lines.append("")
     lines.append("<details>")
@@ -218,19 +244,124 @@ def _generate_bootstrap_env_docs() -> List[str]:
     return lines
 
 
+def _generate_egress_env_docs() -> list[str]:
+    """Generate documentation for VPN/Tor egress environment variables.
+
+    These are startup-only variables consumed by entrypoint.sh / wireguard.sh
+    (before and outside the settings registry) to select and configure the
+    transparent-egress kill-switch. `USING_TOR` has a registry-backed entry
+    under Network and is cross-referenced rather than repeated here so the two
+    mutually exclusive egress modes are discoverable side by side without
+    emitting a duplicate `#### USING_TOR` anchor.
+    """
+    egress_vars = [
+        {
+            "name": "USING_WIREGUARD",
+            "description": "Route all traffic through a WireGuard VPN tunnel with a fail-closed iptables kill-switch (non-tunnel egress is dropped). Requires root startup and NET_ADMIN (plus NET_RAW). Mutually exclusive with USING_TOR.",
+            "type": "boolean",
+            "default": "false",
+        },
+        {
+            "name": "WIREGUARD_CONFIG",
+            "description": "Path to the mounted wg-quick configuration file.",
+            "type": "string (path)",
+            "default": "/config/wg0.conf",
+        },
+        {
+            "name": "WIREGUARD_INTERFACE",
+            "description": "WireGuard interface name brought up by wg-quick.",
+            "type": "string",
+            "default": "wg0",
+        },
+        {
+            "name": "LAN_NETWORK",
+            "description": "Comma-separated CIDRs kept off the tunnel so the WebUI and internal download clients (Prowlarr, qBittorrent) stay reachable.",
+            "type": "string (comma-separated)",
+            "default": "127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16",
+        },
+        {
+            "name": "WIREGUARD_ENFORCE_DNS",
+            "description": "Pin the container's resolver so DNS cannot silently fall back to an off-tunnel path. The resolver used is WIREGUARD_DNS if set, else the tunnel config's DNS = line. This does NOT force queries through the tunnel: it is designed for a trusted LAN resolver kept reachable off-tunnel via LAN_NETWORK (the query leaves over the LAN; the resolver encrypts upstream while the download still egresses via the tunnel). Special case: when Docker's embedded resolver (nameserver 127.0.0.11) is present, it is PRESERVED so container-name resolution (e.g. prowlarr, qbittorrent) keeps working, and the embedded resolver's upstream must be pinned via the container's compose dns: list. Fails closed (refuses to start) only when no embedded resolver is present AND no resolver is defined, or /etc/resolv.conf is not writable.",
+            "type": "boolean",
+            "default": "true",
+        },
+        {
+            "name": "WIREGUARD_DNS",
+            "description": "Explicit resolver(s) (comma/space separated) to pin when WIREGUARD_ENFORCE_DNS is true and Docker's embedded resolver is NOT in use. Use when the VPN's pushed DNS filters domains you need; point it at a resolver reachable via the tunnel or an allowed LAN resolver. NOTE: when the embedded resolver (127.0.0.11) is present it is preserved and this value cannot repoint its upstream from inside the container — set the container's compose dns: list to the trusted resolver instead.",
+            "type": "string (comma-separated)",
+            "default": "unset (uses config DNS = line)",
+        },
+        {
+            "name": "WIREGUARD_DISABLE_IPV6",
+            "description": "Strip IPv6 Address/AllowedIPs/DNS from the tunnel config before wg-quick (many container kernels lack the ip6tables raw table wg-quick needs) and remove IPv6 as a leak surface.",
+            "type": "boolean",
+            "default": "true",
+        },
+        {
+            "name": "WIREGUARD_ALLOW_IPV6_LEAK",
+            "description": "Escape hatch: continue startup even when an IPv6 kill-switch cannot be installed AND IPv6 cannot be disabled. Only set when the container has no IPv6 connectivity, as IPv6 egress may otherwise bypass the tunnel.",
+            "type": "boolean",
+            "default": "false",
+        },
+        {
+            "name": "WIREGUARD_ALLOW_WEBUI_OFFTUNNEL",
+            "description": "When false (default) the kill-switch is strictly fail-closed: the only off-tunnel egress permitted is loopback, the tunnel device and the LAN allowlist. Set true only if a NON-LAN client (e.g. a public reverse proxy on a different segment) must reach the WebUI; it permits app-server REPLY packets (--sport FLASK_PORT, conntrack REPLY) to leave off-tunnel. Server replies only, never client-initiated egress, so it cannot leak outbound browsing/downloads or the real IP for outbound requests, but it is still an off-tunnel path while the tunnel is down, hence opt-in. LAN WebUI clients never need it (covered by LAN_NETWORK).",
+            "type": "boolean",
+            "default": "false",
+        },
+        {
+            "name": "WIREGUARD_STALE_AFTER",
+            "description": "Seconds since the last WireGuard handshake before the healthcheck bounces the tunnel.",
+            "type": "number",
+            "default": "180",
+        },
+    ]
+
+    lines = [
+        "## Egress / VPN Routing",
+        "",
+        "These startup-only variables are consumed by `entrypoint.sh` / `wireguard.sh` to select and configure the WireGuard transparent-egress kill-switch. `USING_WIREGUARD` and [`USING_TOR`](#using_tor) (documented under Network) are mutually exclusive; both require root startup.",
+        "",
+        "| Variable | Description | Type | Default |",
+        "|----------|-------------|------|---------|",
+    ]
+
+    lines.extend(
+        f"| `{var['name']}` | {var['description']} | {var['type']} | `{var['default']}` |"
+        for var in egress_vars
+    )
+
+    lines.append("")
+    lines.append("<details>")
+    lines.append("<summary>Detailed descriptions</summary>")
+    lines.append("")
+
+    for var in egress_vars:
+        lines.append(f"#### `{var['name']}`")
+        lines.append("")
+        lines.append(var["description"])
+        lines.append("")
+        lines.append(f"- **Type:** {var['type']}")
+        lines.append(f"- **Default:** `{var['default']}`")
+        lines.append("")
+
+    lines.append("</details>")
+    lines.append("")
+
+    return lines
+
+
 def generate_env_docs() -> str:
     """Generate markdown documentation for all environment variables."""
     # Import settings modules to ensure all settings are registered
-    import shelfmark.config.settings  # noqa: F401
-    import shelfmark.release_sources.irc.settings  # noqa: F401
+    import shelfmark.config.security
+    import shelfmark.config.settings
+    import shelfmark.metadata_providers.googlebooks
+    import shelfmark.metadata_providers.hardcover
+    import shelfmark.metadata_providers.openlibrary
+    import shelfmark.release_sources.irc.settings
     import shelfmark.release_sources.prowlarr.settings  # noqa: F401
-    import shelfmark.metadata_providers.hardcover  # noqa: F401
-    import shelfmark.metadata_providers.openlibrary  # noqa: F401
-    import shelfmark.metadata_providers.googlebooks  # noqa: F401
-
     from shelfmark.core.settings_registry import (
-        ActionButton,
-        HeadingField,
         get_all_groups,
         get_all_settings_tabs,
     )
@@ -239,7 +370,7 @@ def generate_env_docs() -> str:
     groups = {g.name: g for g in get_all_groups()}
 
     # Organize tabs by group
-    grouped_tabs: Dict[Optional[str], List] = {None: []}
+    grouped_tabs: dict[str | None, list] = {None: []}
     for group_name in groups:
         grouped_tabs[group_name] = []
 
@@ -264,6 +395,7 @@ def generate_env_docs() -> str:
     # Generate TOC
     toc_entries = [
         "- [Bootstrap Configuration](#bootstrap-configuration)",
+        "- [Egress / VPN Routing](#egress--vpn-routing)",
     ]
 
     # Ungrouped tabs first
@@ -289,6 +421,9 @@ def generate_env_docs() -> str:
     # Add bootstrap environment variables documentation
     lines.extend(_generate_bootstrap_env_docs())
 
+    # Add egress / VPN routing (startup-only, shell-driven) documentation
+    lines.extend(_generate_egress_env_docs())
+
     # Generate documentation for ungrouped tabs
     for tab in grouped_tabs.get(None, []):
         lines.extend(_generate_tab_docs(tab))
@@ -308,33 +443,24 @@ def generate_env_docs() -> str:
     return "\n".join(lines)
 
 
-def _generate_tab_docs(tab, group_prefix: Optional[str] = None) -> List[str]:
+def _generate_tab_docs(tab: Any, group_prefix: str | None = None) -> list[str]:
     """Generate documentation for a single settings tab."""
-    from shelfmark.core.settings_registry import ActionButton, HeadingField
+    from shelfmark.core.settings_registry import iter_value_fields
 
     lines = []
 
     # Section header
     if group_prefix:
         lines.append(f"### {group_prefix}: {tab.display_name}")
-        anchor_id = f"{group_prefix}-{tab.display_name}".lower().replace(" ", "-")
     else:
         lines.append(f"## {tab.display_name}")
 
     lines.append("")
 
     # Collect env-supported fields
-    env_fields = []
-    for field in tab.fields:
-        # Skip non-value fields
-        if isinstance(field, (ActionButton, HeadingField)):
-            continue
-
-        # Skip fields that don't support ENV vars
-        if not getattr(field, "env_supported", True):
-            continue
-
-        env_fields.append(field)
+    env_fields = [
+        field for field in iter_value_fields(tab) if getattr(field, "env_supported", True)
+    ]
 
     if not env_fields:
         lines.append("_No environment variables for this section._")
@@ -390,6 +516,7 @@ def _generate_tab_docs(tab, group_prefix: Optional[str] = None) -> List[str]:
 
         # Show constraints for NumberField
         from shelfmark.core.settings_registry import NumberField
+
         if isinstance(field, NumberField):
             constraints = []
             if field.min_value is not None:
@@ -407,7 +534,7 @@ def _generate_tab_docs(tab, group_prefix: Optional[str] = None) -> List[str]:
     return lines
 
 
-def main():
+def main() -> None:
     parser = argparse.ArgumentParser(
         description="Generate markdown documentation for environment variables"
     )

scripts/test_clients.py

@@ -9,7 +9,7 @@ Usage:
     2. Wait for containers to initialize (first run takes ~30s)
 
     3. Run this script to verify clients are accessible:
-       python scripts/test_clients.py
+       uv run python scripts/test_clients.py
 
     4. Access cwabd at http://localhost:8084
        - Go to Settings > Prowlarr > Download Clients
@@ -26,7 +26,7 @@ Web UIs:
     - rTorrent:     http://localhost:8000 (web ui http://localhost:8089 via ruTorrent)
 
 Prerequisites (for running this script locally):
-    pip install requests transmission-rpc qbittorrent-api
+    uv sync --locked
 
 First-Time Setup:
     qBittorrent:
@@ -51,7 +51,8 @@ First-Time Setup:
 
 import sys
 import time
-from xmlrpc import client
+from pathlib import Path
+from typing import Any
 
 # Test configuration - matches docker-compose.test-clients.yml
 CONFIG = {
@@ -89,7 +90,7 @@ CONFIG = {
 TEST_MAGNET = "magnet:?xt=urn:btih:3b245504cf5f11bbdbe1201cea6a6bf45aee1bc0&dn=ubuntu-22.04.3-live-server-amd64.iso"
 
 
-def test_nzbget():
+def test_nzbget() -> bool:
     """Test NZBGet connection."""
     import requests
 
@@ -138,7 +139,7 @@ def test_nzbget():
         return False
 
 
-def test_sabnzbd():
+def test_sabnzbd() -> bool:
     """Test SABnzbd connection."""
     import requests
 
@@ -152,10 +153,9 @@ def test_sabnzbd():
     # Try to get API key from config if not set
     if not api_key:
         try:
-            import os
-            ini_path = ".local/test-clients/sabnzbd/config/sabnzbd.ini"
-            if os.path.exists(ini_path):
-                with open(ini_path) as f:
+            ini_path = Path(".local/test-clients/sabnzbd/config/sabnzbd.ini")
+            if ini_path.exists():
+                with ini_path.open() as f:
                     for line in f:
                         if line.startswith("api_key"):
                             api_key = line.split("=")[1].strip()
@@ -204,7 +204,7 @@ def test_sabnzbd():
         return False
 
 
-def test_qbittorrent():
+def test_qbittorrent() -> bool:
     """Test qBittorrent connection."""
     print("\n" + "=" * 50)
     print("Testing qBittorrent")
@@ -219,6 +219,7 @@ def test_qbittorrent():
 
         # Parse URL for host/port
         from urllib.parse import urlparse
+
         parsed = urlparse(url)
 
         client = qbittorrentapi.Client(
@@ -260,7 +261,7 @@ def test_qbittorrent():
 
     except ImportError:
         print("  ERROR: qbittorrent-api not installed")
-        print("  Run: pip install qbittorrent-api")
+        print("  Run: uv sync --locked")
         return False
     except Exception as e:
         print(f"  ERROR: {e}")
@@ -271,16 +272,17 @@ def test_qbittorrent():
         return False
 
 
-def test_transmission():
+def test_transmission() -> bool:
     """Test Transmission connection."""
     print("\n" + "=" * 50)
     print("Testing Transmission")
     print("=" * 50)
 
     try:
-        from transmission_rpc import Client
         from urllib.parse import urlparse
 
+        from transmission_rpc import Client
+
         url = CONFIG["transmission"]["url"]
         parsed = urlparse(url)
 
@@ -317,14 +319,14 @@ def test_transmission():
 
     except ImportError:
         print("  ERROR: transmission-rpc not installed")
-        print("  Run: pip install transmission-rpc")
+        print("  Run: uv sync --locked")
         return False
     except Exception as e:
         print(f"  ERROR: {e}")
         return False
 
 
-def test_deluge():
+def test_deluge() -> bool:
     """Test Deluge Web UI (JSON-RPC) connection."""
     import requests
 
@@ -336,7 +338,7 @@ def test_deluge():
     password = CONFIG["deluge"]["password"]
     rpc_url = f"{base_url}/json"
 
-    def rpc_call(session: requests.Session, rpc_id: int, method: str, *params):
+    def rpc_call(session: requests.Session, rpc_id: int, method: str, *params: Any) -> Any:
         payload = {"id": rpc_id, "method": method, "params": list(params)}
         resp = session.post(rpc_url, json=payload, timeout=10)
         resp.raise_for_status()
@@ -366,7 +368,11 @@ def test_deluge():
 
             host_id = hosts[0][0]
             for entry in hosts:
-                if isinstance(entry, list) and len(entry) >= 2 and entry[1] in {"127.0.0.1", "localhost"}:
+                if (
+                    isinstance(entry, list)
+                    and len(entry) >= 2
+                    and entry[1] in {"127.0.0.1", "localhost"}
+                ):
                     host_id = entry[0]
                     break
 
@@ -386,13 +392,18 @@ def test_deluge():
 
         # Test adding a torrent (then remove it)
         print("  Testing add/remove torrent...")
-        torrent_id = rpc_call(session, 8, "core.add_torrent_magnet", TEST_MAGNET, {"add_paused": True})
+        torrent_id = rpc_call(
+            session, 8, "core.add_torrent_magnet", TEST_MAGNET, {"add_paused": True}
+        )
 
         if torrent_id:
             torrent_id = str(torrent_id)
             print(f"  Added test torrent: {torrent_id[:20]}...")
 
-            status = rpc_call(session, 9, "core.get_torrent_status", torrent_id, ["state", "progress"]) or {}
+            status = (
+                rpc_call(session, 9, "core.get_torrent_status", torrent_id, ["state", "progress"])
+                or {}
+            )
             state = status.get("state", "unknown") if isinstance(status, dict) else "unknown"
             progress = status.get("progress", 0) if isinstance(status, dict) else 0
             print(f"  Status: {state} ({progress:.1f}%)")
@@ -418,7 +429,8 @@ def test_deluge():
             print("  Check Deluge Web UI password (default: deluge)")
         return False
 
-def test_rtorrent():
+
+def test_rtorrent() -> bool:
     """Test rTorrent connection."""
     print("\n" + "=" * 50)
     print("Testing rTorrent")
@@ -457,19 +469,18 @@ def test_rtorrent():
 
         # rtorrent is weird in that it doesn't return the torrent ID/hash on add
         client.load.start("", TEST_MAGNET, ";".join(commands))
-        
+
         # but we know that it is 3b245504cf5f11bbdbe1201cea6a6bf45aee1bc0 from the magnet link
-        torrent_id = "3B245504CF5F11BBDBE1201CEA6A6BF45AEE1BC0" # rtorrent uses uppercase hashes
+        torrent_id = "3B245504CF5F11BBDBE1201CEA6A6BF45AEE1BC0"  # rtorrent uses uppercase hashes
         print(f"  Added test torrent: {torrent_id}")
 
         torrents = client.download_list()
-        print(f"  Active torrents: {len(torrents)}")        
+        print(f"  Active torrents: {len(torrents)}")
 
         torrent_list = client.d.multicall.filtered(
             "",
             "default",
-            f"equal={{d.hash=,cat={torrent_id}}}"
-            "d.hash=",
+            f"equal={{d.hash=,cat={torrent_id}}}d.hash=",
             "d.state=",
             "d.completed_bytes=",
             "d.size_bytes=",
@@ -483,7 +494,7 @@ def test_rtorrent():
         if not torrent:
             print("  ERROR: Could not find added torrent in list")
             return False
-        
+
         # let's test the base path call
         details = client.d.multicall.filtered(
             "",
@@ -511,7 +522,7 @@ def test_rtorrent():
         return False
 
 
-def main():
+def main() -> int:
     print("Download Client Test Suite")
     print("=" * 50)
     print("Make sure containers are running:")

shelfmark/__main__.py

@@ -1,8 +1,23 @@
 """Package entry point for `python -m shelfmark`."""
 
-from shelfmark.main import app, socketio
 from shelfmark.config.env import FLASK_HOST, FLASK_PORT
 from shelfmark.core.config import config
+from shelfmark.main import app, socketio
+
+
+def _resolve_debug_flag(value: object) -> bool:
+    """Normalize DEBUG config values for Flask-SocketIO startup."""
+    if isinstance(value, bool):
+        return value
+    if isinstance(value, str):
+        return value.strip().lower() in {"1", "true", "yes", "on"}
+    return bool(value)
+
 
 if __name__ == "__main__":
-    socketio.run(app, host=FLASK_HOST, port=FLASK_PORT, debug=config.get("DEBUG", False))
+    socketio.run(
+        app,
+        host=FLASK_HOST,
+        port=FLASK_PORT,
+        debug=_resolve_debug_flag(config.get("DEBUG", False)),
+    )

shelfmark/api/websocket.py

@@ -1,124 +1,81 @@
 """WebSocket manager for real-time status updates."""
 
+from __future__ import annotations
+
 import logging
 import threading
-from typing import Optional, Dict, Any, Callable, List
+from typing import TYPE_CHECKING, Any
 
 from flask_socketio import SocketIO, join_room, leave_room
 
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
+    from flask import Flask
+
 logger = logging.getLogger(__name__)
 
 
 class WebSocketManager:
     """Manages WebSocket connections and broadcasts."""
 
-    def __init__(self):
-        self.socketio: Optional[SocketIO] = None
+    def __init__(self) -> None:
+        """Initialize in-memory connection and room tracking."""
+        self.socketio: SocketIO | None = None
         self._enabled = False
         self._connection_count = 0
         self._connection_lock = threading.Lock()
-        self._on_first_connect_callbacks: List[Callable[[], None]] = []
-        self._on_all_disconnect_callbacks: List[Callable[[], None]] = []
-        self._needs_rewarm = False  # Flag to trigger warmup callbacks on next connect
-        self._user_rooms: Dict[str, int] = {}  # room_name -> ref count
-        self._sid_rooms: Dict[str, str] = {}  # sid -> room_name
+        self._user_rooms: dict[str, int] = {}  # room_name -> ref count
+        self._sid_rooms: dict[str, str] = {}  # sid -> room_name
         self._rooms_lock = threading.Lock()
-        self._queue_status_fn: Optional[Callable] = None  # Reference to queue_status()
+        self._queue_status_fn: Callable | None = None  # Reference to queue_status()
 
-    def init_app(self, app, socketio: SocketIO):
+    def init_app(self, app: Flask, socketio: SocketIO) -> None:
         """Initialize the WebSocket manager with Flask-SocketIO instance."""
         self.socketio = socketio
         self._enabled = True
         logger.info("WebSocket manager initialized")
 
-    def register_on_first_connect(self, callback: Callable[[], None]):
-        """Register a callback for when the first client connects."""
-        self._on_first_connect_callbacks.append(callback)
-        logger.debug(f"Registered on_first_connect callback: {callback.__name__}")
-
-    def register_on_all_disconnect(self, callback: Callable[[], None]):
-        """Register a callback for when all clients disconnect."""
-        self._on_all_disconnect_callbacks.append(callback)
-        logger.debug(f"Registered on_all_disconnect callback: {callback.__name__}")
-
-    def request_warmup_on_next_connect(self):
-        """Request warmup callbacks on the next client connect (e.g., after idle shutdown)."""
-        with self._connection_lock:
-            self._needs_rewarm = True
-            logger.debug("Warmup requested for next client connect")
-
-    def client_connected(self):
+    def client_connected(self) -> None:
         """Track a new client connection. Call this from the connect event handler."""
         with self._connection_lock:
-            was_zero = self._connection_count == 0
-            needs_rewarm = self._needs_rewarm
             self._connection_count += 1
             current_count = self._connection_count
-            # Clear rewarm flag if we're going to trigger warmup
-            if was_zero or needs_rewarm:
-                self._needs_rewarm = False
 
-        logger.debug(f"Client connected. Active connections: {current_count}")
+        logger.debug("Client connected. Active connections: %s", current_count)
 
-        # Trigger warmup callbacks if this is the first connection OR if rewarm was requested
-        # (rewarm is requested when bypasser shuts down due to idle while clients are connected)
-        if was_zero or needs_rewarm:
-            reason = "First client connected" if was_zero else "Rewarm requested after idle shutdown"
-            logger.info(f"{reason}, triggering warmup callbacks...")
-            for callback in self._on_first_connect_callbacks:
-                try:
-                    # Run callbacks in a separate thread to not block the connection
-                    thread = threading.Thread(target=callback, daemon=True)
-                    thread.start()
-                except Exception as e:
-                    logger.error(f"Error in on_first_connect callback {callback.__name__}: {e}")
-
-    def client_disconnected(self):
+    def client_disconnected(self) -> None:
         """Track a client disconnection. Call this from the disconnect event handler."""
         with self._connection_lock:
             self._connection_count = max(0, self._connection_count - 1)
             current_count = self._connection_count
-            is_now_zero = current_count == 0
 
-        logger.debug(f"Client disconnected. Active connections: {current_count}")
-
-        # If all clients have disconnected, trigger cleanup callbacks
-        if is_now_zero:
-            logger.info("All clients disconnected, triggering disconnect callbacks...")
-            for callback in self._on_all_disconnect_callbacks:
-                try:
-                    callback()
-                except Exception as e:
-                    logger.error(f"Error in on_all_disconnect callback {callback.__name__}: {e}")
-
-    def get_connection_count(self) -> int:
-        """Get the current number of active WebSocket connections."""
-        with self._connection_lock:
-            return self._connection_count
-
-    def has_active_connections(self) -> bool:
-        """Check if there are any active WebSocket connections."""
-        return self.get_connection_count() > 0
+        logger.debug("Client disconnected. Active connections: %s", current_count)
 
     def is_enabled(self) -> bool:
         """Check if WebSocket is enabled and ready."""
         return self._enabled and self.socketio is not None
 
-    def set_queue_status_fn(self, fn: Callable):
+    def _get_socketio(self) -> SocketIO | None:
+        if not self._enabled:
+            return None
+        return self.socketio
+
+    def set_queue_status_fn(self, fn: Callable) -> None:
         """Set the queue_status function reference for per-room filtering."""
         self._queue_status_fn = fn
 
-    def _increment_user_room_locked(self, room: str):
+    def _increment_user_room_locked(self, room: str) -> None:
         self._user_rooms[room] = self._user_rooms.get(room, 0) + 1
 
-    def _decrement_user_room_locked(self, room: str):
+    def _decrement_user_room_locked(self, room: str) -> None:
         count = self._user_rooms.get(room, 1) - 1
         if count <= 0:
             self._user_rooms.pop(room, None)
         else:
             self._user_rooms[room] = count
 
-    def _set_sid_room_locked(self, sid: str, room: Optional[str]):
+    def _set_sid_room_locked(self, sid: str, room: str | None) -> None:
         current_room = self._sid_rooms.get(sid)
         if current_room == room:
             return
@@ -135,9 +92,14 @@ class WebSocketManager:
             if room.startswith("user_"):
                 self._increment_user_room_locked(room)
 
-    def sync_user_room(self, sid: str, is_admin: bool, db_user_id: Optional[int] = None):
+    def sync_user_room(
+        self,
+        sid: str,
+        is_admin: bool,
+        db_user_id: int | None = None,
+    ) -> None:
         """Ensure a SID is in exactly one room matching the current session scope."""
-        room: Optional[str] = None
+        room: str | None = None
         if is_admin:
             room = "admins"
         elif db_user_id is not None:
@@ -146,24 +108,36 @@ class WebSocketManager:
         with self._rooms_lock:
             self._set_sid_room_locked(sid, room)
 
-    def join_user_room(self, sid: str, is_admin: bool, db_user_id: Optional[int] = None):
+    def join_user_room(
+        self,
+        sid: str,
+        is_admin: bool,
+        db_user_id: int | None = None,
+    ) -> None:
         """Join the appropriate room based on user role."""
-        self.sync_user_room(sid, is_admin, db_user_id)
+        self.sync_user_room(sid, is_admin=is_admin, db_user_id=db_user_id)
 
-    def leave_user_room(self, sid: str, is_admin: bool = False, db_user_id: Optional[int] = None):
+    def leave_user_room(
+        self,
+        sid: str,
+        *,
+        is_admin: bool = False,
+        db_user_id: int | None = None,
+    ) -> None:
         """Leave whichever room the SID currently belongs to."""
         del is_admin, db_user_id  # Backward-compatible signature; routing is SID-based.
         with self._rooms_lock:
             self._set_sid_room_locked(sid, None)
 
-    def broadcast_status_update(self, status_data: Dict[str, Any]):
+    def broadcast_status_update(self, status_data: dict[str, Any]) -> None:
         """Broadcast status update to all connected clients, filtered by user room."""
-        if not self.is_enabled():
+        socketio = self._get_socketio()
+        if socketio is None:
             return
 
         try:
             # Admins (and no-auth users) get full status
-            self.socketio.emit('status_update', status_data, to="admins")
+            socketio.emit("status_update", status_data, to="admins")
 
             # Each user room gets filtered status
             with self._rooms_lock:
@@ -171,56 +145,48 @@ class WebSocketManager:
 
             if active_rooms and self._queue_status_fn:
                 for room in active_rooms:
-                    try:
-                        # Extract user_id from room name "user_123"
-                        uid = int(room.split("_", 1)[1])
-                        filtered = self._queue_status_fn(user_id=uid)
-                        self.socketio.emit('status_update', filtered, to=room)
-                    except Exception as e:
-                        logger.error(f"Failed to send status update for room {room}: {e}")
+                    self._broadcast_status_update_to_room(room)
 
             logger.debug("Broadcasted status update to all rooms")
-        except Exception as e:
-            logger.error(f"Error broadcasting status update: {e}")
+        except Exception:
+            logger.exception("Error broadcasting status update")
 
-    def broadcast_download_progress(self, book_id: str, progress: float, status: str, user_id: Optional[int] = None):
-        """Broadcast download progress update for a specific book."""
-        if not self.is_enabled():
+    def _broadcast_status_update_to_room(self, room: str) -> None:
+        """Broadcast status update to one user room."""
+        socketio = self._get_socketio()
+        if socketio is None:
             return
 
         try:
-            data = {
-                'book_id': book_id,
-                'progress': progress,
-                'status': status
-            }
+            # Extract user_id from room name "user_123"
+            uid = int(room.split("_", 1)[1])
+            filtered = self._queue_status_fn(user_id=uid) if self._queue_status_fn else None
+            if filtered is not None:
+                socketio.emit("status_update", filtered, to=room)
+        except Exception:
+            logger.exception("Failed to send status update for room %s", room)
+
+    def broadcast_download_progress(
+        self, book_id: str, progress: float, status: str, user_id: int | None = None
+    ) -> None:
+        """Broadcast download progress update for a specific book."""
+        socketio = self._get_socketio()
+        if socketio is None:
+            return
+
+        try:
+            data = {"book_id": book_id, "progress": progress, "status": status}
             # Admins always see all progress
-            self.socketio.emit('download_progress', data, to="admins")
+            socketio.emit("download_progress", data, to="admins")
             # If task belongs to a specific user, send to their room too
             if user_id is not None:
                 room = f"user_{user_id}"
                 with self._rooms_lock:
                     if room in self._user_rooms:
-                        self.socketio.emit('download_progress', data, to=room)
-            logger.debug(f"Broadcasted progress for book {book_id}: {progress}%")
-        except Exception as e:
-            logger.error(f"Error broadcasting download progress: {e}")
-
-    def broadcast_notification(self, message: str, notification_type: str = 'info'):
-        """Broadcast a notification message to all clients."""
-        if not self.is_enabled():
-            return
-
-        try:
-            data = {
-                'message': message,
-                'type': notification_type
-            }
-            # When calling socketio.emit() outside event handlers, it broadcasts by default
-            self.socketio.emit('notification', data)
-            logger.debug(f"Broadcasted notification: {message}")
-        except Exception as e:
-            logger.error(f"Error broadcasting notification: {e}")
+                        socketio.emit("download_progress", data, to=room)
+            logger.debug("Broadcasted progress for book %s: %s%%", book_id, progress)
+        except Exception:
+            logger.exception("Error broadcasting download progress")
 
     def broadcast_search_status(
         self,
@@ -228,23 +194,24 @@ class WebSocketManager:
         provider: str,
         book_id: str,
         message: str,
-        phase: str = 'searching'
-    ):
+        phase: str = "searching",
+    ) -> None:
         """Broadcast search status update for a release source search."""
-        if not self.is_enabled():
+        socketio = self._get_socketio()
+        if socketio is None:
             return
 
         try:
             data = {
-                'source': source,
-                'provider': provider,
-                'book_id': book_id,
-                'message': message,
-                'phase': phase,
+                "source": source,
+                "provider": provider,
+                "book_id": book_id,
+                "message": message,
+                "phase": phase,
             }
-            self.socketio.emit('search_status', data)
-        except Exception as e:
-            logger.error(f"Error broadcasting search status: {e}")
+            socketio.emit("search_status", data)
+        except Exception:
+            logger.exception("Error broadcasting search status")
 
 
 # Global WebSocket manager instance

shelfmark/bypass/__init__.py

@@ -1,5 +1,16 @@
 """Cloudflare bypass utilities."""
 
 
-class BypassCancelledException(Exception):
+class BypassCancelledError(Exception):
     """Raised when a bypass operation is cancelled."""
+
+
+class ChallengeNotSolvedError(Exception):
+    """Raised when a bypasser ran but the site still answered with a challenge.
+
+    Distinct from a bypasser that is broken or unreachable, which is what every
+    "the bypass failed" message used to say. A solver can do its job perfectly and
+    still be handed something it cannot clear - DDoS-Guard's manual CAPTCHA page is
+    the case from #1292 - and telling the user to go check that FlareSolverr is
+    reachable sends them to fix a service that is working.
+    """

shelfmark/bypass/challenge.py

@@ -0,0 +1,52 @@
+"""Challenge-page detection shared by the bypassers and the HTTP retry path.
+
+Kept out of `internal_bypasser` so the HTTP layer can recognise an interstitial
+without importing SeleniumBase: that module is imported lazily precisely because its
+browser dependencies are optional, and external-bypasser setups run without them.
+"""
+
+# Matched against lowercased text, so every entry must be lowercase.
+CLOUDFLARE_INDICATORS = [
+    "just a moment",
+    "verify you are human",
+    "verifying you are human",
+    "cloudflare.com/products/turnstile",
+]
+
+DDOS_GUARD_INDICATORS = [
+    "ddos-guard",
+    "ddos guard",
+    "checking your browser before accessing",
+    "complete the manual check to continue",
+    "could not verify your browser automatically",
+]
+
+# Markers that exist only in raw markup: the bypassers scan rendered innerText, where
+# a script src or a <title> never appears. The title match is scoped to the tag on
+# purpose - hosts word the rest of that sentence differently, and matching "checking
+# your browser" as free text would trip on any page that merely discusses a challenge.
+_RAW_HTML_MARKERS = (
+    "<title>checking your browser",
+    "/cdn-cgi/challenge-platform",
+    "/.well-known/ddos-guard/",
+)
+
+# An interstitial is a few KB of markup. Past that it is a real page that happens to
+# mention a marker - a protected site links its own DDoS-Guard endpoints on every page.
+MAX_CHALLENGE_HTML_CHARS = 64 * 1024
+
+
+def challenge_marker(html: str) -> str | None:
+    """Return the marker proving `html` is an unsolved challenge page, or None.
+
+    Only meaningful for a response that already carries a challenge status: the
+    markers appear on protected sites' real pages too, so the status is what
+    separates "blocked" from "served".
+    """
+    if not html or len(html) > MAX_CHALLENGE_HTML_CHARS:
+        return None
+    lowered = html.lower()
+    for marker in (*_RAW_HTML_MARKERS, *DDOS_GUARD_INDICATORS, *CLOUDFLARE_INDICATORS):
+        if marker in lowered:
+            return marker
+    return None

shelfmark/bypass/cookie_store.py

@@ -0,0 +1,286 @@
+"""Clearance cookies won by a bypass, shared by every bypasser implementation.
+
+Kept in its own module rather than inside a bypasser because both of them feed it and
+both read from it. The internal bypasser cannot host it: it imports seleniumbase at
+module scope, which is exactly the dependency an external-bypasser deployment is
+entitled not to have installed.
+"""
+
+import threading
+import time
+from collections.abc import Mapping
+from typing import Any
+from urllib.parse import urlparse
+
+from shelfmark.core.logger import setup_logger
+
+logger = setup_logger(__name__)
+
+# Cookie storage - shared with requests library for Cloudflare bypass
+# Nested mapping of domain to cookie name to cookie metadata.
+_cf_cookies: dict[str, dict] = {}
+_cf_cookies_lock = threading.Lock()
+
+# User-Agent storage - Cloudflare ties cf_clearance to the UA that solved the challenge
+_cf_user_agents: dict[str, str] = {}
+
+# Protection cookie names we care about (Cloudflare and DDoS-Guard)
+CF_COOKIE_NAMES = {"cf_clearance", "__cf_bm", "cf_chl_2", "cf_chl_prog"}
+DDG_COOKIE_NAMES = {
+    "__ddg1_",
+    "__ddg2_",
+    "__ddg5_",
+    "__ddg8_",
+    "__ddg9_",
+    "__ddg10_",
+    "__ddgid_",
+    "__ddgmark_",
+    "ddg_last_challenge",
+}
+
+# Anna's Archive's own pass for its ?check=1 hop. Without it the hop 302s back
+# forever, however good the __ddg* clearance is.
+AA_COOKIE_NAMES = {"aa_ddg_check"}
+
+# DDoS-Guard cookies that describe *one* check rather than granting clearance, and so
+# must never be replayed on a later request. Observed live on Anna's Archive:
+#
+#   __ddg9_   the client IP address
+#   __ddg10_  the unix timestamp the check was issued
+#   __ddg8_   an opaque token issued with them, same ~40 minute expiry
+#
+# Clearance itself lives in __ddg1_/__ddg2_/__ddgid_ (roughly a year) and __ddg5_.
+# Replaying the trio is actively harmful: once the timestamp ages out - or the egress
+# IP changes, which happens routinely behind a VPN - the values no longer describe the
+# caller, DDoS-Guard re-arms its check and answers every request with a ?check=1
+# redirect. That is the redirect loop, and it is self-inflicted. Dropping them simply
+# lets DDoS-Guard issue a fresh set, exactly as it does for a browser.
+DDG_EPHEMERAL_COOKIE_NAMES = {
+    "__ddg8_",
+    "__ddg9_",
+    "__ddg10_",
+    "ddg_last_challenge",
+}
+
+
+def _get_base_domain(domain: str) -> str:
+    """Extract base domain from hostname (e.g., 'www.example.com' -> 'example.com')."""
+    return ".".join(domain.split(".")[-2:]) if "." in domain else domain
+
+
+def _get_full_cookie_domains() -> set[str]:
+    """Return mirror domains that need full-session cookie extraction."""
+    from shelfmark.core.mirrors import get_zlib_cookie_domains
+
+    return {_get_base_domain(domain) for domain in get_zlib_cookie_domains()}
+
+
+def _replay_per_check_cookies() -> bool:
+    """Whether the per-check trio is kept rather than dropped (see env.py)."""
+    from shelfmark.config import env
+
+    return env.DDG_REPLAY_PER_CHECK_COOKIES
+
+
+def _should_extract_cookie(name: str, *, extract_all: bool) -> bool:
+    """Determine if a cookie should be extracted based on its name."""
+    # Checked before extract_all: a per-check token is wrong to replay for every
+    # domain, including the full-session ones.
+    if name in DDG_EPHEMERAL_COOKIE_NAMES and not _replay_per_check_cookies():
+        return False
+    if extract_all:
+        return True
+    is_cf = name in CF_COOKIE_NAMES or name.startswith("cf_")
+    is_ddg = name in DDG_COOKIE_NAMES or name.startswith("__ddg")
+    is_aa = name in AA_COOKIE_NAMES
+    return is_cf or is_ddg or is_aa
+
+
+def _cookie_field(cookie: Any, name: str) -> Any:
+    """Read one field from a cookie in either shape we are handed.
+
+    The internal bypasser extracts CDP cookie objects; an external bypasser returns
+    the same fields as JSON objects, so the difference is attribute versus key access.
+    """
+    if isinstance(cookie, Mapping):
+        return cookie.get(name)
+    return getattr(cookie, name, None)
+
+
+def _cookie_expiry(cookie: Any) -> float | None:
+    """A cookie's absolute expiry, or None when it is a session cookie.
+
+    The two spellings are not interchangeable and both reach this store. CDP and
+    Playwright cookies carry `expires`; the WebDriver cookie object - what a
+    Selenium-based solver such as FlareSolverr returns - carries `expiry`. Reading
+    only one silently turns every cookie from the other into a never-expiring one,
+    which is exactly how dead clearance ends up replayed forever (see
+    get_cf_cookies_for_domain).
+
+    The value is coerced rather than trusted: it arrives as JSON from a service we
+    do not control, and a string here used to raise straight out of the store.
+    """
+    for field in ("expires", "expiry"):
+        raw = _cookie_field(cookie, field)
+        if raw is None:
+            continue
+        try:
+            expiry = float(raw)
+        except TypeError, ValueError:
+            logger.debug("Unreadable cookie expiry %r; treating as a session cookie", raw)
+            return None
+        # <= 0 is how both shapes spell "session cookie", not "expired in 1970".
+        return expiry if expiry > 0 else None
+    return None
+
+
+def store_extracted_cookies(
+    *,
+    url: str,
+    cookies: list[Any],
+    user_agent: str | None = None,
+) -> None:
+    """Store filtered bypass cookies (and optional UA) for a URL domain."""
+    parsed = urlparse(url)
+    domain = parsed.hostname or ""
+    if not domain:
+        return
+
+    base_domain = _get_base_domain(domain)
+    extract_all = base_domain in _get_full_cookie_domains()
+
+    cookies_found: dict[str, dict[str, Any]] = {}
+    dropped: list[str] = []
+    for cookie in cookies:
+        name = _cookie_field(cookie, "name") or ""
+        if not _should_extract_cookie(name, extract_all=extract_all):
+            dropped.append(name)
+            continue
+        secure = _cookie_field(cookie, "secure")
+        cookies_found[name] = {
+            "value": _cookie_field(cookie, "value") or "",
+            "domain": _cookie_field(cookie, "domain") or domain,
+            "path": _cookie_field(cookie, "path") or "/",
+            "expiry": _cookie_expiry(cookie),
+            "secure": True if secure is None else bool(secure),
+            "httpOnly": True,
+        }
+
+    # Names only, never values. Which cookies a solve won, and which of them were held
+    # back, is the evidence needed to settle what DDoS-Guard actually treats as clearance
+    # (issue #1276) - and without it a debug log shows a solve succeeding and the next
+    # request being challenged with nothing in between to explain why.
+    logger.debug(
+        "Solve on %s won %s; keeping %s; dropping %s",
+        base_domain,
+        sorted({_cookie_field(c, "name") or "" for c in cookies}),
+        sorted(cookies_found),
+        sorted(set(dropped)) or "nothing",
+    )
+
+    if not cookies_found:
+        return
+
+    with _cf_cookies_lock:
+        _cf_cookies[base_domain] = cookies_found
+        if user_agent:
+            _cf_user_agents[base_domain] = user_agent
+            logger.debug("Stored UA for %s: %s...", base_domain, str(user_agent)[:60])
+        else:
+            logger.debug("No UA captured for %s", base_domain)
+
+    cookie_type = "all" if extract_all else "protection"
+    logger.debug("Extracted %s %s cookies for %s", len(cookies_found), cookie_type, base_domain)
+
+
+def _is_cookie_expired(cookie: dict[str, Any]) -> bool:
+    """Whether a stored cookie's expiry has passed. Session cookies never expire here."""
+    expiry = cookie.get("expiry")
+    if expiry is None:
+        expiry = cookie.get("expires")
+    if not expiry or expiry <= 0:
+        return False
+    return time.time() > expiry
+
+
+def get_cf_cookies_for_domain(domain: str) -> dict[str, str]:
+    """Get stored cookies for a domain. Returns empty dict if none available."""
+    if not domain:
+        return {}
+
+    base_domain = _get_base_domain(domain)
+
+    with _cf_cookies_lock:
+        cookies = _cf_cookies.get(base_domain, {})
+        if not cookies:
+            return {}
+
+        cf_clearance = cookies.get("cf_clearance", {})
+        if cf_clearance and _is_cookie_expired(cf_clearance):
+            logger.debug("CF cookies expired for %s", base_domain)
+            _cf_cookies.pop(base_domain, None)
+            return {}
+
+        # Expiry applies to every cookie, not just Cloudflare's. DDoS-Guard domains
+        # have no cf_clearance, so the check above never fired for them and dead
+        # cookies were replayed indefinitely - the server answers those with a
+        # challenge, which is indistinguishable from having sent nothing at all.
+        live = {name: c for name, c in cookies.items() if not _is_cookie_expired(c)}
+        if len(live) != len(cookies):
+            expired = sorted(set(cookies) - set(live))
+            logger.debug("Dropping expired cookies for %s: %s", base_domain, expired)
+            if live:
+                _cf_cookies[base_domain] = live
+            else:
+                _cf_cookies.pop(base_domain, None)
+
+        return {name: c["value"] for name, c in live.items()}
+
+
+def has_valid_cf_cookies(domain: str) -> bool:
+    """Check if we have valid Cloudflare cookies for a domain."""
+    return bool(get_cf_cookies_for_domain(domain))
+
+
+def get_cf_user_agent_for_domain(domain: str) -> str | None:
+    """Get the User-Agent that was used during bypass for a domain."""
+    if not domain:
+        return None
+    with _cf_cookies_lock:
+        return _cf_user_agents.get(_get_base_domain(domain))
+
+
+def export_store() -> tuple[dict[str, dict], dict[str, str]]:
+    """Snapshot the whole store, for handing to another process.
+
+    The internal bypasser's Docker helper solves in a subprocess, so the clearance it
+    wins has to be serialized back to the parent or the solve is lost with the child.
+    """
+    with _cf_cookies_lock:
+        return (
+            {domain: dict(cookies) for domain, cookies in _cf_cookies.items()},
+            dict(_cf_user_agents),
+        )
+
+
+def import_store(cookies: object, user_agents: object) -> None:
+    """Merge a snapshot produced by :func:`export_store` into this process's store."""
+    with _cf_cookies_lock:
+        if isinstance(cookies, dict):
+            _cf_cookies.update(cookies)
+        if isinstance(user_agents, dict):
+            _cf_user_agents.update(
+                {str(domain): str(agent) for domain, agent in user_agents.items()}
+            )
+
+
+def clear_cf_cookies(domain: str | None = None) -> None:
+    """Clear stored Cloudflare cookies and User-Agent. If domain is None, clear all."""
+    with _cf_cookies_lock:
+        if domain:
+            base_domain = _get_base_domain(domain)
+            _cf_cookies.pop(base_domain, None)
+            _cf_user_agents.pop(base_domain, None)
+        else:
+            _cf_cookies.clear()
+            _cf_user_agents.clear()

shelfmark/bypass/external_bypasser.py

@@ -2,20 +2,26 @@
 
 import random
 import time
-from threading import Event
-from typing import TYPE_CHECKING, Optional
+from typing import TYPE_CHECKING, Any
 
 import requests
 
-from shelfmark.bypass import BypassCancelledException
+from shelfmark.bypass import BypassCancelledError, ChallengeNotSolvedError
+from shelfmark.bypass.challenge import challenge_marker
+from shelfmark.bypass.cookie_store import store_extracted_cookies
 from shelfmark.core.config import config
 from shelfmark.core.logger import setup_logger
 from shelfmark.core.utils import normalize_http_url
+from shelfmark.download.network import get_ssl_verify
 
 if TYPE_CHECKING:
+    from collections.abc import Mapping
+    from threading import Event
+
     from shelfmark.download import network
 
 logger = setup_logger(__name__)
+_RNG = random.SystemRandom()
 
 # Timeout constants (seconds)
 CONNECT_TIMEOUT = 10
@@ -28,15 +34,83 @@ BACKOFF_BASE = 1.0
 BACKOFF_CAP = 10.0
 
 
-def _fetch_via_bypasser(target_url: str) -> Optional[str]:
-    """Make a single request to the external bypasser service. Returns HTML or None."""
-    raw_bypasser_url = config.get("EXT_BYPASSER_URL", "http://flaresolverr:8191")
-    bypasser_path = config.get("EXT_BYPASSER_PATH", "/v1")
-    bypasser_timeout = config.get("EXT_BYPASSER_TIMEOUT", 60000)
+def _coerce_config_str(value: object, default: str) -> str:
+    """Return a string config value or a safe default."""
+    if isinstance(value, str):
+        return value
+    return default
+
+
+def _coerce_timeout_ms(value: object, default: int) -> int:
+    """Return a positive timeout in milliseconds or the default."""
+    if isinstance(value, bool):
+        return default
+    if isinstance(value, int) and value > 0:
+        return value
+    return default
+
+
+def max_duration_seconds() -> float:
+    """Upper bound on how long get_bypassed_page() can take for one URL.
+
+    MAX_RETRY attempts at the configured read timeout, plus the exponential backoff waited
+    between them (jitter is < 1s per gap, counted as a full second to stay conservative).
+    Callers use this to declare a stall-detection grace; see shelfmark.download.activity.
+    """
+    bypasser_timeout = _coerce_timeout_ms(config.get("EXT_BYPASSER_TIMEOUT", 60000), 60000)
+    read_timeout = min((bypasser_timeout / 1000) + READ_TIMEOUT_BUFFER, MAX_READ_TIMEOUT)
+    backoff_total = sum(
+        min(BACKOFF_CAP, BACKOFF_BASE * (2 ** (attempt - 1))) + 1.0
+        for attempt in range(1, MAX_RETRY)
+    )
+    return MAX_RETRY * read_timeout + backoff_total
+
+
+def _store_solution_clearance(target_url: str, solution: Mapping[str, Any]) -> None:
+    """Keep the clearance the solver won, so later requests do not re-solve.
+
+    A solve is the expensive part of an external bypass - tens of seconds of real
+    browser - and FlareSolverr-compatible services hand back the cookies and the
+    User-Agent that earned it. Dropping them meant every single request paid a 403
+    plus a full solve, and a file download (which the solver cannot proxy, being
+    binary) never presented clearance at all.
+
+    The UA matters as much as the cookies: Cloudflare ties cf_clearance to the UA
+    that solved the challenge, so replaying the cookie under our own UA is rejected.
+    """
+    cookies = solution.get("cookies") or []
+    if not isinstance(cookies, list):
+        logger.debug("External bypasser returned no usable cookie list for '%s'", target_url)
+        return
+
+    user_agent = solution.get("userAgent")
+    store_extracted_cookies(
+        url=target_url,
+        cookies=cookies,
+        user_agent=user_agent if isinstance(user_agent, str) else None,
+    )
+
+
+def _fetch_via_bypasser(target_url: str) -> str | None:
+    """Make a single request to the external bypasser service. Returns HTML or None.
+
+    Raises:
+        ChallengeNotSolvedError: the service answered with a page that is still a
+            challenge, whatever verdict it reported on itself.
+
+    """
+    raw_bypasser_url = _coerce_config_str(
+        config.get("EXT_BYPASSER_URL", "http://flaresolverr:8191"),
+        "http://flaresolverr:8191",
+    )
+    bypasser_path = _coerce_config_str(config.get("EXT_BYPASSER_PATH", "/v1"), "/v1")
+    bypasser_timeout = _coerce_timeout_ms(config.get("EXT_BYPASSER_TIMEOUT", 60000), 60000)
 
     bypasser_url = normalize_http_url(raw_bypasser_url)
     if not bypasser_url or not bypasser_path:
-        logger.error("External bypasser not configured. Check EXT_BYPASSER_URL and EXT_BYPASSER_PATH.")
+        logger.error(
+            "External bypasser not configured. Check EXT_BYPASSER_URL and EXT_BYPASSER_PATH."
+        )
         return None
 
     read_timeout = min((bypasser_timeout / 1000) + READ_TIMEOUT_BUFFER, MAX_READ_TIMEOUT)
@@ -45,47 +119,98 @@ def _fetch_via_bypasser(target_url: str) -> Optional[str]:
         response = requests.post(
             f"{bypasser_url}{bypasser_path}",
             headers={"Content-Type": "application/json"},
-            json={"cmd": "request.get", "url": target_url, "maxTimeout": bypasser_timeout},
-            timeout=(CONNECT_TIMEOUT, read_timeout)
+            json={
+                "cmd": "request.get",
+                "url": target_url,
+                "maxTimeout": bypasser_timeout,
+            },
+            timeout=(CONNECT_TIMEOUT, read_timeout),
+            verify=get_ssl_verify(bypasser_url),
         )
         response.raise_for_status()
         result = response.json()
 
-        status = result.get('status', 'unknown')
-        message = result.get('message', '')
-        logger.debug(f"External bypasser response for '{target_url}': {status} - {message}")
+        status = result.get("status", "unknown")
+        message = result.get("message", "")
+        logger.debug("External bypasser response for '%s': %s - %s", target_url, status, message)
 
-        if status != 'ok':
-            logger.warning(f"External bypasser failed for '{target_url}': {status} - {message}")
+        if status != "ok":
+            logger.warning(
+                "External bypasser failed for '%s': %s - %s",
+                target_url,
+                status,
+                message,
+            )
             return None
 
-        solution = result.get('solution')
-        html = solution.get('response', '') if solution else ''
+        solution = result.get("solution")
+        html = solution.get("response", "") if solution else ""
 
         if not html:
-            logger.warning(f"External bypasser returned empty response for '{target_url}'")
+            logger.warning("External bypasser returned empty response for '%s'", target_url)
             return None
 
-        return html
+        # "Challenge solved!" is the solver's verdict on its own work, and #1289 showed
+        # it can be reported alongside a page the caller then rejects. Say what actually
+        # came back, so a later report does not have to infer it from downstream errors.
+        marker = challenge_marker(html)
+        logger.debug(
+            "External bypasser page for '%s': %d bytes, challenge_marker=%r",
+            target_url,
+            len(html),
+            marker,
+        )
+        if marker:
+            # The solver's verdict is not evidence; the page is. Returning this one as a
+            # success is what made #1292 unrecoverable: the retry-and-rotate loop that
+            # could still have saved the search - the next mirror is a different
+            # DDoS-Guard host, in its own state - was never entered, and the challenge
+            # page's own __ddg cookies were filed as this host's clearance and replayed
+            # on every later request.
+            logger.warning(
+                "External bypasser reported success but returned a challenge page for "
+                "'%s' (%d bytes, marker=%r) - the solve did not clear the protection",
+                target_url,
+                len(html),
+                marker,
+            )
+            raise ChallengeNotSolvedError(marker)
+
+        try:
+            _store_solution_clearance(target_url, solution)
+        except AttributeError, KeyError, TypeError, ValueError:
+            # Storing clearance is an optimisation; the page is the product. The
+            # solution JSON comes from a service we do not control, so a surprise in
+            # its cookie shape must not discard HTML that already cost a ~30s solve
+            # and send the caller round for up to MAX_RETRY more of them.
+            logger.debug("Could not store bypass clearance for '%s'", target_url, exc_info=True)
 
     except requests.exceptions.Timeout:
-        logger.warning(f"External bypasser timed out for '{target_url}' (connect: {CONNECT_TIMEOUT}s, read: {read_timeout:.0f}s)")
+        logger.warning(
+            "External bypasser timed out for '%s' (connect: %ss, read: %.0fs)",
+            target_url,
+            CONNECT_TIMEOUT,
+            read_timeout,
+        )
     except requests.exceptions.RequestException as e:
-        logger.warning(f"External bypasser request failed for '{target_url}': {e}")
+        logger.warning("External bypasser request failed for '%s': %s", target_url, e)
     except (KeyError, TypeError, ValueError) as e:
-        logger.warning(f"External bypasser returned malformed response for '{target_url}': {e}")
+        logger.warning("External bypasser returned malformed response for '%s': %s", target_url, e)
+    else:
+        return html
 
     return None
 
 
-def _check_cancelled(cancel_flag: Optional[Event], context: str) -> None:
+def _check_cancelled(cancel_flag: Event | None, context: str) -> None:
     """Check if operation was cancelled and raise exception if so."""
     if cancel_flag and cancel_flag.is_set():
-        logger.info(f"External bypasser cancelled {context}")
-        raise BypassCancelledException("Bypass cancelled")
+        logger.info("External bypasser cancelled %s", context)
+        msg = "Bypass cancelled"
+        raise BypassCancelledError(msg)
 
 
-def _sleep_with_cancellation(seconds: float, cancel_flag: Optional[Event]) -> None:
+def _sleep_with_cancellation(seconds: float, cancel_flag: Event | None) -> None:
     """Sleep for the specified duration, checking for cancellation each second."""
     for _ in range(int(seconds)):
         _check_cancelled(cancel_flag, "during backoff")
@@ -97,32 +222,61 @@ def _sleep_with_cancellation(seconds: float, cancel_flag: Optional[Event]) -> No
 
 def get_bypassed_page(
     url: str,
-    selector: Optional["network.AAMirrorSelector"] = None,
-    cancel_flag: Optional[Event] = None
-) -> Optional[str]:
-    """Fetch HTML via external bypasser with retries and mirror rotation."""
+    selector: network.AAMirrorSelector | None = None,
+    cancel_flag: Event | None = None,
+) -> str | None:
+    """Fetch HTML via external bypasser with retries and mirror rotation.
+
+    Raises:
+        ChallengeNotSolvedError: every attempt came back still carrying a challenge.
+            Reported apart from returning None because the two ask the user for
+            opposite things: None means go and check the bypasser, this means the
+            bypasser is fine and the host is the one refusing.
+        BypassCancelledError: the caller's cancel flag was set.
+
+    """
     from shelfmark.download import network as network_module
 
     sel = selector or network_module.AAMirrorSelector()
+    unsolved_marker: str | None = None
 
     for attempt in range(1, MAX_RETRY + 1):
         _check_cancelled(cancel_flag, "by user")
 
         attempt_url = sel.rewrite(url)
-        result = _fetch_via_bypasser(attempt_url)
+        try:
+            result = _fetch_via_bypasser(attempt_url)
+        except ChallengeNotSolvedError as e:
+            # Worth the remaining attempts rather than an immediate give-up: the retry
+            # rotates onto the next mirror, and that is a different DDoS-Guard host with
+            # its own idea of whether this caller needs a CAPTCHA.
+            unsolved_marker = str(e) or unsolved_marker
+            result = None
         if result:
             return result
 
         if attempt == MAX_RETRY:
             break
 
-        delay = min(BACKOFF_CAP, BACKOFF_BASE * (2 ** (attempt - 1))) + random.random()
-        logger.info(f"External bypasser attempt {attempt}/{MAX_RETRY} failed, retrying in {delay:.1f}s")
+        delay = min(BACKOFF_CAP, BACKOFF_BASE * (2 ** (attempt - 1))) + _RNG.random()
+        logger.info(
+            "External bypasser attempt %s/%s failed, retrying in %.1fs",
+            attempt,
+            MAX_RETRY,
+            delay,
+        )
 
         _sleep_with_cancellation(delay, cancel_flag)
 
         new_base, action = sel.next_mirror_or_rotate_dns()
         if action in ("mirror", "dns") and new_base:
-            logger.info(f"Rotated {action} for retry")
+            logger.info("Rotated %s for retry", action)
 
+    if unsolved_marker:
+        msg = (
+            "The bypasser ran, but the site kept answering with a protection challenge "
+            f"(marker={unsolved_marker!r}). That is usually a manual CAPTCHA, which no "
+            "bypasser can answer - the bypasser itself is working. Try again shortly."
+        )
+        raise ChallengeNotSolvedError(msg)
     return None

shelfmark/bypass/fingerprint.py

@@ -1,52 +1,65 @@
 """Browser fingerprint profile management for bypass stealth."""
 
 import random
-from typing import Optional
 
 from shelfmark.core.logger import setup_logger
 
 logger = setup_logger(__name__)
 
 COMMON_RESOLUTIONS = [
-    (1920, 1080, 0.35),  
-    (1366, 768, 0.18),   
-    (1536, 864, 0.10),   
-    (1440, 900, 0.08),   
-    (1280, 720, 0.07),   
-    (1600, 900, 0.06),   
-    (1280, 800, 0.05),   
-    (2560, 1440, 0.04),  
-    (1680, 1050, 0.04),  
-    (1920, 1200, 0.03),  
+    (1920, 1080, 0.35),
+    (1366, 768, 0.18),
+    (1536, 864, 0.10),
+    (1440, 900, 0.08),
+    (1280, 720, 0.07),
+    (1600, 900, 0.06),
+    (1280, 800, 0.05),
+    (2560, 1440, 0.04),
+    (1680, 1050, 0.04),
+    (1920, 1200, 0.03),
 ]
 
 # Current screen size (module-level singleton)
-_current_screen_size: Optional[tuple[int, int]] = None
+_current_screen_size: tuple[int, int] | None = None
+_RNG = random.SystemRandom()
 
 
 def get_screen_size() -> tuple[int, int]:
+    """Return the current synthetic screen size, generating one if needed."""
     global _current_screen_size
     if _current_screen_size is None:
         _current_screen_size = _generate_screen_size()
-        logger.debug(f"Generated initial screen size: {_current_screen_size[0]}x{_current_screen_size[1]}")
+        logger.debug(
+            "Generated initial screen size: %sx%s",
+            _current_screen_size[0],
+            _current_screen_size[1],
+        )
     return _current_screen_size
 
 
 def rotate_screen_size() -> tuple[int, int]:
+    """Rotate to a new synthetic screen size and return it."""
     global _current_screen_size
     old_size = _current_screen_size
     _current_screen_size = _generate_screen_size()
     width, height = _current_screen_size
 
     if old_size:
-        logger.info(f"Rotated screen size: {old_size[0]}x{old_size[1]} -> {width}x{height}")
+        logger.info(
+            "Rotated screen size: %sx%s -> %sx%s",
+            old_size[0],
+            old_size[1],
+            width,
+            height,
+        )
     else:
-        logger.info(f"Generated screen size: {width}x{height}")
+        logger.info("Generated screen size: %sx%s", width, height)
 
     return _current_screen_size
 
 
 def clear_screen_size() -> None:
+    """Clear the cached synthetic screen size."""
     global _current_screen_size
     _current_screen_size = None
 
@@ -54,4 +67,4 @@ def clear_screen_size() -> None:
 def _generate_screen_size() -> tuple[int, int]:
     resolutions = [(w, h) for w, h, _ in COMMON_RESOLUTIONS]
     weights = [weight for _, _, weight in COMMON_RESOLUTIONS]
-    return random.choices(resolutions, weights=weights)[0]
+    return _RNG.choices(resolutions, weights=weights)[0]

shelfmark/bypass/waiting_room.py

@@ -0,0 +1,16 @@
+"""Recognize Anna's Archive pages that require a live JavaScript timer."""
+
+from urllib.parse import urlparse
+
+from bs4 import BeautifulSoup
+
+
+class WaitingRoomTimeoutError(TimeoutError):
+    """The source waiting room did not finish within the browser session budget."""
+
+
+def is_aa_waiting_room(url: str, html: str) -> bool:
+    """Match the download route and actual timer element, not a script reference."""
+    return urlparse(url).path.startswith("/slow_download/") and bool(
+        BeautifulSoup(html, "html.parser").select_one(".js-partner-countdown")
+    )

shelfmark/config/booklore_settings.py

@@ -1,5 +1,8 @@
+"""Helpers for Booklore settings validation, option loading, and connection tests."""
+
 from __future__ import annotations
 
+import hashlib
 from typing import Any
 
 from shelfmark.core.config import config
@@ -21,7 +24,7 @@ _BOOKLORE_OPTIONS_CACHE: dict[str, Any] = {
 
 
 def _get_booklore_cache_key(base_url: str, username: str, password: str) -> str:
-    return f"{base_url}|{username}|{hash(password)}"
+    return f"{base_url}|{username}|{hashlib.sha256(password.encode()).hexdigest()}"
 
 
 def _get_booklore_select_options(
@@ -121,7 +124,7 @@ def get_booklore_library_options() -> list[dict[str, Any]]:
 
     base_url = str(config.get("BOOKLORE_HOST", "") or "").strip().rstrip("/")
     username = str(config.get("BOOKLORE_USERNAME", "") or "").strip()
-    password = config.get("BOOKLORE_PASSWORD", "") or ""
+    password = str(config.get("BOOKLORE_PASSWORD", "") or "")
 
     if not base_url or not username or not password:
         return []
@@ -130,12 +133,13 @@ def get_booklore_library_options() -> list[dict[str, Any]]:
 
     try:
         library_options, _ = _get_booklore_cached_options(base_url, username, password)
-        return library_options
-    except Exception as exc:
-        logger.error(f"Failed to fetch Booklore libraries: {exc}")
+    except Exception:
+        logger.exception("Failed to fetch Booklore libraries")
         if _BOOKLORE_OPTIONS_CACHE.get("key") == cache_key:
             return _BOOKLORE_OPTIONS_CACHE.get("library_options", [])
         return []
+    else:
+        return library_options
 
 
 def get_booklore_path_options() -> list[dict[str, Any]]:
@@ -145,7 +149,7 @@ def get_booklore_path_options() -> list[dict[str, Any]]:
 
     base_url = str(config.get("BOOKLORE_HOST", "") or "").strip().rstrip("/")
     username = str(config.get("BOOKLORE_USERNAME", "") or "").strip()
-    password = config.get("BOOKLORE_PASSWORD", "") or ""
+    password = str(config.get("BOOKLORE_PASSWORD", "") or "")
 
     if not base_url or not username or not password:
         return []
@@ -154,19 +158,22 @@ def get_booklore_path_options() -> list[dict[str, Any]]:
 
     try:
         _, path_options = _get_booklore_cached_options(base_url, username, password)
-        return path_options
-    except Exception as exc:
-        logger.error(f"Failed to fetch Booklore paths: {exc}")
+    except Exception:
+        logger.exception("Failed to fetch Booklore paths")
         if _BOOKLORE_OPTIONS_CACHE.get("key") == cache_key:
             return _BOOKLORE_OPTIONS_CACHE.get("path_options", [])
         return []
+    else:
+        return path_options
 
 
-def test_booklore_connection(current_values: dict[str, Any] | None = None) -> dict[str, Any]:
+def check_booklore_connection(
+    current_values: dict[str, Any] | None = None,
+) -> dict[str, Any]:
     """Test the Booklore connection using current form values."""
     current_values = current_values or {}
 
-    def _get_value(key: str, default: Any = None) -> Any:
+    def _get_value(key: str, default: object = None) -> object:
         value = current_values.get(key)
         if value not in (None, ""):
             return value
@@ -176,22 +183,22 @@ def test_booklore_connection(current_values: dict[str, Any] | None = None) -> di
 
     base_url = str(_get_value("BOOKLORE_HOST", "") or "").strip().rstrip("/")
     username = str(_get_value("BOOKLORE_USERNAME", "") or "").strip()
-    password = _get_value("BOOKLORE_PASSWORD", "") or ""
+    password = str(_get_value("BOOKLORE_PASSWORD", "") or "")
 
     if not base_url:
-        return {"success": False, "message": "Booklore URL is required"}
+        return {"success": False, "message": "Grimmory URL is required"}
     if not username:
-        return {"success": False, "message": "Booklore username is required"}
+        return {"success": False, "message": "Grimmory username is required"}
     if not password:
-        return {"success": False, "message": "Booklore password is required"}
+        return {"success": False, "message": "Grimmory password is required"}
 
     try:
         library_options, _ = _get_booklore_select_options(base_url, username, password)
-
-        message = "Connected to Booklore"
-        if library_options:
-            message = f"Connected to Booklore ({len(library_options)} libraries)"
-
-        return {"success": True, "message": message}
     except BookloreError as exc:
         return {"success": False, "message": str(exc)}
+    else:
+        message = "Connected to Grimmory"
+        if library_options:
+            message = f"Connected to Grimmory ({len(library_options)} libraries)"
+
+        return {"success": True, "message": message}

shelfmark/config/download_settings_handlers.py

@@ -0,0 +1,110 @@
+from __future__ import annotations
+
+import re
+from pathlib import Path
+from typing import Any
+
+_USER_PLACEHOLDER_PATTERN = re.compile(r"\{user\}", re.IGNORECASE)
+
+
+def _get_download_setting_value(
+    current_values: dict[str, Any] | None,
+    key: str,
+    *,
+    default: object = None,
+) -> object:
+    """Read a downloads setting from unsaved form values first, then persisted config."""
+    from shelfmark.core.config import config
+
+    current_values = current_values or {}
+    if key in current_values:
+        return current_values[key]
+    if default is None:
+        return config.get(key)
+    return config.get(key, default)
+
+
+def _resolve_destination_test_path(
+    configured_path: str,
+) -> tuple[Path, str | None]:
+    """Resolve a safe path to validate for destination test actions."""
+    stripped_path = configured_path.strip()
+
+    if not _USER_PLACEHOLDER_PATTERN.search(stripped_path):
+        return Path(stripped_path), None
+
+    base_prefix = _USER_PLACEHOLDER_PATTERN.split(stripped_path, maxsplit=1)[0].rstrip("/")
+    if not base_prefix and not stripped_path.startswith("/"):
+        return Path(stripped_path), None
+
+    base_path = base_prefix or "/"
+    return Path(base_path), (
+        f" (tested base path {base_path} from configured template {stripped_path})"
+    )
+
+
+def _test_folder_destination(
+    *,
+    current_values: dict[str, Any] | None = None,
+    is_audiobook: bool,
+) -> dict[str, Any]:
+    """Validate a folder destination using current form values."""
+    from shelfmark.download.postprocess.destination import validate_destination
+
+    destination_value = _get_download_setting_value(
+        current_values,
+        "DESTINATION",
+        default="/books",
+    )
+    destination = str(destination_value or "").strip()
+
+    label = "Books destination"
+    message_suffix = ""
+
+    if is_audiobook:
+        audiobook_value = _get_download_setting_value(
+            current_values,
+            "DESTINATION_AUDIOBOOK",
+            default="",
+        )
+        audiobook_destination = str(audiobook_value or "").strip()
+        if audiobook_destination:
+            destination = audiobook_destination
+            label = "Audiobook destination"
+        else:
+            label = "Audiobook destination"
+            message_suffix = " (using the Books destination)"
+
+    if not destination:
+        return {"success": False, "message": f"{label} is required"}
+
+    test_path, path_message = _resolve_destination_test_path(destination)
+    if path_message:
+        message_suffix += path_message
+
+    errors: list[str] = []
+
+    def _status_callback(status: str, message: str | None) -> None:
+        if status == "error" and message:
+            errors.append(message)
+
+    if not validate_destination(test_path, _status_callback):
+        message = errors[-1] if errors else f"Cannot access destination: {test_path}"
+        if message_suffix:
+            message = f"{message}{message_suffix}"
+        return {"success": False, "message": message}
+
+    return {
+        "success": True,
+        "message": f"{label} is writable: {test_path}{message_suffix}",
+    }
+
+
+def check_books_destination(current_values: dict[str, Any] | None = None) -> dict[str, Any]:
+    """Validate the configured books destination."""
+    return _test_folder_destination(current_values=current_values, is_audiobook=False)
+
+
+def check_audiobook_destination(current_values: dict[str, Any] | None = None) -> dict[str, Any]:
+    """Validate the configured audiobook destination."""
+    return _test_folder_destination(current_values=current_values, is_audiobook=True)

shelfmark/config/email_settings.py

@@ -1,17 +1,25 @@
+"""Helpers for email settings validation and SMTP connection tests."""
+
 from __future__ import annotations
 
+import smtplib
 from typing import Any
 
 from shelfmark.core.config import config
-from shelfmark.download.outputs.email import EmailOutputError, build_email_smtp_config, test_smtp_connection
+from shelfmark.download.outputs.email import (
+    EmailOutputError,
+    build_email_smtp_config,
+    test_smtp_connection,
+)
 
 
-def test_email_connection(current_values: dict[str, Any] | None = None) -> dict[str, Any]:
+def check_email_connection(
+    current_values: dict[str, Any] | None = None,
+) -> dict[str, Any]:
     """Test SMTP connectivity using current form values (including unsaved changes)."""
-
     current_values = current_values or {}
 
-    def _get_value(key: str, default: Any = None) -> Any:
+    def _get_value(key: str, default: object = None) -> object:
         value = current_values.get(key)
         if value not in (None, ""):
             return value
@@ -28,15 +36,15 @@ def test_email_connection(current_values: dict[str, Any] | None = None) -> dict[
         "EMAIL_FROM": _get_value("EMAIL_FROM", ""),
         "EMAIL_SUBJECT_TEMPLATE": _get_value("EMAIL_SUBJECT_TEMPLATE", "{Title}"),
         "EMAIL_SMTP_TIMEOUT_SECONDS": _get_value("EMAIL_SMTP_TIMEOUT_SECONDS", 60),
-        "EMAIL_ALLOW_UNVERIFIED_TLS": _get_value("EMAIL_ALLOW_UNVERIFIED_TLS", False),
+        "EMAIL_ALLOW_UNVERIFIED_TLS": _get_value("EMAIL_ALLOW_UNVERIFIED_TLS", default=False),
     }
 
     try:
         smtp_config = build_email_smtp_config(settings)
         test_smtp_connection(smtp_config)
-        return {"success": True, "message": "Connected to SMTP server"}
     except EmailOutputError as exc:
         return {"success": False, "message": str(exc)}
-    except Exception as exc:
+    except (OSError, smtplib.SMTPException) as exc:
         return {"success": False, "message": f"SMTP test failed: {exc}"}
-
+    else:
+        return {"success": True, "message": "Connected to SMTP server"}

shelfmark/config/env.py

@@ -3,43 +3,88 @@
 import json
 import os
 import shutil
+import tempfile
 from pathlib import Path
 
+LOG_LEVELS = ("DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL")
+
 
 def string_to_bool(s: str) -> bool:
     """Convert string to boolean."""
     return s.lower() in ["true", "yes", "1", "y"]
 
 
+def _read_advanced_config(key: str) -> object | None:
+    """Read a key from the advanced settings file (import-time safe)."""
+    config_dir = Path(os.getenv("CONFIG_DIR", "/config"))
+    config_file = config_dir / "plugins" / "advanced.json"
+
+    if config_file.exists():
+        try:
+            with config_file.open() as f:
+                config = json.load(f)
+                if key in config:
+                    return config[key]
+        except json.JSONDecodeError, OSError:
+            pass
+
+    return None
+
+
 def _read_debug_from_config() -> bool:
     """Read DEBUG from env var or config file (import-time safe)."""
     env_debug = os.environ.get("DEBUG")
     if env_debug is not None:
         return string_to_bool(env_debug)
 
-    # Try to read from config file
-    config_dir = Path(os.getenv("CONFIG_DIR", "/config"))
-    config_file = config_dir / "plugins" / "advanced.json"
-
-    if config_file.exists():
-        try:
-            with open(config_file, "r") as f:
-                config = json.load(f)
-                if "DEBUG" in config:
-                    return bool(config["DEBUG"])
-        except (json.JSONDecodeError, OSError):
-            pass
+    value = _read_advanced_config("DEBUG")
+    if value is not None:
+        return bool(value)
 
     return False
 
 
+def normalize_log_level(raw: str | None) -> str:
+    """Normalize a log level name, falling back to INFO when unrecognized."""
+    if raw is None:
+        return "INFO"
+
+    normalized = raw.strip().upper()
+    # "WARN" is a logging alias, but gunicorn only accepts "warning".
+    if normalized == "WARN":
+        normalized = "WARNING"
+
+    if normalized not in LOG_LEVELS:
+        return "INFO"
+
+    return normalized
+
+
+def _read_log_level_from_config(debug: bool) -> str:
+    """Resolve the app log level from DEBUG, env var, or config file.
+
+    DEBUG wins when enabled, mirroring how entrypoint.sh picks gunicorn's level.
+    Otherwise LOG_LEVEL is read from the env var, then the settings file, and
+    falls back to INFO when unset or unrecognized.
+    """
+    if debug:
+        return "DEBUG"
+
+    raw = os.environ.get("LOG_LEVEL")
+    if raw is None:
+        value = _read_advanced_config("LOG_LEVEL")
+        raw = value if isinstance(value, str) else None
+
+    return normalize_log_level(raw)
+
+
 def _is_sqlite_file(path: Path) -> bool:
     """Check if a file is a valid SQLite database by reading magic bytes."""
     try:
-        with open(path, "rb") as f:
+        with path.open("rb") as f:
             header = f.read(16)
             return header[:16] == b"SQLite format 3\x00"
-    except (OSError, PermissionError):
+    except OSError, PermissionError:
         return False
 
 
@@ -67,16 +112,20 @@ def _is_config_dir_writable() -> bool:
         test_file = CONFIG_DIR / ".write_test"
         test_file.touch()
         test_file.unlink()
-        return True
-    except (OSError, PermissionError):
+    except OSError, PermissionError:
         return False
+    else:
+        return True
 
 
 def is_covers_cache_enabled() -> bool:
     """Check if cover caching is enabled (requires setting + writable config dir)."""
     from shelfmark.core.config import config
+
     setting_enabled = config.get("COVERS_CACHE_ENABLED", True)
-    return setting_enabled and _is_config_dir_writable()
+    if isinstance(setting_enabled, str):
+        return string_to_bool(setting_enabled) and _is_config_dir_writable()
+    return bool(setting_enabled) and _is_config_dir_writable()
 
 
 # =============================================================================
@@ -87,7 +136,7 @@ CONFIG_DIR = Path(os.getenv("CONFIG_DIR", "/config"))
 LOG_ROOT = Path(os.getenv("LOG_ROOT", "/var/log/"))
 LOG_DIR = LOG_ROOT / "shelfmark"
 LOG_FILE = LOG_DIR / "shelfmark.log"
-TMP_DIR = Path(os.getenv("TMP_DIR", "/tmp/shelfmark"))
+TMP_DIR = Path(os.getenv("TMP_DIR", (Path(tempfile.gettempdir()) / "shelfmark").as_posix()))
 INGEST_DIR = Path(os.getenv("INGEST_DIR", "/books"))
 
 
@@ -96,7 +145,7 @@ INGEST_DIR = Path(os.getenv("INGEST_DIR", "/books"))
 # =============================================================================
 
 DEBUG = _read_debug_from_config()
-LOG_LEVEL = "DEBUG" if DEBUG else "INFO"
+LOG_LEVEL = _read_log_level_from_config(DEBUG)
 ENABLE_LOGGING = string_to_bool(os.getenv("ENABLE_LOGGING", "true"))
 
 
@@ -115,6 +164,12 @@ FLASK_PORT = int(os.getenv("FLASK_PORT", "8084"))
 SESSION_COOKIE_SECURE_ENV = os.getenv("SESSION_COOKIE_SECURE", "false")
 SESSION_COOKIE_NAME = "shelfmark_session"
 CWA_DB_PATH = _resolve_cwa_db_path()
+HIDE_LOCAL_AUTH = string_to_bool(os.getenv("HIDE_LOCAL_AUTH", "false"))
+DISABLE_LOCAL_AUTH = string_to_bool(os.getenv("DISABLE_LOCAL_AUTH", "false"))
+# Optional static API key. When set, requests carrying it as a Bearer token
+# (or X-Api-Key) are authenticated as an admin for that request only.
+SHELFMARK_API_KEY = os.getenv("SHELFMARK_API_KEY", "").strip()
+OIDC_AUTO_REDIRECT = string_to_bool(os.getenv("OIDC_AUTO_REDIRECT", "false"))
 
 
 # =============================================================================
@@ -149,7 +204,22 @@ ONBOARDING = string_to_bool(os.getenv("ONBOARDING", "true"))
 # Debug: skip specific download sources for testing fallback chains
 # Comma-separated values: aa-fast, aa-slow-nowait, aa-slow-wait, libgen, zlib, welib
 _DEBUG_SKIP_SOURCES_RAW = os.getenv("DEBUG_SKIP_SOURCES", "").strip().lower()
-DEBUG_SKIP_SOURCES = set(s.strip() for s in _DEBUG_SKIP_SOURCES_RAW.split(",") if s.strip())
+DEBUG_SKIP_SOURCES = {s.strip() for s in _DEBUG_SKIP_SOURCES_RAW.split(",") if s.strip()}
+
+# Debug: keep DDoS-Guard's __ddg8_/__ddg9_/__ddg10_ in the clearance store instead of
+# dropping them after a solve.
+#
+# Which of DDoS-Guard's cookies actually *are* clearance is not settled. The store treats
+# the trio as describing one check (client IP, timestamp, token) and drops them, on the
+# reasoning that replaying a stale IP/timestamp is what re-arms the ?check=1 loop - see
+# shelfmark.bypass.cookie_store. Field reports on issue #1276 point the other way: every
+# request after a successful solve was challenged again, which is only consistent with
+# what the store keeps not being sufficient clearance on its own.
+#
+# Deliberately env-only and off by default: this is a knob for reproducing the question
+# against a live host, not a setting to offer users. Set it to true, solve once, and watch
+# whether the next search still logs "Redirect loop detected".
+DDG_REPLAY_PER_CHECK_COOKIES = string_to_bool(os.getenv("DDG_REPLAY_PER_CHECK_COOKIES", "false"))
 
 
 # =============================================================================

shelfmark/config/migrations.py

@@ -1,8 +1,14 @@
 """Configuration migration helpers."""
 
-import json
-from typing import Any, Callable
+from __future__ import annotations
 
+import json
+from pathlib import Path
+from typing import TYPE_CHECKING, Any, Protocol
+
+if TYPE_CHECKING:
+    from collections.abc import Callable, Sequence
+    from os import PathLike
 
 _DEPRECATED_SETTINGS_RESTRICTION_KEYS = (
     "PROXY_AUTH_RESTRICT_SETTINGS_TO_ADMIN",
@@ -10,8 +16,25 @@ _DEPRECATED_SETTINGS_RESTRICTION_KEYS = (
     "RESTRICT_SETTINGS_TO_ADMIN",
 )
 
+# The audiobook format list shipped as the default until the format sets were unified.
+# It only covered m4b/mp3, so FLAC/OPUS/OGG/M4A releases were dropped from search results
+# and rejected after download - and the wider default alone would never reach existing
+# installs, because initialize_default_configs() only writes defaults when the config
+# file does not exist yet.
+_LEGACY_AUDIOBOOK_FORMATS_DEFAULT = ("m4b", "mp3")
 
-def _as_bool(value: Any) -> bool:
+
+class MigrationLogger(Protocol):
+    """Logger surface used by config migration helpers."""
+
+    def info(self, msg: str, *args: object) -> object: ...
+
+    def debug(self, msg: str, *args: object) -> object: ...
+
+    def exception(self, msg: str, *args: object) -> object: ...
+
+
+def _as_bool(value: object) -> bool:
     if isinstance(value, bool):
         return value
     if isinstance(value, str):
@@ -23,10 +46,7 @@ def _pick_legacy_settings_restriction(config: dict[str, Any]) -> bool | None:
     """Pick the best legacy admin-restriction value to migrate."""
     auth_method = str(config.get("AUTH_METHOD", "")).strip().lower()
 
-    if (
-        auth_method == "proxy"
-        and "PROXY_AUTH_RESTRICT_SETTINGS_TO_ADMIN" in config
-    ):
+    if auth_method == "proxy" and "PROXY_AUTH_RESTRICT_SETTINGS_TO_ADMIN" in config:
         return _as_bool(config.get("PROXY_AUTH_RESTRICT_SETTINGS_TO_ADMIN"))
 
     if auth_method == "cwa" and "CWA_RESTRICT_SETTINGS_TO_ADMIN" in config:
@@ -44,15 +64,63 @@ def _pick_legacy_settings_restriction(config: dict[str, Any]) -> bool | None:
     return None
 
 
+def migrate_audiobook_formats(
+    *,
+    load_general_config: Callable[[], dict[str, Any]],
+    # `object` rather than `None`: the result is discarded, and savers that report
+    # success (settings_registry.save_config_file returns bool) are not assignable to a
+    # `-> None` callable.
+    save_general_config: Callable[[dict[str, Any]], object],
+    widened_formats: Sequence[str],
+    logger: MigrationLogger,
+) -> None:
+    """Widen an untouched audiobook format list to the current, fuller default.
+
+    Only a list that still matches the old default exactly is rewritten. Any other value
+    means someone chose it deliberately, and a migration that "helpfully" re-enabled
+    formats a user had turned off would be worse than leaving them on the narrow list.
+    """
+    try:
+        config = load_general_config()
+
+        if "SUPPORTED_AUDIOBOOK_FORMATS" not in config:
+            # Nothing persisted, so the field default already applies.
+            logger.debug("No persisted audiobook formats - the current default applies")
+            return
+
+        current = config.get("SUPPORTED_AUDIOBOOK_FORMATS")
+        if not isinstance(current, list):
+            return
+
+        normalized = {str(fmt).strip().lower() for fmt in current if str(fmt).strip()}
+        if normalized != set(_LEGACY_AUDIOBOOK_FORMATS_DEFAULT):
+            logger.debug(
+                "Audiobook formats were customized (%s) - left unchanged", sorted(normalized)
+            )
+            return
+
+        save_general_config({"SUPPORTED_AUDIOBOOK_FORMATS": list(widened_formats)})
+        logger.info(
+            "Widened audiobook formats from the legacy default %s to %s",
+            list(_LEGACY_AUDIOBOOK_FORMATS_DEFAULT),
+            list(widened_formats),
+        )
+
+    except FileNotFoundError:
+        logger.debug("No existing general config file found - nothing to migrate")
+    except Exception:
+        logger.exception("Failed to migrate audiobook formats")
+
+
 def migrate_security_settings(
     *,
     load_security_config: Callable[[], dict[str, Any]],
     load_users_config: Callable[[], dict[str, Any]],
     save_users_config: Callable[[dict[str, Any]], None],
     ensure_config_dir: Callable[[], None],
-    get_config_path: Callable[[], Any],
+    get_config_path: Callable[[], str | PathLike[str]],
     sync_builtin_admin_user: Callable[[str, str], None],
-    logger: Any,
+    logger: MigrationLogger,
 ) -> None:
     """Migrate legacy security keys and sync builtin admin credentials."""
     try:
@@ -67,49 +135,62 @@ def migrate_security_settings(
                 if old_value:
                     config["AUTH_METHOD"] = "cwa"
                     logger.info("Migrated USE_CWA_AUTH=True to AUTH_METHOD='cwa'")
+                elif config.get("BUILTIN_USERNAME") and config.get("BUILTIN_PASSWORD_HASH"):
+                    config["AUTH_METHOD"] = "builtin"
+                    logger.info("Migrated USE_CWA_AUTH=False to AUTH_METHOD='builtin'")
                 else:
-                    if config.get("BUILTIN_USERNAME") and config.get("BUILTIN_PASSWORD_HASH"):
-                        config["AUTH_METHOD"] = "builtin"
-                        logger.info("Migrated USE_CWA_AUTH=False to AUTH_METHOD='builtin'")
-                    else:
-                        config["AUTH_METHOD"] = "none"
-                        logger.info("Migrated USE_CWA_AUTH=False to AUTH_METHOD='none'")
+                    config["AUTH_METHOD"] = "none"
+                    logger.info("Migrated USE_CWA_AUTH=False to AUTH_METHOD='none'")
                 migrated_security = True
             else:
                 logger.info("Removed deprecated USE_CWA_AUTH setting (AUTH_METHOD already exists)")
                 migrated_security = True
 
+        # Backfill AUTH_METHOD for configs that have builtin credentials but
+        # were never migrated from USE_CWA_AUTH (e.g. dev builds that predated
+        # the AUTH_METHOD field).
+        if (
+            "AUTH_METHOD" not in config
+            and config.get("BUILTIN_USERNAME")
+            and config.get("BUILTIN_PASSWORD_HASH")
+        ):
+            config["AUTH_METHOD"] = "builtin"
+            migrated_security = True
+            logger.info(
+                "Backfilled AUTH_METHOD='builtin' from legacy "
+                "BUILTIN_USERNAME/BUILTIN_PASSWORD_HASH credentials"
+            )
+
         if "RESTRICT_SETTINGS_TO_ADMIN" not in users_config:
             legacy_restrict = _pick_legacy_settings_restriction(config)
             if legacy_restrict is not None:
                 save_users_config({"RESTRICT_SETTINGS_TO_ADMIN": legacy_restrict})
                 migrated_users = True
                 logger.info(
-                    "Migrated legacy settings-admin restriction to users.RESTRICT_SETTINGS_TO_ADMIN="
-                    f"{legacy_restrict}"
+                    "Migrated legacy settings-admin restriction to users.RESTRICT_SETTINGS_TO_ADMIN=%s",
+                    legacy_restrict,
                 )
 
         for deprecated_key in _DEPRECATED_SETTINGS_RESTRICTION_KEYS:
             if deprecated_key in config:
                 config.pop(deprecated_key, None)
                 migrated_security = True
-                logger.info(f"Removed deprecated security setting: {deprecated_key}")
+                logger.info("Removed deprecated security setting: %s", deprecated_key)
 
         try:
             sync_builtin_admin_user(
                 config.get("BUILTIN_USERNAME", ""),
                 config.get("BUILTIN_PASSWORD_HASH", ""),
             )
-        except Exception as exc:
-            logger.error(
-                "Failed to sync builtin credentials to users database during migration: "
-                f"{exc}"
+        except Exception:
+            logger.exception(
+                "Failed to sync builtin credentials to users database during migration"
             )
 
         if migrated_security:
             ensure_config_dir()
-            config_path = get_config_path()
-            with open(config_path, "w") as f:
+            config_path = Path(get_config_path())
+            with config_path.open("w") as f:
                 json.dump(config, f, indent=2)
             logger.info("Security settings migration completed successfully")
         elif migrated_users:
@@ -119,5 +200,5 @@ def migrate_security_settings(
 
     except FileNotFoundError:
         logger.debug("No existing security config file found - nothing to migrate")
-    except Exception as exc:
-        logger.error(f"Failed to migrate security settings: {exc}")
+    except Exception:
+        logger.exception("Failed to migrate security settings")

shelfmark/config/notifications_settings.py

@@ -6,10 +6,12 @@ import re
 from typing import Any
 from urllib.parse import urlsplit
 
+from shelfmark.core.config import config as app_config
 from shelfmark.core.notifications import NotificationEvent, send_test_notification
 from shelfmark.core.settings_registry import (
     ActionButton,
     HeadingField,
+    SettingsField,
     TableField,
     load_config_file,
     register_on_save,
@@ -123,7 +125,7 @@ def _count_invalid_route_urls(routes: list[dict[str, Any]]) -> int:
 
 
 def _ensure_default_route_row(routes: list[dict[str, Any]]) -> list[dict[str, Any]]:
-    return routes if routes else [dict(row) for row in _DEFAULT_ROUTE_ROWS]
+    return routes or [dict(row) for row in _DEFAULT_ROUTE_ROWS]
 
 
 def _extract_unique_route_urls(routes: list[dict[str, Any]]) -> list[str]:
@@ -141,6 +143,7 @@ def _extract_unique_route_urls(routes: list[dict[str, Any]]) -> list[str]:
 
 
 def build_notification_test_result(routes_input: Any, *, scope_label: str) -> dict[str, Any]:
+    """Validate routes and return a test-notification result payload."""
     invalid_event_count = _count_invalid_route_events(routes_input)
     if invalid_event_count:
         return {
@@ -245,8 +248,9 @@ def _on_save_notifications(values: dict[str, Any]) -> dict[str, Any]:
 
 
 def _test_admin_notification_action(current_values: dict[str, Any]) -> dict[str, Any]:
-    persisted = load_config_file("notifications")
-    effective: dict[str, Any] = dict(persisted)
+    effective: dict[str, Any] = {
+        "ADMIN_NOTIFICATION_ROUTES": app_config.get("ADMIN_NOTIFICATION_ROUTES", []),
+    }
     if isinstance(current_values, dict):
         effective.update(current_values)
 
@@ -258,7 +262,7 @@ register_on_save("notifications", _on_save_notifications)
 
 
 @register_settings("notifications", "Notifications", icon="bell", order=7)
-def notifications_settings():
+def notifications_settings() -> list[SettingsField]:
     """Global notifications settings."""
     return [
         HeadingField(

shelfmark/config/security.py

@@ -1,26 +1,32 @@
 """Authentication settings registration."""
 
-from typing import Any, Dict, Callable
+from typing import TYPE_CHECKING, Any
 
 from shelfmark.config.migrations import migrate_security_settings
 from shelfmark.config.security_handlers import (
+    check_oidc_connection,
     on_save_security,
-    test_oidc_connection,
 )
+from shelfmark.core.config import config as app_config
 from shelfmark.core.logger import setup_logger
 from shelfmark.core.settings_registry import (
-    register_settings,
-    register_on_save,
-    load_config_file,
-    TextField,
-    SelectField,
-    PasswordField,
-    CheckboxField,
     ActionButton,
+    CheckboxField,
+    CustomComponentField,
+    PasswordField,
+    SelectField,
+    SettingsField,
     TagListField,
+    TextField,
+    load_config_file,
+    register_on_save,
+    register_settings,
 )
 from shelfmark.core.user_db import sync_builtin_admin_user
 
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
 logger = setup_logger(__name__)
 
 
@@ -28,25 +34,24 @@ def _auth_condition(auth_method: str) -> dict[str, str]:
     return {"field": "AUTH_METHOD", "value": auth_method}
 
 
-def _ui_field(factory: Callable[..., Any], **kwargs: Any) -> Any:
-    return factory(env_supported=False, **kwargs)
-
-
-def _auth_ui_field(factory: Callable[..., Any], auth_method: str, **kwargs: Any) -> Any:
-    return _ui_field(factory, show_when=_auth_condition(auth_method), **kwargs)
+def _auth_field(factory: Callable[..., Any], auth_method: str, **kwargs: Any) -> Any:
+    return factory(show_when=_auth_condition(auth_method), **kwargs)
 
 
 def _migrate_security_settings() -> None:
     from shelfmark.core.settings_registry import (
-        _get_config_file_path,
         _ensure_config_dir,
+        _get_config_file_path,
         save_config_file,
     )
 
+    def _save_users_config(values: dict[str, Any]) -> None:
+        save_config_file("users", values)
+
     migrate_security_settings(
         load_security_config=lambda: load_config_file("security"),
         load_users_config=lambda: load_config_file("users"),
-        save_users_config=lambda values: save_config_file("users", values),
+        save_users_config=_save_users_config,
         ensure_config_dir=lambda: _ensure_config_dir("security"),
         get_config_path=lambda: _get_config_file_path("security"),
         sync_builtin_admin_user=sync_builtin_admin_user,
@@ -54,22 +59,24 @@ def _migrate_security_settings() -> None:
     )
 
 
-
-def _on_save_security(values: Dict[str, Any]) -> Dict[str, Any]:
+def _on_save_security(values: dict[str, Any]) -> dict[str, Any]:
     return on_save_security(values)
 
 
-def _test_oidc_connection() -> Dict[str, Any]:
-    return test_oidc_connection(
-        load_security_config=lambda: load_config_file("security"),
+def _test_oidc_connection(current_values: dict[str, Any] | None = None) -> dict[str, Any]:
+    return check_oidc_connection(
+        load_security_config=lambda: {
+            "OIDC_DISCOVERY_URL": app_config.get("OIDC_DISCOVERY_URL", ""),
+        },
+        current_values=current_values or {},
         logger=logger,
     )
 
 
 @register_settings("security", "Security", icon="shield", order=5)
-def security_settings():
+def security_settings() -> list[SettingsField]:
     """Security and authentication settings."""
-    from shelfmark.config.env import CWA_DB_PATH
+    from shelfmark.config.env import CWA_DB_PATH, DISABLE_LOCAL_AUTH
 
     cwa_db_available = CWA_DB_PATH is not None and CWA_DB_PATH.exists()
 
@@ -78,31 +85,65 @@ def security_settings():
         {"label": "Local", "value": "builtin"},
         {"label": "Proxy Authentication", "value": "proxy"},
         {"label": "OIDC (OpenID Connect)", "value": "oidc"},
+        {"label": "Calibre-Web Database", "value": "cwa"},
     ]
-    if cwa_db_available:
-        auth_method_options.append({"label": "Calibre-Web Database", "value": "cwa"})
-
-    auth_method_description = "Select the authentication method for accessing Shelfmark."
-    if not cwa_db_available:
-        auth_method_description += " Calibre-Web database option requires mounting your Calibre-Web app.db to /auth/app.db."
 
     fields = [
         SelectField(
             key="AUTH_METHOD",
             label="Authentication Method",
-            description=auth_method_description,
+            description=(
+                "Select the authentication method for accessing Shelfmark. "
+                "Restart container after changing Calibre-Web passwords."
+            ),
             options=auth_method_options,
             default="none",
-            env_supported=False,
+        ),
+        CustomComponentField(
+            key="builtin_admin_requirement",
+            component="oidc_admin_hint",
+            label=(
+                "Local authentication is inactive until a local admin account with a "
+                "password is created."
+            ),
+            show_when=_auth_condition("builtin"),
+        ),
+        *(
+            []
+            if DISABLE_LOCAL_AUTH
+            else [
+                CustomComponentField(
+                    key="oidc_admin_requirement",
+                    component="oidc_admin_hint",
+                    label="A local admin account is required before OIDC can be enabled.",
+                    show_when=_auth_condition("oidc"),
+                ),
+            ]
+        ),
+        *(
+            []
+            if cwa_db_available
+            else [
+                CustomComponentField(
+                    key="cwa_db_missing",
+                    component="oidc_admin_hint",
+                    label=(
+                        "Calibre-Web database not detected. Mount your app.db to "
+                        "/auth/app.db to enable this method. Authentication will fall "
+                        "back to none until the database is available."
+                    ),
+                    show_when=_auth_condition("cwa"),
+                ),
+            ]
         ),
         ActionButton(
             key="open_users_tab",
             label="Go to Users",
             description="Configure local users and admin access in the Users tab.",
             style="primary",
-            show_when=_auth_condition("builtin"),
+            show_when={"field": "AUTH_METHOD", "value": ["builtin", "oidc"]},
         ),
-        _auth_ui_field(
+        _auth_field(
             TextField,
             "proxy",
             key="PROXY_AUTH_USER_HEADER",
@@ -111,7 +152,7 @@ def security_settings():
             placeholder="e.g. X-Auth-User",
             default="X-Auth-User",
         ),
-        _auth_ui_field(
+        _auth_field(
             TextField,
             "proxy",
             key="PROXY_AUTH_LOGOUT_URL",
@@ -120,7 +161,7 @@ def security_settings():
             placeholder="https://myauth.example.com/logout",
             default="",
         ),
-        _auth_ui_field(
+        _auth_field(
             TextField,
             "proxy",
             key="PROXY_AUTH_ADMIN_GROUP_HEADER",
@@ -129,7 +170,7 @@ def security_settings():
             placeholder="e.g. X-Auth-Groups",
             default="X-Auth-Groups",
         ),
-        _auth_ui_field(
+        _auth_field(
             TextField,
             "proxy",
             key="PROXY_AUTH_ADMIN_GROUP_NAME",
@@ -138,8 +179,30 @@ def security_settings():
             placeholder="e.g. admins",
             default="",
         ),
+        _auth_field(
+            SelectField,
+            "proxy",
+            key="PROXY_AUTH_DEFAULT_ROLE",
+            label="Proxy Auth Default Role",
+            description="Role for users the proxy authenticates for the first time when no admin group is configured. The first account is always an admin so the instance is never left without one.",
+            options=[
+                {"value": "user", "label": "User"},
+                {"value": "admin", "label": "Admin"},
+            ],
+            default="user",
+        ),
     ]
 
+    fields.append(
+        CustomComponentField(
+            key="oidc_callback_url",
+            component="settings_label",
+            label="Callback URL",
+            description="{origin}/api/auth/oidc/callback",
+            show_when=_auth_condition("oidc"),
+        )
+    )
+
     oidc_specs = [
         (
             TextField,
@@ -228,7 +291,7 @@ def security_settings():
             },
         ),
     ]
-    fields.extend(_auth_ui_field(factory, "oidc", **spec) for factory, spec in oidc_specs)
+    fields.extend(_auth_field(factory, "oidc", **spec) for factory, spec in oidc_specs)
     fields.append(
         ActionButton(
             key="test_oidc",
@@ -239,6 +302,16 @@ def security_settings():
             show_when=_auth_condition("oidc"),
         )
     )
+    fields.append(
+        CustomComponentField(
+            key="oidc_env_info",
+            component="oidc_env_info",
+            label="Environment-Only Options",
+            description="These options can only be set via environment variables because changing them through the UI could lock you out of the application.",
+            wrap_in_field_wrapper=True,
+            show_when=_auth_condition("oidc"),
+        )
+    )
     return fields
 
 

shelfmark/config/security_handlers.py

@@ -1,20 +1,56 @@
 """Operational handlers for security settings (save/actions)."""
 
 import os
-from typing import Any, Callable
+from pathlib import Path
+from typing import TYPE_CHECKING, Any
 
-from shelfmark.core.utils import normalize_http_url
+from shelfmark.config.env import DISABLE_LOCAL_AUTH
 from shelfmark.core.user_db import UserDB
+from shelfmark.core.utils import normalize_http_url
+from shelfmark.download.network import get_ssl_verify
 
+if TYPE_CHECKING:
+    from collections.abc import Callable
 
-_OIDC_LOCKOUT_MESSAGE = "Create a local admin account first (Users tab) before enabling OIDC. This ensures you can still log in with a password if SSO is unavailable."
+_OIDC_LOCKOUT_MESSAGE = "A local admin account with a password is required before enabling OIDC. Use the 'Go to Users' button above to create one. This ensures you can still sign in if your identity provider is unavailable."
+_OIDC_REQUIRED_FIELDS = (
+    ("OIDC_DISCOVERY_URL", "Discovery URL"),
+    ("OIDC_CLIENT_ID", "Client ID"),
+    ("OIDC_CLIENT_SECRET", "Client Secret"),
+)
 
 
 def _has_local_password_admin() -> bool:
     root = os.environ.get("CONFIG_DIR", "/config")
-    user_db = UserDB(os.path.join(root, "users.db"))
+    user_db = UserDB(str(Path(root) / "users.db"))
     user_db.initialize()
-    return any(user.get("password_hash") and user.get("role") == "admin" for user in user_db.list_users())
+    return any(
+        user.get("password_hash") and user.get("role") == "admin" for user in user_db.list_users()
+    )
+
+
+def _load_effective_security_values(values: dict[str, Any]) -> dict[str, Any]:
+    """Merge the current save payload onto the persisted security config."""
+    from shelfmark.core.settings_registry import load_config_file
+
+    effective_values = load_config_file("security")
+    effective_values.update(values)
+    return effective_values
+
+
+def _get_missing_oidc_required_fields(effective_values: dict[str, Any]) -> list[str]:
+    """Return missing required OIDC field labels from the effective config."""
+    missing_fields: list[str] = []
+
+    for key, label in _OIDC_REQUIRED_FIELDS:
+        value = effective_values.get(key)
+        if value is None:
+            missing_fields.append(label)
+            continue
+        if isinstance(value, str) and not value.strip():
+            missing_fields.append(label)
+
+    return missing_fields
 
 
 def on_save_security(
@@ -28,6 +64,7 @@ def on_save_security(
         normalized_values["OIDC_DISCOVERY_URL"] = normalize_http_url(
             str(discovery_url),
             default_scheme="https",
+            strip_trailing_slash=False,
         )
 
     proxy_logout_url = normalized_values.get("PROXY_AUTH_LOGOUT_URL")
@@ -38,35 +75,73 @@ def on_save_security(
             strip_trailing_slash=False,
         )
 
-    if normalized_values.get("AUTH_METHOD") == "oidc" and not _has_local_password_admin():
-        return {"error": True, "message": _OIDC_LOCKOUT_MESSAGE, "values": normalized_values}
+    effective_values = _load_effective_security_values(normalized_values)
+    auth_method = str(effective_values.get("AUTH_METHOD", "") or "").strip().lower()
+
+    if auth_method == "oidc":
+        if not DISABLE_LOCAL_AUTH and not _has_local_password_admin():
+            return {"error": True, "message": _OIDC_LOCKOUT_MESSAGE, "values": normalized_values}
+
+        missing_fields = _get_missing_oidc_required_fields(effective_values)
+        if missing_fields:
+            missing_fields_text = ", ".join(missing_fields)
+            return {
+                "error": True,
+                "message": f"OIDC configuration is incomplete: missing {missing_fields_text}.",
+                "values": normalized_values,
+            }
 
     return {"error": False, "values": normalized_values}
 
 
-def test_oidc_connection(
+def check_oidc_connection(
     *,
     load_security_config: Callable[[], dict[str, Any]],
+    current_values: dict[str, Any] | None = None,
     logger: Any,
 ) -> dict[str, Any]:
     """Fetch and validate the configured OIDC discovery document."""
     import requests
 
     try:
-        discovery_url = load_security_config().get("OIDC_DISCOVERY_URL", "")
+        # Prefer the current (unsaved) form value over the saved config
+        discovery_url = (current_values or {}).get(
+            "OIDC_DISCOVERY_URL"
+        ) or load_security_config().get("OIDC_DISCOVERY_URL", "")
         if not discovery_url:
             return {"success": False, "message": "Discovery URL is not configured."}
 
-        response = requests.get(discovery_url, timeout=10)
+        response = requests.get(discovery_url, timeout=10, verify=get_ssl_verify(discovery_url))
         response.raise_for_status()
         document = response.json()
 
-        required_fields = ["issuer", "authorization_endpoint", "token_endpoint"]
+        required_fields = ["issuer", "authorization_endpoint", "token_endpoint", "jwks_uri"]
         missing_fields = [field for field in required_fields if field not in document]
         if missing_fields:
-            return {"success": False, "message": f"Discovery document missing fields: {', '.join(missing_fields)}"}
+            return {
+                "success": False,
+                "message": f"Discovery document missing fields: {', '.join(missing_fields)}",
+            }
+
+        # Logins verify the ID token against the provider's JWKS, so an empty key
+        # set (e.g. an Authentik provider with no Signing Key selected) means every
+        # login will fail even though discovery looks healthy.
+        jwks_uri = str(document["jwks_uri"])
+        jwks_response = requests.get(jwks_uri, timeout=10, verify=get_ssl_verify(jwks_uri))
+        jwks_response.raise_for_status()
+        jwks_document = jwks_response.json()
+        jwks_keys = jwks_document.get("keys") if isinstance(jwks_document, dict) else None
+        if not jwks_keys:
+            return {
+                "success": False,
+                "message": (
+                    "Discovery document is valid, but the provider returned no token "
+                    "signing keys (empty JWKS), so logins will fail. If you use "
+                    "Authentik, select a Signing Key in the provider settings."
+                ),
+            }
 
         return {"success": True, "message": f"Connected to {document['issuer']}"}
     except Exception as exc:
-        logger.error(f"OIDC connection test failed: {exc}")
-        return {"success": False, "message": f"Connection failed: {str(exc)}"}
+        logger.exception("OIDC connection test failed")
+        return {"success": False, "message": f"Connection failed: {exc!s}"}

shelfmark/config/users_settings.py

@@ -5,6 +5,14 @@ The actual user management is handled by a custom frontend component
 that talks to /api/admin/users endpoints.
 """
 
+from typing import Any
+
+from shelfmark.core.languages import normalize_language
+from shelfmark.core.request_policy import (
+    get_source_content_type_capabilities,
+    parse_policy_mode,
+    validate_policy_rules,
+)
 from shelfmark.core.settings_registry import (
     CheckboxField,
     CustomComponentField,
@@ -12,16 +20,11 @@ from shelfmark.core.settings_registry import (
     MultiSelectField,
     NumberField,
     SelectField,
+    SettingsField,
     TableField,
     register_on_save,
     register_settings,
 )
-from shelfmark.core.request_policy import (
-    get_source_content_type_capabilities,
-    parse_policy_mode,
-    validate_policy_rules,
-)
-
 
 _REQUEST_DEFAULT_MODE_OPTIONS = [
     {
@@ -56,6 +59,11 @@ _SELF_SETTINGS_SECTION_OPTIONS = [
         "label": "Delivery Preferences",
         "description": "Show personal delivery output and destination settings.",
     },
+    {
+        "value": "search",
+        "label": "Search Preferences",
+        "description": "Show personal search mode, language, and provider settings.",
+    },
     {
         "value": "notifications",
         "label": "Notifications",
@@ -64,6 +72,21 @@ _SELF_SETTINGS_SECTION_OPTIONS = [
 ]
 _SELF_SETTINGS_SECTION_VALUES = {option["value"] for option in _SELF_SETTINGS_SECTION_OPTIONS}
 _SELF_SETTINGS_SECTION_DEFAULTS = [option["value"] for option in _SELF_SETTINGS_SECTION_OPTIONS]
+_SEARCH_MODE_VALUES = {"direct", "universal"}
+_SEARCH_PREFERENCE_PROVIDER_KEYS = {
+    "METADATA_PROVIDER",
+    "METADATA_PROVIDER_AUDIOBOOK",
+    "METADATA_PROVIDER_COMBINED",
+}
+SEARCH_PREFERENCE_VALIDATABLE_KEYS = {
+    "SEARCH_MODE",
+    "BOOK_LANGUAGE",
+    "DEFAULT_RELEASE_SOURCE",
+    "DEFAULT_RELEASE_SOURCE_AUDIOBOOK",
+    "SHOW_COMBINED_SELECTOR",
+    "FORCE_COMBINED_SEARCH",
+    *_SEARCH_PREFERENCE_PROVIDER_KEYS,
+}
 
 _USERS_HEADING_DESCRIPTION_BY_AUTH_MODE = {
     "builtin": (
@@ -88,35 +111,45 @@ _USERS_HEADING_DESCRIPTION_BY_AUTH_MODE = {
 }
 
 
-def _get_request_source_options():
+def _get_request_source_options() -> list[dict[str, str]]:
     """Build request-policy source options from registered release sources."""
     from shelfmark.release_sources import list_available_sources
 
-    options = []
+    return [
+        {
+            "value": source["name"],
+            "label": source["display_name"],
+        }
+        for source in list_available_sources()
+    ]
+
+
+def _get_valid_release_source_names_for_content_type(content_type: str) -> set[str]:
+    """Return registered release source names that support the requested content type."""
+    from shelfmark.release_sources import list_available_sources
+
+    valid_sources: set[str] = set()
     for source in list_available_sources():
-        options.append(
-            {
-                "value": source["name"],
-                "label": source["display_name"],
-            }
-        )
-    return options
+        supported_types = source.get("supported_content_types", ["ebook", "audiobook"])
+        if content_type in supported_types:
+            valid_sources.add(source["name"])
+    return valid_sources
 
 
-def _get_request_policy_rule_columns():
+def _get_request_policy_rule_columns() -> list[dict[str, object]]:
     source_capabilities = get_source_content_type_capabilities()
     content_type_options = []
 
     for source_name, supported_types in source_capabilities.items():
         normalized_types = [t for t in ("ebook", "audiobook") if t in supported_types]
-        for content_type in normalized_types:
-            content_type_options.append(
-                {
-                    "value": content_type,
-                    "label": "Ebook" if content_type == "ebook" else "Audiobook",
-                    "childOf": source_name,
-                }
-            )
+        content_type_options.extend(
+            {
+                "value": content_type,
+                "label": "Ebook" if content_type == "ebook" else "Audiobook",
+                "childOf": source_name,
+            }
+            for content_type in normalized_types
+        )
 
     return [
         {
@@ -147,7 +180,90 @@ def _get_request_policy_rule_columns():
     ]
 
 
-def _on_save_users(values):
+def _validate_book_languages(value: Any) -> tuple[Any, str | None]:
+    """Validate a per-user default language list against the known languages.
+
+    Accepts the list the settings UI sends as well as a comma-separated string, so an
+    API client can spell the value the way the env var does. Blank entries are skipped
+    rather than rejected, which makes "" and "en," mean the same as [] and ["en"]. An
+    empty result is a deliberate override meaning "no default language filter", so it
+    is kept as-is; ``None`` clears the override further up the chain.
+    """
+    entries = value.split(",") if isinstance(value, str) else value
+    if not isinstance(entries, (list, tuple)):
+        return value, "BOOK_LANGUAGE must be a list of language codes"
+
+    normalized: list[str] = []
+    for entry in entries:
+        if entry is None or (isinstance(entry, str) and not entry.strip()):
+            continue
+        code = normalize_language(entry)
+        if code is None:
+            return value, f"BOOK_LANGUAGE contains an unsupported language: {entry}"
+        if code not in normalized:
+            normalized.append(code)
+
+    return normalized, None
+
+
+def validate_search_preference_value(key: str, value: Any) -> tuple[Any, str | None]:
+    """Validate and normalize a search preference value for user overrides."""
+    if key not in SEARCH_PREFERENCE_VALIDATABLE_KEYS:
+        return value, None
+
+    if value is None:
+        return None, None
+
+    if key == "BOOK_LANGUAGE":
+        return _validate_book_languages(value)
+
+    normalized_value = str(value).strip()
+
+    if key == "SEARCH_MODE":
+        normalized_mode = normalized_value.lower()
+        if normalized_mode not in _SEARCH_MODE_VALUES:
+            return value, "SEARCH_MODE must be 'direct' or 'universal'"
+        return normalized_mode, None
+
+    if key in _SEARCH_PREFERENCE_PROVIDER_KEYS:
+        if normalized_value == "":
+            return "", None
+        from shelfmark.metadata_providers import is_provider_registered
+
+        if not is_provider_registered(normalized_value):
+            return (
+                value,
+                f"{key} must be a valid metadata provider name or empty",
+            )
+        return normalized_value, None
+
+    if key in {"DEFAULT_RELEASE_SOURCE", "DEFAULT_RELEASE_SOURCE_AUDIOBOOK"}:
+        if normalized_value == "":
+            return "", None
+        valid_sources = _get_valid_release_source_names_for_content_type(
+            "audiobook" if key == "DEFAULT_RELEASE_SOURCE_AUDIOBOOK" else "ebook"
+        )
+        if normalized_value not in valid_sources:
+            return (
+                value,
+                f"{key} must be a valid release source name or empty",
+            )
+        return normalized_value, None
+
+    if key == "SHOW_COMBINED_SELECTOR":
+        if isinstance(value, bool):
+            return value, None
+        return bool(value), None
+
+    if key == "FORCE_COMBINED_SEARCH":
+        if isinstance(value, bool):
+            return value, None
+        return bool(value), None
+
+    return value, None
+
+
+def _on_save_users(values: dict[str, object]) -> dict[str, object]:
     """Validate users/request-policy settings before persistence."""
     if "VISIBLE_SELF_SETTINGS_SECTIONS" in values:
         raw_sections = values["VISIBLE_SELF_SETTINGS_SECTIONS"]
@@ -156,7 +272,9 @@ def _on_save_users(values):
         elif isinstance(raw_sections, str):
             candidate_sections = [s.strip() for s in raw_sections.split(",") if s.strip()]
         elif isinstance(raw_sections, (list, tuple, set)):
-            candidate_sections = [str(section).strip() for section in raw_sections if str(section).strip()]
+            candidate_sections = [
+                str(section).strip() for section in raw_sections if str(section).strip()
+            ]
         else:
             return {
                 "error": True,
@@ -181,21 +299,25 @@ def _on_save_users(values):
 
         values["VISIBLE_SELF_SETTINGS_SECTIONS"] = normalized_sections
 
-    if "REQUEST_POLICY_DEFAULT_EBOOK" in values:
-        if parse_policy_mode(values["REQUEST_POLICY_DEFAULT_EBOOK"]) is None:
-            return {
-                "error": True,
-                "message": "REQUEST_POLICY_DEFAULT_EBOOK must be a valid policy mode",
-                "values": values,
-            }
+    if (
+        "REQUEST_POLICY_DEFAULT_EBOOK" in values
+        and parse_policy_mode(values["REQUEST_POLICY_DEFAULT_EBOOK"]) is None
+    ):
+        return {
+            "error": True,
+            "message": "REQUEST_POLICY_DEFAULT_EBOOK must be a valid policy mode",
+            "values": values,
+        }
 
-    if "REQUEST_POLICY_DEFAULT_AUDIOBOOK" in values:
-        if parse_policy_mode(values["REQUEST_POLICY_DEFAULT_AUDIOBOOK"]) is None:
-            return {
-                "error": True,
-                "message": "REQUEST_POLICY_DEFAULT_AUDIOBOOK must be a valid policy mode",
-                "values": values,
-            }
+    if (
+        "REQUEST_POLICY_DEFAULT_AUDIOBOOK" in values
+        and parse_policy_mode(values["REQUEST_POLICY_DEFAULT_AUDIOBOOK"]) is None
+    ):
+        return {
+            "error": True,
+            "message": "REQUEST_POLICY_DEFAULT_AUDIOBOOK must be a valid policy mode",
+            "values": values,
+        }
 
     if "REQUEST_POLICY_RULES" in values:
         normalized_rules, errors = validate_policy_rules(values["REQUEST_POLICY_RULES"])
@@ -207,6 +329,18 @@ def _on_save_users(values):
             }
         values["REQUEST_POLICY_RULES"] = normalized_rules
 
+    for key in SEARCH_PREFERENCE_VALIDATABLE_KEYS:
+        if key not in values:
+            continue
+        normalized_value, validation_error = validate_search_preference_value(key, values[key])
+        if validation_error:
+            return {
+                "error": True,
+                "message": validation_error,
+                "values": values,
+            }
+        values[key] = normalized_value
+
     return {"error": False, "values": values}
 
 
@@ -214,7 +348,7 @@ register_on_save("users", _on_save_users)
 
 
 @register_settings("users", "Users & Requests", icon="users", order=6)
-def users_settings():
+def users_settings() -> list[SettingsField]:
     """User management tab - rendered as a custom component on the frontend."""
     return [
         HeadingField(
@@ -241,9 +375,7 @@ def users_settings():
         HeadingField(
             key="requests_heading",
             title="Requests",
-            description=(
-                "Choose what users can download directly and what needs approval first."
-            ),
+            description=("Choose what users can download directly and what needs approval first."),
         ),
         CheckboxField(
             key="REQUESTS_ENABLED",
@@ -267,9 +399,7 @@ def users_settings():
                 SelectField(
                     key="REQUEST_POLICY_DEFAULT_EBOOK",
                     label="Default Ebook Mode",
-                    description=(
-                        "Sets the baseline for all ebook sources."
-                    ),
+                    description=("Sets the baseline for all ebook sources."),
                     options=_REQUEST_DEFAULT_MODE_OPTIONS,
                     default="download",
                     user_overridable=True,
@@ -277,9 +407,7 @@ def users_settings():
                 SelectField(
                     key="REQUEST_POLICY_DEFAULT_AUDIOBOOK",
                     label="Default Audiobook Mode",
-                    description=(
-                        "Sets the baseline for all audiobook sources."
-                    ),
+                    description=("Sets the baseline for all audiobook sources."),
                     options=_REQUEST_DEFAULT_MODE_OPTIONS,
                     default="download",
                     user_overridable=True,

shelfmark/core/__init__.py

@@ -1,5 +1,14 @@
 """Core module - shared models, queue, and utilities."""
 
-from shelfmark.core.models import BookInfo, QueueItem, SearchFilters, QueueStatus
-from shelfmark.core.queue import BookQueue, book_queue
 from shelfmark.core.logger import setup_logger
+from shelfmark.core.models import QueueItem, QueueStatus, SearchFilters
+from shelfmark.core.queue import BookQueue, book_queue
+
+__all__ = [
+    "BookQueue",
+    "QueueItem",
+    "QueueStatus",
+    "SearchFilters",
+    "book_queue",
+    "setup_logger",
+]

shelfmark/core/activity_service.py

@@ -1,618 +0,0 @@
-"""Persistence helpers for Activity dismissals and terminal snapshots."""
-
-from __future__ import annotations
-
-from datetime import datetime, timezone
-import json
-import sqlite3
-from typing import Any, Iterable
-
-
-VALID_ITEM_TYPES = frozenset({"download", "request"})
-VALID_ORIGINS = frozenset({"direct", "request", "requested"})
-VALID_FINAL_STATUSES = frozenset({"complete", "error", "cancelled", "rejected"})
-
-
-def _now_timestamp() -> str:
-    return datetime.now(timezone.utc).isoformat(timespec="seconds")
-
-
-def _normalize_item_type(item_type: Any) -> str:
-    if not isinstance(item_type, str):
-        raise ValueError("item_type must be a string")
-    normalized = item_type.strip().lower()
-    if normalized not in VALID_ITEM_TYPES:
-        raise ValueError("item_type must be one of: download, request")
-    return normalized
-
-
-def _normalize_item_key(item_key: Any) -> str:
-    if not isinstance(item_key, str):
-        raise ValueError("item_key must be a string")
-    normalized = item_key.strip()
-    if not normalized:
-        raise ValueError("item_key must not be empty")
-    return normalized
-
-
-def _normalize_origin(origin: Any) -> str:
-    if not isinstance(origin, str):
-        raise ValueError("origin must be a string")
-    normalized = origin.strip().lower()
-    if normalized not in VALID_ORIGINS:
-        raise ValueError("origin must be one of: direct, request, requested")
-    return normalized
-
-
-def _normalize_final_status(final_status: Any) -> str:
-    if not isinstance(final_status, str):
-        raise ValueError("final_status must be a string")
-    normalized = final_status.strip().lower()
-    if normalized not in VALID_FINAL_STATUSES:
-        raise ValueError("final_status must be one of: complete, error, cancelled, rejected")
-    return normalized
-
-
-def build_item_key(item_type: str, raw_id: Any) -> str:
-    """Build a stable item key used by dismiss/history APIs."""
-    normalized_type = _normalize_item_type(item_type)
-    if normalized_type == "request":
-        try:
-            request_id = int(raw_id)
-        except (TypeError, ValueError) as exc:
-            raise ValueError("request item IDs must be integers") from exc
-        if request_id < 1:
-            raise ValueError("request item IDs must be positive integers")
-        return f"request:{request_id}"
-
-    if not isinstance(raw_id, str):
-        raise ValueError("download item IDs must be strings")
-    task_id = raw_id.strip()
-    if not task_id:
-        raise ValueError("download item IDs must not be empty")
-    return f"download:{task_id}"
-
-
-def build_request_item_key(request_id: int) -> str:
-    """Build a request item key."""
-    return build_item_key("request", request_id)
-
-
-def build_download_item_key(task_id: str) -> str:
-    """Build a download item key."""
-    return build_item_key("download", task_id)
-
-
-def _parse_request_id_from_item_key(item_key: Any) -> int | None:
-    if not isinstance(item_key, str) or not item_key.startswith("request:"):
-        return None
-    raw_value = item_key.split(":", 1)[1].strip()
-    try:
-        parsed = int(raw_value)
-    except (TypeError, ValueError):
-        return None
-    return parsed if parsed > 0 else None
-
-
-def _request_final_status(request_status: Any, delivery_state: Any) -> str | None:
-    status = str(request_status or "").strip().lower()
-    if status == "pending":
-        return None
-    if status == "rejected":
-        return "rejected"
-    if status == "cancelled":
-        return "cancelled"
-    if status != "fulfilled":
-        return None
-
-    delivery = str(delivery_state or "").strip().lower()
-    if delivery in {"error", "cancelled"}:
-        return delivery
-    return "complete"
-
-
-class ActivityService:
-    """Service for per-user activity dismissals and terminal history snapshots."""
-
-    def __init__(self, db_path: str):
-        self._db_path = db_path
-
-    def _connect(self) -> sqlite3.Connection:
-        conn = sqlite3.connect(self._db_path)
-        conn.row_factory = sqlite3.Row
-        conn.execute("PRAGMA foreign_keys = ON")
-        return conn
-
-    @staticmethod
-    def _coerce_positive_int(value: Any, field: str) -> int:
-        try:
-            parsed = int(value)
-        except (TypeError, ValueError) as exc:
-            raise ValueError(f"{field} must be an integer") from exc
-        if parsed < 1:
-            raise ValueError(f"{field} must be a positive integer")
-        return parsed
-
-    @staticmethod
-    def _row_to_dict(row: sqlite3.Row | None) -> dict[str, Any] | None:
-        return dict(row) if row is not None else None
-
-    @staticmethod
-    def _parse_json_column(value: Any) -> Any:
-        if not isinstance(value, str):
-            return None
-        try:
-            return json.loads(value)
-        except (ValueError, TypeError):
-            return None
-
-    def _build_legacy_request_snapshot(
-        self,
-        conn: sqlite3.Connection,
-        request_id: int,
-    ) -> tuple[dict[str, Any] | None, str | None]:
-        request_row = conn.execute(
-            """
-            SELECT
-                id,
-                user_id,
-                status,
-                delivery_state,
-                request_level,
-                book_data,
-                release_data,
-                note,
-                admin_note,
-                created_at,
-                reviewed_at
-            FROM download_requests
-            WHERE id = ?
-            """,
-            (request_id,),
-        ).fetchone()
-        if request_row is None:
-            return None, None
-
-        row_dict = dict(request_row)
-        book_data = self._parse_json_column(row_dict.get("book_data"))
-        release_data = self._parse_json_column(row_dict.get("release_data"))
-        if not isinstance(book_data, dict):
-            book_data = {}
-        if not isinstance(release_data, dict):
-            release_data = {}
-
-        snapshot = {
-            "kind": "request",
-            "request": {
-                "id": int(row_dict["id"]),
-                "user_id": row_dict.get("user_id"),
-                "status": row_dict.get("status"),
-                "delivery_state": row_dict.get("delivery_state"),
-                "request_level": row_dict.get("request_level"),
-                "book_data": book_data,
-                "release_data": release_data,
-                "note": row_dict.get("note"),
-                "admin_note": row_dict.get("admin_note"),
-                "created_at": row_dict.get("created_at"),
-                "updated_at": row_dict.get("reviewed_at") or row_dict.get("created_at"),
-            },
-        }
-        final_status = _request_final_status(row_dict.get("status"), row_dict.get("delivery_state"))
-        return snapshot, final_status
-
-    def record_terminal_snapshot(
-        self,
-        *,
-        user_id: int | None,
-        item_type: str,
-        item_key: str,
-        origin: str,
-        final_status: str,
-        snapshot: dict[str, Any],
-        request_id: int | None = None,
-        source_id: str | None = None,
-        terminal_at: str | None = None,
-    ) -> dict[str, Any]:
-        """Record a durable terminal-state snapshot for an activity item."""
-        normalized_item_type = _normalize_item_type(item_type)
-        normalized_item_key = _normalize_item_key(item_key)
-        normalized_origin = _normalize_origin(origin)
-        normalized_final_status = _normalize_final_status(final_status)
-        if not isinstance(snapshot, dict):
-            raise ValueError("snapshot must be an object")
-
-        if user_id is not None:
-            user_id = self._coerce_positive_int(user_id, "user_id")
-        if request_id is not None:
-            request_id = self._coerce_positive_int(request_id, "request_id")
-        if source_id is not None and not isinstance(source_id, str):
-            raise ValueError("source_id must be a string when provided")
-        if source_id is not None:
-            source_id = source_id.strip() or None
-
-        effective_terminal_at = terminal_at if isinstance(terminal_at, str) and terminal_at.strip() else _now_timestamp()
-        serialized_snapshot = json.dumps(snapshot, separators=(",", ":"), ensure_ascii=False)
-
-        conn = self._connect()
-        try:
-            cursor = conn.execute(
-                """
-                INSERT INTO activity_log (
-                    user_id,
-                    item_type,
-                    item_key,
-                    request_id,
-                    source_id,
-                    origin,
-                    final_status,
-                    snapshot_json,
-                    terminal_at
-                )
-                VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
-                """,
-                (
-                    user_id,
-                    normalized_item_type,
-                    normalized_item_key,
-                    request_id,
-                    source_id,
-                    normalized_origin,
-                    normalized_final_status,
-                    serialized_snapshot,
-                    effective_terminal_at,
-                ),
-            )
-            snapshot_id = int(cursor.lastrowid)
-            conn.commit()
-            row = conn.execute(
-                "SELECT * FROM activity_log WHERE id = ?",
-                (snapshot_id,),
-            ).fetchone()
-            payload = self._row_to_dict(row)
-            if payload is None:
-                raise ValueError("Failed to read back recorded activity snapshot")
-            return payload
-        finally:
-            conn.close()
-
-    def get_latest_activity_log_id(self, *, item_type: str, item_key: str) -> int | None:
-        """Get the newest snapshot ID for an item key."""
-        normalized_item_type = _normalize_item_type(item_type)
-        normalized_item_key = _normalize_item_key(item_key)
-        conn = self._connect()
-        try:
-            row = conn.execute(
-                """
-                SELECT id
-                FROM activity_log
-                WHERE item_type = ? AND item_key = ?
-                ORDER BY terminal_at DESC, id DESC
-                LIMIT 1
-                """,
-                (normalized_item_type, normalized_item_key),
-            ).fetchone()
-            if row is None:
-                return None
-            return int(row["id"])
-        finally:
-            conn.close()
-
-    def dismiss_item(
-        self,
-        *,
-        user_id: int,
-        item_type: str,
-        item_key: str,
-        activity_log_id: int | None = None,
-    ) -> dict[str, Any]:
-        """Dismiss an item for a specific user (upsert)."""
-        normalized_user_id = self._coerce_positive_int(user_id, "user_id")
-        normalized_item_type = _normalize_item_type(item_type)
-        normalized_item_key = _normalize_item_key(item_key)
-        normalized_log_id = (
-            self._coerce_positive_int(activity_log_id, "activity_log_id")
-            if activity_log_id is not None
-            else self.get_latest_activity_log_id(
-                item_type=normalized_item_type,
-                item_key=normalized_item_key,
-            )
-        )
-
-        conn = self._connect()
-        try:
-            conn.execute(
-                """
-                INSERT INTO activity_dismissals (
-                    user_id,
-                    item_type,
-                    item_key,
-                    activity_log_id,
-                    dismissed_at
-                )
-                VALUES (?, ?, ?, ?, ?)
-                ON CONFLICT(user_id, item_type, item_key)
-                DO UPDATE SET
-                    activity_log_id = excluded.activity_log_id,
-                    dismissed_at = excluded.dismissed_at
-                """,
-                (
-                    normalized_user_id,
-                    normalized_item_type,
-                    normalized_item_key,
-                    normalized_log_id,
-                    _now_timestamp(),
-                ),
-            )
-            conn.commit()
-            row = conn.execute(
-                """
-                SELECT *
-                FROM activity_dismissals
-                WHERE user_id = ? AND item_type = ? AND item_key = ?
-                """,
-                (normalized_user_id, normalized_item_type, normalized_item_key),
-            ).fetchone()
-            payload = self._row_to_dict(row)
-            if payload is None:
-                raise ValueError("Failed to read back dismissal row")
-            return payload
-        finally:
-            conn.close()
-
-    def dismiss_many(self, *, user_id: int, items: Iterable[dict[str, Any]]) -> int:
-        """Dismiss many items for one user."""
-        normalized_user_id = self._coerce_positive_int(user_id, "user_id")
-        normalized_items: list[tuple[str, str, int | None]] = []
-        for item in items:
-            if not isinstance(item, dict):
-                raise ValueError("items must contain objects")
-            normalized_item_type = _normalize_item_type(item.get("item_type"))
-            normalized_item_key = _normalize_item_key(item.get("item_key"))
-            raw_log_id = item.get("activity_log_id")
-            normalized_log_id = (
-                self._coerce_positive_int(raw_log_id, "activity_log_id")
-                if raw_log_id is not None
-                else self.get_latest_activity_log_id(
-                    item_type=normalized_item_type,
-                    item_key=normalized_item_key,
-                )
-            )
-            normalized_items.append((normalized_item_type, normalized_item_key, normalized_log_id))
-
-        if not normalized_items:
-            return 0
-
-        conn = self._connect()
-        try:
-            timestamp = _now_timestamp()
-            for item_type, item_key, activity_log_id in normalized_items:
-                conn.execute(
-                    """
-                    INSERT INTO activity_dismissals (
-                        user_id,
-                        item_type,
-                        item_key,
-                        activity_log_id,
-                        dismissed_at
-                    )
-                    VALUES (?, ?, ?, ?, ?)
-                    ON CONFLICT(user_id, item_type, item_key)
-                    DO UPDATE SET
-                        activity_log_id = excluded.activity_log_id,
-                        dismissed_at = excluded.dismissed_at
-                    """,
-                    (
-                        normalized_user_id,
-                        item_type,
-                        item_key,
-                        activity_log_id,
-                        timestamp,
-                    ),
-                )
-            conn.commit()
-            return len(normalized_items)
-        finally:
-            conn.close()
-
-    def get_dismissal_set(self, user_id: int) -> list[dict[str, str]]:
-        """Return dismissed item keys for one user."""
-        normalized_user_id = self._coerce_positive_int(user_id, "user_id")
-        conn = self._connect()
-        try:
-            rows = conn.execute(
-                """
-                SELECT item_type, item_key
-                FROM activity_dismissals
-                WHERE user_id = ?
-                ORDER BY dismissed_at DESC, id DESC
-                """,
-                (normalized_user_id,),
-            ).fetchall()
-            return [
-                {
-                    "item_type": str(row["item_type"]),
-                    "item_key": str(row["item_key"]),
-                }
-                for row in rows
-            ]
-        finally:
-            conn.close()
-
-    def clear_dismissals_for_item_keys(
-        self,
-        *,
-        user_id: int,
-        item_type: str,
-        item_keys: Iterable[str],
-    ) -> int:
-        """Clear dismissals for one user + item type + item keys."""
-        normalized_user_id = self._coerce_positive_int(user_id, "user_id")
-        normalized_item_type = _normalize_item_type(item_type)
-        normalized_keys = {
-            _normalize_item_key(item_key)
-            for item_key in item_keys
-            if isinstance(item_key, str) and item_key.strip()
-        }
-        if not normalized_keys:
-            return 0
-
-        conn = self._connect()
-        try:
-            cursor = conn.executemany(
-                """
-                DELETE FROM activity_dismissals
-                WHERE user_id = ? AND item_type = ? AND item_key = ?
-                """,
-                (
-                    (normalized_user_id, normalized_item_type, item_key)
-                    for item_key in normalized_keys
-                ),
-            )
-            conn.commit()
-            return int(cursor.rowcount or 0)
-        finally:
-            conn.close()
-
-    def get_history(self, user_id: int, *, limit: int = 50, offset: int = 0) -> list[dict[str, Any]]:
-        """Return paged dismissal history for one user."""
-        normalized_user_id = self._coerce_positive_int(user_id, "user_id")
-        normalized_limit = max(1, min(int(limit), 200))
-        normalized_offset = max(0, int(offset))
-
-        conn = self._connect()
-        try:
-            rows = conn.execute(
-                """
-                SELECT
-                    d.id,
-                    d.user_id,
-                    d.item_type,
-                    d.item_key,
-                    d.activity_log_id,
-                    d.dismissed_at,
-                    l.snapshot_json,
-                    l.origin,
-                    l.final_status,
-                    l.terminal_at,
-                    l.request_id,
-                    l.source_id
-                FROM activity_dismissals d
-                LEFT JOIN activity_log l ON l.id = d.activity_log_id
-                WHERE d.user_id = ?
-                ORDER BY d.dismissed_at DESC, d.id DESC
-                LIMIT ? OFFSET ?
-                """,
-                (normalized_user_id, normalized_limit, normalized_offset),
-            ).fetchall()
-
-            payload: list[dict[str, Any]] = []
-            for row in rows:
-                row_dict = dict(row)
-                raw_snapshot_json = row_dict.pop("snapshot_json", None)
-                snapshot_payload = None
-                if isinstance(raw_snapshot_json, str):
-                    try:
-                        snapshot_payload = json.loads(raw_snapshot_json)
-                    except (ValueError, TypeError):
-                        snapshot_payload = None
-
-                if snapshot_payload is None and row_dict.get("item_type") == "request":
-                    request_id = row_dict.get("request_id")
-                    if request_id is None:
-                        request_id = _parse_request_id_from_item_key(row_dict.get("item_key"))
-                    try:
-                        normalized_request_id = int(request_id) if request_id is not None else None
-                    except (TypeError, ValueError):
-                        normalized_request_id = None
-
-                    if normalized_request_id and normalized_request_id > 0:
-                        fallback_snapshot, fallback_final_status = self._build_legacy_request_snapshot(
-                            conn,
-                            normalized_request_id,
-                        )
-                        if fallback_snapshot is not None:
-                            snapshot_payload = fallback_snapshot
-                            if not row_dict.get("origin"):
-                                row_dict["origin"] = "request"
-                            if not row_dict.get("final_status") and fallback_final_status is not None:
-                                row_dict["final_status"] = fallback_final_status
-
-                row_dict["snapshot"] = snapshot_payload
-                payload.append(row_dict)
-            return payload
-        finally:
-            conn.close()
-
-    def get_undismissed_terminal_downloads(self, user_id: int, *, limit: int = 200) -> list[dict[str, Any]]:
-        """Return latest undismissed terminal download snapshots for one user."""
-        normalized_user_id = self._coerce_positive_int(user_id, "user_id")
-        normalized_limit = max(1, min(int(limit), 500))
-
-        conn = self._connect()
-        try:
-            rows = conn.execute(
-                """
-                SELECT
-                    l.id,
-                    l.user_id,
-                    l.item_type,
-                    l.item_key,
-                    l.request_id,
-                    l.source_id,
-                    l.origin,
-                    l.final_status,
-                    l.snapshot_json,
-                    l.terminal_at
-                FROM activity_log l
-                LEFT JOIN activity_dismissals d
-                    ON d.user_id = ?
-                    AND d.item_type = l.item_type
-                    AND d.item_key = l.item_key
-                WHERE l.user_id = ?
-                    AND l.item_type = 'download'
-                    AND l.final_status IN ('complete', 'error', 'cancelled')
-                    AND d.id IS NULL
-                ORDER BY l.terminal_at DESC, l.id DESC
-                LIMIT ?
-                """,
-                (normalized_user_id, normalized_user_id, normalized_limit * 2),
-            ).fetchall()
-
-            payload: list[dict[str, Any]] = []
-            seen_item_keys: set[str] = set()
-            for row in rows:
-                row_dict = dict(row)
-                item_key = str(row_dict.get("item_key") or "")
-                if not item_key or item_key in seen_item_keys:
-                    continue
-                seen_item_keys.add(item_key)
-
-                raw_snapshot_json = row_dict.pop("snapshot_json", None)
-                snapshot_payload = None
-                if isinstance(raw_snapshot_json, str):
-                    try:
-                        snapshot_payload = json.loads(raw_snapshot_json)
-                    except (ValueError, TypeError):
-                        snapshot_payload = None
-                row_dict["snapshot"] = snapshot_payload
-                payload.append(row_dict)
-                if len(payload) >= normalized_limit:
-                    break
-
-            return payload
-        finally:
-            conn.close()
-
-    def clear_history(self, user_id: int) -> int:
-        """Delete all dismissals for a user and return deleted row count."""
-        normalized_user_id = self._coerce_positive_int(user_id, "user_id")
-        conn = self._connect()
-        try:
-            cursor = conn.execute(
-                "DELETE FROM activity_dismissals WHERE user_id = ?",
-                (normalized_user_id,),
-            )
-            conn.commit()
-            return int(cursor.rowcount or 0)
-        finally:
-            conn.close()

shelfmark/core/activity_view_state_service.py

@@ -0,0 +1,327 @@
+"""Persistence helpers for per-viewer activity visibility state."""
+
+from __future__ import annotations
+
+import sqlite3
+import threading
+from typing import Any
+
+from shelfmark.core.request_helpers import now_utc_iso
+
+VALID_ACTIVITY_ITEM_TYPES = frozenset({"download", "request"})
+ADMIN_VIEWER_SCOPE = "admin:shared"
+NOAUTH_VIEWER_SCOPE = "noauth:shared"
+USER_VIEWER_SCOPE_PREFIX = "user:"
+
+
+def user_viewer_scope(user_id: int) -> str:
+    """Build the persisted viewer scope string for a specific user."""
+    if not isinstance(user_id, int) or user_id < 1:
+        msg = "user_id must be a positive integer"
+        raise ValueError(msg)
+    return f"{USER_VIEWER_SCOPE_PREFIX}{user_id}"
+
+
+def normalize_viewer_scope(viewer_scope: object) -> str:
+    """Validate and normalize a persisted viewer scope string."""
+    if not isinstance(viewer_scope, str) or not viewer_scope.strip():
+        msg = "viewer_scope must be a non-empty string"
+        raise ValueError(msg)
+
+    normalized = viewer_scope.strip()
+    if normalized in {ADMIN_VIEWER_SCOPE, NOAUTH_VIEWER_SCOPE}:
+        return normalized
+
+    if not normalized.startswith(USER_VIEWER_SCOPE_PREFIX):
+        msg = "viewer_scope must be one of: admin:shared, noauth:shared, or user:<id>"
+        raise ValueError(msg)
+
+    raw_user_id = normalized[len(USER_VIEWER_SCOPE_PREFIX) :].strip()
+    try:
+        parsed_user_id = int(raw_user_id)
+    except (TypeError, ValueError) as exc:
+        msg = "viewer_scope user id must be a positive integer"
+        raise ValueError(msg) from exc
+
+    return user_viewer_scope(parsed_user_id)
+
+
+def _normalize_item_type(item_type: object) -> str:
+    if not isinstance(item_type, str) or not item_type.strip():
+        msg = "item_type must be a non-empty string"
+        raise ValueError(msg)
+    normalized = item_type.strip().lower()
+    if normalized not in VALID_ACTIVITY_ITEM_TYPES:
+        msg = "item_type must be one of: download, request"
+        raise ValueError(msg)
+    return normalized
+
+
+def _normalize_item_key(item_key: object, *, item_type: str) -> str:
+    if not isinstance(item_key, str) or not item_key.strip():
+        msg = "item_key must be a non-empty string"
+        raise ValueError(msg)
+
+    normalized = item_key.strip()
+    expected_prefix = f"{item_type}:"
+    if not normalized.startswith(expected_prefix):
+        msg_0 = f"item_key must be in the format {expected_prefix}<id>"
+        raise ValueError(msg_0)
+    if not normalized.split(":", 1)[1].strip():
+        msg_0 = f"item_key must be in the format {expected_prefix}<id>"
+        raise ValueError(msg_0)
+    return normalized
+
+
+class ActivityViewStateService:
+    """Service for per-viewer activity dismissal and history visibility."""
+
+    def __init__(self, db_path: str) -> None:
+        """Initialize the service with the SQLite state database path."""
+        self._db_path = db_path
+        self._lock = threading.Lock()
+
+    def _connect(self) -> sqlite3.Connection:
+        conn = sqlite3.connect(self._db_path)
+        conn.row_factory = sqlite3.Row
+        conn.execute("PRAGMA foreign_keys = ON")
+        return conn
+
+    def list_hidden(
+        self,
+        *,
+        viewer_scope: str,
+        limit: int | None = None,
+    ) -> list[dict[str, Any]]:
+        """Return dismissed rows for a viewer, including cleared history entries."""
+        normalized_scope = normalize_viewer_scope(viewer_scope)
+        normalized_limit = None if limit is None else max(1, int(limit))
+        query = """
+            SELECT item_type, item_key, dismissed_at, cleared_at
+            FROM activity_view_state
+            WHERE viewer_scope = ?
+              AND dismissed_at IS NOT NULL
+            ORDER BY COALESCE(cleared_at, dismissed_at) DESC, id DESC
+        """
+        params: list[Any] = [normalized_scope]
+        if normalized_limit is not None:
+            query += "\nLIMIT ?"
+            params.append(normalized_limit)
+
+        conn = self._connect()
+        try:
+            rows = conn.execute(query, params).fetchall()
+            return [dict(row) for row in rows]
+        finally:
+            conn.close()
+
+    def list_history(
+        self,
+        *,
+        viewer_scope: str,
+        limit: int = 50,
+        offset: int = 0,
+    ) -> list[dict[str, Any]]:
+        """Return active dismissal history rows for a viewer."""
+        normalized_scope = normalize_viewer_scope(viewer_scope)
+        normalized_limit = max(1, min(int(limit), 5000))
+        normalized_offset = max(0, int(offset))
+
+        conn = self._connect()
+        try:
+            rows = conn.execute(
+                """
+                SELECT item_type, item_key, dismissed_at
+                FROM activity_view_state
+                WHERE viewer_scope = ?
+                  AND dismissed_at IS NOT NULL
+                  AND cleared_at IS NULL
+                ORDER BY dismissed_at DESC, id DESC
+                LIMIT ? OFFSET ?
+                """,
+                (normalized_scope, normalized_limit, normalized_offset),
+            ).fetchall()
+            return [dict(row) for row in rows]
+        finally:
+            conn.close()
+
+    def dismiss(
+        self,
+        *,
+        viewer_scope: str,
+        item_type: str,
+        item_key: str,
+    ) -> int:
+        """Mark a single activity item as dismissed for a viewer."""
+        normalized_scope = normalize_viewer_scope(viewer_scope)
+        normalized_type = _normalize_item_type(item_type)
+        normalized_key = _normalize_item_key(item_key, item_type=normalized_type)
+        dismissed_at = now_utc_iso()
+
+        with self._lock:
+            conn = self._connect()
+            try:
+                cursor = conn.execute(
+                    """
+                    INSERT INTO activity_view_state (
+                        viewer_scope,
+                        item_type,
+                        item_key,
+                        dismissed_at,
+                        cleared_at
+                    )
+                    VALUES (?, ?, ?, ?, NULL)
+                    ON CONFLICT(viewer_scope, item_type, item_key) DO UPDATE SET
+                        dismissed_at = excluded.dismissed_at,
+                        cleared_at = NULL
+                    """,
+                    (normalized_scope, normalized_type, normalized_key, dismissed_at),
+                )
+                conn.commit()
+                rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
+                return max(rowcount, 0)
+            finally:
+                conn.close()
+
+    def dismiss_many(
+        self,
+        *,
+        viewer_scope: str,
+        items: list[dict[str, str]],
+    ) -> int:
+        """Mark multiple activity items as dismissed for a viewer."""
+        normalized_scope = normalize_viewer_scope(viewer_scope)
+        if not items:
+            return 0
+
+        seen: set[tuple[str, str]] = set()
+        normalized_items: list[tuple[str, str]] = []
+        for item in items:
+            normalized_type = _normalize_item_type(item.get("item_type"))
+            normalized_key = _normalize_item_key(item.get("item_key"), item_type=normalized_type)
+            marker = (normalized_type, normalized_key)
+            if marker in seen:
+                continue
+            seen.add(marker)
+            normalized_items.append(marker)
+
+        if not normalized_items:
+            return 0
+
+        dismissed_at = now_utc_iso()
+        with self._lock:
+            conn = self._connect()
+            try:
+                total = 0
+                for normalized_type, normalized_key in normalized_items:
+                    cursor = conn.execute(
+                        """
+                        INSERT INTO activity_view_state (
+                            viewer_scope,
+                            item_type,
+                            item_key,
+                            dismissed_at,
+                            cleared_at
+                        )
+                        VALUES (?, ?, ?, ?, NULL)
+                        ON CONFLICT(viewer_scope, item_type, item_key) DO UPDATE SET
+                            dismissed_at = excluded.dismissed_at,
+                            cleared_at = NULL
+                        """,
+                        (
+                            normalized_scope,
+                            normalized_type,
+                            normalized_key,
+                            dismissed_at,
+                        ),
+                    )
+                    rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
+                    total += max(rowcount, 0)
+                conn.commit()
+                return total
+            finally:
+                conn.close()
+
+    def clear_history(self, *, viewer_scope: str) -> int:
+        """Mark all dismissed items as cleared for a viewer."""
+        normalized_scope = normalize_viewer_scope(viewer_scope)
+        cleared_at = now_utc_iso()
+
+        with self._lock:
+            conn = self._connect()
+            try:
+                cursor = conn.execute(
+                    """
+                    UPDATE activity_view_state
+                    SET cleared_at = ?
+                    WHERE viewer_scope = ?
+                      AND dismissed_at IS NOT NULL
+                      AND cleared_at IS NULL
+                    """,
+                    (cleared_at, normalized_scope),
+                )
+                conn.commit()
+                rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
+                return max(rowcount, 0)
+            finally:
+                conn.close()
+
+    def clear_item_for_all_viewers(self, *, item_type: str, item_key: str) -> int:
+        """Delete a dismissed item record for every viewer."""
+        normalized_type = _normalize_item_type(item_type)
+        normalized_key = _normalize_item_key(item_key, item_type=normalized_type)
+
+        with self._lock:
+            conn = self._connect()
+            try:
+                cursor = conn.execute(
+                    """
+                    DELETE FROM activity_view_state
+                    WHERE item_type = ? AND item_key = ?
+                    """,
+                    (normalized_type, normalized_key),
+                )
+                conn.commit()
+                rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
+                return max(rowcount, 0)
+            finally:
+                conn.close()
+
+    def delete_viewer_scope(self, *, viewer_scope: str) -> int:
+        """Delete all activity-view state rows for a viewer scope."""
+        normalized_scope = normalize_viewer_scope(viewer_scope)
+
+        with self._lock:
+            conn = self._connect()
+            try:
+                cursor = conn.execute(
+                    "DELETE FROM activity_view_state WHERE viewer_scope = ?",
+                    (normalized_scope,),
+                )
+                conn.commit()
+                rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
+                return max(rowcount, 0)
+            finally:
+                conn.close()
+
+    def delete_items(self, *, item_type: str, item_keys: list[str]) -> int:
+        """Delete multiple dismissed item records for a given item type."""
+        normalized_type = _normalize_item_type(item_type)
+        normalized_keys = [
+            _normalize_item_key(item_key, item_type=normalized_type) for item_key in item_keys
+        ]
+        if not normalized_keys:
+            return 0
+
+        with self._lock:
+            conn = self._connect()
+            try:
+                cursor = conn.executemany(
+                    "DELETE FROM activity_view_state WHERE item_type = ? AND item_key = ?",
+                    [(normalized_type, normalized_key) for normalized_key in normalized_keys],
+                )
+                conn.commit()
+                rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
+                return max(rowcount, 0)
+            finally:
+                conn.close()

shelfmark/core/admin_routes.py

@@ -4,12 +4,14 @@ Registers /api/admin/users CRUD endpoints for managing users.
 All endpoints require admin session.
 """
 
-from functools import wraps
+from __future__ import annotations
+
 import os
 import sqlite3
-from typing import Any
+from functools import wraps
+from typing import TYPE_CHECKING, Any, ParamSpec
 
-from flask import Flask, jsonify, request, session
+from flask import Flask, Response, g, jsonify, request, session
 from werkzeug.security import generate_password_hash
 
 from shelfmark.config.booklore_settings import (
@@ -26,16 +28,33 @@ from shelfmark.core.auth_modes import (
     AUTH_SOURCE_CWA,
     AUTH_SOURCE_OIDC,
     AUTH_SOURCE_PROXY,
-    determine_auth_mode,
-    has_local_password_admin,
+    is_user_active_for_auth_mode,
+    load_active_auth_mode,
     normalize_auth_source,
 )
+from shelfmark.core.config import config as app_config
 from shelfmark.core.cwa_user_sync import sync_cwa_users_from_rows
 from shelfmark.core.logger import setup_logger
-from shelfmark.core.settings_registry import load_config_file
-from shelfmark.core.user_db import UserDB
+
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
+    from flask.typing import ResponseReturnValue
+
+    from shelfmark.core.user_db import UserDB
+
+P = ParamSpec("P")
 
 logger = setup_logger(__name__)
+MIN_PASSWORD_LENGTH = 4
+_CONFIG_REFRESH_ERRORS = (ImportError, OSError, RuntimeError, TypeError, ValueError)
+
+__all__ = [
+    "get_booklore_library_options",
+    "get_booklore_path_options",
+    "register_admin_routes",
+    "validate_user_settings",
+]
 
 
 def _get_user_edit_capabilities(
@@ -47,10 +66,12 @@ def _get_user_edit_capabilities(
         user.get("auth_source"),
         user.get("oidc_subject"),
     )
-    if security_config is None and auth_source == AUTH_SOURCE_OIDC:
-        security_config = load_config_file("security")
-
-    oidc_use_admin_group = bool((security_config or {}).get("OIDC_USE_ADMIN_GROUP", True))
+    oidc_use_admin_group = bool(
+        (security_config or {}).get(
+            "OIDC_USE_ADMIN_GROUP",
+            app_config.get("OIDC_USE_ADMIN_GROUP", True),
+        )
+    )
     role_managed_by_oidc_group = auth_source == AUTH_SOURCE_OIDC and oidc_use_admin_group
     can_edit_role = auth_source == AUTH_SOURCE_BUILTIN or (
         auth_source == AUTH_SOURCE_OIDC and not role_managed_by_oidc_group
@@ -65,37 +86,6 @@ def _get_user_edit_capabilities(
     }
 
 
-def _get_auth_mode():
-    """Get current auth mode from config."""
-    try:
-        config = load_config_file("security")
-        return determine_auth_mode(
-            config,
-            CWA_DB_PATH,
-            has_local_admin=has_local_password_admin(),
-        )
-    except Exception:
-        return "none"
-
-
-def _require_admin(f):
-    """Decorator to require admin session for admin routes.
-
-    In no-auth mode, everyone has access (is_admin defaults True).
-    In auth-required modes, requires an authenticated session with admin role.
-    """
-    @wraps(f)
-    def decorated(*args, **kwargs):
-        auth_mode = _get_auth_mode()
-        if auth_mode != "none":
-            if "user_id" not in session:
-                return jsonify({"error": "Authentication required"}), 401
-            if not session.get("is_admin", False):
-                return jsonify({"error": "Admin access required"}), 403
-        return f(*args, **kwargs)
-    return decorated
-
-
 def _sanitize_user(user: dict) -> dict:
     """Remove sensitive fields from user dict before returning to client."""
     sanitized = dict(user)
@@ -103,27 +93,21 @@ def _sanitize_user(user: dict) -> dict:
     return sanitized
 
 
-def _oidc_role_management_message(security_config: dict[str, Any]) -> str:
-    admin_group = security_config.get("OIDC_ADMIN_GROUP", "")
+def _oidc_role_management_message(security_config: dict[str, Any] | None = None) -> str:
+    admin_group = (security_config or {}).get(
+        "OIDC_ADMIN_GROUP",
+        app_config.get("OIDC_ADMIN_GROUP", ""),
+    )
     if admin_group:
         return (
             "Admin roles for OIDC users are managed by the "
             f"'{admin_group}' group in your identity provider"
         )
     return (
-        "Disable 'Use Admin Group for Authorization' in security settings "
-        "to manage roles manually"
+        "Disable 'Use Admin Group for Authorization' in security settings to manage roles manually"
     )
 
 
-def _is_user_active(user: dict[str, Any], auth_method: str) -> bool:
-    """Determine whether a user can authenticate in the current auth mode."""
-    source = normalize_auth_source(user.get("auth_source"), user.get("oidc_subject"))
-    if source == AUTH_SOURCE_BUILTIN:
-        return auth_method in (AUTH_SOURCE_BUILTIN, AUTH_SOURCE_OIDC)
-    return source == auth_method
-
-
 def _serialize_user(
     user: dict[str, Any],
     auth_method: str,
@@ -135,7 +119,7 @@ def _serialize_user(
         payload.get("auth_source"),
         payload.get("oidc_subject"),
     )
-    payload["is_active"] = _is_user_active(payload, auth_method)
+    payload["is_active"] = is_user_active_for_auth_mode(payload, auth_method)
     payload["edit_capabilities"] = _get_user_edit_capabilities(
         payload,
         security_config=security_config,
@@ -146,7 +130,8 @@ def _serialize_user(
 def _sync_all_cwa_users(user_db: UserDB) -> dict[str, int]:
     """Sync all users from the Calibre-Web database into users.db."""
     if not CWA_DB_PATH or not CWA_DB_PATH.exists():
-        raise FileNotFoundError("Calibre-Web database is not available")
+        msg = "Calibre-Web database is not available"
+        raise FileNotFoundError(msg)
 
     db_path = os.fspath(CWA_DB_PATH)
     db_uri = f"file:{db_path}?mode=ro&immutable=1"
@@ -164,24 +149,43 @@ def _sync_all_cwa_users(user_db: UserDB) -> dict[str, int]:
 def register_admin_routes(app: Flask, user_db: UserDB) -> None:
     """Register admin user management routes on the Flask app."""
 
+    def _require_admin(
+        f: Callable[P, ResponseReturnValue],
+    ) -> Callable[P, ResponseReturnValue]:
+        """Require an admin session for admin routes.
+
+        In no-auth mode, everyone has access (is_admin defaults True).
+        In auth-required modes, requires an authenticated session with admin role.
+        Caches the resolved auth_mode in ``g.auth_mode`` for the request.
+        """
+
+        @wraps(f)
+        def decorated(*args: P.args, **kwargs: P.kwargs) -> ResponseReturnValue:
+            auth_mode = load_active_auth_mode(CWA_DB_PATH, user_db=user_db)
+            g.auth_mode = auth_mode
+            if auth_mode != "none":
+                if "user_id" not in session:
+                    return jsonify({"error": "Authentication required"}), 401
+                if not session.get("is_admin", False):
+                    return jsonify({"error": "Admin access required"}), 403
+            return f(*args, **kwargs)
+
+        return decorated
+
     @app.route("/api/admin/users", methods=["GET"])
     @_require_admin
-    def admin_list_users():
+    def admin_list_users() -> Response | tuple[Response, int]:
         """List all users."""
         users = user_db.list_users()
-        auth_mode = _get_auth_mode()
-        security_config = load_config_file("security")
-        return jsonify([
-            _serialize_user(u, auth_mode, security_config=security_config)
-            for u in users
-        ])
+        auth_mode = g.auth_mode
+        return jsonify([_serialize_user(u, auth_mode) for u in users])
 
     @app.route("/api/admin/users", methods=["POST"])
     @_require_admin
-    def admin_create_user():
+    def admin_create_user() -> Response | tuple[Response, int]:
         """Create a new user with password authentication."""
         data = request.get_json() or {}
-        auth_mode = _get_auth_mode()
+        auth_mode = g.auth_mode
 
         username = (data.get("username") or "").strip()
         password = data.get("password", "")
@@ -190,23 +194,28 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
         role = data.get("role", "user")
 
         if auth_mode in {AUTH_SOURCE_PROXY, AUTH_SOURCE_CWA}:
-            return jsonify({
-                "error": "Local user creation is disabled in this authentication mode",
-                "message": (
-                    "Users are provisioned by your external authentication source. "
-                    "Switch to builtin or OIDC mode to create local users."
-                ),
-            }), 400
+            return jsonify(
+                {
+                    "error": "Local user creation is disabled in this authentication mode",
+                    "message": (
+                        "Users are provisioned by your external authentication source. "
+                        "Switch to builtin or OIDC mode to create local users."
+                    ),
+                }
+            ), 400
 
         if not username:
             return jsonify({"error": "Username is required"}), 400
-        if not password or len(password) < 4:
-            return jsonify({"error": "Password must be at least 4 characters"}), 400
+        if not password or len(password) < MIN_PASSWORD_LENGTH:
+            return jsonify(
+                {"error": f"Password must be at least {MIN_PASSWORD_LENGTH} characters"}
+            ), 400
         if role not in ("admin", "user"):
             return jsonify({"error": "Role must be 'admin' or 'user'"}), 400
 
         # First user is always admin
-        if not user_db.list_users():
+        existing_users = user_db.list_users()
+        if not existing_users:
             role = "admin"
 
         # Check if username already exists
@@ -226,21 +235,22 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
         except ValueError:
             return jsonify({"error": "Username already exists"}), 409
         logger.info(
-            "Shelfmark user created "
-            f"(source=manual_admin_create, created_by={session.get('user_id', 'unknown')}, "
-            f"username={username}, role={role}, auth_source={AUTH_SOURCE_BUILTIN})"
+            "Shelfmark user created (source=manual_admin_create, created_by=%s, username=%s, role=%s, auth_source=%s)",
+            session.get("user_id", "unknown"),
+            username,
+            role,
+            AUTH_SOURCE_BUILTIN,
         )
         return jsonify(
             _serialize_user(
                 user,
-                _get_auth_mode(),
-                security_config=load_config_file("security"),
+                g.auth_mode,
             )
         ), 201
 
     @app.route("/api/admin/users/<int:user_id>", methods=["GET"])
     @_require_admin
-    def admin_get_user(user_id):
+    def admin_get_user(user_id: int) -> Response | tuple[Response, int]:
         """Get a user by ID with their settings."""
         user = user_db.get_user(user_id=user_id)
         if not user:
@@ -248,39 +258,42 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
 
         result = _serialize_user(
             user,
-            _get_auth_mode(),
-            security_config=load_config_file("security"),
+            g.auth_mode,
         )
         result["settings"] = user_db.get_user_settings(user_id)
         return jsonify(result)
 
     @app.route("/api/admin/users/<int:user_id>", methods=["PUT"])
     @_require_admin
-    def admin_update_user(user_id):
+    def admin_update_user(user_id: int) -> Response | tuple[Response, int]:
         """Update user fields and/or settings."""
         user = user_db.get_user(user_id=user_id)
         if not user:
             return jsonify({"error": "User not found"}), 404
 
         data = request.get_json() or {}
-        security_config = load_config_file("security")
         auth_source = normalize_auth_source(
             user.get("auth_source"),
             user.get("oidc_subject"),
         )
-        capabilities = _get_user_edit_capabilities(user, security_config=security_config)
+        capabilities = _get_user_edit_capabilities(user)
 
         # Handle optional password update
         password = data.get("password", "")
+        password_hash: str | None = None
         if password:
             if not capabilities["canSetPassword"]:
-                return jsonify({
-                    "error": f"Cannot set password for {auth_source.upper()} users",
-                    "message": "Password authentication is only available for local users.",
-                }), 400
-            if len(password) < 4:
-                return jsonify({"error": "Password must be at least 4 characters"}), 400
-            user_db.update_user(user_id, password_hash=generate_password_hash(password))
+                return jsonify(
+                    {
+                        "error": f"Cannot set password for {auth_source.upper()} users",
+                        "message": "Password authentication is only available for local users.",
+                    }
+                ), 400
+            if len(password) < MIN_PASSWORD_LENGTH:
+                return jsonify(
+                    {"error": f"Password must be at least {MIN_PASSWORD_LENGTH} characters"}
+                ), 400
+            password_hash = generate_password_hash(password)
 
         # Update user fields
         user_fields = {}
@@ -293,40 +306,49 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
 
         role_changed = "role" in user_fields and user_fields["role"] != user.get("role")
         email_changed = "email" in user_fields and user_fields["email"] != user.get("email")
-        display_name_changed = (
-            "display_name" in user_fields
-            and user_fields["display_name"] != user.get("display_name")
-        )
+        display_name_changed = "display_name" in user_fields and user_fields[
+            "display_name"
+        ] != user.get("display_name")
 
         if role_changed and not capabilities["canEditRole"]:
             if auth_source == AUTH_SOURCE_OIDC:
-                return jsonify({
-                    "error": "Cannot change role for OIDC user when group-based authorization is enabled",
-                    "message": _oidc_role_management_message(security_config),
-                }), 400
+                return jsonify(
+                    {
+                        "error": "Cannot change role for OIDC user when group-based authorization is enabled",
+                        "message": _oidc_role_management_message(),
+                    }
+                ), 400
 
-            return jsonify({
-                "error": f"Cannot change role for {auth_source.upper()} users",
-                "message": "Role is managed by the external authentication source.",
-            }), 400
+            return jsonify(
+                {
+                    "error": f"Cannot change role for {auth_source.upper()} users",
+                    "message": "Role is managed by the external authentication source.",
+                }
+            ), 400
 
         if email_changed and not capabilities["canEditEmail"]:
             if auth_source == AUTH_SOURCE_CWA:
-                return jsonify({
-                    "error": "Cannot change email for CWA users",
-                    "message": "Email is synced from Calibre-Web.",
-                }), 400
+                return jsonify(
+                    {
+                        "error": "Cannot change email for CWA users",
+                        "message": "Email is synced from Calibre-Web.",
+                    }
+                ), 400
 
-            return jsonify({
-                "error": "Cannot change email for OIDC users",
-                "message": "Email is managed by your identity provider.",
-            }), 400
+            return jsonify(
+                {
+                    "error": "Cannot change email for OIDC users",
+                    "message": "Email is managed by your identity provider.",
+                }
+            ), 400
 
         if display_name_changed and not capabilities["canEditDisplayName"]:
-            return jsonify({
-                "error": "Cannot change display name for OIDC users",
-                "message": "Display name is managed by your identity provider.",
-            }), 400
+            return jsonify(
+                {
+                    "error": "Cannot change display name for OIDC users",
+                    "message": "Display name is managed by your identity provider.",
+                }
+            ), 400
 
         # Allow demoting the last admin account.
         # Auth mode resolution automatically falls back to "none" when no
@@ -337,61 +359,78 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
             if field in user_fields and user_fields[field] == user.get(field):
                 user_fields.pop(field)
 
-        if user_fields:
-            user_db.update_user(user_id, **user_fields)
-
-        # Update per-user settings
+        # Validate per-user settings
+        validated_settings: dict[str, Any] | None = None
         if "settings" in data:
             if not isinstance(data["settings"], dict):
                 return jsonify({"error": "Settings must be an object"}), 400
 
             validated_settings, validation_errors = validate_user_settings(data["settings"])
             if validation_errors:
-                return jsonify({
-                    "error": "Invalid settings payload",
-                    "details": validation_errors,
-                }), 400
+                return jsonify(
+                    {
+                        "error": "Invalid settings payload",
+                        "details": validation_errors,
+                    }
+                ), 400
 
+        # Apply the writes only once the whole payload has been accepted.
+        if password_hash is not None:
+            user_fields["password_hash"] = password_hash
+
+        if user_fields:
+            user_db.update_user(user_id, **user_fields)
+
+        if validated_settings is not None:
             user_db.set_user_settings(user_id, validated_settings)
             # Ensure runtime reads see updated per-user overrides immediately.
             try:
-                from shelfmark.core.config import config as app_config
-                app_config.refresh()
-            except Exception:
-                pass
+                app_config.refresh(force=True)
+            except _CONFIG_REFRESH_ERRORS as exc:
+                logger.warning(
+                    "Updated settings for user %s but failed to refresh runtime config: %s",
+                    user_id,
+                    exc,
+                )
 
         updated = user_db.get_user(user_id=user_id)
+        if not isinstance(updated, dict):
+            return jsonify({"error": "User not found"}), 404
         result = _serialize_user(
             updated,
-            _get_auth_mode(),
-            security_config=security_config,
+            g.auth_mode,
         )
         result["settings"] = user_db.get_user_settings(user_id)
-        logger.info(f"Admin updated user {user_id}")
+        logger.info("Admin updated user %s", user_id)
         return jsonify(result)
 
     @app.route("/api/admin/users/sync-cwa", methods=["POST"])
     @_require_admin
-    def admin_sync_cwa_users():
+    def admin_sync_cwa_users() -> Response | tuple[Response, int]:
         """Manually sync users from Calibre-Web into users.db."""
-        auth_mode = _get_auth_mode()
-        if auth_mode != AUTH_SOURCE_CWA:
-            return jsonify({
-                "error": "CWA sync is only available when CWA authentication is enabled",
-            }), 400
+        if g.auth_mode != AUTH_SOURCE_CWA:
+            return jsonify(
+                {
+                    "error": "CWA sync is only available when CWA authentication is enabled",
+                }
+            ), 400
 
         try:
             summary = _sync_all_cwa_users(user_db)
         except FileNotFoundError:
-            return jsonify({
-                "error": "Calibre-Web database is not available",
-                "message": "Verify app.db is mounted and readable at /auth/app.db.",
-            }), 503
-        except Exception as exc:
-            logger.error(f"Failed to sync CWA users: {exc}")
-            return jsonify({
-                "error": "Failed to sync users from Calibre-Web",
-            }), 500
+            return jsonify(
+                {
+                    "error": "Calibre-Web database is not available",
+                    "message": "Verify app.db is mounted and readable at /auth/app.db.",
+                }
+            ), 503
+        except Exception:
+            logger.exception("Failed to sync CWA users")
+            return jsonify(
+                {
+                    "error": "Failed to sync users from Calibre-Web",
+                }
+            ), 500
 
         message = (
             f"Synced {summary['total']} CWA users "
@@ -399,17 +438,19 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
             f"{summary.get('deleted', 0)} deleted)."
         )
         logger.info(message)
-        return jsonify({
-            "success": True,
-            "message": message,
-            **summary,
-        })
+        return jsonify(
+            {
+                "success": True,
+                "message": message,
+                **summary,
+            }
+        )
 
     register_admin_settings_routes(app, user_db, _require_admin)
 
     @app.route("/api/admin/users/<int:user_id>", methods=["DELETE"])
     @_require_admin
-    def admin_delete_user(user_id):
+    def admin_delete_user(user_id: int) -> Response | tuple[Response, int]:
         """Delete a user."""
         # Prevent self-deletion
         if session.get("db_user_id") == user_id:
@@ -419,21 +460,22 @@ def register_admin_routes(app: Flask, user_db: UserDB) -> None:
         if not user:
             return jsonify({"error": "User not found"}), 404
 
-        auth_mode = _get_auth_mode()
         auth_source = normalize_auth_source(
             user.get("auth_source"),
             user.get("oidc_subject"),
         )
-        if auth_source == AUTH_SOURCE_CWA and auth_source == auth_mode:
-            return jsonify({
-                "error": f"Cannot delete active {auth_source.upper()} users",
-                "message": f"{auth_source.upper()} users are automatically re-provisioned on login.",
-            }), 400
+        if auth_source == AUTH_SOURCE_CWA and auth_source == g.auth_mode:
+            return jsonify(
+                {
+                    "error": f"Cannot delete active {auth_source.upper()} users",
+                    "message": f"{auth_source.upper()} users are automatically re-provisioned on login.",
+                }
+            ), 400
 
         # Allow deleting the last local admin account.
         # Auth mode resolution automatically falls back to "none" when no
         # local password admin remains.
 
         user_db.delete_user(user_id)
-        logger.info(f"Admin deleted user {user_id}: {user['username']}")
+        logger.info("Admin deleted user %s: %s", user_id, user["username"])
         return jsonify({"success": True})

shelfmark/core/admin_settings_routes.py

@@ -1,6 +1,8 @@
 """Admin settings-introspection routes and settings validation helpers."""
 
-from typing import Any, Callable
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
 
 from flask import Flask, jsonify, request
 
@@ -9,17 +11,35 @@ from shelfmark.config.notifications_settings import (
     is_valid_notification_url,
     normalize_notification_routes,
 )
+from shelfmark.config.users_settings import (
+    SEARCH_PREFERENCE_VALIDATABLE_KEYS,
+    validate_search_preference_value,
+)
+from shelfmark.core.config import config as app_config
+from shelfmark.core.request_policy import parse_policy_mode, validate_policy_rules
 from shelfmark.core.settings_registry import load_config_file
 from shelfmark.core.user_settings_overrides import (
     build_user_preferences_payload as _build_user_preferences_payload,
+)
+from shelfmark.core.user_settings_overrides import (
     get_ordered_user_overridable_fields as _get_ordered_user_overridable_fields,
+)
+from shelfmark.core.user_settings_overrides import (
     get_settings_registry as _get_settings_registry,
 )
-from shelfmark.core.user_db import UserDB
-from shelfmark.core.request_policy import parse_policy_mode, validate_policy_rules
+
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
+    from flask.typing import ResponseReturnValue
+
+    from shelfmark.core.user_db import UserDB
 
 
-def validate_user_settings(settings: dict[str, Any]) -> tuple[dict[str, Any], list[str]]:
+def validate_user_settings(
+    settings: dict[str, Any],
+) -> tuple[dict[str, Any], list[str]]:
+    """Validate and normalize per-user settings overrides."""
     settings_registry = _get_settings_registry()
     field_map = settings_registry.get_settings_field_map()
     overridable_map = settings_registry.get_user_overridable_fields()
@@ -37,10 +57,12 @@ def validate_user_settings(settings: dict[str, Any]) -> tuple[dict[str, Any], li
                 valid[key] = None
                 continue
 
-            if key in {"REQUEST_POLICY_DEFAULT_EBOOK", "REQUEST_POLICY_DEFAULT_AUDIOBOOK"}:
-                if parse_policy_mode(value) is None:
-                    errors.append(f"Invalid policy mode for {key}: {value}")
-                    continue
+            if (
+                key in {"REQUEST_POLICY_DEFAULT_EBOOK", "REQUEST_POLICY_DEFAULT_AUDIOBOOK"}
+                and parse_policy_mode(value) is None
+            ):
+                errors.append(f"Invalid policy mode for {key}: {value}")
+                continue
 
             if key == "REQUEST_POLICY_RULES":
                 normalized_rules, rule_errors = validate_policy_rules(value)
@@ -59,15 +81,51 @@ def validate_user_settings(settings: dict[str, Any]) -> tuple[dict[str, Any], li
                 )
                 if invalid_count:
                     errors.append(
-                        (
-                            f"Invalid value for {key}: found {invalid_count} invalid URL(s). "
-                            "Use URL values with a valid scheme, e.g. discord://... or ntfys://..."
-                        )
+                        f"Invalid value for {key}: found {invalid_count} invalid URL(s). "
+                        "Use URL values with a valid scheme, e.g. discord://... or ntfys://..."
                     )
                     continue
                 valid[key] = normalized_routes
                 continue
 
+            normalized_search_value, search_validation_error = validate_search_preference_value(
+                key, value
+            )
+            if search_validation_error:
+                errors.append(search_validation_error)
+                continue
+            # Every key the search validator recognises keeps its normalized value;
+            # a hand-maintained subset here silently dropped normalization for the rest.
+            if key in SEARCH_PREFERENCE_VALIDATABLE_KEYS:
+                valid[key] = normalized_search_value
+                continue
+
+            if key == "DOWNLOAD_TO_BROWSER_CONTENT_TYPES":
+                if not isinstance(value, list):
+                    errors.append(f"Invalid value for {key}: must be a list")
+                    continue
+
+                candidate_values = [
+                    str(entry).strip().lower() for entry in value if str(entry).strip()
+                ]
+                normalized_values: list[str] = []
+                has_invalid_value = False
+                for entry in candidate_values:
+                    if entry not in {"book", "audiobook"}:
+                        errors.append(
+                            f"Invalid value for {key}: unsupported content type '{entry}'"
+                        )
+                        has_invalid_value = True
+                        continue
+                    if entry not in normalized_values:
+                        normalized_values.append(entry)
+
+                if has_invalid_value:
+                    continue
+
+                valid[key] = normalized_values
+                continue
+
             valid[key] = value
 
     return valid, errors
@@ -76,8 +134,9 @@ def validate_user_settings(settings: dict[str, Any]) -> tuple[dict[str, Any], li
 def build_user_notification_test_response(
     *,
     user_id: int,
-    payload: Any,
+    payload: object,
 ) -> tuple[dict[str, Any], int]:
+    """Build a notification test response using effective per-user routes."""
     from shelfmark.core.config import config as app_config
 
     routes_input = app_config.get("USER_NOTIFICATION_ROUTES", [], user_id=user_id)
@@ -95,36 +154,40 @@ def build_user_notification_test_response(
 def register_admin_settings_routes(
     app: Flask,
     user_db: UserDB,
-    require_admin: Callable[[Callable[..., Any]], Callable[..., Any]],
+    require_admin: Callable[
+        [Callable[..., ResponseReturnValue]], Callable[..., ResponseReturnValue]
+    ],
 ) -> None:
+    """Register admin endpoints for user-specific settings and defaults."""
+
     @app.route("/api/admin/download-defaults", methods=["GET"])
     @require_admin
-    def admin_download_defaults():
-        config = load_config_file("downloads")
+    def admin_download_defaults() -> ResponseReturnValue:
         defaults = {
-            key: ("" if (value := config.get(key, field.default)) is None else value)
+            key: ("" if (value := app_config.get(key, field.default)) is None else value)
             for key, field in _get_ordered_user_overridable_fields("downloads")
         }
 
-        security_config = load_config_file("security")
-        defaults["OIDC_ADMIN_GROUP"] = security_config.get("OIDC_ADMIN_GROUP", "")
-        defaults["OIDC_USE_ADMIN_GROUP"] = security_config.get("OIDC_USE_ADMIN_GROUP", True)
-        defaults["OIDC_AUTO_PROVISION"] = security_config.get("OIDC_AUTO_PROVISION", True)
+        defaults["OIDC_ADMIN_GROUP"] = app_config.get("OIDC_ADMIN_GROUP", "")
+        defaults["OIDC_USE_ADMIN_GROUP"] = app_config.get("OIDC_USE_ADMIN_GROUP", True)
+        defaults["OIDC_AUTO_PROVISION"] = app_config.get("OIDC_AUTO_PROVISION", True)
         return jsonify(defaults)
 
     @app.route("/api/admin/booklore-options", methods=["GET"])
     @require_admin
-    def admin_booklore_options():
+    def admin_booklore_options() -> ResponseReturnValue:
         from shelfmark.core import admin_routes
 
-        return jsonify({
-            "libraries": admin_routes.get_booklore_library_options(),
-            "paths": admin_routes.get_booklore_path_options(),
-        })
+        return jsonify(
+            {
+                "libraries": admin_routes.get_booklore_library_options(),
+                "paths": admin_routes.get_booklore_path_options(),
+            }
+        )
 
     @app.route("/api/admin/users/<int:user_id>/delivery-preferences", methods=["GET"])
     @require_admin
-    def admin_get_delivery_preferences(user_id):
+    def admin_get_delivery_preferences(user_id: int) -> ResponseReturnValue:
         user = user_db.get_user(user_id=user_id)
         if not user:
             return jsonify({"error": "User not found"}), 404
@@ -136,9 +199,23 @@ def register_admin_settings_routes(
 
         return jsonify(payload)
 
+    @app.route("/api/admin/users/<int:user_id>/search-preferences", methods=["GET"])
+    @require_admin
+    def admin_get_search_preferences(user_id: int) -> ResponseReturnValue:
+        user = user_db.get_user(user_id=user_id)
+        if not user:
+            return jsonify({"error": "User not found"}), 404
+
+        try:
+            payload = _build_user_preferences_payload(user_db, user_id, "search_mode")
+        except ValueError:
+            return jsonify({"error": "Search mode settings tab not found"}), 500
+
+        return jsonify(payload)
+
     @app.route("/api/admin/users/<int:user_id>/notification-preferences", methods=["GET"])
     @require_admin
-    def admin_get_notification_preferences(user_id):
+    def admin_get_notification_preferences(user_id: int) -> ResponseReturnValue:
         user = user_db.get_user(user_id=user_id)
         if not user:
             return jsonify({"error": "User not found"}), 404
@@ -152,7 +229,7 @@ def register_admin_settings_routes(
 
     @app.route("/api/admin/users/<int:user_id>/notification-preferences/test", methods=["POST"])
     @require_admin
-    def admin_test_notification_preferences(user_id):
+    def admin_test_notification_preferences(user_id: int) -> ResponseReturnValue:
         user = user_db.get_user(user_id=user_id)
         if not user:
             return jsonify({"error": "User not found"}), 404
@@ -166,7 +243,7 @@ def register_admin_settings_routes(
 
     @app.route("/api/admin/settings/overrides-summary", methods=["GET"])
     @require_admin
-    def admin_settings_overrides_summary():
+    def admin_settings_overrides_summary() -> ResponseReturnValue:
         settings_registry = _get_settings_registry()
 
         tab_name = (request.args.get("tab") or "downloads").strip()
@@ -185,11 +262,13 @@ def register_admin_settings_routes(
                 if key not in user_settings or user_settings[key] is None:
                     continue
                 entry = keys_payload.setdefault(key, {"count": 0, "users": []})
-                entry["users"].append({
-                    "userId": user_record["id"],
-                    "username": user_record["username"],
-                    "value": user_settings[key],
-                })
+                entry["users"].append(
+                    {
+                        "userId": user_record["id"],
+                        "username": user_record["username"],
+                        "value": user_settings[key],
+                    }
+                )
 
         for summary in keys_payload.values():
             summary["count"] = len(summary["users"])
@@ -198,7 +277,7 @@ def register_admin_settings_routes(
 
     @app.route("/api/admin/users/<int:user_id>/effective-settings", methods=["GET"])
     @require_admin
-    def admin_get_effective_settings(user_id):
+    def admin_get_effective_settings(user_id: int) -> ResponseReturnValue:
         user = user_db.get_user(user_id=user_id)
         if not user:
             return jsonify({"error": "User not found"}), 404

shelfmark/core/api_key.py

@@ -0,0 +1,39 @@
+"""Static API-key authentication backed by the SHELFMARK_API_KEY environment variable.
+
+When ``SHELFMARK_API_KEY`` is set, a request carrying that value as a Bearer token or in
+``X-Api-Key`` is treated as an admin for that request only. Both headers are
+checked, since a reverse proxy in front of Shelfmark may set its own
+``Authorization`` header, which would otherwise shadow an operator-supplied
+``X-Api-Key``. A candidate that matches neither is ignored so that bearer
+tokens forwarded by reverse proxies keep working. The key is never logged.
+"""
+
+from __future__ import annotations
+
+import hmac
+
+from shelfmark.config.env import SHELFMARK_API_KEY
+
+
+def extract_api_key_candidates(
+    authorization_header: str | None, api_key_header: str | None
+) -> list[str]:
+    """Return the non-empty credentials a client presented, Bearer token first."""
+    candidates: list[str] = []
+    if authorization_header:
+        scheme, _, token = authorization_header.strip().partition(" ")
+        token = token.strip()
+        if scheme.lower() == "bearer" and token:
+            candidates.append(token)
+    if api_key_header:
+        token = api_key_header.strip()
+        if token:
+            candidates.append(token)
+    return candidates
+
+
+def matches_api_key(candidate: str) -> bool:
+    """Constant-time comparison against the configured key. False when unset."""
+    if not SHELFMARK_API_KEY or not candidate:
+        return False
+    return hmac.compare_digest(candidate.encode("utf-8"), SHELFMARK_API_KEY.encode("utf-8"))

shelfmark/core/auth_modes.py

@@ -1,7 +1,14 @@
 """Authentication mode, auth-source normalization, and admin access policy helpers."""
 
+from __future__ import annotations
+
 import os
-from typing import Any, Mapping
+import sqlite3
+from pathlib import Path
+from typing import TYPE_CHECKING, Any, Protocol, TypeGuard
+
+if TYPE_CHECKING:
+    from collections.abc import Mapping
 
 AUTH_SOURCE_BUILTIN = "builtin"
 AUTH_SOURCE_OIDC = "oidc"
@@ -17,7 +24,18 @@ AUTH_SOURCE_SET = frozenset(AUTH_SOURCES)
 _ALWAYS_ADMIN_SETTINGS_TABS = frozenset({"security", "users"})
 
 
-def has_local_password_admin(user_db: Any | None = None) -> bool:
+class _UserDBWithAdminPassword(Protocol):
+    """Minimal user DB surface needed for local-admin checks."""
+
+    def has_admin_with_password(self) -> bool: ...
+
+
+def _has_admin_password_api(candidate: object) -> TypeGuard[_UserDBWithAdminPassword]:
+    """Return True when *candidate* exposes the admin-password lookup we need."""
+    return callable(getattr(candidate, "has_admin_with_password", None))
+
+
+def has_local_password_admin(user_db: object | None = None) -> bool:
     """Return True when at least one local admin with a password exists."""
     try:
         db = user_db
@@ -25,20 +43,19 @@ def has_local_password_admin(user_db: Any | None = None) -> bool:
             from shelfmark.core.user_db import UserDB
 
             config_root = os.environ.get("CONFIG_DIR", "/config")
-            db = UserDB(os.path.join(config_root, "users.db"))
+            db = UserDB(str(Path(config_root) / "users.db"))
             db.initialize()
 
-        return any(
-            user.get("password_hash") and user.get("role") == "admin"
-            for user in db.list_users()
-        )
-    except Exception:
+        if not _has_admin_password_api(db):
+            return False
+        return db.has_admin_with_password()
+    except AttributeError, ImportError, OSError, RuntimeError, TypeError, ValueError, sqlite3.Error:
         return False
 
 
 def normalize_auth_source(
-    source: Any,
-    oidc_subject: Any = None,
+    source: object,
+    oidc_subject: object = None,
 ) -> str:
     """Resolve a stable auth source value from persisted fields."""
     normalized = str(source or "").strip().lower()
@@ -51,17 +68,19 @@ def normalize_auth_source(
 
 def determine_auth_mode(
     security_config: Mapping[str, Any],
-    cwa_db_path: Any | None,
+    cwa_db_path: object | None,
     *,
     has_local_admin: bool = True,
+    disable_local_auth: bool = False,
 ) -> str:
     """Determine active auth mode from security config and runtime prerequisites."""
     auth_mode = security_config.get("AUTH_METHOD", "none")
+    local_admin_available = has_local_admin or disable_local_auth
 
     if auth_mode == AUTH_SOURCE_CWA and cwa_db_path:
         return AUTH_SOURCE_CWA
 
-    if auth_mode == AUTH_SOURCE_BUILTIN and has_local_admin:
+    if auth_mode == AUTH_SOURCE_BUILTIN and local_admin_available:
         return AUTH_SOURCE_BUILTIN
 
     if auth_mode == AUTH_SOURCE_PROXY and security_config.get("PROXY_AUTH_USER_HEADER"):
@@ -69,7 +88,7 @@ def determine_auth_mode(
 
     if (
         auth_mode == AUTH_SOURCE_OIDC
-        and has_local_admin
+        and local_admin_available
         and security_config.get("OIDC_DISCOVERY_URL")
         and security_config.get("OIDC_CLIENT_ID")
     ):
@@ -78,9 +97,43 @@ def determine_auth_mode(
     return "none"
 
 
+def load_active_auth_mode(
+    cwa_db_path: object | None,
+    *,
+    user_db: object | None = None,
+) -> str:
+    """Resolve active auth mode using current security config and runtime prerequisites."""
+    try:
+        from shelfmark.config.env import DISABLE_LOCAL_AUTH
+        from shelfmark.core.config import config as app_config
+
+        security_config = {
+            "AUTH_METHOD": app_config.get("AUTH_METHOD", "none"),
+            "PROXY_AUTH_USER_HEADER": app_config.get("PROXY_AUTH_USER_HEADER", ""),
+            "OIDC_DISCOVERY_URL": app_config.get("OIDC_DISCOVERY_URL", ""),
+            "OIDC_CLIENT_ID": app_config.get("OIDC_CLIENT_ID", ""),
+        }
+        return determine_auth_mode(
+            security_config,
+            cwa_db_path,
+            has_local_admin=has_local_password_admin(user_db),
+            disable_local_auth=DISABLE_LOCAL_AUTH,
+        )
+    except ImportError, OSError, RuntimeError, TypeError, ValueError, sqlite3.Error:
+        return "none"
+
+
+def is_user_active_for_auth_mode(user: Mapping[str, Any], auth_mode: str) -> bool:
+    """Return whether a user can authenticate under the current auth mode."""
+    source = normalize_auth_source(user.get("auth_source"), user.get("oidc_subject"))
+    if source == AUTH_SOURCE_BUILTIN:
+        return auth_mode in (AUTH_SOURCE_BUILTIN, AUTH_SOURCE_OIDC)
+    return source == auth_mode
+
+
 def is_settings_or_onboarding_path(path: str) -> bool:
     """Return True when request path targets protected admin settings routes."""
-    return path.startswith("/api/settings") or path.startswith("/api/onboarding")
+    return path.startswith(("/api/settings", "/api/onboarding"))
 
 
 def get_settings_tab_from_path(path: str) -> str | None:
@@ -88,7 +141,7 @@ def get_settings_tab_from_path(path: str) -> str | None:
     if not path.startswith("/api/settings/"):
         return None
 
-    suffix = path[len("/api/settings/"):]
+    suffix = path[len("/api/settings/") :]
     if not suffix:
         return None
 

shelfmark/core/author_match.py

@@ -0,0 +1,112 @@
+"""Comparing and trimming author names for release search.
+
+Lives in core because more than one release source needs it: Prowlarr ranks
+results on author agreement (#1293), and IRC both trims the name it searches
+for and ranks what comes back.
+"""
+
+import re
+
+_AUTHOR_TOKEN_PATTERN = re.compile(r"\w+", re.UNICODE)
+_AUTHOR_NOISE_TOKENS = frozenset(
+    {"jr", "sr", "ii", "iii", "iv", "phd", "md", "dr", "mr", "mrs", "ms", "et", "al", "and", "the"}
+)
+
+# Ordering tiers for author agreement between the requested book and what an
+# indexer reported. Lower sorts first.
+AUTHOR_MATCH = 0
+AUTHOR_PARTIAL = 1
+AUTHOR_UNKNOWN = 2
+AUTHOR_MISMATCH = 3
+
+# A mononym ("Homer") can only ever agree on one token; a longer name needs a
+# given name and a surname to agree before it counts as the same person.
+_AUTHOR_TOKENS_REQUIRED = 2
+
+
+def _author_tokens(value: object) -> list[str]:
+    """Split an author string into comparable lowercase name tokens."""
+    if not isinstance(value, str):
+        return []
+    tokens = [token.lower() for token in _AUTHOR_TOKEN_PATTERN.findall(value)]
+    return [token for token in tokens if token not in _AUTHOR_NOISE_TOKENS]
+
+
+def _author_tokens_compatible(wanted: str, offered: str) -> bool:
+    """Treat an abbreviated given name as the name it abbreviates."""
+    return wanted == offered or wanted.startswith(offered) or offered.startswith(wanted)
+
+
+def author_affinity(wanted: object, offered: object) -> int:
+    """Rank how far an indexer's author field is from the requested author.
+
+    Shelfmark ranks on this rather than filtering on it, so a wrong verdict only
+    costs a release its position in the list, never its visibility. That is what
+    makes the loose token comparison safe: "Tim"/"Timothy" and "T."/"Timothy"
+    agree, while a transliteration ("Dostoevsky"/"Dostoyevsky") is merely sorted
+    last instead of being hidden.
+
+    Graded, not binary, because the ways of falling short are not equally bad.
+    An indexer that reports no author at all must not sort below one that reports
+    a wrong author, so "no metadata" ranks between agreement and disagreement. And
+    a name that merely says *less* than the one asked for is not evidence of a
+    different person: "Petrie" contradicts nothing about "David Petrie", while
+    "Gordon Petrie" does. That gap matters most where a source is searched by
+    surname alone (#1331) - the filenames such a search is meant to reach are
+    exactly the ones filed under a bare surname, and ranking them as wrong put
+    them below every result that named someone else entirely.
+    """
+    wanted_tokens = _author_tokens(wanted)
+    offered_tokens = _author_tokens(offered)
+    if not wanted_tokens or not offered_tokens:
+        return AUTHOR_UNKNOWN
+
+    matched = sum(
+        1
+        for wanted_token in wanted_tokens
+        if any(
+            _author_tokens_compatible(wanted_token, offered_token)
+            for offered_token in offered_tokens
+        )
+    )
+    required = min(_AUTHOR_TOKENS_REQUIRED, len(wanted_tokens))
+    if matched >= required:
+        return AUTHOR_MATCH
+
+    # Too little agreement to call it the same person, so the question is whether
+    # what was offered *disagrees*. A name every token of which fits the wanted
+    # name is an abbreviation of it; one carrying a token that fits nothing is a
+    # different name that happens to share a surname.
+    if all(
+        any(
+            _author_tokens_compatible(wanted_token, offered_token) for wanted_token in wanted_tokens
+        )
+        for offered_token in offered_tokens
+    ):
+        return AUTHOR_PARTIAL
+
+    return AUTHOR_MISMATCH
+
+
+def search_surname(author: object) -> str:
+    """The one name token worth sending to a source that matches conjunctively.
+
+    Given names are where catalogues disagree - "David Petrie" is filed as
+    "D. Petrie", "Timothy" as "Tim" - so a query carrying one matches nothing on
+    a source that requires every term to appear. The surname is the token both
+    spellings share.
+
+    Keeps the author's own capitalisation, because the result is posted to a
+    public channel, and returns "" when no usable token is left so the caller
+    searches by title alone rather than by noise.
+    """
+    if not isinstance(author, str):
+        return ""
+    tokens = [
+        token
+        for token in _AUTHOR_TOKEN_PATTERN.findall(author)
+        if token.lower() not in _AUTHOR_NOISE_TOKENS
+    ]
+    if not tokens:
+        return ""
+    return tokens[-1]

shelfmark/core/cache.py

@@ -4,32 +4,37 @@ import threading
 import time
 from dataclasses import dataclass
 from functools import wraps
-from typing import Any, Callable, Dict, Optional, TypeVar
+from typing import TYPE_CHECKING, ParamSpec, TypeVar, cast
 
 from shelfmark.core.logger import setup_logger
 
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
 logger = setup_logger(__name__)
 
-T = TypeVar("T")
+P = ParamSpec("P")
+R = TypeVar("R")
 
 
 @dataclass
 class CacheEntry:
     """A cached value with expiration time."""
-    value: Any
+
+    value: object
     expires_at: float
 
 
 class CacheService:
     """Thread-safe in-memory cache with TTL support."""
 
-    def __init__(self, max_size: int = 1000):
+    def __init__(self, max_size: int = 1000) -> None:
         """Initialize cache with max_size entries before eviction."""
-        self._cache: Dict[str, CacheEntry] = {}
+        self._cache: dict[str, CacheEntry] = {}
         self._lock = threading.Lock()
         self._max_size = max_size
 
-    def get(self, key: str) -> Optional[Any]:
+    def get(self, key: str) -> object | None:
         """Get cached value if not expired."""
         with self._lock:
             entry = self._cache.get(key)
@@ -42,17 +47,14 @@ class CacheService:
 
             return entry.value
 
-    def set(self, key: str, value: Any, ttl: int) -> None:
+    def set(self, key: str, value: object, ttl: int) -> None:
         """Cache value with TTL in seconds."""
         with self._lock:
             # Evict oldest entries if at capacity
             if len(self._cache) >= self._max_size:
                 self._evict_oldest()
 
-            self._cache[key] = CacheEntry(
-                value=value,
-                expires_at=time.time() + ttl
-            )
+            self._cache[key] = CacheEntry(value=value, expires_at=time.time() + ttl)
 
     def invalidate(self, key: str) -> bool:
         """Remove specific cache entry. Returns True if found."""
@@ -62,6 +64,14 @@ class CacheService:
                 return True
             return False
 
+    def invalidate_prefix(self, prefix: str) -> int:
+        """Remove all cache entries whose keys start with prefix."""
+        with self._lock:
+            matching_keys = [key for key in self._cache if key.startswith(prefix)]
+            for key in matching_keys:
+                del self._cache[key]
+            return len(matching_keys)
+
     def clear(self) -> None:
         """Clear all cache entries."""
         with self._lock:
@@ -71,10 +81,7 @@ class CacheService:
         """Remove all expired entries. Returns count removed."""
         with self._lock:
             now = time.time()
-            expired_keys = [
-                key for key, entry in self._cache.items()
-                if entry.expires_at < now
-            ]
+            expired_keys = [key for key, entry in self._cache.items() if entry.expires_at < now]
             for key in expired_keys:
                 del self._cache[key]
             return len(expired_keys)
@@ -86,21 +93,15 @@ class CacheService:
 
         # Remove ~10% of entries, oldest first
         entries_to_remove = max(1, len(self._cache) // 10)
-        sorted_entries = sorted(
-            self._cache.items(),
-            key=lambda x: x[1].expires_at
-        )
+        sorted_entries = sorted(self._cache.items(), key=lambda x: x[1].expires_at)
 
         for key, _ in sorted_entries[:entries_to_remove]:
             del self._cache[key]
 
-    def stats(self) -> Dict[str, int]:
+    def stats(self) -> dict[str, int]:
         """Get cache statistics (size, max_size)."""
         with self._lock:
-            return {
-                "size": len(self._cache),
-                "max_size": self._max_size
-            }
+            return {"size": len(self._cache), "max_size": self._max_size}
 
 
 # Global cache instance for metadata providers
@@ -112,23 +113,38 @@ def get_metadata_cache() -> CacheService:
     return _metadata_cache
 
 
-def cache_key(*args, **kwargs) -> str:
+def cache_key(*args: object, **kwargs: object) -> str:
     """Generate cache key from arguments."""
     parts = [str(arg) for arg in args]
     parts.extend(f"{k}={v}" for k, v in sorted(kwargs.items()))
     return ":".join(parts)
 
 
+def _coerce_ttl_seconds(value: object, *, default: int) -> int:
+    """Normalize cache TTL values read from config or decorator arguments."""
+    if isinstance(value, bool):
+        return default
+    if isinstance(value, int):
+        return value if value > 0 else default
+    if isinstance(value, str):
+        stripped = value.strip()
+        if stripped.isdigit():
+            parsed = int(stripped)
+            return parsed if parsed > 0 else default
+    return default
+
+
 def cacheable(
-    ttl: Optional[int] = None,
-    ttl_key: Optional[str] = None,
+    ttl: int | None = None,
+    ttl_key: str | None = None,
     ttl_default: int = 300,
-    key_prefix: str = ""
-):
-    """Decorator for caching function results. Use ttl (static) or ttl_key (from config)."""
-    def decorator(func: Callable[..., T]) -> Callable[..., T]:
+    key_prefix: str = "",
+) -> Callable[[Callable[P, R]], Callable[P, R]]:
+    """Cache function results with a static or config-backed TTL."""
+
+    def decorator(func: Callable[P, R]) -> Callable[P, R]:
         @wraps(func)
-        def wrapper(*args, **kwargs) -> T:
+        def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
             # Check if metadata caching is enabled
             from shelfmark.core.config import config
 
@@ -140,7 +156,10 @@ def cacheable(
             if ttl is not None:
                 effective_ttl = ttl
             elif ttl_key:
-                effective_ttl = config.get(ttl_key, ttl_default)
+                effective_ttl = _coerce_ttl_seconds(
+                    config.get(ttl_key, ttl_default),
+                    default=ttl_default,
+                )
             else:
                 effective_ttl = ttl_default
 
@@ -148,16 +167,12 @@ def cacheable(
             # Skip 'self' argument if present (first arg of method)
             cache_args = args[1:] if args and hasattr(args[0], func.__name__) else args
 
-            key = cache_key(
-                key_prefix or func.__name__,
-                *cache_args,
-                **kwargs
-            )
+            key = cache_key(key_prefix or func.__name__, *cache_args, **kwargs)
 
             # Check cache
             cached = _metadata_cache.get(key)
             if cached is not None:
-                return cached
+                return cast("R", cached)
 
             # Execute function and cache result
             result = func(*args, **kwargs)
@@ -169,4 +184,5 @@ def cacheable(
             return result
 
         return wrapper
+
     return decorator

shelfmark/core/config.py

@@ -2,72 +2,92 @@
 
 import os
 import sqlite3
+import time
+from importlib import import_module
+from pathlib import Path
 from threading import Lock
-from typing import Any, Dict, Optional
+from typing import TYPE_CHECKING, Any, Self
+
+if TYPE_CHECKING:
+    from types import ModuleType
+
+    from shelfmark.core.user_db import UserDB
 
 # Import lazily to avoid circular imports
 _registry_module = None
 _env_module = None
 _user_db_module = None
 
+_SETTINGS_REFRESH_COOLDOWN_SECONDS = 0.05
 
-def _get_registry():
+
+def _get_registry() -> ModuleType:
     """Lazy import of settings registry to avoid circular imports."""
     global _registry_module
     if _registry_module is None:
         from shelfmark.core import settings_registry
+
         _registry_module = settings_registry
     return _registry_module
 
 
-def _get_env():
+def _get_env() -> ModuleType:
     """Lazy import of env module for fallback values."""
     global _env_module
     if _env_module is None:
         from shelfmark.config import env
+
         _env_module = env
     return _env_module
 
 
-def _get_user_db_module():
+def _get_user_db_module() -> type[UserDB]:
     """Lazy import of user DB module to avoid optional dependency loops."""
     global _user_db_module
     if _user_db_module is None:
         from shelfmark.core.user_db import UserDB
+
         _user_db_module = UserDB
     return _user_db_module
 
 
 class Config:
-    """
-    Dynamic configuration singleton that provides live settings access.
+    """Dynamic configuration singleton that provides live settings access.
 
     Settings are resolved with priority: ENV var > config file > default.
     Values are cached for performance and can be refreshed when settings change.
     """
 
-    _instance: Optional['Config'] = None
+    _instance: Self | None = None
     _lock = Lock()
-    def __new__(cls) -> 'Config':
+
+    def __new__(cls) -> Self:
+        """Return the shared configuration singleton instance."""
         if cls._instance is None:
             with cls._lock:
                 if cls._instance is None:
                     cls._instance = super().__new__(cls)
                     cls._instance._initialized = False
-        return cls._instance
+        instance = cls._instance
+        if instance is None:
+            msg = "Config singleton failed to initialize"
+            raise RuntimeError(msg)
+        return instance
 
-    def __init__(self):
+    def __init__(self) -> None:
+        """Initialize caches and backing stores for the singleton."""
         if self._initialized:
             return
-        self._cache: Dict[str, Any] = {}
-        self._field_map: Dict[str, tuple] = {}  # key -> (field, tab_name)
+        self._cache: dict[str, Any] = {}
+        self._field_map: dict[str, tuple] = {}  # key -> (field, tab_name)
         self._cache_lock = Lock()
-        self._user_settings_cache: Dict[int, Dict[str, Any]] = {}
+        self._user_settings_cache: dict[int, dict[str, Any]] = {}
         self._user_settings_cache_lock = Lock()
         self._user_db = None
         self._user_db_load_attempted = False
         self._initialized = True
         self._loaded = False
+        self._last_refresh_time: float = 0.0
 
     def _ensure_loaded(self) -> None:
         """Ensure settings are loaded from the registry."""
@@ -83,10 +103,12 @@ class Config:
         # Ensure all settings modules are imported before loading
         # This handles cases where config is accessed before settings are registered
         try:
-            import shelfmark.config.settings  # noqa: F401 - main app settings
-            import shelfmark.config.notifications_settings  # noqa: F401 - notifications settings
-            import shelfmark.release_sources  # noqa: F401 - plugin settings
-            import shelfmark.metadata_providers  # noqa: F401 - plugin settings
+            import_module("shelfmark.config.notifications_settings")
+            import_module("shelfmark.config.security")
+            import_module("shelfmark.config.settings")
+            import_module("shelfmark.config.users_settings")
+            import_module("shelfmark.metadata_providers")
+            import_module("shelfmark.release_sources")
         except ImportError:
             pass
 
@@ -94,7 +116,7 @@ class Config:
 
         # On first load, sync ENV values to config files
         # This ensures ENV values persist even if ENV vars are later removed
-        if not hasattr(self, '_env_synced'):
+        if not hasattr(self, "_env_synced"):
             registry.sync_env_to_config()
             self._env_synced = True
 
@@ -102,28 +124,27 @@ class Config:
         self._field_map.clear()
         self._cache.clear()
 
-        for tab in registry.get_all_settings_tabs():
-            for field in tab.fields:
-                # Skip action buttons and headings - they don't have values
-                if isinstance(field, (registry.ActionButton, registry.HeadingField)):
-                    continue
-
-                key = field.key
-                self._field_map[key] = (field, tab.name)
-
-                # Load current value
-                value = registry.get_setting_value(field, tab.name)
-                self._cache[key] = value
+        for key, (field, tab_name) in registry.get_settings_field_map().items():
+            self._field_map[key] = (field, tab_name)
+            self._cache[key] = registry.get_setting_value(field, tab_name)
 
         self._loaded = True
 
-    def refresh(self) -> None:
-        """
-        Refresh all cached settings from config files.
+    def refresh(self, *, force: bool = False) -> None:
+        """Refresh all cached settings from config files.
 
         Call this after settings are updated via the UI to ensure
         the config singleton reflects the new values.
+
+        Multiple calls within a short window (50 ms) are coalesced to
+        avoid redundant disk I/O when several helpers each call refresh()
+        during the same request.  Pass ``force=True`` to bypass the guard
+        (e.g. after a settings write).
         """
+        now = time.monotonic()
+        if not force and (now - self._last_refresh_time) < _SETTINGS_REFRESH_COOLDOWN_SECONDS:
+            return
+
         with self._cache_lock:
             self._loaded = False
             self._load_settings()
@@ -131,8 +152,9 @@ class Config:
             self._user_settings_cache.clear()
         self._user_db = None
         self._user_db_load_attempted = False
+        self._last_refresh_time = time.monotonic()
 
-    def _get_user_db(self):
+    def _get_user_db(self) -> UserDB | None:
         """Get or initialize a UserDB handle if available."""
         if self._user_db is not None:
             return self._user_db
@@ -142,16 +164,17 @@ class Config:
         self._user_db_load_attempted = True
         try:
             user_db_cls = _get_user_db_module()
-            db_path = os.path.join(os.environ.get("CONFIG_DIR", "/config"), "users.db")
+            db_path = str(Path(os.environ.get("CONFIG_DIR", "/config")) / "users.db")
             user_db = user_db_cls(db_path)
             user_db.initialize()
-            self._user_db = user_db
-            return self._user_db
-        except Exception:
+        except ImportError, OSError, sqlite3.Error:
             # Multi-user support is optional; fall back to global config when unavailable.
             return None
+        else:
+            self._user_db = user_db
+            return self._user_db
 
-    def _get_user_settings(self, user_id: int) -> Dict[str, Any]:
+    def _get_user_settings(self, user_id: int) -> dict[str, Any]:
         """Get cached per-user settings from user DB."""
         with self._user_settings_cache_lock:
             if user_id in self._user_settings_cache:
@@ -163,7 +186,7 @@ class Config:
 
         try:
             settings = user_db.get_user_settings(user_id)
-        except (sqlite3.OperationalError, OSError, ValueError, TypeError):
+        except sqlite3.OperationalError, OSError, ValueError, TypeError:
             return {}
 
         if not isinstance(settings, dict):
@@ -173,14 +196,13 @@ class Config:
             self._user_settings_cache[user_id] = settings
         return settings
 
-    def _get_user_override(self, user_id: int, key: str) -> Any:
+    def _get_user_override(self, user_id: int, key: str) -> object:
         """Get a user override for a specific key."""
         user_settings = self._get_user_settings(user_id)
         return user_settings.get(key)
 
-    def get(self, key: str, default: Any = None, user_id: Optional[int] = None) -> Any:
-        """
-        Get a setting value by key.
+    def get(self, key: str, default: object = None, user_id: int | None = None) -> object:
+        """Get a setting value by key.
 
         Args:
             key: The setting key (e.g., 'MAX_RETRY')
@@ -189,6 +211,7 @@ class Config:
 
         Returns:
             The setting value, or default if not found
+
         """
         self._ensure_loaded()
 
@@ -208,15 +231,15 @@ class Config:
 
         return self._cache.get(key, default)
 
-    def __getattr__(self, name: str) -> Any:
-        """
-        Allow attribute-style access to settings.
+    def __getattr__(self, name: str) -> object:
+        """Allow attribute-style access to settings.
 
         Example: config.MAX_RETRY instead of config.get('MAX_RETRY')
         """
         # Avoid recursion for internal attributes
-        if name.startswith('_'):
-            raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
+        if name.startswith("_"):
+            msg = f"'{type(self).__name__}' object has no attribute '{name}'"
+            raise AttributeError(msg)
 
         self._ensure_loaded()
 
@@ -229,17 +252,18 @@ class Config:
         if hasattr(env, name):
             return getattr(env, name)
 
-        raise AttributeError(f"Setting '{name}' not found in config or env")
+        msg = f"Setting '{name}' not found in config or env"
+        raise AttributeError(msg)
 
     def is_from_env(self, key: str) -> bool:
-        """
-        Check if a setting's value comes from an environment variable.
+        """Check if a setting's value comes from an environment variable.
 
         Args:
             key: The setting key
 
         Returns:
             True if the value is set via ENV var, False otherwise
+
         """
         self._ensure_loaded()
 
@@ -250,12 +274,12 @@ class Config:
         registry = _get_registry()
         return registry.is_value_from_env(field)
 
-    def get_all(self) -> Dict[str, Any]:
-        """
-        Get all cached settings as a dictionary.
+    def get_all(self) -> dict[str, Any]:
+        """Get all cached settings as a dictionary.
 
         Returns:
             Dict of all setting keys to their current values
+
         """
         self._ensure_loaded()
         return dict(self._cache)

shelfmark/core/cwa_user_sync.py

@@ -2,16 +2,20 @@
 
 from __future__ import annotations
 
-from typing import Any, Iterable
+from typing import TYPE_CHECKING, Any
 
 from shelfmark.core.auth_modes import AUTH_SOURCE_CWA, normalize_auth_source
 from shelfmark.core.external_user_linking import upsert_external_user
-from shelfmark.core.user_db import UserDB
+
+if TYPE_CHECKING:
+    from collections.abc import Iterable
+
+    from shelfmark.core.user_db import UserDB
 
 _CWA_ALIAS_SUFFIX = "__cwa"
 
 
-def _normalize_email(value: Any) -> str | None:
+def _normalize_email(value: object) -> str | None:
     if value is None:
         return None
     email = str(value).strip()
@@ -35,12 +39,14 @@ def upsert_cwa_user(
         email=normalized_email,
         role=role,
         allow_email_link=True,
+        sync_username=True,
         collision_strategy=collision_strategy,
         alias_suffix=_CWA_ALIAS_SUFFIX,
         context=context,
     )
     if user is None:
-        raise RuntimeError("Unexpected CWA user sync result: no user returned")
+        msg = "Unexpected CWA user sync result: no user returned"
+        raise RuntimeError(msg)
     return user, action
 
 
@@ -73,10 +79,13 @@ def sync_cwa_users_from_rows(
 
     deleted = 0
     for existing_user in user_db.list_users():
-        if normalize_auth_source(
-            existing_user.get("auth_source"),
-            existing_user.get("oidc_subject"),
-        ) != AUTH_SOURCE_CWA:
+        if (
+            normalize_auth_source(
+                existing_user.get("auth_source"),
+                existing_user.get("oidc_subject"),
+            )
+            != AUTH_SOURCE_CWA
+        ):
             continue
 
         existing_id = int(existing_user.get("id") or 0)

shelfmark/core/download_history_service.py

@@ -0,0 +1,437 @@
+"""Persistence helpers for canonical download activity rows."""
+
+from __future__ import annotations
+
+import json
+import sqlite3
+import threading
+from datetime import UTC, datetime
+from pathlib import Path
+from typing import Any, SupportsIndex, SupportsInt, TypeGuard
+
+from shelfmark.core.logger import setup_logger
+from shelfmark.core.models import TERMINAL_QUEUE_STATUSES
+from shelfmark.core.request_helpers import (
+    normalize_optional_positive_int,
+    normalize_optional_text,
+    now_utc_iso,
+)
+
+logger = setup_logger(__name__)
+
+
+VALID_TERMINAL_STATUSES = frozenset(s.value for s in TERMINAL_QUEUE_STATUSES)
+ACTIVE_DOWNLOAD_STATUS = "active"
+VALID_ORIGINS = frozenset({"direct", "requested"})
+
+
+def _is_convertible_to_int(
+    value: object,
+) -> TypeGuard[str | bytes | bytearray | SupportsInt | SupportsIndex]:
+    """Return True when *value* can be safely passed to ``int``."""
+    return (
+        isinstance(value, (str, bytes, bytearray))
+        or hasattr(value, "__int__")
+        or hasattr(value, "__index__")
+    )
+
+
+def _coerce_int_value(value: object) -> int:
+    """Normalize int-like values and raise TypeError for unsupported inputs."""
+    if isinstance(value, bool) or not _is_convertible_to_int(value):
+        msg = "limit must be an integer"
+        raise TypeError(msg)
+    return int(value)
+
+
+def _normalize_task_id(task_id: object) -> str:
+    normalized = normalize_optional_text(task_id)
+    if normalized is None:
+        msg = "task_id must be a non-empty string"
+        raise ValueError(msg)
+    return normalized
+
+
+def _normalize_origin(origin: object) -> str:
+    normalized = normalize_optional_text(origin)
+    if normalized is None:
+        return "direct"
+    lowered = normalized.lower()
+    if lowered not in VALID_ORIGINS:
+        msg = "origin must be one of: direct, requested"
+        raise ValueError(msg)
+    return lowered
+
+
+def _normalize_final_status(final_status: object) -> str:
+    normalized = normalize_optional_text(final_status)
+    if normalized is None:
+        msg = "final_status must be a non-empty string"
+        raise ValueError(msg)
+    lowered = normalized.lower()
+    if lowered not in VALID_TERMINAL_STATUSES:
+        msg = "final_status must be one of: complete, error, cancelled"
+        raise ValueError(msg)
+    return lowered
+
+
+def _normalize_limit(value: object, *, default: int, minimum: int, maximum: int) -> int:
+    if value is None:
+        return default
+    try:
+        parsed = _coerce_int_value(value)
+    except (TypeError, ValueError) as exc:
+        msg = "limit must be an integer"
+        raise ValueError(msg) from exc
+    if parsed < minimum:
+        return minimum
+    if parsed > maximum:
+        return maximum
+    return parsed
+
+
+class DownloadHistoryService:
+    """Service for persisted canonical download activity rows."""
+
+    def __init__(self, db_path: str) -> None:
+        """Initialize the service with the SQLite history database path."""
+        self._db_path = db_path
+        self._lock = threading.Lock()
+
+    def _connect(self) -> sqlite3.Connection:
+        conn = sqlite3.connect(self._db_path)
+        conn.row_factory = sqlite3.Row
+        conn.execute("PRAGMA foreign_keys = ON")
+        return conn
+
+    @classmethod
+    def _normalize_row_dict(cls, row: dict[str, Any] | None) -> dict[str, Any] | None:
+        if row is None:
+            return None
+        normalized = dict(row)
+        normalized["retry_payload"] = cls._deserialize_retry_payload(
+            normalized.get("retry_payload")
+        )
+        return normalized
+
+    @classmethod
+    def _row_to_dict(cls, row: sqlite3.Row | None) -> dict[str, Any] | None:
+        return cls._normalize_row_dict(dict(row) if row is not None else None)
+
+    @staticmethod
+    def _to_item_key(task_id: str) -> str:
+        return f"download:{task_id}"
+
+    @staticmethod
+    def _resolve_existing_download_path(value: object) -> str | None:
+        normalized = normalize_optional_text(value)
+        if normalized is None:
+            return None
+        return normalized if Path(normalized).exists() else None
+
+    @staticmethod
+    def _serialize_retry_payload(payload: object) -> str | None:
+        if payload is None:
+            return None
+        try:
+            return json.dumps(payload, ensure_ascii=False, separators=(",", ":"))
+        except (TypeError, ValueError) as exc:
+            msg = "retry_payload must be JSON-serializable"
+            raise ValueError(msg) from exc
+
+    @staticmethod
+    def _deserialize_retry_payload(value: object) -> dict[str, Any] | None:
+        if isinstance(value, dict):
+            return dict(value)
+        normalized = normalize_optional_text(value)
+        if normalized is None:
+            return None
+        try:
+            parsed = json.loads(normalized)
+        except json.JSONDecodeError:
+            return None
+        return parsed if isinstance(parsed, dict) else None
+
+    @staticmethod
+    def _has_staged_retry_source(retry_payload: dict[str, Any]) -> bool:
+        staged_path = retry_payload.get("staged_path")
+        normalized_staged_path = normalize_optional_text(staged_path)
+        if normalized_staged_path is None:
+            return False
+        return Path(normalized_staged_path).exists()
+
+    @staticmethod
+    def _can_retry_without_staged_source(retry_payload: dict[str, Any]) -> bool:
+        return bool(retry_payload.get("can_retry_without_staged_source", True))
+
+    @staticmethod
+    def is_retry_available(row: dict[str, Any]) -> bool:
+        """Return whether a persisted download row can be retried."""
+        final_status = (
+            str(row.get("retry_final_status") or row.get("final_status") or "").strip().lower()
+        )
+        retry_payload = DownloadHistoryService._deserialize_retry_payload(row.get("retry_payload"))
+        if retry_payload is None:
+            return False
+
+        has_staged_retry_source = DownloadHistoryService._has_staged_retry_source(retry_payload)
+        can_retry_without_staged_source = DownloadHistoryService._can_retry_without_staged_source(
+            retry_payload
+        )
+        request_id = normalize_optional_positive_int(row.get("request_id"), "request_id")
+        if request_id is None:
+            if final_status in {ACTIVE_DOWNLOAD_STATUS, "cancelled"}:
+                return can_retry_without_staged_source
+            if final_status == "error":
+                return has_staged_retry_source or can_retry_without_staged_source
+            return False
+
+        if final_status in {ACTIVE_DOWNLOAD_STATUS, "cancelled"}:
+            return can_retry_without_staged_source
+
+        if final_status != "error":
+            return False
+
+        return has_staged_retry_source
+
+    @staticmethod
+    def to_download_payload(row: dict[str, Any]) -> dict[str, Any]:
+        """Build the sidebar/history download payload for a persisted row."""
+        return {
+            "id": row.get("task_id"),
+            "title": row.get("title"),
+            "author": row.get("author"),
+            "format": row.get("format"),
+            "size": row.get("size"),
+            "preview": row.get("preview"),
+            "content_type": row.get("content_type"),
+            "source": row.get("source"),
+            "source_display_name": row.get("source_display_name"),
+            "downloads": row.get("downloads"),
+            "status_message": row.get("status_message"),
+            "download_path": DownloadHistoryService._resolve_existing_download_path(
+                row.get("download_path")
+            ),
+            "added_time": DownloadHistoryService._iso_to_epoch(row.get("queued_at")),
+            "user_id": row.get("user_id"),
+            "username": row.get("username"),
+            "request_id": row.get("request_id"),
+            "retry_available": DownloadHistoryService.is_retry_available(row),
+        }
+
+    @staticmethod
+    def _iso_to_epoch(value: object) -> float | None:
+        if not isinstance(value, str) or not value.strip():
+            return None
+        normalized = value.strip().replace("Z", "+00:00")
+        try:
+            parsed = datetime.fromisoformat(normalized)
+        except ValueError:
+            return None
+        if parsed.tzinfo is None:
+            parsed = parsed.replace(tzinfo=UTC)
+        return parsed.timestamp()
+
+    @classmethod
+    def to_history_row(cls, row: dict[str, Any], *, dismissed_at: str) -> dict[str, Any]:
+        """Build the activity-history payload for a persisted download row."""
+        task_id = str(row.get("task_id") or "").strip()
+        item_key = cls._to_item_key(task_id)
+        download_payload = cls.to_download_payload(row)
+        # Clear stale progress messages for non-error terminal states.
+        if row.get("final_status") in ("complete", "cancelled"):
+            download_payload["status_message"] = None
+        return {
+            "id": item_key,
+            "user_id": row.get("user_id"),
+            "item_type": "download",
+            "item_key": item_key,
+            "dismissed_at": dismissed_at,
+            "snapshot": {
+                "kind": "download",
+                "download": download_payload,
+            },
+            "origin": row.get("origin"),
+            "final_status": row.get("final_status"),
+            "terminal_at": row.get("terminal_at"),
+            "request_id": row.get("request_id"),
+            "source_id": task_id or None,
+        }
+
+    def record_download(
+        self,
+        *,
+        task_id: str,
+        user_id: int | None,
+        username: str | None,
+        request_id: int | None,
+        source: str,
+        source_display_name: str | None,
+        title: str,
+        author: str | None,
+        file_format: str | None,
+        size: str | None,
+        preview: str | None,
+        content_type: str | None,
+        downloads: int | None,
+        origin: str,
+        retry_payload: dict[str, Any] | None = None,
+    ) -> None:
+        """Record a download at queue time with final_status='active'.
+
+        On first queue: inserts a new row.
+        On retry (row already exists): resets the row back to 'active'
+        so the normal finalize path works when the retry completes.
+        """
+        normalized_task_id = _normalize_task_id(task_id)
+        normalized_user_id = normalize_optional_positive_int(user_id, "user_id")
+        normalized_request_id = normalize_optional_positive_int(request_id, "request_id")
+        normalized_source = normalize_optional_text(source)
+        if normalized_source is None:
+            msg = "source must be a non-empty string"
+            raise ValueError(msg)
+        normalized_title = normalize_optional_text(title)
+        if normalized_title is None:
+            msg = "title must be a non-empty string"
+            raise ValueError(msg)
+        normalized_origin = _normalize_origin(origin)
+        normalized_retry_payload = self._serialize_retry_payload(retry_payload)
+        recorded_at = now_utc_iso()
+
+        with self._lock:
+            conn = self._connect()
+            try:
+                conn.execute(
+                    """
+                INSERT INTO download_history (
+                    task_id, user_id, username, request_id,
+                    source, source_display_name,
+                    title, author, format, size, preview, content_type,
+                    origin, final_status,
+                    status_message, download_path, retry_payload,
+                    queued_at, terminal_at, downloads
+                )
+                VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'active', NULL, NULL, ?, ?, ?, ?)
+                ON CONFLICT(task_id) DO UPDATE SET
+                    final_status = 'active',
+                    status_message = NULL,
+                    download_path = NULL,
+                    retry_payload = excluded.retry_payload,
+                    terminal_at = ?,
+                    downloads = excluded.downloads
+                """,
+                    (
+                        normalized_task_id,
+                        normalized_user_id,
+                        normalize_optional_text(username),
+                        normalized_request_id,
+                        normalized_source,
+                        normalize_optional_text(source_display_name),
+                        normalized_title,
+                        normalize_optional_text(author),
+                        normalize_optional_text(file_format),
+                        normalize_optional_text(size),
+                        normalize_optional_text(preview),
+                        normalize_optional_text(content_type),
+                        normalized_origin,
+                        normalized_retry_payload,
+                        recorded_at,
+                        recorded_at,
+                        downloads,
+                        recorded_at,
+                    ),
+                )
+                conn.commit()
+            finally:
+                conn.close()
+
+    def finalize_download(
+        self,
+        *,
+        task_id: str,
+        final_status: str,
+        status_message: str | None = None,
+        download_path: str | None = None,
+        retry_payload: dict[str, Any] | None = None,
+    ) -> None:
+        """Update an existing download row to its terminal state."""
+        normalized_task_id = _normalize_task_id(task_id)
+        normalized_final_status = _normalize_final_status(final_status)
+        normalized_status_message = normalize_optional_text(status_message)
+        normalized_download_path = normalize_optional_text(download_path)
+        normalized_retry_payload = self._serialize_retry_payload(retry_payload)
+        effective_terminal_at = now_utc_iso()
+
+        with self._lock:
+            conn = self._connect()
+            try:
+                cursor = conn.execute(
+                    """
+                    UPDATE download_history
+                    SET final_status = ?,
+                        status_message = ?,
+                        download_path = ?,
+                        retry_payload = COALESCE(?, retry_payload),
+                        terminal_at = ?
+                    WHERE task_id = ? AND final_status = 'active'
+                    """,
+                    (
+                        normalized_final_status,
+                        normalized_status_message,
+                        normalized_download_path,
+                        normalized_retry_payload,
+                        effective_terminal_at,
+                        normalized_task_id,
+                    ),
+                )
+                rowcount = int(cursor.rowcount) if cursor.rowcount is not None else 0
+                if rowcount < 1:
+                    logger.warning(
+                        "finalize_download: no active row found for task_id=%s (may have been missed at queue time)",
+                        normalized_task_id,
+                    )
+                conn.commit()
+            finally:
+                conn.close()
+
+    def get_by_task_id(self, task_id: str) -> dict[str, Any] | None:
+        """Return a persisted download row for the given task id."""
+        normalized_task_id = _normalize_task_id(task_id)
+        conn = self._connect()
+        try:
+            row = conn.execute(
+                "SELECT * FROM download_history WHERE task_id = ?",
+                (normalized_task_id,),
+            ).fetchone()
+            return self._row_to_dict(row)
+        finally:
+            conn.close()
+
+    def list_recent(
+        self,
+        *,
+        user_id: int | None,
+        limit: int = 200,
+    ) -> list[dict[str, Any]]:
+        """Return recent persisted download rows, optionally scoped to one user."""
+        normalized_user_id = normalize_optional_positive_int(user_id, "user_id")
+        normalized_limit = _normalize_limit(limit, default=200, minimum=1, maximum=1000)
+        query = "SELECT * FROM download_history"
+        params: list[Any] = []
+        if normalized_user_id is not None:
+            query += " WHERE user_id = ?"
+            params.append(normalized_user_id)
+        query += " ORDER BY terminal_at DESC, id DESC LIMIT ?"
+        params.append(normalized_limit)
+
+        conn = self._connect()
+        try:
+            rows = conn.execute(query, params).fetchall()
+            result: list[dict[str, Any]] = []
+            for row in rows:
+                normalized = self._normalize_row_dict(dict(row))
+                if normalized is not None:
+                    result.append(normalized)
+            return result
+        finally:
+            conn.close()

shelfmark/core/external_user_linking.py

@@ -3,11 +3,13 @@
 from __future__ import annotations
 
 import re
-from typing import Any, Literal
+from typing import TYPE_CHECKING, Any, Literal
 
 from shelfmark.core.auth_modes import normalize_auth_source
 from shelfmark.core.logger import setup_logger
-from shelfmark.core.user_db import UserDB
+
+if TYPE_CHECKING:
+    from shelfmark.core.user_db import UserDB
 
 UNSET = object()
 
@@ -21,18 +23,18 @@ MatchReason = Literal[
 logger = setup_logger(__name__)
 
 
-def _normalize_username(value: Any) -> str:
+def _normalize_username(value: object) -> str:
     return str(value or "").strip()
 
 
-def _normalize_email(value: Any) -> str | None:
+def _normalize_email(value: object) -> str | None:
     if value is None:
         return None
     email = str(value).strip()
     return email or None
 
 
-def _normalize_display_name(value: Any) -> str | None:
+def _normalize_display_name(value: object) -> str | None:
     if value is None:
         return None
     name = str(value).strip()
@@ -43,11 +45,13 @@ def _email_key(value: str | None) -> str:
     return (value or "").strip().lower()
 
 
-def _normalize_role(value: Any) -> str:
+def _normalize_role(value: object) -> str:
     return "admin" if str(value or "").strip().lower() == "admin" else "user"
 
 
-def _get_by_subject(user_db: UserDB, subject_field: str | None, subject: str | None) -> dict[str, Any] | None:
+def _get_by_subject(
+    user_db: UserDB, subject_field: str | None, subject: str | None
+) -> dict[str, Any] | None:
     if not subject_field or not subject:
         return None
     if subject_field == "oidc_subject":
@@ -56,6 +60,7 @@ def _get_by_subject(user_db: UserDB, subject_field: str | None, subject: str | N
 
 
 def find_unique_user_by_email(user_db: UserDB, email: str | None) -> dict[str, Any] | None:
+    """Return the unique local user matching an email address, if any."""
     key = _email_key(_normalize_email(email))
     if not key:
         return None
@@ -83,10 +88,14 @@ def find_external_user_match(
         return by_subject, "subject_match"
 
     by_username = user_db.get_user(username=normalized_username)
-    if by_username and normalize_auth_source(
-        by_username.get("auth_source"),
-        by_username.get("oidc_subject"),
-    ) == auth_source:
+    if (
+        by_username
+        and normalize_auth_source(
+            by_username.get("auth_source"),
+            by_username.get("oidc_subject"),
+        )
+        == auth_source
+    ):
         return by_username, "existing_source_username_match"
 
     if allow_email_link:
@@ -99,6 +108,7 @@ def _build_updates(
     auth_source: str,
     role: str,
     sync_role: bool,
+    username: str | object,
     email: str | None | object,
     display_name: str | None | object,
     subject_field: str | None,
@@ -107,6 +117,8 @@ def _build_updates(
     updates: dict[str, Any] = {"auth_source": auth_source}
     if sync_role:
         updates["role"] = _normalize_role(role)
+    if username is not UNSET:
+        updates["username"] = _normalize_username(username)
     if email is not UNSET:
         updates["email"] = _normalize_email(email)
     if display_name is not UNSET:
@@ -116,10 +128,17 @@ def _build_updates(
     return updates
 
 
-def _next_suffix_username(user_db: UserDB, base_username: str) -> str:
+def _next_suffix_username(
+    user_db: UserDB,
+    base_username: str,
+    *,
+    exclude_user_id: int | None = None,
+) -> str:
     candidate = base_username
     suffix = 1
-    while user_db.get_user(username=candidate):
+    while existing := user_db.get_user(username=candidate):
+        if exclude_user_id is not None and int(existing.get("id") or 0) == exclude_user_id:
+            return candidate
         candidate = f"{base_username}_{suffix}"
         suffix += 1
     return candidate
@@ -133,13 +152,14 @@ def _find_existing_alias_user(
 ) -> dict[str, Any] | None:
     pattern = re.compile(rf"^{re.escape(alias_base)}(?:_\d+)?$")
     candidates = [
-        user for user in user_db.list_users()
+        user
+        for user in user_db.list_users()
         if pattern.match(str(user.get("username") or ""))
         and normalize_auth_source(user.get("auth_source"), user.get("oidc_subject")) == auth_source
     ]
     if not candidates:
         return None
-    return sorted(candidates, key=lambda user: int(user.get("id") or 0))[0]
+    return min(candidates, key=lambda user: int(user.get("id") or 0), default=None)
 
 
 def _resolve_create_username(
@@ -158,7 +178,11 @@ def _resolve_create_username(
         return None, existing, "username_collision_takeover"
 
     if strategy == "suffix":
-        return _next_suffix_username(user_db, requested_username), None, "username_collision_suffix"
+        return (
+            _next_suffix_username(user_db, requested_username),
+            None,
+            "username_collision_suffix",
+        )
 
     alias_base = f"{requested_username}{alias_suffix}"
     alias_existing = _find_existing_alias_user(
@@ -171,6 +195,38 @@ def _resolve_create_username(
     return _next_suffix_username(user_db, alias_base), None, "username_collision_alias"
 
 
+def _resolve_update_username(
+    user_db: UserDB,
+    *,
+    current_user: dict[str, Any],
+    requested_username: str,
+    strategy: CollisionStrategy,
+    alias_suffix: str,
+) -> str:
+    current_user_id = int(current_user["id"])
+    existing = user_db.get_user(username=requested_username)
+    if existing is None or int(existing.get("id") or 0) == current_user_id:
+        return requested_username
+
+    if strategy == "suffix":
+        return _next_suffix_username(
+            user_db,
+            requested_username,
+            exclude_user_id=current_user_id,
+        )
+    if strategy == "alias":
+        return _next_suffix_username(
+            user_db,
+            f"{requested_username}{alias_suffix}",
+            exclude_user_id=current_user_id,
+        )
+
+    # `takeover` can select an existing row during creation, but once an
+    # identity is already matched it must never replace a different username
+    # owner. Preserve the matched row's current collision-free name instead.
+    return str(current_user["username"])
+
+
 def upsert_external_user(
     user_db: UserDB,
     *,
@@ -183,6 +239,7 @@ def upsert_external_user(
     subject: str | None = None,
     allow_email_link: bool = False,
     sync_role: bool = True,
+    sync_username: bool = False,
     allow_create: bool = True,
     collision_strategy: CollisionStrategy = "takeover",
     alias_suffix: str | None = None,
@@ -197,7 +254,8 @@ def upsert_external_user(
     """
     normalized_username = _normalize_username(username)
     if not normalized_username:
-        raise ValueError("External username is required")
+        msg = "External username is required"
+        raise ValueError(msg)
 
     normalized_email = _normalize_email(email) if email is not UNSET else None
     normalized_display_name = (
@@ -214,10 +272,26 @@ def upsert_external_user(
         subject=subject,
         allow_email_link=allow_email_link,
     )
+    resolved_alias_suffix = alias_suffix or f"__{auth_source}"
+    update_username: str | object = UNSET
+    if (
+        matched is not None
+        and sync_username
+        and normalize_auth_source(matched.get("auth_source"), matched.get("oidc_subject"))
+        == auth_source
+    ):
+        update_username = _resolve_update_username(
+            user_db,
+            current_user=matched,
+            requested_username=normalized_username,
+            strategy=collision_strategy,
+            alias_suffix=resolved_alias_suffix,
+        )
     updates = _build_updates(
         auth_source=auth_source,
         role=normalized_role,
         sync_role=sync_role,
+        username=update_username,
         email=normalized_email if email is not UNSET else UNSET,
         display_name=normalized_display_name if display_name is not UNSET else UNSET,
         subject_field=subject_field,
@@ -227,22 +301,25 @@ def upsert_external_user(
         user_db.update_user(matched["id"], **updates)
         mapped = user_db.get_user(user_id=matched["id"]) or matched
         logger.info(
-            "External user mapped to existing Shelfmark user "
-            f"(source={auth_source}, context={context or 'unspecified'}, reason={match_reason}, "
-            f"external_username={normalized_username}, shelfmark_user_id={mapped['id']}, "
-            f"shelfmark_username={mapped['username']})"
+            "External user mapped to existing Shelfmark user (source=%s, context=%s, reason=%s, external_username=%s, shelfmark_user_id=%s, shelfmark_username=%s)",
+            auth_source,
+            context or "unspecified",
+            match_reason,
+            normalized_username,
+            mapped["id"],
+            mapped["username"],
         )
         return mapped, "updated"
 
     if not allow_create:
         logger.info(
-            "External user could not be mapped and creation is disabled "
-            f"(source={auth_source}, context={context or 'unspecified'}, "
-            f"external_username={normalized_username})"
+            "External user could not be mapped and creation is disabled (source=%s, context=%s, external_username=%s)",
+            auth_source,
+            context or "unspecified",
+            normalized_username,
         )
         return None, "not_found"
 
-    resolved_alias_suffix = alias_suffix or f"__{auth_source}"
     create_username, takeover_target, create_reason = _resolve_create_username(
         user_db,
         auth_source=auth_source,
@@ -254,10 +331,13 @@ def upsert_external_user(
         user_db.update_user(takeover_target["id"], **updates)
         mapped = user_db.get_user(user_id=takeover_target["id"]) or takeover_target
         logger.info(
-            "External user mapped to existing Shelfmark user "
-            f"(source={auth_source}, context={context or 'unspecified'}, reason={create_reason}, "
-            f"external_username={normalized_username}, shelfmark_user_id={mapped['id']}, "
-            f"shelfmark_username={mapped['username']})"
+            "External user mapped to existing Shelfmark user (source=%s, context=%s, reason=%s, external_username=%s, shelfmark_user_id=%s, shelfmark_username=%s)",
+            auth_source,
+            context or "unspecified",
+            create_reason,
+            normalized_username,
+            mapped["id"],
+            mapped["username"],
         )
         return mapped, "updated"
 
@@ -275,9 +355,12 @@ def upsert_external_user(
 
     created = user_db.create_user(**create_kwargs)
     logger.info(
-        "External user created Shelfmark user "
-        f"(source={auth_source}, context={context or 'unspecified'}, reason={create_reason}, "
-        f"external_username={normalized_username}, shelfmark_user_id={created['id']}, "
-        f"shelfmark_username={created['username']})"
+        "External user created Shelfmark user (source=%s, context=%s, reason=%s, external_username=%s, shelfmark_user_id=%s, shelfmark_username=%s)",
+        auth_source,
+        context or "unspecified",
+        create_reason,
+        normalized_username,
+        created["id"],
+        created["username"],
     )
     return created, "created"

shelfmark/core/image_cache.py

@@ -1,37 +1,45 @@
 """Disk-based image cache with LRU eviction."""
 
+import ipaddress
 import json
-import os
+import socket
 import threading
 import time
+from http import HTTPStatus
 from io import BytesIO
-from pathlib import Path
-from typing import Any, Dict, Optional, Tuple
+from typing import TYPE_CHECKING, Any
+from urllib.parse import urljoin, urlparse
 
 import requests
 
 from shelfmark.core.logger import setup_logger
+from shelfmark.core.request_helpers import coerce_int
+from shelfmark.download.network import get_ssl_verify
+
+if TYPE_CHECKING:
+    from pathlib import Path
 
 logger = setup_logger(__name__)
 
 # Image type detection via magic bytes
 IMAGE_SIGNATURES = {
-    b'\xff\xd8\xff': ('image/jpeg', 'jpg'),
-    b'\x89PNG\r\n\x1a\n': ('image/png', 'png'),
-    b'GIF87a': ('image/gif', 'gif'),
-    b'GIF89a': ('image/gif', 'gif'),
-    b'RIFF': ('image/webp', 'webp'),  # WebP starts with RIFF
+    b"\xff\xd8\xff": ("image/jpeg", "jpg"),
+    b"\x89PNG\r\n\x1a\n": ("image/png", "png"),
+    b"GIF87a": ("image/gif", "gif"),
+    b"GIF89a": ("image/gif", "gif"),
+    b"RIFF": ("image/webp", "webp"),  # WebP starts with RIFF
 }
 
 # HTTP headers for image fetching
 FETCH_HEADERS = {
-    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/129.0.0.0 Safari/537.36',
-    'Accept': 'image/webp,image/apng,image/*,*/*;q=0.8',
-    'Accept-Language': 'en-US,en;q=0.5',
+    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/129.0.0.0 Safari/537.36",
+    "Accept": "image/webp,image/apng,image/*,*/*;q=0.8",
+    "Accept-Language": "en-US,en;q=0.5",
 }
 
 # Maximum image size to fetch (5 MB)
 MAX_IMAGE_SIZE = 5 * 1024 * 1024
+MAX_REDIRECTS = 5
 
 # Negative cache TTL (for failed fetches) - 1 hour
 NEGATIVE_CACHE_TTL = 3600
@@ -40,8 +48,11 @@ NEGATIVE_CACHE_TTL = 3600
 # Short enough to retry soon, long enough to prevent spam during one page view
 TRANSIENT_CACHE_TTL = 60
 
+_MIN_WEBP_HEADER_LENGTH = 12
+HTTP_NOT_FOUND = HTTPStatus.NOT_FOUND
 
-def _detect_image_type(data: bytes) -> Optional[Tuple[str, str]]:
+
+def _detect_image_type(data: bytes) -> tuple[str, str] | None:
     """Detect image type from magic bytes.
 
     Args:
@@ -49,14 +60,15 @@ def _detect_image_type(data: bytes) -> Optional[Tuple[str, str]]:
 
     Returns:
         Tuple of (content_type, extension) or None if not recognized
+
     """
     for signature, (content_type, ext) in IMAGE_SIGNATURES.items():
         if data.startswith(signature):
             return content_type, ext
 
     # Special case for WebP - check for WEBP after RIFF
-    if data.startswith(b'RIFF') and len(data) > 12 and data[8:12] == b'WEBP':
-        return 'image/webp', 'webp'
+    if data.startswith(b"RIFF") and len(data) > _MIN_WEBP_HEADER_LENGTH and data[8:12] == b"WEBP":
+        return "image/webp", "webp"
 
     return None
 
@@ -64,20 +76,21 @@ def _detect_image_type(data: bytes) -> Optional[Tuple[str, str]]:
 class ImageCacheService:
     """Persistent image cache with LRU eviction and TTL support."""
 
-    def __init__(self, cache_dir: Path, max_size_mb: int = 500, ttl_seconds: int = 0):
+    def __init__(self, cache_dir: Path, max_size_mb: int = 500, ttl_seconds: int = 0) -> None:
         """Initialize the image cache.
 
         Args:
             cache_dir: Directory to store cached images
             max_size_mb: Maximum cache size in megabytes
             ttl_seconds: Time-to-live in seconds (0 = forever)
+
         """
         self.cache_dir = cache_dir
         self.max_size_bytes = max_size_mb * 1024 * 1024
         self.ttl_seconds = ttl_seconds
         self.index_path = cache_dir / "cache_index.json"
         self._lock = threading.RLock()
-        self._index: Dict[str, Dict[str, Any]] = {}
+        self._index: dict[str, dict[str, Any]] = {}
 
         # Stats tracking
         self._hits = 0
@@ -97,9 +110,9 @@ class ImageCacheService:
             return
 
         try:
-            with open(self.index_path, 'r') as f:
+            with self.index_path.open() as f:
                 self._index = json.load(f)
-        except (json.JSONDecodeError, IOError):
+        except OSError, json.JSONDecodeError:
             self._index = {}
 
     def _sync_index_with_files(self) -> None:
@@ -109,12 +122,12 @@ class ImageCacheService:
         - Removes entries for files that no longer exist (non-negative only)
         - Preserves negative cache entries (they have no files)
         """
-        image_extensions = {'.jpg', '.jpeg', '.png', '.gif', '.webp'}
+        image_extensions = {".jpg", ".jpeg", ".png", ".gif", ".webp"}
         added_count = 0
         removed_count = 0
 
         # Build set of files that exist on disk
-        existing_files: Dict[str, Path] = {}
+        existing_files: dict[str, Path] = {}
         for file_path in self.cache_dir.iterdir():
             if not file_path.is_file():
                 continue
@@ -127,31 +140,31 @@ class ImageCacheService:
             if cache_id in self._index:
                 continue
 
-            ext = file_path.suffix.lstrip('.')
+            ext = file_path.suffix.lstrip(".")
             stat = file_path.stat()
 
             # Detect content type
             try:
-                with open(file_path, 'rb') as f:
+                with file_path.open("rb") as f:
                     header = f.read(16)
                 detected = _detect_image_type(header)
-                content_type = detected[0] if detected else f'image/{ext}'
-            except IOError:
-                content_type = f'image/{ext}'
+                content_type = detected[0] if detected else f"image/{ext}"
+            except OSError:
+                content_type = f"image/{ext}"
 
             self._index[cache_id] = {
-                'ext': ext,
-                'content_type': content_type,
-                'size': stat.st_size,
-                'cached_at': stat.st_mtime,
-                'accessed_at': stat.st_mtime,
+                "ext": ext,
+                "content_type": content_type,
+                "size": stat.st_size,
+                "cached_at": stat.st_mtime,
+                "accessed_at": stat.st_mtime,
             }
             added_count += 1
 
         # Remove index entries for missing files (skip negative cache entries)
         stale_entries = []
         for cache_id, entry in self._index.items():
-            if entry.get('negative', False):
+            if entry.get("negative", False):
                 continue  # Negative entries don't have files
             if cache_id not in existing_files:
                 stale_entries.append(cache_id)
@@ -167,39 +180,39 @@ class ImageCacheService:
         """Save cache index to disk."""
         try:
             # Write to temp file first, then rename for atomicity
-            temp_path = self.index_path.with_suffix('.tmp')
-            with open(temp_path, 'w') as f:
+            temp_path = self.index_path.with_suffix(".tmp")
+            with temp_path.open("w") as f:
                 json.dump(self._index, f)
             temp_path.rename(self.index_path)
-        except IOError:
+        except OSError:
             pass
 
     def _get_image_path(self, cache_id: str, ext: str) -> Path:
         """Get the file path for a cached image."""
         return self.cache_dir / f"{cache_id}.{ext}"
 
-    def _is_expired(self, entry: Dict[str, Any]) -> bool:
+    def _is_expired(self, entry: dict[str, Any]) -> bool:
         """Check if a cache entry is expired."""
         if self.ttl_seconds == 0:
             return False
-        return (time.time() - entry.get('cached_at', 0)) > self.ttl_seconds
+        return (time.time() - entry.get("cached_at", 0)) > self.ttl_seconds
 
-    def _is_negative_expired(self, entry: Dict[str, Any]) -> bool:
+    def _is_negative_expired(self, entry: dict[str, Any]) -> bool:
         """Check if a negative cache entry is expired.
 
         Transient failures (timeouts) expire after TRANSIENT_CACHE_TTL (60s).
         Permanent failures (404s) expire after NEGATIVE_CACHE_TTL (1 hour).
         """
-        if not entry.get('negative', False):
+        if not entry.get("negative", False):
             return False
 
-        cached_at = entry.get('cached_at', 0)
-        ttl = TRANSIENT_CACHE_TTL if entry.get('transient', False) else NEGATIVE_CACHE_TTL
+        cached_at = entry.get("cached_at", 0)
+        ttl = TRANSIENT_CACHE_TTL if entry.get("transient", False) else NEGATIVE_CACHE_TTL
         return (time.time() - cached_at) > ttl
 
     def _calculate_total_size(self) -> int:
         """Calculate total size of cached images."""
-        return sum(entry.get('size', 0) for entry in self._index.values())
+        return sum(entry.get("size", 0) for entry in self._index.values())
 
     def _evict_if_needed(self, required_space: int = 0) -> None:
         """Evict old entries if cache is over size limit.
@@ -213,10 +226,7 @@ class ImageCacheService:
             return
 
         # Sort entries by accessed_at (oldest first)
-        sorted_entries = sorted(
-            self._index.items(),
-            key=lambda x: x[1].get('accessed_at', 0)
-        )
+        sorted_entries = sorted(self._index.items(), key=lambda x: x[1].get("accessed_at", 0))
 
         evicted_count = 0
         for cache_id, entry in sorted_entries:
@@ -224,23 +234,23 @@ class ImageCacheService:
                 break
 
             # Delete the image file
-            ext = entry.get('ext', 'jpg')
+            ext = entry.get("ext", "jpg")
             image_path = self._get_image_path(cache_id, ext)
             try:
                 if image_path.exists():
                     image_path.unlink()
-            except IOError:
+            except OSError:
                 pass
 
             # Update tracking
-            current_size -= entry.get('size', 0)
+            current_size -= entry.get("size", 0)
             del self._index[cache_id]
             evicted_count += 1
 
         if evicted_count > 0:
             self._save_index()
 
-    def get(self, cache_id: str) -> Optional[Tuple[bytes, str]]:
+    def get(self, cache_id: str) -> tuple[bytes, str] | None:
         """Get a cached image.
 
         Args:
@@ -248,6 +258,7 @@ class ImageCacheService:
 
         Returns:
             Tuple of (image_data, content_type) or None if not cached/expired
+
         """
         with self._lock:
             entry = self._index.get(cache_id)
@@ -261,7 +272,7 @@ class ImageCacheService:
                     return None
 
             # Check for negative cache (failed fetch)
-            if entry.get('negative', False):
+            if entry.get("negative", False):
                 if self._is_negative_expired(entry):
                     # Negative cache expired, allow retry
                     del self._index[cache_id]
@@ -274,12 +285,12 @@ class ImageCacheService:
             # Check for expired entry
             if self._is_expired(entry):
                 # Remove expired entry
-                ext = entry.get('ext', 'jpg')
+                ext = entry.get("ext", "jpg")
                 image_path = self._get_image_path(cache_id, ext)
                 try:
                     if image_path.exists():
                         image_path.unlink()
-                except IOError:
+                except OSError:
                     pass
                 del self._index[cache_id]
                 self._save_index()
@@ -287,9 +298,10 @@ class ImageCacheService:
                 return None
 
             # Try to read the cached image
-            ext = entry.get('ext', 'jpg')
-            content_type = entry.get('content_type', 'image/jpeg')
+            ext = entry.get("ext", "jpg")
+            content_type = entry.get("content_type", "image/jpeg")
             image_path = self._get_image_path(cache_id, ext)
+            result: tuple[bytes, str] | None = None
 
             try:
                 if not image_path.exists():
@@ -299,19 +311,20 @@ class ImageCacheService:
                     self._misses += 1
                     return None
 
-                with open(image_path, 'rb') as f:
+                with image_path.open("rb") as f:
                     data = f.read()
 
                 # Update accessed time
-                entry['accessed_at'] = time.time()
+                entry["accessed_at"] = time.time()
                 self._save_index()
+                result = data, content_type
 
-                self._hits += 1
-                return data, content_type
-
-            except IOError:
+            except OSError:
                 self._misses += 1
                 return None
+            else:
+                self._hits += 1
+                return result
 
     def put(self, cache_id: str, data: bytes, content_type: str) -> bool:
         """Store an image in the cache.
@@ -323,24 +336,24 @@ class ImageCacheService:
 
         Returns:
             True if stored successfully
+
         """
         with self._lock:
             # Detect image type for extension
             detected = _detect_image_type(data)
             if detected:
                 content_type, ext = detected
+            # Fall back to content-type header
+            elif "jpeg" in content_type or "jpg" in content_type:
+                ext = "jpg"
+            elif "png" in content_type:
+                ext = "png"
+            elif "gif" in content_type:
+                ext = "gif"
+            elif "webp" in content_type:
+                ext = "webp"
             else:
-                # Fall back to content-type header
-                if 'jpeg' in content_type or 'jpg' in content_type:
-                    ext = 'jpg'
-                elif 'png' in content_type:
-                    ext = 'png'
-                elif 'gif' in content_type:
-                    ext = 'gif'
-                elif 'webp' in content_type:
-                    ext = 'webp'
-                else:
-                    ext = 'jpg'  # Default
+                ext = "jpg"  # Default
 
             image_size = len(data)
 
@@ -350,36 +363,37 @@ class ImageCacheService:
             # Write image to disk
             image_path = self._get_image_path(cache_id, ext)
             try:
-                with open(image_path, 'wb') as f:
+                with image_path.open("wb") as f:
                     f.write(data)
-            except IOError:
+            except OSError:
                 return False
 
             # Update index
             now = time.time()
             self._index[cache_id] = {
-                'ext': ext,
-                'content_type': content_type,
-                'size': image_size,
-                'cached_at': now,
-                'accessed_at': now,
-                'negative': False,
+                "ext": ext,
+                "content_type": content_type,
+                "size": image_size,
+                "cached_at": now,
+                "accessed_at": now,
+                "negative": False,
             }
             self._save_index()
             return True
 
-    def put_negative(self, cache_id: str, transient: bool = False) -> None:
+    def put_negative(self, cache_id: str, *, transient: bool = False) -> None:
         """Store a negative cache entry (failed fetch).
 
         Args:
             cache_id: Cache key
             transient: If True, uses shorter TTL (for timeouts/connection errors)
+
         """
         with self._lock:
             self._index[cache_id] = {
-                'negative': True,
-                'transient': transient,
-                'cached_at': time.time(),
+                "negative": True,
+                "transient": transient,
+                "cached_at": time.time(),
             }
             self._save_index()
 
@@ -391,6 +405,7 @@ class ImageCacheService:
 
         Returns:
             True if entry existed and was deleted
+
         """
         with self._lock:
             entry = self._index.get(cache_id)
@@ -398,13 +413,13 @@ class ImageCacheService:
                 return False
 
             # Delete file if it exists
-            if not entry.get('negative', False):
-                ext = entry.get('ext', 'jpg')
+            if not entry.get("negative", False):
+                ext = entry.get("ext", "jpg")
                 image_path = self._get_image_path(cache_id, ext)
                 try:
                     if image_path.exists():
                         image_path.unlink()
-                except IOError:
+                except OSError:
                     pass
 
             del self._index[cache_id]
@@ -416,19 +431,20 @@ class ImageCacheService:
 
         Returns:
             Number of entries cleared
+
         """
         with self._lock:
             count = len(self._index)
 
             # Delete all image files
             for cache_id, entry in self._index.items():
-                if not entry.get('negative', False):
-                    ext = entry.get('ext', 'jpg')
+                if not entry.get("negative", False):
+                    ext = entry.get("ext", "jpg")
                     image_path = self._get_image_path(cache_id, ext)
                     try:
                         if image_path.exists():
                             image_path.unlink()
-                    except IOError:
+                    except OSError:
                         pass
 
             # Clear index
@@ -441,31 +457,113 @@ class ImageCacheService:
 
             return count
 
-    def stats(self) -> Dict[str, Any]:
+    def stats(self) -> dict[str, Any]:
         """Get cache statistics.
 
         Returns:
             Dict with size, count, hit rate, etc.
+
         """
         with self._lock:
             total_size = self._calculate_total_size()
             entry_count = len(self._index)
-            negative_count = sum(1 for e in self._index.values() if e.get('negative', False))
+            negative_count = sum(1 for e in self._index.values() if e.get("negative", False))
             total_requests = self._hits + self._misses
             hit_rate = (self._hits / total_requests * 100) if total_requests > 0 else 0
 
             return {
-                'entry_count': entry_count,
-                'negative_count': negative_count,
-                'total_size_bytes': total_size,
-                'total_size_mb': round(total_size / (1024 * 1024), 2),
-                'max_size_mb': self.max_size_bytes / (1024 * 1024),
-                'hits': self._hits,
-                'misses': self._misses,
-                'hit_rate': round(hit_rate, 1),
+                "entry_count": entry_count,
+                "negative_count": negative_count,
+                "total_size_bytes": total_size,
+                "total_size_mb": round(total_size / (1024 * 1024), 2),
+                "max_size_mb": self.max_size_bytes / (1024 * 1024),
+                "hits": self._hits,
+                "misses": self._misses,
+                "hit_rate": round(hit_rate, 1),
             }
 
-    def fetch_and_cache(self, cache_id: str, url: str) -> Optional[Tuple[bytes, str]]:
+    @staticmethod
+    def _prepare_safe_url(url: str) -> str | None:
+        """Prepare and validate a URL before fetching it."""
+        if "\\" in url or any(ord(char) < 32 for char in url):
+            return None
+
+        try:
+            prepared = requests.Request("GET", url).prepare()
+            prepared_url = prepared.url
+            if not isinstance(prepared_url, str):
+                return None
+            parsed = urlparse(prepared_url)
+            hostname = parsed.hostname
+        except requests.exceptions.RequestException, ValueError:
+            return None
+
+        if not prepared_url:
+            return None
+
+        if "\\" in prepared_url or any(ord(char) < 32 for char in prepared_url):
+            return None
+
+        netloc_lower = parsed.netloc.lower()
+        if "%2f" in netloc_lower or "%5c" in netloc_lower:
+            return None
+
+        if parsed.scheme not in ("http", "https"):
+            return None
+        if not hostname:
+            return None
+
+        try:
+            resolved = socket.getaddrinfo(hostname, None)
+            for _, _, _, _, sockaddr in resolved:
+                ip = ipaddress.ip_address(sockaddr[0])
+                if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
+                    return None
+        except socket.gaierror, ValueError:
+            return None
+
+        return prepared_url
+
+    @staticmethod
+    def _is_safe_url(url: str) -> bool:
+        """Check that a URL is safe to fetch (no SSRF to internal resources)."""
+        return ImageCacheService._prepare_safe_url(url) is not None
+
+    def _fetch_safe_response(self, url: str) -> requests.Response | None:
+        """Fetch a URL after validating the initial URL and each redirect."""
+        current_url = self._prepare_safe_url(url)
+        if not current_url:
+            logger.warning("Blocked request to disallowed URL: %s", url)
+            return None
+
+        for _ in range(MAX_REDIRECTS + 1):
+            response = requests.get(
+                current_url,
+                timeout=(5, 10),
+                headers=FETCH_HEADERS,
+                stream=True,
+                verify=get_ssl_verify(current_url),
+                allow_redirects=False,
+            )
+
+            if not response.is_redirect:
+                return response
+
+            location = response.headers.get("location")
+            response.close()
+            if not location:
+                return None
+
+            redirect_url = urljoin(current_url, location)
+            next_url = self._prepare_safe_url(redirect_url)
+            if not next_url:
+                logger.warning("Blocked redirect to disallowed URL: %s", redirect_url)
+                return None
+            current_url = next_url
+
+        return None
+
+    def fetch_and_cache(self, cache_id: str, url: str) -> tuple[bytes, str] | None:
         """Fetch an image from URL and cache it.
 
         Args:
@@ -474,20 +572,18 @@ class ImageCacheService:
 
         Returns:
             Tuple of (image_data, content_type) or None on failure
-        """
-        try:
 
-            response = requests.get(
-                url,
-                timeout=(5, 10),
-                headers=FETCH_HEADERS,
-                stream=True,
-            )
+        """
+        cached_data: tuple[bytes, str] | None = None
+        try:
+            response = self._fetch_safe_response(url)
+            if response is None:
+                return None
             response.raise_for_status()
 
             # Validate content type
-            content_type = response.headers.get('content-type', '')
-            if not content_type.startswith('image/'):
+            content_type = response.headers.get("content-type", "")
+            if not content_type.startswith("image/"):
                 self.put_negative(cache_id)
                 return None
 
@@ -511,9 +607,7 @@ class ImageCacheService:
                 detected = _detect_image_type(image_data)
                 if detected:
                     content_type = detected[0]
-                return image_data, content_type
-
-            return None
+                cached_data = image_data, content_type
 
         except requests.exceptions.Timeout:
             self.put_negative(cache_id, transient=True)
@@ -522,15 +616,17 @@ class ImageCacheService:
             self.put_negative(cache_id, transient=True)
             return None
         except requests.exceptions.HTTPError as e:
-            is_404 = e.response is not None and e.response.status_code == 404
+            is_404 = e.response is not None and e.response.status_code == HTTP_NOT_FOUND
             self.put_negative(cache_id, transient=not is_404)
             return None
-        except Exception:
+        except requests.exceptions.RequestException:
             return None
+        else:
+            return cached_data
 
 
 # Singleton instance (initialized lazily when config is available)
-_instance: Optional[ImageCacheService] = None
+_instance: ImageCacheService | None = None
 _instance_lock = threading.Lock()
 
 
@@ -544,12 +640,12 @@ def get_image_cache() -> ImageCacheService:
     if _instance is None:
         with _instance_lock:
             if _instance is None:
-                from shelfmark.core.config import config
                 from shelfmark.config.env import CONFIG_DIR
+                from shelfmark.core.config import config
 
                 cache_dir = CONFIG_DIR / "covers"
-                max_size_mb = config.get("COVERS_CACHE_MAX_SIZE_MB", 500)
-                ttl_days = config.get("COVERS_CACHE_TTL", 0)
+                max_size_mb = coerce_int(config.get("COVERS_CACHE_MAX_SIZE_MB", 500), 500)
+                ttl_days = coerce_int(config.get("COVERS_CACHE_TTL", 0), 0)
                 ttl_seconds = ttl_days * 86400 if ttl_days > 0 else 0
 
                 _instance = ImageCacheService(
@@ -557,7 +653,12 @@ def get_image_cache() -> ImageCacheService:
                     max_size_mb=max_size_mb,
                     ttl_seconds=ttl_seconds,
                 )
-                logger.debug(f"Initialized image cache: {cache_dir} (max {max_size_mb}MB, TTL {ttl_days} days)")
+                logger.debug(
+                    "Initialized image cache: %s (max %sMB, TTL %s days)",
+                    cache_dir,
+                    max_size_mb,
+                    ttl_days,
+                )
 
     return _instance
 

shelfmark/core/languages.py

@@ -0,0 +1,138 @@
+"""Canonical language resolution shared by every release source.
+
+Release sources report a language in whatever shape their upstream uses: a
+two-letter code, an ISO 639-2 three-letter code in either the bibliographic or
+terminological form, or an English name. They all need the same ISO 639-1 code
+out the other side, so the aliases live in one place (``data/book-languages.json``)
+and adding a language means editing one file.
+"""
+
+import json
+import threading
+import unicodedata
+from pathlib import Path
+
+from shelfmark.core.logger import setup_logger
+
+logger = setup_logger(__name__)
+
+LANGUAGE_DATA_PATH = Path(__file__).resolve().parents[1].parent / "data" / "book-languages.json"
+
+# Values a source uses to mean "we could not tell".
+LANGUAGE_PLACEHOLDERS = frozenset({"", "-", "--", "unknown", "unk", "n/a", "na", "none", "null"})
+
+_ALIAS_TO_CODE: dict[str, str] | None = None
+_CODE_TO_NAME: dict[str, str] | None = None
+_LOCK = threading.Lock()
+
+
+# Separators that stand in for the hyphen in a subtag. The dashes turn up in
+# codes copied from web pages -- "zh‑Hant" used U+2011, which renders close
+# enough to both a hyphen and an underscore to go unnoticed -- and the
+# underscore is the spelling Direct Download accepted before this module existed.
+_SUBTAG_SEPARATORS = dict.fromkeys(map(ord, "‐‑‒–—―−﹘﹣－_"), "-")
+
+
+def _fold(value: str) -> str:
+    """Casefold, strip accents, and normalize subtag separators, so 'Español'
+    and 'espanol', or 'zh-Hant', 'zh‑Hant' and 'zh_Hant', all match."""
+    decomposed = unicodedata.normalize("NFKD", value).translate(_SUBTAG_SEPARATORS)
+    stripped = "".join(ch for ch in decomposed if not unicodedata.combining(ch))
+    return " ".join(stripped.split()).casefold()
+
+
+def _load() -> tuple[dict[str, str], dict[str, str]]:
+    global _ALIAS_TO_CODE, _CODE_TO_NAME
+
+    if _ALIAS_TO_CODE is not None and _CODE_TO_NAME is not None:
+        return _ALIAS_TO_CODE, _CODE_TO_NAME
+
+    with _LOCK:
+        if _ALIAS_TO_CODE is not None and _CODE_TO_NAME is not None:
+            return _ALIAS_TO_CODE, _CODE_TO_NAME
+
+        alias_to_code: dict[str, str] = {}
+        code_to_name: dict[str, str] = {}
+
+        try:
+            raw = json.loads(LANGUAGE_DATA_PATH.read_text(encoding="utf-8"))
+        except OSError, ValueError:
+            logger.exception("Failed to load language data from %s", LANGUAGE_DATA_PATH)
+            raw = []
+
+        if not isinstance(raw, list):
+            logger.warning("Language data at %s is not a list", LANGUAGE_DATA_PATH)
+            raw = []
+
+        for item in raw:
+            if not isinstance(item, dict):
+                continue
+            code = str(item.get("code") or "").strip()
+            name = str(item.get("language") or "").strip()
+            if not code:
+                continue
+
+            code_to_name.setdefault(code, name or code)
+
+            for candidate in (code, name, *(item.get("aliases") or [])):
+                folded = _fold(str(candidate))
+                if folded and folded not in LANGUAGE_PLACEHOLDERS:
+                    alias_to_code.setdefault(folded, code)
+
+        _ALIAS_TO_CODE = alias_to_code
+        _CODE_TO_NAME = code_to_name
+        return alias_to_code, code_to_name
+
+
+def normalize_language(value: object) -> str | None:
+    """Resolve any known spelling of a language to its ISO 639-1 code.
+
+    Accepts a two-letter code, an ISO 639-2 three-letter code in either the
+    bibliographic or terminological form, or an English name. Returns None for
+    anything unrecognised or for the placeholders a source uses to say it does
+    not know, so callers can treat "no language" uniformly.
+    """
+    if value is None:
+        return None
+
+    folded = _fold(str(value))
+    if not folded or folded in LANGUAGE_PLACEHOLDERS:
+        return None
+
+    alias_to_code, _ = _load()
+    return alias_to_code.get(folded)
+
+
+def language_name(code: str | None) -> str | None:
+    """Return the English name for a language code, or None if unknown."""
+    if not code:
+        return None
+
+    _, code_to_name = _load()
+    return code_to_name.get(str(code).strip())
+
+
+def language_alias_map() -> dict[str, str]:
+    """Every known alias mapped to its code, for callers doing their own matching.
+
+    Direct Download scans free-text paths and needs the whole alias set up front
+    to look for, rather than resolving one candidate at a time.
+    """
+    alias_to_code, _ = _load()
+    return dict(alias_to_code)
+
+
+def supported_book_languages() -> list[dict[str, str]]:
+    """The selectable languages, as ``{"language": ..., "code": ...}``.
+
+    Aliases are an implementation detail of resolution, so they are left out of
+    what the settings dropdown and the API hand to clients.
+    """
+    _, code_to_name = _load()
+    return [{"language": name, "code": code} for code, name in code_to_name.items()]
+
+
+def known_language_codes() -> frozenset[str]:
+    """Every ISO 639-1 code the bundled language data defines."""
+    _, code_to_name = _load()
+    return frozenset(code_to_name)

shelfmark/core/logger.py

@@ -2,62 +2,91 @@
 
 import logging
 import sys
-from pathlib import Path
+from collections.abc import Mapping
 from logging.handlers import RotatingFileHandler
-from typing import Any
+from threading import Lock
+from typing import TYPE_CHECKING
 
-from shelfmark.config.env import LOG_FILE, ENABLE_LOGGING, LOG_LEVEL
+from shelfmark.config.env import ENABLE_LOGGING, LOG_FILE, LOG_LEVEL
+
+if TYPE_CHECKING:
+    from pathlib import Path
+
+
+_file_handlers: dict[Path, RotatingFileHandler] = {}
+_file_handlers_lock = Lock()
 
 
 class CustomLogger(logging.Logger):
     """Custom logger class with additional error_trace method."""
 
-    def error_trace(self, msg: Any, *args: Any, **kwargs: Any) -> None:
+    def error_trace(self, msg: object, *args: object, **kwargs: object) -> None:
         """Log an error message with full stack trace."""
         self.log_resource_usage()
-        kwargs.pop('exc_info', None)
-        self.error(msg, *args, exc_info=True, **kwargs)
+        stack_info, stacklevel, extra = _extract_log_kwargs(kwargs)
+        self.error(
+            msg,
+            *args,
+            exc_info=True,
+            stack_info=stack_info,
+            stacklevel=stacklevel,
+            extra=extra,
+        )
 
-    def warning_trace(self, msg: Any, *args: Any, **kwargs: Any) -> None:
-        """Log a warning message with full stack trace."""
-        self.log_resource_usage()
-        kwargs.pop('exc_info', None)
-        self.warning(msg, *args, exc_info=True, **kwargs)
-
-    def info_trace(self, msg: Any, *args: Any, **kwargs: Any) -> None:
-        """Log an info message (stack trace only if exception active)."""
-        kwargs.pop('exc_info', None)
-        # Only include exc_info if there's actually an exception
-        has_exception = sys.exc_info()[0] is not None
-        self.info(msg, *args, exc_info=has_exception, **kwargs)
-
-    def debug_trace(self, msg: Any, *args: Any, **kwargs: Any) -> None:
+    def debug_trace(self, msg: object, *args: object, **kwargs: object) -> None:
         """Log a debug message (stack trace only if exception active)."""
-        kwargs.pop('exc_info', None)
+        stack_info, stacklevel, extra = _extract_log_kwargs(kwargs)
         # Only include exc_info if there's actually an exception
         has_exception = sys.exc_info()[0] is not None
-        self.debug(msg, *args, exc_info=has_exception, **kwargs)
+        self.debug(
+            msg,
+            *args,
+            exc_info=has_exception,
+            stack_info=stack_info,
+            stacklevel=stacklevel,
+            extra=extra,
+        )
 
-    def log_resource_usage(self):
-        # Best-effort only; this should never raise during exception logging.
+    def log_resource_usage(self) -> None:
+        """Log best-effort CPU and memory usage for the current container."""
         try:
             import psutil
+        except ImportError:
+            return
+
+        # Best-effort only; this should never raise during exception logging.
+        try:
+
+            def _get_process_rss_mb(proc: object) -> float | None:
+                try:
+                    proc_info = getattr(proc, "info", None)
+                    if not isinstance(proc_info, Mapping):
+                        return None
+                    mem = proc_info.get("memory_info")
+                    rss = getattr(mem, "rss", None)
+                    if isinstance(rss, int | float):
+                        return rss / (1024 * 1024)
+                except (
+                    psutil.NoSuchProcess,
+                    psutil.AccessDenied,
+                    KeyError,
+                    AttributeError,
+                ):
+                    return None
+                return None
 
             # Sum RSS of all processes for actual app memory (container-friendly),
             # but fall back gracefully on platforms that restrict process enumeration.
             app_memory_mb = 0.0
             try:
-                for proc in psutil.process_iter(['memory_info']):
-                    try:
-                        mem = proc.info.get('memory_info')
-                        if mem:
-                            app_memory_mb += mem.rss / (1024 * 1024)
-                    except (psutil.NoSuchProcess, psutil.AccessDenied, KeyError, AttributeError):
-                        continue
-            except (PermissionError, psutil.AccessDenied, OSError):
+                for proc in psutil.process_iter(["memory_info"]):
+                    proc_rss_mb = _get_process_rss_mb(proc)
+                    if proc_rss_mb is not None:
+                        app_memory_mb += proc_rss_mb
+            except PermissionError, psutil.AccessDenied, OSError:
                 try:
                     app_memory_mb = psutil.Process().memory_info().rss / (1024 * 1024)
-                except Exception:
+                except AttributeError, OSError, psutil.Error:
                     app_memory_mb = 0.0
 
             memory = psutil.virtual_memory()
@@ -68,11 +97,52 @@ class CustomLogger(logging.Logger):
                 f"Container Memory: App={app_memory_mb:.2f} MB, System={system_used_mb:.2f} MB, "
                 f"Available={available_mb:.2f} MB, CPU: {cpu_percent:.2f}%"
             )
-        except Exception:
+        except AttributeError, OSError, psutil.Error:
             # Avoid breaking the original log call if psutil is missing or restricted.
             return
 
 
+def _extract_log_kwargs(
+    kwargs: Mapping[str, object],
+) -> tuple[bool, int, Mapping[str, object] | None]:
+    stack_info = kwargs.get("stack_info")
+    normalized_stack_info = stack_info if isinstance(stack_info, bool) else False
+
+    stacklevel = kwargs.get("stacklevel")
+    normalized_stacklevel = stacklevel if isinstance(stacklevel, int) else 1
+
+    extra = kwargs.get("extra")
+    normalized_extra = _normalize_log_extra(extra)
+
+    return normalized_stack_info, normalized_stacklevel, normalized_extra
+
+
+def _normalize_log_extra(value: object) -> Mapping[str, object] | None:
+    if not isinstance(value, Mapping):
+        return None
+
+    if all(isinstance(key, str) for key in value):
+        return value
+
+    return None
+
+
+def _get_file_handler(log_file: Path, formatter: logging.Formatter) -> RotatingFileHandler:
+    """Return the process-wide rotating handler for a log file."""
+    with _file_handlers_lock:
+        handler = _file_handlers.get(log_file)
+        if handler is None:
+            log_file.parent.mkdir(parents=True, exist_ok=True)
+            handler = RotatingFileHandler(
+                log_file,
+                maxBytes=10485760,  # 10MB
+                backupCount=5,
+            )
+            handler.setFormatter(formatter)
+            _file_handlers[log_file] = handler
+        return handler
+
+
 def setup_logger(name: str, log_file: Path = LOG_FILE) -> CustomLogger:
     """Set up and configure a logger instance.
 
@@ -82,6 +152,7 @@ def setup_logger(name: str, log_file: Path = LOG_FILE) -> CustomLogger:
 
     Returns:
         CustomLogger: Configured logger instance with error_trace method
+
     """
     # Register our custom logger class
     logging.setLoggerClass(CustomLogger)
@@ -92,36 +163,29 @@ def setup_logger(name: str, log_file: Path = LOG_FILE) -> CustomLogger:
     logger.setLevel(log_level)
 
     formatter = logging.Formatter(
-        '%(asctime)s - %(name)s - %(levelname)s - %(filename)s:%(lineno)d - %(message)s'
+        "%(asctime)s - %(name)s - %(levelname)s - %(filename)s:%(lineno)d - %(message)s"
     )
 
     # Console handler for Docker output
     console_handler = logging.StreamHandler(sys.stdout)
     console_handler.setFormatter(formatter)
     console_handler.setLevel(log_level)
-    console_handler.addFilter(lambda record: record.levelno < logging.ERROR)  # Only allow logs below ERROR to stdout
+    console_handler.addFilter(
+        lambda record: record.levelno < logging.ERROR
+    )  # Only allow logs below ERROR to stdout
     logger.addHandler(console_handler)
 
     # Error handler for stderr
     error_handler = logging.StreamHandler(sys.stderr)
-    error_handler.setLevel(logging.ERROR) # Error and above go to stderr
+    error_handler.setLevel(logging.ERROR)  # Error and above go to stderr
     error_handler.setFormatter(formatter)
     logger.addHandler(error_handler)
 
     # File handler if log file is specified
     try:
         if ENABLE_LOGGING:
-            # Create log directory if it doesn't exist
-            log_dir = log_file.parent
-            log_dir.mkdir(parents=True, exist_ok=True)
-            file_handler = RotatingFileHandler(
-                log_file,
-                maxBytes=10485760,  # 10MB
-                backupCount=5
-            )
-            file_handler.setFormatter(formatter)
-            logger.addHandler(file_handler)
-    except Exception as e:
+            logger.addHandler(_get_file_handler(log_file, formatter))
+    except (OSError, TypeError, ValueError) as e:
         logger.error_trace(f"Failed to create log file: {e}", exc_info=True)
 
     return logger

shelfmark/core/mirrors.py

@@ -1,254 +1,263 @@
-"""Centralized mirror configuration for all download sources."""
+"""Centralized mirror configuration for direct-download sources."""
 
-from typing import List
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
 
 from shelfmark.core.utils import normalize_http_url
 
-# Lazy import to avoid circular imports
+if TYPE_CHECKING:
+    from shelfmark.core.config import Config
+
 _config_module = None
 
 
-def _get_config():
+def _get_config() -> Config:
     """Lazy import of config module to avoid circular imports."""
     global _config_module
     if _config_module is None:
         from shelfmark.core.config import config
+
         _config_module = config
     return _config_module
 
 
-# Default mirror lists (hardcoded fallbacks)
-DEFAULT_AA_MIRRORS = [
-    "https://annas-archive.gl",
-    "https://annas-archive.li",
-]
+# Mirror URLs are intentionally user-supplied only.
+DEFAULT_AA_MIRRORS: list[str] = []
+DEFAULT_LIBGEN_MIRRORS: list[str] = []
+DEFAULT_ZLIB_MIRRORS: list[str] = []
+DEFAULT_WELIB_MIRRORS: list[str] = []
 
-DEFAULT_LIBGEN_MIRRORS = [
-    "https://libgen.gl",
-    "https://libgen.li",
-    "https://libgen.bz",
-    "https://libgen.la",
-    "https://libgen.vg",
-]
-
-DEFAULT_ZLIB_MIRRORS = [
-    "https://z-lib.fm",
-    "https://z-lib.gs",
-    "https://z-lib.id",
-    "https://z-library.sk",
-    "https://zlibrary-global.se",
-]
-
-DEFAULT_WELIB_MIRRORS = [
-    "https://welib.org",
-]
+_DOWNLOAD_SOURCE_MIRROR_LABELS = {
+    "aa-fast": "Anna's Archive",
+    "aa-slow": "Anna's Archive",
+    "aa-slow-nowait": "Anna's Archive",
+    "aa-slow-wait": "Anna's Archive",
+    "libgen": "LibGen",
+    "zlib": "Z-Library",
+    "welib": "Welib",
+}
 
 
 def _normalize_mirror_url(url: str) -> str:
     return normalize_http_url(url, default_scheme="https")
 
 
-def get_aa_mirrors() -> List[str]:
-    """
-    Get Anna's Archive mirrors.
+def _string_config_value(value: object) -> str:
+    """Normalize mirror-related config values to strings."""
+    return value if isinstance(value, str) else str(value or "")
 
-    Returns:
-        Ordered list of AA mirror URLs.
 
-        If AA_MIRROR_URLS is configured, it is treated as the full list.
-        Otherwise, defaults are used and AA_ADDITIONAL_URLS (legacy) is appended.
+def _normalize_configured_urls(value: object) -> list[str]:
+    """Normalize list or comma-separated mirror config into unique URLs."""
+    if isinstance(value, list):
+        parts = value
+    elif isinstance(value, str) and value.strip():
+        parts = value.split(",")
+    else:
+        return []
 
-        Notes:
-        - The list is used to populate the AA mirror dropdown in Settings.
-        - When AA_BASE_URL is set to 'auto', mirrors are tried in the order listed.
-    """
+    normalized_urls: list[str] = []
+    for raw_url in parts:
+        normalized = _normalize_mirror_url(str(raw_url))
+        if normalized and normalized not in normalized_urls:
+            normalized_urls.append(normalized)
+    return normalized_urls
+
+
+def _get_primary_mirror_url(key: str) -> str | None:
+    """Return a configured primary mirror URL, if present."""
     config = _get_config()
+    primary = _normalize_mirror_url(_string_config_value(config.get(key, "")))
+    return primary or None
 
+
+def _build_primary_and_additional_mirrors(primary_key: str, additional_key: str) -> list[str]:
+    """Build an ordered mirror list from primary + additional config values."""
+    config = _get_config()
     mirrors: list[str] = []
 
-    configured_list = config.get("AA_MIRROR_URLS", None)
-    if isinstance(configured_list, list):
-        for url in configured_list:
-            normalized = _normalize_mirror_url(str(url))
-            if normalized and normalized not in mirrors:
-                mirrors.append(normalized)
-    elif isinstance(configured_list, str) and configured_list.strip():
-        # Allow comma-separated env/manual configs.
-        for url in configured_list.split(","):
-            normalized = _normalize_mirror_url(url)
-            if normalized and normalized not in mirrors:
-                mirrors.append(normalized)
+    primary = _get_primary_mirror_url(primary_key)
+    if primary:
+        mirrors.append(primary)
 
-    if not mirrors:
-        mirrors = [_normalize_mirror_url(url) for url in DEFAULT_AA_MIRRORS]
-        mirrors = [url for url in mirrors if url]
-
-        # Backwards-compatible append-only behavior for legacy configs/env.
-        additional = config.get("AA_ADDITIONAL_URLS", "")
-        if additional:
-            for url in additional.split(","):
-                normalized = _normalize_mirror_url(url)
-                if normalized and normalized not in mirrors:
-                    mirrors.append(normalized)
+    for url in _normalize_configured_urls(config.get(additional_key, "")):
+        if url not in mirrors:
+            mirrors.append(url)
 
     return mirrors
 
 
-def get_libgen_mirrors() -> List[str]:
-    """
-    Get LibGen mirrors: defaults + any additional from config.
+def get_aa_mirrors() -> list[str]:
+    """Get Anna's Archive mirrors.
 
     Returns:
-        List of LibGen mirror URLs (defaults first, then custom additions).
+        Ordered list of user-configured AA mirror URLs.
+
+    Notes:
+        - The list is used to populate the AA mirror dropdown in Settings.
+        - When AA_BASE_URL is set to 'auto', mirrors are tried in the order listed.
+
     """
-    mirrors = [_normalize_mirror_url(url) for url in DEFAULT_LIBGEN_MIRRORS]
-    mirrors = [url for url in mirrors if url]
     config = _get_config()
-
-    additional = config.get("LIBGEN_ADDITIONAL_URLS", "")
-    if additional:
-        for url in additional.split(","):
-            normalized = _normalize_mirror_url(url)
-            if normalized and normalized not in mirrors:
-                mirrors.append(normalized)
-
-    return mirrors
+    configured_list = _normalize_configured_urls(config.get("AA_MIRROR_URLS", None))
+    if configured_list:
+        return configured_list
+    return _normalize_configured_urls(config.get("AA_ADDITIONAL_URLS", ""))
 
 
-def get_zlib_mirrors() -> List[str]:
+def has_aa_mirror_configuration() -> bool:
+    """Return True when direct-download search has at least one AA base URL to use."""
+    if get_aa_mirrors():
+        return True
+
+    configured_base_url = normalize_http_url(
+        _string_config_value(_get_config().get("AA_BASE_URL", "auto")),
+        default_scheme="https",
+        allow_special=("auto",),
+    )
+    return bool(configured_base_url and configured_base_url != "auto")
+
+
+def get_libgen_mirrors() -> list[str]:
+    """Get user-configured LibGen mirrors.
+
+    Returns:
+        List of LibGen mirror URLs.
+
     """
-    Get Z-Library mirrors, with primary first.
+    config = _get_config()
+    configured_list = _normalize_configured_urls(config.get("LIBGEN_MIRROR_URLS", None))
+    if configured_list:
+        return configured_list
+    return _normalize_configured_urls(config.get("LIBGEN_ADDITIONAL_URLS", ""))
+
+
+def has_libgen_mirror_configuration() -> bool:
+    """Return True when at least one LibGen mirror URL is configured."""
+    return bool(get_libgen_mirrors())
+
+
+def get_zlib_mirrors() -> list[str]:
+    """Get user-configured Z-Library mirrors, with primary first.
 
     Returns:
         List of Z-Library mirror URLs, primary first.
+
     """
     config = _get_config()
-
-    primary = _normalize_mirror_url(config.get("ZLIB_PRIMARY_URL", DEFAULT_ZLIB_MIRRORS[0]))
-    if not primary:
-        primary = _normalize_mirror_url(DEFAULT_ZLIB_MIRRORS[0])
-    mirrors = [primary]
-
-    # Add other defaults (excluding primary)
-    for url in DEFAULT_ZLIB_MIRRORS:
-        normalized = _normalize_mirror_url(url)
-        if normalized and normalized != primary:
-            mirrors.append(normalized)
-
-    # Add custom mirrors
-    additional = config.get("ZLIB_ADDITIONAL_URLS", "")
-    if additional:
-        for url in additional.split(","):
-            normalized = _normalize_mirror_url(url)
-            if normalized and normalized not in mirrors:
-                mirrors.append(normalized)
-
-    return mirrors
+    configured_list = _normalize_configured_urls(config.get("ZLIB_MIRROR_URLS", None))
+    if configured_list:
+        return configured_list
+    return _build_primary_and_additional_mirrors("ZLIB_PRIMARY_URL", "ZLIB_ADDITIONAL_URLS")
 
 
-def get_zlib_primary_url() -> str:
-    """
-    Get the primary Z-Library mirror URL.
+def has_zlib_mirror_configuration() -> bool:
+    """Return True when at least one Z-Library mirror URL is configured."""
+    return bool(get_zlib_mirrors())
+
+
+def get_zlib_primary_url() -> str | None:
+    """Get the primary Z-Library mirror URL.
 
     Returns:
-        Primary Z-Library mirror URL.
+        Primary Z-Library mirror URL, if configured.
+
     """
-    config = _get_config()
-    primary = _normalize_mirror_url(config.get("ZLIB_PRIMARY_URL", DEFAULT_ZLIB_MIRRORS[0]))
-    return primary or _normalize_mirror_url(DEFAULT_ZLIB_MIRRORS[0])
+    mirrors = get_zlib_mirrors()
+    return mirrors[0] if mirrors else None
 
 
-def get_zlib_url_template() -> str:
-    """
-    Get Z-Library URL template using configured primary mirror.
+def get_zlib_url_template() -> str | None:
+    """Get Z-Library URL template using configured primary mirror.
 
     Returns:
-        URL template with {md5} placeholder.
+        URL template with {md5} placeholder, if configured.
+
     """
     primary = get_zlib_primary_url()
-    return f"{primary}/md5/{{md5}}"
+    return f"{primary}/md5/{{md5}}" if primary else None
 
 
-def get_welib_mirrors() -> List[str]:
-    """
-    Get Welib mirrors, with primary first.
+def get_welib_mirrors() -> list[str]:
+    """Get user-configured Welib mirrors, with primary first.
 
     Returns:
         List of Welib mirror URLs, primary first.
+
     """
     config = _get_config()
-
-    primary = _normalize_mirror_url(config.get("WELIB_PRIMARY_URL", DEFAULT_WELIB_MIRRORS[0]))
-    if not primary:
-        primary = _normalize_mirror_url(DEFAULT_WELIB_MIRRORS[0])
-    mirrors = [primary]
-
-    # Add other defaults (excluding primary)
-    for url in DEFAULT_WELIB_MIRRORS:
-        normalized = _normalize_mirror_url(url)
-        if normalized and normalized != primary:
-            mirrors.append(normalized)
-
-    # Add custom mirrors
-    additional = config.get("WELIB_ADDITIONAL_URLS", "")
-    if additional:
-        for url in additional.split(","):
-            normalized = _normalize_mirror_url(url)
-            if normalized and normalized not in mirrors:
-                mirrors.append(normalized)
-
-    return mirrors
+    configured_list = _normalize_configured_urls(config.get("WELIB_MIRROR_URLS", None))
+    if configured_list:
+        return configured_list
+    return _build_primary_and_additional_mirrors("WELIB_PRIMARY_URL", "WELIB_ADDITIONAL_URLS")
 
 
-def get_welib_primary_url() -> str:
-    """
-    Get the primary Welib mirror URL.
+def has_welib_mirror_configuration() -> bool:
+    """Return True when at least one Welib mirror URL is configured."""
+    return bool(get_welib_mirrors())
+
+
+def has_download_source_mirror_configuration(source_id: str) -> bool:
+    """Return True when the requested direct-download source has mirror config."""
+    if source_id in {"aa-fast", "aa-slow", "aa-slow-nowait", "aa-slow-wait"}:
+        return has_aa_mirror_configuration()
+    if source_id == "libgen":
+        return has_libgen_mirror_configuration()
+    if source_id == "zlib":
+        return has_zlib_mirror_configuration()
+    if source_id == "welib":
+        return has_welib_mirror_configuration()
+    return False
+
+
+def get_download_source_missing_mirror_reason(source_id: str) -> str | None:
+    """Return a user-facing reason when a direct-download source has no mirror config."""
+    if has_download_source_mirror_configuration(source_id):
+        return None
+
+    label = _DOWNLOAD_SOURCE_MIRROR_LABELS.get(source_id)
+    if not label:
+        return None
+
+    return f"Add at least one {label} mirror in Mirrors"
+
+
+def get_welib_primary_url() -> str | None:
+    """Get the primary Welib mirror URL.
 
     Returns:
-        Primary Welib mirror URL.
+        Primary Welib mirror URL, if configured.
+
     """
-    config = _get_config()
-    primary = _normalize_mirror_url(config.get("WELIB_PRIMARY_URL", DEFAULT_WELIB_MIRRORS[0]))
-    return primary or _normalize_mirror_url(DEFAULT_WELIB_MIRRORS[0])
+    mirrors = get_welib_mirrors()
+    return mirrors[0] if mirrors else None
 
 
-def get_welib_url_template() -> str:
-    """
-    Get Welib URL template using configured primary mirror.
+def get_welib_url_template() -> str | None:
+    """Get Welib URL template using configured primary mirror.
 
     Returns:
-        URL template with {md5} placeholder.
+        URL template with {md5} placeholder, if configured.
+
     """
     primary = get_welib_primary_url()
-    return f"{primary}/md5/{{md5}}"
+    return f"{primary}/md5/{{md5}}" if primary else None
 
 
 def get_zlib_cookie_domains() -> set:
-    """
-    Get set of Z-Library domains that need full cookie handling.
+    """Get set of Z-Library domains that need full cookie handling.
 
     Used by internal_bypasser for CF bypass cookie management.
 
     Returns:
         Set of domain strings (without protocol).
+
     """
     domains = set()
 
-    # Add all default domains
-    for url in DEFAULT_ZLIB_MIRRORS:
-        normalized = _normalize_mirror_url(url)
-        if normalized:
-            domain = normalized.replace("https://", "").replace("http://", "").split("/")[0]
-            domains.add(domain)
-
-    # Add custom domains
-    config = _get_config()
-    additional = config.get("ZLIB_ADDITIONAL_URLS", "")
-    if additional:
-        for url in additional.split(","):
-            normalized = _normalize_mirror_url(url)
-            if normalized:
-                domain = normalized.replace("https://", "").replace("http://", "").split("/")[0]
-                domains.add(domain)
+    for url in get_zlib_mirrors():
+        domain = url.replace("https://", "").replace("http://", "").split("/")[0]
+        domains.add(domain)
 
     return domains

shelfmark/core/models.py

@@ -1,19 +1,20 @@
 """Data structures and models used across the application."""
 
-from dataclasses import dataclass, field
-from pathlib import Path
-from typing import Any, Dict, List, Optional
-from enum import Enum
 import re
 import time
+from dataclasses import dataclass, field
+from enum import StrEnum
+from pathlib import Path
+from typing import Any
 
 
 def build_filename(
     title: str,
-    author: Optional[str] = None,
-    year: Optional[str] = None,
-    fmt: Optional[str] = None,
+    author: str | None = None,
+    year: str | None = None,
+    fmt: str | None = None,
 ) -> str:
+    """Build a filesystem-safe filename from book metadata."""
     parts = []
     if author:
         parts.append(author)
@@ -23,7 +24,7 @@ def build_filename(
         parts.append(f" ({year})")
 
     filename = "".join(parts)
-    filename = re.sub(r'[\\/:*?"<>|]', '_', filename.strip())[:245]
+    filename = re.sub(r'[\\/:*?"<>|]', "_", filename.strip())[:245]
 
     if fmt:
         filename = f"{filename}.{fmt}"
@@ -31,20 +32,39 @@ def build_filename(
     return filename
 
 
-class QueueStatus(str, Enum):
+class QueueStatus(StrEnum):
     """Enum for possible book queue statuses."""
+
     QUEUED = "queued"
     RESOLVING = "resolving"
     LOCATING = "locating"
     DOWNLOADING = "downloading"
     COMPLETE = "complete"
-    AVAILABLE = "available"
     ERROR = "error"
-    DONE = "done"
     CANCELLED = "cancelled"
 
 
-class SearchMode(str, Enum):
+TERMINAL_QUEUE_STATUSES: frozenset[QueueStatus] = frozenset(
+    {
+        QueueStatus.COMPLETE,
+        QueueStatus.ERROR,
+        QueueStatus.CANCELLED,
+    }
+)
+
+ACTIVE_QUEUE_STATUSES: frozenset[QueueStatus] = frozenset(
+    {
+        QueueStatus.QUEUED,
+        QueueStatus.RESOLVING,
+        QueueStatus.LOCATING,
+        QueueStatus.DOWNLOADING,
+    }
+)
+
+
+class SearchMode(StrEnum):
+    """Search modes supported by the Shelfmark UI and API."""
+
     DIRECT = "direct"
     UNIVERSAL = "universal"
 
@@ -52,11 +72,12 @@ class SearchMode(str, Enum):
 @dataclass
 class QueueItem:
     """Queue item with priority and metadata."""
+
     book_id: str
     priority: int
     added_time: float
 
-    def __lt__(self, other):
+    def __lt__(self, other: QueueItem) -> bool:
         """Compare items for priority queue (lower priority number = higher precedence)."""
         if self.priority != other.priority:
             return self.priority < other.priority
@@ -65,50 +86,80 @@ class QueueItem:
 
 @dataclass
 class DownloadTask:
-    task_id: str                                # Unique ID (e.g., AA MD5 hash, Prowlarr GUID)
-    source: str                                 # Handler name ("direct_download", "prowlarr")
-    title: str                                  # Display title for queue sidebar
+    """Mutable download task state tracked throughout the pipeline."""
+
+    task_id: str  # Unique ID (e.g., AA MD5 hash, Prowlarr GUID)
+    source: str  # Handler name ("direct_download", "prowlarr")
+    title: str  # Display title for queue sidebar
 
     # Display info for queue sidebar
-    author: Optional[str] = None
-    year: Optional[str] = None
-    format: Optional[str] = None
-    size: Optional[str] = None
-    preview: Optional[str] = None
-    content_type: Optional[str] = None  # "book (fiction)", "audiobook", "magazine", etc.
-    source_url: Optional[str] = None  # Original release URL used by source-specific handlers
+    author: str | None = None
+    year: str | None = None
+    format: str | None = None
+    size: str | None = None
+    downloads: int | None = None  # Download count from source
+    preview: str | None = None
+    content_type: str | None = None  # "book (fiction)", "audiobook", "magazine", etc.
+    source_url: str | None = None  # Original release URL used by source-specific handlers
+    retry_download_url: str | None = None  # Resolved download URL for restart-safe retries
+    retry_download_protocol: str | None = (
+        None  # Protocol for retry_download_url (e.g. torrent, usenet)
+    )
+    retry_release_name: str | None = None  # Display name to send back to external download clients
+    retry_expected_hash: str | None = None  # Optional torrent hash used to match client downloads
+    retry_ratio_limit: float | None = None  # Optional post-download seeding ratio
+    retry_seeding_time_limit_minutes: int | None = None  # Optional post-download seeding time limit
+    retry_source_context: dict[str, Any] = field(
+        default_factory=dict
+    )  # Source-private context for retry/re-resolution
+    can_retry_without_staged_source: bool = (
+        True  # Whether the source can restart without a preserved staged file
+    )
 
     # Series info (for library naming templates)
-    series_name: Optional[str] = None
-    series_position: Optional[float] = None  # Float for novellas (e.g., 1.5)
-    subtitle: Optional[str] = None  # Book subtitle for naming templates
+    series_name: str | None = None
+    series_position: float | None = None  # Float for novellas (e.g., 1.5)
+    subtitle: str | None = None  # Book subtitle for naming templates
+    language: str | None = None  # Release language code for the {Language} template variable
 
     # Hardlinking support
-    original_download_path: Optional[str] = None  # Path in download client (for hardlinking)
+    original_download_path: str | None = None  # Path in download client (for hardlinking)
 
     # Search mode - determines post-download processing behavior
     # See SearchMode enum for behavioral differences
-    search_mode: Optional[SearchMode] = None
+    search_mode: SearchMode | None = None
 
     # Output selection for post-processing.
     # This is captured at queue time so in-flight tasks are not affected if the user changes settings later.
-    output_mode: Optional[str] = None  # e.g. "folder", "booklore", "email"
-    output_args: Dict[str, Any] = field(default_factory=dict)  # Per-output parameters (e.g. email recipient)
+    output_mode: str | None = None
+
+    output_args: dict[str, Any] = field(
+        default_factory=dict
+    )  # Per-output parameters (e.g. email recipient)
+
+    # Multi-book packs: one release holding several books. `book_plan` is the split the
+    # user approved before download (list of {title, series_position, year, files});
+    # `multi_book` asks post-processing to split heuristically when no plan exists.
+    multi_book: bool = False
+    book_plan: list[dict[str, Any]] | None = None
 
     # User association (multi-user support)
-    user_id: Optional[int] = None  # DB user ID who queued this download
-    username: Optional[str] = None  # Username for {User} template variable
-    request_id: Optional[int] = None  # Origin request ID when queued from request fulfilment
+    user_id: int | None = None  # DB user ID who queued this download
+    username: str | None = None  # Username for {User} template variable
+    request_id: int | None = None  # Origin request ID when queued from request fulfilment
 
     # Runtime state
     priority: int = 0
     added_time: float = field(default_factory=time.time)
     progress: float = 0.0
     status: QueueStatus = QueueStatus.QUEUED
-    status_message: Optional[str] = None
-    download_path: Optional[str] = None
+    status_message: str | None = None
+    download_path: str | None = None
+    last_error_message: str | None = None
+    last_error_type: str | None = None
+    staged_path: str | None = None
 
-    def __lt__(self, other):
+    def __lt__(self, other: DownloadTask) -> bool:
         """Compare tasks for priority queue (lower priority number = higher precedence)."""
         if self.priority != other.priority:
             return self.priority < other.priority
@@ -121,62 +172,14 @@ class DownloadTask:
         return build_filename(self.title, self.author, self.year, self.format)
 
 
-@dataclass
-class BookInfo:
-    """Data class representing book information."""
-    id: str
-    title: str
-    preview: Optional[str] = None
-    author: Optional[str] = None
-    publisher: Optional[str] = None
-    year: Optional[str] = None
-    language: Optional[str] = None
-    content: Optional[str] = None
-    format: Optional[str] = None
-    size: Optional[str] = None
-    info: Optional[Dict[str, List[str]]] = None
-    description: Optional[str] = None
-    download_urls: List[str] = field(default_factory=list)
-    download_path: Optional[str] = None
-    priority: int = 0
-    progress: Optional[float] = None
-    status_message: Optional[str] = None  # Detailed status message for UI display
-    added_time: Optional[float] = None  # Timestamp when added to queue
-    source: str = "direct_download"  # Release source handler to use for downloads
-    source_url: Optional[str] = None  # Link to source page (e.g., Anna's Archive)
-
-    def get_filename(self, fallback_url: Optional[str] = None) -> str:
-        """Build sanitized filename: 'Author - Title (Year).format'
-
-        Resolves format from self.format, download_urls, or fallback_url.
-
-        Args:
-            fallback_url: URL to extract format from if not already known
-
-        Returns:
-            Sanitized filename safe for filesystem use
-        """
-        # Resolve format if needed
-        if not self.format:
-            urls = [self.download_urls[0]] if self.download_urls else []
-            if fallback_url:
-                urls.append(fallback_url)
-            for url in urls:
-                ext = url.split(".")[-1].lower()
-                if ext and len(ext) <= 5 and ext.isalnum():
-                    self.format = ext
-                    break
-
-        return build_filename(self.title, self.author, self.year, self.format)
-
-
 @dataclass
 class SearchFilters:
     """Filters for book search queries."""
-    isbn: Optional[List[str]] = None
-    author: Optional[List[str]] = None
-    title: Optional[List[str]] = None
-    lang: Optional[List[str]] = None
-    sort: Optional[str] = None
-    content: Optional[List[str]] = None
-    format: Optional[List[str]] = None
+
+    isbn: list[str] | None = None
+    author: list[str] | None = None
+    title: list[str] | None = None
+    lang: list[str] | None = None
+    sort: str | None = None
+    content: list[str] | None = None
+    format: list[str] | None = None

shelfmark/core/naming.py

@@ -1,47 +1,64 @@
 """Template-based naming for library organization."""
 
-import os
 import re
 from pathlib import Path
-from typing import Dict, Optional, Union, Mapping
+from typing import TYPE_CHECKING
 
+from shelfmark.core.languages import LANGUAGE_PLACEHOLDERS, normalize_language
 from shelfmark.core.logger import setup_logger
 
+if TYPE_CHECKING:
+    from collections.abc import Mapping
+
 logger = setup_logger(__name__)
 
 
 # Known variable tokens, sorted longest-first to avoid partial matches
-# e.g., "SeriesPosition" must match before "Series"
+# e.g., "SeriesPosition" must match before "Series", "FirstAuthor" before "Author"
 KNOWN_TOKENS = [
-    'seriesposition',
-    'partnumber',
-    'subtitle',
-    'author',
-    'series',
-    'title',
-    'year',
-    'user',
+    "seriesposition",
+    "primarytitle",
+    "originalname",
+    "firstauthor",
+    "partnumber",
+    "language",
+    "subtitle",
+    "author",
+    "series",
+    "title",
+    "year",
+    "user",
 ]
 
+# Authors reach naming already joined as "First Author, Second Author, ...".
+# {FirstAuthor} keeps only the first entry. A single name written "Last, First"
+# is split on the comma too and renders as "Last" -- the source metadata does
+# not mark which form it is (see #930).
+AUTHOR_LIST_SEPARATOR = re.compile(r"\s*[,;]\s*")
+
 # Match any {...} block for template parsing
-BRACE_PATTERN = re.compile(r'\{([^}]+)\}')
+BRACE_PATTERN = re.compile(r"\{([^}]+)\}")
 
 # Characters that are invalid in filenames on various filesystems
 INVALID_CHARS = re.compile(r'[\\/:*?"<>|]')
 
+# Runs of whitespace inside a single placeholder's rendered value, e.g. "Conan Doyle"
+# -- collapsed to the configured word separator (see `parse_naming_template`).
+WHITESPACE_RUN = re.compile(r"\s+")
 
-def _sanitize(name: Optional[str], max_length: int = 245) -> str:
+
+def _sanitize(name: str | None, max_length: int = 245) -> str:
     """Sanitize a string for filesystem use."""
     if not name:
         return ""
 
-    sanitized = INVALID_CHARS.sub('_', name)
-    sanitized = re.sub(r'^[\s.]+|[\s.]+$', '', sanitized)  # Strip whitespace and dots
-    sanitized = re.sub(r'_+', '_', sanitized)  # Collapse underscores
+    sanitized = INVALID_CHARS.sub("_", name)
+    sanitized = re.sub(r"^[\s.]+|[\s.]+$", "", sanitized)  # Strip whitespace and dots
+    sanitized = re.sub(r"_+", "_", sanitized)  # Collapse underscores
     return sanitized[:max_length]
 
 
-def sanitize_filename(name: Optional[str], max_length: int = 245) -> str:
+def sanitize_filename(name: str | None, max_length: int = 245) -> str:
     """Sanitize a string for use as a filename or path component."""
     return _sanitize(name, max_length)
 
@@ -50,7 +67,16 @@ def sanitize_filename(name: Optional[str], max_length: int = 245) -> str:
 sanitize_path_component = sanitize_filename
 
 
-def format_series_position(position: Optional[Union[str, int, float]]) -> str:
+def first_author(value: object) -> str:
+    """Return the first entry from an author string joined with ',' or ';'."""
+    text = " ".join(str(value or "").split())
+    if not text:
+        return ""
+    return AUTHOR_LIST_SEPARATOR.split(text, maxsplit=1)[0].strip()
+
+
+def format_series_position(position: str | float | None) -> str:
+    """Format a series position for naming templates."""
     if position is None:
         return ""
 
@@ -61,14 +87,59 @@ def format_series_position(position: Optional[Union[str, int, float]]) -> str:
     return str(position)
 
 
+def normalize_language_code(language: str | None) -> str:
+    """Resolve a release language to the single spelling used in a path.
+
+    Sources report the same language in different shapes: "en", "eng", "English".
+    All of them have to collapse to one code, or the editions they identify end
+    up in separate folders, which is the collision this token exists to prevent.
+    Placeholder values render empty so `{ (Language)}` disappears entirely
+    rather than labelling a folder "(unknown)".
+
+    A language the bundled data does not know is kept, casefolded, rather than
+    dropped: it still separates editions, and it cannot collide with a resolved
+    code precisely because nothing resolves it.
+    """
+    if not language:
+        return ""
+
+    resolved = normalize_language(language)
+    if resolved is not None:
+        return resolved
+
+    normalized = " ".join(str(language).split()).strip().casefold()
+    if normalized in LANGUAGE_PLACEHOLDERS:
+        return ""
+
+    return normalized
+
+
+def derive_primary_title(title: str | None, subtitle: str | None) -> str:
+    """Return the title without an explicit subtitle suffix when possible."""
+    title_value = " ".join(str(title or "").split()).strip()
+    if not title_value:
+        return ""
+
+    subtitle_value = " ".join(str(subtitle or "").split()).strip()
+    if not subtitle_value:
+        return title_value
+
+    pattern = rf"^(?P<primary>.+?)(?:\s*:\s*|\s+-\s+){re.escape(subtitle_value)}$"
+    match = re.match(pattern, title_value, flags=re.IGNORECASE)
+    if not match:
+        return title_value
+
+    primary = match.group("primary").strip()
+    return primary or title_value
+
+
 # Pads numbers to 9 digits for natural sorting (e.g., "Part 2" -> "Part 000000002")
-PAD_NUMBERS_PATTERN = re.compile(r'\d+')
+PAD_NUMBERS_PATTERN = re.compile(r"\d+")
 
 
-def natural_sort_key(path: Union[str, Path]) -> str:
+def natural_sort_key(path: str | Path) -> str:
     """Generate a sort key with padded numbers for natural sorting."""
-    filename = Path(path).name.lower()
-    return PAD_NUMBERS_PATTERN.sub(lambda m: m.group().zfill(9), filename)
+    return PAD_NUMBERS_PATTERN.sub(lambda m: m.group().zfill(9), str(path).lower())
 
 
 def assign_part_numbers(
@@ -88,43 +159,56 @@ def assign_part_numbers(
 
 def parse_naming_template(
     template: str,
-    metadata: Mapping[str, Optional[Union[str, int, float]]],
+    metadata: Mapping[str, str | int | float | None],
     *,
     allow_path_separators: bool = True,
+    word_separator: str = " ",
 ) -> str:
+    """Render a naming template with Shelfmark metadata placeholders.
+
+    `word_separator` replaces whitespace *inside* each placeholder's rendered
+    value (e.g. "Conan Doyle" -> "Conan.Doyle" for a "." separator). It never
+    touches literal characters typed into the template itself, so a template
+    like "{Author}.-.{Title}" keeps its own dots regardless of this setting.
+    The default (" ") leaves values untouched, matching prior behavior.
+    """
     if not template:
         return ""
 
     # Normalize metadata keys to lowercase for case-insensitive matching
     normalized = {k.lower(): v for k, v in metadata.items()}
 
-    def find_token(content: str) -> tuple[Optional[str], int]:
+    def find_placeholder(content: str) -> tuple[str | None, int]:
         content_lower = content.lower()
-        for token in KNOWN_TOKENS:
-            idx = content_lower.find(token)
+        for placeholder_name in KNOWN_TOKENS:
+            idx = content_lower.find(placeholder_name)
             if idx != -1:
-                return token, idx
+                return placeholder_name, idx
         return None, -1
 
-    def token_value(token: str) -> str:
-        value = normalized.get(token)
-        if token == 'seriesposition':
+    def placeholder_value(placeholder_name: str) -> str:
+        value = normalized.get(placeholder_name)
+        if placeholder_name == "seriesposition":
             value = format_series_position(value)
+        elif placeholder_name == "firstauthor" and not value:
+            value = first_author(normalized.get("author"))
         if value is None:
             return ""
         return str(value).strip()
 
-    def render_block(content: str) -> Optional[str]:
-        token, idx = find_token(content)
-        if token is None:
+    def render_block(content: str) -> str | None:
+        placeholder_name, idx = find_placeholder(content)
+        if placeholder_name is None:
             return None
 
         prefix = content[:idx]
-        suffix = content[idx + len(token):]
-        value = token_value(token)
+        suffix = content[idx + len(placeholder_name) :]
+        value = placeholder_value(placeholder_name)
         if not value:
             return ""
 
+        if word_separator != " ":
+            value = WHITESPACE_RUN.sub(word_separator, value)
         if not allow_path_separators:
             value = value.replace("/", "_")
         value = sanitize_filename(value)
@@ -139,7 +223,7 @@ def parse_naming_template(
         parts: list[str] = []
         cursor = 0
         for idx, match in enumerate(matches):
-            parts.append(template[cursor:match.start()])
+            parts.append(template[cursor : match.start()])
             content = match.group(1)
             rendered = render_block(content)
 
@@ -150,17 +234,16 @@ def parse_naming_template(
                 include_literal = False
                 if idx + 1 < len(matches) and match.end() == matches[idx + 1].start():
                     next_content = matches[idx + 1].group(1)
-                    next_token, _next_idx = find_token(next_content)
-                    if next_token is not None:
+                    next_placeholder_name, _next_idx = find_placeholder(next_content)
+                    if next_placeholder_name is not None:
                         conditional_literal = True
-                        include_literal = bool(token_value(next_token))
+                        include_literal = bool(placeholder_value(next_placeholder_name))
                 if include_literal:
                     parts.append(content)
-                elif not conditional_literal:
+                elif not conditional_literal and re.search(r"\s", content):
                     # Preserve blocks that look like literal text, but treat bare unknown
                     # placeholders as missing variables.
-                    if re.search(r"\s", content):
-                        parts.append(match.group(0))
+                    parts.append(match.group(0))
 
             cursor = match.end()
 
@@ -168,41 +251,44 @@ def parse_naming_template(
         result = "".join(parts)
 
     # Clean up any double slashes that might result from empty tokens
-    result = re.sub(r'/+', '/', result)
+    result = re.sub(r"/+", "/", result)
 
     # Remove leading/trailing slashes
-    result = result.strip('/')
+    result = result.strip("/")
 
     # Clean up any orphaned separators (e.g., " - " at start/end, or " -  - ")
-    result = re.sub(r'^[\s\-_.]+', '', result)
-    result = re.sub(r'[\s\-_.]+$', '', result)
-    result = re.sub(r'(\s*-\s*){2,}', ' - ', result)
+    result = re.sub(r"^[\s\-_.]+", "", result)
+    result = re.sub(r"[\s\-_.]+$", "", result)
+    result = re.sub(r"(\s*-\s*){2,}", " - ", result)
 
     # Clean up empty parentheses/brackets
-    result = re.sub(r'\(\s*\)', '', result)
-    result = re.sub(r'\[\s*\]', '', result)
+    result = re.sub(r"\(\s*\)", "", result)
+    result = re.sub(r"\[\s*\]", "", result)
 
     # Final trim of any trailing separators left after cleanup
-    result = re.sub(r'[\s\-_.]+$', '', result)
-
-    return result
+    return re.sub(r"[\s\-_.]+$", "", result)
 
 
 def build_library_path(
     base_path: str,
     template: str,
-    metadata: Mapping[str, Optional[Union[str, int, float]]],
-    extension: Optional[str] = None,
+    metadata: Mapping[str, str | int | float | None],
+    extension: str | None = None,
+    *,
+    word_separator: str = " ",
 ) -> Path:
-    relative = parse_naming_template(template, metadata, allow_path_separators=True)
+    """Build a final library path from a template and metadata."""
+    relative = parse_naming_template(
+        template, metadata, allow_path_separators=True, word_separator=word_separator
+    )
 
     if not relative:
         # Fallback to title if template produces empty result
-        title = metadata.get('Title') or metadata.get('title') or 'Unknown'
+        title = metadata.get("Title") or metadata.get("title") or "Unknown"
         relative = sanitize_filename(str(title))
 
     # Remove any path traversal attempts
-    relative = relative.replace('..', '')
+    relative = relative.replace("..", "")
 
     base = Path(base_path).resolve()
     full_path = (base / relative).resolve()
@@ -210,11 +296,12 @@ def build_library_path(
     # Verify the path is within the base directory
     try:
         full_path.relative_to(base)
-    except ValueError:
-        raise ValueError(f"Path traversal detected: template would escape library directory")
+    except ValueError as exc:
+        msg = "Path traversal detected: template would escape library directory"
+        raise ValueError(msg) from exc
 
     if extension:
-        ext = extension.lstrip('.')
+        ext = extension.lstrip(".")
         # Don't use with_suffix() - it replaces everything after the first dot
         # e.g., "2.5 - Title" would become "2.epub" instead of "2.5 - Title.epub"
         full_path = Path(f"{full_path}.{ext}")
@@ -222,27 +309,27 @@ def build_library_path(
     return full_path
 
 
-def same_filesystem(path1: Union[str, Path], path2: Union[str, Path]) -> bool:
+def same_filesystem(path1: str | Path, path2: str | Path) -> bool:
     """Check if two paths are on the same filesystem."""
     path1 = Path(path1)
     path2 = Path(path2)
 
-    def get_device(p: Path) -> Optional[int]:
+    def get_device(p: Path) -> int | None:
         try:
             while not p.exists():
                 p = p.parent
                 if p == p.parent:
                     break
-            return os.stat(p).st_dev
+            return p.stat().st_dev
         except (OSError, PermissionError) as e:
-            logger.debug(f"Cannot stat {p}: {e}")
+            logger.debug("Cannot stat %s: %s", p, e)
             return None
 
     dev1 = get_device(path1)
     dev2 = get_device(path2)
 
     if dev1 is None or dev2 is None:
-        logger.warning(f"Cannot determine filesystem for hardlink check, falling back to copy")
+        logger.warning("Cannot determine filesystem for hardlink check, falling back to copy")
         return False
 
     return dev1 == dev2

shelfmark/core/notifications.py

@@ -2,18 +2,26 @@
 
 from __future__ import annotations
 
+import logging
+import threading
 from concurrent.futures import ThreadPoolExecutor
+from contextlib import contextmanager, suppress
 from dataclasses import dataclass
-from enum import Enum
-from typing import Any, Iterable
+from enum import StrEnum
+from typing import TYPE_CHECKING, Any, Protocol, TypeGuard
+from urllib.parse import urlsplit
 
 try:
     import apprise
-except Exception:  # pragma: no cover - exercised in tests via monkeypatch
+except ImportError:  # pragma: no cover - exercised in tests via monkeypatch
     apprise = None  # type: ignore[assignment]
 
 from shelfmark.core.config import config as app_config
 from shelfmark.core.logger import setup_logger
+from shelfmark.core.request_helpers import normalize_positive_int
+
+if TYPE_CHECKING:
+    from collections.abc import Iterable, Iterator
 
 logger = setup_logger(__name__)
 
@@ -25,9 +33,37 @@ _APPRISE_APP_DESC = "Shelfmark notifications"
 _APPRISE_LOGO_URL = (
     "https://raw.githubusercontent.com/calibrain/shelfmark/main/src/frontend/public/logo.png"
 )
+_APPRISE_LOGGER_NAME = "apprise"
+_APPRISE_DISPATCH_ERRORS = (RuntimeError, TypeError, ValueError)
 
 
-class NotificationEvent(str, Enum):
+class _ApprisePluginWithUrl(Protocol):
+    app_id: object
+
+    def url(self, *, privacy: bool = False) -> str:
+        _ = privacy
+        return ""
+
+
+class _AppriseClient(Protocol):
+    asset: object
+
+    def add(self, plugin: object) -> object: ...
+
+    def notify(self, *, title: str, body: str, notify_type: object) -> object: ...
+
+
+def _is_apprise_client(candidate: object) -> TypeGuard[_AppriseClient]:
+    return callable(getattr(candidate, "add", None)) and callable(
+        getattr(candidate, "notify", None)
+    )
+
+
+def _has_plugin_url(candidate: object) -> TypeGuard[_ApprisePluginWithUrl]:
+    return callable(getattr(candidate, "url", None))
+
+
+class NotificationEvent(StrEnum):
     """Global notification event identifiers."""
 
     REQUEST_CREATED = "request_created"
@@ -52,7 +88,7 @@ class NotificationContext:
     error_message: str | None = None
 
 
-def _normalize_urls(value: Any) -> list[str]:
+def _normalize_urls(value: object) -> list[str]:
     if value is None:
         return []
 
@@ -69,6 +105,13 @@ def _normalize_urls(value: Any) -> list[str]:
     seen: set[str] = set()
     for raw_url in raw_values:
         url = str(raw_url or "").strip()
+        if not url:
+            continue
+        # Strip invisible/non-ASCII characters that can sneak in via copy-paste
+        # (zero-width spaces, smart quotes, non-breaking spaces, etc.).
+        # These pass Apprise URL validation but cause UnicodeEncodeError when
+        # requests tries to latin-1 encode credentials for Basic Auth headers.
+        url = url.encode("ascii", errors="ignore").decode("ascii").strip()
         if not url:
             continue
         if url in seen:
@@ -78,7 +121,116 @@ def _normalize_urls(value: Any) -> list[str]:
     return normalized
 
 
-def _normalize_routes(value: Any) -> list[dict[str, str]]:
+def _extract_url_schemes(urls: Iterable[str]) -> list[str]:
+    schemes: list[str] = []
+    seen: set[str] = set()
+    for raw_url in urls:
+        scheme = urlsplit(str(raw_url or "")).scheme.lower()
+        if not scheme or scheme in seen:
+            continue
+        seen.add(scheme)
+        schemes.append(scheme)
+    return schemes
+
+
+class _AppriseLogCapture(logging.Handler):
+    def __init__(self, *, thread_id: int) -> None:
+        super().__init__(level=logging.INFO)
+        self.records: list[tuple[int, str, str, str]] = []
+        self._thread_id = thread_id
+
+    def emit(self, record: logging.LogRecord) -> None:
+        if record.thread != self._thread_id:
+            return
+
+        message = record.getMessage()
+        if message:
+            exception_summary = ""
+            if record.exc_info and record.exc_info[0]:
+                exc_type = getattr(record.exc_info[0], "__name__", "Exception")
+                exc = record.exc_info[1]
+                exception_summary = f"{exc_type}: {exc}"
+            elif record.exc_text:
+                exception_summary = str(record.exc_text).strip()
+
+            self.records.append((record.levelno, record.name, str(message), exception_summary))
+
+
+@contextmanager
+def _capture_apprise_logs(
+    *, min_level: int = logging.INFO
+) -> Iterator[list[tuple[int, str, str, str]]]:
+    apprise_logger = logging.getLogger(_APPRISE_LOGGER_NAME)
+    previous_level = apprise_logger.level
+    handler = _AppriseLogCapture(thread_id=threading.get_ident())
+    apprise_logger.addHandler(handler)
+
+    if previous_level == logging.NOTSET or previous_level > min_level:
+        apprise_logger.setLevel(min_level)
+
+    try:
+        yield handler.records
+    finally:
+        apprise_logger.removeHandler(handler)
+        apprise_logger.setLevel(previous_level)
+
+
+def _log_apprise_records(records: Iterable[tuple[int, str, str, str]]) -> None:
+    seen: set[tuple[int, str, str, str]] = set()
+    for level, source, raw_message, raw_exception_summary in records:
+        message = str(raw_message or "").strip()
+        source_name = str(source or "").strip() or _APPRISE_LOGGER_NAME
+        exception_summary = str(raw_exception_summary or "").strip()
+        key = (int(level), source_name, message, exception_summary)
+        if not message or key in seen:
+            continue
+        seen.add(key)
+
+        full_message = message if not exception_summary else f"{message} ({exception_summary})"
+
+        if level >= logging.ERROR:
+            logger.error("Apprise source [%s]: %s", source_name, full_message)
+        elif level >= logging.WARNING:
+            logger.warning("Apprise source [%s]: %s", source_name, full_message)
+        else:
+            logger.info("Apprise source [%s]: %s", source_name, full_message)
+
+
+def _log_apprise_exception_debug(*, action: str, scheme: str, exc: Exception) -> None:
+    logger.debug(
+        "Apprise %s raised %s for scheme '%s': %s",
+        action,
+        type(exc).__name__,
+        scheme,
+        exc,
+        exc_info=(type(exc), exc, exc.__traceback__),
+    )
+
+
+def _build_apprise_warning_detail(
+    records: Iterable[tuple[int, str, str, str]],
+    *,
+    scheme: str,
+) -> str | None:
+    for level, source, raw_message, raw_exception_summary in records:
+        if level < logging.WARNING:
+            continue
+
+        message = str(raw_message or "").strip()
+        if not message:
+            continue
+
+        source_name = str(source or "").strip()
+        exception_summary = str(raw_exception_summary or "").strip()
+        full_message = message if not exception_summary else f"{message} ({exception_summary})"
+
+        if source_name and source_name != _APPRISE_LOGGER_NAME:
+            return f"{scheme}: {source_name}: {full_message}"
+        return f"{scheme}: {full_message}"
+    return None
+
+
+def _normalize_routes(value: object) -> list[dict[str, str]]:
     if not isinstance(value, list):
         return []
 
@@ -129,14 +281,8 @@ def _resolve_admin_routes() -> list[dict[str, str]]:
     return _normalize_routes(app_config.get("ADMIN_NOTIFICATION_ROUTES", []))
 
 
-def _normalize_user_id(value: Any) -> int | None:
-    try:
-        user_id = int(value)
-    except (TypeError, ValueError):
-        return None
-    if user_id < 1:
-        return None
-    return user_id
+def _normalize_user_id(value: object) -> int | None:
+    return normalize_positive_int(value)
 
 
 def _resolve_user_routes(user_id: int | None) -> list[dict[str, str]]:
@@ -172,7 +318,7 @@ def _resolve_route_urls_for_event(
     return selected
 
 
-def _resolve_notify_type(event: NotificationEvent) -> Any:
+def _resolve_notify_type(event: NotificationEvent) -> object:
     if apprise is None:
         fallback = {
             NotificationEvent.REQUEST_CREATED: "info",
@@ -193,7 +339,7 @@ def _resolve_notify_type(event: NotificationEvent) -> Any:
     return mapping[event]
 
 
-def _clean_text(value: Any, fallback: str) -> str:
+def _clean_text(value: object, fallback: str) -> str:
     text = str(value or "").strip()
     return text or fallback
 
@@ -211,7 +357,10 @@ def _render_message(context: NotificationContext) -> tuple[str, str]:
     if event == NotificationEvent.REQUEST_REJECTED:
         note = _clean_text(context.admin_note, "")
         note_line = f"\nNote: {note}" if note else ""
-        return "Request Rejected", f'Request for "{title}" by {author} was rejected.{note_line}'
+        return (
+            "Request Rejected",
+            f'Request for "{title}" by {author} was rejected.{note_line}',
+        )
     if event == NotificationEvent.DOWNLOAD_COMPLETE:
         return "Download Complete", f'"{title}" by {author} downloaded successfully.'
 
@@ -220,56 +369,209 @@ def _render_message(context: NotificationContext) -> tuple[str, str]:
     return "Download Failed", f'Failed to download "{title}" by {author}.{error_line}'
 
 
+def _plugin_label(plugin: object, fallback_scheme: str) -> str:
+    """Build a human-readable label from a validated Apprise plugin.
+
+    Combines the URL scheme with the plugin's service name (app_id) and
+    privacy-safe URL for richer diagnostics, e.g.
+    ``"slack (Slack - slack://TokenA/To...n/To...n/)"``
+    """
+    parts: list[str] = [fallback_scheme]
+
+    app_id = getattr(plugin, "app_id", None)
+    if app_id and str(app_id) != fallback_scheme:
+        privacy_url: str | None = None
+        if _has_plugin_url(plugin):
+            with suppress(Exception):
+                privacy_url = plugin.url(privacy=True)
+
+        suffix = str(app_id)
+        if privacy_url:
+            suffix = f"{suffix} - {privacy_url}"
+        parts.append(f"({suffix})")
+
+    return " ".join(parts)
+
+
+def _apprise_proxy_env() -> dict[str, str]:
+    """Build proxy env vars from app config so Apprise respects the proxy setting."""
+    import os
+
+    from shelfmark.core.config import config as _cfg
+
+    mode = str(_cfg.get("PROXY_MODE", "") or "").lower()
+    env: dict[str, str] = {}
+
+    if mode == "http":
+        http = str(_cfg.get("HTTP_PROXY", "") or "").strip()
+        https = str(_cfg.get("HTTPS_PROXY", "") or "").strip() or http
+        if http:
+            env["HTTP_PROXY"] = http
+            env["http_proxy"] = http
+        if https:
+            env["HTTPS_PROXY"] = https
+            env["https_proxy"] = https
+    elif mode == "socks5":
+        socks = str(_cfg.get("SOCKS5_PROXY", "") or "").strip()
+        if socks:
+            env["HTTP_PROXY"] = socks
+            env["http_proxy"] = socks
+            env["HTTPS_PROXY"] = socks
+            env["https_proxy"] = socks
+
+    no_proxy = str(_cfg.get("NO_PROXY", "") or "").strip()
+    if no_proxy and env:
+        env["NO_PROXY"] = no_proxy
+        env["no_proxy"] = no_proxy
+
+    # Don't override if the user already set these in the environment directly
+    return {k: v for k, v in env.items() if not os.environ.get(k)}
+
+
 def _dispatch_to_apprise(
     urls: Iterable[str],
     *,
     title: str,
     body: str,
-    notify_type: Any,
+    notify_type: object,
 ) -> dict[str, Any]:
+    import os
+
     normalized_urls = _normalize_urls(list(urls))
+    url_schemes = _extract_url_schemes(normalized_urls)
     if not normalized_urls:
         return {"success": False, "message": "No notification URLs configured"}
 
     if apprise is None:
         return {"success": False, "message": "Apprise is not installed"}
 
-    apobj = _create_apprise_client()
-    if apobj is None:
-        return {"success": False, "message": "Apprise is not installed"}
+    proxy_env = _apprise_proxy_env()
+    if proxy_env:
+        logger.debug("Applying proxy env for Apprise dispatch: %s", list(proxy_env.keys()))
+        os.environ.update(proxy_env)
+
     valid_urls = 0
     invalid_urls = 0
-    for url in normalized_urls:
-        try:
-            added = bool(apobj.add(url))
-        except Exception:
-            added = False
-        if added:
-            valid_urls += 1
-        else:
-            invalid_urls += 1
+    delivered_urls = 0
+    failed_delivery_urls = 0
+    failure_details: list[str] = []
 
+    for url in normalized_urls:
+        scheme = urlsplit(url).scheme or "unknown"
+        apobj = _create_apprise_client()
+        if apobj is None:
+            return {"success": False, "message": "Apprise is not installed"}
+
+        registration_failure_detail: str | None = None
+        with _capture_apprise_logs(min_level=logging.INFO) as apprise_records:
+            try:
+                plugin = apprise.Apprise.instantiate(url, asset=getattr(apobj, "asset", None))
+            except _APPRISE_DISPATCH_ERRORS as exc:
+                logger.warning(
+                    "Failed to register notification route URL for scheme '%s': %s",
+                    scheme,
+                    exc,
+                )
+                _log_apprise_exception_debug(
+                    action="route registration",
+                    scheme=scheme,
+                    exc=exc,
+                )
+                registration_failure_detail = (
+                    f"{scheme}: route registration failed ({type(exc).__name__}: {exc})"
+                )
+                failure_details.append(registration_failure_detail)
+                plugin = None
+
+            if plugin is None:
+                invalid_urls += 1
+                logger.warning("Apprise rejected notification route URL for scheme '%s'", scheme)
+                _log_apprise_records(apprise_records)
+                warning_detail = _build_apprise_warning_detail(apprise_records, scheme=scheme)
+                if warning_detail:
+                    failure_details.append(warning_detail)
+                elif registration_failure_detail is None:
+                    failure_details.append(f"{scheme}: route URL rejected by Apprise")
+                continue
+
+            plugin_label = _plugin_label(plugin, scheme)
+            apobj.add(plugin)
+            valid_urls += 1
+
+            try:
+                delivered = bool(apobj.notify(title=title, body=body, notify_type=notify_type))
+            except _APPRISE_DISPATCH_ERRORS as exc:
+                _log_apprise_records(apprise_records)
+                failed_delivery_urls += 1
+                logger.warning(
+                    "Apprise notify raised %s for %s: %s",
+                    type(exc).__name__,
+                    plugin_label,
+                    exc,
+                )
+                _log_apprise_exception_debug(action="notify", scheme=scheme, exc=exc)
+                warning_detail = _build_apprise_warning_detail(apprise_records, scheme=scheme)
+                if warning_detail:
+                    failure_details.append(warning_detail)
+                else:
+                    failure_details.append(f"{scheme}: notify raised {type(exc).__name__}: {exc}")
+                continue
+
+        _log_apprise_records(apprise_records)
+        if delivered:
+            delivered_urls += 1
+            logger.debug("Notification delivered via %s", plugin_label)
+            continue
+
+        failed_delivery_urls += 1
+        logger.warning("Apprise notify returned False for %s", plugin_label)
+        warning_detail = _build_apprise_warning_detail(apprise_records, scheme=scheme)
+        if warning_detail:
+            failure_details.append(warning_detail)
+        else:
+            failure_details.append(f"{scheme}: delivery failed")
+
+    scheme_summary = ", ".join(url_schemes) if url_schemes else "unknown"
     if valid_urls == 0:
-        return {
+        logger.warning(
+            "No valid Apprise notification routes after registration for scheme(s): %s",
+            scheme_summary,
+        )
+        result: dict[str, Any] = {
             "success": False,
             "message": "No valid notification URLs configured",
         }
+        if failure_details:
+            result["details"] = failure_details
+        return result
 
-    try:
-        delivered = bool(apobj.notify(title=title, body=body, notify_type=notify_type))
-    except Exception as exc:
-        return {"success": False, "message": f"Notification send failed: {type(exc).__name__}: {exc}"}
+    if delivered_urls == 0:
+        logger.warning(
+            (
+                "Apprise notify returned False for scheme(s): %s "
+                "(valid_urls=%s invalid_urls=%s failed_deliveries=%s)"
+            ),
+            scheme_summary,
+            valid_urls,
+            invalid_urls,
+            failed_delivery_urls,
+        )
+        result = {"success": False, "message": "Notification delivery failed"}
+        if failure_details:
+            result["details"] = failure_details
+        return result
 
-    if not delivered:
-        return {"success": False, "message": "Notification delivery failed"}
-
-    message = f"Notification sent to {valid_urls} URL(s)"
-    if invalid_urls:
-        message += f" ({invalid_urls} invalid URL(s) skipped)"
-    return {"success": True, "message": message}
+    message = f"Notification sent to {delivered_urls} URL(s)"
+    failed_urls = invalid_urls + failed_delivery_urls
+    if failed_urls:
+        message += f" ({failed_urls} URL(s) failed)"
+    result = {"success": True, "message": message}
+    if failure_details:
+        result["details"] = failure_details
+    return result
 
 
-def _create_apprise_client() -> Any:
+def _create_apprise_client() -> _AppriseClient | None:
     if apprise is None:
         return None
 
@@ -279,7 +581,8 @@ def _create_apprise_client() -> Any:
 
     apprise_asset_cls = getattr(apprise, "AppriseAsset", None)
     if apprise_asset_cls is None:
-        return apprise_cls()
+        client = apprise_cls()
+        return client if _is_apprise_client(client) else None
 
     try:
         asset = apprise_asset_cls(
@@ -289,20 +592,25 @@ def _create_apprise_client() -> Any:
         )
     except TypeError:
         # Support older Apprise versions that do not expose image_url_logo.
-        asset = apprise_asset_cls(
-            app_id=_APPRISE_APP_ID,
-            app_desc=_APPRISE_APP_DESC,
-        )
-    except Exception:
-        return apprise_cls()
+        try:
+            asset = apprise_asset_cls(
+                app_id=_APPRISE_APP_ID,
+                app_desc=_APPRISE_APP_DESC,
+            )
+        except TypeError:
+            client = apprise_cls()
+            return client if _is_apprise_client(client) else None
 
     try:
-        return apprise_cls(asset=asset)
-    except Exception:
-        return apprise_cls()
+        client = apprise_cls(asset=asset)
+    except TypeError:
+        client = apprise_cls()
+    return client if _is_apprise_client(client) else None
 
 
-def _send_admin_event(event: NotificationEvent, context: NotificationContext, urls: list[str]) -> dict[str, Any]:
+def _send_admin_event(
+    event: NotificationEvent, context: NotificationContext, urls: list[str]
+) -> dict[str, Any]:
     title, body = _render_message(context)
     notify_type = _resolve_notify_type(event)
     return _dispatch_to_apprise(urls, title=title, body=body, notify_type=notify_type)
@@ -317,11 +625,13 @@ def notify_admin(event: NotificationEvent, context: NotificationContext) -> None
 
     try:
         _executor.submit(_dispatch_admin_async, event, context, urls)
-    except Exception as exc:
+    except RuntimeError as exc:
         logger.warning("Failed to queue admin notification '%s': %s", event.value, exc)
 
 
-def notify_user(user_id: int | None, event: NotificationEvent, context: NotificationContext) -> None:
+def notify_user(
+    user_id: int | None, event: NotificationEvent, context: NotificationContext
+) -> None:
     """Send a per-user notification for an event if subscribed."""
     normalized_user_id = _normalize_user_id(user_id)
     if normalized_user_id is None:
@@ -334,7 +644,7 @@ def notify_user(user_id: int | None, event: NotificationEvent, context: Notifica
 
     try:
         _executor.submit(_dispatch_user_async, normalized_user_id, event, context, urls)
-    except Exception as exc:
+    except RuntimeError as exc:
         logger.warning(
             "Failed to queue user notification '%s' for user_id=%s: %s",
             event.value,
@@ -343,10 +653,16 @@ def notify_user(user_id: int | None, event: NotificationEvent, context: Notifica
         )
 
 
-def _dispatch_admin_async(event: NotificationEvent, context: NotificationContext, urls: list[str]) -> None:
+def _dispatch_admin_async(
+    event: NotificationEvent, context: NotificationContext, urls: list[str]
+) -> None:
     result = _send_admin_event(event, context, urls)
     if not result.get("success", False):
-        logger.warning("Admin notification failed for event '%s': %s", event.value, result.get("message"))
+        logger.warning(
+            "Admin notification failed for event '%s': %s",
+            event.value,
+            result.get("message"),
+        )
 
 
 def _dispatch_user_async(

shelfmark/core/oidc_auth.py

@@ -4,12 +4,15 @@ Handles group claim parsing, user info extraction, and user provisioning.
 Flask route handlers are registered separately in main.py.
 """
 
-from typing import Any, Dict, List, Optional
+from typing import TYPE_CHECKING, Any
 
 from shelfmark.core.external_user_linking import upsert_external_user
-from shelfmark.core.user_db import UserDB
 
-def parse_group_claims(id_token: Dict[str, Any], group_claim: str) -> List[str]:
+if TYPE_CHECKING:
+    from shelfmark.core.user_db import UserDB
+
+
+def parse_group_claims(id_token: dict[str, Any], group_claim: str) -> list[str]:
     """Extract group list from an ID token claim.
 
     Supports list, comma-separated string, or pipe-separated string.
@@ -26,7 +29,7 @@ def parse_group_claims(id_token: Dict[str, Any], group_claim: str) -> List[str]:
     return []
 
 
-def extract_user_info(id_token: Dict[str, Any]) -> Dict[str, Any]:
+def extract_user_info(id_token: dict[str, Any]) -> dict[str, Any]:
     """Extract user info from OIDC ID token claims.
 
     Returns a dict with keys: oidc_subject, username, email, display_name.
@@ -47,11 +50,12 @@ def extract_user_info(id_token: Dict[str, Any]) -> Dict[str, Any]:
 
 def provision_oidc_user(
     db: UserDB,
-    user_info: Dict[str, Any],
-    is_admin: Optional[bool] = None,
+    user_info: dict[str, Any],
+    *,
+    is_admin: bool | None = None,
     allow_email_link: bool = False,
     allow_create: bool = True,
-) -> Optional[Dict[str, Any]]:
+) -> dict[str, Any] | None:
     """Create or update a user from OIDC claims.
 
     Matching and collision handling use the shared external user linker:

shelfmark/core/oidc_routes.py

@@ -4,93 +4,226 @@ Registers /api/auth/oidc/login and /api/auth/oidc/callback endpoints.
 Business logic remains in oidc_auth.py.
 """
 
-from typing import Any
+from __future__ import annotations
 
-from authlib.jose.errors import InvalidClaimError
+from collections.abc import Mapping
+from typing import TYPE_CHECKING, Any, Protocol, TypeGuard
+from urllib.parse import urlencode, urlsplit, urlunsplit
+
+from authlib.integrations.base_client.errors import OAuthError
 from authlib.integrations.flask_client import OAuth
+from authlib.jose.errors import InvalidClaimError
 from flask import Flask, jsonify, redirect, request, session
 
+from shelfmark.core.config import config as app_config
 from shelfmark.core.logger import setup_logger
 from shelfmark.core.oidc_auth import (
     extract_user_info,
     parse_group_claims,
     provision_oidc_user,
 )
-from shelfmark.core.settings_registry import load_config_file
-from shelfmark.core.user_db import UserDB
+from shelfmark.download.network import get_ssl_verify
+
+if TYPE_CHECKING:
+    from flask.typing import ResponseReturnValue
+
+    from shelfmark.core.user_db import UserDB
 
 logger = setup_logger(__name__)
 oauth = OAuth()
+_RETURN_TO_SESSION_KEY = "oidc_return_to"
+_OIDC_CLIENT_ERRORS = (OAuthError, OSError, RuntimeError, TypeError, ValueError)
+_EMPTY_JWKS_MESSAGE = (
+    "Authentication failed: the identity provider returned no token signing keys "
+    "(empty JWKS). If you use Authentik, select a Signing Key in the provider "
+    "settings and try again."
+)
 
 
-def _normalize_claims(raw_claims: Any) -> dict[str, Any]:
+class _ClaimsMappingLike(Protocol):
+    """Protocol for Authlib claims payloads that expose a to_dict method."""
+
+    def to_dict(self) -> Mapping[object, object]: ...
+
+
+def _has_claims_to_dict(candidate: object) -> TypeGuard[_ClaimsMappingLike]:
+    """Return True when a claims object exposes a callable to_dict method."""
+    return callable(getattr(candidate, "to_dict", None))
+
+
+def _normalize_claim_mapping(raw_claims: Mapping[object, object]) -> dict[str, Any]:
+    """Return only string-keyed claims for downstream OIDC helpers."""
+    return {key: value for key, value in raw_claims.items() if isinstance(key, str)}
+
+
+def _normalize_claims(raw_claims: object) -> dict[str, Any]:
     """Return a plain dict for claims from Authlib token/userinfo payloads."""
     if raw_claims is None:
         return {}
-    if isinstance(raw_claims, dict):
-        return raw_claims
-    if hasattr(raw_claims, "to_dict"):
-        return raw_claims.to_dict()  # type: ignore[no-any-return]
-    try:
-        return dict(raw_claims)
-    except Exception:
-        return {}
+    if isinstance(raw_claims, Mapping):
+        return _normalize_claim_mapping(raw_claims)
+    if _has_claims_to_dict(raw_claims):
+        converted_claims = raw_claims.to_dict()
+        if isinstance(converted_claims, Mapping):
+            return _normalize_claim_mapping(converted_claims)
+    return {}
+
+
+def _has_username_or_email(claims: dict[str, Any]) -> bool:
+    """Return True when claims include a usable username or email."""
+    for key in ("preferred_username", "email"):
+        value = claims.get(key)
+        if isinstance(value, str) and value.strip():
+            return True
+    return False
 
 
 def _is_email_verified(claims: dict[str, Any]) -> bool:
-    """Normalize provider-specific email_verified values into a strict boolean."""
-    value = claims.get("email_verified", False)
-    if isinstance(value, bool):
-        return value
-    if isinstance(value, str):
-        return value.strip().lower() == "true"
+    """Return True when claims explicitly mark the email address as verified."""
+    email_verified = claims.get("email_verified")
+    if isinstance(email_verified, bool):
+        return email_verified
+    if isinstance(email_verified, str):
+        return email_verified.strip().lower() == "true"
     return False
 
 
+def _login_error_url(message: str) -> str:
+    """Build a login URL (with script_root) that includes an OIDC error message."""
+    script_root = request.script_root.rstrip("/")
+    login_url = f"{script_root}/login" if script_root else "/login"
+    params = {"oidc_error": message}
+    return_to = _get_pending_return_to()
+    if return_to and return_to != "/":
+        params["return_to"] = return_to
+    return f"{login_url}?{urlencode(params)}"
+
+
+def _normalize_return_to(raw_return_to: object) -> str | None:
+    """Return a safe app-relative post-login target."""
+    if not isinstance(raw_return_to, str):
+        return None
+
+    value = raw_return_to.strip()
+    if not value or not value.startswith("/") or value.startswith("//"):
+        return None
+
+    parsed = urlsplit(value)
+    if parsed.scheme or parsed.netloc or "\\" in parsed.path:
+        return None
+
+    script_root = request.script_root.rstrip("/")
+    path = parsed.path or "/"
+    if script_root:
+        if path == script_root:
+            path = "/"
+        elif path.startswith(f"{script_root}/"):
+            path = path[len(script_root) :] or "/"
+
+    if path in {"/login", "/api"} or path.startswith(("/login/", "/api/")):
+        return None
+
+    return urlunsplit(("", "", path, parsed.query, parsed.fragment))
+
+
+def _idp_jwks_has_no_keys(client: Any) -> bool:
+    """Return True when the IdP's JWKS document verifiably contains no signing keys."""
+    try:
+        jwk_set = client.fetch_jwk_set(force=True)
+    except (*_OIDC_CLIENT_ERRORS, KeyError):
+        return False
+    if not isinstance(jwk_set, Mapping):
+        return False
+    return not jwk_set.get("keys")
+
+
+def _get_pending_return_to(*, clear: bool = False) -> str | None:
+    """Read the pending post-login target from the session."""
+    raw_return_to = (
+        session.pop(_RETURN_TO_SESSION_KEY, None) if clear else session.get(_RETURN_TO_SESSION_KEY)
+    )
+    normalized = _normalize_return_to(raw_return_to)
+    if normalized is None and not clear:
+        session.pop(_RETURN_TO_SESSION_KEY, None)
+    return normalized
+
+
+def _post_login_redirect_target(return_to: str | None) -> str:
+    """Build the final redirect target, honoring script_root when present."""
+    normalized = _normalize_return_to(return_to) or "/"
+    script_root = request.script_root.rstrip("/")
+    if not script_root:
+        return normalized
+    if normalized == "/":
+        return f"{script_root}/"
+    return f"{script_root}{normalized}"
+
+
 def _get_oidc_client() -> tuple[Any, dict[str, Any]]:
     """Register and return an OIDC client from the current security config."""
-    config = load_config_file("security")
-    discovery_url = config.get("OIDC_DISCOVERY_URL", "")
-    client_id = config.get("OIDC_CLIENT_ID", "")
+    discovery_url = str(app_config.get("OIDC_DISCOVERY_URL", "") or "")
+    client_id = str(app_config.get("OIDC_CLIENT_ID", "") or "")
 
     if not discovery_url or not client_id:
-        raise ValueError("OIDC not configured")
+        msg = "OIDC not configured"
+        raise ValueError(msg)
 
-    configured_scopes = config.get("OIDC_SCOPES", ["openid", "email", "profile"])
+    configured_scopes = app_config.get("OIDC_SCOPES", ["openid", "email", "profile"])
     if isinstance(configured_scopes, list):
         scope_values = [str(scope).strip() for scope in configured_scopes if str(scope).strip()]
     elif isinstance(configured_scopes, str):
         delimiter = "," if "," in configured_scopes else " "
-        scope_values = [scope.strip() for scope in configured_scopes.split(delimiter) if scope.strip()]
+        scope_values = [
+            scope.strip() for scope in configured_scopes.split(delimiter) if scope.strip()
+        ]
     else:
         scope_values = []
 
-    scopes = list(dict.fromkeys(["openid"] + scope_values))
+    scopes = list(dict.fromkeys(["openid", *scope_values]))
 
-    admin_group = config.get("OIDC_ADMIN_GROUP", "")
-    group_claim = config.get("OIDC_GROUP_CLAIM", "groups")
-    use_admin_group = config.get("OIDC_USE_ADMIN_GROUP", True)
+    admin_group_value = app_config.get("OIDC_ADMIN_GROUP", "")
+    admin_group = admin_group_value.strip() if isinstance(admin_group_value, str) else ""
+    group_claim_value = app_config.get("OIDC_GROUP_CLAIM", "groups")
+    group_claim = (
+        group_claim_value.strip()
+        if isinstance(group_claim_value, str) and group_claim_value.strip()
+        else "groups"
+    )
+    use_admin_group = app_config.get("OIDC_USE_ADMIN_GROUP", True)
     if admin_group and use_admin_group and group_claim and group_claim not in scopes:
         scopes.append(group_claim)
 
+    def _ssl_compliance_fix(session: Any, **kwargs: Any) -> Any:
+        """Set session.verify based on the Certificate Validation setting."""
+        session.verify = get_ssl_verify(discovery_url)
+        return session
+
     oauth._clients.pop("shelfmark_idp", None)
     oauth.register(
         name="shelfmark_idp",
         client_id=client_id,
-        client_secret=config.get("OIDC_CLIENT_SECRET", ""),
+        client_secret=app_config.get("OIDC_CLIENT_SECRET", ""),
         server_metadata_url=discovery_url,
         client_kwargs={
             "scope": " ".join(scopes),
             "code_challenge_method": "S256",
         },
+        compliance_fix=_ssl_compliance_fix,
         overwrite=True,
     )
 
     client = oauth.create_client("shelfmark_idp")
     if client is None:
-        raise RuntimeError("OIDC client initialization failed")
+        msg = "OIDC client initialization failed"
+        raise RuntimeError(msg)
 
-    return client, config
+    return client, {
+        "OIDC_DISCOVERY_URL": discovery_url,
+        "OIDC_GROUP_CLAIM": group_claim,
+        "OIDC_ADMIN_GROUP": admin_group,
+        "OIDC_AUTO_PROVISION": app_config.get("OIDC_AUTO_PROVISION", True),
+        "OIDC_USE_ADMIN_GROUP": use_admin_group,
+    }
 
 
 def register_oidc_routes(app: Flask, user_db: UserDB) -> None:
@@ -98,26 +231,31 @@ def register_oidc_routes(app: Flask, user_db: UserDB) -> None:
     oauth.init_app(app)
 
     @app.route("/api/auth/oidc/login", methods=["GET"])
-    def oidc_login():
+    def oidc_login() -> ResponseReturnValue:
         """Initiate OIDC login flow and redirect to the provider."""
         try:
             client, _ = _get_oidc_client()
+            return_to = _normalize_return_to(request.args.get("return_to"))
+            if return_to and return_to != "/":
+                session[_RETURN_TO_SESSION_KEY] = return_to
+            else:
+                session.pop(_RETURN_TO_SESSION_KEY, None)
             redirect_uri = request.url_root.rstrip("/") + "/api/auth/oidc/callback"
             return client.authorize_redirect(redirect_uri)
         except ValueError:
             return jsonify({"error": "OIDC not configured"}), 500
-        except Exception as e:
-            logger.error(f"OIDC login error: {e}")
+        except Exception:
+            logger.exception("OIDC login error")
             return jsonify({"error": "OIDC login failed"}), 500
 
     @app.route("/api/auth/oidc/callback", methods=["GET"])
-    def oidc_callback():
+    def oidc_callback() -> ResponseReturnValue:
         """Handle OIDC callback from identity provider."""
         try:
             error = request.args.get("error")
             if error:
-                logger.warning(f"OIDC callback error from IdP: {error}")
-                return jsonify({"error": "Authentication failed"}), 400
+                logger.warning("OIDC callback error from IdP: %s", error)
+                return redirect(_login_error_url("Authentication failed"))
 
             client, config = _get_oidc_client()
             try:
@@ -130,43 +268,57 @@ def register_oidc_routes(app: Flask, user_db: UserDB) -> None:
                     metadata = client.load_server_metadata()
                     if isinstance(metadata, dict):
                         provider_issuer = str(metadata.get("issuer", ""))
-                except Exception as metadata_error:
-                    logger.debug(f"OIDC metadata lookup failed during claim diagnostics: {metadata_error}")
+                except _OIDC_CLIENT_ERRORS as metadata_error:
+                    logger.debug(
+                        "OIDC metadata lookup failed during claim diagnostics: %s",
+                        metadata_error,
+                    )
 
-                logger.error(
-                    "OIDC callback claim validation failed: claim=%s error=%s discovery_url=%s provider_issuer=%s",
+                logger.exception(
+                    "OIDC callback claim validation failed: claim=%s discovery_url=%s provider_issuer=%s",
                     claim_name,
-                    e,
                     discovery_url or "<unset>",
                     provider_issuer or "<unknown>",
                 )
                 if claim_name == "iss":
-                    return (
-                        jsonify(
-                            {
-                                "error": (
-                                    "OIDC issuer validation failed. Verify your discovery URL and IdP issuer/"
-                                    "external URL configuration."
-                                )
-                            }
-                        ),
-                        400,
+                    msg = (
+                        "OIDC issuer validation failed. Verify your discovery URL and IdP issuer/"
+                        "external URL configuration."
                     )
+                    return redirect(_login_error_url(msg))
 
-                return jsonify({"error": f"OIDC token claim validation failed: {claim_name}"}), 400
+                return redirect(
+                    _login_error_url(f"OIDC token claim validation failed: {claim_name}")
+                )
+            except KeyError, ValueError:
+                # An IdP serving an empty JWKS document (e.g. an Authentik provider
+                # with no Signing Key selected) surfaces as KeyError('keys') while
+                # importing the key set. Test Connection only validates discovery,
+                # so this is the first place the misconfiguration becomes visible.
+                if _idp_jwks_has_no_keys(client):
+                    logger.exception(
+                        "OIDC callback failed: the IdP JWKS document contains no signing keys"
+                    )
+                    return redirect(_login_error_url(_EMPTY_JWKS_MESSAGE))
+                raise
             claims = _normalize_claims(token.get("userinfo"))
 
-            # If userinfo isn't present in token payload, request it explicitly.
-            if not claims:
+            # If userinfo is missing or claims are too sparse, request it explicitly.
+            if not claims or not _has_username_or_email(claims):
+                fetched_claims: dict[str, Any] = {}
                 try:
-                    claims = _normalize_claims(client.userinfo(token=token))
+                    fetched_claims = _normalize_claims(client.userinfo(token=token))
                 except TypeError:
-                    claims = _normalize_claims(client.userinfo())
-                except Exception as e:
-                    logger.error(f"Failed to fetch OIDC userinfo: {e}")
+                    fetched_claims = _normalize_claims(client.userinfo())
+                except Exception:
+                    logger.exception("Failed to fetch OIDC userinfo")
+                if fetched_claims:
+                    claims = {**claims, **fetched_claims}
 
             if not claims:
-                raise ValueError("OIDC authentication failed: missing user claims")
+                msg = "OIDC authentication failed: missing user claims"
+                logger.error(msg)
+                return redirect(_login_error_url(msg))
 
             group_claim = config.get("OIDC_GROUP_CLAIM", "groups")
             admin_group = config.get("OIDC_ADMIN_GROUP", "")
@@ -181,6 +333,12 @@ def register_oidc_routes(app: Flask, user_db: UserDB) -> None:
                 is_admin = admin_group in groups
 
             allow_email_link = bool(user_info.get("email")) and _is_email_verified(claims)
+            if user_info.get("email") and not allow_email_link:
+                logger.debug(
+                    "OIDC email %s is not marked verified by the IdP; skipping "
+                    "email-based account linking",
+                    user_info["email"],
+                )
             user = provision_oidc_user(
                 user_db,
                 user_info,
@@ -190,21 +348,22 @@ def register_oidc_routes(app: Flask, user_db: UserDB) -> None:
             )
             if user is None:
                 logger.warning(
-                    f"OIDC login rejected: auto-provision disabled for {user_info['username']}"
+                    "OIDC login rejected: auto-provision disabled for %s",
+                    user_info["username"],
                 )
-                return jsonify({"error": "Account not found. Contact your administrator."}), 403
+                return redirect(_login_error_url("Account not found. Contact your administrator."))
 
             session["user_id"] = user["username"]
             session["is_admin"] = user.get("role") == "admin"
             session["db_user_id"] = user["id"]
             session.permanent = True
 
-            logger.info(f"OIDC login successful: {user['username']} (admin={is_admin})")
-            return redirect(request.script_root or "/")
+            logger.info("OIDC login successful: %s (admin=%s)", user["username"], is_admin)
+            return redirect(_post_login_redirect_target(_get_pending_return_to(clear=True)))
 
         except ValueError as e:
-            logger.error(f"OIDC callback error: {e}")
-            return jsonify({"error": str(e)}), 400
-        except Exception as e:
-            logger.error(f"OIDC callback error: {e}")
-            return jsonify({"error": "Authentication failed"}), 500
+            logger.exception("OIDC callback error")
+            return redirect(_login_error_url(str(e)))
+        except Exception:
+            logger.exception("OIDC callback error")
+            return redirect(_login_error_url("Authentication failed"))

shelfmark/core/onboarding.py

@@ -1,5 +1,4 @@
-"""
-Onboarding wizard configuration.
+"""Onboarding wizard configuration.
 
 Defines the steps and fields for the first-run onboarding experience.
 Reuses field definitions from the settings registry where possible.
@@ -8,27 +7,32 @@ Reuses field definitions from the settings registry where possible.
 import json
 from dataclasses import replace
 from pathlib import Path
-from typing import Any, Dict, List, Optional
+from typing import Any
 
 from shelfmark.core.logger import setup_logger
 from shelfmark.core.settings_registry import (
     HeadingField,
+    MultiSelectField,
     SettingsField,
-    get_settings_tab,
-    serialize_field,
-    save_config_file,
     get_setting_value,
+    get_settings_field_map,
+    get_settings_tab,
+    save_config_file,
+    serialize_field,
 )
 
 logger = setup_logger(__name__)
 
 
 ONBOARDING_STORAGE_KEY = "onboarding_complete"
+ONBOARDING_RELEASE_SOURCES_KEY = "ONBOARDING_RELEASE_SOURCES"
+_ONBOARDING_VIRTUAL_KEYS = {ONBOARDING_RELEASE_SOURCES_KEY}
 
 
 def _get_config_dir() -> Path:
     """Get the config directory path."""
     from shelfmark.config.env import CONFIG_DIR
+
     return Path(CONFIG_DIR)
 
 
@@ -45,11 +49,11 @@ def is_onboarding_complete() -> bool:
         return False
 
     try:
-        with open(config_file, 'r') as f:
+        with config_file.open() as f:
             config = json.load(f)
             return config.get(ONBOARDING_STORAGE_KEY, False)
     except (json.JSONDecodeError, OSError) as e:
-        logger.warning(f"Could not read onboarding status from settings.json: {e}")
+        logger.warning("Could not read onboarding status from settings.json: %s", e)
         return False
 
 
@@ -57,14 +61,13 @@ def mark_onboarding_complete() -> bool:
     """Mark onboarding as complete."""
     try:
         return save_config_file("general", {ONBOARDING_STORAGE_KEY: True})
-    except Exception as e:
-        logger.error(f"Failed to mark onboarding complete: {e}")
+    except Exception:
+        logger.exception("Failed to mark onboarding complete")
         return False
 
 
-def _get_field_from_tab(tab_name: str, field_key: str) -> Optional[SettingsField]:
-    """
-    Extract a specific field from a registered settings tab.
+def _get_field_from_tab(tab_name: str, field_key: str) -> SettingsField | None:
+    """Extract a specific field from a registered settings tab.
 
     Args:
         tab_name: Name of the settings tab (e.g., 'search_mode', 'hardcover')
@@ -72,37 +75,145 @@ def _get_field_from_tab(tab_name: str, field_key: str) -> Optional[SettingsField
 
     Returns:
         The field if found, None otherwise
+
     """
     tab = get_settings_tab(tab_name)
     if not tab:
-        logger.warning(f"Settings tab not found: {tab_name}")
+        logger.warning("Settings tab not found: %s", tab_name)
         return None
 
     for field in tab.fields:
-        if hasattr(field, 'key') and field.key == field_key:
+        if hasattr(field, "key") and field.key == field_key:
             return field
 
-    logger.warning(f"Field {field_key} not found in tab {tab_name}")
+    logger.warning("Field %s not found in tab %s", field_key, tab_name)
     return None
 
 
-def _clone_field_with_overrides(field: SettingsField, **overrides) -> SettingsField:
-    """
-    Clone a field with optional attribute overrides.
+def _get_field_tab_name(field: SettingsField, fallback_tab_name: str) -> str:
+    """Return the owning settings tab for a value field."""
+    field_key = getattr(field, "key", None)
+    if not field_key:
+        return fallback_tab_name
+
+    field_map = get_settings_field_map()
+    field_entry = field_map.get(field_key)
+    if field_entry is None:
+        return fallback_tab_name
+
+    return field_entry[1]
+
+
+def _clone_field_with_overrides(field: SettingsField, **overrides: object) -> SettingsField:
+    """Clone a field with optional attribute overrides.
 
     Useful for customizing labels, descriptions, or defaults for onboarding context.
     """
     return replace(field, **overrides)
 
 
+def _get_fields_from_tab(
+    tab_name: str,
+    field_keys: list[str],
+    *,
+    strip_show_when_keys: set[str] | None = None,
+) -> list[SettingsField]:
+    """Return the requested fields from a settings tab in the supplied order."""
+    fields: list[SettingsField] = []
+    for field_key in field_keys:
+        field = _get_field_from_tab(tab_name, field_key)
+        if field:
+            show_when = getattr(field, "show_when", None)
+            stripped_show_when = _strip_show_when_keys(show_when, strip_show_when_keys or set())
+            if stripped_show_when != show_when:
+                field = replace(field, show_when=stripped_show_when)
+            fields.append(field)
+    return fields
+
+
+def _strip_show_when_keys(
+    show_when: dict[str, Any] | list[dict[str, Any]] | None,
+    field_keys: set[str],
+) -> dict[str, Any] | list[dict[str, Any]] | None:
+    """Remove conditions tied to fields that onboarding handles implicitly."""
+    if not show_when or not field_keys:
+        return show_when
+
+    if isinstance(show_when, list):
+        remaining = [
+            condition for condition in show_when if condition.get("field") not in field_keys
+        ]
+        return remaining or None
+
+    if show_when.get("field") in field_keys:
+        return None
+
+    return show_when
+
+
+def _is_release_source_selected(values: dict[str, Any], source_name: str) -> bool:
+    """Return True when a release source has been chosen during onboarding."""
+    raw_sources = values.get(ONBOARDING_RELEASE_SOURCES_KEY, [])
+    if not isinstance(raw_sources, list):
+        return False
+    return source_name in raw_sources
+
+
+def _evaluate_show_when_condition(condition: dict[str, Any], values: dict[str, Any]) -> bool:
+    """Evaluate one onboarding show_when condition against submitted values."""
+    current_value = values.get(condition["field"])
+    expected_value = condition.get("value")
+
+    if condition.get("notEmpty"):
+        if isinstance(current_value, list):
+            return len(current_value) > 0
+        return current_value not in (None, "")
+
+    if isinstance(current_value, list):
+        if isinstance(expected_value, list):
+            return all(item in current_value for item in expected_value)
+        return expected_value in current_value
+
+    if isinstance(expected_value, list):
+        return current_value in expected_value
+
+    return current_value == expected_value
+
+
+def _is_step_visible(step_config: dict[str, Any], values: dict[str, Any]) -> bool:
+    """Return True when a step should be included for the provided values."""
+    show_when = step_config.get("show_when")
+    if not show_when:
+        return True
+    return all(_evaluate_show_when_condition(condition, values) for condition in show_when)
+
+
+def _is_field_visible(field: SettingsField, values: dict[str, Any]) -> bool:
+    """Return True when a field should be included in the onboarding save."""
+    if getattr(field, "hidden_in_ui", False):
+        return False
+
+    if getattr(field, "universal_only", False) and values.get("SEARCH_MODE") != "universal":
+        return False
+
+    show_when = getattr(field, "show_when", None)
+    if not show_when:
+        return True
+
+    if isinstance(show_when, list):
+        return all(_evaluate_show_when_condition(condition, values) for condition in show_when)
+
+    return _evaluate_show_when_condition(show_when, values)
+
+
 # =============================================================================
 # Step Definitions
 # =============================================================================
 
 
-def get_search_mode_fields() -> List[SettingsField]:
+def get_search_mode_fields() -> list[SettingsField]:
     """Step 1: Choose search mode - uses actual SEARCH_MODE field from settings."""
-    fields: List[SettingsField] = [
+    fields: list[SettingsField] = [
         HeadingField(
             key="welcome_heading",
             title="Welcome to Shelfmark",
@@ -114,17 +225,19 @@ def get_search_mode_fields() -> List[SettingsField]:
     search_mode_field = _get_field_from_tab("search_mode", "SEARCH_MODE")
     if search_mode_field:
         # Clone with onboarding-specific description
-        fields.append(_clone_field_with_overrides(
-            search_mode_field,
-            description="Choose how you want to find books.",
-        ))
+        fields.append(
+            _clone_field_with_overrides(
+                search_mode_field,
+                description="Choose how you want to find books.",
+            )
+        )
 
     return fields
 
 
-def get_metadata_provider_fields() -> List[SettingsField]:
+def get_metadata_provider_fields() -> list[SettingsField]:
     """Step 2: Choose metadata provider - uses actual METADATA_PROVIDER field."""
-    fields: List[SettingsField] = [
+    fields: list[SettingsField] = [
         HeadingField(
             key="metadata_heading",
             title="Metadata Provider",
@@ -155,18 +268,20 @@ def get_metadata_provider_fields() -> List[SettingsField]:
         ]
 
         # Clone with onboarding-specific options and default
-        fields.append(_clone_field_with_overrides(
-            provider_field,
-            default="hardcover",
-            options=onboarding_options,
-        ))
+        fields.append(
+            _clone_field_with_overrides(
+                provider_field,
+                default="hardcover",
+                options=onboarding_options,
+            )
+        )
 
     return fields
 
 
-def get_hardcover_setup_fields() -> List[SettingsField]:
+def get_hardcover_setup_fields() -> list[SettingsField]:
     """Step 3a: Configure Hardcover - uses actual API key and test connection fields."""
-    fields: List[SettingsField] = [
+    fields: list[SettingsField] = [
         HeadingField(
             key="hardcover_setup_heading",
             title="Hardcover Setup",
@@ -189,9 +304,9 @@ def get_hardcover_setup_fields() -> List[SettingsField]:
     return fields
 
 
-def get_googlebooks_setup_fields() -> List[SettingsField]:
+def get_googlebooks_setup_fields() -> list[SettingsField]:
     """Step 3b: Configure Google Books - uses actual API key and test connection fields."""
-    fields: List[SettingsField] = [
+    fields: list[SettingsField] = [
         HeadingField(
             key="googlebooks_setup_heading",
             title="Google Books Setup",
@@ -214,129 +329,270 @@ def get_googlebooks_setup_fields() -> List[SettingsField]:
     return fields
 
 
-def get_prowlarr_fields() -> List[SettingsField]:
-    """Step 4: Configure Prowlarr connection - uses actual Prowlarr fields."""
-    fields: List[SettingsField] = [
+def get_release_source_selection_fields() -> list[SettingsField]:
+    """Choose which release sources to configure during onboarding."""
+    fields: list[SettingsField] = [
         HeadingField(
-            key="prowlarr_heading",
-            title="Prowlarr Integration (Optional)",
-            description="Connect to Prowlarr to search your indexers for torrents and NZBs. Skip this step if you only want to use Direct Download.",
+            key="release_sources_heading",
+            title="Release Sources",
+            description=(
+                "Choose the release sources you want to configure now. You can always add or "
+                "change sources later in Settings."
+            ),
+        ),
+        MultiSelectField(
+            key=ONBOARDING_RELEASE_SOURCES_KEY,
+            label="Sources to Set Up",
+            description="Select one or more release sources to configure now.",
+            default=[],
+            variant="dropdown",
+            env_supported=False,
+            options=[
+                {
+                    "value": "direct_download",
+                    "label": "Direct Download",
+                    "description": "Configure your own Anna's Archive mirror URLs for direct ebook downloads.",
+                },
+                {
+                    "value": "prowlarr",
+                    "label": "Prowlarr",
+                    "description": "Search your torrent and Usenet indexers through Prowlarr.",
+                },
+                {
+                    "value": "audiobookbay",
+                    "label": "AudiobookBay",
+                    "description": "Search AudiobookBay directly for audiobook releases.",
+                },
+                {
+                    "value": "irc",
+                    "label": "IRC",
+                    "description": "Connect to IRC for ebook and audiobook release searches.",
+                },
+            ],
         ),
     ]
-
-    # Get actual Prowlarr connection fields
-    prowlarr_fields = ["PROWLARR_ENABLED", "PROWLARR_URL", "PROWLARR_API_KEY", "test_prowlarr"]
-    for field_key in prowlarr_fields:
-        field = _get_field_from_tab("prowlarr_config", field_key)
-        if field:
-            fields.append(field)
-
     return fields
 
 
-def get_prowlarr_indexers_fields() -> List[SettingsField]:
-    """Step 5: Select Prowlarr indexers to search."""
-    fields: List[SettingsField] = [
+def get_direct_download_setup_fields() -> list[SettingsField]:
+    """Render trimmed direct-download essentials for onboarding."""
+    fields: list[SettingsField] = [
         HeadingField(
-            key="prowlarr_indexers_heading",
-            title="Select Indexers",
-            description="Choose which indexers to search for books. Leave empty to search all available indexers.",
-        ),
+            key="direct_download_setup_onboarding_heading",
+            title="Direct Download Setup",
+            description=(
+                "Add at least one Anna's Archive mirror URL to enable Direct Download. If you "
+                "have an Anna's Archive donator key, you can add it here too. You can configure "
+                "alternative mirrors later in Settings."
+            ),
+        )
     ]
-
-    # Get the indexers multi-select field
-    indexers_field = _get_field_from_tab("prowlarr_config", "PROWLARR_INDEXERS")
-    if indexers_field:
-        fields.append(indexers_field)
-
+    fields.extend(_get_fields_from_tab("download_sources", ["AA_DONATOR_KEY"]))
+    fields.extend(_get_fields_from_tab("mirrors", ["AA_MIRROR_URLS"]))
     return fields
 
 
-# =============================================================================
-# Step Configuration
-# =============================================================================
-
-
-ONBOARDING_STEPS = [
-    {
-        "id": "search_mode",
-        "title": "Search Mode",
-        "tab": "search_mode",
-        "get_fields": get_search_mode_fields,
-    },
-    {
-        "id": "metadata_provider",
-        "title": "Metadata Provider",
-        "tab": "search_mode",
-        "get_fields": get_metadata_provider_fields,
-        "show_when": [{"field": "SEARCH_MODE", "value": "universal"}],
-    },
-    {
-        "id": "hardcover_setup",
-        "title": "Hardcover Setup",
-        "tab": "hardcover",
-        "get_fields": get_hardcover_setup_fields,
-        # Must be universal mode AND hardcover selected
-        "show_when": [
-            {"field": "SEARCH_MODE", "value": "universal"},
-            {"field": "METADATA_PROVIDER", "value": "hardcover"},
+def get_direct_download_bypass_fields() -> list[SettingsField]:
+    """Render only the core Cloudflare bypass fields for onboarding."""
+    return _get_fields_from_tab(
+        "cloudflare_bypass",
+        [
+            "USE_CF_BYPASS",
+            "USING_EXTERNAL_BYPASSER",
+            "EXT_BYPASSER_URL",
+            "EXT_BYPASSER_PATH",
         ],
-    },
-    {
-        "id": "googlebooks_setup",
-        "title": "Google Books Setup",
-        "tab": "googlebooks",
-        "get_fields": get_googlebooks_setup_fields,
-        # Must be universal mode AND googlebooks selected
-        "show_when": [
-            {"field": "SEARCH_MODE", "value": "universal"},
-            {"field": "METADATA_PROVIDER", "value": "googlebooks"},
-        ],
-    },
-    {
-        "id": "prowlarr",
-        "title": "Prowlarr",
-        "tab": "prowlarr_config",
-        "get_fields": get_prowlarr_fields,
-        "show_when": [{"field": "SEARCH_MODE", "value": "universal"}],
-        "optional": True,
-    },
-    {
-        "id": "prowlarr_indexers",
-        "title": "Indexers",
-        "tab": "prowlarr_config",
-        "get_fields": get_prowlarr_indexers_fields,
-        # Only show when Prowlarr is enabled
-        "show_when": [
-            {"field": "SEARCH_MODE", "value": "universal"},
-            {"field": "PROWLARR_ENABLED", "value": True},
-        ],
-        "optional": True,
-    },
-]
+    )
 
 
-def get_onboarding_config() -> Dict[str, Any]:
-    """
-    Get the full onboarding configuration including steps and current values.
-    """
+def get_prowlarr_fields() -> list[SettingsField]:
+    """Render trimmed Prowlarr setup fields for onboarding."""
+    return _get_fields_from_tab(
+        "prowlarr_config",
+        [
+            "prowlarr_heading",
+            "PROWLARR_URL",
+            "PROWLARR_API_KEY",
+            "test_prowlarr",
+            "PROWLARR_INDEXERS",
+        ],
+        strip_show_when_keys={"PROWLARR_ENABLED"},
+    )
+
+
+def get_audiobookbay_fields() -> list[SettingsField]:
+    """Render trimmed AudiobookBay setup fields for onboarding."""
+    return [
+        HeadingField(
+            key="audiobookbay_onboarding_heading",
+            title="AudiobookBay",
+            description="Add the AudiobookBay domain you want Shelfmark to search.",
+        ),
+        *_get_fields_from_tab(
+            "audiobookbay_config",
+            ["ABB_HOSTNAME"],
+            strip_show_when_keys={"ABB_ENABLED"},
+        ),
+    ]
+
+
+def get_irc_fields() -> list[SettingsField]:
+    """Render trimmed IRC setup fields for onboarding."""
+    return _get_fields_from_tab(
+        "irc",
+        [
+            "heading",
+            "IRC_SERVER",
+            "IRC_PORT",
+            "IRC_USE_TLS",
+            "IRC_CHANNEL",
+            "IRC_NICK",
+            "IRC_SEARCH_BOT",
+        ],
+    )
+
+
+def get_onboarding_steps() -> list[dict[str, Any]]:
+    """Return the full onboarding step configuration."""
+    return [
+        {
+            "id": "search_mode",
+            "title": "Search Mode",
+            "tab": "search_mode",
+            "get_fields": get_search_mode_fields,
+        },
+        {
+            "id": "metadata_provider",
+            "title": "Metadata Provider",
+            "tab": "search_mode",
+            "get_fields": get_metadata_provider_fields,
+            "show_when": [{"field": "SEARCH_MODE", "value": "universal"}],
+        },
+        {
+            "id": "hardcover_setup",
+            "title": "Hardcover Setup",
+            "tab": "hardcover",
+            "get_fields": get_hardcover_setup_fields,
+            "show_when": [
+                {"field": "SEARCH_MODE", "value": "universal"},
+                {"field": "METADATA_PROVIDER", "value": "hardcover"},
+            ],
+        },
+        {
+            "id": "googlebooks_setup",
+            "title": "Google Books Setup",
+            "tab": "googlebooks",
+            "get_fields": get_googlebooks_setup_fields,
+            "show_when": [
+                {"field": "SEARCH_MODE", "value": "universal"},
+                {"field": "METADATA_PROVIDER", "value": "googlebooks"},
+            ],
+        },
+        {
+            "id": "release_sources",
+            "title": "Release Sources",
+            "tab": "search_mode",
+            "get_fields": get_release_source_selection_fields,
+            "show_when": [{"field": "SEARCH_MODE", "value": "universal"}],
+            "optional": True,
+        },
+        {
+            "id": "direct_download_setup_direct_mode",
+            "title": "Direct Download Setup",
+            "tab": "download_sources",
+            "get_fields": get_direct_download_setup_fields,
+            "show_when": [{"field": "SEARCH_MODE", "value": "direct"}],
+        },
+        {
+            "id": "direct_download_cloudflare_bypass_direct_mode",
+            "title": "Cloudflare Bypass",
+            "tab": "cloudflare_bypass",
+            "get_fields": get_direct_download_bypass_fields,
+            "show_when": [{"field": "SEARCH_MODE", "value": "direct"}],
+        },
+        {
+            "id": "direct_download_setup",
+            "title": "Direct Download Setup",
+            "tab": "download_sources",
+            "get_fields": get_direct_download_setup_fields,
+            "show_when": [
+                {"field": "SEARCH_MODE", "value": "universal"},
+                {"field": ONBOARDING_RELEASE_SOURCES_KEY, "value": "direct_download"},
+            ],
+            "optional": True,
+        },
+        {
+            "id": "direct_download_cloudflare_bypass",
+            "title": "Cloudflare Bypass",
+            "tab": "cloudflare_bypass",
+            "get_fields": get_direct_download_bypass_fields,
+            "show_when": [
+                {"field": "SEARCH_MODE", "value": "universal"},
+                {"field": ONBOARDING_RELEASE_SOURCES_KEY, "value": "direct_download"},
+            ],
+            "optional": True,
+        },
+        {
+            "id": "prowlarr",
+            "title": "Prowlarr",
+            "tab": "prowlarr_config",
+            "get_fields": get_prowlarr_fields,
+            "show_when": [
+                {"field": "SEARCH_MODE", "value": "universal"},
+                {"field": ONBOARDING_RELEASE_SOURCES_KEY, "value": "prowlarr"},
+            ],
+            "optional": True,
+        },
+        {
+            "id": "audiobookbay",
+            "title": "AudiobookBay",
+            "tab": "audiobookbay_config",
+            "get_fields": get_audiobookbay_fields,
+            "show_when": [
+                {"field": "SEARCH_MODE", "value": "universal"},
+                {"field": ONBOARDING_RELEASE_SOURCES_KEY, "value": "audiobookbay"},
+            ],
+            "optional": True,
+        },
+        {
+            "id": "irc",
+            "title": "IRC",
+            "tab": "irc",
+            "get_fields": get_irc_fields,
+            "show_when": [
+                {"field": "SEARCH_MODE", "value": "universal"},
+                {"field": ONBOARDING_RELEASE_SOURCES_KEY, "value": "irc"},
+            ],
+            "optional": True,
+        },
+    ]
+
+
+def get_onboarding_config() -> dict[str, Any]:
+    """Get the full onboarding configuration including steps and current values."""
     steps = []
     all_values = {}
 
-    for step_config in ONBOARDING_STEPS:
+    for step_config in get_onboarding_steps():
         fields = step_config["get_fields"]()
         tab_name = step_config["tab"]
 
         # Serialize fields with current values
         serialized_fields = []
         for field in fields:
-            serialized = serialize_field(field, tab_name, include_value=True)
+            field_tab_name = _get_field_tab_name(field, tab_name)
+            serialized = serialize_field(field, field_tab_name, include_value=True)
             serialized_fields.append(serialized)
 
             # Collect values (skip HeadingFields)
-            if hasattr(field, 'key') and field.key and not isinstance(field, HeadingField):
-                value = get_setting_value(field, tab_name)
-                all_values[field.key] = value if value is not None else getattr(field, 'default', '')
+            if hasattr(field, "env_supported") and getattr(field, "key", None):
+                if field.key in _ONBOARDING_VIRTUAL_KEYS:
+                    value = getattr(field, "default", "")
+                else:
+                    value = get_setting_value(field, field_tab_name)
+                all_values[field.key] = (
+                    value if value is not None else getattr(field, "default", "")
+                )
 
         step = {
             "id": step_config["id"],
@@ -359,22 +615,24 @@ def get_onboarding_config() -> Dict[str, Any]:
     }
 
 
-def save_onboarding_settings(values: Dict[str, Any]) -> Dict[str, Any]:
-    """
-    Save onboarding settings and mark as complete.
+def save_onboarding_settings(values: dict[str, Any]) -> dict[str, Any]:
+    """Save onboarding settings and mark as complete.
 
     Args:
         values: Dict of field key -> value
 
     Returns:
         Dict with success status and message
+
     """
     try:
         # Group values by their target tab
-        tab_values: Dict[str, Dict[str, Any]] = {}
+        tab_values: dict[str, dict[str, Any]] = {}
+
+        for step_config in get_onboarding_steps():
+            if not _is_step_visible(step_config, values):
+                continue
 
-        for step_config in ONBOARDING_STEPS:
-            tab_name = step_config["tab"]
             fields = step_config["get_fields"]()
 
             for field in fields:
@@ -382,7 +640,12 @@ def save_onboarding_settings(values: Dict[str, Any]) -> Dict[str, Any]:
                     continue
 
                 key = field.key
+                if key in _ONBOARDING_VIRTUAL_KEYS:
+                    continue
+                if not _is_field_visible(field, values):
+                    continue
                 if key in values:
+                    tab_name = _get_field_tab_name(field, step_config["tab"])
                     if tab_name not in tab_values:
                         tab_values[tab_name] = {}
                     tab_values[tab_name][key] = values[key]
@@ -391,10 +654,9 @@ def save_onboarding_settings(values: Dict[str, Any]) -> Dict[str, Any]:
         for tab_name, tab_data in tab_values.items():
             if tab_data:
                 save_config_file(tab_name, tab_data)
-                logger.info(f"Saved onboarding settings to {tab_name}: {list(tab_data.keys())}")
+                logger.info("Saved onboarding settings to %s: %s", tab_name, list(tab_data.keys()))
 
-        # Enable the selected metadata provider
-        search_mode = values.get("SEARCH_MODE", "direct")
+        search_mode = values.get("SEARCH_MODE", "universal")
         if search_mode == "universal":
             provider = values.get("METADATA_PROVIDER", "hardcover")
             if provider:
@@ -416,7 +678,52 @@ def save_onboarding_settings(values: Dict[str, Any]) -> Dict[str, Any]:
                     provider_config["GOOGLEBOOKS_API_KEY"] = values["GOOGLEBOOKS_API_KEY"]
 
                 save_config_file(provider, provider_config)
-                logger.info(f"Enabled metadata provider: {provider} with keys: {list(provider_config.keys())}")
+                logger.info(
+                    "Enabled metadata provider: %s with keys: %s",
+                    provider,
+                    list(provider_config.keys()),
+                )
+
+        selected_release_sources = values.get(ONBOARDING_RELEASE_SOURCES_KEY, [])
+        if not isinstance(selected_release_sources, list):
+            selected_release_sources = []
+
+        source_updates: dict[str, dict[str, Any]] = {}
+
+        if search_mode == "direct":
+            source_updates.setdefault("download_sources", {})["DIRECT_DOWNLOAD_ENABLED"] = True
+        else:
+            if _is_release_source_selected(values, "direct_download"):
+                source_updates.setdefault("download_sources", {})["DIRECT_DOWNLOAD_ENABLED"] = True
+            if _is_release_source_selected(values, "prowlarr"):
+                source_updates.setdefault("prowlarr_config", {})["PROWLARR_ENABLED"] = True
+            if _is_release_source_selected(values, "audiobookbay"):
+                source_updates.setdefault("audiobookbay_config", {})["ABB_ENABLED"] = True
+
+            if not values.get("DEFAULT_RELEASE_SOURCE"):
+                for source_name in selected_release_sources:
+                    if source_name in {"direct_download", "prowlarr", "irc"}:
+                        source_updates.setdefault("search_mode", {})["DEFAULT_RELEASE_SOURCE"] = (
+                            source_name
+                        )
+                        break
+
+            if not values.get("DEFAULT_RELEASE_SOURCE_AUDIOBOOK"):
+                for source_name in selected_release_sources:
+                    if source_name in {"prowlarr", "audiobookbay", "irc"}:
+                        source_updates.setdefault("search_mode", {})[
+                            "DEFAULT_RELEASE_SOURCE_AUDIOBOOK"
+                        ] = source_name
+                        break
+
+        for tab_name, tab_data in source_updates.items():
+            if tab_data:
+                save_config_file(tab_name, tab_data)
+                logger.info(
+                    "Enabled onboarding release source settings for %s: %s",
+                    tab_name,
+                    list(tab_data.keys()),
+                )
 
         # Mark onboarding as complete
         mark_onboarding_complete()
@@ -424,12 +731,13 @@ def save_onboarding_settings(values: Dict[str, Any]) -> Dict[str, Any]:
         # Refresh config
         try:
             from shelfmark.core.config import config
+
             config.refresh()
         except ImportError as e:
-            logger.debug(f"Could not refresh config after onboarding: {e}")
-
-        return {"success": True, "message": "Onboarding complete!"}
+            logger.debug("Could not refresh config after onboarding: %s", e)
 
     except Exception as e:
-        logger.error(f"Failed to save onboarding settings: {e}")
+        logger.exception("Failed to save onboarding settings")
         return {"success": False, "message": str(e)}
+    else:
+        return {"success": True, "message": "Onboarding complete!"}

shelfmark/core/path_mappings.py

@@ -10,12 +10,19 @@ A mapping rewrites a remote path prefix into a local path prefix.
 from __future__ import annotations
 
 from dataclasses import dataclass
-from pathlib import Path
-from typing import Any, Iterable, Optional
+from pathlib import Path, PureWindowsPath
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from collections.abc import Iterable
+
+_WINDOWS_DRIVE_PREFIX_LENGTH = 2
 
 
 @dataclass(frozen=True)
 class RemotePathMapping:
+    """Mapping from a remote path prefix to a local path prefix."""
+
     host: str
     remote_path: str
     local_path: str
@@ -36,14 +43,51 @@ def _normalize_prefix(path: str) -> str:
 
 def _is_windows_path(path: str) -> bool:
     """Check if a path looks like a Windows path (has a drive letter like C:/)."""
-    return len(path) >= 2 and path[1] == ":" and path[0].isalpha()
+    return len(path) >= _WINDOWS_DRIVE_PREFIX_LENGTH and path[1] == ":" and path[0].isalpha()
 
 
 def _normalize_host(host: str) -> str:
     return str(host or "").strip().lower()
 
 
-def parse_remote_path_mappings(value: Any) -> list[RemotePathMapping]:
+def _is_relative_to(path: Path, prefix: Path) -> bool:
+    try:
+        path.relative_to(prefix)
+    except ValueError:
+        return False
+
+    return True
+
+
+def _join_contained_path(local_prefix: str, remainder: str) -> Path | None:
+    local_path = Path(local_prefix)
+
+    if remainder:
+        remainder_path = Path(remainder)
+        windows_remainder_path = PureWindowsPath(remainder)
+
+        if (
+            remainder_path.is_absolute()
+            or windows_remainder_path.is_absolute()
+            or ".." in remainder_path.parts
+            or ".." in windows_remainder_path.parts
+        ):
+            return None
+
+        remapped = local_path / remainder_path
+    else:
+        remapped = local_path
+
+    resolved_local_path = local_path.resolve(strict=False)
+    resolved_remapped = remapped.resolve(strict=False)
+    if not _is_relative_to(resolved_remapped, resolved_local_path):
+        return None
+
+    return remapped
+
+
+def parse_remote_path_mappings(value: object) -> list[RemotePathMapping]:
+    """Parse configured remote-path mapping rows into normalized mappings."""
     if not value or not isinstance(value, list):
         return []
 
@@ -60,7 +104,9 @@ def parse_remote_path_mappings(value: Any) -> list[RemotePathMapping]:
         if not host or not remote_path or not local_path:
             continue
 
-        mappings.append(RemotePathMapping(host=host, remote_path=remote_path, local_path=local_path))
+        mappings.append(
+            RemotePathMapping(host=host, remote_path=remote_path, local_path=local_path)
+        )
 
     mappings.sort(key=lambda m: len(m.remote_path), reverse=True)
     return mappings
@@ -71,7 +117,12 @@ def remap_remote_to_local_with_match(
     mappings: Iterable[RemotePathMapping],
     host: str,
     remote_path: str | Path,
-) -> tuple[Path, bool]:
+) -> tuple[Path | None, bool]:
+    """Remap a remote path and report whether a configured mapping matched.
+
+    Returns ``(None, True)`` when a mapping prefix matched but the remainder was
+    unsafe to join under the local prefix.
+    """
     host_normalized = _normalize_host(host)
     remote_normalized = _normalize_prefix(str(remote_path))
 
@@ -96,39 +147,47 @@ def remap_remote_to_local_with_match(
             prefix_lower = remote_prefix.lower()
             matches = remote_lower == prefix_lower or remote_lower.startswith(prefix_lower + "/")
         else:
-            matches = remote_normalized == remote_prefix or remote_normalized.startswith(remote_prefix + "/")
+            matches = remote_normalized == remote_prefix or remote_normalized.startswith(
+                remote_prefix + "/"
+            )
 
         if matches:
             # Use the length of the original prefix to extract remainder
             # This preserves the original case in folder names
-            remainder = remote_normalized[len(remote_prefix):]
+            remainder = remote_normalized[len(remote_prefix) :]
             local_prefix = _normalize_prefix(mapping.local_path)
 
-            if remainder.startswith("/"):
-                remainder = remainder[1:]
+            remainder = remainder.removeprefix("/")
+
+            remapped = _join_contained_path(local_prefix, remainder)
+            if remapped is None:
+                return None, True
 
-            remapped = Path(local_prefix) / remainder if remainder else Path(local_prefix)
             return remapped, True
 
     return Path(remote_normalized), False
 
 
-def remap_remote_to_local(*, mappings: Iterable[RemotePathMapping], host: str, remote_path: str | Path) -> Path:
+def remap_remote_to_local(
+    *, mappings: Iterable[RemotePathMapping], host: str, remote_path: str | Path
+) -> Path:
+    """Remap a remote path to a local path using the configured mappings."""
     remapped, _ = remap_remote_to_local_with_match(
         mappings=mappings,
         host=host,
         remote_path=remote_path,
     )
+    if remapped is None:
+        return Path(str(remote_path))
     return remapped
 
 
-def get_client_host_identifier(client: Any) -> Optional[str]:
+def get_client_host_identifier(client: object) -> str | None:
     """Return a stable identifier used by the mapping UI.
 
     Sonarr uses the download client's configured host. Shelfmark currently uses
     the download client 'name' (e.g. qbittorrent, sabnzbd).
     """
-
     name = getattr(client, "name", None)
     if isinstance(name, str) and name.strip():
         return name.strip().lower()

shelfmark/core/prefix_middleware.py

@@ -2,19 +2,30 @@
 
 from __future__ import annotations
 
-from typing import Iterable, Optional
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from collections.abc import Callable, Iterable
 
 
 class PrefixMiddleware:
     """Strip a configured URL prefix from PATH_INFO before routing."""
 
-    def __init__(self, app, prefix: str, bypass_paths: Optional[Iterable[str]] = None) -> None:
+    def __init__(
+        self,
+        app: Callable[[dict[str, object], Callable[..., object]], object],
+        prefix: str,
+        bypass_paths: Iterable[str] | None = None,
+    ) -> None:
+        """Initialize the middleware with a prefix and optional bypass paths."""
         self.app = app
         self.prefix = prefix.rstrip("/")
         self.bypass_paths = set(bypass_paths or [])
 
-    def __call__(self, environ, start_response):
-        path = environ.get("PATH_INFO", "") or ""
+    def __call__(self, environ: dict[str, object], start_response: Callable[..., object]) -> object:
+        """Rewrite prefixed requests before handing them to the wrapped app."""
+        raw_path = environ.get("PATH_INFO", "")
+        path = raw_path if isinstance(raw_path, str) else str(raw_path or "")
 
         if path in self.bypass_paths:
             return self.app(environ, start_response)
@@ -24,7 +35,7 @@ class PrefixMiddleware:
 
         if path == self.prefix or path.startswith(self.prefix + "/"):
             environ["SCRIPT_NAME"] = self.prefix
-            environ["PATH_INFO"] = path[len(self.prefix):] or "/"
+            environ["PATH_INFO"] = path[len(self.prefix) :] or "/"
             return self.app(environ, start_response)
 
         start_response("404 Not Found", [("Content-Type", "text/plain")])

shelfmark/core/queue.py

@@ -2,19 +2,46 @@
 
 import queue
 import time
-from datetime import datetime, timedelta
+from datetime import UTC, datetime, timedelta
 from pathlib import Path
-from threading import Lock, Event
-from typing import Dict, List, Optional, Tuple, Any, Callable
+from threading import Event, Lock
+from typing import TYPE_CHECKING, Any
 
 from shelfmark.core.config import config as app_config
-from shelfmark.core.models import QueueStatus, QueueItem, DownloadTask
+from shelfmark.core.logger import setup_logger
+from shelfmark.core.models import (
+    TERMINAL_QUEUE_STATUSES,
+    DownloadTask,
+    QueueItem,
+    QueueStatus,
+)
+
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
+logger = setup_logger(__name__)
+_QUEUE_HOOK_ERRORS = (OSError, RuntimeError, TypeError, ValueError)
+
+
+def _coerce_status_timeout_seconds(value: object, *, default: int) -> int:
+    """Normalize STATUS_TIMEOUT into a usable positive integer."""
+    if isinstance(value, bool):
+        return default
+    if isinstance(value, int):
+        return value if value > 0 else default
+    if isinstance(value, str):
+        stripped = value.strip()
+        if stripped.isdigit():
+            parsed = int(stripped)
+            return parsed if parsed > 0 else default
+    return default
 
 
 class BookQueue:
     """Thread-safe download queue manager with priority support and cancellation."""
 
     def __init__(self) -> None:
+        """Initialize queue state, locks, and lifecycle hooks."""
         self._queue: queue.PriorityQueue[QueueItem] = queue.PriorityQueue()
         self._lock = Lock()
         self._status: dict[str, QueueStatus] = {}
@@ -22,22 +49,30 @@ class BookQueue:
         self._status_timestamps: dict[str, datetime] = {}  # Track when each status was last updated
         self._cancel_flags: dict[str, Event] = {}  # Cancellation flags for active downloads
         self._active_downloads: dict[str, bool] = {}  # Track currently downloading tasks
-        self._terminal_status_hook: Optional[
-            Callable[[str, QueueStatus, DownloadTask], None]
-        ] = None
+        self._terminal_status_hook: Callable[[str, QueueStatus, DownloadTask], None] | None = None
+        self._queue_hook: Callable[[str, DownloadTask], None] | None = None
 
     @property
     def _status_timeout(self) -> timedelta:
         """Get status timeout from config (allows live updates)."""
-        return timedelta(seconds=app_config.get("STATUS_TIMEOUT", 3600))
+        return timedelta(
+            seconds=_coerce_status_timeout_seconds(
+                app_config.get("STATUS_TIMEOUT", 3600),
+                default=3600,
+            )
+        )
 
     def add(self, task: DownloadTask) -> bool:
         """Add a download task to the queue. Returns False if already exists."""
+        hook: Callable[[str, DownloadTask], None] | None = None
         with self._lock:
             task_id = task.task_id
 
-            # Don't add if already exists and not in error/done state
-            if task_id in self._status and self._status[task_id] not in [QueueStatus.ERROR, QueueStatus.DONE, QueueStatus.CANCELLED]:
+            # Don't add if already exists and not in error/cancelled state
+            if task_id in self._status and self._status[task_id] not in [
+                QueueStatus.ERROR,
+                QueueStatus.CANCELLED,
+            ]:
                 return False
 
             # Ensure added_time is set
@@ -48,9 +83,16 @@ class BookQueue:
             self._queue.put(queue_item)
             self._task_data[task_id] = task
             self._update_status(task_id, QueueStatus.QUEUED)
-            return True
+            hook = self._queue_hook
 
-    def get_next(self) -> Optional[Tuple[str, Event]]:
+        if hook is not None:
+            try:
+                hook(task_id, task)
+            except _QUEUE_HOOK_ERRORS as exc:
+                logger.warning("Queue hook failed while adding task %s: %s", task_id, exc)
+        return True
+
+    def get_next(self) -> tuple[str, Event] | None:
         """Get next task ID from queue with cancellation flag."""
         # Use iterative approach to avoid stack overflow if many items are cancelled
         while True:
@@ -67,58 +109,72 @@ class BookQueue:
                     cancel_flag = Event()
                     self._cancel_flags[task_id] = cancel_flag
                     self._active_downloads[task_id] = True
-
-                return task_id, cancel_flag
             except queue.Empty:
                 return None
+            else:
+                return task_id, cancel_flag
 
-    def get_task(self, task_id: str) -> Optional[DownloadTask]:
+    def get_task(self, task_id: str) -> DownloadTask | None:
         """Get a task by its ID."""
         with self._lock:
             return self._task_data.get(task_id)
 
+    def get_task_status(self, task_id: str) -> QueueStatus | None:
+        """Get queue status for a task id."""
+        with self._lock:
+            return self._status.get(task_id)
+
     def _update_status(self, book_id: str, status: QueueStatus) -> None:
-        """Internal method to update status and timestamp."""
+        """Update the status and timestamp for a task."""
         self._status[book_id] = status
-        self._status_timestamps[book_id] = datetime.now()
+        self._status_timestamps[book_id] = datetime.now(UTC)
 
     def set_terminal_status_hook(
         self,
-        hook: Optional[Callable[[str, QueueStatus, DownloadTask], None]],
+        hook: Callable[[str, QueueStatus, DownloadTask], None] | None,
     ) -> None:
         """Register a callback invoked when a task first enters a terminal status."""
         with self._lock:
             self._terminal_status_hook = hook
 
+    def set_queue_hook(
+        self,
+        hook: Callable[[str, DownloadTask], None] | None,
+    ) -> None:
+        """Register a callback invoked when a task is added to the queue."""
+        with self._lock:
+            self._queue_hook = hook
+
+    def _apply_status_locked(
+        self, book_id: str, status: QueueStatus
+    ) -> tuple[Callable[[str, QueueStatus, DownloadTask], None] | None, DownloadTask | None]:
+        """Apply a status change; returns the terminal hook to run after releasing the lock."""
+        hook: Callable[[str, QueueStatus, DownloadTask], None] | None = None
+        hook_task: DownloadTask | None = None
+        previous_status = self._status.get(book_id)
+        self._update_status(book_id, status)
+
+        if (
+            status in TERMINAL_QUEUE_STATUSES
+            and previous_status != status
+            and self._terminal_status_hook is not None
+        ):
+            current_task = self._task_data.get(book_id)
+            if current_task is not None:
+                hook = self._terminal_status_hook
+                hook_task = current_task
+
+        # Clean up active download tracking when finished
+        if status in TERMINAL_QUEUE_STATUSES:
+            self._active_downloads.pop(book_id, None)
+            self._cancel_flags.pop(book_id, None)
+
+        return hook, hook_task
+
     def update_status(self, book_id: str, status: QueueStatus) -> None:
         """Update status of a book in the queue."""
-        hook: Optional[Callable[[str, QueueStatus, DownloadTask], None]] = None
-        hook_task: Optional[DownloadTask] = None
         with self._lock:
-            previous_status = self._status.get(book_id)
-            self._update_status(book_id, status)
-
-            terminal_statuses = {
-                QueueStatus.COMPLETE,
-                QueueStatus.AVAILABLE,
-                QueueStatus.ERROR,
-                QueueStatus.DONE,
-                QueueStatus.CANCELLED,
-            }
-            if (
-                status in terminal_statuses
-                and previous_status != status
-                and self._terminal_status_hook is not None
-            ):
-                current_task = self._task_data.get(book_id)
-                if current_task is not None:
-                    hook = self._terminal_status_hook
-                    hook_task = current_task
-
-            # Clean up active download tracking when finished
-            if status in [QueueStatus.COMPLETE, QueueStatus.AVAILABLE, QueueStatus.ERROR, QueueStatus.DONE, QueueStatus.CANCELLED]:
-                self._active_downloads.pop(book_id, None)
-                self._cancel_flags.pop(book_id, None)
+            hook, hook_task = self._apply_status_locked(book_id, status)
 
         if hook is not None and hook_task is not None:
             hook(book_id, status, hook_task)
@@ -141,25 +197,28 @@ class BookQueue:
             if task_id in self._task_data:
                 self._task_data[task_id].status_message = message
 
-    def get_status(self, user_id: Optional[int] = None) -> Dict[QueueStatus, Dict[str, DownloadTask]]:
+    def get_status(self, user_id: int | None = None) -> dict[QueueStatus, dict[str, DownloadTask]]:
         """Get current queue status grouped by status.
 
         Args:
-            user_id: If provided, only return tasks belonging to this user
-                     (plus legacy tasks with no user_id). If None, return all.
+            user_id: If provided, only return tasks belonging to this user.
+                     If None, return all.
+
         """
         self.refresh()
         with self._lock:
-            result: Dict[QueueStatus, Dict[str, DownloadTask]] = {status: {} for status in QueueStatus}
+            result: dict[QueueStatus, dict[str, DownloadTask]] = {
+                status: {} for status in QueueStatus
+            }
             for task_id, status in self._status.items():
                 if task_id in self._task_data:
                     task = self._task_data[task_id]
-                    if user_id is not None and task.user_id is not None and task.user_id != user_id:
+                    if user_id is not None and task.user_id != user_id:
                         continue
                     result[status][task_id] = task
             return result
 
-    def get_queue_order(self) -> List[Dict[str, Any]]:
+    def get_queue_order(self) -> list[dict[str, Any]]:
         """Get current queue order for display."""
         with self._lock:
             queue_items = []
@@ -167,53 +226,51 @@ class BookQueue:
             # Get items from priority queue without removing them
             temp_items = []
             while not self._queue.empty():
-                try:
-                    item = self._queue.get_nowait()
-                    temp_items.append(item)
-                    task_id = item.book_id  # QueueItem uses book_id as the ID field
-                    if task_id in self._task_data:
-                        task = self._task_data[task_id]
-                        queue_items.append({
-                            'id': task_id,
-                            'title': task.title,
-                            'author': task.author,
-                            'priority': item.priority,
-                            'added_time': item.added_time,
-                            'status': self._status.get(task_id, QueueStatus.QUEUED)
-                        })
-                except queue.Empty:
-                    break
+                item = self._queue.get_nowait()
+                temp_items.append(item)
+                task_id = item.book_id  # QueueItem uses book_id as the ID field
+                if task_id in self._task_data:
+                    task = self._task_data[task_id]
+                    queue_items.append(
+                        {
+                            "id": task_id,
+                            "title": task.title,
+                            "author": task.author,
+                            "priority": item.priority,
+                            "added_time": item.added_time,
+                            "status": self._status.get(task_id, QueueStatus.QUEUED),
+                        }
+                    )
 
             # Put items back in queue
             for item in temp_items:
                 self._queue.put(item)
 
-            return sorted(queue_items, key=lambda x: (x['priority'], x['added_time']))
+            return sorted(queue_items, key=lambda x: (x["priority"], x["added_time"]))
 
     def cancel_download(self, task_id: str) -> bool:
-        """Cancel a download or clear a completed/errored item."""
+        """Cancel an active or queued download."""
         with self._lock:
             current_status = self._status.get(task_id)
 
-            # Allow cancellation during any active state
-            if current_status in [QueueStatus.RESOLVING, QueueStatus.LOCATING, QueueStatus.DOWNLOADING]:
+            if current_status in [
+                QueueStatus.RESOLVING,
+                QueueStatus.LOCATING,
+                QueueStatus.DOWNLOADING,
+            ]:
                 # Signal active download to stop
                 if task_id in self._cancel_flags:
                     self._cancel_flags[task_id].set()
-            if current_status in [QueueStatus.COMPLETE, QueueStatus.DONE, QueueStatus.AVAILABLE, QueueStatus.ERROR, QueueStatus.CANCELLED]:
-                # Clear completed/errored/cancelled items from tracking
-                self._status.pop(task_id, None)
-                self._status_timestamps.pop(task_id, None)
-                self._task_data.pop(task_id, None)
-                self._cancel_flags.pop(task_id, None)
-                self._active_downloads.pop(task_id, None)
-                return True
+            elif current_status != QueueStatus.QUEUED:
+                # Not in a cancellable state
+                return False
 
-        if current_status in [QueueStatus.RESOLVING, QueueStatus.LOCATING, QueueStatus.DOWNLOADING, QueueStatus.QUEUED]:
-            self.update_status(task_id, QueueStatus.CANCELLED)
-            return True
+            # Write under the same lock so a download that finishes first is not overwritten
+            hook, hook_task = self._apply_status_locked(task_id, QueueStatus.CANCELLED)
 
-        return False
+        if hook is not None and hook_task is not None:
+            hook(task_id, QueueStatus.CANCELLED, hook_task)
+        return True
 
     def set_priority(self, task_id: str, new_priority: int) -> bool:
         """Change the priority of a queued task (lower = higher priority)."""
@@ -226,20 +283,17 @@ class BookQueue:
             found = False
 
             while not self._queue.empty():
-                try:
-                    item = self._queue.get_nowait()
-                    if item.book_id == task_id:  # QueueItem uses book_id as the ID field
-                        # Create new item with updated priority
-                        new_item = QueueItem(task_id, new_priority, item.added_time)
-                        temp_items.append(new_item)
-                        found = True
-                        # Update task data priority
-                        if task_id in self._task_data:
-                            self._task_data[task_id].priority = new_priority
-                    else:
-                        temp_items.append(item)
-                except queue.Empty:
-                    break
+                item = self._queue.get_nowait()
+                if item.book_id == task_id:  # QueueItem uses book_id as the ID field
+                    # Create new item with updated priority
+                    new_item = QueueItem(task_id, new_priority, item.added_time)
+                    temp_items.append(new_item)
+                    found = True
+                    # Update task data priority
+                    if task_id in self._task_data:
+                        self._task_data[task_id].priority = new_priority
+                else:
+                    temp_items.append(item)
 
             # Put all items back
             for item in temp_items:
@@ -247,25 +301,64 @@ class BookQueue:
 
             return found
 
-    def reorder_queue(self, task_priorities: Dict[str, int]) -> bool:
+    def enqueue_existing(self, task_id: str, *, priority: int | None = None) -> bool:
+        """Requeue an existing task regardless of current status.
+
+        This is used for retries where task metadata should be preserved.
+        """
+        hook: Callable[[str, DownloadTask], None] | None = None
+        hook_task: DownloadTask | None = None
+        with self._lock:
+            task = self._task_data.get(task_id)
+            if task is None:
+                return False
+
+            if priority is not None:
+                task.priority = priority
+
+            # Ensure task doesn't appear active while waiting for retry.
+            self._active_downloads.pop(task_id, None)
+            self._cancel_flags.pop(task_id, None)
+
+            # De-duplicate queue entries for this task id.
+            temp_items: list[QueueItem] = []
+            while not self._queue.empty():
+                item = self._queue.get_nowait()
+                if item.book_id != task_id:
+                    temp_items.append(item)
+
+            for item in temp_items:
+                self._queue.put(item)
+
+            queue_item = QueueItem(task_id, task.priority, time.time())
+            self._queue.put(queue_item)
+            self._update_status(task_id, QueueStatus.QUEUED)
+            hook = self._queue_hook
+            hook_task = task
+
+        if hook is not None and hook_task is not None:
+            try:
+                hook(task_id, hook_task)
+            except _QUEUE_HOOK_ERRORS as exc:
+                logger.warning("Queue hook failed while requeueing task %s: %s", task_id, exc)
+        return True
+
+    def reorder_queue(self, task_priorities: dict[str, int]) -> bool:
         """Bulk reorder queue by mapping task_id to new priority."""
         with self._lock:
             # Extract all items from queue
             all_items = []
             while not self._queue.empty():
-                try:
-                    item = self._queue.get_nowait()
-                    task_id = item.book_id  # QueueItem uses book_id as the ID field
-                    # Update priority if specified
-                    if task_id in task_priorities:
-                        new_priority = task_priorities[task_id]
-                        item = QueueItem(task_id, new_priority, item.added_time)
-                        # Update task data priority
-                        if task_id in self._task_data:
-                            self._task_data[task_id].priority = new_priority
-                    all_items.append(item)
-                except queue.Empty:
-                    break
+                item = self._queue.get_nowait()
+                task_id = item.book_id  # QueueItem uses book_id as the ID field
+                # Update priority if specified
+                if task_id in task_priorities:
+                    new_priority = task_priorities[task_id]
+                    item = QueueItem(task_id, new_priority, item.added_time)
+                    # Update task data priority
+                    if task_id in self._task_data:
+                        self._task_data[task_id].priority = new_priority
+                all_items.append(item)
 
             # Put all items back with updated priorities
             for item in all_items:
@@ -273,7 +366,7 @@ class BookQueue:
 
             return True
 
-    def get_active_downloads(self) -> List[str]:
+    def get_active_downloads(self) -> list[str]:
         """Get list of currently active download task IDs."""
         with self._lock:
             return list(self._active_downloads.keys())
@@ -285,45 +378,11 @@ class BookQueue:
                 return True
             return any(status == QueueStatus.QUEUED for status in self._status.values())
 
-    def clear_completed(self, user_id: Optional[int] = None) -> int:
-        """Remove terminal tasks from tracking, optionally scoped to one user.
-
-        Args:
-            user_id: If provided, only clear tasks belonging to this user,
-                     plus legacy tasks with no user_id. If None, clear all.
-        """
-        terminal_statuses = {QueueStatus.COMPLETE, QueueStatus.DONE, QueueStatus.AVAILABLE, QueueStatus.ERROR, QueueStatus.CANCELLED}
-        with self._lock:
-            to_remove: list[str] = []
-            for task_id, status in self._status.items():
-                if status not in terminal_statuses:
-                    continue
-
-                if user_id is None:
-                    to_remove.append(task_id)
-                    continue
-
-                task = self._task_data.get(task_id)
-                if task is None:
-                    # Without task ownership metadata we cannot safely scope removal.
-                    continue
-                if task.user_id is None or task.user_id == user_id:
-                    to_remove.append(task_id)
-
-            for task_id in to_remove:
-                self._status.pop(task_id, None)
-                self._status_timestamps.pop(task_id, None)
-                self._task_data.pop(task_id, None)
-                self._cancel_flags.pop(task_id, None)
-                self._active_downloads.pop(task_id, None)
-
-            return len(to_remove)
-
     def refresh(self) -> None:
         """Remove any tasks that are done downloading or have stale status."""
-        terminal_statuses = {QueueStatus.COMPLETE, QueueStatus.DONE, QueueStatus.ERROR, QueueStatus.AVAILABLE, QueueStatus.CANCELLED}
+        terminal_statuses = TERMINAL_QUEUE_STATUSES
         with self._lock:
-            current_time = datetime.now()
+            current_time = datetime.now(UTC)
             to_remove = []
 
             for task_id, status in self._status.items():
@@ -335,15 +394,14 @@ class BookQueue:
                 if task.download_path and not Path(task.download_path).exists():
                     task.download_path = None
 
-                # Mark available downloads as done if file is gone
-                if status == QueueStatus.AVAILABLE and not task.download_path:
-                    self._update_status(task_id, QueueStatus.DONE)
-
                 # Check for stale status entries
                 last_update = self._status_timestamps.get(task_id)
-                if last_update and (current_time - last_update) > self._status_timeout:
-                    if status in terminal_statuses:
-                        to_remove.append(task_id)
+                if (
+                    last_update
+                    and (current_time - last_update) > self._status_timeout
+                    and status in terminal_statuses
+                ):
+                    to_remove.append(task_id)
 
             # Remove stale entries
             for task_id in to_remove:
@@ -351,5 +409,6 @@ class BookQueue:
                 self._status_timestamps.pop(task_id, None)
                 self._task_data.pop(task_id, None)
 
+
 # Global instance of BookQueue
 book_queue = BookQueue()

shelfmark/core/release_inspect_routes.py

@@ -0,0 +1,102 @@
+"""Pre-download release inspection: list a release's files and plan a multi-book split."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+from flask import jsonify, request
+
+from shelfmark.core.logger import setup_logger
+from shelfmark.core.utils import is_audiobook
+from shelfmark.download.postprocess.packs import PackFile, PackPlan, plan_pack
+from shelfmark.download.postprocess.policy import (
+    get_supported_audiobook_formats,
+    get_supported_formats,
+)
+from shelfmark.release_sources import get_handler
+
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
+    from flask import Flask, Response
+
+logger = setup_logger(__name__)
+
+_INSPECT_ERRORS = (OSError, RuntimeError, ValueError, TypeError, KeyError, AttributeError)
+NOT_INSPECTABLE_REASON = "This source cannot list the release's files before downloading"
+
+
+def _serialize_plan(plan: PackPlan) -> dict[str, Any]:
+    return {
+        "is_pack": plan.is_pack,
+        "ignored": plan.ignored,
+        "books": [
+            {
+                "title": book.title,
+                "series_position": book.series_position,
+                "year": book.year,
+                "files": book.files,
+            }
+            for book in plan.books
+        ],
+    }
+
+
+def inspect_release(data: dict[str, Any]) -> dict[str, Any]:
+    """Build the inspect response for a release payload (same shape as a download)."""
+    source = str(data["source"])
+    handler = get_handler(source)
+    try:
+        files: list[PackFile] | None = handler.list_files(data)
+    except _INSPECT_ERRORS as exc:
+        logger.warning(
+            "Could not list files for %s release %s: %s", source, data.get("source_id"), exc
+        )
+        return {"inspected": False, "reason": str(exc), "files": [], "plan": None}
+
+    if files is None:
+        return {"inspected": False, "reason": NOT_INSPECTABLE_REASON, "files": [], "plan": None}
+
+    content_type = data.get("content_type")
+    supported = (
+        get_supported_audiobook_formats()
+        if is_audiobook(content_type if isinstance(content_type, str) else None)
+        else get_supported_formats()
+    )
+    series_name = data.get("series_name")
+    author_name = data.get("author")
+    plan = plan_pack(
+        files,
+        supported_extensions=set(supported),
+        series_name=series_name if isinstance(series_name, str) else None,
+        author_name=author_name if isinstance(author_name, str) else None,
+    )
+    return {
+        "inspected": True,
+        "reason": None,
+        "files": [{"path": f.path, "size": f.size} for f in files],
+        "plan": _serialize_plan(plan),
+    }
+
+
+def register_release_inspect_routes(
+    app: Flask,
+    login_required: Callable[..., Any],
+) -> None:
+    """Register POST /api/releases/inspect."""
+
+    @app.route("/api/releases/inspect", methods=["POST"])
+    @login_required
+    def api_inspect_release() -> Response | tuple[Response, int]:
+        data = request.get_json(silent=True)
+        if not isinstance(data, dict):
+            return jsonify({"error": "No data provided"}), 400
+        if not data.get("source_id"):
+            return jsonify({"error": "source_id is required"}), 400
+        if not data.get("source"):
+            return jsonify({"error": "source is required"}), 400
+        try:
+            get_handler(str(data["source"]))
+        except ValueError as exc:
+            return jsonify({"error": str(exc)}), 400
+        return jsonify(inspect_release(data))

shelfmark/core/release_search.py

@@ -0,0 +1,93 @@
+"""Shared release-search helpers.
+
+Extracted from the ``/api/releases`` route so the same per-source search logic can
+be reused outside the HTTP route (for example by background automation) without
+going through Flask. Behaviour for the HTTP route is preserved: the route delegates
+its inner per-source search to :func:`search_source_releases`.
+"""
+
+from __future__ import annotations
+
+import sqlite3
+from typing import TYPE_CHECKING
+
+from shelfmark.core.logger import setup_logger
+from shelfmark.core.search_plan import build_release_search_plan
+
+if TYPE_CHECKING:
+    from shelfmark.core.models import SearchFilters
+    from shelfmark.metadata_providers import BookMetadata
+    from shelfmark.release_sources import Release, ReleaseSource
+
+logger = setup_logger(__name__)
+
+# Mirror of main._OPERATIONAL_ERRORS so a misbehaving source can't crash a caller.
+_OPERATIONAL_ERRORS = (OSError, RuntimeError, TypeError, ValueError, sqlite3.Error)
+
+
+def search_source_releases(
+    source_name: str,
+    search_book: BookMetadata,
+    *,
+    languages: list[str] | None = None,
+    manual_query: str | None = None,
+    indexers: list[str] | None = None,
+    expand_search: bool = False,
+    content_type: str = "ebook",
+    source_filters: SearchFilters | None = None,
+    user_id: int | None = None,
+) -> tuple[ReleaseSource | None, list[Release], str | None]:
+    """Search a single release source, returning any error instead of raising.
+
+    Returns ``(source, releases, error_message)``. On failure ``source`` is ``None``
+    and ``error_message`` describes the problem. ``user_id`` lets the search plan
+    pick up that user's default languages when no explicit filter is given.
+    """
+    from shelfmark.release_sources import SourceUnavailableError, get_source
+
+    try:
+        source = get_source(source_name)
+
+        plan = build_release_search_plan(
+            search_book,
+            languages=languages,
+            manual_query=manual_query,
+            indexers=indexers,
+            source_filters=source_filters,
+            user_id=user_id,
+        )
+
+        if plan.source_filters is not None:
+            planned_query = plan.manual_query or plan.primary_query
+            planned_query_type = "query"
+        elif plan.manual_query:
+            planned_query = plan.manual_query
+            planned_query_type = "manual"
+        elif not expand_search and plan.isbn_candidates:
+            planned_query = plan.isbn_candidates[0]
+            planned_query_type = "isbn"
+        else:
+            planned_query = plan.primary_query
+            planned_query_type = "title_author"
+
+        logger.debug(
+            "Searching %s: %s='%s' (title='%s', authors=%s, expand=%s, content_type=%s)",
+            source_name,
+            planned_query_type,
+            planned_query,
+            search_book.title,
+            search_book.authors,
+            expand_search,
+            content_type,
+        )
+
+        releases = source.search(
+            search_book, plan, expand_search=expand_search, content_type=content_type
+        )
+    except ValueError:
+        return None, [], f"Unknown source: {source_name}"
+    except (SourceUnavailableError, *_OPERATIONAL_ERRORS) as exc:
+        logger.warning("Release search failed for source %s: %s", source_name, exc)
+        return None, [], f"{source_name}: {exc!s}"
+    else:
+        return source, releases, None

shelfmark/core/request_helpers.py

@@ -0,0 +1,184 @@
+"""Shared request-related helper functions used by routes and services."""
+
+from __future__ import annotations
+
+from datetime import UTC, datetime
+from typing import Any, Protocol, SupportsIndex, SupportsInt, TypeGuard
+
+from shelfmark.core.config import config as app_config
+from shelfmark.core.logger import setup_logger
+
+_logger = setup_logger(__name__)
+
+type _ConvertibleToInt = str | bytes | bytearray | SupportsInt | SupportsIndex
+
+
+class _MappingWithGet(Protocol):
+    """Minimal mapping protocol for session-like objects."""
+
+    def get(self, key: str, default: object = None, /) -> object: ...
+
+
+class _UserDBLike(Protocol):
+    """Minimal user DB protocol for username population helpers."""
+
+    def get_user(self, *, user_id: int) -> dict[str, Any] | None: ...
+
+
+def _is_mapping_with_get(candidate: object) -> TypeGuard[_MappingWithGet]:
+    """Return True when *candidate* exposes a mapping-style get method."""
+    return callable(getattr(candidate, "get", None))
+
+
+def _is_user_db_like(candidate: object) -> TypeGuard[_UserDBLike]:
+    """Return True when *candidate* exposes the user lookup API we need."""
+    return callable(getattr(candidate, "get_user", None))
+
+
+def _is_convertible_to_int(value: object) -> TypeGuard[_ConvertibleToInt]:
+    """Return True when *value* can be passed to ``int`` safely."""
+    return (
+        isinstance(value, (str, bytes, bytearray))
+        or hasattr(value, "__int__")
+        or hasattr(value, "__index__")
+    )
+
+
+def now_utc_iso() -> str:
+    """Return the current UTC time as a seconds-precision ISO 8601 string."""
+    return datetime.now(UTC).isoformat(timespec="seconds")
+
+
+def emit_ws_event(
+    ws_manager: object,
+    *,
+    event_name: str,
+    payload: dict[str, Any],
+    room: str,
+) -> None:
+    """Emit a WebSocket event via the shared manager, swallowing failures."""
+    if ws_manager is None:
+        return
+    try:
+        socketio = getattr(ws_manager, "socketio", None)
+        is_enabled = getattr(ws_manager, "is_enabled", None)
+        if socketio is None or not callable(is_enabled) or not is_enabled():
+            return
+        socketio.emit(event_name, payload, to=room)
+    except (AttributeError, RuntimeError, TypeError, ValueError) as exc:
+        _logger.warning(
+            "Failed to emit WebSocket event '%s' to room '%s': %s",
+            event_name,
+            room,
+            exc,
+        )
+
+
+def load_users_request_policy_settings() -> dict[str, Any]:
+    """Load global request-policy settings from the users config file."""
+    from shelfmark.core.request_policy import REQUEST_POLICY_KEYS
+
+    return {key: app_config.get(key) for key in REQUEST_POLICY_KEYS}
+
+
+def coerce_bool(value: object, *, default: bool = False) -> bool:
+    """Coerce arbitrary values into booleans with string-friendly semantics."""
+    if isinstance(value, bool):
+        return value
+    if value is None:
+        return default
+    if isinstance(value, str):
+        normalized = value.strip().lower()
+        if normalized in {"1", "true", "yes", "on"}:
+            return True
+        if normalized in {"0", "false", "no", "off", ""}:
+            return False
+    return bool(value)
+
+
+def get_session_db_user_id(session_obj: object) -> int | None:
+    """Extract and coerce `db_user_id` from a Flask session to ``int | None``."""
+    raw = session_obj.get("db_user_id") if _is_mapping_with_get(session_obj) else None
+    try:
+        return int(raw) if raw is not None and _is_convertible_to_int(raw) else None
+    except TypeError, ValueError:
+        return None
+
+
+def coerce_int(value: object, default: int) -> int:
+    """Best-effort integer coercion with fallback to default."""
+    if not _is_convertible_to_int(value):
+        return default
+    try:
+        return int(value)
+    except TypeError, ValueError:
+        return default
+
+
+def normalize_optional_text(value: object) -> str | None:
+    """Return a trimmed string or None for empty/non-string input."""
+    if not isinstance(value, str):
+        return None
+    normalized = value.strip()
+    return normalized or None
+
+
+def normalize_positive_int(value: object) -> int | None:
+    """Parse *value* as a positive integer, returning ``None`` on failure."""
+    if not _is_convertible_to_int(value):
+        return None
+    try:
+        parsed = int(value)
+    except TypeError, ValueError:
+        return None
+    return parsed if parsed > 0 else None
+
+
+def normalize_optional_positive_int(value: object, field_name: str = "value") -> int | None:
+    """Parse *value* as a positive integer or ``None``.
+
+    Raises ``ValueError`` when *value* is present but not a valid
+    positive integer.
+    """
+    if value is None:
+        return None
+    if not _is_convertible_to_int(value):
+        msg = f"{field_name} must be a positive integer when provided"
+        raise ValueError(msg)
+    try:
+        parsed = int(value)
+    except (TypeError, ValueError) as exc:
+        msg = f"{field_name} must be a positive integer when provided"
+        raise ValueError(msg) from exc
+    if parsed < 1:
+        msg = f"{field_name} must be a positive integer when provided"
+        raise ValueError(msg)
+    return parsed
+
+
+def populate_request_usernames(rows: list[dict[str, Any]], user_db: object) -> None:
+    """Add 'username' to each request row by looking up user_id."""
+    if not _is_user_db_like(user_db):
+        return
+
+    cache: dict[int, str] = {}
+    for row in rows:
+        requester_id = normalize_positive_int(row.get("user_id"))
+        if requester_id is None:
+            row["username"] = ""
+            continue
+        if requester_id not in cache:
+            requester = user_db.get_user(user_id=requester_id)
+            cache[requester_id] = requester.get("username", "") if requester else ""
+        row["username"] = cache[requester_id]
+
+
+def extract_release_source_id(release_data: object) -> str | None:
+    """Extract and normalize release_data.source_id."""
+    if not isinstance(release_data, dict):
+        return None
+    source_id = release_data.get("source_id")
+    if not isinstance(source_id, str):
+        return None
+    normalized = source_id.strip()
+    return normalized or None