feat: optional Hardcover wishlist sync with automated downloading (1 of 5) (#1516)

This replaces #1047, which @InfiniteAvenger opened and calibrain offered
to re-review. It is Caleb's feature commit with his authorship kept,
rebased onto current main, with small fixes, tests and a docs page on
top. Everything is opt-in and off by default, so nothing changes for
anyone who doesn't turn it on.

What it adds is a **Settings, Hardcover Sync** tab with three parts that
can be switched on separately:

- **Shelf sync**: a Hardcover shelf (Want to Read by default) is pulled
into requests on a schedule, or with a Sync now button. Requests carry
the Hardcover metadata and cover.
- **Automatic downloads**: synced requests are approved and a release is
picked from a source priority list. A strict title, author and format
match (plus a seeder minimum for torrents) decides, so anything without
a confident match stays pending for manual review.
- **Skip books you already own**: sync and automatic downloads ask
whichever libraries are enabled under Library Check (Calibre today) and
skip a book that is already there.

On top of Caleb's commit I made the request owner a lookup instead of
assuming user id 1, sent synced cover URLs through the proxy when the
request is created, got ruff and basedpyright clean, and added tests and
`docs/hardcover-sync.md`. His commit also had its own Audiobookshelf
settings and a Test library connection button, written before the
library check was split into providers. The button would have raised a
TypeError against the current `test_connection`, so those settings are
gone from this tab and an Audiobookshelf provider comes as a separate
PR, which is the part of #1047 that is missing here (for now
anyways...).

I've run this daily for about a couple of weeks. Ruff, basedpyright and
the backend suite pass (4426 passed).

This is 1 of 5. As calibrain suggested, the rest are stacked on this
one, each saying which number it is.

Closes
https://github.com/calibrain/shelfmark/pull/1047#issuecomment-6302071588

---------

Co-authored-by: InfiniteAvenger <calebewest02@gmail.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
This commit is contained in:
authored and GitHub committed 2026-10-22 20:27:47 -04:00
1 parent 72b5d04939
commit 48ae3dbef4
14 files changed
+2039

No files matched your search

+79
View File
@@ -0,0 +1,79 @@
# Hardcover Sync
Configure in **Settings → Hardcover Sync**.
Hardcover Sync pulls books from your [Hardcover](https://hardcover.app) reading shelves into Shelfmark as requests, optionally downloads them without an admin picking a release, and can skip books you already own in a library you have set up under [Library Check](library-check.md). Every part is opt-in and off by default; the tab does nothing until you enable it.
## Capabilities
### Shelf sync
On a schedule, Shelfmark reads the selected Hardcover shelf (by default **Want to Read**) through the Hardcover metadata provider and creates a **pending** request for each book that is not already known. Requests carry Hardcover's metadata, including the cover, and are owned by the lowest-id admin account.
A book is skipped when:
- a request with the same Hardcover book id already exists, in any status
- a download with the same title and author is already in the download history
- the library check (below) says it is already in your library
- a pending request with the same title, author, and content type already exists
### Automatic downloads
When enabled, each pending request that came from Hardcover is searched across your release sources in the configured priority order. The first source that yields a **strict match** wins, and the best candidate from that source is queued through the normal request fulfilment path, so it appears in the activity view and download history exactly like a request an admin approved.
A release is a strict match only when all of the following hold:
- at least 85% of the significant title words appear in the release title
- the author's surname appears in the release title, indexer name, or author field
- the release is an audiobook: an audiobook format (`m4b`, `mp3`, or your configured audiobook formats), or "audiobook"/"unabridged" in the title. Ebook-only formats such as `epub` and `pdf` are rejected
- for torrents, seeders meet **Minimum seeders**; other protocols ignore this check
Among matching releases from the winning source, `m4b` outranks `mp3`, then more seeders, then larger size. If no source produces a confident match the request stays **pending** for manual review; nothing is guessed.
### Library check
Both shelf sync and automatic downloads ask the libraries you have enabled under [Library Check](library-check.md) whether the book is already there, for the content type being requested, and skip it if so. There is no separate switch on this tab. A library that is not enabled, or that does not hold the requested content type, never causes a skip.
The check **fails open**: a library that cannot be read is logged and the book is treated as not owned, so processing continues.
## Settings
| Key | Label | Default | Notes |
|-----|-------|---------|-------|
| `HARDCOVER_SYNC_ENABLED` | Enable scheduled sync | Off | Runs shelf sync on the interval below |
| `HARDCOVER_SYNC_TOKEN` | Hardcover API Token | — | Leave blank to reuse the Hardcover metadata provider's token |
| `HARDCOVER_SYNC_STATUSES` | Shelves to sync | Want to Read | Hardcover reading status to pull from |
| `HARDCOVER_SYNC_CONTENT_TYPE` | Request as | Audiobooks | Content type given to synced requests and targeted by automatic downloads |
| `HARDCOVER_SYNC_INTERVAL` | Sync interval | 6 | Combined with the unit below |
| `HARDCOVER_SYNC_INTERVAL_UNIT` | Interval unit | Hours | Minutes or hours. The effective interval never drops below 60 seconds |
| `AUTO_DOWNLOAD_ENABLED` | Enable automatic downloads | Off | Auto-approve and queue strict matches for Hardcover requests |
| `AUTO_DOWNLOAD_SOURCE_PRIORITY` | Source priority | All usable sources | Drag to order. Sources you disable here, or that are not configured, are skipped |
| `AUTO_DOWNLOAD_MIN_SEEDERS` | Minimum seeders (torrents) | 1 | Torrent releases below this are ignored |
One action button sits at the bottom of the tab:
- **Sync now** runs a sync and automatic-download pass immediately using the saved settings. Sync runs even when scheduled sync is off; automatic downloads still require their own toggle.
## Request Flow
1. Shelf sync creates a **pending** request per new Hardcover book, with cover, year, and subtitle where Hardcover has them.
2. If automatic downloads are enabled, the request is searched source by source and the first strict match is queued through the normal fulfil path. The request becomes **fulfilled** and its delivery state is tracked like any other.
3. If no source yields a confident match, the request stays **pending** in the admin queue for manual review, exactly as if a user had requested the book.
Only requests whose provider is Hardcover are considered for automatic downloads, and requests that have already been dispatched are left alone.
## Scheduler
The scheduler starts with the app and waits **60 seconds** before its first cycle. Each cycle runs a shelf sync (when scheduled sync is enabled) followed by an automatic-download pass (when that is enabled), then sleeps for the configured interval. Intervals shorter than 60 seconds are raised to 60 seconds so a typo cannot hammer the Hardcover API. When neither toggle is on the scheduler idles.
Only one run happens at a time. If a scheduled cycle is in progress, **Sync now** reports that a sync is already running.
## Troubleshooting
**Sync adds nothing and the log says "no Hardcover token configured"** — set **Hardcover API Token** on this tab, or configure the token on the Hardcover metadata provider. A sync without a usable token counts as an error and creates no requests.
**Requests appear but nothing downloads** — check **Enable automatic downloads** is on, that **Source priority** contains at least one configured source that supports audiobooks, and look for `no strict match` lines in the log. Lowering **Minimum seeders** helps on quiet trackers; loosening the title or author match is deliberately not an option.
**Books you own keep being added** — enable the library check for the format you request (see [Library Check](library-check.md)) and use its test button. Because the check fails open, a library that cannot be read only produces a log warning rather than blocking the sync.
**Too many requests at once** — the first sync of a large shelf creates one request per book, and the per-user pending limit from **Users & Requests** does not apply to synced requests. Start with a shelf you are happy to see in the queue in full.
+1
View File
@@ -13,6 +13,7 @@ Use the guides below to set up the app, connect your library tools, and understa
## Core Guides
- [Users & Requests](users-and-requests.md)
- [Hardcover Sync](hardcover-sync.md)
- [Reverse Proxy](reverse-proxy.md)
- [OIDC](oidc.md)
- [API Access](api-access.md)
+169
View File
@@ -0,0 +1,169 @@
"""Settings tab for Hardcover wishlist sync + automatic downloads."""
from __future__ import annotations
from typing import Any
from shelfmark.core.settings_registry import (
ActionButton,
CheckboxField,
HeadingField,
NumberField,
OrderableListField,
PasswordField,
SelectField,
SettingsField,
register_settings,
)
_CONTENT_TYPE_OPTIONS = [
{"value": "audiobook", "label": "Audiobooks"},
{"value": "ebook", "label": "Ebooks"},
]
def _status_options() -> list[dict[str, str]]:
from shelfmark.metadata_providers.hardcover import HARDCOVER_STATUSES
return [{"value": str(status["id"]), "label": status["label"]} for status in HARDCOVER_STATUSES]
def _audiobook_source_options() -> list[dict[str, Any]]:
"""Orderable-list options: release sources that can return audiobooks."""
from shelfmark.release_sources import list_available_sources
options: list[dict[str, Any]] = []
for src in list_available_sources():
supported = src.get("supported_content_types") or ["ebook", "audiobook"]
if "audiobook" not in supported:
continue
enabled = bool(src.get("enabled"))
options.append(
{
"id": src["name"],
"label": src.get("display_name") or src["name"],
"description": None if enabled else "Source not configured / unavailable",
"isLocked": not enabled,
"disabledReason": None if enabled else "Source not configured / unavailable",
}
)
return options
def _sync_now(current_values: dict[str, Any] | None = None) -> dict[str, Any]:
"""Action-button callback: trigger an immediate sync + auto-download pass."""
from shelfmark.core import hardcover_scheduler
started = hardcover_scheduler.trigger_async(force=True)
if not started:
return {"success": False, "message": "A sync is already running. Try again shortly."}
return {
"success": True,
"message": "Sync started. New wishlist books will appear as requests; "
"matching audiobooks auto-download if enabled.",
}
@register_settings("hardcover_sync", "Hardcover Sync", icon="book", order=8)
def hardcover_sync_settings() -> list[SettingsField]:
"""Configure Hardcover wishlist sync and automatic downloads."""
return [
HeadingField(
key="hardcover_sync_heading",
title="Hardcover Wishlist Sync",
description=(
"Automatically pull books from your Hardcover reading shelves into "
"Shelfmark as requests, on a schedule."
),
),
CheckboxField(
key="HARDCOVER_SYNC_ENABLED",
label="Enable scheduled sync",
description="Periodically sync the selected Hardcover shelves into requests.",
default=False,
),
PasswordField(
key="HARDCOVER_SYNC_TOKEN",
label="Hardcover API Token",
description=(
"Bearer token for your Hardcover account. Leave blank to reuse the "
"token from the Hardcover metadata provider."
),
placeholder="Reuses metadata provider token if blank",
),
SelectField(
key="HARDCOVER_SYNC_STATUSES",
label="Shelves to sync",
description="Which Hardcover reading shelf to pull from.",
options=_status_options,
default="1",
),
SelectField(
key="HARDCOVER_SYNC_CONTENT_TYPE",
label="Request as",
description="Content type assigned to synced requests and targeted for downloads.",
options=_CONTENT_TYPE_OPTIONS,
default="audiobook",
),
NumberField(
key="HARDCOVER_SYNC_INTERVAL",
label="Sync interval",
description="How often the scheduled sync runs (minimum 1 minute).",
default=6,
min_value=1,
max_value=10000,
),
SelectField(
key="HARDCOVER_SYNC_INTERVAL_UNIT",
label="Interval unit",
description="Whether the sync interval is measured in minutes or hours.",
options=[
{"value": "minutes", "label": "Minutes"},
{"value": "hours", "label": "Hours"},
],
default="hours",
),
HeadingField(
key="auto_download_heading",
title="Automatic Downloads",
description=(
"When enabled, synced requests are auto-approved and downloaded from the "
"first source below that yields a strict title/author/format match. If no "
"confident match is found, the request is left pending for manual review."
),
),
CheckboxField(
key="AUTO_DOWNLOAD_ENABLED",
label="Enable automatic downloads",
description="Auto-approve and auto-download matching releases for synced requests.",
default=False,
),
OrderableListField(
key="AUTO_DOWNLOAD_SOURCE_PRIORITY",
label="Source priority",
description=(
"Drag to set which release sources are tried first. The first source with "
"a strict match wins. Disabled sources are skipped."
),
options=_audiobook_source_options,
# Default left empty so registration never imports release_sources (which would
# deadlock the registry lock). When empty, all available sources are used in
# registry order (see auto_download._configured_source_priority).
default=[],
),
NumberField(
key="AUTO_DOWNLOAD_MIN_SEEDERS",
label="Minimum seeders (torrents)",
description="Skip torrent releases with fewer seeders than this.",
default=1,
min_value=0,
max_value=1000,
),
ActionButton(
key="sync_now",
label="Sync now",
description="Run a sync + auto-download pass immediately (uses saved settings).",
style="primary",
callback=_sync_now,
),
]
+5
View File
@@ -2167,3 +2167,8 @@ def advanced_settings() -> list[SettingsField]:
register_on_save("advanced", _on_save_advanced)
# Register the Hardcover Sync settings tab. Imported here (rather than at each
# settings endpoint) so every code path that imports this module also registers it.
from shelfmark.config import hardcover_sync_settings as _hardcover_sync_settings # noqa: E402,F401
+395
View File
@@ -0,0 +1,395 @@
"""Automatic release selection + download for pending requests.
Given a pending request (typically synced from a Hardcover wishlist), this module
searches release sources in a user-configured priority order, applies a strict
title/author/format match guard, picks the best candidate from the first source
that yields a match, and queues it via the normal request-fulfilment path.
Design goals:
- **Strict by default.** Bias toward leaving a request pending (manual review) over
grabbing the wrong file. Title and author must both match; the format must be a
real audiobook format (no ebook-only fallbacks when targeting audiobooks).
- **Source priority is primary.** Walk the configured source order top-down and take
the first source that produces at least one strict match.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any
from shelfmark.core.config import config as app_config
from shelfmark.core.logger import setup_logger
from shelfmark.core.release_search import search_book_releases
from shelfmark.core.request_helpers import coerce_int
from shelfmark.core.text_match import (
DEFAULT_TITLE_MATCH_THRESHOLD,
author_surname,
title_tokens_match,
)
from shelfmark.core.text_match import (
tokens as _tokens,
)
if TYPE_CHECKING:
from collections.abc import Callable
from shelfmark.core.user_db import UserDB
from shelfmark.metadata_providers import BookMetadata
from shelfmark.release_sources import Release
logger = setup_logger(__name__)
# Fraction of significant book-title tokens that must appear in the release title.
TITLE_MATCH_THRESHOLD = DEFAULT_TITLE_MATCH_THRESHOLD
DEFAULT_AUDIOBOOK_FORMATS = ("m4b", "mp3")
EBOOK_FORMAT_MARKERS = (
"epub",
"mobi",
"azw3",
"azw",
"pdf",
"fb2",
"djvu",
"cbz",
"cbr",
)
# Audiobook signals we recognise in free-text release titles in addition to formats.
AUDIOBOOK_TITLE_MARKERS = ("audiobook", "unabridged", "m4b", "audio book")
# Audiobook format ranking for tie-breaking within a single source.
_FORMAT_RANK = {"m4b": 3, "m4a": 2, "mp3": 1}
@dataclass(frozen=True)
class AutoDownloadOutcome:
"""Result of attempting to auto-download a single request."""
request_id: int
status: str # "queued" | "no_match" | "skipped" | "error"
detail: str = ""
source: str | None = None
def _audiobook_formats() -> set[str]:
configured = app_config.get("SUPPORTED_AUDIOBOOK_FORMATS", list(DEFAULT_AUDIOBOOK_FORMATS))
values: list[object]
if isinstance(configured, str):
values = [configured]
elif isinstance(configured, (list, tuple, set)):
values = list(configured)
else:
values = []
formats = {str(fmt).strip().lower() for fmt in values if str(fmt).strip()}
return formats or set(DEFAULT_AUDIOBOOK_FORMATS)
def _author_surname_tokens(book: BookMetadata) -> list[str]:
"""Return distinctive author tokens (prefer the surname of the primary author)."""
author = book.search_author or (book.authors[0] if book.authors else "")
surname = author_surname(author)
return [surname] if surname else []
def _title_match(book: BookMetadata, release_title: str) -> bool:
return title_tokens_match(
book.search_title or book.title,
set(_tokens(release_title)),
TITLE_MATCH_THRESHOLD,
)
def _author_match(book: BookMetadata, release: Release) -> bool:
surname = _author_surname_tokens(book)
if not surname:
return True # No author metadata to verify against; don't block on it.
haystack = set(_tokens(release.title)) | set(_tokens(release.indexer))
extra_author = release.extra.get("author") if isinstance(release.extra, dict) else None
haystack |= set(_tokens(extra_author if isinstance(extra_author, str) else None))
return all(tok in haystack for tok in surname)
def _format_match(release: Release, audiobook_formats: set[str]) -> bool:
"""Require a real audiobook signal and reject ebook-only releases."""
fmt = (release.format or "").strip().lower()
title_l = (release.title or "").lower()
has_audiobook_signal = (
fmt in audiobook_formats
or any(marker in title_l for marker in AUDIOBOOK_TITLE_MARKERS)
or any(f".{af}" in title_l or f" {af}" in title_l for af in audiobook_formats)
)
if not has_audiobook_signal:
return False
# Reject things that are clearly an ebook and nothing else.
return fmt not in EBOOK_FORMAT_MARKERS
def _seeders_ok(release: Release, min_seeders: int) -> bool:
from shelfmark.release_sources import ReleaseProtocol
if release.protocol != ReleaseProtocol.TORRENT:
return True # Non-torrent protocols have no seeder concept.
if release.seeders is None:
return min_seeders <= 0
return release.seeders >= min_seeders
def strict_match(
release: Release,
book: BookMetadata,
*,
min_seeders: int = 1,
audiobook_formats: set[str] | None = None,
) -> bool:
"""Return True only if the release confidently matches the requested audiobook."""
formats = audiobook_formats if audiobook_formats is not None else _audiobook_formats()
return (
_title_match(book, release.title)
and _author_match(book, release)
and _format_match(release, formats)
and _seeders_ok(release, min_seeders)
)
def _release_sort_key(release: Release) -> tuple[int, int, int]:
fmt = (release.format or "").strip().lower()
if fmt not in _FORMAT_RANK and "m4b" in (release.title or "").lower():
fmt = "m4b"
return (
_FORMAT_RANK.get(fmt, 0),
release.seeders or 0,
release.size_bytes or 0,
)
def pick_best_release(releases: list[Release]) -> Release | None:
if not releases:
return None
return max(releases, key=_release_sort_key)
def build_release_data(release: Release, book: BookMetadata, content_type: str) -> dict[str, Any]:
"""Build a queue_release-compatible payload from a chosen Release."""
author = book.search_author or (book.authors[0] if book.authors else None)
extra = release.extra if isinstance(release.extra, dict) else {}
# Carry the book's cover so the download shows artwork immediately; the orchestrator
# reads release_data["preview"] and proxies it (orchestrator.queue_release / task_to_dict).
preview = book.cover_url or extra.get("preview")
payload: dict[str, Any] = {
"source": release.source,
"source_id": release.source_id,
"title": release.title,
"author": author,
"year": book.publish_year,
"preview": preview,
"format": release.format,
"size": release.size,
"size_bytes": release.size_bytes,
"download_url": release.download_url,
"info_url": release.info_url,
"protocol": release.protocol.value if release.protocol else None,
"indexer": release.indexer,
"seeders": release.seeders,
"language": release.language,
"content_type": release.content_type or content_type,
"series_name": book.series_name,
"series_position": book.series_position,
"subtitle": book.subtitle,
"extra": dict(extra),
}
return {key: value for key, value in payload.items() if value is not None}
def _configured_source_priority() -> list[str]:
"""Return enabled release-source names in configured priority order."""
from shelfmark.release_sources import list_available_sources
available = list_available_sources()
available_by_name = {src["name"]: src for src in available}
raw = app_config.get("AUTO_DOWNLOAD_SOURCE_PRIORITY", [])
ordered: list[str] = []
configured = isinstance(raw, list) and any(isinstance(item, dict) for item in raw)
if isinstance(raw, list):
for item in raw:
if not isinstance(item, dict):
continue
name = item.get("id")
if not isinstance(name, str) or name not in available_by_name:
continue
if not bool(item.get("enabled", True)):
continue
if not available_by_name[name].get("enabled", False):
continue # Source itself not usable (unconfigured/unavailable).
ordered.append(name)
if ordered:
return ordered
if configured:
# A priority list was saved but nothing in it is usable. Falling back to every
# source would search the ones the user turned off.
logger.warning(
"auto-download: every source in the priority list is disabled or unavailable"
)
return []
# Nothing configured yet: every usable source, registry order.
return [src["name"] for src in available if src.get("enabled")]
def auto_download_request(
user_db: UserDB,
request_row: dict[str, Any],
*,
sources: list[str],
content_type: str,
min_seeders: int,
queue_release: Callable[..., tuple[bool, str | None]],
admin_user_id: int,
) -> AutoDownloadOutcome:
"""Search, strict-match, and queue a single pending request."""
from shelfmark.core.requests_service import RequestServiceError, fulfil_request
from shelfmark.metadata_providers import (
get_provider,
get_provider_kwargs,
is_provider_registered,
)
request_id = int(request_row["id"])
book_data = request_row.get("book_data") or {}
if not isinstance(book_data, dict):
return AutoDownloadOutcome(request_id, "skipped", "no book_data")
provider_name = book_data.get("provider")
provider_id = book_data.get("provider_id") or book_data.get("id")
if not provider_name or not provider_id or not is_provider_registered(provider_name):
return AutoDownloadOutcome(request_id, "skipped", "no usable provider/id")
try:
prov = get_provider(provider_name, **get_provider_kwargs(provider_name))
book = prov.get_book(str(provider_id))
except Exception as exc: # noqa: BLE001 - provider/network errors shouldn't kill the loop
logger.warning("auto-download: metadata lookup failed for request %s: %s", request_id, exc)
return AutoDownloadOutcome(request_id, "error", f"metadata lookup failed: {exc}")
if book is None:
return AutoDownloadOutcome(request_id, "skipped", "book not found in provider")
# Final guard: skip if the book is already in a library that is checked for this content type.
from shelfmark.core.library_index import is_in_library
if is_in_library(book, content_type):
logger.info(
"auto-download: request %s (%s) already in library; skipping",
request_id,
book.title,
)
return AutoDownloadOutcome(request_id, "in_library", "already in library")
audiobook_formats = _audiobook_formats()
# Walk sources in priority order; take the first source with a strict match.
for source_name in sources:
_all, by_source, _errors = search_book_releases(
book,
sources=[source_name],
content_type=content_type,
expand_search=True,
)
candidates = [
release
for release in by_source.get(source_name, [])
if strict_match(
release,
book,
min_seeders=min_seeders,
audiobook_formats=audiobook_formats,
)
]
chosen = pick_best_release(candidates)
if chosen is None:
continue
release_data = build_release_data(chosen, book, content_type)
try:
fulfil_request(
user_db,
request_id=request_id,
admin_user_id=admin_user_id,
queue_release=queue_release,
release_data=release_data,
)
except RequestServiceError as exc:
logger.warning("auto-download: fulfil failed for request %s: %s", request_id, exc)
return AutoDownloadOutcome(request_id, "error", str(exc), source=source_name)
logger.info(
"auto-download: queued request %s from %s (%s)",
request_id,
source_name,
chosen.title,
)
return AutoDownloadOutcome(request_id, "queued", chosen.title, source=source_name)
logger.info("auto-download: no strict match for request %s (%s)", request_id, book.title)
return AutoDownloadOutcome(request_id, "no_match", "no strict match across sources")
def auto_download_pending(
user_db: UserDB,
*,
queue_release: Callable[..., tuple[bool, str | None]],
provider_filter: str | None = "hardcover",
admin_user_id: int | None = None,
) -> dict[str, int]:
"""Run the auto-download pass over all eligible pending requests.
Returns a summary count dict. No-ops (returns zeros) unless AUTO_DOWNLOAD_ENABLED.
``admin_user_id`` defaults to the lowest-id admin, who fulfils the requests.
"""
if not bool(app_config.get("AUTO_DOWNLOAD_ENABLED", False)):
return {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
if admin_user_id is None:
from shelfmark.core.hardcover_sync import resolve_request_owner
admin_user_id = resolve_request_owner(user_db)
if admin_user_id is None:
logger.warning("auto-download: no admin user exists to fulfil requests")
return {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
content_type = str(app_config.get("HARDCOVER_SYNC_CONTENT_TYPE", "audiobook") or "audiobook")
min_seeders = coerce_int(app_config.get("AUTO_DOWNLOAD_MIN_SEEDERS", 1), 1)
sources = _configured_source_priority()
if not sources:
logger.warning("auto-download: no usable release sources configured")
return {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
pending = user_db.list_requests(status="pending")
summary = {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
for row in pending:
book_data = row.get("book_data") or {}
if provider_filter and (
not isinstance(book_data, dict) or book_data.get("provider") != provider_filter
):
continue
# Only act on requests that haven't already been dispatched.
if str(row.get("delivery_state") or "none").lower() not in {"none", ""}:
continue
outcome = auto_download_request(
user_db,
row,
sources=sources,
content_type=content_type,
min_seeders=min_seeders,
queue_release=queue_release,
admin_user_id=admin_user_id,
)
summary[outcome.status] = summary.get(outcome.status, 0) + 1
logger.info("auto-download pass complete: %s", summary)
return summary
+138
View File
@@ -0,0 +1,138 @@
"""Background scheduler for Hardcover wishlist sync + auto-download.
Runs an in-process daemon thread (mirroring the download coordinator pattern in
``shelfmark.download.orchestrator``) that periodically syncs configured Hardcover
shelves into requests and auto-downloads them. Also exposes a manual trigger used
by the "Sync now" settings action button.
"""
from __future__ import annotations
import threading
import time
from typing import TYPE_CHECKING, Any
from shelfmark.core.config import config as app_config
from shelfmark.core.logger import setup_logger
if TYPE_CHECKING:
from collections.abc import Callable
from shelfmark.core.user_db import UserDB
logger = setup_logger(__name__)
_WARMUP_SECONDS = 60 # Delay before the first cycle after startup.
_MIN_INTERVAL_SECONDS = 60 # Floor so a misconfigured interval can't hammer the API.
_UNIT_SECONDS = {"minutes": 60.0, "hours": 3600.0}
_ctx_lock = threading.Lock()
_ctx: dict[str, Any] = {}
_thread: threading.Thread | None = None
_thread_lock = threading.Lock()
# Prevents a scheduled cycle and a manual "Sync now" from overlapping.
_run_lock = threading.Lock()
def configure(
user_db: UserDB,
queue_release: Callable[..., tuple[bool, str | None]],
db_path: str | None,
) -> None:
"""Store the runtime context the background work needs."""
with _ctx_lock:
_ctx["user_db"] = user_db
_ctx["queue_release"] = queue_release
_ctx["db_path"] = db_path
def _interval_seconds() -> float:
raw = app_config.get("HARDCOVER_SYNC_INTERVAL", 6)
try:
value = float(raw) if isinstance(raw, (int, float, str)) else 6.0
except ValueError:
value = 6.0
unit = str(app_config.get("HARDCOVER_SYNC_INTERVAL_UNIT", "hours") or "hours").strip().lower()
multiplier = _UNIT_SECONDS.get(unit, _UNIT_SECONDS["hours"])
return max(value * multiplier, _MIN_INTERVAL_SECONDS)
def _enabled() -> bool:
return bool(app_config.get("HARDCOVER_SYNC_ENABLED", False)) or bool(
app_config.get("AUTO_DOWNLOAD_ENABLED", False)
)
def run_once(*, force: bool = False) -> dict[str, Any]:
"""Run one sync + auto-download pass. Returns a combined summary.
``force=True`` (the manual "Sync now" path) syncs regardless of the scheduler
toggle; auto-download always remains gated by ``AUTO_DOWNLOAD_ENABLED`` inside
``auto_download_pending``. Skips (returns ``{"status": "busy"}``) if another run
is in progress.
"""
if not _run_lock.acquire(blocking=False):
return {"status": "busy"}
try:
with _ctx_lock:
user_db = _ctx.get("user_db")
queue_release = _ctx.get("queue_release")
db_path = _ctx.get("db_path")
if user_db is None or queue_release is None:
return {"status": "unconfigured"}
from shelfmark.core.auto_download import auto_download_pending
from shelfmark.core.hardcover_sync import sync_wishlist
sync_summary: dict[str, Any] | None = None
if force or bool(app_config.get("HARDCOVER_SYNC_ENABLED", False)):
sync_summary = sync_wishlist(user_db, db_path=db_path)
auto_summary = auto_download_pending(user_db, queue_release=queue_release)
return {"status": "ok", "sync": sync_summary, "auto_download": auto_summary}
finally:
_run_lock.release()
def trigger_async(*, force: bool = True) -> bool:
"""Kick off a one-off run in a background thread. Returns False if one is active."""
if _run_lock.locked():
return False
thread = threading.Thread(
target=lambda: run_once(force=force), daemon=True, name="HardcoverSyncManual"
)
thread.start()
return True
def _scheduler_loop() -> None:
logger.info("Hardcover scheduler started")
time.sleep(_WARMUP_SECONDS)
while True:
try:
if _enabled():
logger.info("Hardcover scheduler: starting cycle")
result = run_once()
logger.info("Hardcover scheduler: cycle result %s", result)
except Exception:
logger.exception("Hardcover scheduler: cycle failed")
time.sleep(_interval_seconds())
def start(
user_db: UserDB,
queue_release: Callable[..., tuple[bool, str | None]],
db_path: str | None,
) -> None:
"""Configure context and start the scheduler thread (idempotent)."""
configure(user_db, queue_release, db_path)
global _thread
with _thread_lock:
if _thread is not None and _thread.is_alive():
logger.debug("Hardcover scheduler already running")
return
_thread = threading.Thread(target=_scheduler_loop, daemon=True, name="HardcoverScheduler")
_thread.start()
logger.info("Hardcover scheduler thread launched")
+225
View File
@@ -0,0 +1,225 @@
"""Sync a Hardcover reading shelf (e.g. "Want to Read") into Shelfmark requests.
This replaces the standalone ``scripts/sync_hardcover_wishlist.py`` with an in-app
service that reuses the registered Hardcover metadata provider (so synced requests
carry full metadata incl. covers) and the request-service validation/dedup path.
"""
from __future__ import annotations
import sqlite3
from typing import TYPE_CHECKING, Any
from shelfmark.core.config import config as app_config
from shelfmark.core.logger import setup_logger
from shelfmark.core.utils import transform_cover_url
if TYPE_CHECKING:
from shelfmark.core.user_db import UserDB
from shelfmark.metadata_providers import BookMetadata
logger = setup_logger(__name__)
# Hardcover status_id -> shelf. Default sync target is "Want to Read" (1).
DEFAULT_SYNC_STATUSES = ("1",)
_PAGE_LIMIT = 25 # Hardcover API page size.
_MAX_PAGES = 40 # Safety bound (~1000 books) so a bad response can't loop forever.
def _configured_token() -> str:
"""Prefer a dedicated sync token, fall back to the provider API key."""
token = app_config.get("HARDCOVER_SYNC_TOKEN", "") or app_config.get("HARDCOVER_API_KEY", "")
return str(token or "").strip()
def _configured_statuses() -> list[int]:
raw = app_config.get("HARDCOVER_SYNC_STATUSES", list(DEFAULT_SYNC_STATUSES))
values: list[object]
if isinstance(raw, str):
values = [raw]
elif isinstance(raw, (list, tuple)):
values = list(raw)
else:
values = []
statuses: list[int] = []
for value in values:
try:
statuses.append(int(str(value).strip()))
except ValueError:
continue
return statuses or [int(s) for s in DEFAULT_SYNC_STATUSES]
def _build_provider() -> Any | None:
from shelfmark.metadata_providers import get_provider
token = _configured_token()
if not token:
logger.warning("hardcover-sync: no Hardcover token configured")
return None
provider = get_provider("hardcover", api_key=token)
if not provider.is_available():
logger.warning("hardcover-sync: Hardcover provider unavailable (bad token?)")
return None
return provider
def resolve_request_owner(user_db: UserDB) -> int | None:
"""Return the lowest-id admin, who owns synced requests and approves auto-downloads."""
admin = user_db.get_first_admin()
return int(admin["id"]) if admin else None
def _primary_author(book: BookMetadata) -> str:
if book.search_author:
return book.search_author
if book.authors:
return book.authors[0]
return "Unknown"
def _book_to_book_data(book: BookMetadata, content_type: str) -> dict[str, Any]:
book_data: dict[str, Any] = {
"title": book.title,
"author": _primary_author(book),
"provider": "hardcover",
"provider_id": str(book.provider_id),
"content_type": content_type,
}
# Proxy the cover the same way search results are, so the row is ready for the UI.
if book.cover_url:
book_data["preview"] = transform_cover_url(book.cover_url, f"hardcover_{book.provider_id}")
if book.publish_year:
book_data["year"] = book.publish_year
if book.subtitle:
book_data["subtitle"] = book.subtitle
return book_data
def _existing_provider_ids(user_db: UserDB) -> set[str]:
"""Hardcover ids already requested, in any status. Other providers' ids are not counted."""
ids: set[str] = set()
for row in user_db.list_requests():
book_data = row.get("book_data") or {}
if isinstance(book_data, dict) and book_data.get("provider") == "hardcover":
pid = book_data.get("provider_id") or book_data.get("id")
if pid is not None:
ids.add(str(pid))
return ids
def _already_downloaded(db_path: str | None, title: str, author: str) -> bool:
"""Best-effort check that a title/author isn't already in download_history."""
if not db_path:
return False
try:
conn = sqlite3.connect(db_path)
try:
row = conn.execute(
"SELECT 1 FROM download_history "
"WHERE LOWER(title) = ? AND LOWER(author) = ? LIMIT 1",
(title.strip().lower(), author.strip().lower()),
).fetchone()
return row is not None
finally:
conn.close()
except sqlite3.Error as exc:
logger.debug("hardcover-sync: history dedup query failed: %s", exc)
return False
def _fetch_status_books(provider: Any, status_id: int) -> list[BookMetadata]:
"""Page through a Hardcover status shelf, returning all books."""
books: list[BookMetadata] = []
for page in range(1, _MAX_PAGES + 1):
try:
result = provider._fetch_current_user_books_by_status(status_id, page, _PAGE_LIMIT)
except Exception as exc: # noqa: BLE001 - network/provider errors shouldn't abort the sync
logger.warning(
"hardcover-sync: fetch failed (status=%s page=%s): %s", status_id, page, exc
)
break
page_books = list(result.books or [])
books.extend(page_books)
if not result.has_more or not page_books:
break
return books
def sync_wishlist(
user_db: UserDB,
*,
db_path: str | None = None,
user_id: int | None = None,
) -> dict[str, int]:
"""Sync configured Hardcover shelves into pending requests.
Returns a summary dict: ``{"added", "skipped", "in_library", "errors"}``. Requires a
configured token; enable-gating is the caller's responsibility (see hardcover_scheduler).
"""
summary = {"added": 0, "skipped": 0, "in_library": 0, "errors": 0}
if user_id is None:
user_id = resolve_request_owner(user_db)
if user_id is None:
logger.warning("hardcover-sync: no admin user exists to own synced requests")
summary["errors"] += 1
return summary
provider = _build_provider()
if provider is None:
summary["errors"] += 1
return summary
content_type = str(app_config.get("HARDCOVER_SYNC_CONTENT_TYPE", "audiobook") or "audiobook")
known_provider_ids = _existing_provider_ids(user_db)
from shelfmark.core.library_index import is_in_library
from shelfmark.core.requests_service import RequestServiceError, create_request
for status_id in _configured_statuses():
for book in _fetch_status_books(provider, status_id):
provider_id = str(book.provider_id)
author = _primary_author(book)
if provider_id in known_provider_ids:
summary["skipped"] += 1
continue
if _already_downloaded(db_path, book.title, author):
summary["skipped"] += 1
continue
if is_in_library(book, content_type):
summary["in_library"] += 1
logger.info("hardcover-sync: '%s' already in your library; skipping", book.title)
continue
try:
create_request(
user_db,
user_id=user_id,
source_hint=None,
content_type=content_type,
request_level="book",
policy_mode="request_book",
book_data=_book_to_book_data(book, content_type),
note=None,
)
except RequestServiceError as exc:
# Duplicate / max-pending / validation: treat as skip, not failure.
if exc.code in {"duplicate_pending_request", "max_pending_reached"}:
summary["skipped"] += 1
else:
logger.warning("hardcover-sync: could not add '%s': %s", book.title, exc)
summary["errors"] += 1
continue
except Exception:
logger.exception("hardcover-sync: unexpected error adding '%s'", book.title)
summary["errors"] += 1
continue
known_provider_ids.add(provider_id)
summary["added"] += 1
logger.info("hardcover-sync: added request for '%s' by %s", book.title, author)
logger.info("hardcover-sync complete: %s", summary)
return summary
+38
View File
@@ -92,3 +92,41 @@ def search_source_releases(
return None, [], f"{source_name}: {exc!s}"
else:
return source, releases, None
def search_book_releases(
book: BookMetadata,
*,
sources: list[str],
content_type: str = "ebook",
expand_search: bool = True,
languages: list[str] | None = None,
indexers: list[str] | None = None,
) -> tuple[list[Release], dict[str, list[Release]], list[str]]:
"""Search several sources for releases of ``book``.
Returns ``(all_releases, releases_by_source, errors)``. ``releases_by_source``
preserves the requested ``sources`` order so callers can prioritise.
"""
all_releases: list[Release] = []
by_source: dict[str, list[Release]] = {}
errors: list[str] = []
for source_name in sources:
source, releases, error = search_source_releases(
source_name,
book,
languages=languages,
manual_query=None,
indexers=indexers,
expand_search=expand_search,
content_type=content_type,
source_filters=None,
)
if source is not None:
by_source[source_name] = list(releases)
all_releases.extend(releases)
if error is not None:
errors.append(error)
return all_releases, by_source, errors
+9
View File
@@ -291,6 +291,15 @@ if user_db is not None and download_history_service is not None:
# Start download coordinator
backend.start()
# Start the Hardcover wishlist sync + auto-download scheduler (in-process daemon).
if user_db is not None:
try:
from shelfmark.core import hardcover_scheduler
hardcover_scheduler.start(user_db, backend.queue_release, _user_db_path)
except (ImportError, RuntimeError, OSError) as exc:
logger.warning("Failed to start Hardcover scheduler: %s", exc)
# Pre-solve the direct-download source's protection challenge in the background so the
# first user search does not pay for a cold Chrome bypass. Never blocks startup.
warmup.start()
@@ -0,0 +1,71 @@
"""Tests for the Hardcover Sync settings tab registration and action buttons."""
import shelfmark.config.hardcover_sync_settings # noqa: F401 - registers the tab
from shelfmark.core import settings_registry
def _field_map():
tab = settings_registry.get_settings_tab("hardcover_sync")
assert tab is not None
return {field.key: field for field in tab.fields if hasattr(field, "key")}
def test_tab_is_registered():
tab = settings_registry.get_settings_tab("hardcover_sync")
assert tab is not None
assert tab.display_name == "Hardcover Sync"
def test_everything_is_off_by_default():
fields = _field_map()
assert fields["HARDCOVER_SYNC_ENABLED"].default is False
assert fields["AUTO_DOWNLOAD_ENABLED"].default is False
def test_schedule_and_content_type_defaults():
fields = _field_map()
assert fields["HARDCOVER_SYNC_INTERVAL"].default == 6
assert fields["HARDCOVER_SYNC_INTERVAL"].min_value == 1
assert fields["HARDCOVER_SYNC_INTERVAL_UNIT"].default == "hours"
assert fields["HARDCOVER_SYNC_CONTENT_TYPE"].default == "audiobook"
assert fields["HARDCOVER_SYNC_STATUSES"].default == "1"
assert fields["AUTO_DOWNLOAD_SOURCE_PRIORITY"].default == []
def test_shelf_options_come_from_hardcover_statuses():
options = _field_map()["HARDCOVER_SYNC_STATUSES"].options()
assert {"value": "1", "label": "Want to Read"} in options
def test_action_buttons_are_registered():
fields = _field_map()
assert fields["sync_now"].callback is not None
def test_sync_now_reports_busy_when_a_run_is_active(monkeypatch):
monkeypatch.setattr("shelfmark.core.hardcover_scheduler.trigger_async", lambda **_: False)
result = settings_registry.execute_action("hardcover_sync", "sync_now")
assert result["success"] is False
assert "already running" in result["message"]
def test_sync_now_starts_a_forced_run(monkeypatch):
seen = []
def _trigger(**kwargs):
seen.append(kwargs)
return True
monkeypatch.setattr("shelfmark.core.hardcover_scheduler.trigger_async", _trigger)
result = settings_registry.execute_action("hardcover_sync", "sync_now")
assert result["success"] is True
assert seen == [{"force": True}]
+401
View File
@@ -0,0 +1,401 @@
"""Tests for strict release matching and auto-download of pending requests."""
import pytest
from shelfmark.core import auto_download
from shelfmark.core.user_db import UserDB
from shelfmark.metadata_providers import BookMetadata
from shelfmark.release_sources import Release, ReleaseProtocol
AUDIOBOOK_FORMATS = {"m4b", "mp3"}
class _Config:
def __init__(self, **values):
self.values = values
def get(self, key, default=None, user_id=None):
return self.values.get(key, default)
def _book(**overrides):
fields = {
"provider": "hardcover",
"provider_id": "42",
"title": "Dungeon Crawler Carl",
"authors": ["Matt Dinniman"],
}
fields.update(overrides)
return BookMetadata(**fields)
def _release(**overrides):
fields = {
"source": "prowlarr",
"source_id": "rel-1",
"title": "Dungeon Crawler Carl - Matt Dinniman [M4B]",
"format": "m4b",
"protocol": ReleaseProtocol.TORRENT,
"seeders": 10,
}
fields.update(overrides)
return Release(**fields)
def _strict(release, **kwargs):
kwargs.setdefault("audiobook_formats", AUDIOBOOK_FORMATS)
return auto_download.strict_match(release, _book(), **kwargs)
@pytest.fixture
def user_db(tmp_path):
db = UserDB(str(tmp_path / "users.db"))
db.initialize()
return db
def _pending_request(user_db, user_id, **book_data_overrides):
book_data = {
"title": "Dungeon Crawler Carl",
"author": "Matt Dinniman",
"provider": "hardcover",
"provider_id": "42",
"content_type": "audiobook",
}
book_data.update(book_data_overrides)
return user_db.create_request(
user_id=user_id,
content_type="audiobook",
request_level="book",
policy_mode="request_book",
book_data=book_data,
)
def _stub_provider(monkeypatch, book):
class _Provider:
def get_book(self, book_id):
assert book_id == "42"
return book
monkeypatch.setattr(
"shelfmark.metadata_providers.is_provider_registered", lambda name: name == "hardcover"
)
monkeypatch.setattr("shelfmark.metadata_providers.get_provider_kwargs", lambda _name: {})
monkeypatch.setattr(
"shelfmark.metadata_providers.get_provider", lambda _name, **_kwargs: _Provider()
)
class TestStrictMatch:
def test_matching_release_passes(self):
assert _strict(_release()) is True
def test_title_mismatch_rejected(self):
assert _strict(_release(title="Some Other Book - Matt Dinniman [M4B]")) is False
def test_missing_author_surname_rejected(self):
assert _strict(_release(title="Dungeon Crawler Carl [M4B]", indexer=None)) is False
def test_author_found_in_extra_metadata(self):
release = _release(title="Dungeon Crawler Carl [M4B]", extra={"author": "Matt Dinniman"})
assert _strict(release) is True
def test_ebook_only_release_rejected(self):
release = _release(title="Dungeon Crawler Carl - Matt Dinniman", format="epub")
assert _strict(release) is False
def test_audiobook_signal_in_title_accepted_without_format(self):
release = _release(title="Dungeon Crawler Carl by Matt Dinniman (Unabridged)", format=None)
assert _strict(release) is True
@pytest.mark.parametrize("seeders", [0, None])
def test_torrent_below_minimum_seeders_rejected(self, seeders):
assert _strict(_release(seeders=seeders), min_seeders=1) is False
def test_torrent_without_seeder_count_allowed_when_minimum_is_zero(self):
assert _strict(_release(seeders=None), min_seeders=0) is True
def test_non_torrent_ignores_seeders(self):
release = _release(protocol=ReleaseProtocol.NZB, seeders=None)
assert _strict(release, min_seeders=5) is True
class TestPickBestRelease:
def test_empty_returns_none(self):
assert auto_download.pick_best_release([]) is None
def test_m4b_outranks_mp3_regardless_of_seeders(self):
mp3 = _release(source_id="mp3", format="mp3", seeders=500)
m4b = _release(source_id="m4b", format="m4b", seeders=1)
assert auto_download.pick_best_release([mp3, m4b]) is m4b
def test_m4b_in_title_ranks_as_m4b_when_format_missing(self):
mp3 = _release(source_id="mp3", format="mp3", seeders=500)
untagged = _release(source_id="untagged", format=None, seeders=1)
assert auto_download.pick_best_release([mp3, untagged]) is untagged
def test_seeders_break_format_ties(self):
few = _release(source_id="few", seeders=2, size_bytes=10)
many = _release(source_id="many", seeders=20, size_bytes=1)
assert auto_download.pick_best_release([few, many]) is many
def test_size_breaks_seeder_ties(self):
small = _release(source_id="small", seeders=5, size_bytes=100)
large = _release(source_id="large", seeders=5, size_bytes=200)
assert auto_download.pick_best_release([small, large]) is large
class TestBuildReleaseData:
def test_drops_none_values(self):
release = _release(size=None, size_bytes=None, language=None, info_url=None)
data = auto_download.build_release_data(release, _book(), "audiobook")
assert data["source"] == "prowlarr"
assert data["source_id"] == "rel-1"
assert data["protocol"] == "torrent"
assert data["author"] == "Matt Dinniman"
assert data["content_type"] == "audiobook"
assert not {"size", "size_bytes", "language", "info_url", "preview", "year"} & data.keys()
def test_carries_cover_series_and_book_metadata(self):
book = _book(
cover_url="https://img.example/cover.jpg",
publish_year=2020,
subtitle="Book One",
series_name="Dungeon Crawler Carl",
series_position=1.0,
)
release = _release(content_type=None, extra={"preview": "https://img.example/rel.jpg"})
data = auto_download.build_release_data(release, book, "audiobook")
assert data["preview"] == "https://img.example/cover.jpg"
assert data["year"] == 2020
assert data["subtitle"] == "Book One"
assert data["series_name"] == "Dungeon Crawler Carl"
assert data["series_position"] == 1.0
assert data["content_type"] == "audiobook"
assert data["extra"] == {"preview": "https://img.example/rel.jpg"}
def test_falls_back_to_release_preview_without_book_cover(self):
release = _release(extra={"preview": "https://img.example/rel.jpg"})
data = auto_download.build_release_data(release, _book(), "audiobook")
assert data["preview"] == "https://img.example/rel.jpg"
class TestConfiguredSourcePriority:
@pytest.fixture(autouse=True)
def _available_sources(self, monkeypatch):
monkeypatch.setattr(
"shelfmark.release_sources.list_available_sources",
lambda: [
{"name": "first", "enabled": True},
{"name": "second", "enabled": True},
{"name": "unavailable", "enabled": False},
],
)
def _configure(self, monkeypatch, priority):
monkeypatch.setattr(
auto_download, "app_config", _Config(AUTO_DOWNLOAD_SOURCE_PRIORITY=priority)
)
def test_respects_configured_order_and_skips_unusable_entries(self, monkeypatch):
self._configure(
monkeypatch,
[
{"id": "second"},
{"id": "first", "enabled": False},
{"id": "unavailable"},
{"id": "unknown"},
"not-a-dict",
],
)
assert auto_download._configured_source_priority() == ["second"]
@pytest.mark.parametrize("priority", [[], "garbage", ["not-a-dict"]])
def test_falls_back_to_all_enabled_sources_when_nothing_is_configured(
self, monkeypatch, priority
):
self._configure(monkeypatch, priority)
assert auto_download._configured_source_priority() == ["first", "second"]
@pytest.mark.parametrize(
"priority",
[
[{"id": "first", "enabled": False}, {"id": "second", "enabled": False}],
[{"id": "unavailable"}],
[{"id": "unknown"}],
],
)
def test_a_configured_list_with_nothing_usable_searches_nothing(self, monkeypatch, priority):
self._configure(monkeypatch, priority)
assert auto_download._configured_source_priority() == []
class TestAutoDownloadPending:
def test_disabled_returns_zeros_without_touching_the_db(self, monkeypatch):
monkeypatch.setattr(auto_download, "app_config", _Config(AUTO_DOWNLOAD_ENABLED=False))
class _UntouchableDb:
def list_requests(self, **_kwargs):
raise AssertionError("pending requests must not be queried")
summary = auto_download.auto_download_pending(
_UntouchableDb(), queue_release=lambda *_args, **_kwargs: (True, None)
)
assert summary == {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
class TestAutoDownloadRequest:
@pytest.fixture(autouse=True)
def _library_check_off(self, monkeypatch):
monkeypatch.setattr(auto_download, "app_config", _Config())
monkeypatch.setattr(
"shelfmark.core.library_index.is_in_library", lambda *_args, **_kwargs: False
)
def _run(self, user_db, row, admin, *, queue_release=None, sources=("first", "second")):
return auto_download.auto_download_request(
user_db,
row,
sources=list(sources),
content_type="audiobook",
min_seeders=1,
queue_release=queue_release or (lambda *_args, **_kwargs: (True, None)),
admin_user_id=admin["id"],
)
def test_queues_first_strict_match_via_fulfil_path(self, user_db, monkeypatch):
admin = user_db.create_user(username="admin", role="admin")
reader = user_db.create_user(username="reader", role="user")
row = _pending_request(user_db, reader["id"])
_stub_provider(monkeypatch, _book(cover_url="https://img.example/cover.jpg"))
chosen = _release(source="second", source_id="good", seeders=5)
searched = []
def _search(_book_arg, *, sources, **_kwargs):
searched.extend(sources)
if sources == ["first"]:
epub = _release(source="first", title="Dungeon Crawler Carl.epub", format="epub")
return [epub], {"first": [epub]}, []
return [chosen], {"second": [chosen]}, []
monkeypatch.setattr(auto_download, "search_book_releases", _search)
queued = []
def _queue(release_data, priority, *, user_id, username):
queued.append((release_data, priority, user_id, username))
return True, None
outcome = self._run(user_db, row, admin, queue_release=_queue)
assert outcome.status == "queued"
assert outcome.source == "second"
assert searched == ["first", "second"]
stored = user_db.get_request(row["id"])
assert stored["status"] == "fulfilled"
assert stored["delivery_state"] == "queued"
assert stored["reviewed_by"] == admin["id"]
assert stored["release_data"]["source_id"] == "good"
release_data, priority, user_id, username = queued[0]
assert release_data["_request_id"] == row["id"]
assert release_data["preview"] == "https://img.example/cover.jpg"
assert priority == 0
assert user_id == reader["id"]
assert username == "reader"
def test_queue_failure_rolls_request_back_to_pending(self, user_db, monkeypatch):
admin = user_db.create_user(username="admin", role="admin")
reader = user_db.create_user(username="reader", role="user")
row = _pending_request(user_db, reader["id"])
_stub_provider(monkeypatch, _book())
chosen = _release(source="first")
monkeypatch.setattr(
auto_download,
"search_book_releases",
lambda *_args, **_kwargs: ([chosen], {"first": [chosen]}, []),
)
outcome = self._run(
user_db, row, admin, queue_release=lambda *_args, **_kwargs: (False, "client down")
)
assert outcome.status == "error"
assert outcome.detail == "client down"
stored = user_db.get_request(row["id"])
assert stored["status"] == "pending"
assert stored["release_data"] is None
def test_no_strict_match_leaves_request_pending(self, user_db, monkeypatch):
admin = user_db.create_user(username="admin", role="admin")
reader = user_db.create_user(username="reader", role="user")
row = _pending_request(user_db, reader["id"])
_stub_provider(monkeypatch, _book())
wrong = _release(title="A Different Audiobook [M4B]")
monkeypatch.setattr(
auto_download,
"search_book_releases",
lambda *_args, **_kwargs: ([wrong], {"first": [wrong], "second": [wrong]}, []),
)
def _never_queue(*_args, **_kwargs):
raise AssertionError("nothing should be queued")
outcome = self._run(user_db, row, admin, queue_release=_never_queue)
assert outcome.status == "no_match"
assert user_db.get_request(row["id"])["status"] == "pending"
def test_book_already_in_library_is_skipped_before_searching(self, user_db, monkeypatch):
admin = user_db.create_user(username="admin", role="admin")
reader = user_db.create_user(username="reader", role="user")
row = _pending_request(user_db, reader["id"])
_stub_provider(monkeypatch, _book())
monkeypatch.setattr(auto_download, "app_config", _Config())
monkeypatch.setattr(
"shelfmark.core.library_index.is_in_library", lambda *_args, **_kwargs: True
)
def _never_search(*_args, **_kwargs):
raise AssertionError("release search must not run")
monkeypatch.setattr(auto_download, "search_book_releases", _never_search)
outcome = self._run(user_db, row, admin)
assert outcome.status == "in_library"
assert user_db.get_request(row["id"])["status"] == "pending"
def test_unregistered_provider_is_skipped(self, user_db, monkeypatch):
admin = user_db.create_user(username="admin", role="admin")
reader = user_db.create_user(username="reader", role="user")
row = _pending_request(user_db, reader["id"], provider="nope")
monkeypatch.setattr("shelfmark.metadata_providers.is_provider_registered", lambda _n: False)
outcome = self._run(user_db, row, admin)
assert outcome.status == "skipped"
assert outcome.detail == "no usable provider/id"
def test_pending_pass_without_an_admin_queues_nothing(user_db, monkeypatch):
monkeypatch.setattr(auto_download, "app_config", _Config(AUTO_DOWNLOAD_ENABLED=True))
user_db.create_user(username="reader", role="user")
summary = auto_download.auto_download_pending(
user_db, queue_release=lambda *_args, **_kwargs: (True, None)
)
assert summary == {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
+158
View File
@@ -0,0 +1,158 @@
"""Tests for the Hardcover sync scheduler's interval parsing and run gating."""
import threading
import pytest
from shelfmark.core import hardcover_scheduler as scheduler
class _Config:
def __init__(self, **values):
self.values = values
def get(self, key, default=None, user_id=None):
return self.values.get(key, default)
@pytest.fixture(autouse=True)
def _isolated_scheduler(monkeypatch):
monkeypatch.setattr(scheduler, "_ctx", {})
monkeypatch.setattr(scheduler, "_run_lock", threading.Lock())
monkeypatch.setattr(scheduler, "app_config", _Config())
def _configure(monkeypatch, **values):
monkeypatch.setattr(scheduler, "app_config", _Config(**values))
@pytest.mark.parametrize(
("interval", "unit", "expected"),
[
(30, "minutes", 1800),
(2, "hours", 7200),
("1.5", "hours", 5400),
(0.5, "minutes", 60),
(6, "fortnights", 6 * 3600),
("garbage", "hours", 6 * 3600),
([], "hours", 6 * 3600),
],
)
def test_interval_seconds(monkeypatch, interval, unit, expected):
_configure(monkeypatch, HARDCOVER_SYNC_INTERVAL=interval, HARDCOVER_SYNC_INTERVAL_UNIT=unit)
assert scheduler._interval_seconds() == expected
def test_interval_seconds_defaults_to_six_hours(monkeypatch):
_configure(monkeypatch)
assert scheduler._interval_seconds() == 6 * 3600
@pytest.mark.parametrize(
("values", "expected"),
[
({}, False),
({"HARDCOVER_SYNC_ENABLED": True}, True),
({"AUTO_DOWNLOAD_ENABLED": True}, True),
({"HARDCOVER_SYNC_ENABLED": False, "AUTO_DOWNLOAD_ENABLED": False}, False),
],
)
def test_enabled(monkeypatch, values, expected):
_configure(monkeypatch, **values)
assert scheduler._enabled() is expected
def test_run_once_before_configure_is_unconfigured():
assert scheduler.run_once() == {"status": "unconfigured"}
assert scheduler.run_once(force=True) == {"status": "unconfigured"}
def test_run_once_reports_busy_while_another_run_holds_the_lock():
with scheduler._run_lock:
assert scheduler.run_once(force=True) == {"status": "busy"}
assert scheduler.run_once() == {"status": "unconfigured"}
class TestRunOnce:
@pytest.fixture
def calls(self, monkeypatch):
recorded = {"sync": [], "auto": []}
def _sync(user_db, **kwargs):
recorded["sync"].append((user_db, kwargs))
return {"added": 1}
def _auto(user_db, **kwargs):
recorded["auto"].append((user_db, kwargs))
return {"queued": 1}
monkeypatch.setattr("shelfmark.core.hardcover_sync.sync_wishlist", _sync)
monkeypatch.setattr("shelfmark.core.auto_download.auto_download_pending", _auto)
return recorded
def test_scheduled_run_skips_sync_when_disabled(self, calls):
user_db, queue_release = object(), object()
scheduler.configure(user_db, queue_release, "/data/users.db")
result = scheduler.run_once()
assert result == {"status": "ok", "sync": None, "auto_download": {"queued": 1}}
assert calls["sync"] == []
assert calls["auto"] == [(user_db, {"queue_release": queue_release})]
def test_forced_run_syncs_regardless_of_toggle(self, calls):
user_db = object()
scheduler.configure(user_db, object(), "/data/users.db")
result = scheduler.run_once(force=True)
assert result == {"status": "ok", "sync": {"added": 1}, "auto_download": {"queued": 1}}
assert calls["sync"] == [(user_db, {"db_path": "/data/users.db"})]
def test_scheduled_run_syncs_when_enabled(self, monkeypatch, calls):
_configure(monkeypatch, HARDCOVER_SYNC_ENABLED=True)
user_db = object()
scheduler.configure(user_db, object(), None)
result = scheduler.run_once()
assert result["sync"] == {"added": 1}
assert calls["sync"] == [(user_db, {"db_path": None})]
def test_lock_is_released_after_a_run(self, calls):
scheduler.configure(object(), object(), None)
scheduler.run_once()
assert scheduler._run_lock.locked() is False
class TestTriggerAsync:
def test_returns_false_while_a_run_is_active(self, monkeypatch):
def _unexpected(**_kwargs):
raise AssertionError("run_once must not be scheduled")
monkeypatch.setattr(scheduler, "run_once", _unexpected)
with scheduler._run_lock:
assert scheduler.trigger_async() is False
@pytest.mark.parametrize("force", [True, False])
def test_runs_once_in_the_background_with_force_flag(self, monkeypatch, force):
finished = threading.Event()
seen = []
def _run_once(**kwargs):
seen.append(kwargs)
finished.set()
return {"status": "ok"}
monkeypatch.setattr(scheduler, "run_once", _run_once)
assert scheduler.trigger_async(force=force) is True
assert finished.wait(timeout=5)
assert seen == [{"force": force}]
+295
View File
@@ -0,0 +1,295 @@
"""Tests for syncing Hardcover status shelves into pending requests."""
import sqlite3
from types import SimpleNamespace
import pytest
from shelfmark.core import hardcover_sync
from shelfmark.core.user_db import UserDB
from shelfmark.core.utils import transform_cover_url
from shelfmark.metadata_providers import BookMetadata
class _Config:
def __init__(self, **values):
self.values = values
def get(self, key, default=None, user_id=None):
return self.values.get(key, default)
class _Provider:
"""Stub exposing the private paging method ``sync_wishlist`` relies on."""
def __init__(self, pages_by_status):
self.pages_by_status = pages_by_status
self.calls = []
def _fetch_current_user_books_by_status(self, status_id, page, limit):
self.calls.append((status_id, page, limit))
pages = self.pages_by_status.get(status_id, [])
books = pages[page - 1] if page <= len(pages) else []
return SimpleNamespace(books=books, has_more=page < len(pages))
def _book(provider_id, title, author="Matt Dinniman", **overrides):
fields = {
"provider": "hardcover",
"provider_id": provider_id,
"title": title,
"authors": [author],
}
fields.update(overrides)
return BookMetadata(**fields)
def _insert_history(db_path, *, title, author):
conn = sqlite3.connect(db_path)
try:
conn.execute(
"INSERT INTO download_history (task_id, source, title, author, final_status) "
"VALUES (?, ?, ?, ?, ?)",
(f"task-{title}", "prowlarr", title, author, "complete"),
)
conn.commit()
finally:
conn.close()
@pytest.fixture
def db_path(tmp_path):
return str(tmp_path / "users.db")
@pytest.fixture
def user_db(db_path):
db = UserDB(db_path)
db.initialize()
return db
def _configure(monkeypatch, **values):
monkeypatch.setattr(hardcover_sync, "app_config", _Config(**values))
@pytest.mark.parametrize(
("values", "expected"),
[
({"HARDCOVER_SYNC_TOKEN": " sync-token ", "HARDCOVER_API_KEY": "api"}, "sync-token"),
({"HARDCOVER_SYNC_TOKEN": "", "HARDCOVER_API_KEY": "api-key"}, "api-key"),
({}, ""),
],
)
def test_configured_token_falls_back_to_provider_api_key(monkeypatch, values, expected):
_configure(monkeypatch, **values)
assert hardcover_sync._configured_token() == expected
@pytest.mark.parametrize(
("raw", "expected"),
[
("3", [3]),
(" 2 ", [2]),
(["1", "3"], [1, 3]),
((1, "x", None), [1]),
("garbage", [1]),
([], [1]),
(42, [1]),
],
)
def test_configured_statuses_parsing(monkeypatch, raw, expected):
_configure(monkeypatch, HARDCOVER_SYNC_STATUSES=raw)
assert hardcover_sync._configured_statuses() == expected
def test_configured_statuses_default(monkeypatch):
_configure(monkeypatch)
assert hardcover_sync._configured_statuses() == [1]
def test_book_to_book_data_includes_optional_fields_only_when_present():
full = _book(
7,
"Dungeon Crawler Carl",
cover_url="https://img.example/c.jpg",
publish_year=2020,
subtitle="Book One",
search_author="Dinniman",
)
sparse = BookMetadata(provider="hardcover", provider_id="8", title="Untitled")
assert hardcover_sync._book_to_book_data(full, "audiobook") == {
"title": "Dungeon Crawler Carl",
"author": "Dinniman",
"provider": "hardcover",
"provider_id": "7",
"content_type": "audiobook",
"preview": transform_cover_url("https://img.example/c.jpg", "hardcover_7"),
"year": 2020,
"subtitle": "Book One",
}
assert hardcover_sync._book_to_book_data(sparse, "ebook") == {
"title": "Untitled",
"author": "Unknown",
"provider": "hardcover",
"provider_id": "8",
"content_type": "ebook",
}
def test_existing_provider_ids_reads_provider_id_or_id(user_db):
user = user_db.create_user(username="reader", role="user")
for book_data in (
{"title": "A", "author": "X", "provider": "hardcover", "provider_id": 11},
{"title": "B", "author": "X", "provider": "hardcover", "id": "22"},
{"title": "C", "author": "X", "provider": "openlibrary", "id": "OL22W"},
{"title": "D", "author": "X", "provider": "googlebooks", "provider_id": 33},
{"title": "E", "author": "X", "provider": "manual"},
):
user_db.create_request(
user_id=user["id"],
content_type="audiobook",
request_level="book",
policy_mode="request_book",
book_data=book_data,
)
assert hardcover_sync._existing_provider_ids(user_db) == {"11", "22"}
def test_already_downloaded_matches_history_case_insensitively(user_db, db_path):
_insert_history(db_path, title="Dungeon Crawler Carl", author="Matt Dinniman")
assert hardcover_sync._already_downloaded(db_path, "dungeon crawler CARL", "MATT dinniman")
assert not hardcover_sync._already_downloaded(db_path, "Dungeon Crawler Carl", "Someone")
assert not hardcover_sync._already_downloaded(None, "Dungeon Crawler Carl", "Matt Dinniman")
def test_already_downloaded_fails_open_without_history_table(tmp_path):
missing = str(tmp_path / "empty.db")
assert hardcover_sync._already_downloaded(missing, "Anything", "Anyone") is False
class TestSyncWishlist:
@pytest.fixture(autouse=True)
def _defaults(self, monkeypatch):
_configure(monkeypatch, HARDCOVER_SYNC_STATUSES="1")
monkeypatch.setattr(
"shelfmark.core.library_index.is_in_library", lambda *_args, **_kwargs: False
)
def _use_provider(self, monkeypatch, provider):
monkeypatch.setattr(hardcover_sync, "_build_provider", lambda: provider)
def test_adds_new_requests_across_pages(self, user_db, db_path, monkeypatch):
user = user_db.create_user(username="reader", role="user")
provider = _Provider(
{
1: [
[_book(1, "Dungeon Crawler Carl", cover_url="https://img.example/1.jpg")],
[_book(2, "Carl's Doomsday Scenario", publish_year=2021)],
]
}
)
self._use_provider(monkeypatch, provider)
summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path, user_id=user["id"])
assert summary == {"added": 2, "skipped": 0, "in_library": 0, "errors": 0}
assert provider.calls == [(1, 1, 25), (1, 2, 25)]
rows = {row["book_data"]["provider_id"]: row for row in user_db.list_requests()}
assert set(rows) == {"1", "2"}
assert rows["1"]["status"] == "pending"
assert rows["1"]["content_type"] == "audiobook"
assert rows["1"]["book_data"]["preview"] == transform_cover_url(
"https://img.example/1.jpg", "hardcover_1"
)
assert rows["2"]["book_data"]["year"] == 2021
assert "preview" not in rows["2"]["book_data"]
def test_skips_known_downloaded_owned_and_duplicate_books(self, user_db, db_path, monkeypatch):
user = user_db.create_user(username="reader", role="user")
user_db.create_request(
user_id=user["id"],
content_type="audiobook",
request_level="book",
policy_mode="request_book",
book_data={
"title": "Already Requested",
"author": "Matt Dinniman",
"provider": "hardcover",
"provider_id": "10",
},
)
_insert_history(db_path, title="Already Downloaded", author="Matt Dinniman")
_configure(monkeypatch, HARDCOVER_SYNC_STATUSES="1")
monkeypatch.setattr(
"shelfmark.core.library_index.is_in_library",
lambda book, *_args, **_kwargs: book.title == "Already Owned",
)
provider = _Provider(
{
1: [
[
_book(10, "Already Requested"),
_book(11, "Already Downloaded"),
_book(12, "Already Owned"),
_book(13, "Brand New"),
_book(14, "Brand New"),
]
]
}
)
self._use_provider(monkeypatch, provider)
summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path, user_id=user["id"])
assert summary == {"added": 1, "skipped": 3, "in_library": 1, "errors": 0}
titles = sorted(row["book_data"]["title"] for row in user_db.list_requests())
assert titles == ["Already Requested", "Brand New"]
def test_unavailable_provider_counts_as_error(self, user_db, db_path, monkeypatch):
self._use_provider(monkeypatch, None)
summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path)
assert summary == {"added": 0, "skipped": 0, "in_library": 0, "errors": 1}
assert user_db.list_requests() == []
class TestResolveRequestOwner:
def test_returns_lowest_id_admin(self, user_db):
user_db.create_user(username="reader", role="user")
first_admin = user_db.create_user(username="ops", role="admin")
user_db.create_user(username="second", role="admin")
assert hardcover_sync.resolve_request_owner(user_db) == first_admin["id"]
def test_returns_none_without_admins(self, user_db):
user_db.create_user(username="reader", role="user")
assert hardcover_sync.resolve_request_owner(user_db) is None
def test_sync_without_admin_counts_an_error_and_creates_nothing(self, user_db, db_path):
user_db.create_user(username="reader", role="user")
summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path)
assert summary == {"added": 0, "skipped": 0, "in_library": 0, "errors": 1}
assert user_db.list_requests() == []
def test_sync_defaults_to_the_lowest_id_admin(self, user_db, db_path, monkeypatch):
user_db.create_user(username="reader", role="user")
admin = user_db.create_user(username="ops", role="admin")
provider = _Provider({1: [[_book(1, "Dungeon Crawler Carl")]]})
monkeypatch.setattr(hardcover_sync, "_build_provider", lambda: provider)
summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path)
assert summary["added"] == 1
assert user_db.list_requests()[0]["user_id"] == admin["id"]
+55
View File
@@ -117,3 +117,58 @@ def test_plan_receives_filters_and_user_id(monkeypatch: pytest.MonkeyPatch) -> N
"source_filters": None,
"user_id": 42,
}
def test_search_book_releases_preserves_source_order_and_aggregates(
monkeypatch: pytest.MonkeyPatch,
) -> None:
sources = {"a": _Source(result=["a1"]), "b": _Source(result=["b1", "b2"])}
monkeypatch.setattr("shelfmark.release_sources.get_source", lambda name: sources[name])
all_releases, by_source, errors = release_search.search_book_releases(
_book(), sources=["b", "a"], content_type="audiobook"
)
assert list(by_source) == ["b", "a"]
assert by_source == {"b": ["b1", "b2"], "a": ["a1"]}
assert all_releases == ["b1", "b2", "a1"]
assert errors == []
_book_arg, _plan, expand_search, content_type = sources["a"].calls[0]
assert expand_search is True
assert content_type == "audiobook"
def test_search_book_releases_collects_errors_and_keeps_healthy_results(
monkeypatch: pytest.MonkeyPatch,
) -> None:
sources = {
"down": _Source(error=SourceUnavailableError("offline")),
"ok": _Source(result=["r1"]),
}
def _get(name: str) -> Any:
if name not in sources:
msg = f"Unknown release source: {name}"
raise ValueError(msg)
return sources[name]
monkeypatch.setattr("shelfmark.release_sources.get_source", _get)
all_releases, by_source, errors = release_search.search_book_releases(
_book(), sources=["down", "ok", "nope"]
)
assert all_releases == ["r1"]
assert by_source == {"ok": ["r1"]}
assert errors == ["down: offline", "Unknown source: nope"]
def test_search_book_releases_with_no_sources_returns_empty(
monkeypatch: pytest.MonkeyPatch,
) -> None:
def _unexpected(_name: str) -> Any:
raise AssertionError("no source should be resolved")
monkeypatch.setattr("shelfmark.release_sources.get_source", _unexpected)
assert release_search.search_book_releases(_book(), sources=[]) == ([], {}, [])