diff --git a/docs/hardcover-sync.md b/docs/hardcover-sync.md new file mode 100644 index 00000000..3a42bf4c --- /dev/null +++ b/docs/hardcover-sync.md @@ -0,0 +1,79 @@ +# Hardcover Sync + +Configure in **Settings → Hardcover Sync**. + +Hardcover Sync pulls books from your [Hardcover](https://hardcover.app) reading shelves into Shelfmark as requests, optionally downloads them without an admin picking a release, and can skip books you already own in a library you have set up under [Library Check](library-check.md). Every part is opt-in and off by default; the tab does nothing until you enable it. + +## Capabilities + +### Shelf sync + +On a schedule, Shelfmark reads the selected Hardcover shelf (by default **Want to Read**) through the Hardcover metadata provider and creates a **pending** request for each book that is not already known. Requests carry Hardcover's metadata, including the cover, and are owned by the lowest-id admin account. + +A book is skipped when: + +- a request with the same Hardcover book id already exists, in any status +- a download with the same title and author is already in the download history +- the library check (below) says it is already in your library +- a pending request with the same title, author, and content type already exists + +### Automatic downloads + +When enabled, each pending request that came from Hardcover is searched across your release sources in the configured priority order. The first source that yields a **strict match** wins, and the best candidate from that source is queued through the normal request fulfilment path, so it appears in the activity view and download history exactly like a request an admin approved. + +A release is a strict match only when all of the following hold: + +- at least 85% of the significant title words appear in the release title +- the author's surname appears in the release title, indexer name, or author field +- the release is an audiobook: an audiobook format (`m4b`, `mp3`, or your configured audiobook formats), or "audiobook"/"unabridged" in the title. Ebook-only formats such as `epub` and `pdf` are rejected +- for torrents, seeders meet **Minimum seeders**; other protocols ignore this check + +Among matching releases from the winning source, `m4b` outranks `mp3`, then more seeders, then larger size. If no source produces a confident match the request stays **pending** for manual review; nothing is guessed. + +### Library check + +Both shelf sync and automatic downloads ask the libraries you have enabled under [Library Check](library-check.md) whether the book is already there, for the content type being requested, and skip it if so. There is no separate switch on this tab. A library that is not enabled, or that does not hold the requested content type, never causes a skip. + +The check **fails open**: a library that cannot be read is logged and the book is treated as not owned, so processing continues. + +## Settings + +| Key | Label | Default | Notes | +|-----|-------|---------|-------| +| `HARDCOVER_SYNC_ENABLED` | Enable scheduled sync | Off | Runs shelf sync on the interval below | +| `HARDCOVER_SYNC_TOKEN` | Hardcover API Token | — | Leave blank to reuse the Hardcover metadata provider's token | +| `HARDCOVER_SYNC_STATUSES` | Shelves to sync | Want to Read | Hardcover reading status to pull from | +| `HARDCOVER_SYNC_CONTENT_TYPE` | Request as | Audiobooks | Content type given to synced requests and targeted by automatic downloads | +| `HARDCOVER_SYNC_INTERVAL` | Sync interval | 6 | Combined with the unit below | +| `HARDCOVER_SYNC_INTERVAL_UNIT` | Interval unit | Hours | Minutes or hours. The effective interval never drops below 60 seconds | +| `AUTO_DOWNLOAD_ENABLED` | Enable automatic downloads | Off | Auto-approve and queue strict matches for Hardcover requests | +| `AUTO_DOWNLOAD_SOURCE_PRIORITY` | Source priority | All usable sources | Drag to order. Sources you disable here, or that are not configured, are skipped | +| `AUTO_DOWNLOAD_MIN_SEEDERS` | Minimum seeders (torrents) | 1 | Torrent releases below this are ignored | + +One action button sits at the bottom of the tab: + +- **Sync now** runs a sync and automatic-download pass immediately using the saved settings. Sync runs even when scheduled sync is off; automatic downloads still require their own toggle. + +## Request Flow + +1. Shelf sync creates a **pending** request per new Hardcover book, with cover, year, and subtitle where Hardcover has them. +2. If automatic downloads are enabled, the request is searched source by source and the first strict match is queued through the normal fulfil path. The request becomes **fulfilled** and its delivery state is tracked like any other. +3. If no source yields a confident match, the request stays **pending** in the admin queue for manual review, exactly as if a user had requested the book. + +Only requests whose provider is Hardcover are considered for automatic downloads, and requests that have already been dispatched are left alone. + +## Scheduler + +The scheduler starts with the app and waits **60 seconds** before its first cycle. Each cycle runs a shelf sync (when scheduled sync is enabled) followed by an automatic-download pass (when that is enabled), then sleeps for the configured interval. Intervals shorter than 60 seconds are raised to 60 seconds so a typo cannot hammer the Hardcover API. When neither toggle is on the scheduler idles. + +Only one run happens at a time. If a scheduled cycle is in progress, **Sync now** reports that a sync is already running. + +## Troubleshooting + +**Sync adds nothing and the log says "no Hardcover token configured"** — set **Hardcover API Token** on this tab, or configure the token on the Hardcover metadata provider. A sync without a usable token counts as an error and creates no requests. + +**Requests appear but nothing downloads** — check **Enable automatic downloads** is on, that **Source priority** contains at least one configured source that supports audiobooks, and look for `no strict match` lines in the log. Lowering **Minimum seeders** helps on quiet trackers; loosening the title or author match is deliberately not an option. + +**Books you own keep being added** — enable the library check for the format you request (see [Library Check](library-check.md)) and use its test button. Because the check fails open, a library that cannot be read only produces a log warning rather than blocking the sync. + +**Too many requests at once** — the first sync of a large shelf creates one request per book, and the per-user pending limit from **Users & Requests** does not apply to synced requests. Start with a shelf you are happy to see in the queue in full. diff --git a/docs/index.md b/docs/index.md index 44546333..95839cce 100644 --- a/docs/index.md +++ b/docs/index.md @@ -13,6 +13,7 @@ Use the guides below to set up the app, connect your library tools, and understa ## Core Guides - [Users & Requests](users-and-requests.md) +- [Hardcover Sync](hardcover-sync.md) - [Reverse Proxy](reverse-proxy.md) - [OIDC](oidc.md) - [API Access](api-access.md) diff --git a/shelfmark/config/hardcover_sync_settings.py b/shelfmark/config/hardcover_sync_settings.py new file mode 100644 index 00000000..372550d9 --- /dev/null +++ b/shelfmark/config/hardcover_sync_settings.py @@ -0,0 +1,169 @@ +"""Settings tab for Hardcover wishlist sync + automatic downloads.""" + +from __future__ import annotations + +from typing import Any + +from shelfmark.core.settings_registry import ( + ActionButton, + CheckboxField, + HeadingField, + NumberField, + OrderableListField, + PasswordField, + SelectField, + SettingsField, + register_settings, +) + +_CONTENT_TYPE_OPTIONS = [ + {"value": "audiobook", "label": "Audiobooks"}, + {"value": "ebook", "label": "Ebooks"}, +] + + +def _status_options() -> list[dict[str, str]]: + from shelfmark.metadata_providers.hardcover import HARDCOVER_STATUSES + + return [{"value": str(status["id"]), "label": status["label"]} for status in HARDCOVER_STATUSES] + + +def _audiobook_source_options() -> list[dict[str, Any]]: + """Orderable-list options: release sources that can return audiobooks.""" + from shelfmark.release_sources import list_available_sources + + options: list[dict[str, Any]] = [] + for src in list_available_sources(): + supported = src.get("supported_content_types") or ["ebook", "audiobook"] + if "audiobook" not in supported: + continue + enabled = bool(src.get("enabled")) + options.append( + { + "id": src["name"], + "label": src.get("display_name") or src["name"], + "description": None if enabled else "Source not configured / unavailable", + "isLocked": not enabled, + "disabledReason": None if enabled else "Source not configured / unavailable", + } + ) + return options + + +def _sync_now(current_values: dict[str, Any] | None = None) -> dict[str, Any]: + """Action-button callback: trigger an immediate sync + auto-download pass.""" + from shelfmark.core import hardcover_scheduler + + started = hardcover_scheduler.trigger_async(force=True) + if not started: + return {"success": False, "message": "A sync is already running. Try again shortly."} + return { + "success": True, + "message": "Sync started. New wishlist books will appear as requests; " + "matching audiobooks auto-download if enabled.", + } + + +@register_settings("hardcover_sync", "Hardcover Sync", icon="book", order=8) +def hardcover_sync_settings() -> list[SettingsField]: + """Configure Hardcover wishlist sync and automatic downloads.""" + return [ + HeadingField( + key="hardcover_sync_heading", + title="Hardcover Wishlist Sync", + description=( + "Automatically pull books from your Hardcover reading shelves into " + "Shelfmark as requests, on a schedule." + ), + ), + CheckboxField( + key="HARDCOVER_SYNC_ENABLED", + label="Enable scheduled sync", + description="Periodically sync the selected Hardcover shelves into requests.", + default=False, + ), + PasswordField( + key="HARDCOVER_SYNC_TOKEN", + label="Hardcover API Token", + description=( + "Bearer token for your Hardcover account. Leave blank to reuse the " + "token from the Hardcover metadata provider." + ), + placeholder="Reuses metadata provider token if blank", + ), + SelectField( + key="HARDCOVER_SYNC_STATUSES", + label="Shelves to sync", + description="Which Hardcover reading shelf to pull from.", + options=_status_options, + default="1", + ), + SelectField( + key="HARDCOVER_SYNC_CONTENT_TYPE", + label="Request as", + description="Content type assigned to synced requests and targeted for downloads.", + options=_CONTENT_TYPE_OPTIONS, + default="audiobook", + ), + NumberField( + key="HARDCOVER_SYNC_INTERVAL", + label="Sync interval", + description="How often the scheduled sync runs (minimum 1 minute).", + default=6, + min_value=1, + max_value=10000, + ), + SelectField( + key="HARDCOVER_SYNC_INTERVAL_UNIT", + label="Interval unit", + description="Whether the sync interval is measured in minutes or hours.", + options=[ + {"value": "minutes", "label": "Minutes"}, + {"value": "hours", "label": "Hours"}, + ], + default="hours", + ), + HeadingField( + key="auto_download_heading", + title="Automatic Downloads", + description=( + "When enabled, synced requests are auto-approved and downloaded from the " + "first source below that yields a strict title/author/format match. If no " + "confident match is found, the request is left pending for manual review." + ), + ), + CheckboxField( + key="AUTO_DOWNLOAD_ENABLED", + label="Enable automatic downloads", + description="Auto-approve and auto-download matching releases for synced requests.", + default=False, + ), + OrderableListField( + key="AUTO_DOWNLOAD_SOURCE_PRIORITY", + label="Source priority", + description=( + "Drag to set which release sources are tried first. The first source with " + "a strict match wins. Disabled sources are skipped." + ), + options=_audiobook_source_options, + # Default left empty so registration never imports release_sources (which would + # deadlock the registry lock). When empty, all available sources are used in + # registry order (see auto_download._configured_source_priority). + default=[], + ), + NumberField( + key="AUTO_DOWNLOAD_MIN_SEEDERS", + label="Minimum seeders (torrents)", + description="Skip torrent releases with fewer seeders than this.", + default=1, + min_value=0, + max_value=1000, + ), + ActionButton( + key="sync_now", + label="Sync now", + description="Run a sync + auto-download pass immediately (uses saved settings).", + style="primary", + callback=_sync_now, + ), + ] diff --git a/shelfmark/config/settings.py b/shelfmark/config/settings.py index 81119d2c..be173ee5 100644 --- a/shelfmark/config/settings.py +++ b/shelfmark/config/settings.py @@ -2167,3 +2167,8 @@ def advanced_settings() -> list[SettingsField]: register_on_save("advanced", _on_save_advanced) + + +# Register the Hardcover Sync settings tab. Imported here (rather than at each +# settings endpoint) so every code path that imports this module also registers it. +from shelfmark.config import hardcover_sync_settings as _hardcover_sync_settings # noqa: E402,F401 diff --git a/shelfmark/core/auto_download.py b/shelfmark/core/auto_download.py new file mode 100644 index 00000000..5c8fd69d --- /dev/null +++ b/shelfmark/core/auto_download.py @@ -0,0 +1,395 @@ +"""Automatic release selection + download for pending requests. + +Given a pending request (typically synced from a Hardcover wishlist), this module +searches release sources in a user-configured priority order, applies a strict +title/author/format match guard, picks the best candidate from the first source +that yields a match, and queues it via the normal request-fulfilment path. + +Design goals: +- **Strict by default.** Bias toward leaving a request pending (manual review) over + grabbing the wrong file. Title and author must both match; the format must be a + real audiobook format (no ebook-only fallbacks when targeting audiobooks). +- **Source priority is primary.** Walk the configured source order top-down and take + the first source that produces at least one strict match. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING, Any + +from shelfmark.core.config import config as app_config +from shelfmark.core.logger import setup_logger +from shelfmark.core.release_search import search_book_releases +from shelfmark.core.request_helpers import coerce_int +from shelfmark.core.text_match import ( + DEFAULT_TITLE_MATCH_THRESHOLD, + author_surname, + title_tokens_match, +) +from shelfmark.core.text_match import ( + tokens as _tokens, +) + +if TYPE_CHECKING: + from collections.abc import Callable + + from shelfmark.core.user_db import UserDB + from shelfmark.metadata_providers import BookMetadata + from shelfmark.release_sources import Release + +logger = setup_logger(__name__) + +# Fraction of significant book-title tokens that must appear in the release title. +TITLE_MATCH_THRESHOLD = DEFAULT_TITLE_MATCH_THRESHOLD + +DEFAULT_AUDIOBOOK_FORMATS = ("m4b", "mp3") +EBOOK_FORMAT_MARKERS = ( + "epub", + "mobi", + "azw3", + "azw", + "pdf", + "fb2", + "djvu", + "cbz", + "cbr", +) +# Audiobook signals we recognise in free-text release titles in addition to formats. +AUDIOBOOK_TITLE_MARKERS = ("audiobook", "unabridged", "m4b", "audio book") + +# Audiobook format ranking for tie-breaking within a single source. +_FORMAT_RANK = {"m4b": 3, "m4a": 2, "mp3": 1} + + +@dataclass(frozen=True) +class AutoDownloadOutcome: + """Result of attempting to auto-download a single request.""" + + request_id: int + status: str # "queued" | "no_match" | "skipped" | "error" + detail: str = "" + source: str | None = None + + +def _audiobook_formats() -> set[str]: + configured = app_config.get("SUPPORTED_AUDIOBOOK_FORMATS", list(DEFAULT_AUDIOBOOK_FORMATS)) + values: list[object] + if isinstance(configured, str): + values = [configured] + elif isinstance(configured, (list, tuple, set)): + values = list(configured) + else: + values = [] + formats = {str(fmt).strip().lower() for fmt in values if str(fmt).strip()} + return formats or set(DEFAULT_AUDIOBOOK_FORMATS) + + +def _author_surname_tokens(book: BookMetadata) -> list[str]: + """Return distinctive author tokens (prefer the surname of the primary author).""" + author = book.search_author or (book.authors[0] if book.authors else "") + surname = author_surname(author) + return [surname] if surname else [] + + +def _title_match(book: BookMetadata, release_title: str) -> bool: + return title_tokens_match( + book.search_title or book.title, + set(_tokens(release_title)), + TITLE_MATCH_THRESHOLD, + ) + + +def _author_match(book: BookMetadata, release: Release) -> bool: + surname = _author_surname_tokens(book) + if not surname: + return True # No author metadata to verify against; don't block on it. + haystack = set(_tokens(release.title)) | set(_tokens(release.indexer)) + extra_author = release.extra.get("author") if isinstance(release.extra, dict) else None + haystack |= set(_tokens(extra_author if isinstance(extra_author, str) else None)) + return all(tok in haystack for tok in surname) + + +def _format_match(release: Release, audiobook_formats: set[str]) -> bool: + """Require a real audiobook signal and reject ebook-only releases.""" + fmt = (release.format or "").strip().lower() + title_l = (release.title or "").lower() + + has_audiobook_signal = ( + fmt in audiobook_formats + or any(marker in title_l for marker in AUDIOBOOK_TITLE_MARKERS) + or any(f".{af}" in title_l or f" {af}" in title_l for af in audiobook_formats) + ) + if not has_audiobook_signal: + return False + + # Reject things that are clearly an ebook and nothing else. + return fmt not in EBOOK_FORMAT_MARKERS + + +def _seeders_ok(release: Release, min_seeders: int) -> bool: + from shelfmark.release_sources import ReleaseProtocol + + if release.protocol != ReleaseProtocol.TORRENT: + return True # Non-torrent protocols have no seeder concept. + if release.seeders is None: + return min_seeders <= 0 + return release.seeders >= min_seeders + + +def strict_match( + release: Release, + book: BookMetadata, + *, + min_seeders: int = 1, + audiobook_formats: set[str] | None = None, +) -> bool: + """Return True only if the release confidently matches the requested audiobook.""" + formats = audiobook_formats if audiobook_formats is not None else _audiobook_formats() + return ( + _title_match(book, release.title) + and _author_match(book, release) + and _format_match(release, formats) + and _seeders_ok(release, min_seeders) + ) + + +def _release_sort_key(release: Release) -> tuple[int, int, int]: + fmt = (release.format or "").strip().lower() + if fmt not in _FORMAT_RANK and "m4b" in (release.title or "").lower(): + fmt = "m4b" + return ( + _FORMAT_RANK.get(fmt, 0), + release.seeders or 0, + release.size_bytes or 0, + ) + + +def pick_best_release(releases: list[Release]) -> Release | None: + if not releases: + return None + return max(releases, key=_release_sort_key) + + +def build_release_data(release: Release, book: BookMetadata, content_type: str) -> dict[str, Any]: + """Build a queue_release-compatible payload from a chosen Release.""" + author = book.search_author or (book.authors[0] if book.authors else None) + extra = release.extra if isinstance(release.extra, dict) else {} + # Carry the book's cover so the download shows artwork immediately; the orchestrator + # reads release_data["preview"] and proxies it (orchestrator.queue_release / task_to_dict). + preview = book.cover_url or extra.get("preview") + payload: dict[str, Any] = { + "source": release.source, + "source_id": release.source_id, + "title": release.title, + "author": author, + "year": book.publish_year, + "preview": preview, + "format": release.format, + "size": release.size, + "size_bytes": release.size_bytes, + "download_url": release.download_url, + "info_url": release.info_url, + "protocol": release.protocol.value if release.protocol else None, + "indexer": release.indexer, + "seeders": release.seeders, + "language": release.language, + "content_type": release.content_type or content_type, + "series_name": book.series_name, + "series_position": book.series_position, + "subtitle": book.subtitle, + "extra": dict(extra), + } + return {key: value for key, value in payload.items() if value is not None} + + +def _configured_source_priority() -> list[str]: + """Return enabled release-source names in configured priority order.""" + from shelfmark.release_sources import list_available_sources + + available = list_available_sources() + available_by_name = {src["name"]: src for src in available} + + raw = app_config.get("AUTO_DOWNLOAD_SOURCE_PRIORITY", []) + ordered: list[str] = [] + configured = isinstance(raw, list) and any(isinstance(item, dict) for item in raw) + if isinstance(raw, list): + for item in raw: + if not isinstance(item, dict): + continue + name = item.get("id") + if not isinstance(name, str) or name not in available_by_name: + continue + if not bool(item.get("enabled", True)): + continue + if not available_by_name[name].get("enabled", False): + continue # Source itself not usable (unconfigured/unavailable). + ordered.append(name) + + if ordered: + return ordered + if configured: + # A priority list was saved but nothing in it is usable. Falling back to every + # source would search the ones the user turned off. + logger.warning( + "auto-download: every source in the priority list is disabled or unavailable" + ) + return [] + # Nothing configured yet: every usable source, registry order. + return [src["name"] for src in available if src.get("enabled")] + + +def auto_download_request( + user_db: UserDB, + request_row: dict[str, Any], + *, + sources: list[str], + content_type: str, + min_seeders: int, + queue_release: Callable[..., tuple[bool, str | None]], + admin_user_id: int, +) -> AutoDownloadOutcome: + """Search, strict-match, and queue a single pending request.""" + from shelfmark.core.requests_service import RequestServiceError, fulfil_request + from shelfmark.metadata_providers import ( + get_provider, + get_provider_kwargs, + is_provider_registered, + ) + + request_id = int(request_row["id"]) + book_data = request_row.get("book_data") or {} + if not isinstance(book_data, dict): + return AutoDownloadOutcome(request_id, "skipped", "no book_data") + + provider_name = book_data.get("provider") + provider_id = book_data.get("provider_id") or book_data.get("id") + if not provider_name or not provider_id or not is_provider_registered(provider_name): + return AutoDownloadOutcome(request_id, "skipped", "no usable provider/id") + + try: + prov = get_provider(provider_name, **get_provider_kwargs(provider_name)) + book = prov.get_book(str(provider_id)) + except Exception as exc: # noqa: BLE001 - provider/network errors shouldn't kill the loop + logger.warning("auto-download: metadata lookup failed for request %s: %s", request_id, exc) + return AutoDownloadOutcome(request_id, "error", f"metadata lookup failed: {exc}") + + if book is None: + return AutoDownloadOutcome(request_id, "skipped", "book not found in provider") + + # Final guard: skip if the book is already in a library that is checked for this content type. + from shelfmark.core.library_index import is_in_library + + if is_in_library(book, content_type): + logger.info( + "auto-download: request %s (%s) already in library; skipping", + request_id, + book.title, + ) + return AutoDownloadOutcome(request_id, "in_library", "already in library") + + audiobook_formats = _audiobook_formats() + + # Walk sources in priority order; take the first source with a strict match. + for source_name in sources: + _all, by_source, _errors = search_book_releases( + book, + sources=[source_name], + content_type=content_type, + expand_search=True, + ) + candidates = [ + release + for release in by_source.get(source_name, []) + if strict_match( + release, + book, + min_seeders=min_seeders, + audiobook_formats=audiobook_formats, + ) + ] + chosen = pick_best_release(candidates) + if chosen is None: + continue + + release_data = build_release_data(chosen, book, content_type) + try: + fulfil_request( + user_db, + request_id=request_id, + admin_user_id=admin_user_id, + queue_release=queue_release, + release_data=release_data, + ) + except RequestServiceError as exc: + logger.warning("auto-download: fulfil failed for request %s: %s", request_id, exc) + return AutoDownloadOutcome(request_id, "error", str(exc), source=source_name) + logger.info( + "auto-download: queued request %s from %s (%s)", + request_id, + source_name, + chosen.title, + ) + return AutoDownloadOutcome(request_id, "queued", chosen.title, source=source_name) + + logger.info("auto-download: no strict match for request %s (%s)", request_id, book.title) + return AutoDownloadOutcome(request_id, "no_match", "no strict match across sources") + + +def auto_download_pending( + user_db: UserDB, + *, + queue_release: Callable[..., tuple[bool, str | None]], + provider_filter: str | None = "hardcover", + admin_user_id: int | None = None, +) -> dict[str, int]: + """Run the auto-download pass over all eligible pending requests. + + Returns a summary count dict. No-ops (returns zeros) unless AUTO_DOWNLOAD_ENABLED. + ``admin_user_id`` defaults to the lowest-id admin, who fulfils the requests. + """ + if not bool(app_config.get("AUTO_DOWNLOAD_ENABLED", False)): + return {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0} + + if admin_user_id is None: + from shelfmark.core.hardcover_sync import resolve_request_owner + + admin_user_id = resolve_request_owner(user_db) + if admin_user_id is None: + logger.warning("auto-download: no admin user exists to fulfil requests") + return {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0} + + content_type = str(app_config.get("HARDCOVER_SYNC_CONTENT_TYPE", "audiobook") or "audiobook") + min_seeders = coerce_int(app_config.get("AUTO_DOWNLOAD_MIN_SEEDERS", 1), 1) + + sources = _configured_source_priority() + if not sources: + logger.warning("auto-download: no usable release sources configured") + return {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0} + + pending = user_db.list_requests(status="pending") + summary = {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0} + + for row in pending: + book_data = row.get("book_data") or {} + if provider_filter and ( + not isinstance(book_data, dict) or book_data.get("provider") != provider_filter + ): + continue + # Only act on requests that haven't already been dispatched. + if str(row.get("delivery_state") or "none").lower() not in {"none", ""}: + continue + + outcome = auto_download_request( + user_db, + row, + sources=sources, + content_type=content_type, + min_seeders=min_seeders, + queue_release=queue_release, + admin_user_id=admin_user_id, + ) + summary[outcome.status] = summary.get(outcome.status, 0) + 1 + + logger.info("auto-download pass complete: %s", summary) + return summary diff --git a/shelfmark/core/hardcover_scheduler.py b/shelfmark/core/hardcover_scheduler.py new file mode 100644 index 00000000..d84ea876 --- /dev/null +++ b/shelfmark/core/hardcover_scheduler.py @@ -0,0 +1,138 @@ +"""Background scheduler for Hardcover wishlist sync + auto-download. + +Runs an in-process daemon thread (mirroring the download coordinator pattern in +``shelfmark.download.orchestrator``) that periodically syncs configured Hardcover +shelves into requests and auto-downloads them. Also exposes a manual trigger used +by the "Sync now" settings action button. +""" + +from __future__ import annotations + +import threading +import time +from typing import TYPE_CHECKING, Any + +from shelfmark.core.config import config as app_config +from shelfmark.core.logger import setup_logger + +if TYPE_CHECKING: + from collections.abc import Callable + + from shelfmark.core.user_db import UserDB + +logger = setup_logger(__name__) + +_WARMUP_SECONDS = 60 # Delay before the first cycle after startup. +_MIN_INTERVAL_SECONDS = 60 # Floor so a misconfigured interval can't hammer the API. +_UNIT_SECONDS = {"minutes": 60.0, "hours": 3600.0} + +_ctx_lock = threading.Lock() +_ctx: dict[str, Any] = {} + +_thread: threading.Thread | None = None +_thread_lock = threading.Lock() + +# Prevents a scheduled cycle and a manual "Sync now" from overlapping. +_run_lock = threading.Lock() + + +def configure( + user_db: UserDB, + queue_release: Callable[..., tuple[bool, str | None]], + db_path: str | None, +) -> None: + """Store the runtime context the background work needs.""" + with _ctx_lock: + _ctx["user_db"] = user_db + _ctx["queue_release"] = queue_release + _ctx["db_path"] = db_path + + +def _interval_seconds() -> float: + raw = app_config.get("HARDCOVER_SYNC_INTERVAL", 6) + try: + value = float(raw) if isinstance(raw, (int, float, str)) else 6.0 + except ValueError: + value = 6.0 + unit = str(app_config.get("HARDCOVER_SYNC_INTERVAL_UNIT", "hours") or "hours").strip().lower() + multiplier = _UNIT_SECONDS.get(unit, _UNIT_SECONDS["hours"]) + return max(value * multiplier, _MIN_INTERVAL_SECONDS) + + +def _enabled() -> bool: + return bool(app_config.get("HARDCOVER_SYNC_ENABLED", False)) or bool( + app_config.get("AUTO_DOWNLOAD_ENABLED", False) + ) + + +def run_once(*, force: bool = False) -> dict[str, Any]: + """Run one sync + auto-download pass. Returns a combined summary. + + ``force=True`` (the manual "Sync now" path) syncs regardless of the scheduler + toggle; auto-download always remains gated by ``AUTO_DOWNLOAD_ENABLED`` inside + ``auto_download_pending``. Skips (returns ``{"status": "busy"}``) if another run + is in progress. + """ + if not _run_lock.acquire(blocking=False): + return {"status": "busy"} + try: + with _ctx_lock: + user_db = _ctx.get("user_db") + queue_release = _ctx.get("queue_release") + db_path = _ctx.get("db_path") + + if user_db is None or queue_release is None: + return {"status": "unconfigured"} + + from shelfmark.core.auto_download import auto_download_pending + from shelfmark.core.hardcover_sync import sync_wishlist + + sync_summary: dict[str, Any] | None = None + if force or bool(app_config.get("HARDCOVER_SYNC_ENABLED", False)): + sync_summary = sync_wishlist(user_db, db_path=db_path) + auto_summary = auto_download_pending(user_db, queue_release=queue_release) + return {"status": "ok", "sync": sync_summary, "auto_download": auto_summary} + finally: + _run_lock.release() + + +def trigger_async(*, force: bool = True) -> bool: + """Kick off a one-off run in a background thread. Returns False if one is active.""" + if _run_lock.locked(): + return False + thread = threading.Thread( + target=lambda: run_once(force=force), daemon=True, name="HardcoverSyncManual" + ) + thread.start() + return True + + +def _scheduler_loop() -> None: + logger.info("Hardcover scheduler started") + time.sleep(_WARMUP_SECONDS) + while True: + try: + if _enabled(): + logger.info("Hardcover scheduler: starting cycle") + result = run_once() + logger.info("Hardcover scheduler: cycle result %s", result) + except Exception: + logger.exception("Hardcover scheduler: cycle failed") + time.sleep(_interval_seconds()) + + +def start( + user_db: UserDB, + queue_release: Callable[..., tuple[bool, str | None]], + db_path: str | None, +) -> None: + """Configure context and start the scheduler thread (idempotent).""" + configure(user_db, queue_release, db_path) + global _thread + with _thread_lock: + if _thread is not None and _thread.is_alive(): + logger.debug("Hardcover scheduler already running") + return + _thread = threading.Thread(target=_scheduler_loop, daemon=True, name="HardcoverScheduler") + _thread.start() + logger.info("Hardcover scheduler thread launched") diff --git a/shelfmark/core/hardcover_sync.py b/shelfmark/core/hardcover_sync.py new file mode 100644 index 00000000..e7a4cfc7 --- /dev/null +++ b/shelfmark/core/hardcover_sync.py @@ -0,0 +1,225 @@ +"""Sync a Hardcover reading shelf (e.g. "Want to Read") into Shelfmark requests. + +This replaces the standalone ``scripts/sync_hardcover_wishlist.py`` with an in-app +service that reuses the registered Hardcover metadata provider (so synced requests +carry full metadata incl. covers) and the request-service validation/dedup path. +""" + +from __future__ import annotations + +import sqlite3 +from typing import TYPE_CHECKING, Any + +from shelfmark.core.config import config as app_config +from shelfmark.core.logger import setup_logger +from shelfmark.core.utils import transform_cover_url + +if TYPE_CHECKING: + from shelfmark.core.user_db import UserDB + from shelfmark.metadata_providers import BookMetadata + +logger = setup_logger(__name__) + +# Hardcover status_id -> shelf. Default sync target is "Want to Read" (1). +DEFAULT_SYNC_STATUSES = ("1",) +_PAGE_LIMIT = 25 # Hardcover API page size. +_MAX_PAGES = 40 # Safety bound (~1000 books) so a bad response can't loop forever. + + +def _configured_token() -> str: + """Prefer a dedicated sync token, fall back to the provider API key.""" + token = app_config.get("HARDCOVER_SYNC_TOKEN", "") or app_config.get("HARDCOVER_API_KEY", "") + return str(token or "").strip() + + +def _configured_statuses() -> list[int]: + raw = app_config.get("HARDCOVER_SYNC_STATUSES", list(DEFAULT_SYNC_STATUSES)) + values: list[object] + if isinstance(raw, str): + values = [raw] + elif isinstance(raw, (list, tuple)): + values = list(raw) + else: + values = [] + statuses: list[int] = [] + for value in values: + try: + statuses.append(int(str(value).strip())) + except ValueError: + continue + return statuses or [int(s) for s in DEFAULT_SYNC_STATUSES] + + +def _build_provider() -> Any | None: + from shelfmark.metadata_providers import get_provider + + token = _configured_token() + if not token: + logger.warning("hardcover-sync: no Hardcover token configured") + return None + provider = get_provider("hardcover", api_key=token) + if not provider.is_available(): + logger.warning("hardcover-sync: Hardcover provider unavailable (bad token?)") + return None + return provider + + +def resolve_request_owner(user_db: UserDB) -> int | None: + """Return the lowest-id admin, who owns synced requests and approves auto-downloads.""" + admin = user_db.get_first_admin() + return int(admin["id"]) if admin else None + + +def _primary_author(book: BookMetadata) -> str: + if book.search_author: + return book.search_author + if book.authors: + return book.authors[0] + return "Unknown" + + +def _book_to_book_data(book: BookMetadata, content_type: str) -> dict[str, Any]: + book_data: dict[str, Any] = { + "title": book.title, + "author": _primary_author(book), + "provider": "hardcover", + "provider_id": str(book.provider_id), + "content_type": content_type, + } + # Proxy the cover the same way search results are, so the row is ready for the UI. + if book.cover_url: + book_data["preview"] = transform_cover_url(book.cover_url, f"hardcover_{book.provider_id}") + if book.publish_year: + book_data["year"] = book.publish_year + if book.subtitle: + book_data["subtitle"] = book.subtitle + return book_data + + +def _existing_provider_ids(user_db: UserDB) -> set[str]: + """Hardcover ids already requested, in any status. Other providers' ids are not counted.""" + ids: set[str] = set() + for row in user_db.list_requests(): + book_data = row.get("book_data") or {} + if isinstance(book_data, dict) and book_data.get("provider") == "hardcover": + pid = book_data.get("provider_id") or book_data.get("id") + if pid is not None: + ids.add(str(pid)) + return ids + + +def _already_downloaded(db_path: str | None, title: str, author: str) -> bool: + """Best-effort check that a title/author isn't already in download_history.""" + if not db_path: + return False + try: + conn = sqlite3.connect(db_path) + try: + row = conn.execute( + "SELECT 1 FROM download_history " + "WHERE LOWER(title) = ? AND LOWER(author) = ? LIMIT 1", + (title.strip().lower(), author.strip().lower()), + ).fetchone() + return row is not None + finally: + conn.close() + except sqlite3.Error as exc: + logger.debug("hardcover-sync: history dedup query failed: %s", exc) + return False + + +def _fetch_status_books(provider: Any, status_id: int) -> list[BookMetadata]: + """Page through a Hardcover status shelf, returning all books.""" + books: list[BookMetadata] = [] + for page in range(1, _MAX_PAGES + 1): + try: + result = provider._fetch_current_user_books_by_status(status_id, page, _PAGE_LIMIT) + except Exception as exc: # noqa: BLE001 - network/provider errors shouldn't abort the sync + logger.warning( + "hardcover-sync: fetch failed (status=%s page=%s): %s", status_id, page, exc + ) + break + page_books = list(result.books or []) + books.extend(page_books) + if not result.has_more or not page_books: + break + return books + + +def sync_wishlist( + user_db: UserDB, + *, + db_path: str | None = None, + user_id: int | None = None, +) -> dict[str, int]: + """Sync configured Hardcover shelves into pending requests. + + Returns a summary dict: ``{"added", "skipped", "in_library", "errors"}``. Requires a + configured token; enable-gating is the caller's responsibility (see hardcover_scheduler). + """ + summary = {"added": 0, "skipped": 0, "in_library": 0, "errors": 0} + + if user_id is None: + user_id = resolve_request_owner(user_db) + if user_id is None: + logger.warning("hardcover-sync: no admin user exists to own synced requests") + summary["errors"] += 1 + return summary + + provider = _build_provider() + if provider is None: + summary["errors"] += 1 + return summary + + content_type = str(app_config.get("HARDCOVER_SYNC_CONTENT_TYPE", "audiobook") or "audiobook") + known_provider_ids = _existing_provider_ids(user_db) + + from shelfmark.core.library_index import is_in_library + from shelfmark.core.requests_service import RequestServiceError, create_request + + for status_id in _configured_statuses(): + for book in _fetch_status_books(provider, status_id): + provider_id = str(book.provider_id) + author = _primary_author(book) + + if provider_id in known_provider_ids: + summary["skipped"] += 1 + continue + if _already_downloaded(db_path, book.title, author): + summary["skipped"] += 1 + continue + if is_in_library(book, content_type): + summary["in_library"] += 1 + logger.info("hardcover-sync: '%s' already in your library; skipping", book.title) + continue + + try: + create_request( + user_db, + user_id=user_id, + source_hint=None, + content_type=content_type, + request_level="book", + policy_mode="request_book", + book_data=_book_to_book_data(book, content_type), + note=None, + ) + except RequestServiceError as exc: + # Duplicate / max-pending / validation: treat as skip, not failure. + if exc.code in {"duplicate_pending_request", "max_pending_reached"}: + summary["skipped"] += 1 + else: + logger.warning("hardcover-sync: could not add '%s': %s", book.title, exc) + summary["errors"] += 1 + continue + except Exception: + logger.exception("hardcover-sync: unexpected error adding '%s'", book.title) + summary["errors"] += 1 + continue + + known_provider_ids.add(provider_id) + summary["added"] += 1 + logger.info("hardcover-sync: added request for '%s' by %s", book.title, author) + + logger.info("hardcover-sync complete: %s", summary) + return summary diff --git a/shelfmark/core/release_search.py b/shelfmark/core/release_search.py index be082e25..271e5f52 100644 --- a/shelfmark/core/release_search.py +++ b/shelfmark/core/release_search.py @@ -92,3 +92,41 @@ def search_source_releases( return None, [], f"{source_name}: {exc!s}" else: return source, releases, None + + +def search_book_releases( + book: BookMetadata, + *, + sources: list[str], + content_type: str = "ebook", + expand_search: bool = True, + languages: list[str] | None = None, + indexers: list[str] | None = None, +) -> tuple[list[Release], dict[str, list[Release]], list[str]]: + """Search several sources for releases of ``book``. + + Returns ``(all_releases, releases_by_source, errors)``. ``releases_by_source`` + preserves the requested ``sources`` order so callers can prioritise. + """ + all_releases: list[Release] = [] + by_source: dict[str, list[Release]] = {} + errors: list[str] = [] + + for source_name in sources: + source, releases, error = search_source_releases( + source_name, + book, + languages=languages, + manual_query=None, + indexers=indexers, + expand_search=expand_search, + content_type=content_type, + source_filters=None, + ) + if source is not None: + by_source[source_name] = list(releases) + all_releases.extend(releases) + if error is not None: + errors.append(error) + + return all_releases, by_source, errors diff --git a/shelfmark/main.py b/shelfmark/main.py index 2403b735..79b30484 100644 --- a/shelfmark/main.py +++ b/shelfmark/main.py @@ -291,6 +291,15 @@ if user_db is not None and download_history_service is not None: # Start download coordinator backend.start() +# Start the Hardcover wishlist sync + auto-download scheduler (in-process daemon). +if user_db is not None: + try: + from shelfmark.core import hardcover_scheduler + + hardcover_scheduler.start(user_db, backend.queue_release, _user_db_path) + except (ImportError, RuntimeError, OSError) as exc: + logger.warning("Failed to start Hardcover scheduler: %s", exc) + # Pre-solve the direct-download source's protection challenge in the background so the # first user search does not pay for a cold Chrome bypass. Never blocks startup. warmup.start() diff --git a/tests/config/test_hardcover_sync_settings.py b/tests/config/test_hardcover_sync_settings.py new file mode 100644 index 00000000..da341d22 --- /dev/null +++ b/tests/config/test_hardcover_sync_settings.py @@ -0,0 +1,71 @@ +"""Tests for the Hardcover Sync settings tab registration and action buttons.""" + +import shelfmark.config.hardcover_sync_settings # noqa: F401 - registers the tab +from shelfmark.core import settings_registry + + +def _field_map(): + tab = settings_registry.get_settings_tab("hardcover_sync") + assert tab is not None + return {field.key: field for field in tab.fields if hasattr(field, "key")} + + +def test_tab_is_registered(): + tab = settings_registry.get_settings_tab("hardcover_sync") + + assert tab is not None + assert tab.display_name == "Hardcover Sync" + + +def test_everything_is_off_by_default(): + fields = _field_map() + + assert fields["HARDCOVER_SYNC_ENABLED"].default is False + assert fields["AUTO_DOWNLOAD_ENABLED"].default is False + + +def test_schedule_and_content_type_defaults(): + fields = _field_map() + + assert fields["HARDCOVER_SYNC_INTERVAL"].default == 6 + assert fields["HARDCOVER_SYNC_INTERVAL"].min_value == 1 + assert fields["HARDCOVER_SYNC_INTERVAL_UNIT"].default == "hours" + assert fields["HARDCOVER_SYNC_CONTENT_TYPE"].default == "audiobook" + assert fields["HARDCOVER_SYNC_STATUSES"].default == "1" + assert fields["AUTO_DOWNLOAD_SOURCE_PRIORITY"].default == [] + + +def test_shelf_options_come_from_hardcover_statuses(): + options = _field_map()["HARDCOVER_SYNC_STATUSES"].options() + + assert {"value": "1", "label": "Want to Read"} in options + + +def test_action_buttons_are_registered(): + fields = _field_map() + + assert fields["sync_now"].callback is not None + + +def test_sync_now_reports_busy_when_a_run_is_active(monkeypatch): + monkeypatch.setattr("shelfmark.core.hardcover_scheduler.trigger_async", lambda **_: False) + + result = settings_registry.execute_action("hardcover_sync", "sync_now") + + assert result["success"] is False + assert "already running" in result["message"] + + +def test_sync_now_starts_a_forced_run(monkeypatch): + seen = [] + + def _trigger(**kwargs): + seen.append(kwargs) + return True + + monkeypatch.setattr("shelfmark.core.hardcover_scheduler.trigger_async", _trigger) + + result = settings_registry.execute_action("hardcover_sync", "sync_now") + + assert result["success"] is True + assert seen == [{"force": True}] diff --git a/tests/core/test_auto_download.py b/tests/core/test_auto_download.py new file mode 100644 index 00000000..9ca74f6d --- /dev/null +++ b/tests/core/test_auto_download.py @@ -0,0 +1,401 @@ +"""Tests for strict release matching and auto-download of pending requests.""" + +import pytest + +from shelfmark.core import auto_download +from shelfmark.core.user_db import UserDB +from shelfmark.metadata_providers import BookMetadata +from shelfmark.release_sources import Release, ReleaseProtocol + +AUDIOBOOK_FORMATS = {"m4b", "mp3"} + + +class _Config: + def __init__(self, **values): + self.values = values + + def get(self, key, default=None, user_id=None): + return self.values.get(key, default) + + +def _book(**overrides): + fields = { + "provider": "hardcover", + "provider_id": "42", + "title": "Dungeon Crawler Carl", + "authors": ["Matt Dinniman"], + } + fields.update(overrides) + return BookMetadata(**fields) + + +def _release(**overrides): + fields = { + "source": "prowlarr", + "source_id": "rel-1", + "title": "Dungeon Crawler Carl - Matt Dinniman [M4B]", + "format": "m4b", + "protocol": ReleaseProtocol.TORRENT, + "seeders": 10, + } + fields.update(overrides) + return Release(**fields) + + +def _strict(release, **kwargs): + kwargs.setdefault("audiobook_formats", AUDIOBOOK_FORMATS) + return auto_download.strict_match(release, _book(), **kwargs) + + +@pytest.fixture +def user_db(tmp_path): + db = UserDB(str(tmp_path / "users.db")) + db.initialize() + return db + + +def _pending_request(user_db, user_id, **book_data_overrides): + book_data = { + "title": "Dungeon Crawler Carl", + "author": "Matt Dinniman", + "provider": "hardcover", + "provider_id": "42", + "content_type": "audiobook", + } + book_data.update(book_data_overrides) + return user_db.create_request( + user_id=user_id, + content_type="audiobook", + request_level="book", + policy_mode="request_book", + book_data=book_data, + ) + + +def _stub_provider(monkeypatch, book): + class _Provider: + def get_book(self, book_id): + assert book_id == "42" + return book + + monkeypatch.setattr( + "shelfmark.metadata_providers.is_provider_registered", lambda name: name == "hardcover" + ) + monkeypatch.setattr("shelfmark.metadata_providers.get_provider_kwargs", lambda _name: {}) + monkeypatch.setattr( + "shelfmark.metadata_providers.get_provider", lambda _name, **_kwargs: _Provider() + ) + + +class TestStrictMatch: + def test_matching_release_passes(self): + assert _strict(_release()) is True + + def test_title_mismatch_rejected(self): + assert _strict(_release(title="Some Other Book - Matt Dinniman [M4B]")) is False + + def test_missing_author_surname_rejected(self): + assert _strict(_release(title="Dungeon Crawler Carl [M4B]", indexer=None)) is False + + def test_author_found_in_extra_metadata(self): + release = _release(title="Dungeon Crawler Carl [M4B]", extra={"author": "Matt Dinniman"}) + assert _strict(release) is True + + def test_ebook_only_release_rejected(self): + release = _release(title="Dungeon Crawler Carl - Matt Dinniman", format="epub") + assert _strict(release) is False + + def test_audiobook_signal_in_title_accepted_without_format(self): + release = _release(title="Dungeon Crawler Carl by Matt Dinniman (Unabridged)", format=None) + assert _strict(release) is True + + @pytest.mark.parametrize("seeders", [0, None]) + def test_torrent_below_minimum_seeders_rejected(self, seeders): + assert _strict(_release(seeders=seeders), min_seeders=1) is False + + def test_torrent_without_seeder_count_allowed_when_minimum_is_zero(self): + assert _strict(_release(seeders=None), min_seeders=0) is True + + def test_non_torrent_ignores_seeders(self): + release = _release(protocol=ReleaseProtocol.NZB, seeders=None) + assert _strict(release, min_seeders=5) is True + + +class TestPickBestRelease: + def test_empty_returns_none(self): + assert auto_download.pick_best_release([]) is None + + def test_m4b_outranks_mp3_regardless_of_seeders(self): + mp3 = _release(source_id="mp3", format="mp3", seeders=500) + m4b = _release(source_id="m4b", format="m4b", seeders=1) + assert auto_download.pick_best_release([mp3, m4b]) is m4b + + def test_m4b_in_title_ranks_as_m4b_when_format_missing(self): + mp3 = _release(source_id="mp3", format="mp3", seeders=500) + untagged = _release(source_id="untagged", format=None, seeders=1) + assert auto_download.pick_best_release([mp3, untagged]) is untagged + + def test_seeders_break_format_ties(self): + few = _release(source_id="few", seeders=2, size_bytes=10) + many = _release(source_id="many", seeders=20, size_bytes=1) + assert auto_download.pick_best_release([few, many]) is many + + def test_size_breaks_seeder_ties(self): + small = _release(source_id="small", seeders=5, size_bytes=100) + large = _release(source_id="large", seeders=5, size_bytes=200) + assert auto_download.pick_best_release([small, large]) is large + + +class TestBuildReleaseData: + def test_drops_none_values(self): + release = _release(size=None, size_bytes=None, language=None, info_url=None) + + data = auto_download.build_release_data(release, _book(), "audiobook") + + assert data["source"] == "prowlarr" + assert data["source_id"] == "rel-1" + assert data["protocol"] == "torrent" + assert data["author"] == "Matt Dinniman" + assert data["content_type"] == "audiobook" + assert not {"size", "size_bytes", "language", "info_url", "preview", "year"} & data.keys() + + def test_carries_cover_series_and_book_metadata(self): + book = _book( + cover_url="https://img.example/cover.jpg", + publish_year=2020, + subtitle="Book One", + series_name="Dungeon Crawler Carl", + series_position=1.0, + ) + release = _release(content_type=None, extra={"preview": "https://img.example/rel.jpg"}) + + data = auto_download.build_release_data(release, book, "audiobook") + + assert data["preview"] == "https://img.example/cover.jpg" + assert data["year"] == 2020 + assert data["subtitle"] == "Book One" + assert data["series_name"] == "Dungeon Crawler Carl" + assert data["series_position"] == 1.0 + assert data["content_type"] == "audiobook" + assert data["extra"] == {"preview": "https://img.example/rel.jpg"} + + def test_falls_back_to_release_preview_without_book_cover(self): + release = _release(extra={"preview": "https://img.example/rel.jpg"}) + + data = auto_download.build_release_data(release, _book(), "audiobook") + + assert data["preview"] == "https://img.example/rel.jpg" + + +class TestConfiguredSourcePriority: + @pytest.fixture(autouse=True) + def _available_sources(self, monkeypatch): + monkeypatch.setattr( + "shelfmark.release_sources.list_available_sources", + lambda: [ + {"name": "first", "enabled": True}, + {"name": "second", "enabled": True}, + {"name": "unavailable", "enabled": False}, + ], + ) + + def _configure(self, monkeypatch, priority): + monkeypatch.setattr( + auto_download, "app_config", _Config(AUTO_DOWNLOAD_SOURCE_PRIORITY=priority) + ) + + def test_respects_configured_order_and_skips_unusable_entries(self, monkeypatch): + self._configure( + monkeypatch, + [ + {"id": "second"}, + {"id": "first", "enabled": False}, + {"id": "unavailable"}, + {"id": "unknown"}, + "not-a-dict", + ], + ) + + assert auto_download._configured_source_priority() == ["second"] + + @pytest.mark.parametrize("priority", [[], "garbage", ["not-a-dict"]]) + def test_falls_back_to_all_enabled_sources_when_nothing_is_configured( + self, monkeypatch, priority + ): + self._configure(monkeypatch, priority) + + assert auto_download._configured_source_priority() == ["first", "second"] + + @pytest.mark.parametrize( + "priority", + [ + [{"id": "first", "enabled": False}, {"id": "second", "enabled": False}], + [{"id": "unavailable"}], + [{"id": "unknown"}], + ], + ) + def test_a_configured_list_with_nothing_usable_searches_nothing(self, monkeypatch, priority): + self._configure(monkeypatch, priority) + + assert auto_download._configured_source_priority() == [] + + +class TestAutoDownloadPending: + def test_disabled_returns_zeros_without_touching_the_db(self, monkeypatch): + monkeypatch.setattr(auto_download, "app_config", _Config(AUTO_DOWNLOAD_ENABLED=False)) + + class _UntouchableDb: + def list_requests(self, **_kwargs): + raise AssertionError("pending requests must not be queried") + + summary = auto_download.auto_download_pending( + _UntouchableDb(), queue_release=lambda *_args, **_kwargs: (True, None) + ) + + assert summary == {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0} + + +class TestAutoDownloadRequest: + @pytest.fixture(autouse=True) + def _library_check_off(self, monkeypatch): + monkeypatch.setattr(auto_download, "app_config", _Config()) + monkeypatch.setattr( + "shelfmark.core.library_index.is_in_library", lambda *_args, **_kwargs: False + ) + + def _run(self, user_db, row, admin, *, queue_release=None, sources=("first", "second")): + return auto_download.auto_download_request( + user_db, + row, + sources=list(sources), + content_type="audiobook", + min_seeders=1, + queue_release=queue_release or (lambda *_args, **_kwargs: (True, None)), + admin_user_id=admin["id"], + ) + + def test_queues_first_strict_match_via_fulfil_path(self, user_db, monkeypatch): + admin = user_db.create_user(username="admin", role="admin") + reader = user_db.create_user(username="reader", role="user") + row = _pending_request(user_db, reader["id"]) + _stub_provider(monkeypatch, _book(cover_url="https://img.example/cover.jpg")) + + chosen = _release(source="second", source_id="good", seeders=5) + searched = [] + + def _search(_book_arg, *, sources, **_kwargs): + searched.extend(sources) + if sources == ["first"]: + epub = _release(source="first", title="Dungeon Crawler Carl.epub", format="epub") + return [epub], {"first": [epub]}, [] + return [chosen], {"second": [chosen]}, [] + + monkeypatch.setattr(auto_download, "search_book_releases", _search) + queued = [] + + def _queue(release_data, priority, *, user_id, username): + queued.append((release_data, priority, user_id, username)) + return True, None + + outcome = self._run(user_db, row, admin, queue_release=_queue) + + assert outcome.status == "queued" + assert outcome.source == "second" + assert searched == ["first", "second"] + + stored = user_db.get_request(row["id"]) + assert stored["status"] == "fulfilled" + assert stored["delivery_state"] == "queued" + assert stored["reviewed_by"] == admin["id"] + assert stored["release_data"]["source_id"] == "good" + + release_data, priority, user_id, username = queued[0] + assert release_data["_request_id"] == row["id"] + assert release_data["preview"] == "https://img.example/cover.jpg" + assert priority == 0 + assert user_id == reader["id"] + assert username == "reader" + + def test_queue_failure_rolls_request_back_to_pending(self, user_db, monkeypatch): + admin = user_db.create_user(username="admin", role="admin") + reader = user_db.create_user(username="reader", role="user") + row = _pending_request(user_db, reader["id"]) + _stub_provider(monkeypatch, _book()) + chosen = _release(source="first") + monkeypatch.setattr( + auto_download, + "search_book_releases", + lambda *_args, **_kwargs: ([chosen], {"first": [chosen]}, []), + ) + + outcome = self._run( + user_db, row, admin, queue_release=lambda *_args, **_kwargs: (False, "client down") + ) + + assert outcome.status == "error" + assert outcome.detail == "client down" + stored = user_db.get_request(row["id"]) + assert stored["status"] == "pending" + assert stored["release_data"] is None + + def test_no_strict_match_leaves_request_pending(self, user_db, monkeypatch): + admin = user_db.create_user(username="admin", role="admin") + reader = user_db.create_user(username="reader", role="user") + row = _pending_request(user_db, reader["id"]) + _stub_provider(monkeypatch, _book()) + wrong = _release(title="A Different Audiobook [M4B]") + monkeypatch.setattr( + auto_download, + "search_book_releases", + lambda *_args, **_kwargs: ([wrong], {"first": [wrong], "second": [wrong]}, []), + ) + + def _never_queue(*_args, **_kwargs): + raise AssertionError("nothing should be queued") + + outcome = self._run(user_db, row, admin, queue_release=_never_queue) + + assert outcome.status == "no_match" + assert user_db.get_request(row["id"])["status"] == "pending" + + def test_book_already_in_library_is_skipped_before_searching(self, user_db, monkeypatch): + admin = user_db.create_user(username="admin", role="admin") + reader = user_db.create_user(username="reader", role="user") + row = _pending_request(user_db, reader["id"]) + _stub_provider(monkeypatch, _book()) + monkeypatch.setattr(auto_download, "app_config", _Config()) + monkeypatch.setattr( + "shelfmark.core.library_index.is_in_library", lambda *_args, **_kwargs: True + ) + + def _never_search(*_args, **_kwargs): + raise AssertionError("release search must not run") + + monkeypatch.setattr(auto_download, "search_book_releases", _never_search) + + outcome = self._run(user_db, row, admin) + + assert outcome.status == "in_library" + assert user_db.get_request(row["id"])["status"] == "pending" + + def test_unregistered_provider_is_skipped(self, user_db, monkeypatch): + admin = user_db.create_user(username="admin", role="admin") + reader = user_db.create_user(username="reader", role="user") + row = _pending_request(user_db, reader["id"], provider="nope") + monkeypatch.setattr("shelfmark.metadata_providers.is_provider_registered", lambda _n: False) + + outcome = self._run(user_db, row, admin) + + assert outcome.status == "skipped" + assert outcome.detail == "no usable provider/id" + + +def test_pending_pass_without_an_admin_queues_nothing(user_db, monkeypatch): + monkeypatch.setattr(auto_download, "app_config", _Config(AUTO_DOWNLOAD_ENABLED=True)) + user_db.create_user(username="reader", role="user") + + summary = auto_download.auto_download_pending( + user_db, queue_release=lambda *_args, **_kwargs: (True, None) + ) + + assert summary == {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0} diff --git a/tests/core/test_hardcover_scheduler.py b/tests/core/test_hardcover_scheduler.py new file mode 100644 index 00000000..dddf0d86 --- /dev/null +++ b/tests/core/test_hardcover_scheduler.py @@ -0,0 +1,158 @@ +"""Tests for the Hardcover sync scheduler's interval parsing and run gating.""" + +import threading + +import pytest + +from shelfmark.core import hardcover_scheduler as scheduler + + +class _Config: + def __init__(self, **values): + self.values = values + + def get(self, key, default=None, user_id=None): + return self.values.get(key, default) + + +@pytest.fixture(autouse=True) +def _isolated_scheduler(monkeypatch): + monkeypatch.setattr(scheduler, "_ctx", {}) + monkeypatch.setattr(scheduler, "_run_lock", threading.Lock()) + monkeypatch.setattr(scheduler, "app_config", _Config()) + + +def _configure(monkeypatch, **values): + monkeypatch.setattr(scheduler, "app_config", _Config(**values)) + + +@pytest.mark.parametrize( + ("interval", "unit", "expected"), + [ + (30, "minutes", 1800), + (2, "hours", 7200), + ("1.5", "hours", 5400), + (0.5, "minutes", 60), + (6, "fortnights", 6 * 3600), + ("garbage", "hours", 6 * 3600), + ([], "hours", 6 * 3600), + ], +) +def test_interval_seconds(monkeypatch, interval, unit, expected): + _configure(monkeypatch, HARDCOVER_SYNC_INTERVAL=interval, HARDCOVER_SYNC_INTERVAL_UNIT=unit) + + assert scheduler._interval_seconds() == expected + + +def test_interval_seconds_defaults_to_six_hours(monkeypatch): + _configure(monkeypatch) + + assert scheduler._interval_seconds() == 6 * 3600 + + +@pytest.mark.parametrize( + ("values", "expected"), + [ + ({}, False), + ({"HARDCOVER_SYNC_ENABLED": True}, True), + ({"AUTO_DOWNLOAD_ENABLED": True}, True), + ({"HARDCOVER_SYNC_ENABLED": False, "AUTO_DOWNLOAD_ENABLED": False}, False), + ], +) +def test_enabled(monkeypatch, values, expected): + _configure(monkeypatch, **values) + + assert scheduler._enabled() is expected + + +def test_run_once_before_configure_is_unconfigured(): + assert scheduler.run_once() == {"status": "unconfigured"} + assert scheduler.run_once(force=True) == {"status": "unconfigured"} + + +def test_run_once_reports_busy_while_another_run_holds_the_lock(): + with scheduler._run_lock: + assert scheduler.run_once(force=True) == {"status": "busy"} + + assert scheduler.run_once() == {"status": "unconfigured"} + + +class TestRunOnce: + @pytest.fixture + def calls(self, monkeypatch): + recorded = {"sync": [], "auto": []} + + def _sync(user_db, **kwargs): + recorded["sync"].append((user_db, kwargs)) + return {"added": 1} + + def _auto(user_db, **kwargs): + recorded["auto"].append((user_db, kwargs)) + return {"queued": 1} + + monkeypatch.setattr("shelfmark.core.hardcover_sync.sync_wishlist", _sync) + monkeypatch.setattr("shelfmark.core.auto_download.auto_download_pending", _auto) + return recorded + + def test_scheduled_run_skips_sync_when_disabled(self, calls): + user_db, queue_release = object(), object() + scheduler.configure(user_db, queue_release, "/data/users.db") + + result = scheduler.run_once() + + assert result == {"status": "ok", "sync": None, "auto_download": {"queued": 1}} + assert calls["sync"] == [] + assert calls["auto"] == [(user_db, {"queue_release": queue_release})] + + def test_forced_run_syncs_regardless_of_toggle(self, calls): + user_db = object() + scheduler.configure(user_db, object(), "/data/users.db") + + result = scheduler.run_once(force=True) + + assert result == {"status": "ok", "sync": {"added": 1}, "auto_download": {"queued": 1}} + assert calls["sync"] == [(user_db, {"db_path": "/data/users.db"})] + + def test_scheduled_run_syncs_when_enabled(self, monkeypatch, calls): + _configure(monkeypatch, HARDCOVER_SYNC_ENABLED=True) + user_db = object() + scheduler.configure(user_db, object(), None) + + result = scheduler.run_once() + + assert result["sync"] == {"added": 1} + assert calls["sync"] == [(user_db, {"db_path": None})] + + def test_lock_is_released_after_a_run(self, calls): + scheduler.configure(object(), object(), None) + + scheduler.run_once() + + assert scheduler._run_lock.locked() is False + + +class TestTriggerAsync: + def test_returns_false_while_a_run_is_active(self, monkeypatch): + def _unexpected(**_kwargs): + raise AssertionError("run_once must not be scheduled") + + monkeypatch.setattr(scheduler, "run_once", _unexpected) + + with scheduler._run_lock: + assert scheduler.trigger_async() is False + + @pytest.mark.parametrize("force", [True, False]) + def test_runs_once_in_the_background_with_force_flag(self, monkeypatch, force): + finished = threading.Event() + seen = [] + + def _run_once(**kwargs): + seen.append(kwargs) + finished.set() + return {"status": "ok"} + + monkeypatch.setattr(scheduler, "run_once", _run_once) + + assert scheduler.trigger_async(force=force) is True + assert finished.wait(timeout=5) + assert seen == [{"force": force}] diff --git a/tests/core/test_hardcover_sync.py b/tests/core/test_hardcover_sync.py new file mode 100644 index 00000000..89c850ce --- /dev/null +++ b/tests/core/test_hardcover_sync.py @@ -0,0 +1,295 @@ +"""Tests for syncing Hardcover status shelves into pending requests.""" + +import sqlite3 +from types import SimpleNamespace + +import pytest + +from shelfmark.core import hardcover_sync +from shelfmark.core.user_db import UserDB +from shelfmark.core.utils import transform_cover_url +from shelfmark.metadata_providers import BookMetadata + + +class _Config: + def __init__(self, **values): + self.values = values + + def get(self, key, default=None, user_id=None): + return self.values.get(key, default) + + +class _Provider: + """Stub exposing the private paging method ``sync_wishlist`` relies on.""" + + def __init__(self, pages_by_status): + self.pages_by_status = pages_by_status + self.calls = [] + + def _fetch_current_user_books_by_status(self, status_id, page, limit): + self.calls.append((status_id, page, limit)) + pages = self.pages_by_status.get(status_id, []) + books = pages[page - 1] if page <= len(pages) else [] + return SimpleNamespace(books=books, has_more=page < len(pages)) + + +def _book(provider_id, title, author="Matt Dinniman", **overrides): + fields = { + "provider": "hardcover", + "provider_id": provider_id, + "title": title, + "authors": [author], + } + fields.update(overrides) + return BookMetadata(**fields) + + +def _insert_history(db_path, *, title, author): + conn = sqlite3.connect(db_path) + try: + conn.execute( + "INSERT INTO download_history (task_id, source, title, author, final_status) " + "VALUES (?, ?, ?, ?, ?)", + (f"task-{title}", "prowlarr", title, author, "complete"), + ) + conn.commit() + finally: + conn.close() + + +@pytest.fixture +def db_path(tmp_path): + return str(tmp_path / "users.db") + + +@pytest.fixture +def user_db(db_path): + db = UserDB(db_path) + db.initialize() + return db + + +def _configure(monkeypatch, **values): + monkeypatch.setattr(hardcover_sync, "app_config", _Config(**values)) + + +@pytest.mark.parametrize( + ("values", "expected"), + [ + ({"HARDCOVER_SYNC_TOKEN": " sync-token ", "HARDCOVER_API_KEY": "api"}, "sync-token"), + ({"HARDCOVER_SYNC_TOKEN": "", "HARDCOVER_API_KEY": "api-key"}, "api-key"), + ({}, ""), + ], +) +def test_configured_token_falls_back_to_provider_api_key(monkeypatch, values, expected): + _configure(monkeypatch, **values) + + assert hardcover_sync._configured_token() == expected + + +@pytest.mark.parametrize( + ("raw", "expected"), + [ + ("3", [3]), + (" 2 ", [2]), + (["1", "3"], [1, 3]), + ((1, "x", None), [1]), + ("garbage", [1]), + ([], [1]), + (42, [1]), + ], +) +def test_configured_statuses_parsing(monkeypatch, raw, expected): + _configure(monkeypatch, HARDCOVER_SYNC_STATUSES=raw) + + assert hardcover_sync._configured_statuses() == expected + + +def test_configured_statuses_default(monkeypatch): + _configure(monkeypatch) + + assert hardcover_sync._configured_statuses() == [1] + + +def test_book_to_book_data_includes_optional_fields_only_when_present(): + full = _book( + 7, + "Dungeon Crawler Carl", + cover_url="https://img.example/c.jpg", + publish_year=2020, + subtitle="Book One", + search_author="Dinniman", + ) + sparse = BookMetadata(provider="hardcover", provider_id="8", title="Untitled") + + assert hardcover_sync._book_to_book_data(full, "audiobook") == { + "title": "Dungeon Crawler Carl", + "author": "Dinniman", + "provider": "hardcover", + "provider_id": "7", + "content_type": "audiobook", + "preview": transform_cover_url("https://img.example/c.jpg", "hardcover_7"), + "year": 2020, + "subtitle": "Book One", + } + assert hardcover_sync._book_to_book_data(sparse, "ebook") == { + "title": "Untitled", + "author": "Unknown", + "provider": "hardcover", + "provider_id": "8", + "content_type": "ebook", + } + + +def test_existing_provider_ids_reads_provider_id_or_id(user_db): + user = user_db.create_user(username="reader", role="user") + for book_data in ( + {"title": "A", "author": "X", "provider": "hardcover", "provider_id": 11}, + {"title": "B", "author": "X", "provider": "hardcover", "id": "22"}, + {"title": "C", "author": "X", "provider": "openlibrary", "id": "OL22W"}, + {"title": "D", "author": "X", "provider": "googlebooks", "provider_id": 33}, + {"title": "E", "author": "X", "provider": "manual"}, + ): + user_db.create_request( + user_id=user["id"], + content_type="audiobook", + request_level="book", + policy_mode="request_book", + book_data=book_data, + ) + + assert hardcover_sync._existing_provider_ids(user_db) == {"11", "22"} + + +def test_already_downloaded_matches_history_case_insensitively(user_db, db_path): + _insert_history(db_path, title="Dungeon Crawler Carl", author="Matt Dinniman") + + assert hardcover_sync._already_downloaded(db_path, "dungeon crawler CARL", "MATT dinniman") + assert not hardcover_sync._already_downloaded(db_path, "Dungeon Crawler Carl", "Someone") + assert not hardcover_sync._already_downloaded(None, "Dungeon Crawler Carl", "Matt Dinniman") + + +def test_already_downloaded_fails_open_without_history_table(tmp_path): + missing = str(tmp_path / "empty.db") + + assert hardcover_sync._already_downloaded(missing, "Anything", "Anyone") is False + + +class TestSyncWishlist: + @pytest.fixture(autouse=True) + def _defaults(self, monkeypatch): + _configure(monkeypatch, HARDCOVER_SYNC_STATUSES="1") + monkeypatch.setattr( + "shelfmark.core.library_index.is_in_library", lambda *_args, **_kwargs: False + ) + + def _use_provider(self, monkeypatch, provider): + monkeypatch.setattr(hardcover_sync, "_build_provider", lambda: provider) + + def test_adds_new_requests_across_pages(self, user_db, db_path, monkeypatch): + user = user_db.create_user(username="reader", role="user") + provider = _Provider( + { + 1: [ + [_book(1, "Dungeon Crawler Carl", cover_url="https://img.example/1.jpg")], + [_book(2, "Carl's Doomsday Scenario", publish_year=2021)], + ] + } + ) + self._use_provider(monkeypatch, provider) + + summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path, user_id=user["id"]) + + assert summary == {"added": 2, "skipped": 0, "in_library": 0, "errors": 0} + assert provider.calls == [(1, 1, 25), (1, 2, 25)] + rows = {row["book_data"]["provider_id"]: row for row in user_db.list_requests()} + assert set(rows) == {"1", "2"} + assert rows["1"]["status"] == "pending" + assert rows["1"]["content_type"] == "audiobook" + assert rows["1"]["book_data"]["preview"] == transform_cover_url( + "https://img.example/1.jpg", "hardcover_1" + ) + assert rows["2"]["book_data"]["year"] == 2021 + assert "preview" not in rows["2"]["book_data"] + + def test_skips_known_downloaded_owned_and_duplicate_books(self, user_db, db_path, monkeypatch): + user = user_db.create_user(username="reader", role="user") + user_db.create_request( + user_id=user["id"], + content_type="audiobook", + request_level="book", + policy_mode="request_book", + book_data={ + "title": "Already Requested", + "author": "Matt Dinniman", + "provider": "hardcover", + "provider_id": "10", + }, + ) + _insert_history(db_path, title="Already Downloaded", author="Matt Dinniman") + _configure(monkeypatch, HARDCOVER_SYNC_STATUSES="1") + monkeypatch.setattr( + "shelfmark.core.library_index.is_in_library", + lambda book, *_args, **_kwargs: book.title == "Already Owned", + ) + provider = _Provider( + { + 1: [ + [ + _book(10, "Already Requested"), + _book(11, "Already Downloaded"), + _book(12, "Already Owned"), + _book(13, "Brand New"), + _book(14, "Brand New"), + ] + ] + } + ) + self._use_provider(monkeypatch, provider) + + summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path, user_id=user["id"]) + + assert summary == {"added": 1, "skipped": 3, "in_library": 1, "errors": 0} + titles = sorted(row["book_data"]["title"] for row in user_db.list_requests()) + assert titles == ["Already Requested", "Brand New"] + + def test_unavailable_provider_counts_as_error(self, user_db, db_path, monkeypatch): + self._use_provider(monkeypatch, None) + + summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path) + + assert summary == {"added": 0, "skipped": 0, "in_library": 0, "errors": 1} + assert user_db.list_requests() == [] + + +class TestResolveRequestOwner: + def test_returns_lowest_id_admin(self, user_db): + user_db.create_user(username="reader", role="user") + first_admin = user_db.create_user(username="ops", role="admin") + user_db.create_user(username="second", role="admin") + + assert hardcover_sync.resolve_request_owner(user_db) == first_admin["id"] + + def test_returns_none_without_admins(self, user_db): + user_db.create_user(username="reader", role="user") + + assert hardcover_sync.resolve_request_owner(user_db) is None + + def test_sync_without_admin_counts_an_error_and_creates_nothing(self, user_db, db_path): + user_db.create_user(username="reader", role="user") + + summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path) + + assert summary == {"added": 0, "skipped": 0, "in_library": 0, "errors": 1} + assert user_db.list_requests() == [] + + def test_sync_defaults_to_the_lowest_id_admin(self, user_db, db_path, monkeypatch): + user_db.create_user(username="reader", role="user") + admin = user_db.create_user(username="ops", role="admin") + provider = _Provider({1: [[_book(1, "Dungeon Crawler Carl")]]}) + monkeypatch.setattr(hardcover_sync, "_build_provider", lambda: provider) + + summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path) + + assert summary["added"] == 1 + assert user_db.list_requests()[0]["user_id"] == admin["id"] diff --git a/tests/core/test_release_search.py b/tests/core/test_release_search.py index 2755966b..a9a36a3c 100644 --- a/tests/core/test_release_search.py +++ b/tests/core/test_release_search.py @@ -117,3 +117,58 @@ def test_plan_receives_filters_and_user_id(monkeypatch: pytest.MonkeyPatch) -> N "source_filters": None, "user_id": 42, } + + +def test_search_book_releases_preserves_source_order_and_aggregates( + monkeypatch: pytest.MonkeyPatch, +) -> None: + sources = {"a": _Source(result=["a1"]), "b": _Source(result=["b1", "b2"])} + monkeypatch.setattr("shelfmark.release_sources.get_source", lambda name: sources[name]) + + all_releases, by_source, errors = release_search.search_book_releases( + _book(), sources=["b", "a"], content_type="audiobook" + ) + + assert list(by_source) == ["b", "a"] + assert by_source == {"b": ["b1", "b2"], "a": ["a1"]} + assert all_releases == ["b1", "b2", "a1"] + assert errors == [] + _book_arg, _plan, expand_search, content_type = sources["a"].calls[0] + assert expand_search is True + assert content_type == "audiobook" + + +def test_search_book_releases_collects_errors_and_keeps_healthy_results( + monkeypatch: pytest.MonkeyPatch, +) -> None: + sources = { + "down": _Source(error=SourceUnavailableError("offline")), + "ok": _Source(result=["r1"]), + } + + def _get(name: str) -> Any: + if name not in sources: + msg = f"Unknown release source: {name}" + raise ValueError(msg) + return sources[name] + + monkeypatch.setattr("shelfmark.release_sources.get_source", _get) + + all_releases, by_source, errors = release_search.search_book_releases( + _book(), sources=["down", "ok", "nope"] + ) + + assert all_releases == ["r1"] + assert by_source == {"ok": ["r1"]} + assert errors == ["down: offline", "Unknown source: nope"] + + +def test_search_book_releases_with_no_sources_returns_empty( + monkeypatch: pytest.MonkeyPatch, +) -> None: + def _unexpected(_name: str) -> Any: + raise AssertionError("no source should be resolved") + + monkeypatch.setattr("shelfmark.release_sources.get_source", _unexpected) + + assert release_search.search_book_releases(_book(), sources=[]) == ([], {}, [])