mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-11-14 18:08:29 +00:00
feat: optional Hardcover wishlist sync with automated downloading (1 of 5) (#1516)
This replaces #1047, which @InfiniteAvenger opened and calibrain offered to re-review. It is Caleb's feature commit with his authorship kept, rebased onto current main, with small fixes, tests and a docs page on top. Everything is opt-in and off by default, so nothing changes for anyone who doesn't turn it on. What it adds is a **Settings, Hardcover Sync** tab with three parts that can be switched on separately: - **Shelf sync**: a Hardcover shelf (Want to Read by default) is pulled into requests on a schedule, or with a Sync now button. Requests carry the Hardcover metadata and cover. - **Automatic downloads**: synced requests are approved and a release is picked from a source priority list. A strict title, author and format match (plus a seeder minimum for torrents) decides, so anything without a confident match stays pending for manual review. - **Skip books you already own**: sync and automatic downloads ask whichever libraries are enabled under Library Check (Calibre today) and skip a book that is already there. On top of Caleb's commit I made the request owner a lookup instead of assuming user id 1, sent synced cover URLs through the proxy when the request is created, got ruff and basedpyright clean, and added tests and `docs/hardcover-sync.md`. His commit also had its own Audiobookshelf settings and a Test library connection button, written before the library check was split into providers. The button would have raised a TypeError against the current `test_connection`, so those settings are gone from this tab and an Audiobookshelf provider comes as a separate PR, which is the part of #1047 that is missing here (for now anyways...). I've run this daily for about a couple of weeks. Ruff, basedpyright and the backend suite pass (4426 passed). This is 1 of 5. As calibrain suggested, the rest are stacked on this one, each saying which number it is. Closes https://github.com/calibrain/shelfmark/pull/1047#issuecomment-6302071588 --------- Co-authored-by: InfiniteAvenger <calebewest02@gmail.com> Co-authored-by: CaliBrain <calibrain@l4n.xyz>
This commit is contained in:
14 files changed
+2039
No files matched your search
@@ -0,0 +1,79 @@
|
||||
# Hardcover Sync
|
||||
|
||||
Configure in **Settings → Hardcover Sync**.
|
||||
|
||||
Hardcover Sync pulls books from your [Hardcover](https://hardcover.app) reading shelves into Shelfmark as requests, optionally downloads them without an admin picking a release, and can skip books you already own in a library you have set up under [Library Check](library-check.md). Every part is opt-in and off by default; the tab does nothing until you enable it.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### Shelf sync
|
||||
|
||||
On a schedule, Shelfmark reads the selected Hardcover shelf (by default **Want to Read**) through the Hardcover metadata provider and creates a **pending** request for each book that is not already known. Requests carry Hardcover's metadata, including the cover, and are owned by the lowest-id admin account.
|
||||
|
||||
A book is skipped when:
|
||||
|
||||
- a request with the same Hardcover book id already exists, in any status
|
||||
- a download with the same title and author is already in the download history
|
||||
- the library check (below) says it is already in your library
|
||||
- a pending request with the same title, author, and content type already exists
|
||||
|
||||
### Automatic downloads
|
||||
|
||||
When enabled, each pending request that came from Hardcover is searched across your release sources in the configured priority order. The first source that yields a **strict match** wins, and the best candidate from that source is queued through the normal request fulfilment path, so it appears in the activity view and download history exactly like a request an admin approved.
|
||||
|
||||
A release is a strict match only when all of the following hold:
|
||||
|
||||
- at least 85% of the significant title words appear in the release title
|
||||
- the author's surname appears in the release title, indexer name, or author field
|
||||
- the release is an audiobook: an audiobook format (`m4b`, `mp3`, or your configured audiobook formats), or "audiobook"/"unabridged" in the title. Ebook-only formats such as `epub` and `pdf` are rejected
|
||||
- for torrents, seeders meet **Minimum seeders**; other protocols ignore this check
|
||||
|
||||
Among matching releases from the winning source, `m4b` outranks `mp3`, then more seeders, then larger size. If no source produces a confident match the request stays **pending** for manual review; nothing is guessed.
|
||||
|
||||
### Library check
|
||||
|
||||
Both shelf sync and automatic downloads ask the libraries you have enabled under [Library Check](library-check.md) whether the book is already there, for the content type being requested, and skip it if so. There is no separate switch on this tab. A library that is not enabled, or that does not hold the requested content type, never causes a skip.
|
||||
|
||||
The check **fails open**: a library that cannot be read is logged and the book is treated as not owned, so processing continues.
|
||||
|
||||
## Settings
|
||||
|
||||
| Key | Label | Default | Notes |
|
||||
|-----|-------|---------|-------|
|
||||
| `HARDCOVER_SYNC_ENABLED` | Enable scheduled sync | Off | Runs shelf sync on the interval below |
|
||||
| `HARDCOVER_SYNC_TOKEN` | Hardcover API Token | — | Leave blank to reuse the Hardcover metadata provider's token |
|
||||
| `HARDCOVER_SYNC_STATUSES` | Shelves to sync | Want to Read | Hardcover reading status to pull from |
|
||||
| `HARDCOVER_SYNC_CONTENT_TYPE` | Request as | Audiobooks | Content type given to synced requests and targeted by automatic downloads |
|
||||
| `HARDCOVER_SYNC_INTERVAL` | Sync interval | 6 | Combined with the unit below |
|
||||
| `HARDCOVER_SYNC_INTERVAL_UNIT` | Interval unit | Hours | Minutes or hours. The effective interval never drops below 60 seconds |
|
||||
| `AUTO_DOWNLOAD_ENABLED` | Enable automatic downloads | Off | Auto-approve and queue strict matches for Hardcover requests |
|
||||
| `AUTO_DOWNLOAD_SOURCE_PRIORITY` | Source priority | All usable sources | Drag to order. Sources you disable here, or that are not configured, are skipped |
|
||||
| `AUTO_DOWNLOAD_MIN_SEEDERS` | Minimum seeders (torrents) | 1 | Torrent releases below this are ignored |
|
||||
|
||||
One action button sits at the bottom of the tab:
|
||||
|
||||
- **Sync now** runs a sync and automatic-download pass immediately using the saved settings. Sync runs even when scheduled sync is off; automatic downloads still require their own toggle.
|
||||
|
||||
## Request Flow
|
||||
|
||||
1. Shelf sync creates a **pending** request per new Hardcover book, with cover, year, and subtitle where Hardcover has them.
|
||||
2. If automatic downloads are enabled, the request is searched source by source and the first strict match is queued through the normal fulfil path. The request becomes **fulfilled** and its delivery state is tracked like any other.
|
||||
3. If no source yields a confident match, the request stays **pending** in the admin queue for manual review, exactly as if a user had requested the book.
|
||||
|
||||
Only requests whose provider is Hardcover are considered for automatic downloads, and requests that have already been dispatched are left alone.
|
||||
|
||||
## Scheduler
|
||||
|
||||
The scheduler starts with the app and waits **60 seconds** before its first cycle. Each cycle runs a shelf sync (when scheduled sync is enabled) followed by an automatic-download pass (when that is enabled), then sleeps for the configured interval. Intervals shorter than 60 seconds are raised to 60 seconds so a typo cannot hammer the Hardcover API. When neither toggle is on the scheduler idles.
|
||||
|
||||
Only one run happens at a time. If a scheduled cycle is in progress, **Sync now** reports that a sync is already running.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Sync adds nothing and the log says "no Hardcover token configured"** — set **Hardcover API Token** on this tab, or configure the token on the Hardcover metadata provider. A sync without a usable token counts as an error and creates no requests.
|
||||
|
||||
**Requests appear but nothing downloads** — check **Enable automatic downloads** is on, that **Source priority** contains at least one configured source that supports audiobooks, and look for `no strict match` lines in the log. Lowering **Minimum seeders** helps on quiet trackers; loosening the title or author match is deliberately not an option.
|
||||
|
||||
**Books you own keep being added** — enable the library check for the format you request (see [Library Check](library-check.md)) and use its test button. Because the check fails open, a library that cannot be read only produces a log warning rather than blocking the sync.
|
||||
|
||||
**Too many requests at once** — the first sync of a large shelf creates one request per book, and the per-user pending limit from **Users & Requests** does not apply to synced requests. Start with a shelf you are happy to see in the queue in full.
|
||||
@@ -13,6 +13,7 @@ Use the guides below to set up the app, connect your library tools, and understa
|
||||
## Core Guides
|
||||
|
||||
- [Users & Requests](users-and-requests.md)
|
||||
- [Hardcover Sync](hardcover-sync.md)
|
||||
- [Reverse Proxy](reverse-proxy.md)
|
||||
- [OIDC](oidc.md)
|
||||
- [API Access](api-access.md)
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
"""Settings tab for Hardcover wishlist sync + automatic downloads."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from shelfmark.core.settings_registry import (
|
||||
ActionButton,
|
||||
CheckboxField,
|
||||
HeadingField,
|
||||
NumberField,
|
||||
OrderableListField,
|
||||
PasswordField,
|
||||
SelectField,
|
||||
SettingsField,
|
||||
register_settings,
|
||||
)
|
||||
|
||||
_CONTENT_TYPE_OPTIONS = [
|
||||
{"value": "audiobook", "label": "Audiobooks"},
|
||||
{"value": "ebook", "label": "Ebooks"},
|
||||
]
|
||||
|
||||
|
||||
def _status_options() -> list[dict[str, str]]:
|
||||
from shelfmark.metadata_providers.hardcover import HARDCOVER_STATUSES
|
||||
|
||||
return [{"value": str(status["id"]), "label": status["label"]} for status in HARDCOVER_STATUSES]
|
||||
|
||||
|
||||
def _audiobook_source_options() -> list[dict[str, Any]]:
|
||||
"""Orderable-list options: release sources that can return audiobooks."""
|
||||
from shelfmark.release_sources import list_available_sources
|
||||
|
||||
options: list[dict[str, Any]] = []
|
||||
for src in list_available_sources():
|
||||
supported = src.get("supported_content_types") or ["ebook", "audiobook"]
|
||||
if "audiobook" not in supported:
|
||||
continue
|
||||
enabled = bool(src.get("enabled"))
|
||||
options.append(
|
||||
{
|
||||
"id": src["name"],
|
||||
"label": src.get("display_name") or src["name"],
|
||||
"description": None if enabled else "Source not configured / unavailable",
|
||||
"isLocked": not enabled,
|
||||
"disabledReason": None if enabled else "Source not configured / unavailable",
|
||||
}
|
||||
)
|
||||
return options
|
||||
|
||||
|
||||
def _sync_now(current_values: dict[str, Any] | None = None) -> dict[str, Any]:
|
||||
"""Action-button callback: trigger an immediate sync + auto-download pass."""
|
||||
from shelfmark.core import hardcover_scheduler
|
||||
|
||||
started = hardcover_scheduler.trigger_async(force=True)
|
||||
if not started:
|
||||
return {"success": False, "message": "A sync is already running. Try again shortly."}
|
||||
return {
|
||||
"success": True,
|
||||
"message": "Sync started. New wishlist books will appear as requests; "
|
||||
"matching audiobooks auto-download if enabled.",
|
||||
}
|
||||
|
||||
|
||||
@register_settings("hardcover_sync", "Hardcover Sync", icon="book", order=8)
|
||||
def hardcover_sync_settings() -> list[SettingsField]:
|
||||
"""Configure Hardcover wishlist sync and automatic downloads."""
|
||||
return [
|
||||
HeadingField(
|
||||
key="hardcover_sync_heading",
|
||||
title="Hardcover Wishlist Sync",
|
||||
description=(
|
||||
"Automatically pull books from your Hardcover reading shelves into "
|
||||
"Shelfmark as requests, on a schedule."
|
||||
),
|
||||
),
|
||||
CheckboxField(
|
||||
key="HARDCOVER_SYNC_ENABLED",
|
||||
label="Enable scheduled sync",
|
||||
description="Periodically sync the selected Hardcover shelves into requests.",
|
||||
default=False,
|
||||
),
|
||||
PasswordField(
|
||||
key="HARDCOVER_SYNC_TOKEN",
|
||||
label="Hardcover API Token",
|
||||
description=(
|
||||
"Bearer token for your Hardcover account. Leave blank to reuse the "
|
||||
"token from the Hardcover metadata provider."
|
||||
),
|
||||
placeholder="Reuses metadata provider token if blank",
|
||||
),
|
||||
SelectField(
|
||||
key="HARDCOVER_SYNC_STATUSES",
|
||||
label="Shelves to sync",
|
||||
description="Which Hardcover reading shelf to pull from.",
|
||||
options=_status_options,
|
||||
default="1",
|
||||
),
|
||||
SelectField(
|
||||
key="HARDCOVER_SYNC_CONTENT_TYPE",
|
||||
label="Request as",
|
||||
description="Content type assigned to synced requests and targeted for downloads.",
|
||||
options=_CONTENT_TYPE_OPTIONS,
|
||||
default="audiobook",
|
||||
),
|
||||
NumberField(
|
||||
key="HARDCOVER_SYNC_INTERVAL",
|
||||
label="Sync interval",
|
||||
description="How often the scheduled sync runs (minimum 1 minute).",
|
||||
default=6,
|
||||
min_value=1,
|
||||
max_value=10000,
|
||||
),
|
||||
SelectField(
|
||||
key="HARDCOVER_SYNC_INTERVAL_UNIT",
|
||||
label="Interval unit",
|
||||
description="Whether the sync interval is measured in minutes or hours.",
|
||||
options=[
|
||||
{"value": "minutes", "label": "Minutes"},
|
||||
{"value": "hours", "label": "Hours"},
|
||||
],
|
||||
default="hours",
|
||||
),
|
||||
HeadingField(
|
||||
key="auto_download_heading",
|
||||
title="Automatic Downloads",
|
||||
description=(
|
||||
"When enabled, synced requests are auto-approved and downloaded from the "
|
||||
"first source below that yields a strict title/author/format match. If no "
|
||||
"confident match is found, the request is left pending for manual review."
|
||||
),
|
||||
),
|
||||
CheckboxField(
|
||||
key="AUTO_DOWNLOAD_ENABLED",
|
||||
label="Enable automatic downloads",
|
||||
description="Auto-approve and auto-download matching releases for synced requests.",
|
||||
default=False,
|
||||
),
|
||||
OrderableListField(
|
||||
key="AUTO_DOWNLOAD_SOURCE_PRIORITY",
|
||||
label="Source priority",
|
||||
description=(
|
||||
"Drag to set which release sources are tried first. The first source with "
|
||||
"a strict match wins. Disabled sources are skipped."
|
||||
),
|
||||
options=_audiobook_source_options,
|
||||
# Default left empty so registration never imports release_sources (which would
|
||||
# deadlock the registry lock). When empty, all available sources are used in
|
||||
# registry order (see auto_download._configured_source_priority).
|
||||
default=[],
|
||||
),
|
||||
NumberField(
|
||||
key="AUTO_DOWNLOAD_MIN_SEEDERS",
|
||||
label="Minimum seeders (torrents)",
|
||||
description="Skip torrent releases with fewer seeders than this.",
|
||||
default=1,
|
||||
min_value=0,
|
||||
max_value=1000,
|
||||
),
|
||||
ActionButton(
|
||||
key="sync_now",
|
||||
label="Sync now",
|
||||
description="Run a sync + auto-download pass immediately (uses saved settings).",
|
||||
style="primary",
|
||||
callback=_sync_now,
|
||||
),
|
||||
]
|
||||
@@ -2167,3 +2167,8 @@ def advanced_settings() -> list[SettingsField]:
|
||||
|
||||
|
||||
register_on_save("advanced", _on_save_advanced)
|
||||
|
||||
|
||||
# Register the Hardcover Sync settings tab. Imported here (rather than at each
|
||||
# settings endpoint) so every code path that imports this module also registers it.
|
||||
from shelfmark.config import hardcover_sync_settings as _hardcover_sync_settings # noqa: E402,F401
|
||||
@@ -0,0 +1,395 @@
|
||||
"""Automatic release selection + download for pending requests.
|
||||
|
||||
Given a pending request (typically synced from a Hardcover wishlist), this module
|
||||
searches release sources in a user-configured priority order, applies a strict
|
||||
title/author/format match guard, picks the best candidate from the first source
|
||||
that yields a match, and queues it via the normal request-fulfilment path.
|
||||
|
||||
Design goals:
|
||||
- **Strict by default.** Bias toward leaving a request pending (manual review) over
|
||||
grabbing the wrong file. Title and author must both match; the format must be a
|
||||
real audiobook format (no ebook-only fallbacks when targeting audiobooks).
|
||||
- **Source priority is primary.** Walk the configured source order top-down and take
|
||||
the first source that produces at least one strict match.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from shelfmark.core.config import config as app_config
|
||||
from shelfmark.core.logger import setup_logger
|
||||
from shelfmark.core.release_search import search_book_releases
|
||||
from shelfmark.core.request_helpers import coerce_int
|
||||
from shelfmark.core.text_match import (
|
||||
DEFAULT_TITLE_MATCH_THRESHOLD,
|
||||
author_surname,
|
||||
title_tokens_match,
|
||||
)
|
||||
from shelfmark.core.text_match import (
|
||||
tokens as _tokens,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
from shelfmark.core.user_db import UserDB
|
||||
from shelfmark.metadata_providers import BookMetadata
|
||||
from shelfmark.release_sources import Release
|
||||
|
||||
logger = setup_logger(__name__)
|
||||
|
||||
# Fraction of significant book-title tokens that must appear in the release title.
|
||||
TITLE_MATCH_THRESHOLD = DEFAULT_TITLE_MATCH_THRESHOLD
|
||||
|
||||
DEFAULT_AUDIOBOOK_FORMATS = ("m4b", "mp3")
|
||||
EBOOK_FORMAT_MARKERS = (
|
||||
"epub",
|
||||
"mobi",
|
||||
"azw3",
|
||||
"azw",
|
||||
"pdf",
|
||||
"fb2",
|
||||
"djvu",
|
||||
"cbz",
|
||||
"cbr",
|
||||
)
|
||||
# Audiobook signals we recognise in free-text release titles in addition to formats.
|
||||
AUDIOBOOK_TITLE_MARKERS = ("audiobook", "unabridged", "m4b", "audio book")
|
||||
|
||||
# Audiobook format ranking for tie-breaking within a single source.
|
||||
_FORMAT_RANK = {"m4b": 3, "m4a": 2, "mp3": 1}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AutoDownloadOutcome:
|
||||
"""Result of attempting to auto-download a single request."""
|
||||
|
||||
request_id: int
|
||||
status: str # "queued" | "no_match" | "skipped" | "error"
|
||||
detail: str = ""
|
||||
source: str | None = None
|
||||
|
||||
|
||||
def _audiobook_formats() -> set[str]:
|
||||
configured = app_config.get("SUPPORTED_AUDIOBOOK_FORMATS", list(DEFAULT_AUDIOBOOK_FORMATS))
|
||||
values: list[object]
|
||||
if isinstance(configured, str):
|
||||
values = [configured]
|
||||
elif isinstance(configured, (list, tuple, set)):
|
||||
values = list(configured)
|
||||
else:
|
||||
values = []
|
||||
formats = {str(fmt).strip().lower() for fmt in values if str(fmt).strip()}
|
||||
return formats or set(DEFAULT_AUDIOBOOK_FORMATS)
|
||||
|
||||
|
||||
def _author_surname_tokens(book: BookMetadata) -> list[str]:
|
||||
"""Return distinctive author tokens (prefer the surname of the primary author)."""
|
||||
author = book.search_author or (book.authors[0] if book.authors else "")
|
||||
surname = author_surname(author)
|
||||
return [surname] if surname else []
|
||||
|
||||
|
||||
def _title_match(book: BookMetadata, release_title: str) -> bool:
|
||||
return title_tokens_match(
|
||||
book.search_title or book.title,
|
||||
set(_tokens(release_title)),
|
||||
TITLE_MATCH_THRESHOLD,
|
||||
)
|
||||
|
||||
|
||||
def _author_match(book: BookMetadata, release: Release) -> bool:
|
||||
surname = _author_surname_tokens(book)
|
||||
if not surname:
|
||||
return True # No author metadata to verify against; don't block on it.
|
||||
haystack = set(_tokens(release.title)) | set(_tokens(release.indexer))
|
||||
extra_author = release.extra.get("author") if isinstance(release.extra, dict) else None
|
||||
haystack |= set(_tokens(extra_author if isinstance(extra_author, str) else None))
|
||||
return all(tok in haystack for tok in surname)
|
||||
|
||||
|
||||
def _format_match(release: Release, audiobook_formats: set[str]) -> bool:
|
||||
"""Require a real audiobook signal and reject ebook-only releases."""
|
||||
fmt = (release.format or "").strip().lower()
|
||||
title_l = (release.title or "").lower()
|
||||
|
||||
has_audiobook_signal = (
|
||||
fmt in audiobook_formats
|
||||
or any(marker in title_l for marker in AUDIOBOOK_TITLE_MARKERS)
|
||||
or any(f".{af}" in title_l or f" {af}" in title_l for af in audiobook_formats)
|
||||
)
|
||||
if not has_audiobook_signal:
|
||||
return False
|
||||
|
||||
# Reject things that are clearly an ebook and nothing else.
|
||||
return fmt not in EBOOK_FORMAT_MARKERS
|
||||
|
||||
|
||||
def _seeders_ok(release: Release, min_seeders: int) -> bool:
|
||||
from shelfmark.release_sources import ReleaseProtocol
|
||||
|
||||
if release.protocol != ReleaseProtocol.TORRENT:
|
||||
return True # Non-torrent protocols have no seeder concept.
|
||||
if release.seeders is None:
|
||||
return min_seeders <= 0
|
||||
return release.seeders >= min_seeders
|
||||
|
||||
|
||||
def strict_match(
|
||||
release: Release,
|
||||
book: BookMetadata,
|
||||
*,
|
||||
min_seeders: int = 1,
|
||||
audiobook_formats: set[str] | None = None,
|
||||
) -> bool:
|
||||
"""Return True only if the release confidently matches the requested audiobook."""
|
||||
formats = audiobook_formats if audiobook_formats is not None else _audiobook_formats()
|
||||
return (
|
||||
_title_match(book, release.title)
|
||||
and _author_match(book, release)
|
||||
and _format_match(release, formats)
|
||||
and _seeders_ok(release, min_seeders)
|
||||
)
|
||||
|
||||
|
||||
def _release_sort_key(release: Release) -> tuple[int, int, int]:
|
||||
fmt = (release.format or "").strip().lower()
|
||||
if fmt not in _FORMAT_RANK and "m4b" in (release.title or "").lower():
|
||||
fmt = "m4b"
|
||||
return (
|
||||
_FORMAT_RANK.get(fmt, 0),
|
||||
release.seeders or 0,
|
||||
release.size_bytes or 0,
|
||||
)
|
||||
|
||||
|
||||
def pick_best_release(releases: list[Release]) -> Release | None:
|
||||
if not releases:
|
||||
return None
|
||||
return max(releases, key=_release_sort_key)
|
||||
|
||||
|
||||
def build_release_data(release: Release, book: BookMetadata, content_type: str) -> dict[str, Any]:
|
||||
"""Build a queue_release-compatible payload from a chosen Release."""
|
||||
author = book.search_author or (book.authors[0] if book.authors else None)
|
||||
extra = release.extra if isinstance(release.extra, dict) else {}
|
||||
# Carry the book's cover so the download shows artwork immediately; the orchestrator
|
||||
# reads release_data["preview"] and proxies it (orchestrator.queue_release / task_to_dict).
|
||||
preview = book.cover_url or extra.get("preview")
|
||||
payload: dict[str, Any] = {
|
||||
"source": release.source,
|
||||
"source_id": release.source_id,
|
||||
"title": release.title,
|
||||
"author": author,
|
||||
"year": book.publish_year,
|
||||
"preview": preview,
|
||||
"format": release.format,
|
||||
"size": release.size,
|
||||
"size_bytes": release.size_bytes,
|
||||
"download_url": release.download_url,
|
||||
"info_url": release.info_url,
|
||||
"protocol": release.protocol.value if release.protocol else None,
|
||||
"indexer": release.indexer,
|
||||
"seeders": release.seeders,
|
||||
"language": release.language,
|
||||
"content_type": release.content_type or content_type,
|
||||
"series_name": book.series_name,
|
||||
"series_position": book.series_position,
|
||||
"subtitle": book.subtitle,
|
||||
"extra": dict(extra),
|
||||
}
|
||||
return {key: value for key, value in payload.items() if value is not None}
|
||||
|
||||
|
||||
def _configured_source_priority() -> list[str]:
|
||||
"""Return enabled release-source names in configured priority order."""
|
||||
from shelfmark.release_sources import list_available_sources
|
||||
|
||||
available = list_available_sources()
|
||||
available_by_name = {src["name"]: src for src in available}
|
||||
|
||||
raw = app_config.get("AUTO_DOWNLOAD_SOURCE_PRIORITY", [])
|
||||
ordered: list[str] = []
|
||||
configured = isinstance(raw, list) and any(isinstance(item, dict) for item in raw)
|
||||
if isinstance(raw, list):
|
||||
for item in raw:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
name = item.get("id")
|
||||
if not isinstance(name, str) or name not in available_by_name:
|
||||
continue
|
||||
if not bool(item.get("enabled", True)):
|
||||
continue
|
||||
if not available_by_name[name].get("enabled", False):
|
||||
continue # Source itself not usable (unconfigured/unavailable).
|
||||
ordered.append(name)
|
||||
|
||||
if ordered:
|
||||
return ordered
|
||||
if configured:
|
||||
# A priority list was saved but nothing in it is usable. Falling back to every
|
||||
# source would search the ones the user turned off.
|
||||
logger.warning(
|
||||
"auto-download: every source in the priority list is disabled or unavailable"
|
||||
)
|
||||
return []
|
||||
# Nothing configured yet: every usable source, registry order.
|
||||
return [src["name"] for src in available if src.get("enabled")]
|
||||
|
||||
|
||||
def auto_download_request(
|
||||
user_db: UserDB,
|
||||
request_row: dict[str, Any],
|
||||
*,
|
||||
sources: list[str],
|
||||
content_type: str,
|
||||
min_seeders: int,
|
||||
queue_release: Callable[..., tuple[bool, str | None]],
|
||||
admin_user_id: int,
|
||||
) -> AutoDownloadOutcome:
|
||||
"""Search, strict-match, and queue a single pending request."""
|
||||
from shelfmark.core.requests_service import RequestServiceError, fulfil_request
|
||||
from shelfmark.metadata_providers import (
|
||||
get_provider,
|
||||
get_provider_kwargs,
|
||||
is_provider_registered,
|
||||
)
|
||||
|
||||
request_id = int(request_row["id"])
|
||||
book_data = request_row.get("book_data") or {}
|
||||
if not isinstance(book_data, dict):
|
||||
return AutoDownloadOutcome(request_id, "skipped", "no book_data")
|
||||
|
||||
provider_name = book_data.get("provider")
|
||||
provider_id = book_data.get("provider_id") or book_data.get("id")
|
||||
if not provider_name or not provider_id or not is_provider_registered(provider_name):
|
||||
return AutoDownloadOutcome(request_id, "skipped", "no usable provider/id")
|
||||
|
||||
try:
|
||||
prov = get_provider(provider_name, **get_provider_kwargs(provider_name))
|
||||
book = prov.get_book(str(provider_id))
|
||||
except Exception as exc: # noqa: BLE001 - provider/network errors shouldn't kill the loop
|
||||
logger.warning("auto-download: metadata lookup failed for request %s: %s", request_id, exc)
|
||||
return AutoDownloadOutcome(request_id, "error", f"metadata lookup failed: {exc}")
|
||||
|
||||
if book is None:
|
||||
return AutoDownloadOutcome(request_id, "skipped", "book not found in provider")
|
||||
|
||||
# Final guard: skip if the book is already in a library that is checked for this content type.
|
||||
from shelfmark.core.library_index import is_in_library
|
||||
|
||||
if is_in_library(book, content_type):
|
||||
logger.info(
|
||||
"auto-download: request %s (%s) already in library; skipping",
|
||||
request_id,
|
||||
book.title,
|
||||
)
|
||||
return AutoDownloadOutcome(request_id, "in_library", "already in library")
|
||||
|
||||
audiobook_formats = _audiobook_formats()
|
||||
|
||||
# Walk sources in priority order; take the first source with a strict match.
|
||||
for source_name in sources:
|
||||
_all, by_source, _errors = search_book_releases(
|
||||
book,
|
||||
sources=[source_name],
|
||||
content_type=content_type,
|
||||
expand_search=True,
|
||||
)
|
||||
candidates = [
|
||||
release
|
||||
for release in by_source.get(source_name, [])
|
||||
if strict_match(
|
||||
release,
|
||||
book,
|
||||
min_seeders=min_seeders,
|
||||
audiobook_formats=audiobook_formats,
|
||||
)
|
||||
]
|
||||
chosen = pick_best_release(candidates)
|
||||
if chosen is None:
|
||||
continue
|
||||
|
||||
release_data = build_release_data(chosen, book, content_type)
|
||||
try:
|
||||
fulfil_request(
|
||||
user_db,
|
||||
request_id=request_id,
|
||||
admin_user_id=admin_user_id,
|
||||
queue_release=queue_release,
|
||||
release_data=release_data,
|
||||
)
|
||||
except RequestServiceError as exc:
|
||||
logger.warning("auto-download: fulfil failed for request %s: %s", request_id, exc)
|
||||
return AutoDownloadOutcome(request_id, "error", str(exc), source=source_name)
|
||||
logger.info(
|
||||
"auto-download: queued request %s from %s (%s)",
|
||||
request_id,
|
||||
source_name,
|
||||
chosen.title,
|
||||
)
|
||||
return AutoDownloadOutcome(request_id, "queued", chosen.title, source=source_name)
|
||||
|
||||
logger.info("auto-download: no strict match for request %s (%s)", request_id, book.title)
|
||||
return AutoDownloadOutcome(request_id, "no_match", "no strict match across sources")
|
||||
|
||||
|
||||
def auto_download_pending(
|
||||
user_db: UserDB,
|
||||
*,
|
||||
queue_release: Callable[..., tuple[bool, str | None]],
|
||||
provider_filter: str | None = "hardcover",
|
||||
admin_user_id: int | None = None,
|
||||
) -> dict[str, int]:
|
||||
"""Run the auto-download pass over all eligible pending requests.
|
||||
|
||||
Returns a summary count dict. No-ops (returns zeros) unless AUTO_DOWNLOAD_ENABLED.
|
||||
``admin_user_id`` defaults to the lowest-id admin, who fulfils the requests.
|
||||
"""
|
||||
if not bool(app_config.get("AUTO_DOWNLOAD_ENABLED", False)):
|
||||
return {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
|
||||
|
||||
if admin_user_id is None:
|
||||
from shelfmark.core.hardcover_sync import resolve_request_owner
|
||||
|
||||
admin_user_id = resolve_request_owner(user_db)
|
||||
if admin_user_id is None:
|
||||
logger.warning("auto-download: no admin user exists to fulfil requests")
|
||||
return {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
|
||||
|
||||
content_type = str(app_config.get("HARDCOVER_SYNC_CONTENT_TYPE", "audiobook") or "audiobook")
|
||||
min_seeders = coerce_int(app_config.get("AUTO_DOWNLOAD_MIN_SEEDERS", 1), 1)
|
||||
|
||||
sources = _configured_source_priority()
|
||||
if not sources:
|
||||
logger.warning("auto-download: no usable release sources configured")
|
||||
return {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
|
||||
|
||||
pending = user_db.list_requests(status="pending")
|
||||
summary = {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
|
||||
|
||||
for row in pending:
|
||||
book_data = row.get("book_data") or {}
|
||||
if provider_filter and (
|
||||
not isinstance(book_data, dict) or book_data.get("provider") != provider_filter
|
||||
):
|
||||
continue
|
||||
# Only act on requests that haven't already been dispatched.
|
||||
if str(row.get("delivery_state") or "none").lower() not in {"none", ""}:
|
||||
continue
|
||||
|
||||
outcome = auto_download_request(
|
||||
user_db,
|
||||
row,
|
||||
sources=sources,
|
||||
content_type=content_type,
|
||||
min_seeders=min_seeders,
|
||||
queue_release=queue_release,
|
||||
admin_user_id=admin_user_id,
|
||||
)
|
||||
summary[outcome.status] = summary.get(outcome.status, 0) + 1
|
||||
|
||||
logger.info("auto-download pass complete: %s", summary)
|
||||
return summary
|
||||
@@ -0,0 +1,138 @@
|
||||
"""Background scheduler for Hardcover wishlist sync + auto-download.
|
||||
|
||||
Runs an in-process daemon thread (mirroring the download coordinator pattern in
|
||||
``shelfmark.download.orchestrator``) that periodically syncs configured Hardcover
|
||||
shelves into requests and auto-downloads them. Also exposes a manual trigger used
|
||||
by the "Sync now" settings action button.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from shelfmark.core.config import config as app_config
|
||||
from shelfmark.core.logger import setup_logger
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
from shelfmark.core.user_db import UserDB
|
||||
|
||||
logger = setup_logger(__name__)
|
||||
|
||||
_WARMUP_SECONDS = 60 # Delay before the first cycle after startup.
|
||||
_MIN_INTERVAL_SECONDS = 60 # Floor so a misconfigured interval can't hammer the API.
|
||||
_UNIT_SECONDS = {"minutes": 60.0, "hours": 3600.0}
|
||||
|
||||
_ctx_lock = threading.Lock()
|
||||
_ctx: dict[str, Any] = {}
|
||||
|
||||
_thread: threading.Thread | None = None
|
||||
_thread_lock = threading.Lock()
|
||||
|
||||
# Prevents a scheduled cycle and a manual "Sync now" from overlapping.
|
||||
_run_lock = threading.Lock()
|
||||
|
||||
|
||||
def configure(
|
||||
user_db: UserDB,
|
||||
queue_release: Callable[..., tuple[bool, str | None]],
|
||||
db_path: str | None,
|
||||
) -> None:
|
||||
"""Store the runtime context the background work needs."""
|
||||
with _ctx_lock:
|
||||
_ctx["user_db"] = user_db
|
||||
_ctx["queue_release"] = queue_release
|
||||
_ctx["db_path"] = db_path
|
||||
|
||||
|
||||
def _interval_seconds() -> float:
|
||||
raw = app_config.get("HARDCOVER_SYNC_INTERVAL", 6)
|
||||
try:
|
||||
value = float(raw) if isinstance(raw, (int, float, str)) else 6.0
|
||||
except ValueError:
|
||||
value = 6.0
|
||||
unit = str(app_config.get("HARDCOVER_SYNC_INTERVAL_UNIT", "hours") or "hours").strip().lower()
|
||||
multiplier = _UNIT_SECONDS.get(unit, _UNIT_SECONDS["hours"])
|
||||
return max(value * multiplier, _MIN_INTERVAL_SECONDS)
|
||||
|
||||
|
||||
def _enabled() -> bool:
|
||||
return bool(app_config.get("HARDCOVER_SYNC_ENABLED", False)) or bool(
|
||||
app_config.get("AUTO_DOWNLOAD_ENABLED", False)
|
||||
)
|
||||
|
||||
|
||||
def run_once(*, force: bool = False) -> dict[str, Any]:
|
||||
"""Run one sync + auto-download pass. Returns a combined summary.
|
||||
|
||||
``force=True`` (the manual "Sync now" path) syncs regardless of the scheduler
|
||||
toggle; auto-download always remains gated by ``AUTO_DOWNLOAD_ENABLED`` inside
|
||||
``auto_download_pending``. Skips (returns ``{"status": "busy"}``) if another run
|
||||
is in progress.
|
||||
"""
|
||||
if not _run_lock.acquire(blocking=False):
|
||||
return {"status": "busy"}
|
||||
try:
|
||||
with _ctx_lock:
|
||||
user_db = _ctx.get("user_db")
|
||||
queue_release = _ctx.get("queue_release")
|
||||
db_path = _ctx.get("db_path")
|
||||
|
||||
if user_db is None or queue_release is None:
|
||||
return {"status": "unconfigured"}
|
||||
|
||||
from shelfmark.core.auto_download import auto_download_pending
|
||||
from shelfmark.core.hardcover_sync import sync_wishlist
|
||||
|
||||
sync_summary: dict[str, Any] | None = None
|
||||
if force or bool(app_config.get("HARDCOVER_SYNC_ENABLED", False)):
|
||||
sync_summary = sync_wishlist(user_db, db_path=db_path)
|
||||
auto_summary = auto_download_pending(user_db, queue_release=queue_release)
|
||||
return {"status": "ok", "sync": sync_summary, "auto_download": auto_summary}
|
||||
finally:
|
||||
_run_lock.release()
|
||||
|
||||
|
||||
def trigger_async(*, force: bool = True) -> bool:
|
||||
"""Kick off a one-off run in a background thread. Returns False if one is active."""
|
||||
if _run_lock.locked():
|
||||
return False
|
||||
thread = threading.Thread(
|
||||
target=lambda: run_once(force=force), daemon=True, name="HardcoverSyncManual"
|
||||
)
|
||||
thread.start()
|
||||
return True
|
||||
|
||||
|
||||
def _scheduler_loop() -> None:
|
||||
logger.info("Hardcover scheduler started")
|
||||
time.sleep(_WARMUP_SECONDS)
|
||||
while True:
|
||||
try:
|
||||
if _enabled():
|
||||
logger.info("Hardcover scheduler: starting cycle")
|
||||
result = run_once()
|
||||
logger.info("Hardcover scheduler: cycle result %s", result)
|
||||
except Exception:
|
||||
logger.exception("Hardcover scheduler: cycle failed")
|
||||
time.sleep(_interval_seconds())
|
||||
|
||||
|
||||
def start(
|
||||
user_db: UserDB,
|
||||
queue_release: Callable[..., tuple[bool, str | None]],
|
||||
db_path: str | None,
|
||||
) -> None:
|
||||
"""Configure context and start the scheduler thread (idempotent)."""
|
||||
configure(user_db, queue_release, db_path)
|
||||
global _thread
|
||||
with _thread_lock:
|
||||
if _thread is not None and _thread.is_alive():
|
||||
logger.debug("Hardcover scheduler already running")
|
||||
return
|
||||
_thread = threading.Thread(target=_scheduler_loop, daemon=True, name="HardcoverScheduler")
|
||||
_thread.start()
|
||||
logger.info("Hardcover scheduler thread launched")
|
||||
@@ -0,0 +1,225 @@
|
||||
"""Sync a Hardcover reading shelf (e.g. "Want to Read") into Shelfmark requests.
|
||||
|
||||
This replaces the standalone ``scripts/sync_hardcover_wishlist.py`` with an in-app
|
||||
service that reuses the registered Hardcover metadata provider (so synced requests
|
||||
carry full metadata incl. covers) and the request-service validation/dedup path.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from shelfmark.core.config import config as app_config
|
||||
from shelfmark.core.logger import setup_logger
|
||||
from shelfmark.core.utils import transform_cover_url
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from shelfmark.core.user_db import UserDB
|
||||
from shelfmark.metadata_providers import BookMetadata
|
||||
|
||||
logger = setup_logger(__name__)
|
||||
|
||||
# Hardcover status_id -> shelf. Default sync target is "Want to Read" (1).
|
||||
DEFAULT_SYNC_STATUSES = ("1",)
|
||||
_PAGE_LIMIT = 25 # Hardcover API page size.
|
||||
_MAX_PAGES = 40 # Safety bound (~1000 books) so a bad response can't loop forever.
|
||||
|
||||
|
||||
def _configured_token() -> str:
|
||||
"""Prefer a dedicated sync token, fall back to the provider API key."""
|
||||
token = app_config.get("HARDCOVER_SYNC_TOKEN", "") or app_config.get("HARDCOVER_API_KEY", "")
|
||||
return str(token or "").strip()
|
||||
|
||||
|
||||
def _configured_statuses() -> list[int]:
|
||||
raw = app_config.get("HARDCOVER_SYNC_STATUSES", list(DEFAULT_SYNC_STATUSES))
|
||||
values: list[object]
|
||||
if isinstance(raw, str):
|
||||
values = [raw]
|
||||
elif isinstance(raw, (list, tuple)):
|
||||
values = list(raw)
|
||||
else:
|
||||
values = []
|
||||
statuses: list[int] = []
|
||||
for value in values:
|
||||
try:
|
||||
statuses.append(int(str(value).strip()))
|
||||
except ValueError:
|
||||
continue
|
||||
return statuses or [int(s) for s in DEFAULT_SYNC_STATUSES]
|
||||
|
||||
|
||||
def _build_provider() -> Any | None:
|
||||
from shelfmark.metadata_providers import get_provider
|
||||
|
||||
token = _configured_token()
|
||||
if not token:
|
||||
logger.warning("hardcover-sync: no Hardcover token configured")
|
||||
return None
|
||||
provider = get_provider("hardcover", api_key=token)
|
||||
if not provider.is_available():
|
||||
logger.warning("hardcover-sync: Hardcover provider unavailable (bad token?)")
|
||||
return None
|
||||
return provider
|
||||
|
||||
|
||||
def resolve_request_owner(user_db: UserDB) -> int | None:
|
||||
"""Return the lowest-id admin, who owns synced requests and approves auto-downloads."""
|
||||
admin = user_db.get_first_admin()
|
||||
return int(admin["id"]) if admin else None
|
||||
|
||||
|
||||
def _primary_author(book: BookMetadata) -> str:
|
||||
if book.search_author:
|
||||
return book.search_author
|
||||
if book.authors:
|
||||
return book.authors[0]
|
||||
return "Unknown"
|
||||
|
||||
|
||||
def _book_to_book_data(book: BookMetadata, content_type: str) -> dict[str, Any]:
|
||||
book_data: dict[str, Any] = {
|
||||
"title": book.title,
|
||||
"author": _primary_author(book),
|
||||
"provider": "hardcover",
|
||||
"provider_id": str(book.provider_id),
|
||||
"content_type": content_type,
|
||||
}
|
||||
# Proxy the cover the same way search results are, so the row is ready for the UI.
|
||||
if book.cover_url:
|
||||
book_data["preview"] = transform_cover_url(book.cover_url, f"hardcover_{book.provider_id}")
|
||||
if book.publish_year:
|
||||
book_data["year"] = book.publish_year
|
||||
if book.subtitle:
|
||||
book_data["subtitle"] = book.subtitle
|
||||
return book_data
|
||||
|
||||
|
||||
def _existing_provider_ids(user_db: UserDB) -> set[str]:
|
||||
"""Hardcover ids already requested, in any status. Other providers' ids are not counted."""
|
||||
ids: set[str] = set()
|
||||
for row in user_db.list_requests():
|
||||
book_data = row.get("book_data") or {}
|
||||
if isinstance(book_data, dict) and book_data.get("provider") == "hardcover":
|
||||
pid = book_data.get("provider_id") or book_data.get("id")
|
||||
if pid is not None:
|
||||
ids.add(str(pid))
|
||||
return ids
|
||||
|
||||
|
||||
def _already_downloaded(db_path: str | None, title: str, author: str) -> bool:
|
||||
"""Best-effort check that a title/author isn't already in download_history."""
|
||||
if not db_path:
|
||||
return False
|
||||
try:
|
||||
conn = sqlite3.connect(db_path)
|
||||
try:
|
||||
row = conn.execute(
|
||||
"SELECT 1 FROM download_history "
|
||||
"WHERE LOWER(title) = ? AND LOWER(author) = ? LIMIT 1",
|
||||
(title.strip().lower(), author.strip().lower()),
|
||||
).fetchone()
|
||||
return row is not None
|
||||
finally:
|
||||
conn.close()
|
||||
except sqlite3.Error as exc:
|
||||
logger.debug("hardcover-sync: history dedup query failed: %s", exc)
|
||||
return False
|
||||
|
||||
|
||||
def _fetch_status_books(provider: Any, status_id: int) -> list[BookMetadata]:
|
||||
"""Page through a Hardcover status shelf, returning all books."""
|
||||
books: list[BookMetadata] = []
|
||||
for page in range(1, _MAX_PAGES + 1):
|
||||
try:
|
||||
result = provider._fetch_current_user_books_by_status(status_id, page, _PAGE_LIMIT)
|
||||
except Exception as exc: # noqa: BLE001 - network/provider errors shouldn't abort the sync
|
||||
logger.warning(
|
||||
"hardcover-sync: fetch failed (status=%s page=%s): %s", status_id, page, exc
|
||||
)
|
||||
break
|
||||
page_books = list(result.books or [])
|
||||
books.extend(page_books)
|
||||
if not result.has_more or not page_books:
|
||||
break
|
||||
return books
|
||||
|
||||
|
||||
def sync_wishlist(
|
||||
user_db: UserDB,
|
||||
*,
|
||||
db_path: str | None = None,
|
||||
user_id: int | None = None,
|
||||
) -> dict[str, int]:
|
||||
"""Sync configured Hardcover shelves into pending requests.
|
||||
|
||||
Returns a summary dict: ``{"added", "skipped", "in_library", "errors"}``. Requires a
|
||||
configured token; enable-gating is the caller's responsibility (see hardcover_scheduler).
|
||||
"""
|
||||
summary = {"added": 0, "skipped": 0, "in_library": 0, "errors": 0}
|
||||
|
||||
if user_id is None:
|
||||
user_id = resolve_request_owner(user_db)
|
||||
if user_id is None:
|
||||
logger.warning("hardcover-sync: no admin user exists to own synced requests")
|
||||
summary["errors"] += 1
|
||||
return summary
|
||||
|
||||
provider = _build_provider()
|
||||
if provider is None:
|
||||
summary["errors"] += 1
|
||||
return summary
|
||||
|
||||
content_type = str(app_config.get("HARDCOVER_SYNC_CONTENT_TYPE", "audiobook") or "audiobook")
|
||||
known_provider_ids = _existing_provider_ids(user_db)
|
||||
|
||||
from shelfmark.core.library_index import is_in_library
|
||||
from shelfmark.core.requests_service import RequestServiceError, create_request
|
||||
|
||||
for status_id in _configured_statuses():
|
||||
for book in _fetch_status_books(provider, status_id):
|
||||
provider_id = str(book.provider_id)
|
||||
author = _primary_author(book)
|
||||
|
||||
if provider_id in known_provider_ids:
|
||||
summary["skipped"] += 1
|
||||
continue
|
||||
if _already_downloaded(db_path, book.title, author):
|
||||
summary["skipped"] += 1
|
||||
continue
|
||||
if is_in_library(book, content_type):
|
||||
summary["in_library"] += 1
|
||||
logger.info("hardcover-sync: '%s' already in your library; skipping", book.title)
|
||||
continue
|
||||
|
||||
try:
|
||||
create_request(
|
||||
user_db,
|
||||
user_id=user_id,
|
||||
source_hint=None,
|
||||
content_type=content_type,
|
||||
request_level="book",
|
||||
policy_mode="request_book",
|
||||
book_data=_book_to_book_data(book, content_type),
|
||||
note=None,
|
||||
)
|
||||
except RequestServiceError as exc:
|
||||
# Duplicate / max-pending / validation: treat as skip, not failure.
|
||||
if exc.code in {"duplicate_pending_request", "max_pending_reached"}:
|
||||
summary["skipped"] += 1
|
||||
else:
|
||||
logger.warning("hardcover-sync: could not add '%s': %s", book.title, exc)
|
||||
summary["errors"] += 1
|
||||
continue
|
||||
except Exception:
|
||||
logger.exception("hardcover-sync: unexpected error adding '%s'", book.title)
|
||||
summary["errors"] += 1
|
||||
continue
|
||||
|
||||
known_provider_ids.add(provider_id)
|
||||
summary["added"] += 1
|
||||
logger.info("hardcover-sync: added request for '%s' by %s", book.title, author)
|
||||
|
||||
logger.info("hardcover-sync complete: %s", summary)
|
||||
return summary
|
||||
@@ -92,3 +92,41 @@ def search_source_releases(
|
||||
return None, [], f"{source_name}: {exc!s}"
|
||||
else:
|
||||
return source, releases, None
|
||||
|
||||
|
||||
def search_book_releases(
|
||||
book: BookMetadata,
|
||||
*,
|
||||
sources: list[str],
|
||||
content_type: str = "ebook",
|
||||
expand_search: bool = True,
|
||||
languages: list[str] | None = None,
|
||||
indexers: list[str] | None = None,
|
||||
) -> tuple[list[Release], dict[str, list[Release]], list[str]]:
|
||||
"""Search several sources for releases of ``book``.
|
||||
|
||||
Returns ``(all_releases, releases_by_source, errors)``. ``releases_by_source``
|
||||
preserves the requested ``sources`` order so callers can prioritise.
|
||||
"""
|
||||
all_releases: list[Release] = []
|
||||
by_source: dict[str, list[Release]] = {}
|
||||
errors: list[str] = []
|
||||
|
||||
for source_name in sources:
|
||||
source, releases, error = search_source_releases(
|
||||
source_name,
|
||||
book,
|
||||
languages=languages,
|
||||
manual_query=None,
|
||||
indexers=indexers,
|
||||
expand_search=expand_search,
|
||||
content_type=content_type,
|
||||
source_filters=None,
|
||||
)
|
||||
if source is not None:
|
||||
by_source[source_name] = list(releases)
|
||||
all_releases.extend(releases)
|
||||
if error is not None:
|
||||
errors.append(error)
|
||||
|
||||
return all_releases, by_source, errors
|
||||
@@ -291,6 +291,15 @@ if user_db is not None and download_history_service is not None:
|
||||
# Start download coordinator
|
||||
backend.start()
|
||||
|
||||
# Start the Hardcover wishlist sync + auto-download scheduler (in-process daemon).
|
||||
if user_db is not None:
|
||||
try:
|
||||
from shelfmark.core import hardcover_scheduler
|
||||
|
||||
hardcover_scheduler.start(user_db, backend.queue_release, _user_db_path)
|
||||
except (ImportError, RuntimeError, OSError) as exc:
|
||||
logger.warning("Failed to start Hardcover scheduler: %s", exc)
|
||||
|
||||
# Pre-solve the direct-download source's protection challenge in the background so the
|
||||
# first user search does not pay for a cold Chrome bypass. Never blocks startup.
|
||||
warmup.start()
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
"""Tests for the Hardcover Sync settings tab registration and action buttons."""
|
||||
|
||||
import shelfmark.config.hardcover_sync_settings # noqa: F401 - registers the tab
|
||||
from shelfmark.core import settings_registry
|
||||
|
||||
|
||||
def _field_map():
|
||||
tab = settings_registry.get_settings_tab("hardcover_sync")
|
||||
assert tab is not None
|
||||
return {field.key: field for field in tab.fields if hasattr(field, "key")}
|
||||
|
||||
|
||||
def test_tab_is_registered():
|
||||
tab = settings_registry.get_settings_tab("hardcover_sync")
|
||||
|
||||
assert tab is not None
|
||||
assert tab.display_name == "Hardcover Sync"
|
||||
|
||||
|
||||
def test_everything_is_off_by_default():
|
||||
fields = _field_map()
|
||||
|
||||
assert fields["HARDCOVER_SYNC_ENABLED"].default is False
|
||||
assert fields["AUTO_DOWNLOAD_ENABLED"].default is False
|
||||
|
||||
|
||||
def test_schedule_and_content_type_defaults():
|
||||
fields = _field_map()
|
||||
|
||||
assert fields["HARDCOVER_SYNC_INTERVAL"].default == 6
|
||||
assert fields["HARDCOVER_SYNC_INTERVAL"].min_value == 1
|
||||
assert fields["HARDCOVER_SYNC_INTERVAL_UNIT"].default == "hours"
|
||||
assert fields["HARDCOVER_SYNC_CONTENT_TYPE"].default == "audiobook"
|
||||
assert fields["HARDCOVER_SYNC_STATUSES"].default == "1"
|
||||
assert fields["AUTO_DOWNLOAD_SOURCE_PRIORITY"].default == []
|
||||
|
||||
|
||||
def test_shelf_options_come_from_hardcover_statuses():
|
||||
options = _field_map()["HARDCOVER_SYNC_STATUSES"].options()
|
||||
|
||||
assert {"value": "1", "label": "Want to Read"} in options
|
||||
|
||||
|
||||
def test_action_buttons_are_registered():
|
||||
fields = _field_map()
|
||||
|
||||
assert fields["sync_now"].callback is not None
|
||||
|
||||
|
||||
def test_sync_now_reports_busy_when_a_run_is_active(monkeypatch):
|
||||
monkeypatch.setattr("shelfmark.core.hardcover_scheduler.trigger_async", lambda **_: False)
|
||||
|
||||
result = settings_registry.execute_action("hardcover_sync", "sync_now")
|
||||
|
||||
assert result["success"] is False
|
||||
assert "already running" in result["message"]
|
||||
|
||||
|
||||
def test_sync_now_starts_a_forced_run(monkeypatch):
|
||||
seen = []
|
||||
|
||||
def _trigger(**kwargs):
|
||||
seen.append(kwargs)
|
||||
return True
|
||||
|
||||
monkeypatch.setattr("shelfmark.core.hardcover_scheduler.trigger_async", _trigger)
|
||||
|
||||
result = settings_registry.execute_action("hardcover_sync", "sync_now")
|
||||
|
||||
assert result["success"] is True
|
||||
assert seen == [{"force": True}]
|
||||
@@ -0,0 +1,401 @@
|
||||
"""Tests for strict release matching and auto-download of pending requests."""
|
||||
|
||||
import pytest
|
||||
|
||||
from shelfmark.core import auto_download
|
||||
from shelfmark.core.user_db import UserDB
|
||||
from shelfmark.metadata_providers import BookMetadata
|
||||
from shelfmark.release_sources import Release, ReleaseProtocol
|
||||
|
||||
AUDIOBOOK_FORMATS = {"m4b", "mp3"}
|
||||
|
||||
|
||||
class _Config:
|
||||
def __init__(self, **values):
|
||||
self.values = values
|
||||
|
||||
def get(self, key, default=None, user_id=None):
|
||||
return self.values.get(key, default)
|
||||
|
||||
|
||||
def _book(**overrides):
|
||||
fields = {
|
||||
"provider": "hardcover",
|
||||
"provider_id": "42",
|
||||
"title": "Dungeon Crawler Carl",
|
||||
"authors": ["Matt Dinniman"],
|
||||
}
|
||||
fields.update(overrides)
|
||||
return BookMetadata(**fields)
|
||||
|
||||
|
||||
def _release(**overrides):
|
||||
fields = {
|
||||
"source": "prowlarr",
|
||||
"source_id": "rel-1",
|
||||
"title": "Dungeon Crawler Carl - Matt Dinniman [M4B]",
|
||||
"format": "m4b",
|
||||
"protocol": ReleaseProtocol.TORRENT,
|
||||
"seeders": 10,
|
||||
}
|
||||
fields.update(overrides)
|
||||
return Release(**fields)
|
||||
|
||||
|
||||
def _strict(release, **kwargs):
|
||||
kwargs.setdefault("audiobook_formats", AUDIOBOOK_FORMATS)
|
||||
return auto_download.strict_match(release, _book(), **kwargs)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def user_db(tmp_path):
|
||||
db = UserDB(str(tmp_path / "users.db"))
|
||||
db.initialize()
|
||||
return db
|
||||
|
||||
|
||||
def _pending_request(user_db, user_id, **book_data_overrides):
|
||||
book_data = {
|
||||
"title": "Dungeon Crawler Carl",
|
||||
"author": "Matt Dinniman",
|
||||
"provider": "hardcover",
|
||||
"provider_id": "42",
|
||||
"content_type": "audiobook",
|
||||
}
|
||||
book_data.update(book_data_overrides)
|
||||
return user_db.create_request(
|
||||
user_id=user_id,
|
||||
content_type="audiobook",
|
||||
request_level="book",
|
||||
policy_mode="request_book",
|
||||
book_data=book_data,
|
||||
)
|
||||
|
||||
|
||||
def _stub_provider(monkeypatch, book):
|
||||
class _Provider:
|
||||
def get_book(self, book_id):
|
||||
assert book_id == "42"
|
||||
return book
|
||||
|
||||
monkeypatch.setattr(
|
||||
"shelfmark.metadata_providers.is_provider_registered", lambda name: name == "hardcover"
|
||||
)
|
||||
monkeypatch.setattr("shelfmark.metadata_providers.get_provider_kwargs", lambda _name: {})
|
||||
monkeypatch.setattr(
|
||||
"shelfmark.metadata_providers.get_provider", lambda _name, **_kwargs: _Provider()
|
||||
)
|
||||
|
||||
|
||||
class TestStrictMatch:
|
||||
def test_matching_release_passes(self):
|
||||
assert _strict(_release()) is True
|
||||
|
||||
def test_title_mismatch_rejected(self):
|
||||
assert _strict(_release(title="Some Other Book - Matt Dinniman [M4B]")) is False
|
||||
|
||||
def test_missing_author_surname_rejected(self):
|
||||
assert _strict(_release(title="Dungeon Crawler Carl [M4B]", indexer=None)) is False
|
||||
|
||||
def test_author_found_in_extra_metadata(self):
|
||||
release = _release(title="Dungeon Crawler Carl [M4B]", extra={"author": "Matt Dinniman"})
|
||||
assert _strict(release) is True
|
||||
|
||||
def test_ebook_only_release_rejected(self):
|
||||
release = _release(title="Dungeon Crawler Carl - Matt Dinniman", format="epub")
|
||||
assert _strict(release) is False
|
||||
|
||||
def test_audiobook_signal_in_title_accepted_without_format(self):
|
||||
release = _release(title="Dungeon Crawler Carl by Matt Dinniman (Unabridged)", format=None)
|
||||
assert _strict(release) is True
|
||||
|
||||
@pytest.mark.parametrize("seeders", [0, None])
|
||||
def test_torrent_below_minimum_seeders_rejected(self, seeders):
|
||||
assert _strict(_release(seeders=seeders), min_seeders=1) is False
|
||||
|
||||
def test_torrent_without_seeder_count_allowed_when_minimum_is_zero(self):
|
||||
assert _strict(_release(seeders=None), min_seeders=0) is True
|
||||
|
||||
def test_non_torrent_ignores_seeders(self):
|
||||
release = _release(protocol=ReleaseProtocol.NZB, seeders=None)
|
||||
assert _strict(release, min_seeders=5) is True
|
||||
|
||||
|
||||
class TestPickBestRelease:
|
||||
def test_empty_returns_none(self):
|
||||
assert auto_download.pick_best_release([]) is None
|
||||
|
||||
def test_m4b_outranks_mp3_regardless_of_seeders(self):
|
||||
mp3 = _release(source_id="mp3", format="mp3", seeders=500)
|
||||
m4b = _release(source_id="m4b", format="m4b", seeders=1)
|
||||
assert auto_download.pick_best_release([mp3, m4b]) is m4b
|
||||
|
||||
def test_m4b_in_title_ranks_as_m4b_when_format_missing(self):
|
||||
mp3 = _release(source_id="mp3", format="mp3", seeders=500)
|
||||
untagged = _release(source_id="untagged", format=None, seeders=1)
|
||||
assert auto_download.pick_best_release([mp3, untagged]) is untagged
|
||||
|
||||
def test_seeders_break_format_ties(self):
|
||||
few = _release(source_id="few", seeders=2, size_bytes=10)
|
||||
many = _release(source_id="many", seeders=20, size_bytes=1)
|
||||
assert auto_download.pick_best_release([few, many]) is many
|
||||
|
||||
def test_size_breaks_seeder_ties(self):
|
||||
small = _release(source_id="small", seeders=5, size_bytes=100)
|
||||
large = _release(source_id="large", seeders=5, size_bytes=200)
|
||||
assert auto_download.pick_best_release([small, large]) is large
|
||||
|
||||
|
||||
class TestBuildReleaseData:
|
||||
def test_drops_none_values(self):
|
||||
release = _release(size=None, size_bytes=None, language=None, info_url=None)
|
||||
|
||||
data = auto_download.build_release_data(release, _book(), "audiobook")
|
||||
|
||||
assert data["source"] == "prowlarr"
|
||||
assert data["source_id"] == "rel-1"
|
||||
assert data["protocol"] == "torrent"
|
||||
assert data["author"] == "Matt Dinniman"
|
||||
assert data["content_type"] == "audiobook"
|
||||
assert not {"size", "size_bytes", "language", "info_url", "preview", "year"} & data.keys()
|
||||
|
||||
def test_carries_cover_series_and_book_metadata(self):
|
||||
book = _book(
|
||||
cover_url="https://img.example/cover.jpg",
|
||||
publish_year=2020,
|
||||
subtitle="Book One",
|
||||
series_name="Dungeon Crawler Carl",
|
||||
series_position=1.0,
|
||||
)
|
||||
release = _release(content_type=None, extra={"preview": "https://img.example/rel.jpg"})
|
||||
|
||||
data = auto_download.build_release_data(release, book, "audiobook")
|
||||
|
||||
assert data["preview"] == "https://img.example/cover.jpg"
|
||||
assert data["year"] == 2020
|
||||
assert data["subtitle"] == "Book One"
|
||||
assert data["series_name"] == "Dungeon Crawler Carl"
|
||||
assert data["series_position"] == 1.0
|
||||
assert data["content_type"] == "audiobook"
|
||||
assert data["extra"] == {"preview": "https://img.example/rel.jpg"}
|
||||
|
||||
def test_falls_back_to_release_preview_without_book_cover(self):
|
||||
release = _release(extra={"preview": "https://img.example/rel.jpg"})
|
||||
|
||||
data = auto_download.build_release_data(release, _book(), "audiobook")
|
||||
|
||||
assert data["preview"] == "https://img.example/rel.jpg"
|
||||
|
||||
|
||||
class TestConfiguredSourcePriority:
|
||||
@pytest.fixture(autouse=True)
|
||||
def _available_sources(self, monkeypatch):
|
||||
monkeypatch.setattr(
|
||||
"shelfmark.release_sources.list_available_sources",
|
||||
lambda: [
|
||||
{"name": "first", "enabled": True},
|
||||
{"name": "second", "enabled": True},
|
||||
{"name": "unavailable", "enabled": False},
|
||||
],
|
||||
)
|
||||
|
||||
def _configure(self, monkeypatch, priority):
|
||||
monkeypatch.setattr(
|
||||
auto_download, "app_config", _Config(AUTO_DOWNLOAD_SOURCE_PRIORITY=priority)
|
||||
)
|
||||
|
||||
def test_respects_configured_order_and_skips_unusable_entries(self, monkeypatch):
|
||||
self._configure(
|
||||
monkeypatch,
|
||||
[
|
||||
{"id": "second"},
|
||||
{"id": "first", "enabled": False},
|
||||
{"id": "unavailable"},
|
||||
{"id": "unknown"},
|
||||
"not-a-dict",
|
||||
],
|
||||
)
|
||||
|
||||
assert auto_download._configured_source_priority() == ["second"]
|
||||
|
||||
@pytest.mark.parametrize("priority", [[], "garbage", ["not-a-dict"]])
|
||||
def test_falls_back_to_all_enabled_sources_when_nothing_is_configured(
|
||||
self, monkeypatch, priority
|
||||
):
|
||||
self._configure(monkeypatch, priority)
|
||||
|
||||
assert auto_download._configured_source_priority() == ["first", "second"]
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"priority",
|
||||
[
|
||||
[{"id": "first", "enabled": False}, {"id": "second", "enabled": False}],
|
||||
[{"id": "unavailable"}],
|
||||
[{"id": "unknown"}],
|
||||
],
|
||||
)
|
||||
def test_a_configured_list_with_nothing_usable_searches_nothing(self, monkeypatch, priority):
|
||||
self._configure(monkeypatch, priority)
|
||||
|
||||
assert auto_download._configured_source_priority() == []
|
||||
|
||||
|
||||
class TestAutoDownloadPending:
|
||||
def test_disabled_returns_zeros_without_touching_the_db(self, monkeypatch):
|
||||
monkeypatch.setattr(auto_download, "app_config", _Config(AUTO_DOWNLOAD_ENABLED=False))
|
||||
|
||||
class _UntouchableDb:
|
||||
def list_requests(self, **_kwargs):
|
||||
raise AssertionError("pending requests must not be queried")
|
||||
|
||||
summary = auto_download.auto_download_pending(
|
||||
_UntouchableDb(), queue_release=lambda *_args, **_kwargs: (True, None)
|
||||
)
|
||||
|
||||
assert summary == {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
|
||||
|
||||
|
||||
class TestAutoDownloadRequest:
|
||||
@pytest.fixture(autouse=True)
|
||||
def _library_check_off(self, monkeypatch):
|
||||
monkeypatch.setattr(auto_download, "app_config", _Config())
|
||||
monkeypatch.setattr(
|
||||
"shelfmark.core.library_index.is_in_library", lambda *_args, **_kwargs: False
|
||||
)
|
||||
|
||||
def _run(self, user_db, row, admin, *, queue_release=None, sources=("first", "second")):
|
||||
return auto_download.auto_download_request(
|
||||
user_db,
|
||||
row,
|
||||
sources=list(sources),
|
||||
content_type="audiobook",
|
||||
min_seeders=1,
|
||||
queue_release=queue_release or (lambda *_args, **_kwargs: (True, None)),
|
||||
admin_user_id=admin["id"],
|
||||
)
|
||||
|
||||
def test_queues_first_strict_match_via_fulfil_path(self, user_db, monkeypatch):
|
||||
admin = user_db.create_user(username="admin", role="admin")
|
||||
reader = user_db.create_user(username="reader", role="user")
|
||||
row = _pending_request(user_db, reader["id"])
|
||||
_stub_provider(monkeypatch, _book(cover_url="https://img.example/cover.jpg"))
|
||||
|
||||
chosen = _release(source="second", source_id="good", seeders=5)
|
||||
searched = []
|
||||
|
||||
def _search(_book_arg, *, sources, **_kwargs):
|
||||
searched.extend(sources)
|
||||
if sources == ["first"]:
|
||||
epub = _release(source="first", title="Dungeon Crawler Carl.epub", format="epub")
|
||||
return [epub], {"first": [epub]}, []
|
||||
return [chosen], {"second": [chosen]}, []
|
||||
|
||||
monkeypatch.setattr(auto_download, "search_book_releases", _search)
|
||||
queued = []
|
||||
|
||||
def _queue(release_data, priority, *, user_id, username):
|
||||
queued.append((release_data, priority, user_id, username))
|
||||
return True, None
|
||||
|
||||
outcome = self._run(user_db, row, admin, queue_release=_queue)
|
||||
|
||||
assert outcome.status == "queued"
|
||||
assert outcome.source == "second"
|
||||
assert searched == ["first", "second"]
|
||||
|
||||
stored = user_db.get_request(row["id"])
|
||||
assert stored["status"] == "fulfilled"
|
||||
assert stored["delivery_state"] == "queued"
|
||||
assert stored["reviewed_by"] == admin["id"]
|
||||
assert stored["release_data"]["source_id"] == "good"
|
||||
|
||||
release_data, priority, user_id, username = queued[0]
|
||||
assert release_data["_request_id"] == row["id"]
|
||||
assert release_data["preview"] == "https://img.example/cover.jpg"
|
||||
assert priority == 0
|
||||
assert user_id == reader["id"]
|
||||
assert username == "reader"
|
||||
|
||||
def test_queue_failure_rolls_request_back_to_pending(self, user_db, monkeypatch):
|
||||
admin = user_db.create_user(username="admin", role="admin")
|
||||
reader = user_db.create_user(username="reader", role="user")
|
||||
row = _pending_request(user_db, reader["id"])
|
||||
_stub_provider(monkeypatch, _book())
|
||||
chosen = _release(source="first")
|
||||
monkeypatch.setattr(
|
||||
auto_download,
|
||||
"search_book_releases",
|
||||
lambda *_args, **_kwargs: ([chosen], {"first": [chosen]}, []),
|
||||
)
|
||||
|
||||
outcome = self._run(
|
||||
user_db, row, admin, queue_release=lambda *_args, **_kwargs: (False, "client down")
|
||||
)
|
||||
|
||||
assert outcome.status == "error"
|
||||
assert outcome.detail == "client down"
|
||||
stored = user_db.get_request(row["id"])
|
||||
assert stored["status"] == "pending"
|
||||
assert stored["release_data"] is None
|
||||
|
||||
def test_no_strict_match_leaves_request_pending(self, user_db, monkeypatch):
|
||||
admin = user_db.create_user(username="admin", role="admin")
|
||||
reader = user_db.create_user(username="reader", role="user")
|
||||
row = _pending_request(user_db, reader["id"])
|
||||
_stub_provider(monkeypatch, _book())
|
||||
wrong = _release(title="A Different Audiobook [M4B]")
|
||||
monkeypatch.setattr(
|
||||
auto_download,
|
||||
"search_book_releases",
|
||||
lambda *_args, **_kwargs: ([wrong], {"first": [wrong], "second": [wrong]}, []),
|
||||
)
|
||||
|
||||
def _never_queue(*_args, **_kwargs):
|
||||
raise AssertionError("nothing should be queued")
|
||||
|
||||
outcome = self._run(user_db, row, admin, queue_release=_never_queue)
|
||||
|
||||
assert outcome.status == "no_match"
|
||||
assert user_db.get_request(row["id"])["status"] == "pending"
|
||||
|
||||
def test_book_already_in_library_is_skipped_before_searching(self, user_db, monkeypatch):
|
||||
admin = user_db.create_user(username="admin", role="admin")
|
||||
reader = user_db.create_user(username="reader", role="user")
|
||||
row = _pending_request(user_db, reader["id"])
|
||||
_stub_provider(monkeypatch, _book())
|
||||
monkeypatch.setattr(auto_download, "app_config", _Config())
|
||||
monkeypatch.setattr(
|
||||
"shelfmark.core.library_index.is_in_library", lambda *_args, **_kwargs: True
|
||||
)
|
||||
|
||||
def _never_search(*_args, **_kwargs):
|
||||
raise AssertionError("release search must not run")
|
||||
|
||||
monkeypatch.setattr(auto_download, "search_book_releases", _never_search)
|
||||
|
||||
outcome = self._run(user_db, row, admin)
|
||||
|
||||
assert outcome.status == "in_library"
|
||||
assert user_db.get_request(row["id"])["status"] == "pending"
|
||||
|
||||
def test_unregistered_provider_is_skipped(self, user_db, monkeypatch):
|
||||
admin = user_db.create_user(username="admin", role="admin")
|
||||
reader = user_db.create_user(username="reader", role="user")
|
||||
row = _pending_request(user_db, reader["id"], provider="nope")
|
||||
monkeypatch.setattr("shelfmark.metadata_providers.is_provider_registered", lambda _n: False)
|
||||
|
||||
outcome = self._run(user_db, row, admin)
|
||||
|
||||
assert outcome.status == "skipped"
|
||||
assert outcome.detail == "no usable provider/id"
|
||||
|
||||
|
||||
def test_pending_pass_without_an_admin_queues_nothing(user_db, monkeypatch):
|
||||
monkeypatch.setattr(auto_download, "app_config", _Config(AUTO_DOWNLOAD_ENABLED=True))
|
||||
user_db.create_user(username="reader", role="user")
|
||||
|
||||
summary = auto_download.auto_download_pending(
|
||||
user_db, queue_release=lambda *_args, **_kwargs: (True, None)
|
||||
)
|
||||
|
||||
assert summary == {"queued": 0, "no_match": 0, "in_library": 0, "skipped": 0, "error": 0}
|
||||
@@ -0,0 +1,158 @@
|
||||
"""Tests for the Hardcover sync scheduler's interval parsing and run gating."""
|
||||
|
||||
import threading
|
||||
|
||||
import pytest
|
||||
|
||||
from shelfmark.core import hardcover_scheduler as scheduler
|
||||
|
||||
|
||||
class _Config:
|
||||
def __init__(self, **values):
|
||||
self.values = values
|
||||
|
||||
def get(self, key, default=None, user_id=None):
|
||||
return self.values.get(key, default)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _isolated_scheduler(monkeypatch):
|
||||
monkeypatch.setattr(scheduler, "_ctx", {})
|
||||
monkeypatch.setattr(scheduler, "_run_lock", threading.Lock())
|
||||
monkeypatch.setattr(scheduler, "app_config", _Config())
|
||||
|
||||
|
||||
def _configure(monkeypatch, **values):
|
||||
monkeypatch.setattr(scheduler, "app_config", _Config(**values))
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("interval", "unit", "expected"),
|
||||
[
|
||||
(30, "minutes", 1800),
|
||||
(2, "hours", 7200),
|
||||
("1.5", "hours", 5400),
|
||||
(0.5, "minutes", 60),
|
||||
(6, "fortnights", 6 * 3600),
|
||||
("garbage", "hours", 6 * 3600),
|
||||
([], "hours", 6 * 3600),
|
||||
],
|
||||
)
|
||||
def test_interval_seconds(monkeypatch, interval, unit, expected):
|
||||
_configure(monkeypatch, HARDCOVER_SYNC_INTERVAL=interval, HARDCOVER_SYNC_INTERVAL_UNIT=unit)
|
||||
|
||||
assert scheduler._interval_seconds() == expected
|
||||
|
||||
|
||||
def test_interval_seconds_defaults_to_six_hours(monkeypatch):
|
||||
_configure(monkeypatch)
|
||||
|
||||
assert scheduler._interval_seconds() == 6 * 3600
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("values", "expected"),
|
||||
[
|
||||
({}, False),
|
||||
({"HARDCOVER_SYNC_ENABLED": True}, True),
|
||||
({"AUTO_DOWNLOAD_ENABLED": True}, True),
|
||||
({"HARDCOVER_SYNC_ENABLED": False, "AUTO_DOWNLOAD_ENABLED": False}, False),
|
||||
],
|
||||
)
|
||||
def test_enabled(monkeypatch, values, expected):
|
||||
_configure(monkeypatch, **values)
|
||||
|
||||
assert scheduler._enabled() is expected
|
||||
|
||||
|
||||
def test_run_once_before_configure_is_unconfigured():
|
||||
assert scheduler.run_once() == {"status": "unconfigured"}
|
||||
assert scheduler.run_once(force=True) == {"status": "unconfigured"}
|
||||
|
||||
|
||||
def test_run_once_reports_busy_while_another_run_holds_the_lock():
|
||||
with scheduler._run_lock:
|
||||
assert scheduler.run_once(force=True) == {"status": "busy"}
|
||||
|
||||
assert scheduler.run_once() == {"status": "unconfigured"}
|
||||
|
||||
|
||||
class TestRunOnce:
|
||||
@pytest.fixture
|
||||
def calls(self, monkeypatch):
|
||||
recorded = {"sync": [], "auto": []}
|
||||
|
||||
def _sync(user_db, **kwargs):
|
||||
recorded["sync"].append((user_db, kwargs))
|
||||
return {"added": 1}
|
||||
|
||||
def _auto(user_db, **kwargs):
|
||||
recorded["auto"].append((user_db, kwargs))
|
||||
return {"queued": 1}
|
||||
|
||||
monkeypatch.setattr("shelfmark.core.hardcover_sync.sync_wishlist", _sync)
|
||||
monkeypatch.setattr("shelfmark.core.auto_download.auto_download_pending", _auto)
|
||||
return recorded
|
||||
|
||||
def test_scheduled_run_skips_sync_when_disabled(self, calls):
|
||||
user_db, queue_release = object(), object()
|
||||
scheduler.configure(user_db, queue_release, "/data/users.db")
|
||||
|
||||
result = scheduler.run_once()
|
||||
|
||||
assert result == {"status": "ok", "sync": None, "auto_download": {"queued": 1}}
|
||||
assert calls["sync"] == []
|
||||
assert calls["auto"] == [(user_db, {"queue_release": queue_release})]
|
||||
|
||||
def test_forced_run_syncs_regardless_of_toggle(self, calls):
|
||||
user_db = object()
|
||||
scheduler.configure(user_db, object(), "/data/users.db")
|
||||
|
||||
result = scheduler.run_once(force=True)
|
||||
|
||||
assert result == {"status": "ok", "sync": {"added": 1}, "auto_download": {"queued": 1}}
|
||||
assert calls["sync"] == [(user_db, {"db_path": "/data/users.db"})]
|
||||
|
||||
def test_scheduled_run_syncs_when_enabled(self, monkeypatch, calls):
|
||||
_configure(monkeypatch, HARDCOVER_SYNC_ENABLED=True)
|
||||
user_db = object()
|
||||
scheduler.configure(user_db, object(), None)
|
||||
|
||||
result = scheduler.run_once()
|
||||
|
||||
assert result["sync"] == {"added": 1}
|
||||
assert calls["sync"] == [(user_db, {"db_path": None})]
|
||||
|
||||
def test_lock_is_released_after_a_run(self, calls):
|
||||
scheduler.configure(object(), object(), None)
|
||||
|
||||
scheduler.run_once()
|
||||
|
||||
assert scheduler._run_lock.locked() is False
|
||||
|
||||
|
||||
class TestTriggerAsync:
|
||||
def test_returns_false_while_a_run_is_active(self, monkeypatch):
|
||||
def _unexpected(**_kwargs):
|
||||
raise AssertionError("run_once must not be scheduled")
|
||||
|
||||
monkeypatch.setattr(scheduler, "run_once", _unexpected)
|
||||
|
||||
with scheduler._run_lock:
|
||||
assert scheduler.trigger_async() is False
|
||||
|
||||
@pytest.mark.parametrize("force", [True, False])
|
||||
def test_runs_once_in_the_background_with_force_flag(self, monkeypatch, force):
|
||||
finished = threading.Event()
|
||||
seen = []
|
||||
|
||||
def _run_once(**kwargs):
|
||||
seen.append(kwargs)
|
||||
finished.set()
|
||||
return {"status": "ok"}
|
||||
|
||||
monkeypatch.setattr(scheduler, "run_once", _run_once)
|
||||
|
||||
assert scheduler.trigger_async(force=force) is True
|
||||
assert finished.wait(timeout=5)
|
||||
assert seen == [{"force": force}]
|
||||
@@ -0,0 +1,295 @@
|
||||
"""Tests for syncing Hardcover status shelves into pending requests."""
|
||||
|
||||
import sqlite3
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from shelfmark.core import hardcover_sync
|
||||
from shelfmark.core.user_db import UserDB
|
||||
from shelfmark.core.utils import transform_cover_url
|
||||
from shelfmark.metadata_providers import BookMetadata
|
||||
|
||||
|
||||
class _Config:
|
||||
def __init__(self, **values):
|
||||
self.values = values
|
||||
|
||||
def get(self, key, default=None, user_id=None):
|
||||
return self.values.get(key, default)
|
||||
|
||||
|
||||
class _Provider:
|
||||
"""Stub exposing the private paging method ``sync_wishlist`` relies on."""
|
||||
|
||||
def __init__(self, pages_by_status):
|
||||
self.pages_by_status = pages_by_status
|
||||
self.calls = []
|
||||
|
||||
def _fetch_current_user_books_by_status(self, status_id, page, limit):
|
||||
self.calls.append((status_id, page, limit))
|
||||
pages = self.pages_by_status.get(status_id, [])
|
||||
books = pages[page - 1] if page <= len(pages) else []
|
||||
return SimpleNamespace(books=books, has_more=page < len(pages))
|
||||
|
||||
|
||||
def _book(provider_id, title, author="Matt Dinniman", **overrides):
|
||||
fields = {
|
||||
"provider": "hardcover",
|
||||
"provider_id": provider_id,
|
||||
"title": title,
|
||||
"authors": [author],
|
||||
}
|
||||
fields.update(overrides)
|
||||
return BookMetadata(**fields)
|
||||
|
||||
|
||||
def _insert_history(db_path, *, title, author):
|
||||
conn = sqlite3.connect(db_path)
|
||||
try:
|
||||
conn.execute(
|
||||
"INSERT INTO download_history (task_id, source, title, author, final_status) "
|
||||
"VALUES (?, ?, ?, ?, ?)",
|
||||
(f"task-{title}", "prowlarr", title, author, "complete"),
|
||||
)
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def db_path(tmp_path):
|
||||
return str(tmp_path / "users.db")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def user_db(db_path):
|
||||
db = UserDB(db_path)
|
||||
db.initialize()
|
||||
return db
|
||||
|
||||
|
||||
def _configure(monkeypatch, **values):
|
||||
monkeypatch.setattr(hardcover_sync, "app_config", _Config(**values))
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("values", "expected"),
|
||||
[
|
||||
({"HARDCOVER_SYNC_TOKEN": " sync-token ", "HARDCOVER_API_KEY": "api"}, "sync-token"),
|
||||
({"HARDCOVER_SYNC_TOKEN": "", "HARDCOVER_API_KEY": "api-key"}, "api-key"),
|
||||
({}, ""),
|
||||
],
|
||||
)
|
||||
def test_configured_token_falls_back_to_provider_api_key(monkeypatch, values, expected):
|
||||
_configure(monkeypatch, **values)
|
||||
|
||||
assert hardcover_sync._configured_token() == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("raw", "expected"),
|
||||
[
|
||||
("3", [3]),
|
||||
(" 2 ", [2]),
|
||||
(["1", "3"], [1, 3]),
|
||||
((1, "x", None), [1]),
|
||||
("garbage", [1]),
|
||||
([], [1]),
|
||||
(42, [1]),
|
||||
],
|
||||
)
|
||||
def test_configured_statuses_parsing(monkeypatch, raw, expected):
|
||||
_configure(monkeypatch, HARDCOVER_SYNC_STATUSES=raw)
|
||||
|
||||
assert hardcover_sync._configured_statuses() == expected
|
||||
|
||||
|
||||
def test_configured_statuses_default(monkeypatch):
|
||||
_configure(monkeypatch)
|
||||
|
||||
assert hardcover_sync._configured_statuses() == [1]
|
||||
|
||||
|
||||
def test_book_to_book_data_includes_optional_fields_only_when_present():
|
||||
full = _book(
|
||||
7,
|
||||
"Dungeon Crawler Carl",
|
||||
cover_url="https://img.example/c.jpg",
|
||||
publish_year=2020,
|
||||
subtitle="Book One",
|
||||
search_author="Dinniman",
|
||||
)
|
||||
sparse = BookMetadata(provider="hardcover", provider_id="8", title="Untitled")
|
||||
|
||||
assert hardcover_sync._book_to_book_data(full, "audiobook") == {
|
||||
"title": "Dungeon Crawler Carl",
|
||||
"author": "Dinniman",
|
||||
"provider": "hardcover",
|
||||
"provider_id": "7",
|
||||
"content_type": "audiobook",
|
||||
"preview": transform_cover_url("https://img.example/c.jpg", "hardcover_7"),
|
||||
"year": 2020,
|
||||
"subtitle": "Book One",
|
||||
}
|
||||
assert hardcover_sync._book_to_book_data(sparse, "ebook") == {
|
||||
"title": "Untitled",
|
||||
"author": "Unknown",
|
||||
"provider": "hardcover",
|
||||
"provider_id": "8",
|
||||
"content_type": "ebook",
|
||||
}
|
||||
|
||||
|
||||
def test_existing_provider_ids_reads_provider_id_or_id(user_db):
|
||||
user = user_db.create_user(username="reader", role="user")
|
||||
for book_data in (
|
||||
{"title": "A", "author": "X", "provider": "hardcover", "provider_id": 11},
|
||||
{"title": "B", "author": "X", "provider": "hardcover", "id": "22"},
|
||||
{"title": "C", "author": "X", "provider": "openlibrary", "id": "OL22W"},
|
||||
{"title": "D", "author": "X", "provider": "googlebooks", "provider_id": 33},
|
||||
{"title": "E", "author": "X", "provider": "manual"},
|
||||
):
|
||||
user_db.create_request(
|
||||
user_id=user["id"],
|
||||
content_type="audiobook",
|
||||
request_level="book",
|
||||
policy_mode="request_book",
|
||||
book_data=book_data,
|
||||
)
|
||||
|
||||
assert hardcover_sync._existing_provider_ids(user_db) == {"11", "22"}
|
||||
|
||||
|
||||
def test_already_downloaded_matches_history_case_insensitively(user_db, db_path):
|
||||
_insert_history(db_path, title="Dungeon Crawler Carl", author="Matt Dinniman")
|
||||
|
||||
assert hardcover_sync._already_downloaded(db_path, "dungeon crawler CARL", "MATT dinniman")
|
||||
assert not hardcover_sync._already_downloaded(db_path, "Dungeon Crawler Carl", "Someone")
|
||||
assert not hardcover_sync._already_downloaded(None, "Dungeon Crawler Carl", "Matt Dinniman")
|
||||
|
||||
|
||||
def test_already_downloaded_fails_open_without_history_table(tmp_path):
|
||||
missing = str(tmp_path / "empty.db")
|
||||
|
||||
assert hardcover_sync._already_downloaded(missing, "Anything", "Anyone") is False
|
||||
|
||||
|
||||
class TestSyncWishlist:
|
||||
@pytest.fixture(autouse=True)
|
||||
def _defaults(self, monkeypatch):
|
||||
_configure(monkeypatch, HARDCOVER_SYNC_STATUSES="1")
|
||||
monkeypatch.setattr(
|
||||
"shelfmark.core.library_index.is_in_library", lambda *_args, **_kwargs: False
|
||||
)
|
||||
|
||||
def _use_provider(self, monkeypatch, provider):
|
||||
monkeypatch.setattr(hardcover_sync, "_build_provider", lambda: provider)
|
||||
|
||||
def test_adds_new_requests_across_pages(self, user_db, db_path, monkeypatch):
|
||||
user = user_db.create_user(username="reader", role="user")
|
||||
provider = _Provider(
|
||||
{
|
||||
1: [
|
||||
[_book(1, "Dungeon Crawler Carl", cover_url="https://img.example/1.jpg")],
|
||||
[_book(2, "Carl's Doomsday Scenario", publish_year=2021)],
|
||||
]
|
||||
}
|
||||
)
|
||||
self._use_provider(monkeypatch, provider)
|
||||
|
||||
summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path, user_id=user["id"])
|
||||
|
||||
assert summary == {"added": 2, "skipped": 0, "in_library": 0, "errors": 0}
|
||||
assert provider.calls == [(1, 1, 25), (1, 2, 25)]
|
||||
rows = {row["book_data"]["provider_id"]: row for row in user_db.list_requests()}
|
||||
assert set(rows) == {"1", "2"}
|
||||
assert rows["1"]["status"] == "pending"
|
||||
assert rows["1"]["content_type"] == "audiobook"
|
||||
assert rows["1"]["book_data"]["preview"] == transform_cover_url(
|
||||
"https://img.example/1.jpg", "hardcover_1"
|
||||
)
|
||||
assert rows["2"]["book_data"]["year"] == 2021
|
||||
assert "preview" not in rows["2"]["book_data"]
|
||||
|
||||
def test_skips_known_downloaded_owned_and_duplicate_books(self, user_db, db_path, monkeypatch):
|
||||
user = user_db.create_user(username="reader", role="user")
|
||||
user_db.create_request(
|
||||
user_id=user["id"],
|
||||
content_type="audiobook",
|
||||
request_level="book",
|
||||
policy_mode="request_book",
|
||||
book_data={
|
||||
"title": "Already Requested",
|
||||
"author": "Matt Dinniman",
|
||||
"provider": "hardcover",
|
||||
"provider_id": "10",
|
||||
},
|
||||
)
|
||||
_insert_history(db_path, title="Already Downloaded", author="Matt Dinniman")
|
||||
_configure(monkeypatch, HARDCOVER_SYNC_STATUSES="1")
|
||||
monkeypatch.setattr(
|
||||
"shelfmark.core.library_index.is_in_library",
|
||||
lambda book, *_args, **_kwargs: book.title == "Already Owned",
|
||||
)
|
||||
provider = _Provider(
|
||||
{
|
||||
1: [
|
||||
[
|
||||
_book(10, "Already Requested"),
|
||||
_book(11, "Already Downloaded"),
|
||||
_book(12, "Already Owned"),
|
||||
_book(13, "Brand New"),
|
||||
_book(14, "Brand New"),
|
||||
]
|
||||
]
|
||||
}
|
||||
)
|
||||
self._use_provider(monkeypatch, provider)
|
||||
|
||||
summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path, user_id=user["id"])
|
||||
|
||||
assert summary == {"added": 1, "skipped": 3, "in_library": 1, "errors": 0}
|
||||
titles = sorted(row["book_data"]["title"] for row in user_db.list_requests())
|
||||
assert titles == ["Already Requested", "Brand New"]
|
||||
|
||||
def test_unavailable_provider_counts_as_error(self, user_db, db_path, monkeypatch):
|
||||
self._use_provider(monkeypatch, None)
|
||||
|
||||
summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path)
|
||||
|
||||
assert summary == {"added": 0, "skipped": 0, "in_library": 0, "errors": 1}
|
||||
assert user_db.list_requests() == []
|
||||
|
||||
|
||||
class TestResolveRequestOwner:
|
||||
def test_returns_lowest_id_admin(self, user_db):
|
||||
user_db.create_user(username="reader", role="user")
|
||||
first_admin = user_db.create_user(username="ops", role="admin")
|
||||
user_db.create_user(username="second", role="admin")
|
||||
|
||||
assert hardcover_sync.resolve_request_owner(user_db) == first_admin["id"]
|
||||
|
||||
def test_returns_none_without_admins(self, user_db):
|
||||
user_db.create_user(username="reader", role="user")
|
||||
|
||||
assert hardcover_sync.resolve_request_owner(user_db) is None
|
||||
|
||||
def test_sync_without_admin_counts_an_error_and_creates_nothing(self, user_db, db_path):
|
||||
user_db.create_user(username="reader", role="user")
|
||||
|
||||
summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path)
|
||||
|
||||
assert summary == {"added": 0, "skipped": 0, "in_library": 0, "errors": 1}
|
||||
assert user_db.list_requests() == []
|
||||
|
||||
def test_sync_defaults_to_the_lowest_id_admin(self, user_db, db_path, monkeypatch):
|
||||
user_db.create_user(username="reader", role="user")
|
||||
admin = user_db.create_user(username="ops", role="admin")
|
||||
provider = _Provider({1: [[_book(1, "Dungeon Crawler Carl")]]})
|
||||
monkeypatch.setattr(hardcover_sync, "_build_provider", lambda: provider)
|
||||
|
||||
summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path)
|
||||
|
||||
assert summary["added"] == 1
|
||||
assert user_db.list_requests()[0]["user_id"] == admin["id"]
|
||||
@@ -117,3 +117,58 @@ def test_plan_receives_filters_and_user_id(monkeypatch: pytest.MonkeyPatch) -> N
|
||||
"source_filters": None,
|
||||
"user_id": 42,
|
||||
}
|
||||
|
||||
|
||||
def test_search_book_releases_preserves_source_order_and_aggregates(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
sources = {"a": _Source(result=["a1"]), "b": _Source(result=["b1", "b2"])}
|
||||
monkeypatch.setattr("shelfmark.release_sources.get_source", lambda name: sources[name])
|
||||
|
||||
all_releases, by_source, errors = release_search.search_book_releases(
|
||||
_book(), sources=["b", "a"], content_type="audiobook"
|
||||
)
|
||||
|
||||
assert list(by_source) == ["b", "a"]
|
||||
assert by_source == {"b": ["b1", "b2"], "a": ["a1"]}
|
||||
assert all_releases == ["b1", "b2", "a1"]
|
||||
assert errors == []
|
||||
_book_arg, _plan, expand_search, content_type = sources["a"].calls[0]
|
||||
assert expand_search is True
|
||||
assert content_type == "audiobook"
|
||||
|
||||
|
||||
def test_search_book_releases_collects_errors_and_keeps_healthy_results(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
sources = {
|
||||
"down": _Source(error=SourceUnavailableError("offline")),
|
||||
"ok": _Source(result=["r1"]),
|
||||
}
|
||||
|
||||
def _get(name: str) -> Any:
|
||||
if name not in sources:
|
||||
msg = f"Unknown release source: {name}"
|
||||
raise ValueError(msg)
|
||||
return sources[name]
|
||||
|
||||
monkeypatch.setattr("shelfmark.release_sources.get_source", _get)
|
||||
|
||||
all_releases, by_source, errors = release_search.search_book_releases(
|
||||
_book(), sources=["down", "ok", "nope"]
|
||||
)
|
||||
|
||||
assert all_releases == ["r1"]
|
||||
assert by_source == {"ok": ["r1"]}
|
||||
assert errors == ["down: offline", "Unknown source: nope"]
|
||||
|
||||
|
||||
def test_search_book_releases_with_no_sources_returns_empty(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
def _unexpected(_name: str) -> Any:
|
||||
raise AssertionError("no source should be resolved")
|
||||
|
||||
monkeypatch.setattr("shelfmark.release_sources.get_source", _unexpected)
|
||||
|
||||
assert release_search.search_book_releases(_book(), sources=[]) == ([], {}, [])
|
||||
Reference in new issue
Block a user