feat(library): Audiobookshelf library check, and the sync fails closed when a library cannot be read (#1519)

Adds an Audiobookshelf provider for the library check, next to the Calibre one. It reads the server over its REST API (title, author, ISBN and ASIN per item), answers for audiobooks, and has its settings in General with a test button that uses the unsaved form values. Read only.

The Hardcover sync and automatic downloads now fail closed when an enabled library cannot be read: the sync skips its cycle and an automatic download holds the request. A library that listed items at the last index and now lists none counts as a failed read. Badges still fail open.
This commit is contained in:
splitsec2 authored and GitHub committed 2026-10-23 00:21:40 -06:00
1 parent 99a6aa206b
commit ea69db3f8e
13 files changed
+835 -28

No files matched your search

+2 -2
View File
@@ -34,7 +34,7 @@ Among matching releases from the winning source, `m4b` outranks `mp3`, then more
Both shelf sync and automatic downloads ask the libraries you have enabled under [Library Check](library-check.md) whether the book is already there, for the content type being requested, and skip it if so. There is no separate switch on this tab. A library that is not enabled, or that does not hold the requested content type, never causes a skip.
The check **fails open**: a library that cannot be read is logged and the book is treated as not owned, so processing continues.
The check **fails closed** here: if an enabled library cannot be read and has not been read before, the sync cycle is skipped and an automatic download is held, and the request stays pending until the next cycle. Skipping is the safe side, because carrying on would request or download books the library already holds. Badges in search results still fail open.
## Settings
@@ -74,6 +74,6 @@ Only one run happens at a time. If a scheduled cycle is in progress, **Sync now*
**Requests appear but nothing downloads** — check **Enable automatic downloads** is on, that **Source priority** contains at least one configured source that supports audiobooks, and look for `no strict match` lines in the log. Lowering **Minimum seeders** helps on quiet trackers; loosening the title or author match is deliberately not an option.
**Books you own keep being added** — enable the library check for the format you request (see [Library Check](library-check.md)) and use its test button. Because the check fails open, a library that cannot be read only produces a log warning rather than blocking the sync.
**Books you own keep being added** — enable the library check for the format you request (see [Library Check](library-check.md)) and use its test button. A library that cannot be read shows up in the log as a skipped cycle.
**Too many requests at once** — the first sync of a large shelf creates one request per book, and the per-user pending limit from **Users & Requests** does not apply to synced requests. Start with a shelf you are happy to see in the queue in full.
+31 -5
View File
@@ -21,6 +21,26 @@ database directly. No HTTP call, no API token, and nothing is ever written back.
A result that matches the library then carries an **In library** badge in the card, list
and compact views, and in the details dialog.
## Audiobookshelf
Shelfmark asks the Audiobookshelf server over its REST API for the title, author, ISBN and
ASIN of every item, and nothing is written back.
1. In **Settings, General**, turn on **Mark audiobooks already in your Audiobookshelf
library**.
2. Set **Audiobookshelf URL** to an address the Shelfmark container can reach (often the
host's LAN IP, not the public URL), for example `http://10.0.0.91:13378`.
3. Paste an **Audiobookshelf API Token** (Audiobookshelf **Settings, Users, your user, API
Token**). Read access is enough.
4. Leave **Audiobookshelf Library IDs** blank to check every book library, or list the IDs
to check (comma separated).
5. Press **Test Audiobookshelf library**. It reports how many items it indexed, using the
values on the form, so it works before you save.
Audiobookshelf titles are often raw folder names such as `Alex Cross 02: Kiss the Girls`.
When the server reports a series for the item, the leading label and number are read as the
series, not as a different book, so a search for `Kiss the Girls` still matches.
## How a match is decided
In order of confidence:
@@ -32,12 +52,18 @@ In order of confidence:
## Behaviour worth knowing
- **It fails open.** If the database cannot be read, Shelfmark logs a warning, reuses the
last successful read if it has one, and otherwise treats the book as not owned. A broken
path degrades the badge, it never blocks a search.
- **It fails open for badges.** If a library cannot be read, Shelfmark logs a warning, reuses
the last successful read if it has one, and otherwise treats the book as not owned. A
broken path degrades the badge, it never blocks a search.
- **It fails closed for anything that acts on the answer.** The Hardcover sync and automatic
downloads would otherwise request or download books the library already holds, so when an
enabled library cannot be read and has not been read before, the sync cycle is skipped and
the request stays pending until the next one. A library that listed items a moment ago and
now lists none is treated as unreadable too, not as an emptied shelf.
- **Results are cached** for ten minutes, and refreshed early when the database file
changes, so a large library costs one read rather than one per search.
- **Ebooks only.** A Calibre library holds ebooks, so the badge answers for ebooks. The
provider interface in `shelfmark/core/library_providers/` takes more libraries: add a
- **Each library answers for its own format.** Calibre holds ebooks and Audiobookshelf holds
audiobooks, so an ebook result is only checked against Calibre and an audiobook against
Audiobookshelf. The provider interface in `shelfmark/core/library_providers/` takes more libraries: add a
module with the `LibraryProvider` shape and list it in `all_providers()`. Nothing above
that function knows which libraries exist.
+42
View File
@@ -396,6 +396,13 @@ def _test_calibre_library(current_values: dict[str, Any] | None = None) -> dict[
return library_index.test_connection("calibre", current_values)
def _test_audiobookshelf_library(current_values: dict[str, Any] | None = None) -> dict[str, Any]:
"""Action-button callback: read the Audiobookshelf libraries and count the items."""
from shelfmark.core import library_index
return library_index.test_connection("audiobookshelf", current_values)
@register_settings("general", "General", icon="settings", order=0)
def general_settings() -> list[SettingsField]:
"""Core application settings."""
@@ -444,6 +451,41 @@ def general_settings() -> list[SettingsField]:
description="Check that Shelfmark can read the Calibre database and count the books.",
callback=_test_calibre_library,
),
CheckboxField(
key="LIBRARY_CHECK_AUDIOBOOKSHELF_ENABLED",
label="Mark audiobooks already in your Audiobookshelf library",
description=(
"Ask your Audiobookshelf server which audiobooks you already own, so synced "
"and automatic downloads skip them. Read only, nothing is written to Audiobookshelf."
),
default=False,
),
TextField(
key="AUDIOBOOKSHELF_URL",
label="Audiobookshelf URL",
description=(
"Base URL reachable from the Shelfmark container (not the public URL). Often the "
"host LAN IP, e.g. http://10.0.0.91:13378."
),
placeholder="http://10.0.0.91:13378",
),
PasswordField(
key="AUDIOBOOKSHELF_TOKEN",
label="Audiobookshelf API Token",
description="API token from Audiobookshelf (Settings > Users > your user > API Token).",
),
TextField(
key="AUDIOBOOKSHELF_LIBRARY_IDS",
label="Audiobookshelf Library IDs (optional)",
description="Comma-separated library IDs to check. Leave blank for all book libraries.",
placeholder="Blank = all book libraries",
),
ActionButton(
key="test_audiobookshelf_library",
label="Test Audiobookshelf library",
description="Check that Shelfmark can reach Audiobookshelf and count the items.",
callback=_test_audiobookshelf_library,
),
HeadingField(
key="search_defaults_heading",
title="Default Search Filters",
+11 -3
View File
@@ -147,7 +147,7 @@ def strict_match(
"""Return True only if the release confidently matches the requested audiobook."""
formats = audiobook_formats if audiobook_formats is not None else _audiobook_formats()
return (
_title_match(book, release.title)
_title_match(book, release.title or "")
and _author_match(book, release)
and _format_match(release, formats)
and _seeders_ok(release, min_seeders)
@@ -278,9 +278,17 @@ def auto_download_request(
return AutoDownloadOutcome(request_id, "skipped", "book not found in provider")
# Final guard: skip if the book is already in a library that is checked for this content type.
from shelfmark.core.library_index import is_in_library
from shelfmark.core import library_index
if is_in_library(book, content_type):
try:
in_library = library_index.is_in_library(book, content_type, strict=True)
except library_index.LibraryUnavailableError as exc:
# Unattended: do not download blind. The request stays pending for the next cycle.
logger.warning(
"auto-download: request %s held, library check unavailable: %s", request_id, exc
)
return AutoDownloadOutcome(request_id, "error", f"library check unavailable: {exc}")
if in_library:
logger.info(
"auto-download: request %s (%s) already in library; skipping",
request_id,
+10 -2
View File
@@ -172,9 +172,17 @@ def sync_wishlist(
return summary
content_type = str(app_config.get("HARDCOVER_SYNC_CONTENT_TYPE", "audiobook") or "audiobook")
from shelfmark.core import library_index
try:
library_index.require_ready(content_type)
except library_index.LibraryUnavailableError as exc:
# Requesting now would queue books the library may already hold.
logger.warning("hardcover-sync: %s; skipping this sync", exc)
summary["errors"] += 1
return summary
known_provider_ids = _existing_provider_ids(user_db)
from shelfmark.core.library_index import is_in_library
from shelfmark.core.requests_service import RequestServiceError, create_request
for status_id in _configured_statuses():
@@ -188,7 +196,7 @@ def sync_wishlist(
if _already_downloaded(db_path, book.title, author):
summary["skipped"] += 1
continue
if is_in_library(book, content_type):
if library_index.is_in_library(book, content_type):
summary["in_library"] += 1
logger.info("hardcover-sync: '%s' already in your library; skipping", book.title)
continue
+61 -13
View File
@@ -69,14 +69,31 @@ def _store(provider_name: str, entries: list[LibraryEntry], fingerprint: object
slot.failed_at = None
def _entries_for(provider: LibraryProvider) -> list[LibraryEntry]:
"""Cached entries for one provider, re-indexed past the TTL or when the library changed."""
class LibraryUnavailableError(RuntimeError):
"""An enabled library could not be read and there is no earlier index to fall back on."""
def _entries_for(provider: LibraryProvider, *, strict: bool = False) -> list[LibraryEntry]:
"""Cached entries for one provider, re-indexed past the TTL or when the library changed.
A library that cannot be read answers from its stale cache when there is one. With
nothing cached the default is to fail open (no entries, so nothing counts as owned),
which suits badges. ``strict`` raises instead, for callers that act on the answer.
"""
slot = _slot(provider.name)
with _lock:
if slot.failed_at is not None and time.monotonic() - slot.failed_at < (
_FAILURE_BACKOFF_SECONDS
):
return slot.entries or []
backing_off = slot.failed_at is not None and (
time.monotonic() - slot.failed_at < _FAILURE_BACKOFF_SECONDS
)
stale = slot.entries
if backing_off:
# Same answer the failure gave: the stale index, or (strict) a refusal to guess.
if stale is not None:
return stale
if strict:
msg = f"{provider.describe()} is unavailable (retrying after a short back-off)"
raise LibraryUnavailableError(msg)
return []
try:
fingerprint = provider.fingerprint()
with _lock:
@@ -86,11 +103,30 @@ def _entries_for(provider: LibraryProvider) -> list[LibraryEntry]:
if cached is not None and fresh and unchanged:
return cached
entries = provider.fetch_entries()
except Exception as exc: # noqa: BLE001 - any failure must fail open
logger.warning("library check: %s unavailable (%s); failing open", provider.describe(), exc)
with _lock:
had_entries = bool(slot.entries)
if had_entries and not entries:
# A library that held books a moment ago and now lists none is far more likely a
# changed answer from the server than an emptied shelf. Indexing it would make
# every owned book look missing, so it is treated like any other failed read
# (the earlier index keeps answering, strict callers hold). A library that was
# really emptied clears on the next restart.
msg = "it listed no items although it held some at the last index"
raise ValueError(msg) # noqa: TRY301 - handled by the failure path below
except Exception as exc:
with _lock:
slot.failed_at = time.monotonic()
return slot.entries or [] # Use the stale cache if we have one.
stale = slot.entries
if stale is not None:
logger.warning(
"library check: %s unavailable (%s); using stale index", provider.describe(), exc
)
return stale
if strict:
msg = f"{provider.describe()} is unavailable: {exc}"
raise LibraryUnavailableError(msg) from exc
logger.warning("library check: %s unavailable (%s); failing open", provider.describe(), exc)
return []
_store(provider.name, entries, fingerprint)
logger.info("library check: indexed %d %s item(s)", len(entries), provider.display_name)
@@ -153,13 +189,25 @@ def book_matches_entries(book: BookMetadata, entries: list[LibraryEntry]) -> boo
return match_entries(book, entries) is not None
def is_in_library(book: BookMetadata, content_type: str | None = None) -> bool:
"""True if an enabled library holding ``content_type`` already has ``book`` (fail-open).
def require_ready(content_type: str | None = None) -> None:
"""Raise LibraryUnavailableError unless every enabled library can be read (or is cached).
``content_type`` None consults every enabled provider.
``content_type`` limits the check to the libraries holding that type.
"""
for provider in _enabled_providers(content_type):
_entries_for(provider, strict=True)
def is_in_library(
book: BookMetadata, content_type: str | None = None, *, strict: bool = False
) -> bool:
"""True if an enabled library holding ``content_type`` already has ``book``.
``content_type`` None consults every enabled provider. A library that cannot be read
counts as not holding the book, unless ``strict``, which raises LibraryUnavailableError.
"""
return any(
book_matches_entries(book, _entries_for(provider))
book_matches_entries(book, _entries_for(provider, strict=strict))
for provider in _enabled_providers(content_type)
)
+2 -1
View File
@@ -68,6 +68,7 @@ def all_providers(overrides: Mapping[str, Any] | None = None) -> list[LibraryPro
Adding a library means writing a module with the :class:`LibraryProvider` shape and
listing it here. Nothing above this function knows which libraries exist.
"""
from shelfmark.core.library_providers.audiobookshelf import AudiobookshelfLibrary
from shelfmark.core.library_providers.calibre import CalibreLibrary
return [CalibreLibrary(overrides)]
return [CalibreLibrary(overrides), AudiobookshelfLibrary(overrides)]
@@ -0,0 +1,159 @@
"""Audiobookshelf library provider.
Fetches the libraries of an Audiobookshelf (ABS) server over its REST API, with title,
author, ISBN and ASIN per item. ABS metadata is often messy (titles are raw folder
names, the author is frequently empty), so each item is tokenized from title, author and
folder path.
"""
from __future__ import annotations
import re
from typing import TYPE_CHECKING, Any
import requests
from shelfmark.core.config import config as app_config
from shelfmark.core.library_providers import LibraryEntry, setting
from shelfmark.core.text_match import isbn_variants, tokens
from shelfmark.core.utils import normalize_http_url
if TYPE_CHECKING:
from collections.abc import Mapping
_REQUEST_TIMEOUT = 15
_ITEMS_PAGE_LIMIT = 500
# "Alex Cross 02: ", "DCC 1: ", "Reacher 19.5: ": a short label, a number and a colon.
_SERIES_PREFIX = re.compile(r"^(?P<label>[^\d:]{1,40}?)\s+(?P<number>\d+(?:\.\d+)?)\s*:\s*(?=\S)")
def _config(overrides: Mapping[str, Any] | None = None) -> tuple[bool, str, str, list[str]]:
enabled = bool(setting(app_config, "LIBRARY_CHECK_AUDIOBOOKSHELF_ENABLED", False, overrides))
url = normalize_http_url(str(setting(app_config, "AUDIOBOOKSHELF_URL", "", overrides) or ""))
token = str(setting(app_config, "AUDIOBOOKSHELF_TOKEN", "", overrides) or "").strip()
lib_ids_raw = str(setting(app_config, "AUDIOBOOKSHELF_LIBRARY_IDS", "", overrides) or "")
lib_ids = [s.strip() for s in lib_ids_raw.split(",") if s.strip()]
return enabled, url, token, lib_ids
def _session(url: str, token: str) -> requests.Session:
from shelfmark.download.network import get_ssl_verify
session = requests.Session()
session.headers.update({"Authorization": f"Bearer {token}"})
# The URL lets CERTIFICATE_VALIDATION=disabled_local skip verification for a LAN
# server, which is where Audiobookshelf usually lives.
session.verify = get_ssl_verify(url)
return session
def _first_series_name(value: object) -> str | None:
"""The first series in Audiobookshelf's ``seriesName`` ("Name #1, Other Name #3")."""
first = str(value or "").split(", ", 1)[0].strip()
name = re.sub(r"\s+#\S+$", "", first).strip()
return name or None
def _item_to_entry(item: dict[str, Any]) -> LibraryEntry:
media = item.get("media") or {}
meta = media.get("metadata") or {}
title = meta.get("title") or ""
author = meta.get("authorName") or ""
rel_path = item.get("relPath") or ""
context_tok = set(tokens(author))
# A book in a series is often titled "Alex Cross 02: Kiss the Girls" so players that
# ignore the series number still show the order. Those leading words name the series,
# not another work, so they go with the author as context.
prefix = _SERIES_PREFIX.match(title) if _first_series_name(meta.get("seriesName")) else None
if prefix:
context_tok |= set(tokens(f"{prefix['label']} {prefix['number']}"))
title = title[prefix.end() :]
title_tok = set(tokens(title))
tok = title_tok | context_tok | set(tokens(rel_path))
asin = str(meta.get("asin") or "").strip().upper()
return LibraryEntry(
frozenset(tok),
isbn_variants(meta.get("isbn")),
frozenset({asin} if asin else set()),
frozenset(),
frozenset(title_tok),
frozenset(context_tok),
)
def _fetch_library_entries(url: str, token: str, lib_ids: list[str]) -> list[LibraryEntry]:
session = _session(url, token)
resp = session.get(f"{url}/api/libraries", timeout=_REQUEST_TIMEOUT)
resp.raise_for_status()
libraries = resp.json().get("libraries", [])
selected: list[str] = []
for lib in libraries:
lib_id = lib.get("id")
if not lib_id:
continue
if lib_ids:
if lib_id in lib_ids:
selected.append(lib_id)
elif lib.get("mediaType") == "book":
selected.append(lib_id)
entries: list[LibraryEntry] = []
for lib_id in selected:
page = 0
while True:
resp = session.get(
f"{url}/api/libraries/{lib_id}/items",
params={"limit": _ITEMS_PAGE_LIMIT, "page": page},
timeout=_REQUEST_TIMEOUT,
)
resp.raise_for_status()
data = resp.json()
results = data.get("results") if isinstance(data, dict) else None
if not isinstance(results, list):
# An answer without an item list is not an empty library.
msg = f"Audiobookshelf's item list for library {lib_id} has an unexpected shape"
raise TypeError(msg)
total = int(data.get("total", 0) or 0)
if page == 0 and not results and total > 0:
msg = f"Audiobookshelf reports {total} items in library {lib_id} but listed none"
raise ValueError(msg)
entries.extend(_item_to_entry(item) for item in results)
page += 1
if not results or page * _ITEMS_PAGE_LIMIT >= total:
break
return entries
class AudiobookshelfLibrary:
"""Audiobook ownership via the Audiobookshelf REST API."""
name = "audiobookshelf"
display_name = "Audiobookshelf"
content_types = frozenset({"audiobook"})
def __init__(self, overrides: Mapping[str, Any] | None = None) -> None:
self._overrides = overrides
def is_enabled(self) -> bool:
enabled, url, token, _lib_ids = _config(self._overrides)
return enabled and bool(url) and bool(token)
def describe(self) -> str:
_enabled, url, _token, _lib_ids = _config(self._overrides)
return f"Audiobookshelf at {url}" if url else "Audiobookshelf"
def fingerprint(self) -> None:
return None
def fetch_entries(self) -> list[LibraryEntry]:
_enabled, url, token, lib_ids = _config(self._overrides)
if not url or not token:
msg = "Set the Audiobookshelf URL and token first."
raise ValueError(msg)
return _fetch_library_entries(url, token, lib_ids)
+30
View File
@@ -1,6 +1,7 @@
"""Tests for general settings definitions."""
from shelfmark.config.settings import general_settings
from shelfmark.core import settings_registry
def test_supported_formats_stay_admin_only():
@@ -9,3 +10,32 @@ def test_supported_formats_stay_admin_only():
assert fields["SUPPORTED_FORMATS"].user_overridable is False
assert fields["SUPPORTED_AUDIOBOOK_FORMATS"].user_overridable is False
def _fields():
return {field.key: field for field in general_settings() if hasattr(field, "key")}
def test_the_audiobookshelf_library_check_is_off_by_default():
fields = _fields()
assert fields["LIBRARY_CHECK_AUDIOBOOKSHELF_ENABLED"].default is False
assert fields["AUDIOBOOKSHELF_URL"].placeholder
assert fields["AUDIOBOOKSHELF_TOKEN"].key == "AUDIOBOOKSHELF_TOKEN"
assert fields["test_audiobookshelf_library"].callback is not None
def test_the_audiobookshelf_test_button_checks_the_unsaved_form_values(monkeypatch):
seen = {}
def _fake_test_connection(provider_name, current_values):
seen["call"] = (provider_name, current_values)
return {"success": True, "message": "Connected. Indexed 3 library item(s)."}
monkeypatch.setattr("shelfmark.core.library_index.test_connection", _fake_test_connection)
form = {"AUDIOBOOKSHELF_URL": "http://abs:13378", "AUDIOBOOKSHELF_TOKEN": "t"}
result = settings_registry.execute_action("general", "test_audiobookshelf_library", form)
assert result["success"] is True
assert seen["call"] == ("audiobookshelf", form)
+31 -1
View File
@@ -2,7 +2,7 @@
import pytest
from shelfmark.core import auto_download
from shelfmark.core import auto_download, library_index
from shelfmark.core.user_db import UserDB
from shelfmark.metadata_providers import BookMetadata
from shelfmark.release_sources import Release, ReleaseProtocol
@@ -120,6 +120,9 @@ class TestStrictMatch:
release = _release(protocol=ReleaseProtocol.NZB, seeders=None)
assert _strict(release, min_seeders=5) is True
def test_a_release_without_a_title_does_not_match(self):
assert _strict(_release(title=None)) is False
class TestPickBestRelease:
def test_empty_returns_none(self):
@@ -378,6 +381,33 @@ class TestAutoDownloadRequest:
assert outcome.status == "in_library"
assert user_db.get_request(row["id"])["status"] == "pending"
def test_a_library_that_cannot_be_read_stops_the_request_before_searching(
self, user_db, monkeypatch
):
admin = user_db.create_user(username="admin", role="admin")
reader = user_db.create_user(username="reader", role="user")
row = _pending_request(user_db, reader["id"])
_stub_provider(monkeypatch, _book())
monkeypatch.setattr(auto_download, "app_config", _Config())
def _is_in_library(_book, _content_type=None, *, strict=False):
if strict:
raise library_index.LibraryUnavailableError("Audiobookshelf is unavailable")
return False # what a fail-open check would have answered
monkeypatch.setattr("shelfmark.core.library_index.is_in_library", _is_in_library)
monkeypatch.setattr(
auto_download,
"search_book_releases",
lambda *_a, **_k: pytest.fail("a request must not be searched for blind"),
)
outcome = self._run(user_db, row, admin)
assert outcome.status == "error"
assert "library" in outcome.detail
assert user_db.get_request(row["id"])["status"] == "pending"
def test_unregistered_provider_is_skipped(self, user_db, monkeypatch):
admin = user_db.create_user(username="admin", role="admin")
reader = user_db.create_user(username="reader", role="user")
+21 -1
View File
@@ -5,7 +5,7 @@ from types import SimpleNamespace
import pytest
from shelfmark.core import hardcover_sync
from shelfmark.core import hardcover_sync, library_index
from shelfmark.core.user_db import UserDB
from shelfmark.core.utils import transform_cover_url
from shelfmark.metadata_providers import BookMetadata
@@ -253,6 +253,26 @@ class TestSyncWishlist:
titles = sorted(row["book_data"]["title"] for row in user_db.list_requests())
assert titles == ["Already Requested", "Brand New"]
def test_a_library_that_cannot_be_read_stops_the_sync_before_anything_is_requested(
self, user_db, db_path, monkeypatch
):
user = user_db.create_user(username="reader", role="user")
provider = _Provider({1: [[_book(1, "Dungeon Crawler Carl")]]})
self._use_provider(monkeypatch, provider)
_configure(monkeypatch, HARDCOVER_SYNC_STATUSES="1")
def _down(*_args, **_kwargs):
raise library_index.LibraryUnavailableError("Audiobookshelf is unavailable")
monkeypatch.setattr("shelfmark.core.library_index.require_ready", _down)
summary = hardcover_sync.sync_wishlist(user_db, db_path=db_path, user_id=user["id"])
# Skipping is the safe side: syncing now would request books the library already holds.
assert summary == {"added": 0, "skipped": 0, "in_library": 0, "errors": 1}
assert provider.calls == [] # Hardcover was not even asked
assert user_db.list_requests() == []
def test_unavailable_provider_counts_as_error(self, user_db, db_path, monkeypatch):
self._use_provider(monkeypatch, None)
+105
View File
@@ -468,6 +468,111 @@ def test_an_exact_shelf_title_outranks_a_collection_holding_the_same_book() -> N
assert library_index.match_entries(book, [collection, single]) == "owned"
# --------------------------------------------------------------------------- #
# Fail closed for callers that act on the answer
# --------------------------------------------------------------------------- #
def test_a_library_that_suddenly_lists_nothing_keeps_answering_from_the_earlier_index(
providers: list[_Provider], monkeypatch: pytest.MonkeyPatch, clock: list[float]
) -> None:
provider = _Provider("audio", {"audiobook"}, [_DCC_ENTRY])
providers.append(provider)
assert library_index.is_in_library(_book(), "audiobook") is True
provider.entries = [] # the server's answer changed shape and every item vanished
clock[0] += library_index._CACHE_TTL_SECONDS + 1
warnings = _warnings(monkeypatch)
assert library_index.is_in_library(_book(), "audiobook") is True
assert library_index.is_in_library(_book(), "audiobook", strict=True) is True
assert provider.fetches == 2 # the second answer came from the back-off, not a refetch
assert len(warnings) == 1
assert "listed no items" in warnings[0]
def test_a_library_that_was_never_indexed_may_be_empty(providers: list[_Provider]) -> None:
providers.append(_Provider("audio", {"audiobook"}, []))
assert library_index.is_in_library(_book(), "audiobook") is False
assert library_index.is_in_library(_book(), "audiobook", strict=True) is False
def test_a_strict_check_still_refuses_during_the_back_off(
providers: list[_Provider], clock: list[float]
) -> None:
provider = _Provider("ebook", {"ebook"}, error=OSError("mount gone"))
providers.append(provider)
with pytest.raises(library_index.LibraryUnavailableError):
library_index.is_in_library(_book(), "ebook", strict=True)
# Inside the back-off nothing is fetched, and the answer must not fall back to "not owned".
with pytest.raises(library_index.LibraryUnavailableError):
library_index.is_in_library(_book(), "ebook", strict=True)
assert provider.fetches == 1
def test_a_strict_check_refuses_to_guess_when_a_library_is_down_and_nothing_is_cached(
providers: list[_Provider],
) -> None:
providers.append(_Provider("ebook", {"ebook"}, error=OSError("mount gone")))
with pytest.raises(library_index.LibraryUnavailableError, match=r"Ebook \(test\)"):
library_index.is_in_library(_book(), "ebook", strict=True)
def test_a_strict_check_still_answers_from_a_stale_cache(
providers: list[_Provider], clock: list[float]
) -> None:
provider = _Provider("ebook", {"ebook"}, [_DCC_ENTRY])
providers.append(provider)
assert library_index.is_in_library(_book(), "ebook", strict=True) is True
provider.error = RuntimeError("boom")
clock[0] += library_index._CACHE_TTL_SECONDS + 1
assert library_index.is_in_library(_book(), "ebook", strict=True) is True
def test_a_down_library_leaves_the_badges_fail_open(providers: list[_Provider]) -> None:
providers.append(_Provider("ebook", {"ebook"}, error=OSError("mount gone")))
assert library_index.ownership(_book()) == {"ebook": None}
def test_require_ready_names_the_library_that_cannot_be_read(providers: list[_Provider]) -> None:
providers.append(_Provider("audiobook", {"audiobook"}, [_DCC_ENTRY]))
providers.append(_Provider("ebook", {"ebook"}, error=OSError("mount gone")))
with pytest.raises(library_index.LibraryUnavailableError, match=r"Ebook \(test\)"):
library_index.require_ready()
def test_require_ready_passes_once_every_enabled_library_has_been_read(
providers: list[_Provider],
) -> None:
providers.append(_Provider("ebook", {"ebook"}, [_DCC_ENTRY]))
providers.append(_Provider("audiobook", {"audiobook"}, [_DCC_ENTRY]))
library_index.require_ready() # does not raise
def test_require_ready_ignores_a_disabled_library(providers: list[_Provider]) -> None:
providers.append(_Provider("ebook", {"ebook"}, enabled=False, error=OSError("mount gone")))
library_index.require_ready() # does not raise
def test_require_ready_only_checks_the_libraries_for_the_content_type(
providers: list[_Provider],
) -> None:
providers.append(_Provider("ebook", {"ebook"}, error=OSError("no such file")))
providers.append(_Provider("audio", {"audiobook"}, [_entry("carl")]))
library_index.require_ready("audiobook")
with pytest.raises(library_index.LibraryUnavailableError, match="Ebook"):
library_index.require_ready("ebook")
def test_matching_tokenises_the_title_once_for_all_entries(monkeypatch: pytest.MonkeyPatch) -> None:
"""The title does not change between entries, so it is split into words once."""
from shelfmark.core import text_match
@@ -0,0 +1,330 @@
"""Tests for the Audiobookshelf library provider."""
from __future__ import annotations
from typing import Any
import pytest
from shelfmark.core import library_index
from shelfmark.core.library_providers import audiobookshelf
from shelfmark.metadata_providers import BookMetadata
pytestmark = pytest.mark.usefixtures("config")
@pytest.fixture
def config(monkeypatch: pytest.MonkeyPatch) -> dict[str, Any]:
"""Settings the provider reads, starting empty."""
values: dict[str, Any] = {}
monkeypatch.setattr(
audiobookshelf.app_config,
"get",
lambda key, default=None, user_id=None: values.get(key, default),
)
monkeypatch.setattr(library_index, "_cache", {})
return values
def _item(title: str, author: str, rel_path: str = "", **meta: Any) -> dict[str, Any]:
return {
"relPath": rel_path,
"media": {"metadata": {"title": title, "authorName": author, **meta}},
}
def _searching(title: str, author: str = "James Patterson") -> BookMetadata:
return BookMetadata(
provider="hardcover",
provider_id="1",
title=title,
authors=[author],
search_title=title,
search_author=author,
)
class _Response:
def __init__(self, payload: dict[str, Any]) -> None:
self.payload = payload
def raise_for_status(self) -> None:
return None
def json(self) -> dict[str, Any]:
return self.payload
class _Session:
def __init__(self, libraries: list[dict[str, Any]], pages: dict[int, dict[str, Any]]) -> None:
self.libraries = libraries
self.pages = pages
self.calls: list[tuple[str, dict[str, Any]]] = []
def get(self, url: str, params: dict[str, Any] | None = None, timeout: int = 0) -> _Response:
self.calls.append((url, dict(params or {})))
if url.endswith("/api/libraries"):
return _Response({"libraries": self.libraries})
return _Response(self.pages[int(self.calls[-1][1]["page"])])
def _install(monkeypatch: pytest.MonkeyPatch, session: _Session) -> None:
monkeypatch.setattr(audiobookshelf, "_session", lambda url, token: session)
monkeypatch.setattr(audiobookshelf, "_ITEMS_PAGE_LIMIT", 2)
def test_item_to_entry_tokenizes_title_author_and_path() -> None:
item = _item(
"Dungeon Crawler Carl",
"Matt Dinniman",
"Matt Dinniman/Dungeon Crawler Carl (2020)",
isbn="978-0-593-82024-7",
asin=" b08bkgyqxw ",
)
entry = audiobookshelf._item_to_entry(item)
assert {"dungeon", "crawler", "carl", "matt", "dinniman", "2020"} <= entry.tokens
assert entry.isbns == {"9780593820247"}
assert entry.asins == {"B08BKGYQXW"}
assert entry.external_ids == frozenset()
def test_item_to_entry_indexes_both_forms_of_an_isbn10() -> None:
entry = audiobookshelf._item_to_entry(_item("Carl", "Dinniman", isbn="059382024X"))
assert entry.isbns == {"059382024X", "9780593820247"}
def test_item_to_entry_tolerates_missing_metadata() -> None:
entry = audiobookshelf._item_to_entry({})
assert (entry.tokens, entry.isbns, entry.asins) == (frozenset(), frozenset(), frozenset())
@pytest.mark.parametrize(
("shelf_title", "series", "author", "searching"),
[
("Alex Cross 02: Kiss the Girls", "Alex Cross #2", "James Patterson", "Kiss the Girls"),
("Alex Cross 12: Cross", "Alex Cross #12", "James Patterson", "Cross"),
("Alex Cross 13: Double Cross", "Alex Cross #13", "James Patterson", "Double Cross"),
(
"DCC 2: Carl's Doomsday Scenario",
"Dungeon Crawler Carl #2",
"Matt Dinniman",
"Carl's Doomsday Scenario",
),
("Reacher 19: Personal", "Jack Reacher #19", "Lee Child", "Personal"),
("Reacher 01: Killing Floor", "Jack Reacher #1", "Lee Child", "Killing Floor"),
],
)
def test_a_series_number_in_the_shelf_title_does_not_hide_the_book(
shelf_title: str, series: str, author: str, searching: str
) -> None:
entry = audiobookshelf._item_to_entry(_item(shelf_title, author, seriesName=series))
assert library_index.match_entries(_searching(searching, author), [entry]) == "owned"
def test_a_search_title_that_carries_the_series_words_still_matches() -> None:
"""Metadata sources often title the same book "Kiss the Girls: Alex Cross"."""
entry = audiobookshelf._item_to_entry(
_item("Alex Cross 02: Kiss the Girls", "James Patterson", seriesName="Alex Cross #2")
)
assert library_index.match_entries(_searching("Kiss the Girls: Alex Cross"), [entry]) == "owned"
def test_a_numbered_shelf_title_still_does_not_match_a_different_book() -> None:
entry = audiobookshelf._item_to_entry(
_item("Alex Cross 02: Kiss the Girls", "James Patterson", seriesName="Alex Cross #2")
)
assert library_index.match_entries(_searching("Along Came a Spider"), [entry]) is None
# A shorter search title is still a different book, as with "Dune" and "Dune Messiah".
assert library_index.match_entries(_searching("Cross"), [entry]) is None
def test_a_number_before_a_colon_is_only_a_series_label_when_abs_reports_a_series() -> None:
entry = audiobookshelf._item_to_entry(_item("Apollo 13: The Movie", "Someone"))
assert library_index.match_entries(_searching("The Movie", "Someone"), [entry]) is None
def test_a_plain_shelf_title_is_unchanged_by_the_series_handling() -> None:
entry = audiobookshelf._item_to_entry(
_item("Kiss the Girls", "James Patterson", seriesName="Alex Cross #2")
)
assert library_index.match_entries(_searching("Kiss the Girls"), [entry]) == "owned"
def test_fetch_pages_through_book_libraries(monkeypatch: pytest.MonkeyPatch) -> None:
session = _Session(
libraries=[{"id": "books", "mediaType": "book"}, {"id": "pods", "mediaType": "podcast"}],
pages={
0: {"results": [_item("One", "A"), _item("Two", "B")], "total": 3},
1: {"results": [_item("Three", "C")], "total": 3},
},
)
_install(monkeypatch, session)
entries = audiobookshelf._fetch_library_entries("http://abs", "token", [])
assert [sorted(entry.tokens) for entry in entries] == [
["a", "one"],
["b", "two"],
["c", "three"],
]
item_calls = [(url, params) for url, params in session.calls if url.endswith("/items")]
assert [url for url, _ in item_calls] == ["http://abs/api/libraries/books/items"] * 2
assert [params["page"] for _, params in item_calls] == [0, 1]
assert all(params["limit"] == 2 for _, params in item_calls)
def test_fetch_honours_the_library_id_filter(monkeypatch: pytest.MonkeyPatch) -> None:
session = _Session(
libraries=[{"id": "books", "mediaType": "book"}, {"id": "kids", "mediaType": "book"}],
pages={0: {"results": [_item("One", "A")], "total": 1}},
)
_install(monkeypatch, session)
entries = audiobookshelf._fetch_library_entries("http://abs", "token", ["kids"])
assert len(entries) == 1
assert [url for url, _ in session.calls if url.endswith("/items")] == [
"http://abs/api/libraries/kids/items"
]
def test_a_library_that_reports_items_but_lists_none_is_an_error_not_an_empty_library(
monkeypatch: pytest.MonkeyPatch,
) -> None:
session = _Session(
libraries=[{"id": "books", "mediaType": "book"}],
pages={0: {"results": [], "total": 50}},
)
_install(monkeypatch, session)
with pytest.raises(ValueError, match="reports 50 items"):
audiobookshelf._fetch_library_entries("http://abs", "token", [])
@pytest.mark.parametrize("payload", [{"total": 3}, {"results": None, "total": 3}, [1, 2]])
def test_an_item_list_in_an_unexpected_shape_is_an_error(
monkeypatch: pytest.MonkeyPatch, payload: Any
) -> None:
session = _Session(
libraries=[{"id": "books", "mediaType": "book"}],
pages={0: payload},
)
_install(monkeypatch, session)
with pytest.raises(TypeError, match="unexpected shape"):
audiobookshelf._fetch_library_entries("http://abs", "token", [])
def test_a_really_empty_library_indexes_as_empty(monkeypatch: pytest.MonkeyPatch) -> None:
session = _Session(
libraries=[{"id": "books", "mediaType": "book"}],
pages={0: {"results": [], "total": 0}},
)
_install(monkeypatch, session)
assert audiobookshelf._fetch_library_entries("http://abs", "token", []) == []
def test_fetch_stops_on_an_empty_later_page(monkeypatch: pytest.MonkeyPatch) -> None:
session = _Session(
libraries=[{"id": "books", "mediaType": "book"}],
pages={
0: {"results": [{"id": "1", "media": {"metadata": {"title": "One"}}}], "total": 50},
1: {"results": [], "total": 50},
},
)
_install(monkeypatch, session)
assert len(audiobookshelf._fetch_library_entries("http://abs", "token", [])) == 1
assert len([url for url, _ in session.calls if url.endswith("/items")]) == 2
@pytest.mark.parametrize(
("values", "enabled"),
[
(
{
"LIBRARY_CHECK_AUDIOBOOKSHELF_ENABLED": True,
"AUDIOBOOKSHELF_URL": "http://abs/",
"AUDIOBOOKSHELF_TOKEN": "t",
},
True,
),
(
{"LIBRARY_CHECK_AUDIOBOOKSHELF_ENABLED": True, "AUDIOBOOKSHELF_URL": "http://abs/"},
False,
),
(
{
"LIBRARY_CHECK_AUDIOBOOKSHELF_ENABLED": False,
"AUDIOBOOKSHELF_URL": "http://abs/",
"AUDIOBOOKSHELF_TOKEN": "t",
},
False,
),
],
ids=["configured", "missing-token", "disabled"],
)
def test_provider_is_enabled_only_when_configured(
config: dict[str, Any], values: dict[str, Any], enabled: bool
) -> None:
config.update(values)
provider = audiobookshelf.AudiobookshelfLibrary()
assert provider.is_enabled() is enabled
assert provider.describe() == "Audiobookshelf at http://abs"
assert provider.fingerprint() is None
def test_fetch_entries_requires_url_and_token() -> None:
provider = audiobookshelf.AudiobookshelfLibrary()
assert provider.describe() == "Audiobookshelf"
with pytest.raises(ValueError, match="URL and token"):
provider.fetch_entries()
@pytest.mark.parametrize(
("mode", "url", "verify"),
[
("disabled_local", "https://10.0.0.91:13378", False),
("disabled_local", "https://abs.example.com", True),
("enabled", "https://10.0.0.91:13378", True),
],
)
def test_session_honours_certificate_validation_for_the_server_url(
config: dict[str, Any], mode: str, url: str, verify: bool
) -> None:
config["CERTIFICATE_VALIDATION"] = mode
assert audiobookshelf._session(url, "t").verify is verify
def test_configured_url_is_normalized(config: dict[str, Any]) -> None:
config.update(
LIBRARY_CHECK_AUDIOBOOKSHELF_ENABLED=True,
AUDIOBOOKSHELF_URL=' "abs.lan:13378/" ',
AUDIOBOOKSHELF_TOKEN="t",
)
assert (
audiobookshelf.AudiobookshelfLibrary().describe()
== "Audiobookshelf at http://abs.lan:13378"
)
def test_unsaved_form_values_override_the_saved_settings(config: dict[str, Any]) -> None:
config.update(AUDIOBOOKSHELF_URL="http://saved:13378", AUDIOBOOKSHELF_TOKEN="saved")
provider = audiobookshelf.AudiobookshelfLibrary({"AUDIOBOOKSHELF_URL": "http://form:13378"})
assert provider.describe() == "Audiobookshelf at http://form:13378"