Files
shelfmark/shelfmark/metadata_providers
splitsec2 9bd6c51388 fix(hardcover): read the featured series from a search result (#1589)
Closes #1585. @joeRob2468 traced this one to the parser, and the answer
turned out to be sitting in the search response already.

A Hardcover search hit carries the book's featured series as
`featured_series`, with the same shape as the `featured_book_series` of
a full record: the position and a nested series with its id, name and
book count. `_parse_search_result` never read it, so a book requested
from a search card was stored with `series_name: null` and the
`{Series}` folder and the custom script payload came out empty. It now
reads it, through a small helper that `_parse_book` uses too, so there
is no extra Hardcover call per request (the issue's other option was
fetching the full record, which costs one against the rate limit).

I checked it against a live search for "Mistborn The Final Empire". The
top hit parses to series "The Mistborn Saga: The Original Trilogy",
position 1, id 10001, and its search title is still "Mistborn".

That last part is on purpose. I left `_compute_search_title` alone,
since passing it the series would change what gets searched for everyone
and the series name here doesn't equal the "Mistborn" prefix anyway.

Two things that behave differently. A search result now has
`series_name` set, so anything in the UI that shows it for a full record
(the series link, the bigger cover in the release modal) will show it on
search results too. I ran the branch and looked at the release modal for
the top Mistborn result: it shows "#1 of 3 in The Mistborn Saga: The
Original Trilogy" with the View series link. The modal fetches the full
record when it opens, so I can't say how much of that is new on this
path. Requests already stored keep the empty series they were saved
with.

Tests: four in a new file (a hit with a series, hits with none or an
incomplete one, a hit nested in a `document`, and the search title
staying the same), the first and third fail on current main. The
metadata and core suites, ruff and basedpyright are clean.
2026-10-31 01:35:37 -06:00
..

Metadata Providers

This module provides a plugin architecture for fetching book metadata from various sources with a unified interface.

Overview

Metadata providers allow searching for books and retrieving detailed metadata (title, authors, cover images, descriptions, etc.) from external services. The system uses a decorator-based registration pattern, making it easy to add new providers.

Available Providers

Provider Auth Required Description
Hardcover Yes (API key) Modern book tracking platform with GraphQL API. Get your key at hardcover.app/account/api
Open Library No Free, open-source library catalog from the Internet Archive. Rate limited to ~100 requests/minute
Google Books Yes (API key) Google's book database with broad coverage and a free API key option
DNB No Deutsche Nationalbibliothek, the German national library catalog. Best coverage for German-language editions
Databáze knih No Czech and Slovak book catalog. It has no public API, so Shelfmark reads the web pages
Moly.hu No Hungarian book catalog, read from the web pages. No API key required

Core Components

BookMetadata

Dataclass representing a book from a metadata provider:

@dataclass
class BookMetadata:
    provider: str  # Internal provider name (e.g., "hardcover")
    provider_id: str  # ID in that provider's system
    title: str

    # Optional fields
    provider_display_name: str  # Human-readable name (e.g., "Hardcover")
    authors: List[str]
    isbn_10: str
    isbn_13: str
    cover_url: str
    description: str
    publisher: str
    publish_year: int
    language: str
    genres: List[str]
    source_url: str  # Link to book on provider's site
    display_fields: List[DisplayField]  # Provider-specific display data

DisplayField

Provider-specific metadata for UI cards (ratings, page counts, reader counts, etc.):

@dataclass
class DisplayField:
    label: str  # e.g., "Rating", "Pages", "Readers"
    value: str  # e.g., "4.5", "496", "8,041"
    icon: str  # Icon name: "star", "book", "users", "editions"

MetadataSearchOptions

Unified search options that work across all providers:

@dataclass
class MetadataSearchOptions:
    query: str
    search_type: SearchType = SearchType.GENERAL  # GENERAL, TITLE, AUTHOR, ISBN
    language: str = None  # ISO 639-1 code (e.g., "en")
    sort: SortOrder = SortOrder.RELEVANCE
    limit: int = 40
    page: int = 1
    user_id: int | None = None  # Searching user, for per-user settings

SortOrder

Available sort options (provider support varies):

Sort Order Description Hardcover Open Library
RELEVANCE Best match first (default) ✓ ✓
POPULARITY Most popular first ✓ ✗
RATING Highest rated first ✓ ✗
NEWEST Most recently published ✓ ✓
OLDEST Oldest published first ✓ ✓

MetadataProvider (Abstract Base Class)

All providers must implement this interface:

class MetadataProvider(ABC):
    name: str  # Internal identifier
    display_name: str  # Human-readable name
    requires_auth: bool  # True if API key required
    supported_sorts: List[SortOrder]  # Supported sort options

    @abstractmethod
    def search(self, options: MetadataSearchOptions) -> List[BookMetadata]:
        """Search for books using the provided options."""
        pass

    @abstractmethod
    def get_book(self, book_id: str) -> Optional[BookMetadata]:
        """Get a specific book by provider ID."""
        pass

    @abstractmethod
    def search_by_isbn(self, isbn: str) -> Optional[BookMetadata]:
        """Search for a book by ISBN."""
        pass

    @abstractmethod
    def is_available(self) -> bool:
        """Check if this provider is configured and available."""
        pass

Registry Functions

Provider Registration

from shelfmark.metadata_providers import register_provider


@register_provider("my_provider")
class MyProvider(MetadataProvider): ...

Getting Providers

from shelfmark.metadata_providers import (
    get_provider,
    get_configured_provider,
    get_provider_kwargs,
    list_providers,
    is_provider_registered,
)

# Get specific provider with kwargs
provider = get_provider("hardcover", api_key="...")

# Get currently configured provider (from settings)
provider = get_configured_provider()

# Get provider-specific kwargs from config
kwargs = get_provider_kwargs("hardcover")  # {"api_key": "..."}

# List all registered providers
providers = list_providers()
# [{"name": "hardcover", "display_name": "Hardcover", "requires_auth": True}, ...]

# Check if provider exists
exists = is_provider_registered("hardcover")  # True

Sort Options

from shelfmark.metadata_providers import get_provider_sort_options

# Get sort options for a specific provider
options = get_provider_sort_options("hardcover")
# [{"value": "relevance", "label": "Most relevant"}, ...]

# Get sort options for configured provider
options = get_provider_sort_options()  # Uses METADATA_PROVIDER from config

Creating a New Provider

  1. Create a new file in shelfmark/metadata_providers/ (e.g., my_provider.py)

  2. Implement the provider:

from shelfmark.metadata_providers import (
    BookMetadata,
    DisplayField,
    MetadataProvider,
    MetadataSearchOptions,
    SearchType,
    SortOrder,
    register_provider,
)
from shelfmark.core.settings_registry import (
    register_settings,
    HeadingField,
    PasswordField,
    ActionButton,
)
from shelfmark.core.config import config


@register_provider("my_provider")
class MyProvider(MetadataProvider):
    name = "my_provider"
    display_name = "My Provider"
    requires_auth = True
    supported_sorts = [SortOrder.RELEVANCE, SortOrder.NEWEST]

    def __init__(self, api_key: str = None):
        self.api_key = api_key or config.get("MY_PROVIDER_API_KEY", "")

    def is_available(self) -> bool:
        return bool(self.api_key)

    def search(self, options: MetadataSearchOptions) -> List[BookMetadata]:
        # Handle ISBN search separately
        if options.search_type == SearchType.ISBN:
            result = self.search_by_isbn(options.query)
            return [result] if result else []

        # Implement search logic...
        return []

    def get_book(self, book_id: str) -> Optional[BookMetadata]:
        # Implement get book logic...
        return None

    def search_by_isbn(self, isbn: str) -> Optional[BookMetadata]:
        # Implement ISBN search logic...
        return None


# Settings for the UI
@register_settings("my_provider", "My Provider", icon="book", order=53, group="metadata_providers")
def my_provider_settings():
    return [
        HeadingField(
            key="my_provider_heading",
            title="My Provider",
            description="Description of your provider",
            link_url="https://myprovider.com",
            link_text="myprovider.com",
        ),
        PasswordField(
            key="MY_PROVIDER_API_KEY",
            label="API Key",
            description="Your API key",
            required=True,
        ),
        ActionButton(
            key="test_connection",
            label="Test Connection",
            style="primary",
            callback=_test_connection,
        ),
    ]
  1. Import your provider in __init__.py:
try:
    from shelfmark.metadata_providers import my_provider  # noqa: F401
except ImportError:
    pass  # Provider is optional
  1. Add provider kwargs to get_provider_kwargs() in __init__.py:
def get_provider_kwargs(provider_name: str) -> Dict:
    kwargs: Dict = {}
    if provider_name == "hardcover":
        kwargs["api_key"] = app_config.get("HARDCOVER_API_KEY", "")
    elif provider_name == "my_provider":
        kwargs["api_key"] = app_config.get("MY_PROVIDER_API_KEY", "")
    return kwargs

Caching

Providers should use the @cacheable decorator for API calls:

from shelfmark.core.cache import cacheable
from shelfmark.config.env import (
    METADATA_CACHE_SEARCH_TTL,
    METADATA_CACHE_BOOK_TTL,
)


@cacheable(ttl=METADATA_CACHE_SEARCH_TTL, key_prefix="myprovider:search")
def _search_cached(self, cache_key: str, options: MetadataSearchOptions):
    # Cached search implementation
    pass


@cacheable(ttl=METADATA_CACHE_BOOK_TTL, key_prefix="myprovider:book")
def get_book(self, book_id: str):
    # Cached book lookup
    pass

Rate Limiting

For providers with rate limits (like Open Library), implement a rate limiter:

from shelfmark.metadata_providers.openlibrary import RateLimiter

# 90 requests per 60 seconds
rate_limiter = RateLimiter(max_requests=90, window_seconds=60)


def make_request(self):
    rate_limiter.wait_if_needed()  # Blocks if rate limited
    # ... make request

Configuration

Provider settings are stored in CONFIG_DIR/plugins/<provider_name>.json and managed via the Settings UI. See Plugin Settings Guide for detailed documentation on adding settings to your provider.

Environment Variables

Variable Default Description
METADATA_PROVIDER "" Active metadata provider name
METADATA_CACHE_SEARCH_TTL 3600 Search cache TTL in seconds
METADATA_CACHE_BOOK_TTL 86400 Book lookup cache TTL in seconds