Closes #1585. @joeRob2468 traced this one to the parser, and the answer turned out to be sitting in the search response already. A Hardcover search hit carries the book's featured series as `featured_series`, with the same shape as the `featured_book_series` of a full record: the position and a nested series with its id, name and book count. `_parse_search_result` never read it, so a book requested from a search card was stored with `series_name: null` and the `{Series}` folder and the custom script payload came out empty. It now reads it, through a small helper that `_parse_book` uses too, so there is no extra Hardcover call per request (the issue's other option was fetching the full record, which costs one against the rate limit). I checked it against a live search for "Mistborn The Final Empire". The top hit parses to series "The Mistborn Saga: The Original Trilogy", position 1, id 10001, and its search title is still "Mistborn". That last part is on purpose. I left `_compute_search_title` alone, since passing it the series would change what gets searched for everyone and the series name here doesn't equal the "Mistborn" prefix anyway. Two things that behave differently. A search result now has `series_name` set, so anything in the UI that shows it for a full record (the series link, the bigger cover in the release modal) will show it on search results too. I ran the branch and looked at the release modal for the top Mistborn result: it shows "#1 of 3 in The Mistborn Saga: The Original Trilogy" with the View series link. The modal fetches the full record when it opens, so I can't say how much of that is new on this path. Requests already stored keep the empty series they were saved with. Tests: four in a new file (a hit with a series, hits with none or an incomplete one, a hit nested in a `document`, and the search title staying the same), the first and third fail on current main. The metadata and core suites, ruff and basedpyright are clean.
Metadata Providers
This module provides a plugin architecture for fetching book metadata from various sources with a unified interface.
Overview
Metadata providers allow searching for books and retrieving detailed metadata (title, authors, cover images, descriptions, etc.) from external services. The system uses a decorator-based registration pattern, making it easy to add new providers.
Available Providers
| Provider | Auth Required | Description |
|---|---|---|
| Hardcover | Yes (API key) | Modern book tracking platform with GraphQL API. Get your key at hardcover.app/account/api |
| Open Library | No | Free, open-source library catalog from the Internet Archive. Rate limited to ~100 requests/minute |
| Google Books | Yes (API key) | Google's book database with broad coverage and a free API key option |
| DNB | No | Deutsche Nationalbibliothek, the German national library catalog. Best coverage for German-language editions |
| Databáze knih | No | Czech and Slovak book catalog. It has no public API, so Shelfmark reads the web pages |
| Moly.hu | No | Hungarian book catalog, read from the web pages. No API key required |
Core Components
BookMetadata
Dataclass representing a book from a metadata provider:
@dataclass
class BookMetadata:
provider: str # Internal provider name (e.g., "hardcover")
provider_id: str # ID in that provider's system
title: str
# Optional fields
provider_display_name: str # Human-readable name (e.g., "Hardcover")
authors: List[str]
isbn_10: str
isbn_13: str
cover_url: str
description: str
publisher: str
publish_year: int
language: str
genres: List[str]
source_url: str # Link to book on provider's site
display_fields: List[DisplayField] # Provider-specific display data
DisplayField
Provider-specific metadata for UI cards (ratings, page counts, reader counts, etc.):
@dataclass
class DisplayField:
label: str # e.g., "Rating", "Pages", "Readers"
value: str # e.g., "4.5", "496", "8,041"
icon: str # Icon name: "star", "book", "users", "editions"
MetadataSearchOptions
Unified search options that work across all providers:
@dataclass
class MetadataSearchOptions:
query: str
search_type: SearchType = SearchType.GENERAL # GENERAL, TITLE, AUTHOR, ISBN
language: str = None # ISO 639-1 code (e.g., "en")
sort: SortOrder = SortOrder.RELEVANCE
limit: int = 40
page: int = 1
user_id: int | None = None # Searching user, for per-user settings
SortOrder
Available sort options (provider support varies):
| Sort Order | Description | Hardcover | Open Library |
|---|---|---|---|
RELEVANCE |
Best match first (default) | ✓ | ✓ |
POPULARITY |
Most popular first | ✓ | ✗ |
RATING |
Highest rated first | ✓ | ✗ |
NEWEST |
Most recently published | ✓ | ✓ |
OLDEST |
Oldest published first | ✓ | ✓ |
MetadataProvider (Abstract Base Class)
All providers must implement this interface:
class MetadataProvider(ABC):
name: str # Internal identifier
display_name: str # Human-readable name
requires_auth: bool # True if API key required
supported_sorts: List[SortOrder] # Supported sort options
@abstractmethod
def search(self, options: MetadataSearchOptions) -> List[BookMetadata]:
"""Search for books using the provided options."""
pass
@abstractmethod
def get_book(self, book_id: str) -> Optional[BookMetadata]:
"""Get a specific book by provider ID."""
pass
@abstractmethod
def search_by_isbn(self, isbn: str) -> Optional[BookMetadata]:
"""Search for a book by ISBN."""
pass
@abstractmethod
def is_available(self) -> bool:
"""Check if this provider is configured and available."""
pass
Registry Functions
Provider Registration
from shelfmark.metadata_providers import register_provider
@register_provider("my_provider")
class MyProvider(MetadataProvider): ...
Getting Providers
from shelfmark.metadata_providers import (
get_provider,
get_configured_provider,
get_provider_kwargs,
list_providers,
is_provider_registered,
)
# Get specific provider with kwargs
provider = get_provider("hardcover", api_key="...")
# Get currently configured provider (from settings)
provider = get_configured_provider()
# Get provider-specific kwargs from config
kwargs = get_provider_kwargs("hardcover") # {"api_key": "..."}
# List all registered providers
providers = list_providers()
# [{"name": "hardcover", "display_name": "Hardcover", "requires_auth": True}, ...]
# Check if provider exists
exists = is_provider_registered("hardcover") # True
Sort Options
from shelfmark.metadata_providers import get_provider_sort_options
# Get sort options for a specific provider
options = get_provider_sort_options("hardcover")
# [{"value": "relevance", "label": "Most relevant"}, ...]
# Get sort options for configured provider
options = get_provider_sort_options() # Uses METADATA_PROVIDER from config
Creating a New Provider
-
Create a new file in
shelfmark/metadata_providers/(e.g.,my_provider.py) -
Implement the provider:
from shelfmark.metadata_providers import (
BookMetadata,
DisplayField,
MetadataProvider,
MetadataSearchOptions,
SearchType,
SortOrder,
register_provider,
)
from shelfmark.core.settings_registry import (
register_settings,
HeadingField,
PasswordField,
ActionButton,
)
from shelfmark.core.config import config
@register_provider("my_provider")
class MyProvider(MetadataProvider):
name = "my_provider"
display_name = "My Provider"
requires_auth = True
supported_sorts = [SortOrder.RELEVANCE, SortOrder.NEWEST]
def __init__(self, api_key: str = None):
self.api_key = api_key or config.get("MY_PROVIDER_API_KEY", "")
def is_available(self) -> bool:
return bool(self.api_key)
def search(self, options: MetadataSearchOptions) -> List[BookMetadata]:
# Handle ISBN search separately
if options.search_type == SearchType.ISBN:
result = self.search_by_isbn(options.query)
return [result] if result else []
# Implement search logic...
return []
def get_book(self, book_id: str) -> Optional[BookMetadata]:
# Implement get book logic...
return None
def search_by_isbn(self, isbn: str) -> Optional[BookMetadata]:
# Implement ISBN search logic...
return None
# Settings for the UI
@register_settings("my_provider", "My Provider", icon="book", order=53, group="metadata_providers")
def my_provider_settings():
return [
HeadingField(
key="my_provider_heading",
title="My Provider",
description="Description of your provider",
link_url="https://myprovider.com",
link_text="myprovider.com",
),
PasswordField(
key="MY_PROVIDER_API_KEY",
label="API Key",
description="Your API key",
required=True,
),
ActionButton(
key="test_connection",
label="Test Connection",
style="primary",
callback=_test_connection,
),
]
- Import your provider in
__init__.py:
try:
from shelfmark.metadata_providers import my_provider # noqa: F401
except ImportError:
pass # Provider is optional
- Add provider kwargs to
get_provider_kwargs()in__init__.py:
def get_provider_kwargs(provider_name: str) -> Dict:
kwargs: Dict = {}
if provider_name == "hardcover":
kwargs["api_key"] = app_config.get("HARDCOVER_API_KEY", "")
elif provider_name == "my_provider":
kwargs["api_key"] = app_config.get("MY_PROVIDER_API_KEY", "")
return kwargs
Caching
Providers should use the @cacheable decorator for API calls:
from shelfmark.core.cache import cacheable
from shelfmark.config.env import (
METADATA_CACHE_SEARCH_TTL,
METADATA_CACHE_BOOK_TTL,
)
@cacheable(ttl=METADATA_CACHE_SEARCH_TTL, key_prefix="myprovider:search")
def _search_cached(self, cache_key: str, options: MetadataSearchOptions):
# Cached search implementation
pass
@cacheable(ttl=METADATA_CACHE_BOOK_TTL, key_prefix="myprovider:book")
def get_book(self, book_id: str):
# Cached book lookup
pass
Rate Limiting
For providers with rate limits (like Open Library), implement a rate limiter:
from shelfmark.metadata_providers.openlibrary import RateLimiter
# 90 requests per 60 seconds
rate_limiter = RateLimiter(max_requests=90, window_seconds=60)
def make_request(self):
rate_limiter.wait_if_needed() # Blocks if rate limited
# ... make request
Configuration
Provider settings are stored in CONFIG_DIR/plugins/<provider_name>.json and managed via the Settings UI. See Plugin Settings Guide for detailed documentation on adding settings to your provider.
Environment Variables
| Variable | Default | Description |
|---|---|---|
METADATA_PROVIDER |
"" |
Active metadata provider name |
METADATA_CACHE_SEARCH_TTL |
3600 |
Search cache TTL in seconds |
METADATA_CACHE_BOOK_TTL |
86400 |
Book lookup cache TTL in seconds |