diff --git a/docs/dev/index.md b/docs/dev/index.md index c066acaf..f00c7632 100644 --- a/docs/dev/index.md +++ b/docs/dev/index.md @@ -1,3 +1,7 @@ # Developer Documentation -TODO +Guides for adding to Shelfmark. + +- [Release Sources Plugin Guide](release-sources-plugin-guide.md): add a source that searches for releases and a handler that downloads them +- [Plugin Settings Guide](plugin-settings.md): add settings tabs and fields for a plugin to the Settings UI +- [Metadata Providers](../../shelfmark/metadata_providers/README.md): add a metadata provider diff --git a/docs/dev/plugin-settings.md b/docs/dev/plugin-settings.md index 6bab9a0e..003f7a62 100644 --- a/docs/dev/plugin-settings.md +++ b/docs/dev/plugin-settings.md @@ -137,9 +137,11 @@ SelectField( ) ``` +`SelectField` also takes `filter_by_field`, the key of another field whose value filters this field's options through each option's `childOf` property. `options` can be a list or a function that returns one, evaluated when the schema is built. + ### MultiSelectField -Multi-choice selection from a list of options. +Multi-choice selection from a list of options. `variant` is `"pills"` (default) or `"dropdown"` for a checkbox dropdown. ```python MultiSelectField( @@ -156,6 +158,56 @@ MultiSelectField( ) ``` +### TagListField + +Editable list of free-form strings, shown as tags. The value is a list of strings. + +```python +TagListField( + key="MY_PLUGIN_MIRROR_URLS", + label="Mirror URLs", + description="Type a URL and press Enter to add it", + placeholder="https://mirror.example", + normalize_urls=True, # default: entries are cleaned up as URLs +) +``` + +### OrderableListField + +A list of options that can be toggled on and off and reordered. The value is a list of `{"id": ..., "enabled": ...}` objects in priority order. + +```python +OrderableListField( + key="MY_PLUGIN_SOURCE_PRIORITY", + label="Source priority", + options=[ + {"id": "a", "label": "Source A"}, + {"id": "b", "label": "Source B", "isLocked": True, "disabledReason": "Needs an API key"}, + ], + default=[{"id": "a", "enabled": True}, {"id": "b", "enabled": False}], +) +``` + +An option can set `isLocked` (the toggle is disabled), `isPinned` (it cannot be reordered), `description` and `section`. + +### TableField + +Editable table of structured rows. The value is a list of objects, one per row. + +```python +TableField( + key="MY_PLUGIN_RULES", + label="Rules", + columns=[ + {"key": "name", "label": "Name", "type": "text", "placeholder": "Rule name"}, + {"key": "mode", "label": "Mode", "type": "select", "options": [...]}, + ], + add_label="Add rule", + empty_message="No rules yet", + default=[], +) +``` + ### ActionButton Button that executes a callback function. Does not store a value. @@ -179,6 +231,16 @@ def my_callback_fn(): return {"success": False, "message": f"Failed: {str(e)}"} ``` +A callback that declares a `current_values` parameter receives the form's current, unsaved values as a dict, so a "Test connection" button can test what the user just typed instead of what was last saved: + +```python +def my_callback_fn(current_values: dict): + api_key = current_values.get("MY_PLUGIN_API_KEY", "") + ... +``` + +The decorator calls your settings function as soon as the module is imported, so define the callback above `@register_settings`, not below it. + ### HeadingField Display-only section heading with optional link. Does not store a value. @@ -235,6 +297,10 @@ All field types support these common properties: | `description` | `str` | `""` | Help text shown below the field | | `default` | `Any` | `None` | Default value if not set | | `required` | `bool` | `False` | Whether the field must have a value | +| `env_var` | `str` | `None` | Environment variable name to use instead of `key` | +| `env_supported` | `bool` | `True` | Set `False` for a UI-only setting that cannot come from an environment variable | +| `user_overridable` | `bool` | `False` | Let an admin set a per-user override of this field | +| `universal_only` | `bool` | `False` | Only show the field in Universal search mode | | `disabled` | `bool` | `False` | Disable the field (greyed out) | | `disabled_reason` | `str` | `""` | Explanation shown when disabled | | `requires_restart` | `bool` | `False` | Whether changes require container restart | @@ -309,16 +375,34 @@ def plugin_a_settings(): **Existing groups:** - `direct_download` (order=20): For download-related settings -- `metadata_providers` (order=50): For metadata provider plugins +- `metadata_providers` (order=12): For metadata provider plugins ## Value Resolution Priority Settings values are resolved in this order (highest priority first): -1. **Config File** - Stored in `CONFIG_DIR/plugins/.json` -2. **Field Default** - Value specified in the field definition +1. **Environment variable** - The field's key, or `env_var` when set. Skipped for a field with `env_supported=False`. A value that comes from an environment variable is shown locked in the UI +2. **Config File** - Stored in `CONFIG_DIR/plugins/.json` +3. **Field Default** - Value specified in the field definition -The `general` tab uses `CONFIG_DIR/settings.json` instead of the plugins subdirectory. +The `general` and `search_mode` tabs use `CONFIG_DIR/settings.json` instead of the plugins subdirectory. + +## Validating or Transforming on Save + +`register_on_save(tab_name, handler)` runs a function each time the tab is saved. The handler receives a copy of the values being saved and returns a dict. Return `{"error": True, "message": "..."}` to refuse the save and show the message, or `{"values": {...}}` to save changed values. + +```python +from shelfmark.core.settings_registry import register_on_save + + +def _validate_my_plugin(values: dict) -> dict: + if values.get("MY_PLUGIN_TIMEOUT", 30) < 5: + return {"error": True, "message": "Timeout must be at least 5 seconds"} + return {"values": values} + + +register_on_save("my_plugin", _validate_my_plugin) +``` ## Reading Setting Values @@ -339,7 +423,7 @@ all_settings = config.get_all() ``` The config singleton: -- Automatically resolves values from config files with field defaults as fallback +- Resolves values in the order above, with the field default as the last fallback - Caches values for performance - Refreshes automatically when settings are updated via the UI @@ -350,9 +434,12 @@ Here's a complete example for a metadata provider plugin: ```python # shelfmark/metadata_providers/my_provider.py -from shelfmark.metadata_providers.base import ( +from shelfmark.metadata_providers import ( + BookMetadata, MetadataProvider, + MetadataSearchOptions, register_provider, + register_provider_kwargs, ) from shelfmark.core.settings_registry import ( register_settings, @@ -426,6 +513,12 @@ def my_provider_settings(): ] +@register_provider_kwargs("my_provider") +def _my_provider_kwargs() -> dict: + """Keyword arguments for MyProvider(...), read from settings each time one is created.""" + return {"api_key": config.get("MY_PROVIDER_API_KEY", "")} + + @register_provider("my_provider") class MyProvider(MetadataProvider): """My Provider metadata implementation.""" @@ -434,22 +527,28 @@ class MyProvider(MetadataProvider): display_name = "My Provider" requires_auth = True - def __init__(self, api_key: str = None): - self.api_key = api_key or config.get("MY_PROVIDER_API_KEY", "") + def __init__(self, api_key: str = ""): + self.api_key = api_key self.base_url = config.get("MY_PROVIDER_BASE_URL", "https://api.myprovider.com/v1") def is_available(self) -> bool: return bool(self.api_key) - def search(self, query: str): + def search(self, options: MetadataSearchOptions) -> list[BookMetadata]: # Implementation... - pass + return [] - def get_book(self, book_id: str): + def get_book(self, book_id: str) -> BookMetadata | None: # Implementation... - pass + return None + + def search_by_isbn(self, isbn: str) -> BookMetadata | None: + # Implementation... + return None ``` +All four methods above are abstract, so a provider that leaves one out cannot be instantiated. The provider module also has to be imported once so its decorators run: add it to the list of imports at the bottom of `shelfmark/metadata_providers/__init__.py`. + ## Complete Example: Release Source Here's a complete example for a release source plugin: @@ -457,11 +556,11 @@ Here's a complete example for a release source plugin: ```python # shelfmark/release_sources/my_source.py -from shelfmark.release_sources.base import ( - ReleaseSource, +from shelfmark.release_sources import ( DownloadHandler, - register_source, + ReleaseSource, register_handler, + register_source, ) from shelfmark.core.settings_registry import ( register_settings, @@ -559,22 +658,25 @@ class MySource(ReleaseSource): def is_available(self) -> bool: return self.enabled - def search(self, book): + def search(self, book, plan, *, expand_search=False, content_type="ebook"): # Implementation... - pass + return [] @register_handler("my_source") class MySourceHandler(DownloadHandler): """Handler for downloading from My Source.""" - name = "my_source" - - def download(self, release, output_path): + def download(self, task, cancel_flag, progress_callback, status_callback): # Implementation... - pass + return None + + def cancel(self, task_id): + return False ``` +The full source and handler interfaces, and how to register the module, are in the [Release Sources Plugin Guide](release-sources-plugin-guide.md). + ## Best Practices 1. **Use descriptive keys**: Keys should be uppercase and prefixed with your plugin name (e.g., `MY_PLUGIN_API_KEY`) diff --git a/docs/dev/release-sources-plugin-guide.md b/docs/dev/release-sources-plugin-guide.md index a4fbaa4f..ce17bd9e 100644 --- a/docs/dev/release-sources-plugin-guide.md +++ b/docs/dev/release-sources-plugin-guide.md @@ -54,12 +54,12 @@ This separation allows: ### Plugin Lifecycle 1. **Registration** (import time): Decorators register your classes in the global registry -2. **Discovery**: `list_available_sources()` checks `is_available()` on each source -3. **Search**: User selects source, `search(book_metadata)` returns releases +2. **Discovery**: `list_available_sources()` lists every registered source and reports `enabled` from its `is_available()` +3. **Search**: User selects source, `search(book, plan, ...)` returns releases 4. **Queue**: User selects release, it becomes a `DownloadTask` in the queue 5. **Download**: Orchestrator calls `handler.download()` with callbacks 6. **Progress**: Handler reports via `progress_callback` and `status_callback` -7. **Completion**: Handler returns file path or `None` on failure +7. **Completion**: Handler returns a file path (or a `HandoffResult` for an external handoff) and `None` on failure ### Source vs Handler @@ -85,6 +85,7 @@ from typing import Callable, List, Optional from threading import Event from shelfmark.release_sources import ( + HandoffResult, Release, ReleaseSource, DownloadHandler, @@ -93,6 +94,7 @@ from shelfmark.release_sources import ( ) from shelfmark.metadata_providers import BookMetadata from shelfmark.core.models import DownloadTask +from shelfmark.core.search_plan import ReleaseSearchPlan @register_source("my_source") @@ -100,7 +102,14 @@ class MySource(ReleaseSource): name = "my_source" display_name = "My Source" - def search(self, book: BookMetadata) -> List[Release]: + def search( + self, + book: BookMetadata, + plan: ReleaseSearchPlan, + *, + expand_search: bool = False, + content_type: str = "ebook", + ) -> List[Release]: # Your search logic here return [] @@ -116,7 +125,7 @@ class MyHandler(DownloadHandler): cancel_flag: Event, progress_callback: Callable[[float], None], status_callback: Callable[[str, Optional[str]], None], - ) -> Optional[str]: + ) -> str | HandoffResult | None: # Your download logic here return None # Return file path on success @@ -124,13 +133,18 @@ class MyHandler(DownloadHandler): return False # Cancellation via cancel_flag ``` -Register the import in `shelfmark/release_sources/__init__.py`: +Add the module to `_BUILTIN_SOURCE_MODULES` in `shelfmark/release_sources/__init__.py`: ```python -# At the bottom of the file -from shelfmark.release_sources import my_source # noqa: F401, E402 +_BUILTIN_SOURCE_MODULES = ( + "shelfmark.release_sources.audiobookbay", + # ... + "shelfmark.release_sources.my_source", +) ``` +The built-in sources are packages (`jackett/`, `webshare/`, `slskd/` and so on) with the source, handler and settings in separate modules that the package `__init__.py` imports. A single module is fine for a small source. + --- ## ReleaseSource Interface @@ -139,7 +153,8 @@ The `ReleaseSource` abstract base class defines the search interface: ```python from abc import ABC, abstractmethod -from typing import List +from typing import ClassVar, List +from shelfmark.core.search_plan import ReleaseSearchPlan from shelfmark.metadata_providers import BookMetadata from shelfmark.release_sources import Release, ReleaseColumnConfig @@ -149,18 +164,29 @@ class ReleaseSource(ABC): name: str # Internal identifier: "direct_download", "prowlarr" display_name: str # User-facing name: "Direct Download", "Prowlarr" + supported_content_types: ClassVar[list[str]] = ["ebook", "audiobook"] + can_be_default: bool = True # Whether it can be picked as the default source in settings @abstractmethod - def search(self, book: BookMetadata) -> List[Release]: + def search( + self, + book: BookMetadata, + plan: ReleaseSearchPlan, + *, + expand_search: bool = False, + content_type: str = "ebook", + ) -> List[Release]: """Search for releases of a book. Args: book: Book metadata including title, authors, ISBN, language + plan: Pre-computed search inputs shared by all sources (see below) + expand_search: True when the user asked for a wider search + content_type: "ebook" or "audiobook" Returns: List of Release objects found from this source """ - pass @abstractmethod def is_available(self) -> bool: @@ -169,10 +195,8 @@ class ReleaseSource(ABC): Returns: True if source can be used for searching """ - pass - @classmethod - def get_column_config(cls) -> ReleaseColumnConfig: + def get_column_config(self) -> ReleaseColumnConfig: """Get the column configuration for this source's release list UI. Override to customize how releases are displayed in the modal. @@ -181,17 +205,39 @@ class ReleaseSource(ABC): return _default_column_config() ``` +Optional hooks, all with working defaults: + +| Method | Purpose | +|--------|---------| +| `search_results_are_releases()` | Return `True` when the source's browse results are already concrete releases. Request policy then treats "request book" as "request release" for it | +| `get_record(record_id, *, fetch_download_count=True)` | Resolve a source-native record for browse flows. Raises `NotImplementedError` by default | +| `get_destination_override(task)` | Return a source-specific destination `Path` for a queued download, or `None` | + +### The search plan + +`plan` is a `ReleaseSearchPlan` (`shelfmark/core/search_plan.py`) that Shelfmark builds once per search, so each source does not redo the same work: + +| Field | Description | +|-------|-------------| +| `languages` | Language codes to filter by, or `None` | +| `isbn_candidates` | ISBNs to try, best first | +| `author` | Author string to search with | +| `title_variants`, `grouped_title_variants` | Title queries, best first. `plan.primary_query` is the first one | +| `manual_query` | Text the user typed into the search box, when they did | +| `indexers` | Indexer names for indexer-based sources | +| `source_filters` | Filters from the source's own search form | + ### Search Method -The `search` method receives complete book metadata and returns standardized releases: +The `search` method receives the book metadata and the search plan, and returns standardized releases: ```python -def search(self, book: BookMetadata) -> List[Release]: +def search(self, book, plan, *, expand_search=False, content_type="ebook") -> List[Release]: results = [] # Strategy 1: Search by ISBN (most accurate) - if book.isbn_13: - api_results = self.api.search_isbn(book.isbn_13) + if plan.isbn_candidates: + api_results = self.api.search_isbn(plan.isbn_candidates[0]) for r in api_results: results.append( Release( @@ -204,9 +250,9 @@ def search(self, book: BookMetadata) -> List[Release]: ) ) - # Strategy 2: Fallback to title + author search + # Strategy 2: Fallback to the planned title query if not results: - query = f"{book.title} {book.authors[0] if book.authors else ''}" + query = plan.manual_query or plan.primary_query api_results = self.api.search_text(query) for r in api_results: results.append(Release(...)) @@ -242,6 +288,7 @@ from abc import ABC, abstractmethod from typing import Callable, Optional from threading import Event from shelfmark.core.models import DownloadTask +from shelfmark.release_sources import HandoffResult class DownloadHandler(ABC): @@ -254,7 +301,7 @@ class DownloadHandler(ABC): cancel_flag: Event, progress_callback: Callable[[float], None], status_callback: Callable[[str, Optional[str]], None], - ) -> Optional[str]: + ) -> str | HandoffResult | None: """Execute download and return path to downloaded file. Args: @@ -264,7 +311,9 @@ class DownloadHandler(ABC): status_callback: Call with (status, optional_message) Returns: - Absolute path to downloaded file, or None if failed/cancelled + Absolute path to downloaded file, a HandoffResult when the download + was handed to an external client and finished without a book payload, + or None if failed/cancelled """ pass @@ -303,6 +352,15 @@ def list_files(self, release_data: dict[str, Any]) -> list[PackFile] | None: Prowlarr parses the `.torrent` it already fetches (magnet-only releases return `None`), and AudiobookBay reads the file table off its detail page. +### Optional: Other Handler Hooks + +| Method | Purpose | +|--------|---------| +| `post_process_cleanup(task, *, success)` | Runs after Shelfmark has post-processed the download. External clients use it to clean up on their side only once the files are safely imported | +| `build_retry_resolution_fields(release_data)` | Return private queue-time fields the handler needs to retry after a restart | +| `list_authors(release_data)` | Return authors read from a source page, for sources whose search results can leave the author out. `None` when there is nothing more to say | +| `seed_complaints(release_data)` | How many comments on the release's page say nobody is seeding it, for sources with no seeder count of their own. `None` when there is no such signal | + ### Download Method Parameters | Parameter | Type | Description | @@ -318,12 +376,13 @@ Prowlarr parses the `.torrent` it already fetches (magnet-only releases return |--------|-------------|-------------| | `"queued"` | Waiting in queue | Set by orchestrator, rarely needed in handler | | `"resolving"` | Pre-download phase | Fetching metadata, extracting URLs, connecting | +| `"locating"` | Waiting for the finished files to show up | Set by the handler with `status_callback` once an external client or peer reports done, as the torrent, usenet and Soulseek handlers do | | `"downloading"` | Active download | File transfer in progress | | `"complete"` | Successfully finished | Set by orchestrator when handler returns a file path | | `"error"` | Failed | Any unrecoverable error (handler should set this) | | `"cancelled"` | User cancelled | Set by orchestrator when `cancel_flag` is triggered | -**Note**: The orchestrator automatically sets `"complete"` when your handler returns a valid file path, and `"cancelled"` when cancellation is detected. Your handler should primarily use `"resolving"`, `"downloading"`, and `"error"`. +**Note**: The orchestrator automatically sets `"complete"` when your handler returns a valid file path, and `"cancelled"` when cancellation is detected. Your handler should primarily use `"resolving"`, `"downloading"`, and `"error"`, plus `"locating"` if it waits on an external client. ### Typical Download Flow @@ -391,19 +450,32 @@ class Release: title: str # Display title format: Optional[str] = None # File format: "epub", "mobi", "pdf" + language: Optional[str] = None # ISO 639-1 code: "en", "de", "fr" size: Optional[str] = None # Human-readable: "5.2 MB" size_bytes: Optional[int] = None # For sorting download_url: Optional[str] = None # Direct download URL if available info_url: Optional[str] = None # Link to tracker/info page - protocol: Optional[str] = None # "http", "torrent", "usenet" + protocol: Optional[ReleaseProtocol] = None # See below indexer: Optional[str] = None # Display name: "Direct Download", "My Indexer" seeders: Optional[int] = None # For torrents + peers: Optional[str] = None # For torrents: "seeders/leechers" display string + content_type: Optional[str] = None # "ebook" or "audiobook", kept from the search extra: Dict = field(default_factory=dict) # Source-specific metadata ``` +`ReleaseProtocol` is a string enum: + +| Member | Value | Used for | +|--------|-------|----------| +| `HTTP` | `"http"` | Direct HTTP download | +| `TORRENT` | `"torrent"` | BitTorrent | +| `NZB` | `"nzb"` | Usenet NZB | +| `DCC` | `"dcc"` | IRC DCC | +| `SOULSEEK` | `"soulseek"` | Soulseek peer transfer (via slskd) | + #### Using the `extra` Field Store source-specific data in `extra` that doesn't fit standard fields: @@ -440,22 +512,36 @@ class DownloadTask: # Display info (from Release.extra or Release fields) author: Optional[str] = None + year: Optional[str] = None format: Optional[str] = None size: Optional[str] = None preview: Optional[str] = None # Cover image URL + content_type: Optional[str] = None + source_url: Optional[str] = None # Original release URL, for source-specific handlers + + # Library naming templates + series_name: Optional[str] = None + series_position: Optional[float] = None + subtitle: Optional[str] = None + narrator: Optional[str] = None + language: Optional[str] = None # Runtime state (managed by orchestrator) priority: int = 0 added_time: float = field(default_factory=time.time) progress: float = 0.0 - status: str = "queued" + status: QueueStatus = QueueStatus.QUEUED status_message: Optional[str] = None download_path: Optional[str] = None + + # ... plus retry fields (retry_*), the output selection (output_mode, output_args), + # multi-book pack fields (multi_book, book_plan, ignored_files) and user fields + # (user_id, username, request_id). See shelfmark/core/models.py for the full class. ``` ### BookMetadata -Book information passed to `search()`: +Book information passed to `search()` (the class lives in `shelfmark/metadata_providers/__init__.py`): ```python @dataclass @@ -475,7 +561,20 @@ class BookMetadata: language: Optional[str] = None # ISO 639-1 code: "en", "de", "fr" genres: List[str] = field(default_factory=list) source_url: Optional[str] = None + subtitle: Optional[str] = None + search_title: Optional[str] = None # Cleaner title for search queries, when the provider has one + search_author: Optional[str] = None # Cleaner author for search queries + cover_aspect: Optional[str] = None # "portrait" or "square" display_fields: List[DisplayField] = field(default_factory=list) + + # Series info, when the book is part of a series + series_id: Optional[str] = None + series_name: Optional[str] = None + series_position: Optional[float] = None # 3, or 1.5 for a novella + series_count: Optional[int] = None + + # Localized titles for searches in other languages: {"de": "Der Titel"} + titles_by_language: Dict[str, str] = field(default_factory=dict) ``` --- @@ -507,34 +606,39 @@ from shelfmark.release_sources import ( # Get a source instance by name source = get_source("my_source") -releases = source.search(book_metadata) +releases = source.search(book_metadata, plan, expand_search=False, content_type="ebook") # Get a handler instance by name handler = get_handler("my_source") file_path = handler.download(task, cancel_flag, progress_cb, status_cb) -# List all sources where is_available() returns True -available = list_available_sources() -# Returns: [{"name": "direct_download", "display_name": "Direct Download"}, ...] +# List every registered source, with an `enabled` flag taken from is_available() +sources = list_available_sources() +# Returns: [{"name": "direct_download", "display_name": "Direct Download", "enabled": True, +# "supported_content_types": ["ebook", "audiobook"], +# "browse_results_are_releases": False, "can_be_default": True}, ...] ``` ### Import Registration -**Critical**: Your plugin must be imported at module load time. Add to `release_sources/__init__.py`: +**Critical**: Your plugin must be imported for its decorators to run. Built-in sources are listed in `_BUILTIN_SOURCE_MODULES` in `release_sources/__init__.py`, and `_ensure_builtin_sources_registered()` imports each one the first time a source or handler is looked up: ```python -# At the bottom of __init__.py -from shelfmark.release_sources import direct_download # noqa: F401, E402 -from shelfmark.release_sources import my_source # noqa: F401, E402 +_BUILTIN_SOURCE_MODULES = ( + "shelfmark.release_sources.audiobookbay", + "shelfmark.release_sources.direct_download", + # ... + "shelfmark.release_sources.my_source", # add your module here +) ``` -The `noqa` comments suppress linter warnings about unused imports. +Use the dotted path of the module (or package) that imports your source, handler and settings. --- ## Settings Integration -Register plugin settings using the settings registry decorator: +Register plugin settings using the settings registry decorator. The decorator calls your function as soon as the module is imported, so define anything the fields refer to, such as a button callback, above it: ```python from shelfmark.core.settings_registry import ( @@ -551,12 +655,39 @@ from shelfmark.core.settings_registry import ( ) +def test_my_source_connection(current_values: dict): + """Test connection callback - returns result dict. + + `current_values` holds the form's unsaved values, so the button tests what the user just + typed. A password the user has not touched may not be sent, so fall back to the saved one. + """ + try: + api_key = current_values.get("MY_SOURCE_API_KEY") or get_setting_value( + PasswordField(key="MY_SOURCE_API_KEY", label=""), "my_source" + ) + + if not api_key: + return {"success": False, "message": "API key not configured"} + + # Test the connection + response = requests.get( + "https://api.mysource.com/ping", headers={"Authorization": f"Bearer {api_key}"} + ) + + if response.status_code == 200: + return {"success": True, "message": "Connected successfully!"} + else: + return {"success": False, "message": f"API returned {response.status_code}"} + + except Exception as e: + return {"success": False, "message": str(e)} + + @register_settings( name="my_source", # Unique identifier display_name="My Source", # Shown in settings UI icon="globe", # Icon name (optional) order=60, # Sort order in sidebar - group="sources", # Grouping in sidebar ) def my_source_settings(): """Register settings for My Source plugin.""" @@ -600,31 +731,10 @@ def my_source_settings(): style="primary", ), ] - - -def test_my_source_connection(): - """Test connection callback - returns result dict.""" - try: - # Get current API key - api_key = get_setting_value(PasswordField(key="MY_SOURCE_API_KEY", label=""), "my_source") - - if not api_key: - return {"success": False, "message": "API key not configured"} - - # Test the connection - response = requests.get( - "https://api.mysource.com/ping", headers={"Authorization": f"Bearer {api_key}"} - ) - - if response.status_code == 200: - return {"success": True, "message": "Connected successfully!"} - else: - return {"success": False, "message": f"API returned {response.status_code}"} - - except Exception as e: - return {"success": False, "message": str(e)} ``` +A source's settings tab is normally top level, as Jackett and Webshare.cz are. The optional `group=` argument files a tab under a sidebar group registered with `register_group()`. The only groups today are `direct_download` and `metadata_providers`. + ### Field Types Reference | Field Type | Purpose | Key Properties | @@ -688,8 +798,7 @@ from shelfmark.release_sources import ( ) -@classmethod -def get_column_config(cls) -> ReleaseColumnConfig: +def get_column_config(self) -> ReleaseColumnConfig: return ReleaseColumnConfig( columns=[ ColumnSchema( @@ -724,7 +833,7 @@ def get_column_config(cls) -> ReleaseColumnConfig: ColumnSchema( key="seeders", label="Seeders", - render_type=ColumnRenderType.SEEDERS, + render_type=ColumnRenderType.NUMBER, align=ColumnAlign.CENTER, width="60px", hide_mobile=True, @@ -744,13 +853,28 @@ def get_column_config(cls) -> ReleaseColumnConfig: |----------|------|-------------| | `key` | `str` | Data path: `"format"`, `"extra.language"`, `"extra.seeders"` | | `label` | `str` | Accessibility label | -| `render_type` | `ColumnRenderType` | `TEXT`, `BADGE`, `SIZE`, `NUMBER`, `SEEDERS` | +| `render_type` | `ColumnRenderType` | `TEXT`, `BADGE`, `TAGS`, `SIZE`, `NUMBER`, `PEERS`, `INDEXER_PROTOCOL`, `FLAG_ICON`, `FORMAT_CONTENT_TYPE` | | `align` | `ColumnAlign` | `LEFT`, `CENTER`, `RIGHT` | | `width` | `str` | CSS width: `"80px"`, `"minmax(0, 2fr)"` | | `hide_mobile` | `bool` | Hide on small screens | | `color_hint` | `ColumnColorHint` | For `BADGE` type coloring | | `fallback` | `str` | Shown when data missing (default: `"-"`) | | `uppercase` | `bool` | Force uppercase text | +| `sortable` | `bool` | Offer this column in the sort dropdown (opt-in) | +| `sort_key` | `str` | Field to sort by, defaults to `key` | +| `setting_key` | `str` | Name of a boolean setting that shows or hides the column | +| `content_types` | `tuple[str, ...]` | Only show the column for these content types (`"ebook"`, `"audiobook"`) | + +`ReleaseColumnConfig` takes more than `columns`, `grid_template` and `leading_cell`: + +| Property | Description | +|----------|-------------| +| `supported_filters` | Which filters the source supports: `["format", "language", "indexer"]` | +| `extra_sort_options` | `SortOption(label, sort_key, default_direction)` entries for sorts not tied to a column | +| `action_button` | `SourceActionButton(label, action)` that replaces the default "expand search" button | +| `cache_ttl_seconds` | How long results are cached (default 5 minutes) | +| `online_servers` | IRC: nicks of the servers currently online | +| `available_indexers`, `default_indexers` | Indexer sources: every enabled indexer, and the ones selected in settings | ### Color Hints @@ -841,16 +965,20 @@ error → "All download sources failed" ### In Search Methods -Return empty list on errors, log for debugging: +Return an empty list when the search simply found nothing or failed in a way the user cannot act on, and log for debugging. When the source is configured but unreachable, raise `SourceUnavailableError` with a message written for the user, so the UI can say why there are no results instead of showing an empty list: ```python -def search(self, book: BookMetadata) -> List[Release]: +from shelfmark.release_sources import SourceUnavailableError + + +def search(self, book, plan, *, expand_search=False, content_type="ebook") -> List[Release]: try: - results = self.api.search(book.title) + results = self.api.search(plan.primary_query) return [self._to_release(r) for r in results] except ConnectionError as e: logger.warning(f"Connection error searching {self.name}: {e}") - return [] + msg = f"{self.display_name} is not reachable" + raise SourceUnavailableError(msg) from e except AuthenticationError: logger.error(f"{self.name} API key invalid") return [] @@ -928,6 +1056,7 @@ from typing import Callable, Dict, List, Optional from threading import Event from shelfmark.release_sources import ( + HandoffResult, Release, ReleaseSource, DownloadHandler, @@ -943,6 +1072,7 @@ from shelfmark.release_sources import ( ) from shelfmark.metadata_providers import BookMetadata from shelfmark.core.models import DownloadTask +from shelfmark.core.search_plan import ReleaseSearchPlan from shelfmark.core.logger import setup_logger from shelfmark.core.settings_registry import ( register_settings, @@ -963,12 +1093,48 @@ logger = setup_logger(__name__) # --------------------------------------------------------------------------- +def _test_booknet_connection(current_values: Dict) -> Dict: + """Test BookNet API connection using the form's unsaved values, then the saved ones.""" + try: + saved = load_config_file("booknet") + api_key = current_values.get("BOOKNET_API_KEY") or saved.get("BOOKNET_API_KEY") + base_url = ( + current_values.get("BOOKNET_BASE_URL") + or saved.get("BOOKNET_BASE_URL") + or "https://api.booknet.example.com" + ) + + if not api_key: + return {"success": False, "message": "API key not configured"} + + response = requests.get( + f"{base_url}/v1/ping", + headers={"Authorization": f"Bearer {api_key}"}, + timeout=10, + ) + + if response.status_code == 200: + data = response.json() + return { + "success": True, + "message": f"Connected! {data.get('books_available', 0):,} books available", + } + elif response.status_code == 401: + return {"success": False, "message": "Invalid API key"} + else: + return {"success": False, "message": f"API error: {response.status_code}"} + + except requests.Timeout: + return {"success": False, "message": "Connection timeout"} + except Exception as e: + return {"success": False, "message": str(e)} + + @register_settings( name="booknet", display_name="BookNet", icon="book", order=55, - group="sources", ) def booknet_settings(): """Register BookNet settings.""" @@ -1016,39 +1182,6 @@ def booknet_settings(): ] -def _test_booknet_connection() -> Dict: - """Test BookNet API connection.""" - try: - config = load_config_file("booknet") - api_key = config.get("BOOKNET_API_KEY") - base_url = config.get("BOOKNET_BASE_URL", "https://api.booknet.example.com") - - if not api_key: - return {"success": False, "message": "API key not configured"} - - response = requests.get( - f"{base_url}/v1/ping", - headers={"Authorization": f"Bearer {api_key}"}, - timeout=10, - ) - - if response.status_code == 200: - data = response.json() - return { - "success": True, - "message": f"Connected! {data.get('books_available', 0):,} books available", - } - elif response.status_code == 401: - return {"success": False, "message": "Invalid API key"} - else: - return {"success": False, "message": f"API error: {response.status_code}"} - - except requests.Timeout: - return {"success": False, "message": "Connection timeout"} - except Exception as e: - return {"success": False, "message": str(e)} - - # --------------------------------------------------------------------------- # Release Source Implementation # --------------------------------------------------------------------------- @@ -1068,7 +1201,14 @@ class BookNetSource(ReleaseSource): self.timeout = config.get("BOOKNET_TIMEOUT", 30) self.prefer_epub = config.get("BOOKNET_PREFER_EPUB", True) - def search(self, book: BookMetadata) -> List[Release]: + def search( + self, + book: BookMetadata, + plan: ReleaseSearchPlan, + *, + expand_search: bool = False, + content_type: str = "ebook", + ) -> List[Release]: """Search BookNet for releases.""" if not self.is_available(): return [] @@ -1077,8 +1217,8 @@ class BookNetSource(ReleaseSource): try: # Strategy 1: Search by ISBN (most accurate) - if book.isbn_13: - releases = self._search_isbn(book.isbn_13) + if plan.isbn_candidates: + releases = self._search_isbn(plan.isbn_candidates[0]) # Strategy 2: Fallback to title + author if not releases: @@ -1149,8 +1289,7 @@ class BookNetSource(ReleaseSource): """Check if BookNet is configured.""" return bool(self.api_key) - @classmethod - def get_column_config(cls) -> ReleaseColumnConfig: + def get_column_config(self) -> ReleaseColumnConfig: """Custom column layout for BookNet releases.""" return ReleaseColumnConfig( columns=[ @@ -1217,7 +1356,7 @@ class BookNetHandler(DownloadHandler): cancel_flag: Event, progress_callback: Callable[[float], None], status_callback: Callable[[str, Optional[str]], None], - ) -> Optional[str]: + ) -> str | HandoffResult | None: """Execute download from BookNet.""" try: # Phase 1: Get download URL @@ -1364,10 +1503,10 @@ def is_available(self) -> bool: return False -def search(self, book: BookMetadata) -> List[Release]: - """Return empty list on errors, don't raise.""" +def search(self, book, plan, *, expand_search=False, content_type="ebook") -> List[Release]: + """Return empty list on errors, don't raise (SourceUnavailableError aside).""" try: - return self._do_search(book) + return self._do_search(book, plan) except Exception as e: logger.warning(f"Search failed: {e}") return [] @@ -1459,14 +1598,11 @@ logger.error("Failures") # Unrecoverable errors When creating a new plugin: -1. Create `shelfmark/release_sources/my_plugin.py` +1. Create `shelfmark/release_sources/my_plugin.py` (or a `my_plugin/` package, as the built-in sources are) 2. Implement `ReleaseSource` subclass with `@register_source("my_plugin")` 3. Implement `DownloadHandler` subclass with `@register_handler("my_plugin")` 4. Add settings with `@register_settings("my_plugin", ...)` -5. Add import to `shelfmark/release_sources/__init__.py`: - ```python - from shelfmark.release_sources import my_plugin # noqa: F401, E402 - ``` +5. Add `"shelfmark.release_sources.my_plugin"` to `_BUILTIN_SOURCE_MODULES` in `shelfmark/release_sources/__init__.py` 6. Test `is_available()` returns `True` when configured 7. Test search returns valid `Release` objects 8. Test download handles cancellation and errors gracefully diff --git a/docs/index.md b/docs/index.md index 95839cce..2e11e0c5 100644 --- a/docs/index.md +++ b/docs/index.md @@ -22,6 +22,10 @@ Use the guides below to set up the app, connect your library tools, and understa - [Library Check](library-check.md) - [Custom Scripts](custom-scripts.md) +## Developers + +- [Developer Documentation](dev/index.md) + ## Help - [Troubleshooting](troubleshooting.md) diff --git a/shelfmark/metadata_providers/README.md b/shelfmark/metadata_providers/README.md index 61607177..3ac6c7bf 100644 --- a/shelfmark/metadata_providers/README.md +++ b/shelfmark/metadata_providers/README.md @@ -14,6 +14,8 @@ Metadata providers allow searching for books and retrieving detailed metadata (t | **Open Library** | No | Free, open-source library catalog from the Internet Archive. Rate limited to ~100 requests/minute | | **Google Books** | Yes (API key) | Google's book database with broad coverage and a free API key option | | **DNB** | No | Deutsche Nationalbibliothek, the German national library catalog. Best coverage for German-language editions | +| **Databáze knih** | No | Czech and Slovak book catalog. It has no public API, so Shelfmark reads the web pages | +| **Moly.hu** | No | Hungarian book catalog, read from the web pages. No API key required | ## Core Components @@ -314,7 +316,7 @@ def make_request(self): ## Configuration -Provider settings are stored in `CONFIG_DIR/plugins/.json` and managed via the Settings UI. See [Plugin Settings Guide](../../docs/plugin-settings.md) for detailed documentation on adding settings to your provider. +Provider settings are stored in `CONFIG_DIR/plugins/.json` and managed via the Settings UI. See [Plugin Settings Guide](../../docs/dev/plugin-settings.md) for detailed documentation on adding settings to your provider. ## Environment Variables