mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-09-24 11:40:46 +01:00
feat(packs): inspect multi-book releases and file each book separately (#1270)
## Multi-book packs: inspect a release before download and file each book separately Closes #576 ### Problem One queued release is always treated as one book. When a torrent is actually a whole series (`Series/Book 1 - Title/…`, or a flat folder of `Series 1.0 - Title.m4b` files), post-processing walks the whole tree, flattens every file into one list and renames them `Title - 01…10` under the searched book's `{Author}/{Title}`. Audiobookshelf then sees a single 10-file "book" and the user has to re-file everything by hand. ### What this does Most releases expose their file list *before* anything is downloaded, so the split is decided up front and approved by the user, then the download is fire-and-forget: 1. **Inspect** – clicking a release's download button now calls `POST /api/releases/inspect` first. A new optional `DownloadHandler.list_files(release_data)` hook returns the release's files without downloading: - **AudiobookBay** reads the torrent file table off the detail page it already fetches (the page is now cached for 120 s, so inspect + download cost ABB one request). - **Prowlarr** parses `info.files` from the `.torrent` it already fetches (the existing 120 s torrent-fetch cache is reused). Magnet-only and usenet releases report "can't inspect". - Other sources default to `None`. 2. **Review** – if the plan contains more than one book, the Find Releases modal swaps the list for a review panel: one row per book with editable title / series position / year, expandable file lists, non-book sidecars (`.txt`, covers) shown as ignored, a "Treat as a single book" switch, and **Download N books**. Single-book releases queue immediately, exactly as before. 3. **File** – the approved plan travels with the task (`DownloadTask.book_plan`, retry-safe) and post-processing files each book through the existing transfer code, one book at a time (`dataclasses.replace(task, title=…, series_position=…, year=…)`), so organize/rename templates, part numbering (now scoped per book), hardlinks, torrent copy-preserve and usenet handling are unchanged. Status reads `Complete (N books, M files)`. 4. **Fallback** – when a release can't be inspected the user gets a toast, and a small "Multi-book pack" toggle in the modal header forces a heuristic split (subfolder = book, or one book per file when the file names carry series positions). Planning lives in `shelfmark/download/postprocess/packs.py` and is shared by the inspect endpoint and post-processing, so what the user approved is what gets filed. The name parser strips `Book 3 -`, `03 -`, `1.0 -`, `3.`, `[03]`, `#3`, a leading series name, labels like "An Expanse Novella -", repeated titles (`Gods of Risk 2.5 - Gods of Risk`) and a trailing `(Year)`; author and series name come from the book that was searched, and the searched book's own series position is never applied to its siblings. ### Files - `shelfmark/download/postprocess/packs.py` (new) – `PackFile/PackBook/PackPlan`, `plan_pack`, `parse_pack_book_name`, `group_files_into_books`, `match_plan_to_files` - `shelfmark/core/release_inspect_routes.py` (new) – `POST /api/releases/inspect` - `shelfmark/release_sources/__init__.py` – `DownloadHandler.list_files` hook - `shelfmark/release_sources/audiobookbay/{scraper,handler}.py` – detail-page cache, `extract_file_list`, `list_files` - `shelfmark/release_sources/prowlarr/handler.py`, `download/clients/torrent_utils.py` – `extract_file_list_from_torrent`, `list_files` - `shelfmark/core/models.py`, `download/orchestrator.py` – `multi_book` / `book_plan` fields, queue + retry serialization - `shelfmark/download/postprocess/transfer.py`, `pipeline.py`, `outputs/folder.py` – per-book transfer branch and status message - `src/frontend`: `components/PackReviewPanel.tsx` (new), `ReleaseModal.tsx`, `App.tsx`, `services/api.ts`, `types/index.ts`, `utils/releasePayload.ts` (payload builder moved out of `App.tsx`), `utils/packReview.ts` - `docs/dev/release-sources-plugin-guide.md` – documents the `list_files` hook ### Out of scope (follow-ups) - Listing files from an NZB (Shelfmark already fetches the bytes; `<file subject>` names are noisy) - Inspecting magnet links via qBittorrent's files API after a paused add - BookLore / email outputs (they ignore `book_plan`; noted in code) - The combined ebook + audiobook flow ### Testing **Automated** (`make checks`, `make python-test`, `make frontend-test` all green; the only failures on my machine are the pre-existing `tests/config/test_entrypoint_permissions.py` cases, which need bash ≥ 4 and fail identically on `main` under macOS bash 3.2): - `tests/download/test_packs.py` – name parsing (markers, series name, novella labels, repeated titles, bare numeric titles like `1984`), nested / flat / mixed / deeper-nested packs, single wrapping folder not treated as a pack, plan-to-disk matching with basename fallback - `tests/core/test_processing_packs.py` – full `post_process_download` runs on a real temp filesystem: approved plan files each book under its own `{Author}/{Title}`, heuristic split of a nested pack, searched book's series position does not leak, multi-file book inside a pack keeps `- 01/- 02` per book, hardlinked torrent pack leaves the seeding tree intact, no pack fields ⇒ behaviour unchanged, single group degrades to the searched title, status message - `tests/core/test_release_inspect_routes.py` – plan response, not-inspectable, handler errors never 500, unknown source / missing `source_id` ⇒ 400, login required - `tests/audiobookbay/test_file_list.py` – file-table scraping from real ABB markup (multi-file and single-file pages), handler host validation, one page fetch shared by magnet + file list - `tests/prowlarr/test_torrent_file_list.py` – multi-file / single-file `.torrent` parsing, handler behaviour for torrent URL vs magnet vs usenet vs cache miss - `tests/download/test_orchestrator_pack_fields.py` – queue-time parsing and retry round-trip - Frontend: `releasePayload.test.ts`, `packReview.test.ts` (vitest) **Manual, on a real deployment** (arm64 image built from this branch, run as a side container next to production with the same qBittorrent / Audiobookshelf setup, `FILE_ORGANIZATION_AUDIOBOOK=organize`, hardlinks on): - AudiobookBay "The Expanse Complete 2.0" (7.87 GB, 36 files): clicking download opened the review panel in ~1 s showing **18 books · 18 files · 18 files ignored** (the `.txt` sidecars), with series positions 0.1–9.5 and years parsed from the file names; novella labels stripped ("The Churn", "The Butcher of Anderson Station"). Editing a title in the panel works. Confirming queued one task; the magnet resolved from the cached page in ~30 ms; after the download the task reported `Complete (18 books, 18 files)`, 18 hardlinks landed as `audiobooks/James S. A. Corey/<Title>/<Title>.m4b`, the torrent kept seeding, and Audiobookshelf scanned each folder as its own book (title, author, embedded chapters). - A second pack ("Expanse [01 - 9.5]", `Title N - Title` naming) was inspected to verify the repeated-title rule and the Back button, without downloading. - Single-book releases still queue immediately with no extra UI.
This commit is contained in:
@@ -236,6 +236,7 @@ pyrightconfig.json
|
|||||||
*.local.*
|
*.local.*
|
||||||
AGENTS.md
|
AGENTS.md
|
||||||
.claude/
|
.claude/
|
||||||
|
CLAUDE.md
|
||||||
.nvmrc
|
.nvmrc
|
||||||
.playwright-mcp/
|
.playwright-mcp/
|
||||||
frontend-dist/
|
frontend-dist/
|
||||||
|
|||||||
@@ -276,6 +276,27 @@ class DownloadHandler(ABC):
|
|||||||
pass
|
pass
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Optional: Listing Files Before Download
|
||||||
|
|
||||||
|
Some releases bundle several books (a whole-series torrent). Shelfmark inspects a
|
||||||
|
release before queueing it so the user can review how it will be split into books.
|
||||||
|
Override `list_files` when your source can enumerate a release's files without
|
||||||
|
downloading it; the default returns `None`, which the UI reports as "can't inspect":
|
||||||
|
|
||||||
|
```python
|
||||||
|
from shelfmark.download.postprocess.packs import PackFile
|
||||||
|
|
||||||
|
def list_files(self, release_data: dict[str, Any]) -> list[PackFile] | None:
|
||||||
|
"""Return the release's files (release-relative paths + sizes), or None."""
|
||||||
|
torrent_bytes = ... # e.g. fetch the .torrent, or scrape the indexer's detail page
|
||||||
|
return extract_file_list_from_torrent(torrent_bytes) # from download.clients.torrent_utils
|
||||||
|
```
|
||||||
|
|
||||||
|
`release_data` is the same payload the frontend sends to `/api/releases/download`
|
||||||
|
(`source_id`, `download_url`, `content_type`, `series_name`, ...). Built-in examples:
|
||||||
|
Prowlarr parses the `.torrent` it already fetches (magnet-only releases return
|
||||||
|
`None`), and AudiobookBay reads the file table off its detail page.
|
||||||
|
|
||||||
### Download Method Parameters
|
### Download Method Parameters
|
||||||
|
|
||||||
| Parameter | Type | Description |
|
| Parameter | Type | Description |
|
||||||
|
|||||||
@@ -136,6 +136,12 @@ class DownloadTask:
|
|||||||
default_factory=dict
|
default_factory=dict
|
||||||
) # Per-output parameters (e.g. email recipient)
|
) # Per-output parameters (e.g. email recipient)
|
||||||
|
|
||||||
|
# Multi-book packs: one release holding several books. `book_plan` is the split the
|
||||||
|
# user approved before download (list of {title, series_position, year, files});
|
||||||
|
# `multi_book` asks post-processing to split heuristically when no plan exists.
|
||||||
|
multi_book: bool = False
|
||||||
|
book_plan: list[dict[str, Any]] | None = None
|
||||||
|
|
||||||
# User association (multi-user support)
|
# User association (multi-user support)
|
||||||
user_id: int | None = None # DB user ID who queued this download
|
user_id: int | None = None # DB user ID who queued this download
|
||||||
username: str | None = None # Username for {User} template variable
|
username: str | None = None # Username for {User} template variable
|
||||||
|
|||||||
@@ -0,0 +1,102 @@
|
|||||||
|
"""Pre-download release inspection: list a release's files and plan a multi-book split."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
from flask import jsonify, request
|
||||||
|
|
||||||
|
from shelfmark.core.logger import setup_logger
|
||||||
|
from shelfmark.core.utils import is_audiobook
|
||||||
|
from shelfmark.download.postprocess.packs import PackFile, PackPlan, plan_pack
|
||||||
|
from shelfmark.download.postprocess.policy import (
|
||||||
|
get_supported_audiobook_formats,
|
||||||
|
get_supported_formats,
|
||||||
|
)
|
||||||
|
from shelfmark.release_sources import get_handler
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Callable
|
||||||
|
|
||||||
|
from flask import Flask, Response
|
||||||
|
|
||||||
|
logger = setup_logger(__name__)
|
||||||
|
|
||||||
|
_INSPECT_ERRORS = (OSError, RuntimeError, ValueError, TypeError, KeyError, AttributeError)
|
||||||
|
NOT_INSPECTABLE_REASON = "This source cannot list the release's files before downloading"
|
||||||
|
|
||||||
|
|
||||||
|
def _serialize_plan(plan: PackPlan) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"is_pack": plan.is_pack,
|
||||||
|
"ignored": plan.ignored,
|
||||||
|
"books": [
|
||||||
|
{
|
||||||
|
"title": book.title,
|
||||||
|
"series_position": book.series_position,
|
||||||
|
"year": book.year,
|
||||||
|
"files": book.files,
|
||||||
|
}
|
||||||
|
for book in plan.books
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def inspect_release(data: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
"""Build the inspect response for a release payload (same shape as a download)."""
|
||||||
|
source = str(data["source"])
|
||||||
|
handler = get_handler(source)
|
||||||
|
try:
|
||||||
|
files: list[PackFile] | None = handler.list_files(data)
|
||||||
|
except _INSPECT_ERRORS as exc:
|
||||||
|
logger.warning(
|
||||||
|
"Could not list files for %s release %s: %s", source, data.get("source_id"), exc
|
||||||
|
)
|
||||||
|
return {"inspected": False, "reason": str(exc), "files": [], "plan": None}
|
||||||
|
|
||||||
|
if files is None:
|
||||||
|
return {"inspected": False, "reason": NOT_INSPECTABLE_REASON, "files": [], "plan": None}
|
||||||
|
|
||||||
|
content_type = data.get("content_type")
|
||||||
|
supported = (
|
||||||
|
get_supported_audiobook_formats()
|
||||||
|
if is_audiobook(content_type if isinstance(content_type, str) else None)
|
||||||
|
else get_supported_formats()
|
||||||
|
)
|
||||||
|
series_name = data.get("series_name")
|
||||||
|
author_name = data.get("author")
|
||||||
|
plan = plan_pack(
|
||||||
|
files,
|
||||||
|
supported_extensions=set(supported),
|
||||||
|
series_name=series_name if isinstance(series_name, str) else None,
|
||||||
|
author_name=author_name if isinstance(author_name, str) else None,
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"inspected": True,
|
||||||
|
"reason": None,
|
||||||
|
"files": [{"path": f.path, "size": f.size} for f in files],
|
||||||
|
"plan": _serialize_plan(plan),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def register_release_inspect_routes(
|
||||||
|
app: Flask,
|
||||||
|
login_required: Callable[..., Any],
|
||||||
|
) -> None:
|
||||||
|
"""Register POST /api/releases/inspect."""
|
||||||
|
|
||||||
|
@app.route("/api/releases/inspect", methods=["POST"])
|
||||||
|
@login_required
|
||||||
|
def api_inspect_release() -> Response | tuple[Response, int]:
|
||||||
|
data = request.get_json(silent=True)
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
return jsonify({"error": "No data provided"}), 400
|
||||||
|
if not data.get("source_id"):
|
||||||
|
return jsonify({"error": "source_id is required"}), 400
|
||||||
|
if not data.get("source"):
|
||||||
|
return jsonify({"error": "source is required"}), 400
|
||||||
|
try:
|
||||||
|
get_handler(str(data["source"]))
|
||||||
|
except ValueError as exc:
|
||||||
|
return jsonify({"error": str(exc)}), 400
|
||||||
|
return jsonify(inspect_release(data))
|
||||||
@@ -17,6 +17,7 @@ from shelfmark.core.config import config
|
|||||||
from shelfmark.core.logger import setup_logger
|
from shelfmark.core.logger import setup_logger
|
||||||
from shelfmark.core.utils import normalize_http_url
|
from shelfmark.core.utils import normalize_http_url
|
||||||
from shelfmark.download.network import get_ssl_verify
|
from shelfmark.download.network import get_ssl_verify
|
||||||
|
from shelfmark.download.postprocess.packs import PackFile
|
||||||
|
|
||||||
logger = setup_logger(__name__)
|
logger = setup_logger(__name__)
|
||||||
|
|
||||||
@@ -433,6 +434,56 @@ def extract_info_hash_from_torrent(torrent_data: bytes) -> str | None:
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _decode_torrent_text(value: object) -> str | None:
|
||||||
|
if isinstance(value, bytes):
|
||||||
|
return value.decode("utf-8", errors="replace")
|
||||||
|
if isinstance(value, str):
|
||||||
|
return value
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def extract_file_list_from_torrent(torrent_data: bytes) -> list[PackFile] | None:
|
||||||
|
"""List the files a .torrent describes, release-relative, without downloading it.
|
||||||
|
|
||||||
|
Multi-file torrents nest every path under the torrent name (which becomes the
|
||||||
|
client's save folder); single-file torrents are just the named file.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
decoded, _ = bencode_decode(torrent_data)
|
||||||
|
except _TORRENT_PARSE_ERRORS as e:
|
||||||
|
logger.debug("Failed to parse torrent file list: %s", e)
|
||||||
|
return None
|
||||||
|
if not isinstance(decoded, dict):
|
||||||
|
return None
|
||||||
|
info = decoded.get(b"info")
|
||||||
|
if not isinstance(info, dict):
|
||||||
|
return None
|
||||||
|
|
||||||
|
name = _decode_torrent_text(info.get(b"name")) or ""
|
||||||
|
raw_files = info.get(b"files")
|
||||||
|
if not isinstance(raw_files, list):
|
||||||
|
length = info.get(b"length")
|
||||||
|
if not name:
|
||||||
|
return None
|
||||||
|
return [PackFile(name, length if isinstance(length, int) else None)]
|
||||||
|
|
||||||
|
files: list[PackFile] = []
|
||||||
|
for entry in raw_files:
|
||||||
|
if not isinstance(entry, dict):
|
||||||
|
continue
|
||||||
|
raw_path = entry.get(b"path")
|
||||||
|
if not isinstance(raw_path, list):
|
||||||
|
continue
|
||||||
|
segments = [seg for seg in (_decode_torrent_text(part) for part in raw_path) if seg]
|
||||||
|
if not segments:
|
||||||
|
continue
|
||||||
|
if name:
|
||||||
|
segments.insert(0, name)
|
||||||
|
length = entry.get(b"length")
|
||||||
|
files.append(PackFile("/".join(segments), length if isinstance(length, int) else None))
|
||||||
|
return files
|
||||||
|
|
||||||
|
|
||||||
def extract_hash_from_magnet(magnet_url: str) -> str | None:
|
def extract_hash_from_magnet(magnet_url: str) -> str | None:
|
||||||
"""Extract info_hash from a magnet URL."""
|
"""Extract info_hash from a magnet URL."""
|
||||||
if not magnet_url.startswith("magnet:"):
|
if not magnet_url.startswith("magnet:"):
|
||||||
|
|||||||
@@ -265,6 +265,8 @@ def queue_release(
|
|||||||
series_position = release_data.get("series_position") or extra.get("series_position")
|
series_position = release_data.get("series_position") or extra.get("series_position")
|
||||||
subtitle = release_data.get("subtitle") or extra.get("subtitle")
|
subtitle = release_data.get("subtitle") or extra.get("subtitle")
|
||||||
language = release_data.get("language") or extra.get("language")
|
language = release_data.get("language") or extra.get("language")
|
||||||
|
multi_book = bool(release_data.get("multi_book") or extra.get("multi_book"))
|
||||||
|
book_plan = _normalize_book_plan(release_data.get("book_plan") or extra.get("book_plan"))
|
||||||
|
|
||||||
books_output_mode = (
|
books_output_mode = (
|
||||||
str(config.get("BOOKS_OUTPUT_MODE", "folder", user_id=user_id) or "folder")
|
str(config.get("BOOKS_OUTPUT_MODE", "folder", user_id=user_id) or "folder")
|
||||||
@@ -300,6 +302,8 @@ def queue_release(
|
|||||||
series_position=series_position,
|
series_position=series_position,
|
||||||
subtitle=subtitle,
|
subtitle=subtitle,
|
||||||
language=language,
|
language=language,
|
||||||
|
multi_book=multi_book or book_plan is not None,
|
||||||
|
book_plan=book_plan,
|
||||||
search_mode=search_mode,
|
search_mode=search_mode,
|
||||||
output_mode=output_mode,
|
output_mode=output_mode,
|
||||||
output_args=output_args,
|
output_args=output_args,
|
||||||
@@ -408,6 +412,33 @@ def can_retry_download_task(
|
|||||||
return _has_staged_retry_source(task)
|
return _has_staged_retry_source(task)
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_book_plan(value: object) -> list[dict[str, Any]] | None:
|
||||||
|
"""Keep only well-formed pack books: a title plus a non-empty list of file paths."""
|
||||||
|
if not isinstance(value, list):
|
||||||
|
return None
|
||||||
|
books: list[dict[str, Any]] = []
|
||||||
|
for entry in value:
|
||||||
|
if not isinstance(entry, dict):
|
||||||
|
continue
|
||||||
|
title = normalize_optional_text(entry.get("title"))
|
||||||
|
raw_files = entry.get("files")
|
||||||
|
if title is None or not isinstance(raw_files, list):
|
||||||
|
continue
|
||||||
|
files = [f for f in raw_files if isinstance(f, str) and f.strip()]
|
||||||
|
if not files:
|
||||||
|
continue
|
||||||
|
year = entry.get("year")
|
||||||
|
books.append(
|
||||||
|
{
|
||||||
|
"title": title,
|
||||||
|
"series_position": _optional_number(entry.get("series_position")),
|
||||||
|
"year": year if isinstance(year, int) and not isinstance(year, bool) else None,
|
||||||
|
"files": files,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return books or None
|
||||||
|
|
||||||
|
|
||||||
def serialize_task_for_retry(task: DownloadTask) -> dict[str, Any]:
|
def serialize_task_for_retry(task: DownloadTask) -> dict[str, Any]:
|
||||||
"""Serialize the task state needed for restart-safe retries."""
|
"""Serialize the task state needed for restart-safe retries."""
|
||||||
raw_search_mode = getattr(task, "search_mode", None)
|
raw_search_mode = getattr(task, "search_mode", None)
|
||||||
@@ -437,6 +468,8 @@ def serialize_task_for_retry(task: DownloadTask) -> dict[str, Any]:
|
|||||||
"subtitle": getattr(task, "subtitle", None),
|
"subtitle": getattr(task, "subtitle", None),
|
||||||
"language": getattr(task, "language", None),
|
"language": getattr(task, "language", None),
|
||||||
"search_mode": search_mode,
|
"search_mode": search_mode,
|
||||||
|
"multi_book": bool(getattr(task, "multi_book", False)),
|
||||||
|
"book_plan": _normalize_book_plan(getattr(task, "book_plan", None)),
|
||||||
"output_mode": getattr(task, "output_mode", None),
|
"output_mode": getattr(task, "output_mode", None),
|
||||||
"output_args": dict(raw_output_args) if isinstance(raw_output_args, dict) else {},
|
"output_args": dict(raw_output_args) if isinstance(raw_output_args, dict) else {},
|
||||||
"user_id": getattr(task, "user_id", None),
|
"user_id": getattr(task, "user_id", None),
|
||||||
@@ -495,6 +528,8 @@ def _restore_task_from_retry_payload(payload: object) -> DownloadTask | None:
|
|||||||
subtitle=normalize_optional_text(payload.get("subtitle")),
|
subtitle=normalize_optional_text(payload.get("subtitle")),
|
||||||
language=normalize_optional_text(payload.get("language")),
|
language=normalize_optional_text(payload.get("language")),
|
||||||
search_mode=search_mode,
|
search_mode=search_mode,
|
||||||
|
multi_book=bool(payload.get("multi_book", False)),
|
||||||
|
book_plan=_normalize_book_plan(payload.get("book_plan")),
|
||||||
output_mode=normalize_optional_text(payload.get("output_mode")),
|
output_mode=normalize_optional_text(payload.get("output_mode")),
|
||||||
output_args=dict(output_args) if isinstance(output_args, dict) else {},
|
output_args=dict(output_args) if isinstance(output_args, dict) else {},
|
||||||
user_id=normalize_positive_int(payload.get("user_id")),
|
user_id=normalize_positive_int(payload.get("user_id")),
|
||||||
|
|||||||
@@ -105,6 +105,7 @@ def process_folder_output(
|
|||||||
maybe_run_custom_script,
|
maybe_run_custom_script,
|
||||||
prepare_output_files,
|
prepare_output_files,
|
||||||
record_step,
|
record_step,
|
||||||
|
resolve_book_groups,
|
||||||
transfer_book_files,
|
transfer_book_files,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -260,7 +261,15 @@ def process_folder_output(
|
|||||||
prepared.cleanup_paths,
|
prepared.cleanup_paths,
|
||||||
)
|
)
|
||||||
|
|
||||||
message = "Complete" if len(final_paths) == 1 else f"Complete ({len(final_paths)} files)"
|
pack_groups = resolve_book_groups(
|
||||||
|
task, prepared.files, organization_mode=plan.organization_mode
|
||||||
|
)
|
||||||
|
if pack_groups is not None:
|
||||||
|
message = f"Complete ({len(pack_groups)} books, {len(final_paths)} files)"
|
||||||
|
elif len(final_paths) == 1:
|
||||||
|
message = "Complete"
|
||||||
|
else:
|
||||||
|
message = f"Complete ({len(final_paths)} files)"
|
||||||
status_callback("complete", message)
|
status_callback("complete", message)
|
||||||
|
|
||||||
return str(final_paths[0])
|
return str(final_paths[0])
|
||||||
|
|||||||
@@ -0,0 +1,406 @@
|
|||||||
|
"""Multi-book ("pack") release planning.
|
||||||
|
|
||||||
|
A pack is one release that contains several books: a whole-series torrent with one
|
||||||
|
subfolder per book, or a flat folder of `Series 1.0 - Title.m4b` files. The same
|
||||||
|
planning rules serve pre-download inspection (the file list comes from the release
|
||||||
|
source) and post-processing (the file list comes from disk), so what the user
|
||||||
|
approved in the modal is what gets filed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path, PurePosixPath
|
||||||
|
|
||||||
|
_YEAR_SUFFIX_RE = re.compile(r"\s*\(\s*(?P<year>\d{4})\s*\)\s*$")
|
||||||
|
_SERIES_MARKER_RE = re.compile(
|
||||||
|
r"""
|
||||||
|
^\s*
|
||||||
|
(?:
|
||||||
|
\[\s*\#?(?P<bracket>\d+(?:\.\d+)?)\s*\] # [03] / [#3]
|
||||||
|
| \#(?P<hash>\d+(?:\.\d+)?) # #3
|
||||||
|
| book\.?\s*(?P<book>\d+(?:\.\d+)?) # Book 3 / Book. 03
|
||||||
|
| (?P<plain>\d+(?:\.\d+)?)(?=[\s\-:.]) # 03 - / 1.0 - / 3.
|
||||||
|
)
|
||||||
|
\s*(?:[-:.]\s*)?
|
||||||
|
""",
|
||||||
|
re.IGNORECASE | re.VERBOSE,
|
||||||
|
)
|
||||||
|
_SEPARATOR_CHARS = " \t-_:."
|
||||||
|
# "Gods of Risk 2.5 - Gods of Risk": the title repeated on both sides of the position.
|
||||||
|
_REPEATED_TITLE_RE = re.compile(
|
||||||
|
r"^(?P<left>.+?)\s+(?P<position>\d+(?:\.\d+)?)\s*[-:\u2013]\s*(?P<right>.+)$"
|
||||||
|
)
|
||||||
|
_SERIES_LABEL_WORDS = r"(?:novella|novellas|short\s+story|short|story|novel)"
|
||||||
|
# "Uncrowned Cradle, Book 7" / "Reaper Cradle, Volume 10" / "Wintersteel (Cradle, Book 8)":
|
||||||
|
# an explicit word marks the position at the END of the name. A bare trailing number
|
||||||
|
# is deliberately not matched — "Title - 02" is a chapter, not a series position.
|
||||||
|
_TRAILING_MARKER_RE = re.compile(
|
||||||
|
r"""
|
||||||
|
[\s,\-:\u2013(]*
|
||||||
|
(?:book|volume|vol\.?)\s*\#?(?P<position>\d+(?:\.\d+)?)
|
||||||
|
\s*\)?\s*$
|
||||||
|
""",
|
||||||
|
re.IGNORECASE | re.VERBOSE,
|
||||||
|
)
|
||||||
|
# AudiobookBay renders a file inside a folder as "<folder> <file>" with no separator,
|
||||||
|
# so a pack row reads "Author - Title Series, Book 1 Title Series, Book 1".
|
||||||
|
_GLUED_FOLDER_RE = re.compile(
|
||||||
|
r"^(?P<prefix>.+?\s[-\u2013]\s)?(?P<core>.+?)\s+(?P=core)$", re.IGNORECASE
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PackFile:
|
||||||
|
"""One file inside a release, path relative to the release root."""
|
||||||
|
|
||||||
|
path: str
|
||||||
|
size: int | None = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PackBook:
|
||||||
|
"""One book split out of a pack, files as release-relative paths."""
|
||||||
|
|
||||||
|
title: str
|
||||||
|
series_position: float | None
|
||||||
|
year: int | None
|
||||||
|
files: list[str]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PackPlan:
|
||||||
|
books: list[PackBook]
|
||||||
|
ignored: list[str]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_pack(self) -> bool:
|
||||||
|
return len(self.books) > 1
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class BookGroup:
|
||||||
|
"""One book's on-disk files, ready for transfer."""
|
||||||
|
|
||||||
|
title: str
|
||||||
|
series_position: float | None
|
||||||
|
year: int | None
|
||||||
|
files: list[Path]
|
||||||
|
|
||||||
|
|
||||||
|
def _strip_series_name(name: str, series_name: str | None) -> str:
|
||||||
|
if not series_name:
|
||||||
|
return name
|
||||||
|
prefix = series_name.strip()
|
||||||
|
if not prefix or not name.lower().startswith(prefix.lower()):
|
||||||
|
return name
|
||||||
|
remainder = name[len(prefix) :]
|
||||||
|
if remainder and remainder[0].isalnum():
|
||||||
|
return name
|
||||||
|
return remainder.lstrip(_SEPARATOR_CHARS)
|
||||||
|
|
||||||
|
|
||||||
|
def _strip_series_label(work: str, series_name: str | None) -> str:
|
||||||
|
"""Drop a leading "An <Series> Novella - " style label that some packs prepend."""
|
||||||
|
if not series_name:
|
||||||
|
return work
|
||||||
|
# "The Expanse" is labelled "An Expanse Novella", so match without the article.
|
||||||
|
core = re.sub(r"^(?:the|an?)\s+", "", series_name.strip(), flags=re.IGNORECASE)
|
||||||
|
if not core:
|
||||||
|
return work
|
||||||
|
pattern = re.compile(
|
||||||
|
rf"^(?:an?\s+|the\s+)?{re.escape(core)}\s+{_SERIES_LABEL_WORDS}\s*[-:\u2013]\s*",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
return pattern.sub("", work, count=1)
|
||||||
|
|
||||||
|
|
||||||
|
def _collapse_glued_folder(name: str) -> str:
|
||||||
|
match = _GLUED_FOLDER_RE.match(name)
|
||||||
|
if not match:
|
||||||
|
return name
|
||||||
|
prefix = match.group("prefix") or ""
|
||||||
|
core = match.group("core")
|
||||||
|
# "Author - X X" → "Author - X" (the folder carried the author, the file did not).
|
||||||
|
return (prefix + core).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def _strip_author_name(name: str, author_name: str | None) -> str:
|
||||||
|
"""Drop a leading "Author - " (packs are often filed as `Author - Title`)."""
|
||||||
|
if not author_name:
|
||||||
|
return name
|
||||||
|
prefix = author_name.strip()
|
||||||
|
if not prefix or not name.lower().startswith(prefix.lower()):
|
||||||
|
return name
|
||||||
|
remainder = name[len(prefix) :]
|
||||||
|
stripped = remainder.lstrip(_SEPARATOR_CHARS + "\u2013")
|
||||||
|
if stripped == remainder: # no separator after the author: part of the title
|
||||||
|
return name
|
||||||
|
return stripped
|
||||||
|
|
||||||
|
|
||||||
|
def _strip_trailing_series_name(work: str, series_name: str | None) -> str:
|
||||||
|
"""Drop a trailing series name left behind by a trailing position marker."""
|
||||||
|
if not series_name:
|
||||||
|
return work
|
||||||
|
suffix = series_name.strip()
|
||||||
|
if not suffix or not work.lower().endswith(suffix.lower()):
|
||||||
|
return work
|
||||||
|
remainder = work[: -len(suffix)]
|
||||||
|
stripped = remainder.rstrip(_SEPARATOR_CHARS + ",(\u2013")
|
||||||
|
if not stripped or stripped == remainder:
|
||||||
|
return work
|
||||||
|
return stripped
|
||||||
|
|
||||||
|
|
||||||
|
def parse_pack_book_name(
|
||||||
|
name: str, *, series_name: str | None, author_name: str | None = None
|
||||||
|
) -> tuple[str, float | None, int | None]:
|
||||||
|
"""Split a book folder/file-stem name into (title, series position, year).
|
||||||
|
|
||||||
|
Strips a leading series name, a leading position marker (`Book 3 - `, `03 - `,
|
||||||
|
`1.0 - `, `3. `, `[03] `, `#3 `) and a trailing `(YYYY)`. Also understands a
|
||||||
|
trailing marker (`Title Series, Book 3`, `Title (Series, Volume 3)`), a leading
|
||||||
|
`Author - `, and AudiobookBay's glued `<folder> <file>` names. Returns the name
|
||||||
|
unchanged with no position/year when nothing would be left of the title.
|
||||||
|
"""
|
||||||
|
work = _collapse_glued_folder(name.strip())
|
||||||
|
work = _strip_author_name(work, author_name)
|
||||||
|
work = _strip_series_name(work, series_name)
|
||||||
|
|
||||||
|
year: int | None = None
|
||||||
|
year_match = _YEAR_SUFFIX_RE.search(work)
|
||||||
|
if year_match:
|
||||||
|
year = int(year_match.group("year"))
|
||||||
|
work = work[: year_match.start()]
|
||||||
|
|
||||||
|
position: float | None = None
|
||||||
|
repeated = _REPEATED_TITLE_RE.match(work.strip())
|
||||||
|
if (
|
||||||
|
repeated
|
||||||
|
and repeated.group("left").strip().lower() == repeated.group("right").strip().lower()
|
||||||
|
):
|
||||||
|
return repeated.group("right").strip(), float(repeated.group("position")), year
|
||||||
|
|
||||||
|
marker = _SERIES_MARKER_RE.match(work)
|
||||||
|
if marker:
|
||||||
|
raw = (
|
||||||
|
marker.group("bracket")
|
||||||
|
or marker.group("hash")
|
||||||
|
or marker.group("book")
|
||||||
|
or marker.group("plain")
|
||||||
|
)
|
||||||
|
position = float(raw)
|
||||||
|
work = work[marker.end() :]
|
||||||
|
else:
|
||||||
|
trailing = _TRAILING_MARKER_RE.search(work)
|
||||||
|
if trailing and trailing.start() > 0:
|
||||||
|
position = float(trailing.group("position"))
|
||||||
|
work = _strip_trailing_series_name(work[: trailing.start()], series_name)
|
||||||
|
|
||||||
|
work = _strip_series_label(work, series_name)
|
||||||
|
title = work.strip().strip(_SEPARATOR_CHARS).strip()
|
||||||
|
if not title:
|
||||||
|
return name, None, None
|
||||||
|
return title, position, year
|
||||||
|
|
||||||
|
|
||||||
|
def _book_from_name(
|
||||||
|
name: str, files: list[str], series_name: str | None, author_name: str | None = None
|
||||||
|
) -> PackBook:
|
||||||
|
title, position, year = parse_pack_book_name(
|
||||||
|
name, series_name=series_name, author_name=author_name
|
||||||
|
)
|
||||||
|
return PackBook(title=title, series_position=position, year=year, files=files)
|
||||||
|
|
||||||
|
|
||||||
|
def _common_root_parts(paths: list[PurePosixPath]) -> tuple[str, ...]:
|
||||||
|
parents = [p.parent.parts for p in paths]
|
||||||
|
common: list[str] = []
|
||||||
|
for parts in zip(*parents, strict=False):
|
||||||
|
if len(set(parts)) != 1:
|
||||||
|
break
|
||||||
|
common.append(parts[0])
|
||||||
|
return tuple(common)
|
||||||
|
|
||||||
|
|
||||||
|
def plan_pack(
|
||||||
|
files: list[PackFile],
|
||||||
|
*,
|
||||||
|
supported_extensions: set[str],
|
||||||
|
series_name: str | None,
|
||||||
|
author_name: str | None = None,
|
||||||
|
root_depth: int | None = None,
|
||||||
|
) -> PackPlan:
|
||||||
|
"""Group a release's file list into books.
|
||||||
|
|
||||||
|
Files in a subfolder (relative to the common root) group by that subfolder. Files
|
||||||
|
directly in the root split one-book-per-file only when at least two of them carry
|
||||||
|
a series position in their names; otherwise they are one book (a chaptered
|
||||||
|
audiobook, e.g. `01.mp3`, `02.mp3`). `root_depth` fixes how many leading path
|
||||||
|
components form the root instead of deriving it from the files' common parent.
|
||||||
|
"""
|
||||||
|
supported = {ext.lower().lstrip(".") for ext in supported_extensions}
|
||||||
|
book_files: list[PurePosixPath] = []
|
||||||
|
ignored: list[str] = []
|
||||||
|
for pack_file in files:
|
||||||
|
rel = PurePosixPath(pack_file.path.replace("\\", "/").lstrip("./"))
|
||||||
|
if rel.suffix.lower().lstrip(".") in supported:
|
||||||
|
book_files.append(rel)
|
||||||
|
else:
|
||||||
|
ignored.append(pack_file.path)
|
||||||
|
|
||||||
|
if not book_files:
|
||||||
|
return PackPlan(books=[], ignored=ignored)
|
||||||
|
|
||||||
|
root_parts = (
|
||||||
|
_common_root_parts(book_files) if root_depth is None else book_files[0].parts[:root_depth]
|
||||||
|
)
|
||||||
|
depth = len(root_parts)
|
||||||
|
|
||||||
|
root_files: list[PurePosixPath] = []
|
||||||
|
folders: dict[str, list[str]] = {}
|
||||||
|
for rel in book_files:
|
||||||
|
remainder = rel.parts[depth:]
|
||||||
|
if len(remainder) > 1:
|
||||||
|
folders.setdefault(remainder[0], []).append(str(rel))
|
||||||
|
else:
|
||||||
|
root_files.append(rel)
|
||||||
|
|
||||||
|
books: list[PackBook] = []
|
||||||
|
if root_files:
|
||||||
|
parsed = [
|
||||||
|
parse_pack_book_name(f.stem, series_name=series_name, author_name=author_name)
|
||||||
|
for f in root_files
|
||||||
|
]
|
||||||
|
positions = {p[1] for p in parsed if p[1] is not None}
|
||||||
|
if len(positions) >= 2:
|
||||||
|
books.extend(
|
||||||
|
PackBook(title=title, series_position=position, year=year, files=[str(f)])
|
||||||
|
for f, (title, position, year) in zip(root_files, parsed, strict=True)
|
||||||
|
)
|
||||||
|
elif len(root_files) == 1:
|
||||||
|
books.append(
|
||||||
|
_book_from_name(root_files[0].stem, [str(root_files[0])], series_name, author_name)
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
group_name = root_parts[-1] if root_parts else ""
|
||||||
|
books.append(
|
||||||
|
_book_from_name(group_name, [str(f) for f in root_files], series_name, author_name)
|
||||||
|
)
|
||||||
|
|
||||||
|
books.extend(
|
||||||
|
_book_from_name(folder, paths, series_name, author_name)
|
||||||
|
for folder, paths in folders.items()
|
||||||
|
)
|
||||||
|
return PackPlan(books=books, ignored=ignored)
|
||||||
|
|
||||||
|
|
||||||
|
def _relative_paths(
|
||||||
|
book_files: list[Path], root: Path | None = None
|
||||||
|
) -> tuple[Path, dict[Path, str]]:
|
||||||
|
if root is None:
|
||||||
|
root = Path(os.path.commonpath([str(f.parent) for f in book_files]))
|
||||||
|
return root, {f: f.relative_to(root).as_posix() for f in book_files}
|
||||||
|
|
||||||
|
|
||||||
|
def group_files_into_books(
|
||||||
|
book_files: list[Path],
|
||||||
|
*,
|
||||||
|
series_name: str | None,
|
||||||
|
author_name: str | None = None,
|
||||||
|
root: Path | None = None,
|
||||||
|
) -> list[BookGroup]:
|
||||||
|
"""Heuristically split on-disk files into books (see `plan_pack`).
|
||||||
|
|
||||||
|
`root` pins the release root when grouping a subset of a larger file set.
|
||||||
|
"""
|
||||||
|
if not book_files:
|
||||||
|
return []
|
||||||
|
_root, rel_by_path = _relative_paths(book_files, root)
|
||||||
|
path_by_rel = {rel: path for path, rel in rel_by_path.items()}
|
||||||
|
extensions = {f.suffix.lower().lstrip(".") for f in book_files}
|
||||||
|
plan = plan_pack(
|
||||||
|
[PackFile(rel) for rel in rel_by_path.values()],
|
||||||
|
supported_extensions=extensions,
|
||||||
|
series_name=series_name,
|
||||||
|
author_name=author_name,
|
||||||
|
root_depth=None if root is None else 0,
|
||||||
|
)
|
||||||
|
return [
|
||||||
|
BookGroup(
|
||||||
|
title=book.title,
|
||||||
|
series_position=book.series_position,
|
||||||
|
year=book.year,
|
||||||
|
files=[path_by_rel[rel] for rel in book.files],
|
||||||
|
)
|
||||||
|
for book in plan.books
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def match_plan_to_files(
|
||||||
|
plan: list[PackBook],
|
||||||
|
book_files: list[Path],
|
||||||
|
*,
|
||||||
|
series_name: str | None = None,
|
||||||
|
author_name: str | None = None,
|
||||||
|
) -> list[BookGroup]:
|
||||||
|
"""Apply an approved plan to on-disk files.
|
||||||
|
|
||||||
|
Files match by release-relative path first, then by basename (archive extraction
|
||||||
|
and client save paths can shift the root), then by the on-disk basename being a
|
||||||
|
suffix of the planned name (sources that glue folder and file names together).
|
||||||
|
Book files the plan does not mention fall back to heuristic grouping so nothing
|
||||||
|
is silently dropped.
|
||||||
|
"""
|
||||||
|
if not book_files:
|
||||||
|
return []
|
||||||
|
root, rel_by_path = _relative_paths(book_files)
|
||||||
|
by_rel = {rel: path for path, rel in rel_by_path.items()}
|
||||||
|
by_name: dict[str, list[Path]] = {}
|
||||||
|
for path in book_files:
|
||||||
|
by_name.setdefault(path.name, []).append(path)
|
||||||
|
|
||||||
|
claimed: set[Path] = set()
|
||||||
|
groups: list[BookGroup] = []
|
||||||
|
for book in plan:
|
||||||
|
matched: list[Path] = []
|
||||||
|
for wanted in book.files:
|
||||||
|
wanted_rel = wanted.replace("\\", "/").lstrip("./")
|
||||||
|
candidate = by_rel.get(wanted_rel)
|
||||||
|
if candidate is None:
|
||||||
|
candidates = [
|
||||||
|
p for p in by_name.get(PurePosixPath(wanted_rel).name, []) if p not in claimed
|
||||||
|
]
|
||||||
|
candidate = candidates[0] if candidates else None
|
||||||
|
if candidate is None:
|
||||||
|
wanted_name = PurePosixPath(wanted_rel).name.lower()
|
||||||
|
candidates = [
|
||||||
|
p
|
||||||
|
for p in book_files
|
||||||
|
if p not in claimed and wanted_name.endswith(p.name.lower())
|
||||||
|
]
|
||||||
|
candidate = candidates[0] if len(candidates) == 1 else None
|
||||||
|
if candidate is not None and candidate not in claimed:
|
||||||
|
claimed.add(candidate)
|
||||||
|
matched.append(candidate)
|
||||||
|
if matched:
|
||||||
|
groups.append(
|
||||||
|
BookGroup(
|
||||||
|
title=book.title,
|
||||||
|
series_position=book.series_position,
|
||||||
|
year=book.year,
|
||||||
|
files=matched,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
unmatched = [p for p in book_files if p not in claimed]
|
||||||
|
if unmatched:
|
||||||
|
groups.extend(
|
||||||
|
group_files_into_books(
|
||||||
|
unmatched, series_name=series_name, author_name=author_name, root=root
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return groups
|
||||||
@@ -40,6 +40,7 @@ from .transfer import (
|
|||||||
build_metadata_dict,
|
build_metadata_dict,
|
||||||
is_torrent_source,
|
is_torrent_source,
|
||||||
process_directory,
|
process_directory,
|
||||||
|
resolve_book_groups,
|
||||||
resolve_hardlink_source,
|
resolve_hardlink_source,
|
||||||
should_hardlink,
|
should_hardlink,
|
||||||
transfer_book_files,
|
transfer_book_files,
|
||||||
@@ -80,6 +81,7 @@ __all__ = [
|
|||||||
"process_directory",
|
"process_directory",
|
||||||
"record_step",
|
"record_step",
|
||||||
"resolve_custom_script_target",
|
"resolve_custom_script_target",
|
||||||
|
"resolve_book_groups",
|
||||||
"resolve_hardlink_source",
|
"resolve_hardlink_source",
|
||||||
"run_custom_script",
|
"run_custom_script",
|
||||||
"safe_cleanup_path",
|
"safe_cleanup_path",
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
import os
|
import os
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
@@ -26,6 +27,7 @@ from shelfmark.download.fs import (
|
|||||||
)
|
)
|
||||||
from shelfmark.download.postprocess.policy import get_file_organization, get_template
|
from shelfmark.download.postprocess.policy import get_file_organization, get_template
|
||||||
|
|
||||||
|
from .packs import BookGroup, PackBook, group_files_into_books, match_plan_to_files
|
||||||
from .scan import collect_directory_files, scan_directory_tree
|
from .scan import collect_directory_files, scan_directory_tree
|
||||||
from .types import TransferPlan
|
from .types import TransferPlan
|
||||||
from .workspace import safe_cleanup_path
|
from .workspace import safe_cleanup_path
|
||||||
@@ -196,6 +198,19 @@ def transfer_book_files(
|
|||||||
|
|
||||||
is_audiobook = check_audiobook(task.content_type)
|
is_audiobook = check_audiobook(task.content_type)
|
||||||
organization_mode = organization_mode or get_file_organization(is_audiobook=is_audiobook)
|
organization_mode = organization_mode or get_file_organization(is_audiobook=is_audiobook)
|
||||||
|
|
||||||
|
groups = resolve_book_groups(task, book_files, organization_mode=organization_mode)
|
||||||
|
if groups is not None:
|
||||||
|
return _transfer_book_groups(
|
||||||
|
groups,
|
||||||
|
destination,
|
||||||
|
task,
|
||||||
|
use_hardlink=use_hardlink,
|
||||||
|
is_torrent=is_torrent,
|
||||||
|
preserve_source=preserve_source,
|
||||||
|
organization_mode=organization_mode,
|
||||||
|
)
|
||||||
|
|
||||||
max_attempts = _max_attempts_for_batch(len(book_files))
|
max_attempts = _max_attempts_for_batch(len(book_files))
|
||||||
|
|
||||||
final_paths: list[Path] = []
|
final_paths: list[Path] = []
|
||||||
@@ -299,6 +314,101 @@ def transfer_book_files(
|
|||||||
return final_paths, None, op_counts
|
return final_paths, None, op_counts
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_book_groups(
|
||||||
|
task: DownloadTask,
|
||||||
|
book_files: list[Path],
|
||||||
|
*,
|
||||||
|
organization_mode: str,
|
||||||
|
) -> list[BookGroup] | None:
|
||||||
|
"""Split a multi-book pack into per-book groups, or None to file as one book.
|
||||||
|
|
||||||
|
An approved `book_plan` wins; a bare `multi_book` flag falls back to heuristic
|
||||||
|
grouping. Organization `none` keeps files as-is, and a split that yields a single
|
||||||
|
group is not a pack at all.
|
||||||
|
"""
|
||||||
|
if organization_mode == "none" or not (task.book_plan or task.multi_book):
|
||||||
|
return None
|
||||||
|
if task.book_plan:
|
||||||
|
plan = [
|
||||||
|
PackBook(
|
||||||
|
title=str(entry.get("title") or ""),
|
||||||
|
series_position=entry.get("series_position"),
|
||||||
|
year=entry.get("year"),
|
||||||
|
files=list(entry.get("files") or []),
|
||||||
|
)
|
||||||
|
for entry in task.book_plan
|
||||||
|
if isinstance(entry, dict)
|
||||||
|
]
|
||||||
|
groups = match_plan_to_files(
|
||||||
|
plan, book_files, series_name=task.series_name, author_name=task.author
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
groups = group_files_into_books(
|
||||||
|
book_files, series_name=task.series_name, author_name=task.author
|
||||||
|
)
|
||||||
|
return groups if len(groups) > 1 else None
|
||||||
|
|
||||||
|
|
||||||
|
def _transfer_book_groups(
|
||||||
|
groups: list[BookGroup],
|
||||||
|
destination: Path,
|
||||||
|
task: DownloadTask,
|
||||||
|
*,
|
||||||
|
use_hardlink: bool,
|
||||||
|
is_torrent: bool,
|
||||||
|
preserve_source: bool,
|
||||||
|
organization_mode: str,
|
||||||
|
) -> tuple[list[Path], str | None, dict[str, int]]:
|
||||||
|
"""Transfer each book of a pack through the normal single-book path.
|
||||||
|
|
||||||
|
Each book gets an isolated task copy (the single-file path mutates `task.format`)
|
||||||
|
carrying its own title, position and year; the searched book's position must not
|
||||||
|
leak onto its siblings, while author and series name apply to all of them.
|
||||||
|
"""
|
||||||
|
all_paths: list[Path] = []
|
||||||
|
totals: dict[str, int] = {"hardlink": 0, "copy": 0, "move": 0}
|
||||||
|
errors: list[str] = []
|
||||||
|
|
||||||
|
for group in groups:
|
||||||
|
book_task = dataclasses.replace(
|
||||||
|
task,
|
||||||
|
title=group.title or task.title,
|
||||||
|
year=str(group.year) if group.year is not None else None,
|
||||||
|
subtitle=None,
|
||||||
|
series_position=group.series_position,
|
||||||
|
multi_book=False,
|
||||||
|
book_plan=None,
|
||||||
|
)
|
||||||
|
paths, error, op_counts = transfer_book_files(
|
||||||
|
group.files,
|
||||||
|
destination,
|
||||||
|
book_task,
|
||||||
|
use_hardlink=use_hardlink,
|
||||||
|
is_torrent=is_torrent,
|
||||||
|
preserve_source=preserve_source,
|
||||||
|
organization_mode=organization_mode,
|
||||||
|
source_root=group.files[0].parent,
|
||||||
|
)
|
||||||
|
for op, count in op_counts.items():
|
||||||
|
totals[op] = totals.get(op, 0) + count
|
||||||
|
if error:
|
||||||
|
errors.append(f"{group.title}: {error}")
|
||||||
|
logger.warning("Task %s: pack book %r failed: %s", task.task_id, group.title, error)
|
||||||
|
continue
|
||||||
|
all_paths.extend(paths)
|
||||||
|
|
||||||
|
if not all_paths:
|
||||||
|
return [], "; ".join(errors) or "No book files found", totals
|
||||||
|
if errors:
|
||||||
|
logger.warning(
|
||||||
|
"Task %s: pack filed with %d failed book(s): %s",
|
||||||
|
task.task_id,
|
||||||
|
len(errors),
|
||||||
|
"; ".join(errors),
|
||||||
|
)
|
||||||
|
return all_paths, None, totals
|
||||||
|
|
||||||
|
|
||||||
def process_directory(
|
def process_directory(
|
||||||
directory: Path,
|
directory: Path,
|
||||||
ingest_dir: Path,
|
ingest_dir: Path,
|
||||||
|
|||||||
@@ -62,6 +62,7 @@ from shelfmark.core.notifications import (
|
|||||||
notify_user,
|
notify_user,
|
||||||
)
|
)
|
||||||
from shelfmark.core.prefix_middleware import PrefixMiddleware
|
from shelfmark.core.prefix_middleware import PrefixMiddleware
|
||||||
|
from shelfmark.core.release_inspect_routes import register_release_inspect_routes
|
||||||
from shelfmark.core.request_helpers import (
|
from shelfmark.core.request_helpers import (
|
||||||
coerce_bool,
|
coerce_bool,
|
||||||
emit_ws_event,
|
emit_ws_event,
|
||||||
@@ -1024,6 +1025,9 @@ def _serialize_release(release: Release) -> dict:
|
|||||||
return result
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
register_release_inspect_routes(app, login_required)
|
||||||
|
|
||||||
|
|
||||||
@app.route("/api/releases/download", methods=["POST"])
|
@app.route("/api/releases/download", methods=["POST"])
|
||||||
@login_required
|
@login_required
|
||||||
def api_download_release() -> Response | tuple[Response, int]:
|
def api_download_release() -> Response | tuple[Response, int]:
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ if TYPE_CHECKING:
|
|||||||
|
|
||||||
from shelfmark.core.models import DownloadTask
|
from shelfmark.core.models import DownloadTask
|
||||||
from shelfmark.core.search_plan import ReleaseSearchPlan
|
from shelfmark.core.search_plan import ReleaseSearchPlan
|
||||||
|
from shelfmark.download.postprocess.packs import PackFile
|
||||||
|
|
||||||
from shelfmark.metadata_providers import BookMetadata
|
from shelfmark.metadata_providers import BookMetadata
|
||||||
|
|
||||||
@@ -400,6 +401,14 @@ class DownloadHandler(ABC):
|
|||||||
"""Return private queue-time fields needed for restart-safe retry."""
|
"""Return private queue-time fields needed for restart-safe retry."""
|
||||||
return {}
|
return {}
|
||||||
|
|
||||||
|
def list_files(self, release_data: dict[str, Any]) -> list[PackFile] | None:
|
||||||
|
"""Return the release's file list without downloading it.
|
||||||
|
|
||||||
|
Lets the UI review a multi-book pack before queueing. Return None when the
|
||||||
|
source cannot know the files ahead of time (magnet links, usenet, ...).
|
||||||
|
"""
|
||||||
|
return None
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def cancel(self, task_id: str) -> bool:
|
def cancel(self, task_id: str) -> bool:
|
||||||
"""Cancel an in-progress download."""
|
"""Cancel an in-progress download."""
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
"""AudiobookBay download handler - resolves magnet links and uses shared client lifecycle."""
|
"""AudiobookBay download handler - resolves magnet links and uses shared client lifecycle."""
|
||||||
|
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING, Any
|
||||||
from urllib.parse import urlparse
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
from shelfmark.core.config import config
|
from shelfmark.core.config import config
|
||||||
@@ -22,6 +22,7 @@ if TYPE_CHECKING:
|
|||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
|
|
||||||
from shelfmark.core.models import DownloadTask
|
from shelfmark.core.models import DownloadTask
|
||||||
|
from shelfmark.download.postprocess.packs import PackFile
|
||||||
|
|
||||||
logger = setup_logger(__name__)
|
logger = setup_logger(__name__)
|
||||||
DEFAULT_ABB_HOSTNAME = "audiobookbay.lu"
|
DEFAULT_ABB_HOSTNAME = "audiobookbay.lu"
|
||||||
@@ -68,6 +69,19 @@ class AudiobookBayHandler(ExternalClientHandler):
|
|||||||
return task_id
|
return task_id
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
def list_files(self, release_data: dict[str, Any]) -> list[PackFile] | None:
|
||||||
|
"""Read the torrent's file list off the detail page, without downloading."""
|
||||||
|
raw_url = release_data.get("download_url") or release_data.get("source_url")
|
||||||
|
detail_url = raw_url.strip() if isinstance(raw_url, str) else ""
|
||||||
|
hostname = _resolve_allowed_detail_hostname()
|
||||||
|
if not detail_url or not _detail_url_matches_host(detail_url, hostname):
|
||||||
|
logger.debug("Cannot list files for AudiobookBay release without a valid detail URL")
|
||||||
|
return None
|
||||||
|
detail_html = scraper.fetch_detail_html(detail_url, hostname)
|
||||||
|
if not detail_html:
|
||||||
|
return None
|
||||||
|
return scraper.extract_file_list(detail_html)
|
||||||
|
|
||||||
def _get_client(self, protocol: str) -> DownloadClient | None:
|
def _get_client(self, protocol: str) -> DownloadClient | None:
|
||||||
"""Compatibility shim so module-level patching still works in tests."""
|
"""Compatibility shim so module-level patching still works in tests."""
|
||||||
return get_client(protocol)
|
return get_client(protocol)
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
|
|
||||||
import re
|
import re
|
||||||
import time
|
import time
|
||||||
|
from threading import Lock
|
||||||
from urllib.parse import quote, quote_plus
|
from urllib.parse import quote, quote_plus
|
||||||
|
|
||||||
import requests
|
import requests
|
||||||
@@ -10,6 +11,7 @@ from bs4 import BeautifulSoup
|
|||||||
from shelfmark.core.config import config
|
from shelfmark.core.config import config
|
||||||
from shelfmark.core.logger import setup_logger
|
from shelfmark.core.logger import setup_logger
|
||||||
from shelfmark.download import http as downloader
|
from shelfmark.download import http as downloader
|
||||||
|
from shelfmark.download.postprocess.packs import PackFile
|
||||||
from shelfmark.release_sources.audiobookbay.utils import normalize_search_punctuation
|
from shelfmark.release_sources.audiobookbay.utils import normalize_search_punctuation
|
||||||
|
|
||||||
logger = setup_logger(__name__)
|
logger = setup_logger(__name__)
|
||||||
@@ -32,6 +34,13 @@ FIRST_PAGE_SESSION_REFRESH_ATTEMPTS = 2
|
|||||||
# Legacy search parameter used by older ABB flows
|
# Legacy search parameter used by older ABB flows
|
||||||
LEGACY_CATEGORY_QUERY = "undefined%2Cundefined"
|
LEGACY_CATEGORY_QUERY = "undefined%2Cundefined"
|
||||||
|
|
||||||
|
# Detail pages are fetched once and shared by inspection (file list) and download
|
||||||
|
# (magnet link) so a "review then download" round trip costs ABB a single request.
|
||||||
|
DETAIL_PAGE_CACHE_TTL_SECONDS = 120.0
|
||||||
|
DETAIL_PAGE_CACHE_MAX_ENTRIES = 8
|
||||||
|
_detail_page_cache: dict[str, tuple[float, str]] = {}
|
||||||
|
_detail_page_cache_lock = Lock()
|
||||||
|
|
||||||
# Precompiled patterns used while parsing result cards
|
# Precompiled patterns used while parsing result cards
|
||||||
LANGUAGE_PATTERN = re.compile(r"Language:\s*([A-Za-z]+)")
|
LANGUAGE_PATTERN = re.compile(r"Language:\s*([A-Za-z]+)")
|
||||||
POSTED_PATTERN = re.compile(r"Posted:\s*(\d+\s+[A-Za-z]+\s+\d{4})")
|
POSTED_PATTERN = re.compile(r"Posted:\s*(\d+\s+[A-Za-z]+\s+\d{4})")
|
||||||
@@ -39,6 +48,11 @@ FORMAT_PATTERN = re.compile(r"Format:\s*([A-Za-z0-9]+)")
|
|||||||
BITRATE_PATTERN = re.compile(r"Bitrate:\s*([\d]+\s*[A-Za-z/]+)")
|
BITRATE_PATTERN = re.compile(r"Bitrate:\s*([\d]+\s*[A-Za-z/]+)")
|
||||||
SIZE_PATTERN = re.compile(r"File Size:\s*([\d.]+)\s*([A-Za-z]+)")
|
SIZE_PATTERN = re.compile(r"File Size:\s*([\d.]+)\s*([A-Za-z]+)")
|
||||||
INFO_HASH_LABEL_PATTERN = re.compile(r"Info Hash", re.IGNORECASE)
|
INFO_HASH_LABEL_PATTERN = re.compile(r"Info Hash", re.IGNORECASE)
|
||||||
|
FILE_ROW_SIZE_PATTERN = re.compile(
|
||||||
|
r"^(?P<name>.+?)\s+(?P<size>\d+(?:\.\d+)?)\s*(?P<unit>Bytes?|KBs?|MBs?|GBs?|TBs?)$",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
_FILE_SIZE_MULTIPLIERS = {"b": 1, "k": 1024, "m": 1024**2, "g": 1024**3, "t": 1024**4}
|
||||||
|
|
||||||
|
|
||||||
def _coerce_non_negative_float(value: object, default: float) -> float:
|
def _coerce_non_negative_float(value: object, default: float) -> float:
|
||||||
@@ -348,6 +362,98 @@ def search_audiobookbay(
|
|||||||
return results
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def _get_cached_detail_page(details_url: str) -> str | None:
|
||||||
|
with _detail_page_cache_lock:
|
||||||
|
entry = _detail_page_cache.get(details_url)
|
||||||
|
if entry is None:
|
||||||
|
return None
|
||||||
|
fetched_at, html = entry
|
||||||
|
if time.monotonic() - fetched_at > DETAIL_PAGE_CACHE_TTL_SECONDS:
|
||||||
|
del _detail_page_cache[details_url]
|
||||||
|
return None
|
||||||
|
return html
|
||||||
|
|
||||||
|
|
||||||
|
def _store_cached_detail_page(details_url: str, html: str) -> None:
|
||||||
|
with _detail_page_cache_lock:
|
||||||
|
_detail_page_cache[details_url] = (time.monotonic(), html)
|
||||||
|
while len(_detail_page_cache) > DETAIL_PAGE_CACHE_MAX_ENTRIES:
|
||||||
|
oldest = min(_detail_page_cache, key=lambda key: _detail_page_cache[key][0])
|
||||||
|
del _detail_page_cache[oldest]
|
||||||
|
|
||||||
|
|
||||||
|
def clear_detail_page_cache() -> None:
|
||||||
|
"""Drop cached detail pages (used by tests)."""
|
||||||
|
with _detail_page_cache_lock:
|
||||||
|
_detail_page_cache.clear()
|
||||||
|
|
||||||
|
|
||||||
|
def _fetch_detail_page_once(details_url: str, hostname: str) -> str:
|
||||||
|
session = requests.Session()
|
||||||
|
_bootstrap_abb_session(hostname, session, DETAIL_PAGE_RETRY_ATTEMPTS)
|
||||||
|
return _coerce_markup_to_html(
|
||||||
|
downloader.html_get_page(
|
||||||
|
details_url,
|
||||||
|
retry=DETAIL_PAGE_RETRY_ATTEMPTS,
|
||||||
|
use_bypasser=False,
|
||||||
|
allow_bypasser_fallback=False,
|
||||||
|
success_delay=0,
|
||||||
|
session=session,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_detail_html(details_url: str, hostname: str = "audiobookbay.lu") -> str:
|
||||||
|
"""Fetch a detail page (one retry with a fresh session), cached briefly per URL."""
|
||||||
|
cached = _get_cached_detail_page(details_url)
|
||||||
|
if cached is not None:
|
||||||
|
logger.debug("Reusing recently fetched detail page: %s", details_url)
|
||||||
|
return cached
|
||||||
|
detail_html = _fetch_detail_page_once(details_url, hostname)
|
||||||
|
if not detail_html:
|
||||||
|
detail_html = _fetch_detail_page_once(details_url, hostname)
|
||||||
|
if detail_html:
|
||||||
|
_store_cached_detail_page(details_url, detail_html)
|
||||||
|
return detail_html
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_file_row(text: str) -> PackFile | None:
|
||||||
|
match = FILE_ROW_SIZE_PATTERN.match(text.strip())
|
||||||
|
if not match:
|
||||||
|
return None
|
||||||
|
multiplier = _FILE_SIZE_MULTIPLIERS[match.group("unit")[0].lower()]
|
||||||
|
return PackFile(match.group("name"), int(float(match.group("size")) * multiplier))
|
||||||
|
|
||||||
|
|
||||||
|
def extract_file_list(detail_html: str) -> list[PackFile] | None:
|
||||||
|
"""Read the torrent file rows off a detail page.
|
||||||
|
|
||||||
|
ABB renders the torrent's file table as single-cell rows between the
|
||||||
|
"This is a Multifile Torrent" marker (absent for single-file torrents) and the
|
||||||
|
"Combined File Size" row. Returns None when the page has no such table.
|
||||||
|
"""
|
||||||
|
soup = BeautifulSoup(detail_html, "html.parser")
|
||||||
|
rows: list[PackFile] = []
|
||||||
|
for row in soup.find_all("tr"):
|
||||||
|
cells = row.find_all("td")
|
||||||
|
if not cells:
|
||||||
|
continue
|
||||||
|
label = cells[0].get_text(" ", strip=True)
|
||||||
|
if label.lower().startswith("combined file size"):
|
||||||
|
return rows or None
|
||||||
|
if len(cells) != 1:
|
||||||
|
rows = [] # a two-column metadata row means we're not in the file table yet
|
||||||
|
continue
|
||||||
|
text = cells[0].get_text(" ", strip=True)
|
||||||
|
if "multifile torrent" in text.lower():
|
||||||
|
rows = []
|
||||||
|
continue
|
||||||
|
parsed = _parse_file_row(text)
|
||||||
|
if parsed is not None:
|
||||||
|
rows.append(parsed)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
def extract_magnet_link(details_url: str, hostname: str = "audiobookbay.lu") -> str | None:
|
def extract_magnet_link(details_url: str, hostname: str = "audiobookbay.lu") -> str | None:
|
||||||
"""Extract info hash and trackers from book detail page, then construct magnet link.
|
"""Extract info hash and trackers from book detail page, then construct magnet link.
|
||||||
|
|
||||||
@@ -360,35 +466,7 @@ def extract_magnet_link(details_url: str, hostname: str = "audiobookbay.lu") ->
|
|||||||
|
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
session = requests.Session()
|
detail_html = fetch_detail_html(details_url, hostname)
|
||||||
_bootstrap_abb_session(hostname, session, DETAIL_PAGE_RETRY_ATTEMPTS)
|
|
||||||
|
|
||||||
# Fetch detail page
|
|
||||||
detail_html = _coerce_markup_to_html(
|
|
||||||
downloader.html_get_page(
|
|
||||||
details_url,
|
|
||||||
retry=DETAIL_PAGE_RETRY_ATTEMPTS,
|
|
||||||
use_bypasser=False,
|
|
||||||
allow_bypasser_fallback=False,
|
|
||||||
success_delay=0,
|
|
||||||
session=session,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
if not detail_html:
|
|
||||||
session = requests.Session()
|
|
||||||
_bootstrap_abb_session(hostname, session, DETAIL_PAGE_RETRY_ATTEMPTS)
|
|
||||||
detail_html = _coerce_markup_to_html(
|
|
||||||
downloader.html_get_page(
|
|
||||||
details_url,
|
|
||||||
retry=DETAIL_PAGE_RETRY_ATTEMPTS,
|
|
||||||
use_bypasser=False,
|
|
||||||
allow_bypasser_fallback=False,
|
|
||||||
success_delay=0,
|
|
||||||
session=session,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
if not detail_html:
|
if not detail_html:
|
||||||
logger.warning("Failed to fetch details page")
|
logger.warning("Failed to fetch details page")
|
||||||
return None
|
return None
|
||||||
|
|||||||
@@ -28,6 +28,10 @@ from shelfmark.download.clients.base_handler import (
|
|||||||
DownloadRequest,
|
DownloadRequest,
|
||||||
ExternalClientHandler,
|
ExternalClientHandler,
|
||||||
)
|
)
|
||||||
|
from shelfmark.download.clients.torrent_utils import (
|
||||||
|
extract_file_list_from_torrent,
|
||||||
|
extract_torrent_info,
|
||||||
|
)
|
||||||
from shelfmark.metadata_providers import BookMetadata
|
from shelfmark.metadata_providers import BookMetadata
|
||||||
from shelfmark.release_sources import register_handler
|
from shelfmark.release_sources import register_handler
|
||||||
from shelfmark.release_sources.prowlarr.api import IndexerSeedSettings, ProwlarrClient
|
from shelfmark.release_sources.prowlarr.api import IndexerSeedSettings, ProwlarrClient
|
||||||
@@ -38,12 +42,14 @@ from shelfmark.release_sources.prowlarr.utils import (
|
|||||||
coerce_int_like,
|
coerce_int_like,
|
||||||
get_preferred_download_url,
|
get_preferred_download_url,
|
||||||
get_protocol,
|
get_protocol,
|
||||||
|
sanitize_download_url,
|
||||||
)
|
)
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
|
|
||||||
from shelfmark.core.models import DownloadTask
|
from shelfmark.core.models import DownloadTask
|
||||||
|
from shelfmark.download.postprocess.packs import PackFile
|
||||||
|
|
||||||
logger = setup_logger(__name__)
|
logger = setup_logger(__name__)
|
||||||
|
|
||||||
@@ -127,6 +133,24 @@ class ProwlarrHandler(ExternalClientHandler):
|
|||||||
|
|
||||||
return settings.get(indexer_id)
|
return settings.get(indexer_id)
|
||||||
|
|
||||||
|
def list_files(self, release_data: dict[str, Any]) -> list[PackFile] | None:
|
||||||
|
"""List a cached torrent release's files from its .torrent, without downloading.
|
||||||
|
|
||||||
|
Magnet-only and usenet releases cannot be listed ahead of time.
|
||||||
|
"""
|
||||||
|
source_id = str(release_data.get("source_id") or "")
|
||||||
|
prowlarr_result = get_release(source_id) if source_id else None
|
||||||
|
if not prowlarr_result or get_protocol(prowlarr_result) != "torrent":
|
||||||
|
return None
|
||||||
|
download_url = sanitize_download_url(str(prowlarr_result.get("downloadUrl") or "").strip())
|
||||||
|
if not download_url or download_url.startswith("magnet:"):
|
||||||
|
return None
|
||||||
|
expected_hash = str(prowlarr_result.get("infoHash") or "").strip() or None
|
||||||
|
info = extract_torrent_info(download_url, expected_hash=expected_hash)
|
||||||
|
if not info.torrent_data:
|
||||||
|
return None
|
||||||
|
return extract_file_list_from_torrent(info.torrent_data)
|
||||||
|
|
||||||
def _get_client(self, protocol: str) -> DownloadClient | None:
|
def _get_client(self, protocol: str) -> DownloadClient | None:
|
||||||
"""Compatibility shim so module-level patching still works in tests."""
|
"""Compatibility shim so module-level patching still works in tests."""
|
||||||
return get_client(protocol)
|
return get_client(protocol)
|
||||||
|
|||||||
@@ -58,7 +58,6 @@ import {
|
|||||||
isApiResponseError,
|
isApiResponseError,
|
||||||
updateSelfUser,
|
updateSelfUser,
|
||||||
setBookTargetState,
|
setBookTargetState,
|
||||||
type DownloadReleasePayload,
|
|
||||||
} from './services/api';
|
} from './services/api';
|
||||||
import type {
|
import type {
|
||||||
Book,
|
Book,
|
||||||
@@ -93,6 +92,7 @@ import { getEffectiveMetadataSort } from './utils/metadataSort';
|
|||||||
import { isRecord } from './utils/objectHelpers';
|
import { isRecord } from './utils/objectHelpers';
|
||||||
import { policyTrace } from './utils/policyTrace';
|
import { policyTrace } from './utils/policyTrace';
|
||||||
import { buildQueryTargets, getDefaultQueryTargetKey } from './utils/queryTargets';
|
import { buildQueryTargets, getDefaultQueryTargetKey } from './utils/queryTargets';
|
||||||
|
import { buildReleaseDownloadPayload, type ReleaseDownloadOptions } from './utils/releasePayload';
|
||||||
import { applyRequestNoteToPayload } from './utils/requestConfirmation';
|
import { applyRequestNoteToPayload } from './utils/requestConfirmation';
|
||||||
import { bookFromRequestData } from './utils/requestFulfil';
|
import { bookFromRequestData } from './utils/requestFulfil';
|
||||||
import {
|
import {
|
||||||
@@ -219,6 +219,7 @@ type PendingOnBehalfDownload =
|
|||||||
release: Release;
|
release: Release;
|
||||||
releaseContentType: ContentType;
|
releaseContentType: ContentType;
|
||||||
actingAsUser: ActingAsUserSelection;
|
actingAsUser: ActingAsUserSelection;
|
||||||
|
options?: ReleaseDownloadOptions;
|
||||||
}
|
}
|
||||||
| {
|
| {
|
||||||
type: 'combined';
|
type: 'combined';
|
||||||
@@ -1072,41 +1073,6 @@ function App() {
|
|||||||
[],
|
[],
|
||||||
);
|
);
|
||||||
|
|
||||||
const buildReleaseDownloadPayload = useCallback(
|
|
||||||
(book: Book, release: Release, releaseContentType: ContentType): DownloadReleasePayload => {
|
|
||||||
const isManual = book.provider === 'manual';
|
|
||||||
const releasePreview =
|
|
||||||
typeof release.extra?.preview === 'string' ? release.extra.preview : undefined;
|
|
||||||
const releaseAuthor =
|
|
||||||
typeof release.extra?.author === 'string' ? release.extra.author : undefined;
|
|
||||||
|
|
||||||
return {
|
|
||||||
source: release.source,
|
|
||||||
source_id: release.source_id,
|
|
||||||
title: isManual ? release.title : book.title,
|
|
||||||
author: isManual ? releaseAuthor || '' : book.author,
|
|
||||||
year: book.year,
|
|
||||||
format: release.format,
|
|
||||||
size: release.size,
|
|
||||||
size_bytes: release.size_bytes,
|
|
||||||
download_url: release.download_url,
|
|
||||||
protocol: release.protocol,
|
|
||||||
indexer: release.indexer,
|
|
||||||
seeders: release.seeders,
|
|
||||||
extra: release.extra,
|
|
||||||
preview: isManual ? releasePreview || undefined : book.preview,
|
|
||||||
content_type: releaseContentType,
|
|
||||||
series_name: book.series_name,
|
|
||||||
series_position: book.series_position,
|
|
||||||
subtitle: book.subtitle,
|
|
||||||
// From the release, never the book: book.language is the provider's
|
|
||||||
// canonical edition, which would mislabel a translated release.
|
|
||||||
language: release.language ?? undefined,
|
|
||||||
};
|
|
||||||
},
|
|
||||||
[],
|
|
||||||
);
|
|
||||||
|
|
||||||
// When downloading a book while browsing a Hardcover list the user owns,
|
// When downloading a book while browsing a Hardcover list the user owns,
|
||||||
// automatically remove it from that list (fire-and-forget).
|
// automatically remove it from that list (fire-and-forget).
|
||||||
const searchFieldLabelsRef = useRef(searchFieldLabels);
|
const searchFieldLabelsRef = useRef(searchFieldLabels);
|
||||||
@@ -1214,12 +1180,13 @@ function App() {
|
|||||||
release: Release,
|
release: Release,
|
||||||
releaseContentType: ContentType,
|
releaseContentType: ContentType,
|
||||||
onBehalfOfUserId?: number,
|
onBehalfOfUserId?: number,
|
||||||
|
options?: ReleaseDownloadOptions,
|
||||||
): Promise<void> => {
|
): Promise<void> => {
|
||||||
const requestStartedAtSeconds = Date.now() / 1000;
|
const requestStartedAtSeconds = Date.now() / 1000;
|
||||||
try {
|
try {
|
||||||
trackRelease(book.id, release.source_id);
|
trackRelease(book.id, release.source_id);
|
||||||
await downloadRelease(
|
await downloadRelease(
|
||||||
buildReleaseDownloadPayload(book, release, releaseContentType),
|
buildReleaseDownloadPayload(book, release, releaseContentType, options),
|
||||||
onBehalfOfUserId,
|
onBehalfOfUserId,
|
||||||
);
|
);
|
||||||
await fetchStatus();
|
await fetchStatus();
|
||||||
@@ -1301,7 +1268,6 @@ function App() {
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
[
|
[
|
||||||
buildReleaseDownloadPayload,
|
|
||||||
fetchStatus,
|
fetchStatus,
|
||||||
openRequestConfirmation,
|
openRequestConfirmation,
|
||||||
refreshRequestPolicy,
|
refreshRequestPolicy,
|
||||||
@@ -1416,6 +1382,7 @@ function App() {
|
|||||||
effectivePendingOnBehalfDownload.release,
|
effectivePendingOnBehalfDownload.release,
|
||||||
effectivePendingOnBehalfDownload.releaseContentType,
|
effectivePendingOnBehalfDownload.releaseContentType,
|
||||||
onBehalfOfUserId,
|
onBehalfOfUserId,
|
||||||
|
effectivePendingOnBehalfDownload.options,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
setPendingOnBehalfDownload(null);
|
setPendingOnBehalfDownload(null);
|
||||||
@@ -1640,6 +1607,7 @@ function App() {
|
|||||||
book: Book,
|
book: Book,
|
||||||
release: Release,
|
release: Release,
|
||||||
releaseContentType: ContentType,
|
releaseContentType: ContentType,
|
||||||
|
options?: ReleaseDownloadOptions,
|
||||||
) => {
|
) => {
|
||||||
policyTrace('release.action:start', {
|
policyTrace('release.action:start', {
|
||||||
bookId: book.id,
|
bookId: book.id,
|
||||||
@@ -1655,11 +1623,12 @@ function App() {
|
|||||||
release,
|
release,
|
||||||
releaseContentType,
|
releaseContentType,
|
||||||
actingAsUser: effectiveActingAsUser,
|
actingAsUser: effectiveActingAsUser,
|
||||||
|
options,
|
||||||
});
|
});
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
await executeReleaseDownload(book, release, releaseContentType);
|
await executeReleaseDownload(book, release, releaseContentType, undefined, options);
|
||||||
};
|
};
|
||||||
|
|
||||||
const handleReleaseRequest = useCallback(
|
const handleReleaseRequest = useCallback(
|
||||||
|
|||||||
@@ -0,0 +1,193 @@
|
|||||||
|
import { useState } from 'react';
|
||||||
|
|
||||||
|
import type { PackBook, PackPlan, Release } from '../types';
|
||||||
|
import {
|
||||||
|
describePackPlan,
|
||||||
|
parseSeriesPositionInput,
|
||||||
|
toBookPlanPayload,
|
||||||
|
updateReviewBook,
|
||||||
|
} from '../utils/packReview';
|
||||||
|
import { ToggleSwitch } from './shared/ToggleSwitch';
|
||||||
|
|
||||||
|
interface PackReviewPanelProps {
|
||||||
|
release: Release;
|
||||||
|
plan: PackPlan;
|
||||||
|
books: PackBook[];
|
||||||
|
onChange: (books: PackBook[]) => void;
|
||||||
|
onBack: () => void;
|
||||||
|
/** `null` means "treat the whole release as one book". */
|
||||||
|
onConfirm: (books: PackBook[] | null) => Promise<void>;
|
||||||
|
isSubmitting: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
const inputClassName =
|
||||||
|
'w-full rounded-md border border-(--border-muted) bg-(--bg) px-2 py-1 text-sm text-(--text) focus:border-emerald-500 focus:outline-none';
|
||||||
|
|
||||||
|
export const PackReviewPanel = ({
|
||||||
|
release,
|
||||||
|
plan,
|
||||||
|
books,
|
||||||
|
onChange,
|
||||||
|
onBack,
|
||||||
|
onConfirm,
|
||||||
|
isSubmitting,
|
||||||
|
}: PackReviewPanelProps) => {
|
||||||
|
const [singleBook, setSingleBook] = useState(false);
|
||||||
|
const [expandedFiles, setExpandedFiles] = useState<number | null>(null);
|
||||||
|
const [showIgnored, setShowIgnored] = useState(false);
|
||||||
|
|
||||||
|
const payloadBooks = toBookPlanPayload(books);
|
||||||
|
const canConfirm = !isSubmitting && (singleBook || payloadBooks.length > 0);
|
||||||
|
const confirmLabel = singleBook
|
||||||
|
? 'Download as one book'
|
||||||
|
: `Download ${payloadBooks.length} ${payloadBooks.length === 1 ? 'book' : 'books'}`;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="flex flex-col gap-4 px-5 py-4" data-testid="pack-review-panel">
|
||||||
|
<div>
|
||||||
|
<h3 className="text-base font-semibold text-(--text)">
|
||||||
|
This release contains several books
|
||||||
|
</h3>
|
||||||
|
<p className="mt-1 text-sm text-zinc-500 dark:text-zinc-400">
|
||||||
|
<span className="font-medium text-(--text)">{release.title}</span> ·{' '}
|
||||||
|
{describePackPlan(books, plan.ignored)}
|
||||||
|
</p>
|
||||||
|
<p className="mt-1 text-xs text-zinc-500 dark:text-zinc-400">
|
||||||
|
Each book below is filed separately with its own title. Fix any titles before downloading
|
||||||
|
— the author and series come from the book you searched.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div className="flex items-center justify-between rounded-lg border border-(--border-muted) px-3 py-2">
|
||||||
|
<div>
|
||||||
|
<p className="text-sm font-medium text-(--text)">Treat as a single book</p>
|
||||||
|
<p className="text-xs text-zinc-500 dark:text-zinc-400">
|
||||||
|
Use this if the split is wrong and the files are really one audiobook.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<ToggleSwitch
|
||||||
|
checked={singleBook}
|
||||||
|
onChange={setSingleBook}
|
||||||
|
color="emerald"
|
||||||
|
ariaLabel="Treat as a single book"
|
||||||
|
disabled={isSubmitting}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div
|
||||||
|
className={`flex flex-col divide-y divide-zinc-200/60 dark:divide-zinc-800/60 ${
|
||||||
|
singleBook ? 'pointer-events-none opacity-40' : ''
|
||||||
|
}`}
|
||||||
|
>
|
||||||
|
<div className="grid grid-cols-[minmax(0,1fr)_72px_72px_80px] gap-2 pb-1 text-xs font-medium tracking-wide text-zinc-500 uppercase dark:text-zinc-400">
|
||||||
|
<span>Title</span>
|
||||||
|
<span>Series #</span>
|
||||||
|
<span>Year</span>
|
||||||
|
<span className="text-right">Files</span>
|
||||||
|
</div>
|
||||||
|
{books.map((book, index) => (
|
||||||
|
<div key={book.files[0] ?? index} className="py-2">
|
||||||
|
<div className="grid grid-cols-[minmax(0,1fr)_72px_72px_80px] items-center gap-2">
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
value={book.title}
|
||||||
|
onChange={(e) =>
|
||||||
|
onChange(updateReviewBook(books, index, { title: e.target.value }))
|
||||||
|
}
|
||||||
|
aria-label={`Title for book ${index + 1}`}
|
||||||
|
className={inputClassName}
|
||||||
|
disabled={isSubmitting}
|
||||||
|
/>
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
inputMode="decimal"
|
||||||
|
value={book.series_position ?? ''}
|
||||||
|
onChange={(e) =>
|
||||||
|
onChange(
|
||||||
|
updateReviewBook(books, index, {
|
||||||
|
series_position: parseSeriesPositionInput(e.target.value),
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
aria-label={`Series position for book ${index + 1}`}
|
||||||
|
className={inputClassName}
|
||||||
|
disabled={isSubmitting}
|
||||||
|
/>
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
inputMode="numeric"
|
||||||
|
value={book.year ?? ''}
|
||||||
|
onChange={(e) => {
|
||||||
|
const parsed = parseSeriesPositionInput(e.target.value);
|
||||||
|
onChange(
|
||||||
|
updateReviewBook(books, index, {
|
||||||
|
year: parsed === null ? null : Math.trunc(parsed),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
}}
|
||||||
|
aria-label={`Year for book ${index + 1}`}
|
||||||
|
className={inputClassName}
|
||||||
|
disabled={isSubmitting}
|
||||||
|
/>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={() => setExpandedFiles(expandedFiles === index ? null : index)}
|
||||||
|
className="hover-surface rounded-md px-2 py-1 text-right text-sm text-zinc-500 transition-colors dark:text-zinc-400"
|
||||||
|
aria-expanded={expandedFiles === index}
|
||||||
|
>
|
||||||
|
{book.files.length} {book.files.length === 1 ? 'file' : 'files'}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
{expandedFiles === index && (
|
||||||
|
<ul className="mt-2 max-h-40 overflow-y-auto rounded-md bg-(--bg-soft) px-3 py-2 font-mono text-xs break-all text-zinc-600 dark:text-zinc-300">
|
||||||
|
{book.files.map((file) => (
|
||||||
|
<li key={file}>{file}</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{plan.ignored.length > 0 && (
|
||||||
|
<div>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={() => setShowIgnored(!showIgnored)}
|
||||||
|
className="text-xs text-zinc-500 underline-offset-2 hover:underline dark:text-zinc-400"
|
||||||
|
aria-expanded={showIgnored}
|
||||||
|
>
|
||||||
|
{plan.ignored.length} {plan.ignored.length === 1 ? 'file' : 'files'} ignored (not a book
|
||||||
|
format)
|
||||||
|
</button>
|
||||||
|
{showIgnored && (
|
||||||
|
<ul className="mt-2 max-h-32 overflow-y-auto rounded-md bg-(--bg-soft) px-3 py-2 font-mono text-xs break-all text-zinc-600 dark:text-zinc-300">
|
||||||
|
{plan.ignored.map((file) => (
|
||||||
|
<li key={file}>{file}</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<div className="flex items-center justify-end gap-3 border-t border-(--border-muted) pt-4">
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={onBack}
|
||||||
|
disabled={isSubmitting}
|
||||||
|
className="hover-surface rounded-lg px-3 py-1.5 text-sm font-medium text-(--text) transition-colors disabled:opacity-50"
|
||||||
|
>
|
||||||
|
← Back
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={() => void onConfirm(singleBook ? null : payloadBooks)}
|
||||||
|
disabled={!canConfirm}
|
||||||
|
className="rounded-lg bg-emerald-600 px-4 py-1.5 text-sm font-medium text-white transition-colors hover:bg-emerald-700 disabled:cursor-not-allowed disabled:opacity-50"
|
||||||
|
>
|
||||||
|
{isSubmitting ? 'Queuing…' : confirmLabel}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
};
|
||||||
@@ -7,6 +7,7 @@ import { useReleaseSearchSession } from '../hooks/releaseModal/useReleaseSearchS
|
|||||||
import { useTabIndicator } from '../hooks/ui/useTabIndicator';
|
import { useTabIndicator } from '../hooks/ui/useTabIndicator';
|
||||||
import { useBodyScrollLock } from '../hooks/useBodyScrollLock';
|
import { useBodyScrollLock } from '../hooks/useBodyScrollLock';
|
||||||
import { useEscapeKey } from '../hooks/useEscapeKey';
|
import { useEscapeKey } from '../hooks/useEscapeKey';
|
||||||
|
import { inspectRelease } from '../services/api';
|
||||||
import type {
|
import type {
|
||||||
Book,
|
Book,
|
||||||
Release,
|
Release,
|
||||||
@@ -18,6 +19,8 @@ import type {
|
|||||||
LeadingCellConfig,
|
LeadingCellConfig,
|
||||||
ContentType,
|
ContentType,
|
||||||
RequestPolicyMode,
|
RequestPolicyMode,
|
||||||
|
PackBook,
|
||||||
|
PackPlan,
|
||||||
} from '../types';
|
} from '../types';
|
||||||
import { isMetadataBook } from '../types';
|
import { isMetadataBook } from '../types';
|
||||||
import { bookSupportsTargets } from '../utils/bookTargetLoader';
|
import { bookSupportsTargets } from '../utils/bookTargetLoader';
|
||||||
@@ -29,7 +32,9 @@ import {
|
|||||||
buildLanguageNormalizer,
|
buildLanguageNormalizer,
|
||||||
} from '../utils/languageFilters';
|
} from '../utils/languageFilters';
|
||||||
import { getNestedValue, toComparableText, toStringValue } from '../utils/objectHelpers';
|
import { getNestedValue, toComparableText, toStringValue } from '../utils/objectHelpers';
|
||||||
|
import { toBookPlanPayload } from '../utils/packReview';
|
||||||
import { getReleaseFormats } from '../utils/releaseFormats';
|
import { getReleaseFormats } from '../utils/releaseFormats';
|
||||||
|
import { buildReleaseDownloadPayload, type ReleaseDownloadOptions } from '../utils/releasePayload';
|
||||||
import {
|
import {
|
||||||
getBookTitleCandidates,
|
getBookTitleCandidates,
|
||||||
getBookAuthorCandidates,
|
getBookAuthorCandidates,
|
||||||
@@ -50,6 +55,7 @@ import { BookTargetDropdown } from './BookTargetDropdown';
|
|||||||
import { Dropdown } from './Dropdown';
|
import { Dropdown } from './Dropdown';
|
||||||
import { DropdownList } from './DropdownList';
|
import { DropdownList } from './DropdownList';
|
||||||
import { LanguageMultiSelect } from './LanguageMultiSelect';
|
import { LanguageMultiSelect } from './LanguageMultiSelect';
|
||||||
|
import { PackReviewPanel } from './PackReviewPanel';
|
||||||
import { ReleaseCell } from './ReleaseCell';
|
import { ReleaseCell } from './ReleaseCell';
|
||||||
|
|
||||||
// Combined mode configuration for the ReleaseModal
|
// Combined mode configuration for the ReleaseModal
|
||||||
@@ -140,7 +146,12 @@ const DEFAULT_COLUMN_CONFIG: ReleaseColumnConfig = {
|
|||||||
interface ReleaseModalProps {
|
interface ReleaseModalProps {
|
||||||
book: Book | null;
|
book: Book | null;
|
||||||
onClose: () => void;
|
onClose: () => void;
|
||||||
onDownload: (book: Book, release: Release, contentType: ContentType) => Promise<void>;
|
onDownload: (
|
||||||
|
book: Book,
|
||||||
|
release: Release,
|
||||||
|
contentType: ContentType,
|
||||||
|
options?: ReleaseDownloadOptions,
|
||||||
|
) => Promise<void>;
|
||||||
onRequestRelease?: (book: Book, release: Release, contentType: ContentType) => Promise<void>;
|
onRequestRelease?: (book: Book, release: Release, contentType: ContentType) => Promise<void>;
|
||||||
onRequestBook?: (book: Book, contentType: ContentType) => Promise<void>;
|
onRequestBook?: (book: Book, contentType: ContentType) => Promise<void>;
|
||||||
getPolicyModeForSource?: (source: string, contentType: ContentType) => RequestPolicyMode;
|
getPolicyModeForSource?: (source: string, contentType: ContentType) => RequestPolicyMode;
|
||||||
@@ -762,6 +773,15 @@ const ReleaseModalSession = ({
|
|||||||
: supportedFormats;
|
: supportedFormats;
|
||||||
const [isRequestingBook, setIsRequestingBook] = useState(false);
|
const [isRequestingBook, setIsRequestingBook] = useState(false);
|
||||||
const [selectedRelease, setSelectedRelease] = useState<Release | null>(null);
|
const [selectedRelease, setSelectedRelease] = useState<Release | null>(null);
|
||||||
|
// Multi-book packs: `multiBook` is the manual header toggle (heuristic split for
|
||||||
|
// releases we can't inspect); `packReview` holds an inspected pack awaiting approval.
|
||||||
|
const [multiBook, setMultiBook] = useState(false);
|
||||||
|
const [packReview, setPackReview] = useState<{
|
||||||
|
release: Release;
|
||||||
|
plan: PackPlan;
|
||||||
|
books: PackBook[];
|
||||||
|
} | null>(null);
|
||||||
|
const [packSubmitting, setPackSubmitting] = useState(false);
|
||||||
const isCombinedMode = combinedMode != null;
|
const isCombinedMode = combinedMode != null;
|
||||||
const combinedPhase = combinedMode?.phase ?? null;
|
const combinedPhase = combinedMode?.phase ?? null;
|
||||||
const combinedStepLabel = combinedMode?.stepLabel ?? '';
|
const combinedStepLabel = combinedMode?.stepLabel ?? '';
|
||||||
@@ -1196,7 +1216,30 @@ const ReleaseModalSession = ({
|
|||||||
|
|
||||||
const mode = getReleaseActionMode(release);
|
const mode = getReleaseActionMode(release);
|
||||||
if (mode === 'download') {
|
if (mode === 'download') {
|
||||||
await onDownload(book, release, contentType);
|
// Look at the release's files before queueing so a whole-series pack can be
|
||||||
|
// reviewed and filed as separate books instead of one mangled item.
|
||||||
|
let inspected = false;
|
||||||
|
let plan: PackPlan | null = null;
|
||||||
|
try {
|
||||||
|
const inspection = await inspectRelease(
|
||||||
|
buildReleaseDownloadPayload(book, release, contentType),
|
||||||
|
);
|
||||||
|
inspected = inspection.inspected;
|
||||||
|
plan = inspection.plan;
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Release inspection failed:', error);
|
||||||
|
}
|
||||||
|
if (inspected && plan?.is_pack) {
|
||||||
|
setPackReview({ release, plan, books: plan.books });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (!inspected && !multiBook) {
|
||||||
|
onShowToast?.(
|
||||||
|
"Couldn't check this release's files before download. If it contains several books, turn on the multi-book pack toggle first.",
|
||||||
|
'info',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
await onDownload(book, release, contentType, multiBook ? { multiBook: true } : {});
|
||||||
handleClose();
|
handleClose();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -1215,9 +1258,32 @@ const ReleaseModalSession = ({
|
|||||||
onRequestRelease,
|
onRequestRelease,
|
||||||
contentType,
|
contentType,
|
||||||
handleClose,
|
handleClose,
|
||||||
|
multiBook,
|
||||||
|
onShowToast,
|
||||||
],
|
],
|
||||||
);
|
);
|
||||||
|
|
||||||
|
const handlePackConfirm = useCallback(
|
||||||
|
async (books: PackBook[] | null): Promise<void> => {
|
||||||
|
if (!book || !packReview) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
setPackSubmitting(true);
|
||||||
|
try {
|
||||||
|
await onDownload(
|
||||||
|
book,
|
||||||
|
packReview.release,
|
||||||
|
contentType,
|
||||||
|
books ? { multiBook: true, bookPlan: toBookPlanPayload(books) } : {},
|
||||||
|
);
|
||||||
|
handleClose();
|
||||||
|
} finally {
|
||||||
|
setPackSubmitting(false);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[book, packReview, onDownload, contentType, handleClose],
|
||||||
|
);
|
||||||
|
|
||||||
const titleId = `release-modal-title-${book.id}`;
|
const titleId = `release-modal-title-${book.id}`;
|
||||||
const providerDisplay =
|
const providerDisplay =
|
||||||
book.provider_display_name ||
|
book.provider_display_name ||
|
||||||
@@ -1720,6 +1786,37 @@ const ReleaseModalSession = ({
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div className="flex items-center gap-3 pr-1 pl-2">
|
<div className="flex items-center gap-3 pr-1 pl-2">
|
||||||
|
{/* Multi-book pack toggle (fallback for releases that can't be inspected) */}
|
||||||
|
{!isCombinedMode && (
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={() => setMultiBook((prev) => !prev)}
|
||||||
|
className={`hover-surface relative rounded-full p-2.5 text-zinc-500 transition-colors dark:text-zinc-400 ${
|
||||||
|
multiBook ? 'text-emerald-600 dark:text-emerald-400' : ''
|
||||||
|
}`}
|
||||||
|
aria-label="Multi-book pack"
|
||||||
|
aria-pressed={multiBook}
|
||||||
|
title="Multi-book pack: file each subfolder (or each file) as a separate book. Only needed when a release can't be inspected before download."
|
||||||
|
>
|
||||||
|
<svg
|
||||||
|
className="h-4 w-4"
|
||||||
|
fill="none"
|
||||||
|
stroke="currentColor"
|
||||||
|
viewBox="0 0 24 24"
|
||||||
|
strokeWidth={1.5}
|
||||||
|
>
|
||||||
|
<path
|
||||||
|
strokeLinecap="round"
|
||||||
|
strokeLinejoin="round"
|
||||||
|
d="M6.429 9.75 2.25 12l4.179 2.25m0-4.5 5.571 3 5.571-3m-11.142 0L2.25 7.5 12 2.25l9.75 5.25-4.179 2.25m0 0L21.75 12l-4.179 2.25m0 0 4.179 2.25L12 21.75 2.25 16.5l4.179-2.25m11.142 0-5.571 3-5.571-3"
|
||||||
|
/>
|
||||||
|
</svg>
|
||||||
|
{multiBook && (
|
||||||
|
<span className="absolute top-1 right-1 h-2 w-2 rounded-full bg-emerald-500" />
|
||||||
|
)}
|
||||||
|
</button>
|
||||||
|
)}
|
||||||
|
|
||||||
{/* Manual query button */}
|
{/* Manual query button */}
|
||||||
<button
|
<button
|
||||||
type="button"
|
type="button"
|
||||||
@@ -2120,6 +2217,19 @@ const ReleaseModalSession = ({
|
|||||||
{/* Release list content */}
|
{/* Release list content */}
|
||||||
<div className="min-h-[200px]">
|
<div className="min-h-[200px]">
|
||||||
{(() => {
|
{(() => {
|
||||||
|
if (packReview) {
|
||||||
|
return (
|
||||||
|
<PackReviewPanel
|
||||||
|
release={packReview.release}
|
||||||
|
plan={packReview.plan}
|
||||||
|
books={packReview.books}
|
||||||
|
onChange={(books) => setPackReview({ ...packReview, books })}
|
||||||
|
onBack={() => setPackReview(null)}
|
||||||
|
onConfirm={handlePackConfirm}
|
||||||
|
isSubmitting={packSubmitting}
|
||||||
|
/>
|
||||||
|
);
|
||||||
|
}
|
||||||
if (sourcesLoading) {
|
if (sourcesLoading) {
|
||||||
return <ReleaseSkeleton />;
|
return <ReleaseSkeleton />;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,6 +12,8 @@ import type {
|
|||||||
RequestSubmissionResult,
|
RequestSubmissionResult,
|
||||||
MetadataProvidersResponse,
|
MetadataProvidersResponse,
|
||||||
MetadataSearchConfig,
|
MetadataSearchConfig,
|
||||||
|
PackBook,
|
||||||
|
InspectReleaseResponse,
|
||||||
} from '../types';
|
} from '../types';
|
||||||
import type {
|
import type {
|
||||||
ActionResult,
|
ActionResult,
|
||||||
@@ -510,6 +512,18 @@ export type DownloadReleasePayload = {
|
|||||||
language?: string; // Release language code, for the {Language} naming variable
|
language?: string; // Release language code, for the {Language} naming variable
|
||||||
search_author?: string;
|
search_author?: string;
|
||||||
search_mode?: 'direct' | 'universal';
|
search_mode?: 'direct' | 'universal';
|
||||||
|
multi_book?: boolean; // Split a multi-book pack into one book per subfolder/file
|
||||||
|
book_plan?: PackBook[]; // The split the user approved before download
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Inspect a release's file list before download (same body as downloadRelease). */
|
||||||
|
export const inspectRelease = async (
|
||||||
|
release: DownloadReleasePayload,
|
||||||
|
): Promise<InspectReleaseResponse> => {
|
||||||
|
return fetchJSON<InspectReleaseResponse>(`${API_BASE}/releases/inspect`, {
|
||||||
|
method: 'POST',
|
||||||
|
body: JSON.stringify(release),
|
||||||
|
});
|
||||||
};
|
};
|
||||||
|
|
||||||
export const downloadRelease = async (
|
export const downloadRelease = async (
|
||||||
|
|||||||
@@ -0,0 +1,53 @@
|
|||||||
|
import { describe, it, expect } from 'vitest';
|
||||||
|
|
||||||
|
import type { PackBook } from '../types';
|
||||||
|
import {
|
||||||
|
describePackPlan,
|
||||||
|
parseSeriesPositionInput,
|
||||||
|
toBookPlanPayload,
|
||||||
|
updateReviewBook,
|
||||||
|
} from '../utils/packReview';
|
||||||
|
|
||||||
|
const books: PackBook[] = [
|
||||||
|
{ title: 'Leviathan Wakes', series_position: 1, year: 2011, files: ['a.m4b'] },
|
||||||
|
{ title: 'Caliban’s War', series_position: 2, year: 2012, files: ['b.m4b', 'b2.m4b'] },
|
||||||
|
];
|
||||||
|
|
||||||
|
describe('packReview.updateReviewBook', () => {
|
||||||
|
it('replaces one book without touching the others', () => {
|
||||||
|
const next = updateReviewBook(books, 1, { title: 'Caliban’s War (Unabridged)' });
|
||||||
|
expect(next[0]).toBe(books[0]);
|
||||||
|
expect(next[1]).toEqual({ ...books[1], title: 'Caliban’s War (Unabridged)' });
|
||||||
|
expect(books[1].title).toBe('Caliban’s War');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('packReview.parseSeriesPositionInput', () => {
|
||||||
|
it('accepts whole and fractional positions', () => {
|
||||||
|
expect(parseSeriesPositionInput('3')).toBe(3);
|
||||||
|
expect(parseSeriesPositionInput('2.5')).toBe(2.5);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('treats blank or junk as no position', () => {
|
||||||
|
expect(parseSeriesPositionInput('')).toBeNull();
|
||||||
|
expect(parseSeriesPositionInput('abc')).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('packReview.toBookPlanPayload', () => {
|
||||||
|
it('trims titles, drops books without a title, and keeps file lists', () => {
|
||||||
|
const edited = updateReviewBook(books, 0, { title: ' ' });
|
||||||
|
expect(toBookPlanPayload(edited)).toEqual([
|
||||||
|
{ title: 'Caliban’s War', series_position: 2, year: 2012, files: ['b.m4b', 'b2.m4b'] },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('packReview.describePackPlan', () => {
|
||||||
|
it('summarises books, files and ignored sidecars', () => {
|
||||||
|
expect(describePackPlan(books, ['a.txt', 'cover.jpg'])).toBe(
|
||||||
|
'2 books · 3 files · 2 files ignored',
|
||||||
|
);
|
||||||
|
expect(describePackPlan([books[0]], [])).toBe('1 book · 1 file');
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
import { describe, it, expect } from 'vitest';
|
||||||
|
|
||||||
|
import type { Book, Release } from '../types';
|
||||||
|
import { buildReleaseDownloadPayload } from '../utils/releasePayload';
|
||||||
|
|
||||||
|
const book: Book = {
|
||||||
|
id: 'hc-1',
|
||||||
|
title: 'Drive',
|
||||||
|
author: 'James S. A. Corey',
|
||||||
|
year: '2012',
|
||||||
|
preview: 'https://img/drive.jpg',
|
||||||
|
series_name: 'The Expanse',
|
||||||
|
series_position: 2.6,
|
||||||
|
subtitle: 'An Expanse Short Story',
|
||||||
|
provider: 'hardcover',
|
||||||
|
provider_id: 'hc-1',
|
||||||
|
source: 'direct_download',
|
||||||
|
};
|
||||||
|
|
||||||
|
const release: Release = {
|
||||||
|
source: 'audiobookbay',
|
||||||
|
source_id: 'abb-1',
|
||||||
|
title: 'James S. A. Corey - The Expanse Complete 2.0',
|
||||||
|
format: 'm4b',
|
||||||
|
language: 'en',
|
||||||
|
download_url: 'https://audiobookbay.lu/abss/expanse/',
|
||||||
|
};
|
||||||
|
|
||||||
|
describe('buildReleaseDownloadPayload', () => {
|
||||||
|
it('describes the searched book and the chosen release', () => {
|
||||||
|
const payload = buildReleaseDownloadPayload(book, release, 'audiobook');
|
||||||
|
expect(payload).toMatchObject({
|
||||||
|
source: 'audiobookbay',
|
||||||
|
source_id: 'abb-1',
|
||||||
|
title: 'Drive',
|
||||||
|
author: 'James S. A. Corey',
|
||||||
|
series_name: 'The Expanse',
|
||||||
|
series_position: 2.6,
|
||||||
|
language: 'en',
|
||||||
|
content_type: 'audiobook',
|
||||||
|
});
|
||||||
|
expect(payload.multi_book).toBeUndefined();
|
||||||
|
expect(payload.book_plan).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('uses the release title and author for manual books', () => {
|
||||||
|
const manual: Book = { ...book, provider: 'manual', title: 'ignored' };
|
||||||
|
const withAuthor = { ...release, extra: { author: 'Release Author' } };
|
||||||
|
const payload = buildReleaseDownloadPayload(manual, withAuthor, 'audiobook');
|
||||||
|
expect(payload.title).toBe(release.title);
|
||||||
|
expect(payload.author).toBe('Release Author');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('flags a manual multi-book pack', () => {
|
||||||
|
const payload = buildReleaseDownloadPayload(book, release, 'audiobook', { multiBook: true });
|
||||||
|
expect(payload.multi_book).toBe(true);
|
||||||
|
expect(payload.book_plan).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('attaches the approved book plan', () => {
|
||||||
|
const plan = [{ title: 'Leviathan Wakes', series_position: 1, year: 2011, files: ['a.m4b'] }];
|
||||||
|
const payload = buildReleaseDownloadPayload(book, release, 'audiobook', {
|
||||||
|
multiBook: true,
|
||||||
|
bookPlan: plan,
|
||||||
|
});
|
||||||
|
expect(payload.multi_book).toBe(true);
|
||||||
|
expect(payload.book_plan).toEqual(plan);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -460,6 +460,27 @@ export interface SourceSearchInfo {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Response from /api/releases endpoint
|
// Response from /api/releases endpoint
|
||||||
|
/** One book split out of a multi-book pack release, files as release-relative paths. */
|
||||||
|
export interface PackBook {
|
||||||
|
title: string;
|
||||||
|
series_position: number | null;
|
||||||
|
year: number | null;
|
||||||
|
files: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PackPlan {
|
||||||
|
is_pack: boolean;
|
||||||
|
books: PackBook[];
|
||||||
|
ignored: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface InspectReleaseResponse {
|
||||||
|
inspected: boolean;
|
||||||
|
reason: string | null;
|
||||||
|
files: { path: string; size: number | null }[];
|
||||||
|
plan: PackPlan | null;
|
||||||
|
}
|
||||||
|
|
||||||
export interface ReleasesResponse {
|
export interface ReleasesResponse {
|
||||||
releases: Release[];
|
releases: Release[];
|
||||||
book: {
|
book: {
|
||||||
|
|||||||
@@ -0,0 +1,39 @@
|
|||||||
|
import type { PackBook } from '../types';
|
||||||
|
|
||||||
|
/** Return a copy of `books` with one entry patched; the input is not mutated. */
|
||||||
|
export function updateReviewBook(
|
||||||
|
books: PackBook[],
|
||||||
|
index: number,
|
||||||
|
patch: Partial<PackBook>,
|
||||||
|
): PackBook[] {
|
||||||
|
return books.map((book, i) => (i === index ? { ...book, ...patch } : book));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Parse a series-position text field: "3" → 3, "2.5" → 2.5, blank/junk → null. */
|
||||||
|
export function parseSeriesPositionInput(value: string): number | null {
|
||||||
|
const trimmed = value.trim();
|
||||||
|
if (!trimmed) return null;
|
||||||
|
const parsed = Number(trimmed);
|
||||||
|
return Number.isFinite(parsed) ? parsed : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The plan sent with the download: trimmed titles, untitled books dropped. */
|
||||||
|
export function toBookPlanPayload(books: PackBook[]): PackBook[] {
|
||||||
|
return books
|
||||||
|
.map((book) => ({ ...book, title: book.title.trim() }))
|
||||||
|
.filter((book) => book.title.length > 0 && book.files.length > 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
function plural(count: number, noun: string): string {
|
||||||
|
return `${count} ${noun}${count === 1 ? '' : 's'}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** "2 books · 3 files · 2 files ignored" */
|
||||||
|
export function describePackPlan(books: PackBook[], ignored: string[]): string {
|
||||||
|
const fileCount = books.reduce((sum, book) => sum + book.files.length, 0);
|
||||||
|
const parts = [plural(books.length, 'book'), plural(fileCount, 'file')];
|
||||||
|
if (ignored.length > 0) {
|
||||||
|
parts.push(`${plural(ignored.length, 'file')} ignored`);
|
||||||
|
}
|
||||||
|
return parts.join(' · ');
|
||||||
|
}
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
import type { DownloadReleasePayload } from '../services/api';
|
||||||
|
import type { Book, ContentType, PackBook, Release } from '../types';
|
||||||
|
|
||||||
|
export interface ReleaseDownloadOptions {
|
||||||
|
/** Ask post-processing to split the release into one book per subfolder/file. */
|
||||||
|
multiBook?: boolean;
|
||||||
|
/** The split the user approved in the pack review panel. */
|
||||||
|
bookPlan?: PackBook[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Build the body for /api/releases/download (and /api/releases/inspect). */
|
||||||
|
export function buildReleaseDownloadPayload(
|
||||||
|
book: Book,
|
||||||
|
release: Release,
|
||||||
|
releaseContentType: ContentType,
|
||||||
|
options: ReleaseDownloadOptions = {},
|
||||||
|
): DownloadReleasePayload {
|
||||||
|
const isManual = book.provider === 'manual';
|
||||||
|
const releasePreview =
|
||||||
|
typeof release.extra?.preview === 'string' ? release.extra.preview : undefined;
|
||||||
|
const releaseAuthor =
|
||||||
|
typeof release.extra?.author === 'string' ? release.extra.author : undefined;
|
||||||
|
|
||||||
|
const payload: DownloadReleasePayload = {
|
||||||
|
source: release.source,
|
||||||
|
source_id: release.source_id,
|
||||||
|
title: isManual ? release.title : book.title,
|
||||||
|
author: isManual ? releaseAuthor || '' : book.author,
|
||||||
|
year: book.year,
|
||||||
|
format: release.format,
|
||||||
|
size: release.size,
|
||||||
|
size_bytes: release.size_bytes,
|
||||||
|
download_url: release.download_url,
|
||||||
|
protocol: release.protocol,
|
||||||
|
indexer: release.indexer,
|
||||||
|
seeders: release.seeders,
|
||||||
|
extra: release.extra,
|
||||||
|
preview: isManual ? releasePreview || undefined : book.preview,
|
||||||
|
content_type: releaseContentType,
|
||||||
|
series_name: book.series_name,
|
||||||
|
series_position: book.series_position,
|
||||||
|
subtitle: book.subtitle,
|
||||||
|
// From the release, never the book: book.language is the provider's
|
||||||
|
// canonical edition, which would mislabel a translated release.
|
||||||
|
language: release.language ?? undefined,
|
||||||
|
};
|
||||||
|
|
||||||
|
if (options.multiBook || options.bookPlan) {
|
||||||
|
payload.multi_book = true;
|
||||||
|
}
|
||||||
|
if (options.bookPlan) {
|
||||||
|
payload.book_plan = options.bookPlan;
|
||||||
|
}
|
||||||
|
return payload;
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
"""AudiobookBay test fixtures."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shelfmark.release_sources.audiobookbay import scraper
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _clear_detail_page_cache():
|
||||||
|
"""Detail pages are cached briefly in production; tests must not share them."""
|
||||||
|
scraper.clear_detail_page_cache()
|
||||||
|
yield
|
||||||
|
scraper.clear_detail_page_cache()
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
"""Tests for reading the torrent file list off an AudiobookBay detail page."""
|
||||||
|
|
||||||
|
from shelfmark.download.postprocess.packs import PackFile
|
||||||
|
from shelfmark.release_sources.audiobookbay import scraper
|
||||||
|
|
||||||
|
# Trimmed from a real detail page (2026-08): the file rows sit between the
|
||||||
|
# "Multifile Torrent" marker and the "Combined File Size" row.
|
||||||
|
MULTIFILE_DETAIL_HTML = """
|
||||||
|
<table>
|
||||||
|
<tr><td>Tracker:</td><td>udp://tracker.torrent.eu.org:451/announce</td></tr>
|
||||||
|
<tr><td>Creation Date:</td><td>Sun, 29 Mar 2026 21:09:39 +0200</td></tr>
|
||||||
|
<tr><td colspan='2'>This is a Multifile Torrent</td></tr>
|
||||||
|
<tr><td colspan='2'>The Expanse 9.0 - Leviathan Falls (2021).m4b 1.05 GBs</td></tr>
|
||||||
|
<tr><td colspan='2'>The Expanse 0.1 - An Expanse Novella - Drive (2012).txt 340 Bytes</td></tr>
|
||||||
|
<tr><td colspan='2'>The Expanse 0.2 - An Expanse Novella - The Churn (2014).m4b 125.72 MBs</td></tr>
|
||||||
|
<tr><td colspan='2'>The Expanse 2.0 - Caliban’s War (2012).m4b 578.97 MBs</td></tr>
|
||||||
|
<tr><td>Combined File Size:</td><td><span style='color:#00f;'>7.87</span> GBs</td></tr>
|
||||||
|
<tr><td>Info Hash:</td><td>e4a5538e26987ee58a43aa629ec2c4f2b2d46526</td></tr>
|
||||||
|
</table>
|
||||||
|
"""
|
||||||
|
|
||||||
|
SINGLE_FILE_DETAIL_HTML = """
|
||||||
|
<table>
|
||||||
|
<tr><td>Creation Date:</td><td>Sun, 29 Mar 2026 21:09:39 +0200</td></tr>
|
||||||
|
<tr><td colspan='2'>Drive.m4b 41.55 MBs</td></tr>
|
||||||
|
<tr><td>Combined File Size:</td><td><span style='color:#00f;'>41.55</span> MBs</td></tr>
|
||||||
|
<tr><td>Info Hash:</td><td>e4a5538e26987ee58a43aa629ec2c4f2b2d46526</td></tr>
|
||||||
|
</table>
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def test_extracts_multifile_rows_with_byte_sizes():
|
||||||
|
files = scraper.extract_file_list(MULTIFILE_DETAIL_HTML)
|
||||||
|
assert files == [
|
||||||
|
PackFile("The Expanse 9.0 - Leviathan Falls (2021).m4b", int(1.05 * 1024**3)),
|
||||||
|
PackFile("The Expanse 0.1 - An Expanse Novella - Drive (2012).txt", 340),
|
||||||
|
PackFile(
|
||||||
|
"The Expanse 0.2 - An Expanse Novella - The Churn (2014).m4b", int(125.72 * 1024**2)
|
||||||
|
),
|
||||||
|
PackFile("The Expanse 2.0 - Caliban’s War (2012).m4b", int(578.97 * 1024**2)),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_single_file_torrent_lists_the_row_before_combined_size():
|
||||||
|
assert scraper.extract_file_list(SINGLE_FILE_DETAIL_HTML) == [
|
||||||
|
PackFile("Drive.m4b", int(41.55 * 1024**2))
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_page_without_file_table_returns_none():
|
||||||
|
assert scraper.extract_file_list("<html><body><p>nothing here</p></body></html>") is None
|
||||||
|
|
||||||
|
|
||||||
|
class TestHandlerListFiles:
|
||||||
|
def test_lists_files_from_detail_page(self):
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from shelfmark.release_sources.audiobookbay.handler import AudiobookBayHandler
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"shelfmark.release_sources.audiobookbay.handler.scraper.fetch_detail_html",
|
||||||
|
return_value=SINGLE_FILE_DETAIL_HTML,
|
||||||
|
) as fetch:
|
||||||
|
files = AudiobookBayHandler().list_files(
|
||||||
|
{"source_id": "abc", "download_url": "https://audiobookbay.lu/abss/drive/"}
|
||||||
|
)
|
||||||
|
assert files == [PackFile("Drive.m4b", int(41.55 * 1024**2))]
|
||||||
|
fetch.assert_called_once_with("https://audiobookbay.lu/abss/drive/", "audiobookbay.lu")
|
||||||
|
|
||||||
|
def test_rejects_detail_url_on_other_host(self):
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from shelfmark.release_sources.audiobookbay.handler import AudiobookBayHandler
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"shelfmark.release_sources.audiobookbay.handler.scraper.fetch_detail_html"
|
||||||
|
) as fetch:
|
||||||
|
files = AudiobookBayHandler().list_files(
|
||||||
|
{"source_id": "abc", "download_url": "https://evil.example/abss/drive/"}
|
||||||
|
)
|
||||||
|
assert files is None
|
||||||
|
fetch.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
def test_extract_magnet_link_and_file_list_share_one_page_fetch():
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
page = MULTIFILE_DETAIL_HTML
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"shelfmark.release_sources.audiobookbay.scraper.downloader.html_get_page",
|
||||||
|
return_value=page,
|
||||||
|
) as get_page,
|
||||||
|
patch("shelfmark.release_sources.audiobookbay.scraper._bootstrap_abb_session"),
|
||||||
|
):
|
||||||
|
scraper.clear_detail_page_cache()
|
||||||
|
url = "https://audiobookbay.lu/abss/expanse/"
|
||||||
|
assert scraper.fetch_detail_html(url, "audiobookbay.lu") == page
|
||||||
|
magnet = scraper.extract_magnet_link(url, "audiobookbay.lu")
|
||||||
|
assert magnet is not None
|
||||||
|
assert "e4a5538e26987ee58a43aa629ec2c4f2b2d46526".upper() in magnet
|
||||||
|
assert get_page.call_count == 1
|
||||||
@@ -0,0 +1,212 @@
|
|||||||
|
"""Multi-book packs are filed one book at a time through the normal pipeline."""
|
||||||
|
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
from threading import Event
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from shelfmark.core.models import DownloadTask, SearchMode
|
||||||
|
from tests.core.test_processing_integration import _build_config, _sync_config
|
||||||
|
|
||||||
|
|
||||||
|
def _run(temp_path: Path, task: DownloadTask, ingest: Path, staging: Path, **config_kwargs):
|
||||||
|
from shelfmark.download.postprocess.router import post_process_download
|
||||||
|
|
||||||
|
statuses: list[tuple[str, str | None]] = []
|
||||||
|
with (
|
||||||
|
patch("shelfmark.core.config.config") as mock_config,
|
||||||
|
patch("shelfmark.config.env.TMP_DIR", staging),
|
||||||
|
):
|
||||||
|
mock_config.get = _build_config(
|
||||||
|
ingest,
|
||||||
|
organization=config_kwargs.pop("organization", "organize"),
|
||||||
|
supported_audiobook_formats=["m4b", "mp3"],
|
||||||
|
audiobook_organize_template=config_kwargs.pop(
|
||||||
|
"audiobook_organize_template", "{Author}/{Title}/{Title}{ - PartNumber}"
|
||||||
|
),
|
||||||
|
**config_kwargs,
|
||||||
|
)
|
||||||
|
mock_config.CUSTOM_SCRIPT = None
|
||||||
|
_sync_config(mock_config, mock_config)
|
||||||
|
result = post_process_download(
|
||||||
|
temp_path, task, Event(), lambda s, m=None: statuses.append((s, m))
|
||||||
|
)
|
||||||
|
return result, statuses
|
||||||
|
|
||||||
|
|
||||||
|
def _nested_pack(root: Path) -> Path:
|
||||||
|
pack = root / "Sun Eater"
|
||||||
|
for folder, name in (
|
||||||
|
("Book 1 - Empire of Silence", "empire.m4b"),
|
||||||
|
("Book 2 - Howling Dark", "howling.m4b"),
|
||||||
|
):
|
||||||
|
(pack / folder).mkdir(parents=True)
|
||||||
|
(pack / folder / name).write_text(name)
|
||||||
|
(pack / "cover.jpg").write_text("img")
|
||||||
|
return pack
|
||||||
|
|
||||||
|
|
||||||
|
def _audiobook_task(**overrides) -> DownloadTask:
|
||||||
|
fields = {
|
||||||
|
"task_id": "pack-1",
|
||||||
|
"source": "direct_download",
|
||||||
|
"title": "Drive",
|
||||||
|
"author": "James S. A. Corey",
|
||||||
|
"content_type": "audiobook",
|
||||||
|
"series_name": "The Expanse",
|
||||||
|
"series_position": 2.6,
|
||||||
|
"search_mode": SearchMode.UNIVERSAL,
|
||||||
|
}
|
||||||
|
fields.update(overrides)
|
||||||
|
return DownloadTask(**fields)
|
||||||
|
|
||||||
|
|
||||||
|
def test_approved_plan_files_each_book_with_its_own_title(tmp_path):
|
||||||
|
staging = tmp_path / "staging"
|
||||||
|
ingest = tmp_path / "ingest"
|
||||||
|
staging.mkdir()
|
||||||
|
ingest.mkdir()
|
||||||
|
pack = staging / "Expanse"
|
||||||
|
pack.mkdir()
|
||||||
|
for name in (
|
||||||
|
"The Expanse 1.0 - Leviathan Wakes (2011).m4b",
|
||||||
|
"The Expanse 2.0 - Caliban's War (2012).m4b",
|
||||||
|
):
|
||||||
|
(pack / name).write_text(name)
|
||||||
|
(pack / "The Expanse 1.0 - Leviathan Wakes (2011).txt").write_text("notes")
|
||||||
|
|
||||||
|
task = _audiobook_task(
|
||||||
|
title="Sun Eater", # the searched book; must not name the pack's books
|
||||||
|
book_plan=[
|
||||||
|
{
|
||||||
|
"title": "Leviathan Wakes (edited)",
|
||||||
|
"series_position": 1.0,
|
||||||
|
"year": 2011,
|
||||||
|
"files": ["The Expanse 1.0 - Leviathan Wakes (2011).m4b"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"title": "Caliban's War",
|
||||||
|
"series_position": 2.0,
|
||||||
|
"year": 2012,
|
||||||
|
"files": ["The Expanse 2.0 - Caliban's War (2012).m4b"],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
result, statuses = _run(pack, task, ingest, staging)
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
author_dir = ingest / "James S. A. Corey"
|
||||||
|
assert sorted(p.name for p in author_dir.iterdir()) == [
|
||||||
|
"Caliban's War",
|
||||||
|
"Leviathan Wakes (edited)",
|
||||||
|
]
|
||||||
|
assert (author_dir / "Leviathan Wakes (edited)" / "Leviathan Wakes (edited).m4b").exists()
|
||||||
|
assert (author_dir / "Caliban's War" / "Caliban's War.m4b").exists()
|
||||||
|
assert statuses[-1] == ("complete", "Complete (2 books, 2 files)")
|
||||||
|
|
||||||
|
|
||||||
|
def test_multi_book_flag_splits_nested_pack_heuristically(tmp_path):
|
||||||
|
staging = tmp_path / "staging"
|
||||||
|
ingest = tmp_path / "ingest"
|
||||||
|
staging.mkdir()
|
||||||
|
ingest.mkdir()
|
||||||
|
pack = _nested_pack(staging)
|
||||||
|
|
||||||
|
result, _ = _run(pack, _audiobook_task(multi_book=True), ingest, staging)
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
author_dir = ingest / "James S. A. Corey"
|
||||||
|
assert (author_dir / "Empire of Silence" / "Empire of Silence.m4b").exists()
|
||||||
|
assert (author_dir / "Howling Dark" / "Howling Dark.m4b").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_pack_book_series_position_does_not_leak_from_searched_book(tmp_path):
|
||||||
|
staging = tmp_path / "staging"
|
||||||
|
ingest = tmp_path / "ingest"
|
||||||
|
staging.mkdir()
|
||||||
|
ingest.mkdir()
|
||||||
|
pack = staging / "Two"
|
||||||
|
for folder in ("Alpha", "Beta"):
|
||||||
|
(pack / folder).mkdir(parents=True)
|
||||||
|
(pack / folder / f"{folder.lower()}.m4b").write_text(folder)
|
||||||
|
|
||||||
|
result, _ = _run(
|
||||||
|
pack,
|
||||||
|
_audiobook_task(multi_book=True),
|
||||||
|
ingest,
|
||||||
|
staging,
|
||||||
|
audiobook_organize_template="{Author}/{SeriesPosition - }{Title}/{Title}",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert sorted(p.name for p in (ingest / "James S. A. Corey").iterdir()) == ["Alpha", "Beta"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_multifile_book_inside_pack_keeps_part_numbers_per_book(tmp_path):
|
||||||
|
staging = tmp_path / "staging"
|
||||||
|
ingest = tmp_path / "ingest"
|
||||||
|
staging.mkdir()
|
||||||
|
ingest.mkdir()
|
||||||
|
pack = staging / "Pack"
|
||||||
|
(pack / "Book 1 - One").mkdir(parents=True)
|
||||||
|
(pack / "Book 2 - Two").mkdir(parents=True)
|
||||||
|
for i in (1, 2, 3):
|
||||||
|
(pack / "Book 1 - One" / f"part{i}.mp3").write_text(str(i))
|
||||||
|
(pack / "Book 2 - Two" / "two.mp3").write_text("t")
|
||||||
|
|
||||||
|
result, statuses = _run(pack, _audiobook_task(multi_book=True), ingest, staging)
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
one = ingest / "James S. A. Corey" / "One"
|
||||||
|
assert sorted(p.name for p in one.iterdir()) == ["One - 01.mp3", "One - 02.mp3", "One - 03.mp3"]
|
||||||
|
assert (ingest / "James S. A. Corey" / "Two" / "Two.mp3").exists()
|
||||||
|
assert statuses[-1] == ("complete", "Complete (2 books, 4 files)")
|
||||||
|
|
||||||
|
|
||||||
|
def test_hardlinked_torrent_pack_leaves_source_tree_intact(tmp_path):
|
||||||
|
downloads = tmp_path / "downloads"
|
||||||
|
ingest = tmp_path / "ingest"
|
||||||
|
downloads.mkdir()
|
||||||
|
ingest.mkdir()
|
||||||
|
pack = _nested_pack(downloads)
|
||||||
|
task = _audiobook_task(source="prowlarr", multi_book=True, original_download_path=str(pack))
|
||||||
|
|
||||||
|
result, _ = _run(pack, task, ingest, tmp_path / "staging", hardlink=True)
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
empire_src = pack / "Book 1 - Empire of Silence" / "empire.m4b"
|
||||||
|
empire_dst = ingest / "James S. A. Corey" / "Empire of Silence" / "Empire of Silence.m4b"
|
||||||
|
assert empire_src.exists()
|
||||||
|
assert empire_dst.exists()
|
||||||
|
assert os.stat(empire_src).st_ino == os.stat(empire_dst).st_ino
|
||||||
|
|
||||||
|
|
||||||
|
def test_without_pack_fields_nested_pack_is_still_one_book(tmp_path):
|
||||||
|
staging = tmp_path / "staging"
|
||||||
|
ingest = tmp_path / "ingest"
|
||||||
|
staging.mkdir()
|
||||||
|
ingest.mkdir()
|
||||||
|
pack = _nested_pack(staging)
|
||||||
|
|
||||||
|
result, _ = _run(pack, _audiobook_task(), ingest, staging)
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
drive = ingest / "James S. A. Corey" / "Drive"
|
||||||
|
assert sorted(p.name for p in drive.iterdir()) == ["Drive - 01.m4b", "Drive - 02.m4b"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_single_group_with_multi_book_flag_uses_searched_title(tmp_path):
|
||||||
|
staging = tmp_path / "staging"
|
||||||
|
ingest = tmp_path / "ingest"
|
||||||
|
staging.mkdir()
|
||||||
|
ingest.mkdir()
|
||||||
|
pack = staging / "Series" / "Book 1 - Solo"
|
||||||
|
pack.mkdir(parents=True)
|
||||||
|
(pack / "solo.m4b").write_text("s")
|
||||||
|
|
||||||
|
result, statuses = _run(pack.parent, _audiobook_task(multi_book=True), ingest, staging)
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert (ingest / "James S. A. Corey" / "Drive" / "Drive.m4b").exists()
|
||||||
|
assert statuses[-1] == ("complete", "Complete")
|
||||||
@@ -0,0 +1,134 @@
|
|||||||
|
"""API tests for POST /api/releases/inspect."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shelfmark.download.postprocess.packs import PackFile
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(scope="module")
|
||||||
|
def main_module():
|
||||||
|
with patch("shelfmark.download.orchestrator.start"):
|
||||||
|
import shelfmark.main as main
|
||||||
|
|
||||||
|
importlib.reload(main)
|
||||||
|
return main
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client(main_module):
|
||||||
|
client = main_module.app.test_client()
|
||||||
|
with client.session_transaction() as sess:
|
||||||
|
sess["user_id"] = "tester"
|
||||||
|
sess["is_admin"] = False
|
||||||
|
return client
|
||||||
|
|
||||||
|
|
||||||
|
class _Handler:
|
||||||
|
def __init__(self, files):
|
||||||
|
self._files = files
|
||||||
|
|
||||||
|
def list_files(self, release_data):
|
||||||
|
if isinstance(self._files, Exception):
|
||||||
|
raise self._files
|
||||||
|
return self._files
|
||||||
|
|
||||||
|
|
||||||
|
def _inspect(client, handler, payload=None):
|
||||||
|
body = {
|
||||||
|
"source": "audiobookbay",
|
||||||
|
"source_id": "abc",
|
||||||
|
"title": "Drive",
|
||||||
|
"content_type": "audiobook",
|
||||||
|
"series_name": "The Expanse",
|
||||||
|
**(payload or {}),
|
||||||
|
}
|
||||||
|
with patch("shelfmark.core.release_inspect_routes.get_handler", return_value=handler):
|
||||||
|
return client.post("/api/releases/inspect", json=body)
|
||||||
|
|
||||||
|
|
||||||
|
def test_pack_release_returns_a_plan(client):
|
||||||
|
files = [
|
||||||
|
PackFile("The Expanse 1.0 - Leviathan Wakes (2011).m4b", 100),
|
||||||
|
PackFile("The Expanse 1.0 - Leviathan Wakes (2011).txt", 1),
|
||||||
|
PackFile("The Expanse 2.0 - Caliban's War (2012).m4b", 100),
|
||||||
|
]
|
||||||
|
resp = _inspect(client, _Handler(files))
|
||||||
|
assert resp.status_code == 200
|
||||||
|
data = resp.get_json()
|
||||||
|
assert data["inspected"] is True
|
||||||
|
assert data["reason"] is None
|
||||||
|
assert data["files"] == [
|
||||||
|
{"path": "The Expanse 1.0 - Leviathan Wakes (2011).m4b", "size": 100},
|
||||||
|
{"path": "The Expanse 1.0 - Leviathan Wakes (2011).txt", "size": 1},
|
||||||
|
{"path": "The Expanse 2.0 - Caliban's War (2012).m4b", "size": 100},
|
||||||
|
]
|
||||||
|
assert data["plan"]["is_pack"] is True
|
||||||
|
assert data["plan"]["ignored"] == ["The Expanse 1.0 - Leviathan Wakes (2011).txt"]
|
||||||
|
assert data["plan"]["books"] == [
|
||||||
|
{
|
||||||
|
"title": "Leviathan Wakes",
|
||||||
|
"series_position": 1.0,
|
||||||
|
"year": 2011,
|
||||||
|
"files": ["The Expanse 1.0 - Leviathan Wakes (2011).m4b"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"title": "Caliban's War",
|
||||||
|
"series_position": 2.0,
|
||||||
|
"year": 2012,
|
||||||
|
"files": ["The Expanse 2.0 - Caliban's War (2012).m4b"],
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_single_book_release_is_not_a_pack(client):
|
||||||
|
resp = _inspect(client, _Handler([PackFile("Drive.m4b", 5)]))
|
||||||
|
data = resp.get_json()
|
||||||
|
assert data["inspected"] is True
|
||||||
|
assert data["plan"]["is_pack"] is False
|
||||||
|
assert len(data["plan"]["books"]) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_handler_without_file_list_reports_not_inspected(client):
|
||||||
|
resp = _inspect(client, _Handler(None))
|
||||||
|
assert resp.status_code == 200
|
||||||
|
data = resp.get_json()
|
||||||
|
assert data["inspected"] is False
|
||||||
|
assert data["reason"]
|
||||||
|
assert data["plan"] is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_handler_failure_reports_not_inspected_without_500(client):
|
||||||
|
resp = _inspect(client, _Handler(RuntimeError("boom")))
|
||||||
|
assert resp.status_code == 200
|
||||||
|
data = resp.get_json()
|
||||||
|
assert data["inspected"] is False
|
||||||
|
assert "boom" in data["reason"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_source_is_a_client_error(client):
|
||||||
|
with patch(
|
||||||
|
"shelfmark.core.release_inspect_routes.get_handler", side_effect=ValueError("no source")
|
||||||
|
):
|
||||||
|
resp = client.post(
|
||||||
|
"/api/releases/inspect", json={"source": "nope", "source_id": "x", "title": "t"}
|
||||||
|
)
|
||||||
|
assert resp.status_code == 400
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_source_id_is_a_client_error(client):
|
||||||
|
resp = client.post("/api/releases/inspect", json={"source": "audiobookbay"})
|
||||||
|
assert resp.status_code == 400
|
||||||
|
|
||||||
|
|
||||||
|
def test_requires_login(main_module):
|
||||||
|
anonymous = main_module.app.test_client()
|
||||||
|
with patch.object(main_module, "load_active_auth_mode", return_value="builtin"):
|
||||||
|
resp = anonymous.post(
|
||||||
|
"/api/releases/inspect", json={"source": "audiobookbay", "source_id": "a"}
|
||||||
|
)
|
||||||
|
assert resp.status_code == 401
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
"""Pack fields (multi_book / book_plan) survive queueing and restart-safe retry."""
|
||||||
|
|
||||||
|
from shelfmark.core.models import DownloadTask
|
||||||
|
from shelfmark.download import orchestrator
|
||||||
|
|
||||||
|
PLAN = [
|
||||||
|
{"title": "Leviathan Wakes", "series_position": 1.0, "year": 2011, "files": ["a.m4b"]},
|
||||||
|
{"title": "Caliban's War", "series_position": 2.0, "year": 2012, "files": ["b.m4b"]},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_task_defaults_to_single_book():
|
||||||
|
task = DownloadTask(task_id="t", source="prowlarr", title="T")
|
||||||
|
assert task.multi_book is False
|
||||||
|
assert task.book_plan is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_retry_payload_round_trips_pack_fields():
|
||||||
|
task = DownloadTask(task_id="t", source="prowlarr", title="T", multi_book=True, book_plan=PLAN)
|
||||||
|
payload = orchestrator.serialize_task_for_retry(task)
|
||||||
|
restored = orchestrator._restore_task_from_retry_payload(payload)
|
||||||
|
assert restored is not None
|
||||||
|
assert restored.multi_book is True
|
||||||
|
assert restored.book_plan == PLAN
|
||||||
|
|
||||||
|
|
||||||
|
def test_retry_payload_drops_malformed_plan():
|
||||||
|
payload = orchestrator.serialize_task_for_retry(
|
||||||
|
DownloadTask(task_id="t", source="prowlarr", title="T")
|
||||||
|
)
|
||||||
|
payload["book_plan"] = "not a list"
|
||||||
|
restored = orchestrator._restore_task_from_retry_payload(payload)
|
||||||
|
assert restored is not None
|
||||||
|
assert restored.book_plan is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_queue_release_reads_pack_fields(monkeypatch):
|
||||||
|
captured: dict[str, DownloadTask] = {}
|
||||||
|
|
||||||
|
def fake_add(task: DownloadTask) -> bool:
|
||||||
|
captured["task"] = task
|
||||||
|
return True
|
||||||
|
|
||||||
|
monkeypatch.setattr(orchestrator.config, "get", lambda _key, default=None, **_kw: default)
|
||||||
|
monkeypatch.setattr(orchestrator, "_source_unavailable_message", lambda _source: None)
|
||||||
|
monkeypatch.setattr(orchestrator.book_queue, "add", fake_add)
|
||||||
|
monkeypatch.setattr(orchestrator, "ws_manager", None)
|
||||||
|
|
||||||
|
ok, error = orchestrator.queue_release(
|
||||||
|
{
|
||||||
|
"source": "direct_download",
|
||||||
|
"source_id": "abc",
|
||||||
|
"title": "The Expanse",
|
||||||
|
"content_type": "audiobook",
|
||||||
|
"multi_book": True,
|
||||||
|
"book_plan": PLAN,
|
||||||
|
},
|
||||||
|
0,
|
||||||
|
)
|
||||||
|
assert ok, error
|
||||||
|
assert captured["task"].multi_book is True
|
||||||
|
assert captured["task"].book_plan == PLAN
|
||||||
@@ -0,0 +1,282 @@
|
|||||||
|
"""Tests for multi-book pack planning (shelfmark.download.postprocess.packs)."""
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shelfmark.download.postprocess.packs import (
|
||||||
|
PackBook,
|
||||||
|
PackFile,
|
||||||
|
group_files_into_books,
|
||||||
|
match_plan_to_files,
|
||||||
|
parse_pack_book_name,
|
||||||
|
plan_pack,
|
||||||
|
)
|
||||||
|
|
||||||
|
AUDIO = {"m4b", "mp3"}
|
||||||
|
|
||||||
|
|
||||||
|
class TestParsePackBookName:
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("name", "expected"),
|
||||||
|
[
|
||||||
|
("Book 3 - Howling Dark", ("Howling Dark", 3.0, None)),
|
||||||
|
("Book 03: Howling Dark", ("Howling Dark", 3.0, None)),
|
||||||
|
("03 - Empire of Silence", ("Empire of Silence", 3.0, None)),
|
||||||
|
("2.5 - Interlude", ("Interlude", 2.5, None)),
|
||||||
|
("[03] Empire of Silence", ("Empire of Silence", 3.0, None)),
|
||||||
|
("#3 Empire of Silence", ("Empire of Silence", 3.0, None)),
|
||||||
|
("3. Empire of Silence", ("Empire of Silence", 3.0, None)),
|
||||||
|
("Empire of Silence", ("Empire of Silence", None, None)),
|
||||||
|
("Empire of Silence (2018)", ("Empire of Silence", None, 2018)),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_strips_series_markers(self, name, expected):
|
||||||
|
assert parse_pack_book_name(name, series_name=None) == expected
|
||||||
|
|
||||||
|
def test_strips_leading_series_name_and_trailing_year(self):
|
||||||
|
assert parse_pack_book_name(
|
||||||
|
"The Expanse 1.0 - Leviathan Wakes (2011)", series_name="The Expanse"
|
||||||
|
) == ("Leviathan Wakes", 1.0, 2011)
|
||||||
|
|
||||||
|
def test_series_name_match_is_case_insensitive(self):
|
||||||
|
assert parse_pack_book_name(
|
||||||
|
"the expanse 2.5 - Gods of Risk", series_name="The Expanse"
|
||||||
|
) == (
|
||||||
|
"Gods of Risk",
|
||||||
|
2.5,
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"name",
|
||||||
|
[
|
||||||
|
"The Expanse 0.2 - An Expanse Novella - The Churn (2014)",
|
||||||
|
"The Expanse 0.2 - The Expanse Novella - The Churn (2014)",
|
||||||
|
"The Expanse 0.2 - An Expanse Short Story - The Churn (2014)",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_strips_series_novella_label(self, name):
|
||||||
|
assert parse_pack_book_name(name, series_name="The Expanse") == ("The Churn", 0.2, 2014)
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("name", "expected"),
|
||||||
|
[
|
||||||
|
("Uncrowned Cradle, Book 7", ("Uncrowned", 7.0, None)),
|
||||||
|
("Reaper Cradle, Volume 10", ("Reaper", 10.0, None)),
|
||||||
|
("Soulsmith Cradle, Book 2", ("Soulsmith", 2.0, None)),
|
||||||
|
("Wintersteel - Cradle Book 8", ("Wintersteel", 8.0, None)),
|
||||||
|
("Wintersteel (Cradle, Book 8)", ("Wintersteel", 8.0, None)),
|
||||||
|
# A trailing bare number is a chapter/part, never a series position.
|
||||||
|
("Unsouled - 02", ("Unsouled - 02", None, None)),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_trailing_series_marker(self, name, expected):
|
||||||
|
assert parse_pack_book_name(name, series_name="Cradle") == expected
|
||||||
|
|
||||||
|
def test_trailing_series_name_only_stripped_with_a_marker(self):
|
||||||
|
# "Stories from Cradle" is the title; nothing marks a position, so keep it.
|
||||||
|
assert parse_pack_book_name("Threshold: Stories from Cradle", series_name="Cradle") == (
|
||||||
|
"Threshold: Stories from Cradle",
|
||||||
|
None,
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("name", "expected"),
|
||||||
|
[
|
||||||
|
# AudiobookBay renders "<folder> <file>" as one flat string.
|
||||||
|
(
|
||||||
|
"Will Wight - Unsouled Cradle, Book 1 Will Wight - Unsouled Cradle, Book 1",
|
||||||
|
("Unsouled", 1.0, None),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"Will Wight - Skysworn Cradle, Book 4 Skysworn Cradle, Book 4",
|
||||||
|
("Skysworn", 4.0, None),
|
||||||
|
),
|
||||||
|
("Will Wight - Bloodline Cradle, Book 9", ("Bloodline", 9.0, None)),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_strips_author_prefix_and_glued_folder_name(self, name, expected):
|
||||||
|
assert (
|
||||||
|
parse_pack_book_name(name, series_name="Cradle", author_name="Will Wight") == expected
|
||||||
|
)
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("name", "expected"),
|
||||||
|
[
|
||||||
|
("Gods of Risk 2.5 - Gods of Risk", ("Gods of Risk", 2.5, None)),
|
||||||
|
("Cibola Burn 4 - Cibola Burn (2014)", ("Cibola Burn", 4.0, 2014)),
|
||||||
|
("cibola burn 4 - Cibola Burn", ("Cibola Burn", 4.0, None)),
|
||||||
|
# Different text on each side is a real "Series N - Title" name, not a repeat.
|
||||||
|
("Sun Eater 2 - Howling Dark", ("Sun Eater 2 - Howling Dark", None, None)),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_collapses_title_repeated_around_the_position(self, name, expected):
|
||||||
|
assert parse_pack_book_name(name, series_name=None) == expected
|
||||||
|
|
||||||
|
def test_bare_numeric_title_is_left_alone(self):
|
||||||
|
assert parse_pack_book_name("1984", series_name=None) == ("1984", None, None)
|
||||||
|
|
||||||
|
def test_marker_that_would_leave_nothing_is_left_alone(self):
|
||||||
|
assert parse_pack_book_name("Book 3", series_name=None) == ("Book 3", None, None)
|
||||||
|
|
||||||
|
|
||||||
|
class TestPlanPack:
|
||||||
|
def test_nested_subfolders_become_separate_books(self):
|
||||||
|
files = [
|
||||||
|
PackFile("Sun Eater/Book 1 - Empire of Silence/Empire of Silence.m4b", 10),
|
||||||
|
PackFile("Sun Eater/Book 2 - Howling Dark/Howling Dark.m4b", 20),
|
||||||
|
PackFile("Sun Eater/cover.jpg", 1),
|
||||||
|
]
|
||||||
|
plan = plan_pack(files, supported_extensions=AUDIO, series_name=None)
|
||||||
|
assert plan.is_pack
|
||||||
|
assert [b.title for b in plan.books] == ["Empire of Silence", "Howling Dark"]
|
||||||
|
assert [b.series_position for b in plan.books] == [1.0, 2.0]
|
||||||
|
assert plan.books[0].files == ["Sun Eater/Book 1 - Empire of Silence/Empire of Silence.m4b"]
|
||||||
|
assert plan.ignored == ["Sun Eater/cover.jpg"]
|
||||||
|
|
||||||
|
def test_flat_pack_becomes_one_book_per_file_and_ignores_sidecars(self):
|
||||||
|
files = [
|
||||||
|
PackFile("The Expanse 1.0 - Leviathan Wakes (2011).m4b", 100),
|
||||||
|
PackFile("The Expanse 1.0 - Leviathan Wakes (2011).txt", 1),
|
||||||
|
PackFile("The Expanse 2.0 - Caliban's War (2012).m4b", 100),
|
||||||
|
]
|
||||||
|
plan = plan_pack(files, supported_extensions=AUDIO, series_name="The Expanse")
|
||||||
|
assert plan.is_pack
|
||||||
|
assert [(b.title, b.series_position, b.year) for b in plan.books] == [
|
||||||
|
("Leviathan Wakes", 1.0, 2011),
|
||||||
|
("Caliban's War", 2.0, 2012),
|
||||||
|
]
|
||||||
|
assert plan.ignored == ["The Expanse 1.0 - Leviathan Wakes (2011).txt"]
|
||||||
|
|
||||||
|
def test_audiobookbay_flat_list_with_trailing_markers_is_a_pack(self):
|
||||||
|
# ABB's file table has no folder separators: "<folder> <file> <size>".
|
||||||
|
files = [
|
||||||
|
PackFile(
|
||||||
|
"Will Wight - Unsouled Cradle, Book 1 Will Wight - Unsouled Cradle, Book 1.sfv", 1
|
||||||
|
),
|
||||||
|
PackFile(
|
||||||
|
"Will Wight - Unsouled Cradle, Book 1 Will Wight - Unsouled Cradle, Book 1.m4a", 9
|
||||||
|
),
|
||||||
|
PackFile("Will Wight - Skysworn Cradle, Book 4 Skysworn Cradle, Book 4.m4b", 9),
|
||||||
|
PackFile("Uncrowned Cradle, Book 7.m4b", 9),
|
||||||
|
PackFile(
|
||||||
|
"Will Wight - Reaper Cradle, Volume 10 Will Wight - Reaper Cradle, Volume 10.m4b", 9
|
||||||
|
),
|
||||||
|
]
|
||||||
|
plan = plan_pack(
|
||||||
|
files,
|
||||||
|
supported_extensions={"m4a", "m4b"},
|
||||||
|
series_name="Cradle",
|
||||||
|
author_name="Will Wight",
|
||||||
|
)
|
||||||
|
assert plan.is_pack
|
||||||
|
assert [(b.title, b.series_position) for b in plan.books] == [
|
||||||
|
("Unsouled", 1.0),
|
||||||
|
("Skysworn", 4.0),
|
||||||
|
("Uncrowned", 7.0),
|
||||||
|
("Reaper", 10.0),
|
||||||
|
]
|
||||||
|
|
||||||
|
def test_deeper_nesting_collapses_onto_book_folder(self):
|
||||||
|
files = [
|
||||||
|
PackFile("Book 1/CD1/01.mp3"),
|
||||||
|
PackFile("Book 1/CD2/01.mp3"),
|
||||||
|
PackFile("Book 2/01.mp3"),
|
||||||
|
]
|
||||||
|
plan = plan_pack(files, supported_extensions=AUDIO, series_name=None)
|
||||||
|
assert [b.files for b in plan.books] == [
|
||||||
|
["Book 1/CD1/01.mp3", "Book 1/CD2/01.mp3"],
|
||||||
|
["Book 2/01.mp3"],
|
||||||
|
]
|
||||||
|
|
||||||
|
def test_single_wrapping_folder_is_not_a_book_boundary(self):
|
||||||
|
# A torrent named "Series" containing one multi-part book is a single book.
|
||||||
|
files = [PackFile("Series/Book 1/01.mp3"), PackFile("Series/Book 1/02.mp3")]
|
||||||
|
plan = plan_pack(files, supported_extensions=AUDIO, series_name=None)
|
||||||
|
assert not plan.is_pack
|
||||||
|
assert len(plan.books) == 1
|
||||||
|
|
||||||
|
def test_root_files_and_subfolders_coexist(self):
|
||||||
|
files = [PackFile("Novella.m4b"), PackFile("Book 1/a.m4b"), PackFile("Book 1/b.m4b")]
|
||||||
|
plan = plan_pack(files, supported_extensions=AUDIO, series_name=None)
|
||||||
|
assert [b.files for b in plan.books] == [["Novella.m4b"], ["Book 1/a.m4b", "Book 1/b.m4b"]]
|
||||||
|
|
||||||
|
def test_single_file_is_not_a_pack(self):
|
||||||
|
plan = plan_pack([PackFile("Book.m4b")], supported_extensions=AUDIO, series_name=None)
|
||||||
|
assert not plan.is_pack
|
||||||
|
assert plan.books[0].title == "Book"
|
||||||
|
|
||||||
|
def test_empty_input(self):
|
||||||
|
plan = plan_pack([], supported_extensions=AUDIO, series_name=None)
|
||||||
|
assert plan.books == []
|
||||||
|
assert not plan.is_pack
|
||||||
|
|
||||||
|
|
||||||
|
class TestGroupFilesIntoBooks:
|
||||||
|
def test_groups_on_disk_files_by_top_level_folder(self, tmp_path: Path):
|
||||||
|
a = tmp_path / "Book 1 - A" / "a.m4b"
|
||||||
|
b = tmp_path / "Book 2 - B" / "b.m4b"
|
||||||
|
for f in (a, b):
|
||||||
|
f.parent.mkdir(parents=True)
|
||||||
|
f.write_bytes(b"x")
|
||||||
|
groups = group_files_into_books([a, b], series_name=None)
|
||||||
|
assert [(g.title, g.series_position, g.files) for g in groups] == [
|
||||||
|
("A", 1.0, [a]),
|
||||||
|
("B", 2.0, [b]),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class TestMatchPlanToFiles:
|
||||||
|
def test_matches_by_relative_path_then_basename(self, tmp_path: Path):
|
||||||
|
root = tmp_path / "staging" / "Sun Eater"
|
||||||
|
a = root / "Book 1 - A" / "a.m4b"
|
||||||
|
b = root / "Book 2 - B" / "b.m4b"
|
||||||
|
for f in (a, b):
|
||||||
|
f.parent.mkdir(parents=True)
|
||||||
|
f.write_bytes(b"x")
|
||||||
|
plan = [
|
||||||
|
PackBook(title="Alpha", series_position=1.0, year=2001, files=["Book 1 - A/a.m4b"]),
|
||||||
|
PackBook(title="Beta", series_position=2.0, year=None, files=["b.m4b"]),
|
||||||
|
]
|
||||||
|
groups = match_plan_to_files(plan, [a, b])
|
||||||
|
assert [(g.title, g.series_position, g.year, g.files) for g in groups] == [
|
||||||
|
("Alpha", 1.0, 2001, [a]),
|
||||||
|
("Beta", 2.0, None, [b]),
|
||||||
|
]
|
||||||
|
|
||||||
|
def test_matches_glued_plan_path_by_basename_suffix(self, tmp_path: Path):
|
||||||
|
# The plan came from ABB's "<folder> <file>" strings; on disk the file sits in a folder.
|
||||||
|
root = tmp_path / "Cradle - Will Wight Books 1-10"
|
||||||
|
a = root / "Will Wight - Skysworn Cradle, Book 4" / "Skysworn Cradle, Book 4.m4b"
|
||||||
|
b = root / "Uncrowned Cradle, Book 7.m4b"
|
||||||
|
for f in (a, b):
|
||||||
|
f.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
f.write_bytes(b"x")
|
||||||
|
plan = [
|
||||||
|
PackBook(
|
||||||
|
title="Skysworn",
|
||||||
|
series_position=4.0,
|
||||||
|
year=None,
|
||||||
|
files=["Will Wight - Skysworn Cradle, Book 4 Skysworn Cradle, Book 4.m4b"],
|
||||||
|
),
|
||||||
|
PackBook(
|
||||||
|
title="Uncrowned",
|
||||||
|
series_position=7.0,
|
||||||
|
year=None,
|
||||||
|
files=["Uncrowned Cradle, Book 7.m4b"],
|
||||||
|
),
|
||||||
|
]
|
||||||
|
groups = match_plan_to_files(plan, [a, b])
|
||||||
|
assert [(g.title, g.files) for g in groups] == [("Skysworn", [a]), ("Uncrowned", [b])]
|
||||||
|
|
||||||
|
def test_unmatched_files_fall_back_to_heuristic_groups(self, tmp_path: Path):
|
||||||
|
a = tmp_path / "Book 1 - A" / "a.m4b"
|
||||||
|
c = tmp_path / "Book 3 - C" / "c.m4b"
|
||||||
|
for f in (a, c):
|
||||||
|
f.parent.mkdir(parents=True)
|
||||||
|
f.write_bytes(b"x")
|
||||||
|
plan = [PackBook(title="Alpha", series_position=1.0, year=None, files=["Book 1 - A/a.m4b"])]
|
||||||
|
groups = match_plan_to_files(plan, [a, c])
|
||||||
|
assert [(g.title, g.files) for g in groups] == [("Alpha", [a]), ("C", [c])]
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
"""Tests for listing the files inside a .torrent without downloading it."""
|
||||||
|
|
||||||
|
from shelfmark.download.clients.torrent_utils import (
|
||||||
|
bencode_encode,
|
||||||
|
extract_file_list_from_torrent,
|
||||||
|
)
|
||||||
|
from shelfmark.download.postprocess.packs import PackFile
|
||||||
|
|
||||||
|
|
||||||
|
def _torrent(info: dict) -> bytes:
|
||||||
|
return bencode_encode({b"announce": b"http://t/announce", b"info": info})
|
||||||
|
|
||||||
|
|
||||||
|
def test_multi_file_torrent_lists_release_relative_paths():
|
||||||
|
data = _torrent(
|
||||||
|
{
|
||||||
|
b"name": b"Sun Eater",
|
||||||
|
b"piece length": 16384,
|
||||||
|
b"pieces": b"x" * 20,
|
||||||
|
b"files": [
|
||||||
|
{b"length": 10, b"path": [b"Book 1 - Empire of Silence", b"empire.m4b"]},
|
||||||
|
{b"length": 20, b"path": [b"Book 2 - Howling Dark", b"howling.m4b"]},
|
||||||
|
{b"length": 1, b"path": [b"cover.jpg"]},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
assert extract_file_list_from_torrent(data) == [
|
||||||
|
PackFile("Sun Eater/Book 1 - Empire of Silence/empire.m4b", 10),
|
||||||
|
PackFile("Sun Eater/Book 2 - Howling Dark/howling.m4b", 20),
|
||||||
|
PackFile("Sun Eater/cover.jpg", 1),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_single_file_torrent_lists_its_one_file():
|
||||||
|
data = _torrent(
|
||||||
|
{b"name": b"Book.m4b", b"length": 42, b"piece length": 16384, b"pieces": b"x" * 20}
|
||||||
|
)
|
||||||
|
assert extract_file_list_from_torrent(data) == [PackFile("Book.m4b", 42)]
|
||||||
|
|
||||||
|
|
||||||
|
def test_unparseable_data_returns_none():
|
||||||
|
assert extract_file_list_from_torrent(b"not a torrent") is None
|
||||||
|
|
||||||
|
|
||||||
|
class TestProwlarrHandlerListFiles:
|
||||||
|
def _handler(self):
|
||||||
|
from shelfmark.release_sources.prowlarr.handler import ProwlarrHandler
|
||||||
|
|
||||||
|
return ProwlarrHandler()
|
||||||
|
|
||||||
|
def test_lists_files_from_torrent_url(self):
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from shelfmark.download.clients.torrent_utils import TorrentInfo
|
||||||
|
|
||||||
|
data = _torrent(
|
||||||
|
{b"name": b"Book.m4b", b"length": 42, b"piece length": 16384, b"pieces": b"x" * 20}
|
||||||
|
)
|
||||||
|
release = {
|
||||||
|
"protocol": "torrent",
|
||||||
|
"downloadUrl": "http://prowlarr/dl.torrent",
|
||||||
|
"magnetUrl": "magnet:?xt=urn:btih:abc",
|
||||||
|
"infoHash": "abc",
|
||||||
|
}
|
||||||
|
with (
|
||||||
|
patch("shelfmark.release_sources.prowlarr.handler.get_release", return_value=release),
|
||||||
|
patch(
|
||||||
|
"shelfmark.release_sources.prowlarr.handler.extract_torrent_info",
|
||||||
|
return_value=TorrentInfo(info_hash="abc", torrent_data=data, is_magnet=False),
|
||||||
|
) as extract,
|
||||||
|
):
|
||||||
|
files = self._handler().list_files({"source_id": "rel-1"})
|
||||||
|
assert files == [PackFile("Book.m4b", 42)]
|
||||||
|
extract.assert_called_once_with("http://prowlarr/dl.torrent", expected_hash="abc")
|
||||||
|
|
||||||
|
def test_magnet_only_release_cannot_be_listed(self):
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
release = {"protocol": "torrent", "magnetUrl": "magnet:?xt=urn:btih:abc"}
|
||||||
|
with (
|
||||||
|
patch("shelfmark.release_sources.prowlarr.handler.get_release", return_value=release),
|
||||||
|
patch("shelfmark.release_sources.prowlarr.handler.extract_torrent_info") as extract,
|
||||||
|
):
|
||||||
|
assert self._handler().list_files({"source_id": "rel-1"}) is None
|
||||||
|
extract.assert_not_called()
|
||||||
|
|
||||||
|
def test_usenet_release_cannot_be_listed(self):
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
release = {"protocol": "usenet", "downloadUrl": "http://prowlarr/dl.nzb"}
|
||||||
|
with patch("shelfmark.release_sources.prowlarr.handler.get_release", return_value=release):
|
||||||
|
assert self._handler().list_files({"source_id": "rel-1"}) is None
|
||||||
|
|
||||||
|
def test_unknown_release_cannot_be_listed(self):
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
with patch("shelfmark.release_sources.prowlarr.handler.get_release", return_value=None):
|
||||||
|
assert self._handler().list_files({"source_id": "missing"}) is None
|
||||||
Reference in New Issue
Block a user