mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-11-14 20:28:27 +00:00
2007384058
## What Adds [Webshare.cz](https://webshare.cz), the Czech file host, as a release source for ebooks and audiobooks. It follows the Libgen source: disabled by default, with a settings tab for credentials and a **Test Login** button that reports whether the account is VIP. ## Why Webshare is one of the main places Czech and Slovak ebooks and audiobooks are shared, and none of the existing sources reach it. Together with the Databáze knih metadata provider (#1479), this lets Universal mode find Czech and Slovak titles end to end. ## How - **API client** (`webshare/api.py`): Webshare's XML API (`POST /api/<endpoint>/`). Login is the salted exchange the API documents: fetch the salt, send `SHA1(MD5_CRYPT(password, salt))` plus an MD5 digest. Python dropped the `crypt` module in 3.13, so md5-crypt is a small pure-Python port, checked against `openssl passwd -1`. The session token is cached per account and refreshed once when an authenticated call fails. File-level errors (`FILE_LINK_FATAL_*`) are not retried. XML is parsed with `defusedxml`. - **Search** (`webshare/source.py`): Webshare matches file names only, so the file name becomes the release title and its extension the format. Queries are title + author, then title only, or the manual query. Results are filtered to the configured `SUPPORTED_FORMATS` / `SUPPORTED_AUDIOBOOK_FORMATS`, plus `zip`/`rar` for audiobooks, since they are usually uploaded as archives. Password-protected files are skipped. The language is guessed from tags in the name (`CZ`, `SK`, `ENG`, ...), and the vote balance is a sortable column. API failures raise `SourceUnavailableError`. - **Download** (`webshare/handler.py`): resolves a time-limited link per attempt with `file_link`, then streams it into `TMP_DIR` with progress and cancel support. Large audiobook archives are not buffered in memory. - **Settings**: `WEBSHARE_ENABLED`, `WEBSHARE_USERNAME`, `WEBSHARE_PASSWORD` (a `PasswordField`, already covered by the `_PASSWORD` redaction in `genDebug.sh`), and `WEBSHARE_MAX_RESULTS`. The `docs/environment-variables.md` section was added. ## Testing - `tests/webshare/`: 31 offline tests. They cover md5-crypt vectors, password hashing, XML parsing, token reuse and re-login, non-retried file errors, availability gating, format filtering, query fallback, release mapping, language guessing, and the handler's streaming, small-file rejection, link errors and cancel. - `ruff check` and `ruff format --check` are clean. The rest of the unit suite fails the same way with and without this change: Windows-only tests on my machine. - Verified live against webshare.cz: search parsing and the salt endpoint's error path. - Still a draft: I haven't yet verified login and download with a real account. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: CaliBrain <calibrain@l4n.xyz>
88 lines
3.1 KiB
Python
88 lines
3.1 KiB
Python
"""Tests for WebshareHandler: link resolution, streaming, cleanup on failure."""
|
|
|
|
import threading
|
|
from unittest.mock import MagicMock, patch
|
|
|
|
from shelfmark.core.models import DownloadTask
|
|
from shelfmark.release_sources.webshare import handler as webshare_handler
|
|
from shelfmark.release_sources.webshare.api import WebshareError
|
|
from shelfmark.release_sources.webshare.handler import WebshareHandler
|
|
|
|
|
|
def _task(task_id="webshare:abc123", fmt="epub"):
|
|
return DownloadTask(task_id=task_id, source="webshare", title="Krysar", format=fmt)
|
|
|
|
|
|
def _http_response(body: bytes):
|
|
response = MagicMock()
|
|
response.__enter__.return_value = response
|
|
response.__exit__.return_value = False
|
|
response.headers = {"Content-Length": str(len(body))}
|
|
response.raise_for_status.return_value = None
|
|
response.iter_content.return_value = [body[i : i + 4096] for i in range(0, len(body), 4096)]
|
|
return response
|
|
|
|
|
|
def _run(tmp_path, *, file_link, body=b"x" * 20000, cancel=None, task=None):
|
|
status = MagicMock()
|
|
progress = MagicMock()
|
|
cancel = cancel or threading.Event()
|
|
with (
|
|
patch.object(webshare_handler, "TMP_DIR", tmp_path),
|
|
patch.object(
|
|
webshare_handler.config,
|
|
"get",
|
|
side_effect=lambda k, d=None: "none" if k == "FILE_ORGANIZATION" else d,
|
|
),
|
|
patch.object(webshare_handler, "get_credentials", return_value=("user", "pw")),
|
|
patch.object(webshare_handler.WebshareClient, "file_link", side_effect=file_link),
|
|
patch.object(webshare_handler.requests, "get", return_value=_http_response(body)),
|
|
):
|
|
result = WebshareHandler().download(task or _task(), cancel, progress, status)
|
|
return result, status, progress
|
|
|
|
|
|
def test_download_streams_file_named_after_link(tmp_path):
|
|
idents = []
|
|
|
|
def file_link(ident):
|
|
idents.append(ident)
|
|
return "https://dl.webshare.cz/abc/Krysar%20%28CZ%29.epub"
|
|
|
|
result, _, progress = _run(tmp_path, file_link=file_link)
|
|
|
|
expected = tmp_path / "Krysar (CZ).epub"
|
|
assert result == str(expected)
|
|
assert expected.read_bytes() == b"x" * 20000
|
|
assert idents == ["abc123"]
|
|
progress.assert_called_with(100.0)
|
|
|
|
|
|
def test_too_small_file_is_rejected_and_removed(tmp_path):
|
|
result, status, _ = _run(
|
|
tmp_path, file_link=lambda ident: "https://dl.webshare.cz/f.epub", body=b"tiny"
|
|
)
|
|
assert result is None
|
|
assert not (tmp_path / "f.epub").exists()
|
|
status.assert_called_with("error", "Webshare returned an invalid file")
|
|
|
|
|
|
def test_link_error_reports_message(tmp_path):
|
|
def file_link(ident):
|
|
msg = "File not found."
|
|
raise WebshareError(msg, "FILE_LINK_FATAL_1")
|
|
|
|
result, status, _ = _run(tmp_path, file_link=file_link)
|
|
assert result is None
|
|
status.assert_called_with("error", "File not found.")
|
|
|
|
|
|
def test_cancel_before_start(tmp_path):
|
|
cancel = threading.Event()
|
|
cancel.set()
|
|
file_link = MagicMock()
|
|
result, status, _ = _run(tmp_path, file_link=file_link, cancel=cancel)
|
|
assert result is None
|
|
file_link.assert_not_called()
|
|
status.assert_called_with("cancelled", "Cancelled")
|