mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-11-10 12:38:32 +00:00
## Add Soulseek (slskd) as a release source, building on #1335 This is @dskvr's work from #1335 rebased onto current `main`, with my changes on top. #1335 has been quiet since 2026-09-13 and I needed the changes below, so I carried it forward instead of starting over. Their two commits are the first two in this branch with their authorship intact, and the source plugin, handler, API client, settings tab, tests, `ReleaseProtocol.SOULSEEK`, test client compose file and frontend bits are all theirs. I haven't changed how a search turns into releases beyond what is listed here. If @dskvr would rather keep #1335 alive and take these as commits on their branch, I can do that instead and close this. ### What I changed **Downloads survive other apps using the same slskd.** The handler found its files by listing slskd's downloads, and another app on the same instance (SoulSync here) sends `DELETE /transfers/downloads/all/completed` after each of its batches. Once that ran, a finished download no longer appeared in the list and the task never saw it complete, even though the files were on disk. With the new `SLSKD_ISOLATE_DOWNLOADS` setting (on by default) each task: - enqueues one batch (`POST /transfers/downloads/batches`) into its own folder, `<SLSKD_DESTINATION_PREFIX>/<task id>` - polls each transfer by id, which slskd still answers after the list has been cleared - decides it is done from the files on disk (sizes, not names), with the `/events` log as a fallback when the transfers are gone and the files aren't visible yet - copies out of that folder, and removes it only if `SLSKD_KEEP_COMPLETED` is off and the import worked With isolation off the handler behaves as it did in #1335. If the batch endpoint isn't there (404/405) it falls back to the original per-file enqueue. **Source changes.** m4b/m4a files are their own release next to loose chapter files, so auto-download can prefer one file over a chapter set. Results from the query variants are merged per peer. The folder text is exposed as `extra["author"]` and `extra["title_raw"]`, which is what auto-download's strict match and pack detection read. `is_available()` also needs an API key now. **Smaller fixes.** Cancelling while files are still being located now removes the transfer records it created. In the settings tab, a cleared form field is no longer replaced by the saved value when testing the connection. The new settings are in `docs/environment-variables.md`. ### Testing `tests/slskd` has 192 passing and 6 live tests that skip without an instance. I wrote the new tests first and checked the handler and source rules by breaking them and confirming a test failed. Ruff is clean. On the whole suite the only failures are 4 in `tests/download` that import `seleniumbase`, which isn't installed in my environment, and they fail the same on `main`. I've been running this on my own deployment against slskd 0.26.0 behind a VPN, with SoulSync active on a second instance. A search returned a couple hundred results from about 70 peers, an epub came down and was imported, and auto-download fetched five audiobooks as single m4b files. That is a day of use, so I don't know how it behaves on other slskd versions or on Windows path mappings. --------- Co-authored-by: sandwich <dskvr@users.noreply.github.com> Co-authored-by: sandwich <299465+dskvr@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
22 files changed
+5434
-2
No files matched your search
@@ -16,6 +16,7 @@
|
||||
# - NZBHydra: http://localhost:5076 (no auth by default)
|
||||
# - SABnzbd: http://localhost:8085 (complete setup wizard for API key)
|
||||
# - rTorrent: http://localhost:8000 (admin / admin - if auth enabled)
|
||||
# - slskd: http://localhost:5030 (slskd / slskdslskd; API key in .local/test-clients/slskd/config/slskd.yml)
|
||||
#
|
||||
|
||||
services:
|
||||
@@ -39,6 +40,7 @@ services:
|
||||
# - NZBHydra: http://nzbhydra:5076
|
||||
# - SABnzbd: http://sabnzbd:8080
|
||||
# - rTorrent: http://rtorrent:80 (XMLRPC via HTTP) or rtorrent (port 5000 for SCGI)
|
||||
# - slskd: http://slskd:5030 (downloads land in /downloads/slskd/complete)
|
||||
ports:
|
||||
- "8084:8084"
|
||||
volumes:
|
||||
@@ -68,6 +70,7 @@ services:
|
||||
- transmission
|
||||
- deluge
|
||||
- rtorrent
|
||||
- slskd
|
||||
restart: unless-stopped
|
||||
|
||||
prowlarr:
|
||||
@@ -198,3 +201,20 @@ services:
|
||||
- 5076:5076
|
||||
restart: unless-stopped
|
||||
|
||||
slskd:
|
||||
image: slskd/slskd:latest
|
||||
container_name: test-slskd
|
||||
environment:
|
||||
- TZ=UTC
|
||||
# Bootstrap: write .local/test-clients/slskd/config/slskd.yml first (see
|
||||
# scripts/test_clients.py for a template). It holds the Soulseek login, the
|
||||
# API key Shelfmark uses, and the download directories below.
|
||||
- SLSKD_CONFIG=/app/slskd.yml
|
||||
volumes:
|
||||
- ./.local/test-clients/slskd/config:/app
|
||||
- ./.local/test-clients/slskd/shares:/shares:ro
|
||||
- ./.local/test-clients/downloads:/downloads
|
||||
ports:
|
||||
- "5030:5030" # Web UI / API
|
||||
- "50300:50300" # Soulseek peer listen port
|
||||
restart: unless-stopped
|
||||
@@ -16,6 +16,7 @@ This document lists all configuration options that can be set via environment va
|
||||
- [Advanced](#advanced)
|
||||
- [Prowlarr](#prowlarr)
|
||||
- [Newznab](#newznab)
|
||||
- [Soulseek (slskd)](#soulseek-slskd)
|
||||
- [AudiobookBay](#audiobookbay)
|
||||
- [Libgen Search](#libgen-search)
|
||||
- [IRC](#irc)
|
||||
@@ -1498,6 +1499,129 @@ Automatically retry search without category filtering if no results are found
|
||||
|
||||
</details>
|
||||
|
||||
## Soulseek (slskd)
|
||||
|
||||
| Variable | Description | Type | Default |
|
||||
|----------|-------------|------|---------|
|
||||
| `SLSKD_ENABLED` | Enable searching and downloading books through slskd | boolean | `false` |
|
||||
| `SLSKD_URL` | Base URL of the slskd web interface | string | _none_ |
|
||||
| `SLSKD_API_KEY` | An API key from slskd's configuration (web.authentication.api_keys) with the readwrite role | string (secret) | _none_ |
|
||||
| `SLSKD_DOWNLOAD_PATH` | Where slskd's completed downloads folder is mounted inside Shelfmark. Leave empty when both containers see it at the same path, or use Settings > Advanced > Remote Path Mappings with client 'slskd'. | string | _none_ |
|
||||
| `SLSKD_ISOLATE_DOWNLOADS` | Download into a folder of its own inside slskd's downloads folder (needs slskd 0.26 or newer). Other apps using the same slskd cannot mix files with it or lose track of it when they clear slskd's list of finished transfers, and Shelfmark confirms a download by the files on disk. | boolean | `true` |
|
||||
| `SLSKD_DESTINATION_PREFIX` | The folder, inside slskd's downloads folder, that holds one folder per download. It must not climb out of slskd's downloads folder. | string | `shelfmark` |
|
||||
| `SLSKD_KEEP_COMPLETED` | Leave the finished files (and their transfer records) in slskd after Shelfmark copies them to your library, so they stay shared with other Soulseek users. Turn off to delete them once imported. | boolean | `true` |
|
||||
| `SLSKD_SEARCH_TIMEOUT` | How long a Soulseek search collects peer responses. Longer waits find more peers but make every search slower. | number | `15` |
|
||||
| `SLSKD_RESPONSE_LIMIT` | Stop a search early once this many peers have answered | number | `100` |
|
||||
| `SLSKD_QUEUE_TIMEOUT_MINUTES` | Give up when the peer has not started sending within this long. Set to 0 to wait indefinitely. | number | `60` |
|
||||
| `SLSKD_REMOVE_COMPLETED` | Clear the transfer from slskd's download list after Shelfmark imports it. Applies when each download does not get its own folder. | boolean | `true` |
|
||||
|
||||
<details>
|
||||
<summary>Detailed descriptions</summary>
|
||||
|
||||
#### `SLSKD_ENABLED`
|
||||
|
||||
**Enable Soulseek source**
|
||||
|
||||
Enable searching and downloading books through slskd
|
||||
|
||||
- **Type:** boolean
|
||||
- **Default:** `false`
|
||||
|
||||
#### `SLSKD_URL`
|
||||
|
||||
**slskd URL**
|
||||
|
||||
Base URL of the slskd web interface
|
||||
|
||||
- **Type:** string
|
||||
- **Default:** _none_
|
||||
|
||||
#### `SLSKD_API_KEY`
|
||||
|
||||
**API Key**
|
||||
|
||||
An API key from slskd's configuration (web.authentication.api_keys) with the readwrite role
|
||||
|
||||
- **Type:** string (secret)
|
||||
- **Default:** _none_
|
||||
|
||||
#### `SLSKD_DOWNLOAD_PATH`
|
||||
|
||||
**Downloads Path**
|
||||
|
||||
Where slskd's completed downloads folder is mounted inside Shelfmark. Leave empty when both containers see it at the same path, or use Settings > Advanced > Remote Path Mappings with client 'slskd'.
|
||||
|
||||
- **Type:** string
|
||||
- **Default:** _none_
|
||||
|
||||
#### `SLSKD_ISOLATE_DOWNLOADS`
|
||||
|
||||
**Give each download its own folder**
|
||||
|
||||
Download into a folder of its own inside slskd's downloads folder (needs slskd 0.26 or newer). Other apps using the same slskd cannot mix files with it or lose track of it when they clear slskd's list of finished transfers, and Shelfmark confirms a download by the files on disk.
|
||||
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
|
||||
#### `SLSKD_DESTINATION_PREFIX`
|
||||
|
||||
**Download Folder**
|
||||
|
||||
The folder, inside slskd's downloads folder, that holds one folder per download. It must not climb out of slskd's downloads folder.
|
||||
|
||||
- **Type:** string
|
||||
- **Default:** `shelfmark`
|
||||
|
||||
#### `SLSKD_KEEP_COMPLETED`
|
||||
|
||||
**Keep finished files in slskd**
|
||||
|
||||
Leave the finished files (and their transfer records) in slskd after Shelfmark copies them to your library, so they stay shared with other Soulseek users. Turn off to delete them once imported.
|
||||
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
|
||||
#### `SLSKD_SEARCH_TIMEOUT`
|
||||
|
||||
**Search Wait (seconds)**
|
||||
|
||||
How long a Soulseek search collects peer responses. Longer waits find more peers but make every search slower.
|
||||
|
||||
- **Type:** number
|
||||
- **Default:** `15`
|
||||
- **Constraints:** min: 3, max: 120
|
||||
|
||||
#### `SLSKD_RESPONSE_LIMIT`
|
||||
|
||||
**Max Peer Responses**
|
||||
|
||||
Stop a search early once this many peers have answered
|
||||
|
||||
- **Type:** number
|
||||
- **Default:** `100`
|
||||
- **Constraints:** min: 10, max: 1000
|
||||
|
||||
#### `SLSKD_QUEUE_TIMEOUT_MINUTES`
|
||||
|
||||
**Queue Timeout (minutes)**
|
||||
|
||||
Give up when the peer has not started sending within this long. Set to 0 to wait indefinitely.
|
||||
|
||||
- **Type:** number
|
||||
- **Default:** `60`
|
||||
- **Constraints:** min: 0, max: 1440
|
||||
|
||||
#### `SLSKD_REMOVE_COMPLETED`
|
||||
|
||||
**Remove completed transfers from slskd**
|
||||
|
||||
Clear the transfer from slskd's download list after Shelfmark imports it. Applies when each download does not get its own folder.
|
||||
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
|
||||
</details>
|
||||
|
||||
## AudiobookBay
|
||||
|
||||
| Variable | Description | Type | Default |
|
||||
|
||||
@@ -17,7 +17,7 @@ Works great alongside the following library tools, with support for automatic im
|
||||
## ✨ Features
|
||||
|
||||
- **One-Stop Interface** - A clean, modern UI to search, browse, and download from multiple configured sources in one place
|
||||
- **Multiple Sources** - Configurable web, torrent, usenet, and IRC source support
|
||||
- **Multiple Sources** - Configurable web, torrent, usenet, IRC, and Soulseek (slskd) source support
|
||||
- **Audiobook Support** - Full audiobook search and download with dedicated processing
|
||||
- **Flexible Search** - Search metadata providers (Hardcover, Open Library, Google Books) for rich book and audiobook discovery, or query configured sources directly
|
||||
- **Multi-User & Requests** - Share your instance with others, let users browse and request books, and manage approvals with configurable notifications
|
||||
@@ -149,6 +149,7 @@ Some of the additional options available in Settings:
|
||||
- **Additional audiobook sources** - Configure additional sources for audiobook discovery
|
||||
- **Direct Download mirrors** - Supply your own Anna's Archive mirror URLs; Auto mode tries them in the order listed. The `annas-archive.is` domain does not currently work as a source — use `annas-archive.gl` instead (checked August 2026; mirror availability changes)
|
||||
- **IRC** - Add details for IRC book sources and download directly from the UI. Most networks serve audiobooks from the same channel as ebooks (on `irc.irchighway.net` that's `#ebooks`, while `#bookz` is effectively inactive), so leave the separate audiobook channel blank unless your network actually indexes one. IRC audiobooks usually arrive as ZIP/RAR archives — keep those enabled under Supported Audiobook Formats or the releases are filtered out of results
|
||||
- **Soulseek (slskd)** - Search the Soulseek network and download through a [slskd](https://github.com/slskd/slskd) instance. Point Shelfmark at slskd's URL with an API key (`web.authentication.api_keys` in `slskd.yml`, role `readwrite`), and mount slskd's downloads folder into the Shelfmark container. If the folder sits at a different path inside Shelfmark, set **Downloads Path** in the slskd settings or add a Remote Path Mapping for client `slskd`. Ebook results are single files; audiobook results group a peer's folder of audio files into one release
|
||||
- **Library Link** - Add a link to your Calibre-Web or Grimmory instance in the UI header
|
||||
- **File processing** - Customiseable download paths, file renaming and directory creation with template-based renaming
|
||||
- **Network Settings** - Custom proxy support (SOCKS5 + HTTP/S) and configurable DNS
|
||||
|
||||
@@ -47,6 +47,14 @@ First-Time Setup:
|
||||
- Complete the setup wizard at http://localhost:8085
|
||||
- API key will be auto-detected by this script
|
||||
- In cwabd, copy API key from SABnzbd Config > General
|
||||
|
||||
slskd (Soulseek):
|
||||
- Before the first `up`, write .local/test-clients/slskd/config/slskd.yml.
|
||||
Run `uv run python scripts/test_clients.py --write-slskd-config` to create
|
||||
one with a throwaway Soulseek login and a fresh API key, then create the
|
||||
download folders it names (this script does both).
|
||||
- In Shelfmark: Settings > Soulseek (slskd), URL http://slskd:5030 and the
|
||||
API key from that file; Downloads Path /downloads/slskd/complete.
|
||||
"""
|
||||
|
||||
import sys
|
||||
@@ -84,8 +92,38 @@ CONFIG = {
|
||||
"rtorrent": {
|
||||
"url": "http://localhost:8000/RPC2",
|
||||
},
|
||||
# Soulseek
|
||||
"slskd": {
|
||||
"url": "http://localhost:5030",
|
||||
"config_file": Path(".local/test-clients/slskd/config/slskd.yml"),
|
||||
},
|
||||
}
|
||||
|
||||
SLSKD_CONFIG_TEMPLATE = """# slskd test-stack config (generated by scripts/test_clients.py; not committed).
|
||||
# Soulseek creates the account on first login, so any unused username works.
|
||||
web:
|
||||
port: 5030
|
||||
authentication:
|
||||
disabled: false
|
||||
username: slskd
|
||||
password: slskdslskd
|
||||
api_keys:
|
||||
shelfmark:
|
||||
key: {api_key}
|
||||
role: readwrite
|
||||
cidr: 0.0.0.0/0,::/0
|
||||
soulseek:
|
||||
username: {soulseek_username}
|
||||
password: {soulseek_password}
|
||||
description: shelfmark test-stack client
|
||||
directories:
|
||||
downloads: /downloads/slskd/complete
|
||||
incomplete: /downloads/slskd/incomplete
|
||||
shares:
|
||||
directories:
|
||||
- /shares
|
||||
"""
|
||||
|
||||
# Test magnet link (Ubuntu ISO - legal, small metadata)
|
||||
TEST_MAGNET = "magnet:?xt=urn:btih:3b245504cf5f11bbdbe1201cea6a6bf45aee1bc0&dn=ubuntu-22.04.3-live-server-amd64.iso"
|
||||
|
||||
@@ -522,7 +560,85 @@ def test_rtorrent() -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _read_slskd_api_key() -> str | None:
|
||||
"""Pull the shelfmark API key out of the generated slskd.yml (no YAML dependency)."""
|
||||
config_file = CONFIG["slskd"]["config_file"]
|
||||
if not config_file.exists():
|
||||
return None
|
||||
for line in config_file.read_text().splitlines():
|
||||
stripped = line.strip()
|
||||
if stripped.startswith("key:"):
|
||||
return stripped.split(":", 1)[1].strip()
|
||||
return None
|
||||
|
||||
|
||||
def write_slskd_config() -> Path:
|
||||
"""Write a fresh slskd.yml with a throwaway Soulseek login and API key."""
|
||||
import secrets
|
||||
|
||||
config_file = CONFIG["slskd"]["config_file"]
|
||||
config_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
suffix = secrets.token_hex(2)
|
||||
config_file.write_text(
|
||||
SLSKD_CONFIG_TEMPLATE.format(
|
||||
api_key=secrets.token_hex(24),
|
||||
soulseek_username=f"shelfmark-dev-{suffix}",
|
||||
soulseek_password=secrets.token_urlsafe(12),
|
||||
)
|
||||
)
|
||||
for folder in (
|
||||
"shares/TestBooks",
|
||||
"../downloads/slskd/complete",
|
||||
"../downloads/slskd/incomplete",
|
||||
):
|
||||
(config_file.parent.parent / folder).resolve().mkdir(parents=True, exist_ok=True)
|
||||
return config_file
|
||||
|
||||
|
||||
def test_slskd() -> bool:
|
||||
"""Test slskd connection and Soulseek login state."""
|
||||
print("\n" + "=" * 50)
|
||||
print("Testing slskd (Soulseek)")
|
||||
print("=" * 50)
|
||||
|
||||
# The slskd client needs the app's log/config dirs; keep them out of the way.
|
||||
import os
|
||||
import tempfile
|
||||
|
||||
scratch = tempfile.mkdtemp(prefix="shelfmark-test-clients-")
|
||||
for var, sub in (("LOG_ROOT", ""), ("CONFIG_DIR", "config"), ("TMP_DIR", "tmp")):
|
||||
os.environ.setdefault(var, str(Path(scratch, sub)))
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
from shelfmark.release_sources.slskd.api import SlskdClient
|
||||
|
||||
api_key = _read_slskd_api_key()
|
||||
if not api_key:
|
||||
print(f" ERROR: no API key found in {CONFIG['slskd']['config_file']}")
|
||||
print(" Run: uv run python scripts/test_clients.py --write-slskd-config")
|
||||
return False
|
||||
|
||||
client = SlskdClient(CONFIG["slskd"]["url"], api_key, timeout=10)
|
||||
success, message = client.test_connection()
|
||||
print(f" {message}")
|
||||
if not success:
|
||||
print(" Is the container running? docker ps | grep slskd")
|
||||
return False
|
||||
|
||||
print(f" Downloads directory (in slskd): {client.get_download_directory()}")
|
||||
search_id = client.start_search(
|
||||
"shelfmark-connectivity-probe", timeout_ms=1000, response_limit=1
|
||||
)
|
||||
print(f" Search API ok (id {search_id[:8]}...)")
|
||||
client.delete_search(search_id)
|
||||
print(" SUCCESS: slskd is working!")
|
||||
return True
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if "--write-slskd-config" in sys.argv:
|
||||
print(f"Wrote {write_slskd_config()}")
|
||||
return 0
|
||||
|
||||
print("Download Client Test Suite")
|
||||
print("=" * 50)
|
||||
print("Make sure containers are running:")
|
||||
@@ -546,6 +662,12 @@ def main() -> int:
|
||||
results["deluge"] = test_deluge()
|
||||
results["rtorrent"] = test_rtorrent()
|
||||
|
||||
# Test Soulseek
|
||||
print("\n" + "=" * 50)
|
||||
print("SOULSEEK")
|
||||
print("=" * 50)
|
||||
results["slskd"] = test_slskd()
|
||||
|
||||
# Summary
|
||||
print("\n" + "=" * 50)
|
||||
print("SUMMARY")
|
||||
|
||||
@@ -2036,6 +2036,7 @@ def advanced_settings() -> list[SettingsField]:
|
||||
{"value": "rtorrent", "label": "rTorrent"},
|
||||
{"value": "nzbget", "label": "NZBGet"},
|
||||
{"value": "sabnzbd", "label": "SABnzbd"},
|
||||
{"value": "slskd", "label": "slskd"},
|
||||
],
|
||||
"defaultValue": "qbittorrent",
|
||||
},
|
||||
|
||||
@@ -25,6 +25,7 @@ class ReleaseProtocol(StrEnum):
|
||||
TORRENT = "torrent" # BitTorrent
|
||||
NZB = "nzb" # Usenet NZB
|
||||
DCC = "dcc" # IRC DCC
|
||||
SOULSEEK = "soulseek" # Soulseek peer transfer (via slskd)
|
||||
|
||||
|
||||
class SourceUnavailableError(Exception):
|
||||
@@ -492,6 +493,7 @@ _BUILTIN_SOURCE_MODULES = (
|
||||
"shelfmark.release_sources.libgen",
|
||||
"shelfmark.release_sources.newznab",
|
||||
"shelfmark.release_sources.prowlarr",
|
||||
"shelfmark.release_sources.slskd",
|
||||
)
|
||||
_builtin_source_state = {"loaded": False}
|
||||
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
"""
|
||||
slskd release source plugin.
|
||||
|
||||
Searches the Soulseek network through a slskd instance and downloads the chosen
|
||||
files with it. Unlike torrent/usenet sources, search and download are the same
|
||||
service: a result is a file on a specific peer, and only slskd can fetch it.
|
||||
|
||||
Includes:
|
||||
- SlskdSource: Search integration
|
||||
- SlskdHandler: Download handling via slskd transfers
|
||||
"""
|
||||
|
||||
# Import submodules to trigger decorator registration
|
||||
from shelfmark.release_sources.slskd import (
|
||||
handler as handler,
|
||||
)
|
||||
from shelfmark.release_sources.slskd import (
|
||||
settings as settings,
|
||||
)
|
||||
from shelfmark.release_sources.slskd import (
|
||||
source as source,
|
||||
)
|
||||
@@ -0,0 +1,452 @@
|
||||
"""slskd API client - talks to a slskd (Soulseek daemon) instance over its REST API.
|
||||
|
||||
Only the small slice of the v0 API that Shelfmark needs is wrapped here: searches,
|
||||
download transfers, and the two read-only endpoints used to test the connection and
|
||||
learn where slskd writes completed files.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from typing import Any
|
||||
from urllib.parse import quote
|
||||
|
||||
import requests
|
||||
|
||||
from shelfmark.core.logger import setup_logger
|
||||
from shelfmark.core.utils import normalize_http_url
|
||||
from shelfmark.download.network import get_ssl_verify
|
||||
|
||||
logger = setup_logger(__name__)
|
||||
|
||||
API_PREFIX = "/api/v0"
|
||||
_HTTP_UNAUTHORIZED = 401
|
||||
_HTTP_FORBIDDEN = 403
|
||||
_HTTP_NOT_FOUND = 404
|
||||
_HTTP_METHOD_NOT_ALLOWED = 405
|
||||
|
||||
# slskd transfer states are "Phase" or "Completed, Outcome" strings.
|
||||
TRANSFER_STATE_COMPLETED_PREFIX = "Completed"
|
||||
TRANSFER_STATE_SUCCEEDED = "Completed, Succeeded"
|
||||
|
||||
|
||||
class SlskdError(Exception):
|
||||
"""Raised when slskd returns an error or an unexpected payload."""
|
||||
|
||||
|
||||
class SlskdAuthError(SlskdError):
|
||||
"""Raised when slskd rejects the configured API key."""
|
||||
|
||||
|
||||
class SlskdBatchUnsupportedError(SlskdError):
|
||||
"""slskd has no batch download endpoint (it is older than 0.26)."""
|
||||
|
||||
|
||||
def transfer_is_complete(state: str | None) -> bool:
|
||||
"""Whether a slskd transfer state string is terminal."""
|
||||
return str(state or "").startswith(TRANSFER_STATE_COMPLETED_PREFIX)
|
||||
|
||||
|
||||
# A search ends on slskd's own clock; this is how far past the wait one may run before we
|
||||
# stop waiting for it (its responses keep resetting slskd's timer while peers answer).
|
||||
SEARCH_FINISH_MARGIN_SECONDS = 60.0
|
||||
# How long a search that reports itself complete may take to show ``endedAt``. slskd sets
|
||||
# it in the write that saves the responses; a slskd that never reports it is read after this.
|
||||
SEARCH_SAVE_WAIT_SECONDS = 10.0
|
||||
|
||||
|
||||
def transfer_succeeded(state: str | None) -> bool:
|
||||
"""Whether a slskd transfer state string is the successful terminal state."""
|
||||
return str(state or "").strip() == TRANSFER_STATE_SUCCEEDED
|
||||
|
||||
|
||||
class SlskdClient:
|
||||
"""Client for the slskd v0 REST API."""
|
||||
|
||||
def __init__(self, url: str, api_key: str, timeout: int = 30) -> None:
|
||||
self.base_url = normalize_http_url(url).rstrip("/")
|
||||
self.api_key = api_key
|
||||
self.timeout = timeout
|
||||
self._session = requests.Session()
|
||||
|
||||
# ── plumbing ──────────────────────────────────────────────────────────────
|
||||
|
||||
def _api_url(self, path: str) -> str:
|
||||
base = self.base_url
|
||||
base = base.removesuffix(API_PREFIX)
|
||||
return f"{base}{API_PREFIX}/{path.lstrip('/')}"
|
||||
|
||||
def _request(
|
||||
self,
|
||||
method: str,
|
||||
path: str,
|
||||
*,
|
||||
json: Any = None,
|
||||
params: dict[str, Any] | None = None,
|
||||
) -> requests.Response:
|
||||
url = self._api_url(path)
|
||||
headers = {"Accept": "application/json"}
|
||||
if self.api_key:
|
||||
headers["X-API-Key"] = self.api_key
|
||||
|
||||
logger.debug("slskd API: %s %s", method, url)
|
||||
response = self._session.request(
|
||||
method,
|
||||
url,
|
||||
headers=headers,
|
||||
json=json,
|
||||
params=params,
|
||||
timeout=self.timeout,
|
||||
verify=get_ssl_verify(url),
|
||||
)
|
||||
if response.status_code in (_HTTP_UNAUTHORIZED, _HTTP_FORBIDDEN):
|
||||
raise SlskdAuthError("slskd rejected the API key")
|
||||
response.raise_for_status()
|
||||
return response
|
||||
|
||||
@staticmethod
|
||||
def _json(response: requests.Response) -> Any:
|
||||
if not response.content:
|
||||
return None
|
||||
try:
|
||||
return response.json()
|
||||
except ValueError as e:
|
||||
raise SlskdError(f"slskd returned a non-JSON response: {e}") from e
|
||||
|
||||
# ── application / options ─────────────────────────────────────────────────
|
||||
|
||||
def get_application(self) -> dict[str, Any]:
|
||||
"""Return slskd's application state (version, Soulseek server connection...)."""
|
||||
data = self._json(self._request("GET", "application"))
|
||||
if not isinstance(data, dict):
|
||||
raise SlskdError("slskd application state has an unexpected shape")
|
||||
return data
|
||||
|
||||
def get_download_directory(self) -> str | None:
|
||||
"""Return the directory slskd writes completed downloads to (as slskd sees it)."""
|
||||
data = self._json(self._request("GET", "options"))
|
||||
if not isinstance(data, dict):
|
||||
return None
|
||||
directories = data.get("directories")
|
||||
if not isinstance(directories, dict):
|
||||
return None
|
||||
downloads = str(directories.get("downloads") or "").strip()
|
||||
return downloads or None
|
||||
|
||||
def test_connection(self) -> tuple[bool, str]:
|
||||
"""Test connectivity and Soulseek login state. Returns (success, message)."""
|
||||
logger.info("Testing slskd connection to: %s", self.base_url)
|
||||
try:
|
||||
app = self.get_application()
|
||||
except SlskdAuthError:
|
||||
return False, "Invalid API key"
|
||||
except requests.exceptions.ConnectionError:
|
||||
return False, "Could not connect. Check the URL."
|
||||
except requests.exceptions.HTTPError as e:
|
||||
status = e.response.status_code if e.response is not None else "unknown"
|
||||
return False, f"HTTP error {status}"
|
||||
except (requests.exceptions.RequestException, SlskdError) as e:
|
||||
return False, f"Connection failed: {e!s}"
|
||||
|
||||
version = app.get("version")
|
||||
version_text = ""
|
||||
if isinstance(version, dict):
|
||||
version_text = str(version.get("current") or version.get("full") or "").strip()
|
||||
elif isinstance(version, str):
|
||||
version_text = version.strip()
|
||||
label = f"slskd {version_text}".strip()
|
||||
|
||||
server = app.get("server")
|
||||
server_state = ""
|
||||
logged_in = False
|
||||
if isinstance(server, dict):
|
||||
server_state = str(server.get("state") or "").strip()
|
||||
logged_in = bool(server.get("isLoggedIn"))
|
||||
|
||||
if not logged_in:
|
||||
state_text = server_state or "unknown"
|
||||
return (
|
||||
False,
|
||||
f"Connected to {label}, but it is not logged in to Soulseek "
|
||||
f"(server state: {state_text}). Check the Soulseek credentials in slskd.",
|
||||
)
|
||||
return True, f"Connected to {label} (logged in to Soulseek)"
|
||||
|
||||
def is_logged_in(self) -> bool:
|
||||
"""Whether slskd currently has a logged-in Soulseek session."""
|
||||
try:
|
||||
server = self.get_application().get("server")
|
||||
except requests.exceptions.RequestException, SlskdError:
|
||||
return False
|
||||
return isinstance(server, dict) and bool(server.get("isLoggedIn"))
|
||||
|
||||
# ── searches ──────────────────────────────────────────────────────────────
|
||||
|
||||
def start_search(
|
||||
self,
|
||||
search_text: str,
|
||||
*,
|
||||
timeout_ms: int,
|
||||
response_limit: int,
|
||||
file_limit: int = 10000,
|
||||
) -> str:
|
||||
"""Start a network search and return its slskd id."""
|
||||
payload: dict[str, Any] = {
|
||||
"searchText": search_text,
|
||||
"searchTimeout": int(timeout_ms),
|
||||
"responseLimit": int(response_limit),
|
||||
"fileLimit": int(file_limit),
|
||||
}
|
||||
data = self._json(self._request("POST", "searches", json=payload))
|
||||
search_id = data.get("id") if isinstance(data, dict) else None
|
||||
if not search_id:
|
||||
raise SlskdError("slskd did not return a search id")
|
||||
return str(search_id)
|
||||
|
||||
def get_search(self, search_id: str) -> dict[str, Any]:
|
||||
"""Return the state of a search (without its responses)."""
|
||||
data = self._json(self._request("GET", f"searches/{quote(search_id, safe='')}"))
|
||||
if not isinstance(data, dict):
|
||||
raise SlskdError("slskd search state has an unexpected shape")
|
||||
return data
|
||||
|
||||
def get_search_responses(self, search_id: str) -> list[dict[str, Any]]:
|
||||
"""Return the peer responses collected so far for a search."""
|
||||
data = self._json(self._request("GET", f"searches/{quote(search_id, safe='')}/responses"))
|
||||
if not isinstance(data, list):
|
||||
return []
|
||||
return [r for r in data if isinstance(r, dict)]
|
||||
|
||||
def delete_search(self, search_id: str) -> None:
|
||||
"""Forget a search. Best effort; slskd retains them otherwise."""
|
||||
try:
|
||||
self._request("DELETE", f"searches/{quote(search_id, safe='')}")
|
||||
except (requests.exceptions.RequestException, SlskdError) as e:
|
||||
logger.debug("Failed to delete slskd search %s: %s", search_id, e)
|
||||
|
||||
def search(
|
||||
self,
|
||||
search_text: str,
|
||||
*,
|
||||
wait_seconds: float,
|
||||
response_limit: int,
|
||||
poll_interval: float = 0.5,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Run a search to the end and return its responses.
|
||||
|
||||
slskd searches are asynchronous: one ends ``searchTimeout`` after its last response,
|
||||
or when ``responseLimit`` peers have answered, so a busy one can outlast
|
||||
``wait_seconds``. A started search is never cut short: it is not cancelled, and it
|
||||
is not deleted until slskd has saved its responses.
|
||||
|
||||
Complete is not saved. slskd flags a search complete on its last state change and
|
||||
saves the responses in a second write, which also sets ``endedAt``.
|
||||
``GET .../responses`` reads what was saved, so reading earlier returns nothing, and
|
||||
deleting earlier makes that second write fail. So this waits for ``endedAt``. A
|
||||
search still running ``SEARCH_FINISH_MARGIN_SECONDS`` past the wait is left alone
|
||||
(slskd keeps its record) and this returns nothing.
|
||||
"""
|
||||
if not search_text.strip():
|
||||
return []
|
||||
|
||||
search_id = self.start_search(
|
||||
search_text,
|
||||
timeout_ms=int(wait_seconds * 1000),
|
||||
response_limit=response_limit,
|
||||
)
|
||||
|
||||
deadline = time.monotonic() + wait_seconds + SEARCH_FINISH_MARGIN_SECONDS
|
||||
completed_at: float | None = None
|
||||
responses: list[dict[str, Any]] | None = None
|
||||
while True:
|
||||
state = self.get_search(search_id)
|
||||
now = time.monotonic()
|
||||
if state.get("endedAt"):
|
||||
responses = self.get_search_responses(search_id) # saved: endedAt came with them
|
||||
break
|
||||
if state.get("isComplete") or transfer_is_complete(state.get("state")):
|
||||
completed_at = completed_at if completed_at is not None else now
|
||||
if now - completed_at >= SEARCH_SAVE_WAIT_SECONDS:
|
||||
# No endedAt: an older slskd. Take its word, unless it heard peers and
|
||||
# nothing is saved.
|
||||
responses = self.get_search_responses(search_id)
|
||||
if not responses and (state.get("responseCount") or 0) > 0:
|
||||
responses = None
|
||||
break
|
||||
elif now >= deadline:
|
||||
break
|
||||
time.sleep(poll_interval)
|
||||
|
||||
if responses is None:
|
||||
logger.warning(
|
||||
"slskd search '%s' did not finish saving; leaving it for slskd", search_text
|
||||
)
|
||||
return []
|
||||
|
||||
self.delete_search(search_id)
|
||||
logger.debug("slskd search '%s': %d responses", search_text, len(responses))
|
||||
return responses
|
||||
|
||||
# ── transfers ─────────────────────────────────────────────────────────────
|
||||
|
||||
def enqueue_downloads(
|
||||
self,
|
||||
username: str,
|
||||
files: list[dict[str, Any]],
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Ask slskd to download ``files`` (``{"filename", "size"}``) from ``username``.
|
||||
|
||||
Returns the transfer records slskd created. Newer slskd versions return them in
|
||||
the response body; older ones return an empty 201, in which case the transfers
|
||||
are looked up by filename from the user's download list.
|
||||
"""
|
||||
payload = [{"filename": str(f["filename"]), "size": int(f.get("size") or 0)} for f in files]
|
||||
response = self._request(
|
||||
"POST", f"transfers/downloads/{quote(username, safe='')}", json=payload
|
||||
)
|
||||
data = self._json(response)
|
||||
|
||||
if isinstance(data, dict):
|
||||
failed = data.get("failed")
|
||||
if isinstance(failed, list) and failed:
|
||||
first = failed[0] if isinstance(failed[0], dict) else {}
|
||||
reason = first.get("reason") or first.get("message") or "unknown reason"
|
||||
raise SlskdError(f"slskd refused {len(failed)} file(s): {reason}")
|
||||
enqueued = data.get("enqueued")
|
||||
if isinstance(enqueued, list):
|
||||
return [t for t in enqueued if isinstance(t, dict)]
|
||||
|
||||
wanted = {str(f["filename"]) for f in files}
|
||||
return [t for t in self.list_downloads(username) if str(t.get("filename") or "") in wanted]
|
||||
|
||||
def enqueue_batch(
|
||||
self,
|
||||
username: str,
|
||||
files: list[dict[str, Any]],
|
||||
destination: str,
|
||||
) -> dict[str, Any]:
|
||||
"""Queue ``files`` from ``username`` as one batch landing in ``destination``.
|
||||
|
||||
``destination`` is a path relative to slskd's downloads folder, so every download of
|
||||
ours gets a folder of its own. Returns ``{"batch_id", "transfers", "failures"}``:
|
||||
the transfers slskd created and the files it refused (already queued, say). A slskd
|
||||
that predates batches answers 404 or 405, reported as ``SlskdBatchUnsupportedError``
|
||||
so the caller can fall back to the plain enqueue.
|
||||
"""
|
||||
payload = {
|
||||
"username": username,
|
||||
"files": [
|
||||
{"filename": str(f["filename"]), "size": int(f.get("size") or 0)} for f in files
|
||||
],
|
||||
"options": {"destination": destination},
|
||||
}
|
||||
try:
|
||||
response = self._request("POST", "transfers/downloads/batches", json=payload)
|
||||
except requests.exceptions.HTTPError as e:
|
||||
if e.response is not None and e.response.status_code in (
|
||||
_HTTP_NOT_FOUND,
|
||||
_HTTP_METHOD_NOT_ALLOWED,
|
||||
):
|
||||
raise SlskdBatchUnsupportedError("slskd has no batch download endpoint") from e
|
||||
raise
|
||||
data = self._json(response)
|
||||
data = data if isinstance(data, dict) else {}
|
||||
raw_batch = data.get("batch")
|
||||
batch = raw_batch if isinstance(raw_batch, dict) else {}
|
||||
failures = data.get("failures")
|
||||
return {
|
||||
"batch_id": str(batch.get("id") or ""),
|
||||
"transfers": [t for t in batch.get("transfers") or [] if isinstance(t, dict)],
|
||||
"failures": [f for f in failures or [] if isinstance(f, dict)],
|
||||
}
|
||||
|
||||
def get_batch(self, batch_id: str) -> dict[str, Any] | None:
|
||||
"""One batch with all its transfers, removed ones included; None when it is gone."""
|
||||
try:
|
||||
response = self._request(
|
||||
"GET", f"transfers/downloads/batches/{quote(batch_id, safe='')}"
|
||||
)
|
||||
except requests.exceptions.HTTPError as e:
|
||||
if e.response is not None and e.response.status_code == _HTTP_NOT_FOUND:
|
||||
return None
|
||||
raise
|
||||
data = self._json(response)
|
||||
return data if isinstance(data, dict) else None
|
||||
|
||||
def get_events(self, *, limit: int = 200, offset: int = 0) -> list[dict[str, Any]]:
|
||||
"""slskd's stored events, NEWEST first (``DownloadFileComplete`` and so on).
|
||||
|
||||
They live in slskd's database, so they outlive anything clearing the transfer list.
|
||||
A slskd without the endpoint yields nothing.
|
||||
"""
|
||||
try:
|
||||
response = self._request("GET", "events", params={"offset": offset, "limit": limit})
|
||||
except requests.exceptions.HTTPError as e:
|
||||
if e.response is not None and e.response.status_code == _HTTP_NOT_FOUND:
|
||||
return []
|
||||
raise
|
||||
data = self._json(response)
|
||||
return [e for e in data if isinstance(e, dict)] if isinstance(data, list) else []
|
||||
|
||||
def list_downloads(self, username: str) -> list[dict[str, Any]]:
|
||||
"""Return every download transfer slskd tracks for ``username``, flattened."""
|
||||
try:
|
||||
response = self._request("GET", f"transfers/downloads/{quote(username, safe='')}")
|
||||
except requests.exceptions.HTTPError as e:
|
||||
if e.response is not None and e.response.status_code == _HTTP_NOT_FOUND:
|
||||
return []
|
||||
raise
|
||||
data = self._json(response)
|
||||
return _flatten_transfers(data)
|
||||
|
||||
def get_transfer(self, username: str, transfer_id: str) -> dict[str, Any] | None:
|
||||
"""Return one transfer record, or None when slskd no longer tracks it."""
|
||||
try:
|
||||
response = self._request(
|
||||
"GET",
|
||||
f"transfers/downloads/{quote(username, safe='')}/{quote(transfer_id, safe='')}",
|
||||
)
|
||||
except requests.exceptions.HTTPError as e:
|
||||
if e.response is not None and e.response.status_code == _HTTP_NOT_FOUND:
|
||||
return None
|
||||
raise
|
||||
data = self._json(response)
|
||||
return data if isinstance(data, dict) else None
|
||||
|
||||
def cancel_transfer(self, username: str, transfer_id: str, *, remove: bool) -> bool:
|
||||
"""Cancel a transfer; with ``remove`` also drop it from slskd's list."""
|
||||
try:
|
||||
self._request(
|
||||
"DELETE",
|
||||
f"transfers/downloads/{quote(username, safe='')}/{quote(transfer_id, safe='')}",
|
||||
params={"remove": "true" if remove else "false"},
|
||||
)
|
||||
except requests.exceptions.HTTPError as e:
|
||||
if e.response is not None and e.response.status_code == _HTTP_NOT_FOUND:
|
||||
return False
|
||||
raise
|
||||
return True
|
||||
|
||||
|
||||
def _flatten_transfers(data: Any) -> list[dict[str, Any]]:
|
||||
"""Flatten slskd's ``{username, directories: [{directory, files: [...]}]}`` shape."""
|
||||
if isinstance(data, list):
|
||||
# GET /transfers/downloads returns one entry per user.
|
||||
transfers: list[dict[str, Any]] = []
|
||||
for entry in data:
|
||||
transfers.extend(_flatten_transfers(entry))
|
||||
return transfers
|
||||
if not isinstance(data, dict):
|
||||
return []
|
||||
directories = data.get("directories")
|
||||
if not isinstance(directories, list):
|
||||
return []
|
||||
transfers = []
|
||||
for directory in directories:
|
||||
if not isinstance(directory, dict):
|
||||
continue
|
||||
files = directory.get("files")
|
||||
if not isinstance(files, list):
|
||||
continue
|
||||
transfers.extend(f for f in files if isinstance(f, dict))
|
||||
return transfers
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,194 @@
|
||||
"""slskd settings registration."""
|
||||
|
||||
from typing import Any
|
||||
|
||||
from shelfmark.core.settings_registry import (
|
||||
ActionButton,
|
||||
CheckboxField,
|
||||
HeadingField,
|
||||
NumberField,
|
||||
PasswordField,
|
||||
SettingsField,
|
||||
TextField,
|
||||
register_settings,
|
||||
)
|
||||
from shelfmark.core.utils import normalize_http_url
|
||||
|
||||
|
||||
def _test_slskd_connection(current_values: dict[str, Any] | None = None) -> dict[str, Any]:
|
||||
"""Test the slskd connection using unsaved form values when available."""
|
||||
from shelfmark.core.config import config
|
||||
from shelfmark.release_sources.slskd.api import SlskdClient
|
||||
|
||||
current_values = current_values or {}
|
||||
|
||||
def _value(key: str) -> str:
|
||||
# A key the form sent, even empty, is what the user typed; only a key the form did not
|
||||
# send falls back to the saved setting. `or` would turn a cleared field back into the
|
||||
# saved value and make Test Connection validate something the user just removed.
|
||||
if key in current_values:
|
||||
return str(current_values.get(key) or "")
|
||||
return str(config.get(key, "") or "")
|
||||
|
||||
raw_url = _value("SLSKD_URL")
|
||||
api_key = _value("SLSKD_API_KEY")
|
||||
|
||||
if not raw_url.strip():
|
||||
return {"success": False, "message": "slskd URL is required"}
|
||||
|
||||
url = normalize_http_url(raw_url)
|
||||
if not url:
|
||||
return {"success": False, "message": "slskd URL is invalid"}
|
||||
|
||||
if not api_key.strip():
|
||||
return {"success": False, "message": "slskd API key is required"}
|
||||
|
||||
try:
|
||||
success, message = SlskdClient(url, api_key.strip()).test_connection()
|
||||
except Exception as e: # noqa: BLE001 — surface any unexpected error to the UI
|
||||
return {"success": False, "message": f"Connection failed: {e!s}"}
|
||||
else:
|
||||
return {"success": success, "message": message}
|
||||
|
||||
|
||||
@register_settings(
|
||||
name="slskd_config",
|
||||
display_name="Soulseek (slskd)",
|
||||
icon="download",
|
||||
order=43,
|
||||
)
|
||||
def slskd_config_settings() -> list[SettingsField]:
|
||||
"""slskd connection and download settings."""
|
||||
return [
|
||||
HeadingField(
|
||||
key="slskd_heading",
|
||||
title="Soulseek via slskd",
|
||||
description=(
|
||||
"Search the Soulseek network and download files through a "
|
||||
"[slskd](https://github.com/slskd/slskd) instance. slskd must be logged in "
|
||||
"to Soulseek, and its downloads folder must be visible to Shelfmark."
|
||||
),
|
||||
),
|
||||
CheckboxField(
|
||||
key="SLSKD_ENABLED",
|
||||
label="Enable Soulseek source",
|
||||
default=False,
|
||||
description="Enable searching and downloading books through slskd",
|
||||
),
|
||||
TextField(
|
||||
key="SLSKD_URL",
|
||||
label="slskd URL",
|
||||
description="Base URL of the slskd web interface",
|
||||
placeholder="http://slskd:5030",
|
||||
required=False,
|
||||
show_when={"field": "SLSKD_ENABLED", "value": True},
|
||||
),
|
||||
PasswordField(
|
||||
key="SLSKD_API_KEY",
|
||||
label="API Key",
|
||||
description=(
|
||||
"An API key from slskd's configuration (web.authentication.api_keys) with "
|
||||
"the readwrite role"
|
||||
),
|
||||
required=False,
|
||||
show_when={"field": "SLSKD_ENABLED", "value": True},
|
||||
),
|
||||
ActionButton(
|
||||
key="test_slskd",
|
||||
label="Test Connection",
|
||||
description="Verify the URL and API key, and that slskd is logged in to Soulseek",
|
||||
style="primary",
|
||||
callback=_test_slskd_connection,
|
||||
show_when={"field": "SLSKD_ENABLED", "value": True},
|
||||
),
|
||||
TextField(
|
||||
key="SLSKD_DOWNLOAD_PATH",
|
||||
label="Downloads Path",
|
||||
description=(
|
||||
"Where slskd's completed downloads folder is mounted inside Shelfmark. "
|
||||
"Leave empty when both containers see it at the same path, or use "
|
||||
"Settings > Advanced > Remote Path Mappings with client 'slskd'."
|
||||
),
|
||||
placeholder="/downloads/slskd/complete",
|
||||
required=False,
|
||||
show_when={"field": "SLSKD_ENABLED", "value": True},
|
||||
),
|
||||
CheckboxField(
|
||||
key="SLSKD_ISOLATE_DOWNLOADS",
|
||||
label="Give each download its own folder",
|
||||
default=True,
|
||||
description=(
|
||||
"Download into a folder of its own inside slskd's downloads folder (needs "
|
||||
"slskd 0.26 or newer). Other apps using the same slskd cannot mix files with "
|
||||
"it or lose track of it when they clear slskd's list of finished transfers, "
|
||||
"and Shelfmark confirms a download by the files on disk."
|
||||
),
|
||||
show_when={"field": "SLSKD_ENABLED", "value": True},
|
||||
),
|
||||
TextField(
|
||||
key="SLSKD_DESTINATION_PREFIX",
|
||||
label="Download Folder",
|
||||
description=(
|
||||
"The folder, inside slskd's downloads folder, that holds one folder per "
|
||||
"download. It must not climb out of slskd's downloads folder."
|
||||
),
|
||||
placeholder="shelfmark",
|
||||
default="shelfmark",
|
||||
required=False,
|
||||
show_when={"field": "SLSKD_ISOLATE_DOWNLOADS", "value": True},
|
||||
),
|
||||
CheckboxField(
|
||||
key="SLSKD_KEEP_COMPLETED",
|
||||
label="Keep finished files in slskd",
|
||||
default=True,
|
||||
description=(
|
||||
"Leave the finished files (and their transfer records) in slskd after "
|
||||
"Shelfmark copies them to your library, so they stay shared with other "
|
||||
"Soulseek users. Turn off to delete them once imported."
|
||||
),
|
||||
show_when={"field": "SLSKD_ISOLATE_DOWNLOADS", "value": True},
|
||||
),
|
||||
NumberField(
|
||||
key="SLSKD_SEARCH_TIMEOUT",
|
||||
label="Search Wait (seconds)",
|
||||
description=(
|
||||
"How long a Soulseek search collects peer responses. Longer waits find "
|
||||
"more peers but make every search slower."
|
||||
),
|
||||
default=15,
|
||||
min_value=3,
|
||||
max_value=120,
|
||||
show_when={"field": "SLSKD_ENABLED", "value": True},
|
||||
),
|
||||
NumberField(
|
||||
key="SLSKD_RESPONSE_LIMIT",
|
||||
label="Max Peer Responses",
|
||||
description="Stop a search early once this many peers have answered",
|
||||
default=100,
|
||||
min_value=10,
|
||||
max_value=1000,
|
||||
show_when={"field": "SLSKD_ENABLED", "value": True},
|
||||
),
|
||||
NumberField(
|
||||
key="SLSKD_QUEUE_TIMEOUT_MINUTES",
|
||||
label="Queue Timeout (minutes)",
|
||||
description=(
|
||||
"Give up when the peer has not started sending within this long. "
|
||||
"Set to 0 to wait indefinitely."
|
||||
),
|
||||
default=60,
|
||||
min_value=0,
|
||||
max_value=1440,
|
||||
show_when={"field": "SLSKD_ENABLED", "value": True},
|
||||
),
|
||||
CheckboxField(
|
||||
key="SLSKD_REMOVE_COMPLETED",
|
||||
label="Remove completed transfers from slskd",
|
||||
default=True,
|
||||
description=(
|
||||
"Clear the transfer from slskd's download list after Shelfmark imports it. "
|
||||
"Applies when each download does not get its own folder."
|
||||
),
|
||||
show_when={"field": "SLSKD_ISOLATE_DOWNLOADS", "value": False},
|
||||
),
|
||||
]
|
||||
@@ -0,0 +1,541 @@
|
||||
"""slskd release source - searches the Soulseek network through a slskd instance.
|
||||
|
||||
Soulseek results are files shared by peers, not curated releases. Ebooks map one
|
||||
file to one release. Audiobooks are usually a folder of per-chapter audio files, so
|
||||
audiobook results are grouped by (peer, directory) into a single multi-file release.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from hashlib import sha256
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from shelfmark.core.search_plan import ReleaseSearchPlan
|
||||
from shelfmark.metadata_providers import BookMetadata
|
||||
|
||||
from shelfmark.core.config import config
|
||||
from shelfmark.core.logger import setup_logger
|
||||
from shelfmark.core.utils import ARCHIVE_FORMATS, AUDIOBOOK_FORMATS, normalize_http_url
|
||||
from shelfmark.core.utils import is_audiobook as check_audiobook
|
||||
from shelfmark.release_sources import (
|
||||
ColumnAlign,
|
||||
ColumnColorHint,
|
||||
ColumnRenderType,
|
||||
ColumnSchema,
|
||||
LeadingCellConfig,
|
||||
LeadingCellType,
|
||||
Release,
|
||||
ReleaseColumnConfig,
|
||||
ReleaseProtocol,
|
||||
ReleaseSource,
|
||||
SortOption,
|
||||
SourceActionButton,
|
||||
register_source,
|
||||
)
|
||||
from shelfmark.release_sources.prowlarr.source import _parse_size
|
||||
from shelfmark.release_sources.slskd.api import SlskdClient
|
||||
|
||||
logger = setup_logger(__name__)
|
||||
|
||||
SOURCE_NAME = "slskd"
|
||||
DISPLAY_NAME = "Soulseek (slskd)"
|
||||
|
||||
# Soulseek searches complete on their own after slskd's per-search timeout; this caps
|
||||
# the whole Shelfmark-side search across every query variant.
|
||||
SLSKD_SEARCH_TIMEOUT_SECONDS = 90.0
|
||||
DEFAULT_SEARCH_WAIT_SECONDS = 15
|
||||
DEFAULT_RESPONSE_LIMIT = 100
|
||||
MAX_QUERY_VARIANTS = 3
|
||||
|
||||
# Whole-book audio files: one file holds the entire book, unlike chapter files.
|
||||
CONTAINER_FORMATS = frozenset({"m4b", "m4a"})
|
||||
_DEFAULT_EBOOK_FORMATS = ["epub", "mobi", "azw3", "fb2", "djvu", "cbz", "cbr"]
|
||||
_DEFAULT_AUDIOBOOK_FORMATS = [*AUDIOBOOK_FORMATS, *ARCHIVE_FORMATS]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PeerFile:
|
||||
"""One shared file from a search response, with its remote path split apart."""
|
||||
|
||||
username: str
|
||||
filename: str # Full remote path exactly as slskd reports it
|
||||
directory: str # Remote directory (Soulseek separators preserved)
|
||||
basename: str
|
||||
extension: str # Lowercase, no dot
|
||||
size: int
|
||||
|
||||
|
||||
def split_remote_path(filename: str) -> tuple[str, str]:
|
||||
"""Split a Soulseek path into (directory, basename).
|
||||
|
||||
Soulseek paths use backslashes, but some clients share with forward slashes.
|
||||
"""
|
||||
text = str(filename or "")
|
||||
cut = max(text.rfind("\\"), text.rfind("/"))
|
||||
if cut < 0:
|
||||
return "", text
|
||||
return text[:cut], text[cut + 1 :]
|
||||
|
||||
|
||||
def remote_directory_name(directory: str) -> str:
|
||||
"""Return the last segment of a remote directory path."""
|
||||
return split_remote_path(directory)[1] or directory
|
||||
|
||||
|
||||
def file_extension(basename: str) -> str:
|
||||
"""Return the lowercase extension of a file name, without the dot."""
|
||||
name = str(basename or "")
|
||||
if "." not in name:
|
||||
return ""
|
||||
return name.rsplit(".", 1)[1].strip().lower()
|
||||
|
||||
|
||||
def make_source_id(username: str, remote_path: str) -> str:
|
||||
"""Stable, cache-safe id for a (peer, remote path) pair."""
|
||||
digest = sha256(f"{username}\0{remote_path}".encode()).hexdigest()[:24]
|
||||
return f"{SOURCE_NAME}:{digest}"
|
||||
|
||||
|
||||
def _normalize_formats(raw: object, defaults: list[str]) -> set[str]:
|
||||
if isinstance(raw, str):
|
||||
raw = [part for part in raw.replace(",", " ").split() if part]
|
||||
if not isinstance(raw, (list, tuple)):
|
||||
return {f.lower() for f in defaults}
|
||||
formats = {str(item).strip().lower().lstrip(".") for item in raw if str(item).strip()}
|
||||
return formats or {f.lower() for f in defaults}
|
||||
|
||||
|
||||
def get_supported_formats(content_type: str | None) -> set[str]:
|
||||
"""Return the configured formats for the requested content type."""
|
||||
if check_audiobook(content_type):
|
||||
return _normalize_formats(
|
||||
config.get("SUPPORTED_AUDIOBOOK_FORMATS", None), _DEFAULT_AUDIOBOOK_FORMATS
|
||||
)
|
||||
return _normalize_formats(config.get("SUPPORTED_FORMATS", None), _DEFAULT_EBOOK_FORMATS)
|
||||
|
||||
|
||||
def _coerce_int(value: object) -> int:
|
||||
if isinstance(value, bool):
|
||||
return 0
|
||||
if isinstance(value, (int, float)):
|
||||
return int(value)
|
||||
if isinstance(value, str):
|
||||
try:
|
||||
return int(float(value.strip()))
|
||||
except ValueError:
|
||||
return 0
|
||||
return 0
|
||||
|
||||
|
||||
def iter_peer_files(response: dict[str, Any]) -> list[PeerFile]:
|
||||
"""Extract the downloadable files from one slskd search response.
|
||||
|
||||
Locked files (the peer requires a friend/user list) are skipped: slskd would
|
||||
just queue them forever.
|
||||
"""
|
||||
username = str(response.get("username") or "").strip()
|
||||
raw_files = response.get("files")
|
||||
if not username or not isinstance(raw_files, list):
|
||||
return []
|
||||
|
||||
peer_files: list[PeerFile] = []
|
||||
for raw in raw_files:
|
||||
if not isinstance(raw, dict) or raw.get("isLocked"):
|
||||
continue
|
||||
filename = str(raw.get("filename") or "").strip()
|
||||
if not filename:
|
||||
continue
|
||||
directory, basename = split_remote_path(filename)
|
||||
if not basename:
|
||||
continue
|
||||
peer_files.append(
|
||||
PeerFile(
|
||||
username=username,
|
||||
filename=filename,
|
||||
directory=directory,
|
||||
basename=basename,
|
||||
extension=file_extension(basename),
|
||||
size=max(_coerce_int(raw.get("size")), 0),
|
||||
)
|
||||
)
|
||||
return peer_files
|
||||
|
||||
|
||||
def _peer_extra(response: dict[str, Any]) -> dict[str, Any]:
|
||||
upload_speed = _coerce_int(response.get("uploadSpeed"))
|
||||
queue_length = _coerce_int(response.get("queueLength"))
|
||||
has_free_slot = bool(response.get("hasFreeUploadSlot"))
|
||||
if has_free_slot:
|
||||
availability = "Free slot"
|
||||
elif queue_length > 0:
|
||||
availability = f"Queue {queue_length}"
|
||||
else:
|
||||
availability = "Busy"
|
||||
return {
|
||||
"username": str(response.get("username") or ""),
|
||||
"upload_speed": upload_speed,
|
||||
"queue_length": queue_length,
|
||||
"has_free_upload_slot": has_free_slot,
|
||||
"availability": availability,
|
||||
}
|
||||
|
||||
|
||||
def _strip_extension(basename: str) -> str:
|
||||
if "." not in basename:
|
||||
return basename
|
||||
stem = basename.rsplit(".", 1)[0]
|
||||
return stem or basename
|
||||
|
||||
|
||||
def _path_text(directory: str) -> str:
|
||||
"""A remote folder path as plain words. A Soulseek result has no author field, but the
|
||||
folders nearly always name one ("Books\\Frank Herbert\\Dune"), so the path is what
|
||||
auto-download's author check reads (it looks at ``extra["author"]``)."""
|
||||
return " ".join(part for part in re.split(r"[\\/]", directory) if part and part != "@@")
|
||||
|
||||
|
||||
def file_release(peer_file: PeerFile, response: dict[str, Any], content_type: str) -> Release:
|
||||
"""Build a single-file release."""
|
||||
is_audio = check_audiobook(content_type)
|
||||
return Release(
|
||||
source=SOURCE_NAME,
|
||||
source_id=make_source_id(peer_file.username, peer_file.filename),
|
||||
title=_strip_extension(peer_file.basename),
|
||||
format=peer_file.extension or None,
|
||||
language=None,
|
||||
size=_parse_size(peer_file.size),
|
||||
size_bytes=peer_file.size or None,
|
||||
download_url=None,
|
||||
info_url=None,
|
||||
protocol=ReleaseProtocol.SOULSEEK,
|
||||
indexer=peer_file.username,
|
||||
content_type="audiobook" if is_audio else "ebook",
|
||||
extra={
|
||||
**_peer_extra(response),
|
||||
"directory": peer_file.directory,
|
||||
"directory_name": remote_directory_name(peer_file.directory),
|
||||
"author": _path_text(peer_file.directory),
|
||||
"title_raw": f"{remote_directory_name(peer_file.directory)} {_strip_extension(peer_file.basename)}".strip(),
|
||||
"file_count": 1,
|
||||
"files": [{"filename": peer_file.filename, "size": peer_file.size}],
|
||||
"formats": [peer_file.extension] if peer_file.extension else [],
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def directory_release(
|
||||
files: list[PeerFile], response: dict[str, Any], content_type: str
|
||||
) -> Release:
|
||||
"""Build a multi-file release from every matching file in one peer directory."""
|
||||
first = files[0]
|
||||
total_size = sum(f.size for f in files)
|
||||
formats: list[str] = []
|
||||
for f in files:
|
||||
if f.extension and f.extension not in formats:
|
||||
formats.append(f.extension)
|
||||
formats.sort(key=lambda ext: -sum(1 for f in files if f.extension == ext))
|
||||
directory_name = remote_directory_name(first.directory) or first.username
|
||||
is_audio = check_audiobook(content_type)
|
||||
|
||||
return Release(
|
||||
source=SOURCE_NAME,
|
||||
source_id=make_source_id(first.username, first.directory + "\\"),
|
||||
title=f"{directory_name} ({len(files)} files)",
|
||||
format=formats[0] if formats else None,
|
||||
language=None,
|
||||
size=_parse_size(total_size),
|
||||
size_bytes=total_size or None,
|
||||
download_url=None,
|
||||
info_url=None,
|
||||
protocol=ReleaseProtocol.SOULSEEK,
|
||||
indexer=first.username,
|
||||
content_type="audiobook" if is_audio else "ebook",
|
||||
extra={
|
||||
**_peer_extra(response),
|
||||
"directory": first.directory,
|
||||
"directory_name": directory_name,
|
||||
"author": _path_text(first.directory),
|
||||
"title_raw": directory_name,
|
||||
"file_count": len(files),
|
||||
"files": [{"filename": f.filename, "size": f.size} for f in files],
|
||||
"formats": formats,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def build_releases(
|
||||
responses: list[dict[str, Any]],
|
||||
*,
|
||||
content_type: str,
|
||||
formats: set[str],
|
||||
) -> list[Release]:
|
||||
"""Turn slskd search responses into releases, filtered to ``formats``."""
|
||||
releases: list[Release] = []
|
||||
seen_ids: set[str] = set()
|
||||
group_audio = check_audiobook(content_type)
|
||||
|
||||
for response in responses:
|
||||
matching = [f for f in iter_peer_files(response) if f.extension in formats]
|
||||
if not matching:
|
||||
continue
|
||||
|
||||
if not group_audio:
|
||||
candidates = [file_release(f, response, content_type) for f in matching]
|
||||
else:
|
||||
by_directory: dict[str, list[PeerFile]] = {}
|
||||
for peer_file in matching:
|
||||
# A bare filename has no folder, so grouping on "" would bundle unrelated
|
||||
# files from the same peer into one audiobook; give each its own group.
|
||||
key = peer_file.directory or f"\0{peer_file.filename}"
|
||||
by_directory.setdefault(key, []).append(peer_file)
|
||||
candidates = []
|
||||
for grouped in by_directory.values():
|
||||
# An archive is a whole audiobook on its own; keep those separate from
|
||||
# the loose chapter files next to them.
|
||||
archives = [f for f in grouped if f.extension in ARCHIVE_FORMATS]
|
||||
# An m4b/m4a is a whole book in one file too. Next to a folder of chapters it
|
||||
# is a different copy of the book, not another chapter.
|
||||
containers = [f for f in grouped if f.extension in CONTAINER_FORMATS]
|
||||
loose = [
|
||||
f
|
||||
for f in grouped
|
||||
if f.extension not in ARCHIVE_FORMATS and f.extension not in CONTAINER_FORMATS
|
||||
]
|
||||
candidates.extend(
|
||||
file_release(f, response, content_type) for f in [*archives, *containers]
|
||||
)
|
||||
if len(loose) == 1:
|
||||
candidates.append(file_release(loose[0], response, content_type))
|
||||
elif loose:
|
||||
candidates.append(directory_release(loose, response, content_type))
|
||||
|
||||
for release in candidates:
|
||||
if release.source_id in seen_ids:
|
||||
continue
|
||||
seen_ids.add(release.source_id)
|
||||
releases.append(release)
|
||||
|
||||
return releases
|
||||
|
||||
|
||||
def merge_responses(batches: list[list[dict[str, Any]]]) -> list[dict[str, Any]]:
|
||||
"""Combine the responses of several searches, one per peer.
|
||||
|
||||
Different queries can return different parts of the same peer's folder (the title+author
|
||||
query some chapters, the bare title others). Building releases per query would keep the
|
||||
first partial folder and drop the rest, so files are merged by peer first. The peer's
|
||||
slot and queue details are those of its first answer.
|
||||
"""
|
||||
merged: dict[str, dict[str, Any]] = {}
|
||||
for batch in batches:
|
||||
for response in batch:
|
||||
username = str(response.get("username") or "").strip()
|
||||
if not username:
|
||||
continue
|
||||
known = merged.get(username)
|
||||
if known is None:
|
||||
merged[username] = {**response, "files": list(response.get("files") or [])}
|
||||
continue
|
||||
have = {str(f.get("filename") or "") for f in known["files"] if isinstance(f, dict)}
|
||||
for raw in response.get("files") or []:
|
||||
name = str(raw.get("filename") or "") if isinstance(raw, dict) else ""
|
||||
if name and name not in have:
|
||||
known["files"].append(raw)
|
||||
have.add(name)
|
||||
return list(merged.values())
|
||||
|
||||
|
||||
def rank_key(release: Release) -> tuple:
|
||||
"""Sort releases so the peers most likely to actually serve the file come first."""
|
||||
extra = release.extra
|
||||
return (
|
||||
0 if extra.get("has_free_upload_slot") else 1,
|
||||
_coerce_int(extra.get("queue_length")),
|
||||
-_coerce_int(extra.get("upload_speed")),
|
||||
release.title.lower(),
|
||||
)
|
||||
|
||||
|
||||
def _config_int(key: str, default: int, *, minimum: int, maximum: int) -> int:
|
||||
value = _coerce_int(config.get(key, default))
|
||||
if value <= 0:
|
||||
value = default
|
||||
return max(minimum, min(maximum, value))
|
||||
|
||||
|
||||
def build_client_from_config() -> SlskdClient | None:
|
||||
"""Build a client from settings, or None when the URL is missing/invalid."""
|
||||
raw_url = str(config.get("SLSKD_URL", "") or "").strip()
|
||||
if not raw_url:
|
||||
return None
|
||||
url = normalize_http_url(raw_url)
|
||||
if not url:
|
||||
return None
|
||||
api_key = str(config.get("SLSKD_API_KEY", "") or "").strip()
|
||||
return SlskdClient(url, api_key)
|
||||
|
||||
|
||||
def _plan_queries(plan: ReleaseSearchPlan) -> list[str]:
|
||||
"""Search strings to try, most specific first, with duplicates removed."""
|
||||
queries: list[str] = []
|
||||
if plan.manual_query:
|
||||
queries.append(plan.manual_query)
|
||||
for variant in plan.title_variants:
|
||||
queries.append(variant.query)
|
||||
# Peers rarely name files with the author; the bare title is the fallback.
|
||||
if variant.author:
|
||||
queries.append(variant.title)
|
||||
|
||||
unique: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for query in queries:
|
||||
cleaned = " ".join(str(query or "").split())
|
||||
key = cleaned.lower()
|
||||
if not cleaned or key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
unique.append(cleaned)
|
||||
return unique[:MAX_QUERY_VARIANTS]
|
||||
|
||||
|
||||
@register_source(SOURCE_NAME)
|
||||
class SlskdSource(ReleaseSource):
|
||||
"""Release source for the Soulseek network via slskd."""
|
||||
|
||||
name = SOURCE_NAME
|
||||
display_name = DISPLAY_NAME
|
||||
supported_content_types: ClassVar[list[str]] = ["ebook", "audiobook"]
|
||||
|
||||
def get_column_config(self) -> ReleaseColumnConfig:
|
||||
return ReleaseColumnConfig(
|
||||
columns=[
|
||||
ColumnSchema(
|
||||
key="extra.username",
|
||||
label="Peer",
|
||||
render_type=ColumnRenderType.TEXT,
|
||||
align=ColumnAlign.LEFT,
|
||||
width="minmax(100px, 1fr)",
|
||||
hide_mobile=False,
|
||||
sortable=True,
|
||||
sort_key="indexer",
|
||||
),
|
||||
ColumnSchema(
|
||||
key="extra.availability",
|
||||
label="Slot",
|
||||
render_type=ColumnRenderType.TEXT,
|
||||
align=ColumnAlign.CENTER,
|
||||
width="80px",
|
||||
hide_mobile=True,
|
||||
),
|
||||
ColumnSchema(
|
||||
key="format",
|
||||
label="Format",
|
||||
render_type=ColumnRenderType.BADGE,
|
||||
align=ColumnAlign.CENTER,
|
||||
width="70px",
|
||||
hide_mobile=False,
|
||||
color_hint=ColumnColorHint(type="map", value="format"),
|
||||
uppercase=True,
|
||||
sortable=True,
|
||||
),
|
||||
ColumnSchema(
|
||||
key="size",
|
||||
label="Size",
|
||||
render_type=ColumnRenderType.SIZE,
|
||||
align=ColumnAlign.CENTER,
|
||||
width="80px",
|
||||
hide_mobile=False,
|
||||
sortable=True,
|
||||
sort_key="size_bytes",
|
||||
),
|
||||
],
|
||||
grid_template="minmax(0,2fr) minmax(100px,1fr) 80px 70px 80px",
|
||||
leading_cell=LeadingCellConfig(type=LeadingCellType.NONE),
|
||||
cache_ttl_seconds=600,
|
||||
supported_filters=["format"],
|
||||
extra_sort_options=[
|
||||
SortOption(label="Upload speed", sort_key="extra.upload_speed"),
|
||||
SortOption(
|
||||
label="Queue length", sort_key="extra.queue_length", default_direction="asc"
|
||||
),
|
||||
],
|
||||
action_button=SourceActionButton(label="Search again"),
|
||||
)
|
||||
|
||||
def _get_client(self) -> SlskdClient | None:
|
||||
return build_client_from_config()
|
||||
|
||||
def search(
|
||||
self,
|
||||
book: BookMetadata,
|
||||
plan: ReleaseSearchPlan,
|
||||
*,
|
||||
expand_search: bool = False,
|
||||
content_type: str = "ebook",
|
||||
) -> list[Release]:
|
||||
"""Search Soulseek for files matching the book."""
|
||||
client = self._get_client()
|
||||
if client is None:
|
||||
logger.warning("slskd not configured - skipping search")
|
||||
return []
|
||||
|
||||
queries = _plan_queries(plan)
|
||||
if not queries:
|
||||
logger.warning("slskd: no search query available")
|
||||
return []
|
||||
|
||||
formats = get_supported_formats(content_type)
|
||||
wait_seconds = _config_int(
|
||||
"SLSKD_SEARCH_TIMEOUT", DEFAULT_SEARCH_WAIT_SECONDS, minimum=3, maximum=120
|
||||
)
|
||||
response_limit = _config_int(
|
||||
"SLSKD_RESPONSE_LIMIT", DEFAULT_RESPONSE_LIMIT, minimum=10, maximum=1000
|
||||
)
|
||||
# Searches now run to slskd's own end rather than being cut at the wait, so the
|
||||
# budget must let every planned query start even after slow ones.
|
||||
budget = max(SLSKD_SEARCH_TIMEOUT_SECONDS, len(queries) * (wait_seconds + 30.0))
|
||||
deadline = time.monotonic() + budget
|
||||
|
||||
releases: list[Release] = []
|
||||
answered: list[list[dict[str, Any]]] = []
|
||||
# Expanding keeps going through every variant even after one produced results.
|
||||
for index, query in enumerate(queries, start=1):
|
||||
if releases and not expand_search:
|
||||
break
|
||||
if time.monotonic() >= deadline:
|
||||
logger.warning("slskd search timed out before query %d/%d", index, len(queries))
|
||||
break
|
||||
logger.debug("slskd query %d/%d: '%s'", index, len(queries), query)
|
||||
try:
|
||||
responses = client.search(
|
||||
query,
|
||||
wait_seconds=wait_seconds,
|
||||
response_limit=response_limit,
|
||||
)
|
||||
except Exception:
|
||||
logger.exception("slskd search failed for '%s'", query)
|
||||
continue
|
||||
|
||||
answered.append(responses)
|
||||
releases = build_releases(
|
||||
merge_responses(answered), content_type=content_type, formats=formats
|
||||
)
|
||||
|
||||
releases.sort(key=rank_key)
|
||||
if releases:
|
||||
peers = len({r.indexer for r in releases})
|
||||
logger.info("slskd: %d results from %d peers", len(releases), peers)
|
||||
else:
|
||||
logger.debug("slskd: no results found")
|
||||
return releases
|
||||
|
||||
def is_available(self) -> bool:
|
||||
if not config.get("SLSKD_ENABLED", False):
|
||||
return False
|
||||
if not str(config.get("SLSKD_API_KEY", "") or "").strip():
|
||||
return False
|
||||
return bool(normalize_http_url(str(config.get("SLSKD_URL", "") or "")))
|
||||
@@ -474,7 +474,7 @@ export interface Release {
|
||||
size_bytes?: number; // Size in bytes for sorting
|
||||
download_url?: string;
|
||||
info_url?: string; // Link to release info page (e.g., tracker page) - makes title clickable
|
||||
protocol?: 'http' | 'torrent' | 'nzb' | 'dcc';
|
||||
protocol?: 'http' | 'torrent' | 'nzb' | 'dcc' | 'soulseek';
|
||||
indexer?: string; // Display name for the source/indexer
|
||||
seeders?: number; // For torrents
|
||||
peers?: string; // For torrents: "seeders/leechers" display string
|
||||
|
||||
@@ -117,6 +117,7 @@ export function getProtocolDotColor(protocol?: string): string {
|
||||
const p = protocol.toLowerCase();
|
||||
if (p === 'torrent') return 'bg-orange-500';
|
||||
if (p === 'nzb' || p === 'usenet') return 'bg-sky-500';
|
||||
if (p === 'soulseek') return 'bg-emerald-500';
|
||||
return 'bg-gray-400';
|
||||
}
|
||||
|
||||
|
||||
Whitespace-only changes.
@@ -0,0 +1,24 @@
|
||||
"""
|
||||
slskd test configuration.
|
||||
|
||||
Stubs out optional heavy dependencies (flask_socketio) that are not available
|
||||
in lightweight dev/CI environments. These are only needed by the IRC source,
|
||||
which is unrelated to the slskd plugin under test.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import types
|
||||
|
||||
|
||||
def _stub_module(name: str) -> None:
|
||||
"""Insert a minimal stub module into sys.modules if not already present."""
|
||||
if name not in sys.modules:
|
||||
stub = types.ModuleType(name)
|
||||
# Provide no-op stand-ins for the symbols IRC source imports.
|
||||
stub.SocketIO = object
|
||||
stub.join_room = lambda *a, **kw: None
|
||||
stub.leave_room = lambda *a, **kw: None
|
||||
sys.modules[name] = stub
|
||||
|
||||
|
||||
_stub_module("flask_socketio")
|
||||
@@ -0,0 +1,556 @@
|
||||
"""Unit tests for the slskd API client."""
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
import requests
|
||||
|
||||
from shelfmark.release_sources.slskd.api import (
|
||||
SEARCH_FINISH_MARGIN_SECONDS,
|
||||
SEARCH_SAVE_WAIT_SECONDS,
|
||||
SlskdAuthError,
|
||||
SlskdBatchUnsupportedError,
|
||||
SlskdClient,
|
||||
SlskdError,
|
||||
_flatten_transfers,
|
||||
transfer_is_complete,
|
||||
transfer_succeeded,
|
||||
)
|
||||
|
||||
# ── helpers ────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _make_response(payload=None, status: int = 200, *, raw: bytes | None = None) -> MagicMock:
|
||||
r = MagicMock(spec=requests.Response)
|
||||
r.status_code = status
|
||||
r.ok = status < 400
|
||||
if raw is not None:
|
||||
r.content = raw
|
||||
r.json.side_effect = ValueError("not json")
|
||||
elif payload is None:
|
||||
r.content = b""
|
||||
r.json.side_effect = ValueError("no content")
|
||||
else:
|
||||
r.content = b"{}"
|
||||
r.json.return_value = payload
|
||||
r.raise_for_status = MagicMock()
|
||||
if status >= 400:
|
||||
r.raise_for_status.side_effect = requests.exceptions.HTTPError(response=r)
|
||||
return r
|
||||
|
||||
|
||||
def _app_payload(*, logged_in: bool = True, state: str = "Connected, LoggedIn") -> dict:
|
||||
return {
|
||||
"version": {"full": "0.26.0.0 (0.26.0.0+e42a525d)", "current": "0.26.0.0"},
|
||||
"server": {"state": state, "isConnected": True, "isLoggedIn": logged_in},
|
||||
}
|
||||
|
||||
|
||||
TRANSFERS_PAYLOAD = {
|
||||
"username": "peer",
|
||||
"directories": [
|
||||
{
|
||||
"directory": "@@share\\Books\\Author",
|
||||
"fileCount": 2,
|
||||
"files": [
|
||||
{"id": "t1", "filename": "@@share\\Books\\Author\\one.epub", "state": "InProgress"},
|
||||
{
|
||||
"id": "t2",
|
||||
"filename": "@@share\\Books\\Author\\two.epub",
|
||||
"state": "Completed, Succeeded",
|
||||
},
|
||||
],
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
# ── URL construction / auth ────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestApiUrl:
|
||||
def test_appends_api_prefix(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
assert client._api_url("application") == "http://slskd:5030/api/v0/application"
|
||||
|
||||
def test_strips_trailing_slash(self):
|
||||
client = SlskdClient("http://slskd:5030/", "key")
|
||||
assert client._api_url("/searches") == "http://slskd:5030/api/v0/searches"
|
||||
|
||||
def test_does_not_double_prefix(self):
|
||||
client = SlskdClient("http://slskd:5030/api/v0", "key")
|
||||
assert client._api_url("options") == "http://slskd:5030/api/v0/options"
|
||||
|
||||
def test_adds_scheme_when_missing(self):
|
||||
client = SlskdClient("slskd:5030", "key")
|
||||
assert client.base_url.startswith("http")
|
||||
|
||||
|
||||
class TestRequest:
|
||||
def test_sends_api_key_header(self):
|
||||
client = SlskdClient("http://slskd:5030", "secret")
|
||||
with patch.object(
|
||||
client._session, "request", return_value=_make_response(_app_payload())
|
||||
) as req:
|
||||
client.get_application()
|
||||
kwargs = req.call_args.kwargs
|
||||
assert kwargs["headers"]["X-API-Key"] == "secret"
|
||||
assert kwargs["method"] == "GET" if "method" in kwargs else req.call_args.args[0] == "GET"
|
||||
|
||||
def test_omits_header_without_key(self):
|
||||
client = SlskdClient("http://slskd:5030", "")
|
||||
with patch.object(
|
||||
client._session, "request", return_value=_make_response(_app_payload())
|
||||
) as req:
|
||||
client.get_application()
|
||||
assert "X-API-Key" not in req.call_args.kwargs["headers"]
|
||||
|
||||
def test_401_raises_auth_error(self):
|
||||
client = SlskdClient("http://slskd:5030", "bad")
|
||||
with (
|
||||
patch.object(client._session, "request", return_value=_make_response(status=401)),
|
||||
pytest.raises(SlskdAuthError),
|
||||
):
|
||||
client.get_application()
|
||||
|
||||
def test_500_raises_http_error(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with (
|
||||
patch.object(client._session, "request", return_value=_make_response(status=500)),
|
||||
pytest.raises(requests.exceptions.HTTPError),
|
||||
):
|
||||
client.get_application()
|
||||
|
||||
def test_non_json_body_raises_slskd_error(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with (
|
||||
patch.object(client._session, "request", return_value=_make_response(raw=b"<html>")),
|
||||
pytest.raises(SlskdError),
|
||||
):
|
||||
client.get_application()
|
||||
|
||||
|
||||
# ── test_connection ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestTestConnection:
|
||||
def test_success_when_logged_in(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with patch.object(client, "get_application", return_value=_app_payload()):
|
||||
ok, msg = client.test_connection()
|
||||
assert ok is True
|
||||
assert "0.26.0" in msg
|
||||
assert "logged in" in msg.lower()
|
||||
|
||||
def test_failure_when_not_logged_in(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
payload = _app_payload(logged_in=False, state="Disconnected")
|
||||
with patch.object(client, "get_application", return_value=payload):
|
||||
ok, msg = client.test_connection()
|
||||
assert ok is False
|
||||
assert "Disconnected" in msg
|
||||
assert "not logged in" in msg
|
||||
|
||||
def test_invalid_api_key(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with patch.object(client, "get_application", side_effect=SlskdAuthError("no")):
|
||||
ok, msg = client.test_connection()
|
||||
assert ok is False
|
||||
assert msg == "Invalid API key"
|
||||
|
||||
def test_connection_error(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with patch.object(
|
||||
client, "get_application", side_effect=requests.exceptions.ConnectionError()
|
||||
):
|
||||
ok, msg = client.test_connection()
|
||||
assert ok is False
|
||||
assert "Could not connect" in msg
|
||||
|
||||
def test_http_error(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
err = requests.exceptions.HTTPError(response=_make_response(status=502))
|
||||
with patch.object(client, "get_application", side_effect=err):
|
||||
ok, msg = client.test_connection()
|
||||
assert ok is False
|
||||
assert "502" in msg
|
||||
|
||||
|
||||
# ── options ────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestGetDownloadDirectory:
|
||||
def test_reads_directories_downloads(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
payload = {"directories": {"downloads": "/downloads/complete", "incomplete": "/x"}}
|
||||
with patch.object(client, "_request", return_value=_make_response(payload)):
|
||||
assert client.get_download_directory() == "/downloads/complete"
|
||||
|
||||
def test_returns_none_when_missing(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with patch.object(client, "_request", return_value=_make_response({"web": {}})):
|
||||
assert client.get_download_directory() is None
|
||||
|
||||
|
||||
# ── searches ───────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestSearch:
|
||||
def test_start_search_posts_expected_payload(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with patch.object(
|
||||
client, "_request", return_value=_make_response({"id": "abc", "state": "InProgress"})
|
||||
) as req:
|
||||
search_id = client.start_search("dune herbert", timeout_ms=5000, response_limit=50)
|
||||
assert search_id == "abc"
|
||||
method, path = req.call_args.args
|
||||
assert (method, path) == ("POST", "searches")
|
||||
body = req.call_args.kwargs["json"]
|
||||
assert body["searchText"] == "dune herbert"
|
||||
assert body["searchTimeout"] == 5000
|
||||
assert body["responseLimit"] == 50
|
||||
|
||||
def test_start_search_without_id_raises(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with (
|
||||
patch.object(client, "_request", return_value=_make_response({})),
|
||||
pytest.raises(SlskdError),
|
||||
):
|
||||
client.start_search("x", timeout_ms=1000, response_limit=10)
|
||||
|
||||
@staticmethod
|
||||
def _search(states, *, responses=None, wait_seconds=15, poll_interval=1.0):
|
||||
"""Run client.search over scripted slskd states on a fake clock.
|
||||
|
||||
Returns (result, calls, clock): ``calls`` lists the client calls in order, and each
|
||||
sleep advances the clock, so a search "taking" minutes costs nothing.
|
||||
"""
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
clock = [0.0]
|
||||
calls: list[str] = []
|
||||
seq = iter(states)
|
||||
last = [None]
|
||||
|
||||
def _state(_id):
|
||||
calls.append("state")
|
||||
try:
|
||||
last[0] = next(seq)
|
||||
except StopIteration:
|
||||
pass
|
||||
return last[0]
|
||||
|
||||
def _responses(_id):
|
||||
calls.append("responses")
|
||||
return list(responses or [])
|
||||
|
||||
def _sleep(seconds):
|
||||
clock[0] += seconds
|
||||
|
||||
with (
|
||||
patch.object(client, "start_search", return_value="abc"),
|
||||
patch.object(client, "get_search", side_effect=_state),
|
||||
patch.object(client, "get_search_responses", side_effect=_responses),
|
||||
patch.object(client, "delete_search", side_effect=lambda _id: calls.append("delete")),
|
||||
patch("shelfmark.release_sources.slskd.api.time.sleep", side_effect=_sleep),
|
||||
patch(
|
||||
"shelfmark.release_sources.slskd.api.time.monotonic", side_effect=lambda: clock[0]
|
||||
),
|
||||
):
|
||||
result = client.search(
|
||||
"dune", wait_seconds=wait_seconds, response_limit=10, poll_interval=poll_interval
|
||||
)
|
||||
return result, calls, clock[0]
|
||||
|
||||
def test_search_waits_for_slskd_to_save_the_responses_then_deletes(self):
|
||||
running = {"isComplete": False, "state": "InProgress"}
|
||||
# Complete, but slskd has not written the responses yet: no endedAt.
|
||||
complete = {"isComplete": True, "state": "Completed, TimedOut", "responseCount": 3}
|
||||
saved = {**complete, "endedAt": "2026-10-12T05:00:00Z"}
|
||||
responses = [{"username": "peer", "files": []}]
|
||||
|
||||
result, calls, _ = self._search([running, complete, complete, saved], responses=responses)
|
||||
|
||||
assert result == responses
|
||||
# Nothing is read or deleted before endedAt appears, and the delete comes last.
|
||||
assert calls == ["state"] * 4 + ["responses", "delete"]
|
||||
|
||||
def test_a_busy_search_that_outlasts_the_wait_is_not_cut_short(self):
|
||||
running = {"isComplete": False, "state": "InProgress"}
|
||||
saved = {"isComplete": True, "endedAt": "2026-10-12T05:00:00Z", "responseCount": 40}
|
||||
responses = [{"username": "peer"}]
|
||||
|
||||
# Still running 40 s in, against a 15 s wait (peers keep resetting slskd's timer).
|
||||
result, calls, elapsed = self._search([running] * 40 + [saved], responses=responses)
|
||||
|
||||
assert result == responses
|
||||
assert elapsed > 15
|
||||
assert calls[-2:] == ["responses", "delete"]
|
||||
|
||||
def test_a_search_that_never_finishes_is_left_running_not_deleted(self):
|
||||
running = {"isComplete": False, "state": "InProgress"}
|
||||
|
||||
result, calls, elapsed = self._search([running])
|
||||
|
||||
assert result == []
|
||||
assert "delete" not in calls
|
||||
assert "responses" not in calls
|
||||
assert elapsed >= 15 + SEARCH_FINISH_MARGIN_SECONDS # it was given the whole margin
|
||||
|
||||
def test_a_search_with_no_responses_is_deleted_once_slskd_has_finished_it(self):
|
||||
saved = {"isComplete": True, "endedAt": "2026-10-12T05:00:00Z", "responseCount": 0}
|
||||
|
||||
result, calls, _ = self._search([saved])
|
||||
|
||||
assert result == []
|
||||
assert calls == ["state", "responses", "delete"]
|
||||
|
||||
def test_a_slskd_without_endedat_is_read_after_the_save_wait(self):
|
||||
complete = {"isComplete": True, "state": "Completed, TimedOut", "responseCount": 2}
|
||||
responses = [{"username": "peer"}]
|
||||
|
||||
result, calls, elapsed = self._search([complete], responses=responses)
|
||||
|
||||
assert result == responses
|
||||
assert elapsed >= SEARCH_SAVE_WAIT_SECONDS
|
||||
assert calls[-2:] == ["responses", "delete"]
|
||||
|
||||
def test_a_slskd_without_endedat_that_heard_peers_but_saved_nothing_is_left_alone(self):
|
||||
complete = {"isComplete": True, "state": "Completed, TimedOut", "responseCount": 2}
|
||||
|
||||
result, calls, _ = self._search([complete], responses=[])
|
||||
|
||||
assert result == []
|
||||
assert "delete" not in calls
|
||||
|
||||
def test_search_does_not_delete_when_polling_fails(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with (
|
||||
patch.object(client, "start_search", return_value="abc"),
|
||||
patch.object(client, "get_search", side_effect=requests.exceptions.Timeout()),
|
||||
patch.object(client, "delete_search") as delete,
|
||||
pytest.raises(requests.exceptions.Timeout),
|
||||
):
|
||||
client.search("dune", wait_seconds=1, response_limit=10)
|
||||
delete.assert_not_called() # slskd may still be writing the search
|
||||
|
||||
def test_empty_query_short_circuits(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with patch.object(client, "start_search") as start:
|
||||
assert client.search(" ", wait_seconds=1, response_limit=10) == []
|
||||
start.assert_not_called()
|
||||
|
||||
def test_get_search_responses_filters_non_dicts(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with patch.object(
|
||||
client, "_request", return_value=_make_response([{"username": "a"}, "junk", 3])
|
||||
):
|
||||
assert client.get_search_responses("abc") == [{"username": "a"}]
|
||||
|
||||
def test_delete_search_swallows_errors(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with patch.object(client, "_request", side_effect=requests.exceptions.ConnectionError()):
|
||||
client.delete_search("abc") # must not raise
|
||||
|
||||
|
||||
# ── transfers ──────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestTransfers:
|
||||
def test_enqueue_returns_enqueued_records(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
payload = {
|
||||
"enqueued": [{"id": "t1", "filename": "a\\b.epub", "state": "Queued, Locally"}],
|
||||
"failed": [],
|
||||
}
|
||||
with patch.object(client, "_request", return_value=_make_response(payload)) as req:
|
||||
result = client.enqueue_downloads("peer name", [{"filename": "a\\b.epub", "size": 5}])
|
||||
assert result == payload["enqueued"]
|
||||
method, path = req.call_args.args
|
||||
assert method == "POST"
|
||||
assert path == "transfers/downloads/peer%20name"
|
||||
assert req.call_args.kwargs["json"] == [{"filename": "a\\b.epub", "size": 5}]
|
||||
|
||||
def test_enqueue_raises_when_slskd_reports_failures(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
payload = {"enqueued": [], "failed": [{"filename": "x", "reason": "Not shared"}]}
|
||||
with (
|
||||
patch.object(client, "_request", return_value=_make_response(payload)),
|
||||
pytest.raises(SlskdError, match="Not shared"),
|
||||
):
|
||||
client.enqueue_downloads("peer", [{"filename": "x", "size": 1}])
|
||||
|
||||
def test_enqueue_falls_back_to_listing_on_empty_body(self):
|
||||
# Older slskd versions answer 201 with no body.
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with (
|
||||
patch.object(client, "_request", return_value=_make_response(None, status=201)),
|
||||
patch.object(
|
||||
client, "list_downloads", return_value=_flatten_transfers(TRANSFERS_PAYLOAD)
|
||||
),
|
||||
):
|
||||
result = client.enqueue_downloads(
|
||||
"peer", [{"filename": "@@share\\Books\\Author\\two.epub", "size": 1}]
|
||||
)
|
||||
assert [t["id"] for t in result] == ["t2"]
|
||||
|
||||
def test_list_downloads_flattens_directories(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with patch.object(client, "_request", return_value=_make_response(TRANSFERS_PAYLOAD)):
|
||||
transfers = client.list_downloads("peer")
|
||||
assert [t["id"] for t in transfers] == ["t1", "t2"]
|
||||
|
||||
def test_list_downloads_404_is_empty(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
err = requests.exceptions.HTTPError(response=_make_response(status=404))
|
||||
with patch.object(client, "_request", side_effect=err):
|
||||
assert client.list_downloads("peer") == []
|
||||
|
||||
def test_get_transfer_404_is_none(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
err = requests.exceptions.HTTPError(response=_make_response(status=404))
|
||||
with patch.object(client, "_request", side_effect=err):
|
||||
assert client.get_transfer("peer", "t1") is None
|
||||
|
||||
def test_get_transfer_returns_record(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
record = {"id": "t1", "state": "InProgress"}
|
||||
with patch.object(client, "_request", return_value=_make_response(record)) as req:
|
||||
assert client.get_transfer("peer", "t1") == record
|
||||
assert req.call_args.args == ("GET", "transfers/downloads/peer/t1")
|
||||
|
||||
def test_cancel_transfer_sends_remove_flag(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
with patch.object(client, "_request", return_value=_make_response(None, 204)) as req:
|
||||
assert client.cancel_transfer("peer", "t1", remove=True) is True
|
||||
assert req.call_args.args == ("DELETE", "transfers/downloads/peer/t1")
|
||||
assert req.call_args.kwargs["params"] == {"remove": "true"}
|
||||
|
||||
def test_cancel_transfer_404_is_false(self):
|
||||
client = SlskdClient("http://slskd:5030", "key")
|
||||
err = requests.exceptions.HTTPError(response=_make_response(status=404))
|
||||
with patch.object(client, "_request", side_effect=err):
|
||||
assert client.cancel_transfer("peer", "t1", remove=False) is False
|
||||
|
||||
|
||||
# ── helpers ────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestHelpers:
|
||||
def test_flatten_handles_list_of_users(self):
|
||||
data = [TRANSFERS_PAYLOAD, {"username": "other", "directories": []}]
|
||||
assert [t["id"] for t in _flatten_transfers(data)] == ["t1", "t2"]
|
||||
|
||||
def test_flatten_ignores_garbage(self):
|
||||
assert _flatten_transfers(None) == []
|
||||
assert _flatten_transfers({"directories": "nope"}) == []
|
||||
assert _flatten_transfers({"directories": [{"files": "nope"}, 3]}) == []
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("state", "complete", "succeeded"),
|
||||
[
|
||||
("Completed, Succeeded", True, True),
|
||||
("Completed, Errored", True, False),
|
||||
("Completed, Cancelled", True, False),
|
||||
("InProgress", False, False),
|
||||
("Queued, Remotely", False, False),
|
||||
(None, False, False),
|
||||
],
|
||||
)
|
||||
def test_state_predicates(self, state, complete, succeeded):
|
||||
assert transfer_is_complete(state) is complete
|
||||
assert transfer_succeeded(state) is succeeded
|
||||
|
||||
|
||||
# ── batch downloads, batches by id, events (fork additions) ─────────────────────
|
||||
|
||||
|
||||
class TestBatchDownloads:
|
||||
def _client(self):
|
||||
return SlskdClient("http://slskd:5030", "key")
|
||||
|
||||
def test_enqueue_batch_posts_files_into_a_relative_destination(self):
|
||||
client = self._client()
|
||||
reply = {
|
||||
"batch": {
|
||||
"id": "b-1",
|
||||
"transfers": [{"id": "t1", "filename": "a.epub", "state": "Requested"}],
|
||||
},
|
||||
"failures": [],
|
||||
}
|
||||
with patch.object(
|
||||
client._session, "request", return_value=_make_response(reply, 201)
|
||||
) as request:
|
||||
result = client.enqueue_batch(
|
||||
"peer", [{"filename": "a.epub", "size": 7}], "shelfmark/task-1"
|
||||
)
|
||||
|
||||
_, url = request.call_args.args[:2]
|
||||
assert url.endswith("/api/v0/transfers/downloads/batches")
|
||||
assert request.call_args.kwargs["json"] == {
|
||||
"username": "peer",
|
||||
"files": [{"filename": "a.epub", "size": 7}],
|
||||
"options": {"destination": "shelfmark/task-1"},
|
||||
}
|
||||
assert result["batch_id"] == "b-1"
|
||||
assert [t["id"] for t in result["transfers"]] == ["t1"]
|
||||
assert result["failures"] == []
|
||||
|
||||
def test_enqueue_batch_returns_the_files_slskd_refused(self):
|
||||
client = self._client()
|
||||
reply = {
|
||||
"batch": {"id": "b-1", "transfers": []},
|
||||
"failures": [{"filename": "a.epub", "message": "already queued"}],
|
||||
}
|
||||
with patch.object(client._session, "request", return_value=_make_response(reply, 207)):
|
||||
result = client.enqueue_batch("peer", [{"filename": "a.epub", "size": 7}], "d")
|
||||
|
||||
assert result["failures"] == [{"filename": "a.epub", "message": "already queued"}]
|
||||
|
||||
@pytest.mark.parametrize("status", [404, 405])
|
||||
def test_a_slskd_without_the_batch_endpoint_is_reported_as_unsupported(self, status):
|
||||
client = self._client()
|
||||
with (
|
||||
patch.object(client._session, "request", return_value=_make_response(status=status)),
|
||||
pytest.raises(SlskdBatchUnsupportedError),
|
||||
):
|
||||
client.enqueue_batch("peer", [{"filename": "a.epub", "size": 7}], "d")
|
||||
|
||||
def test_other_enqueue_errors_are_not_mistaken_for_an_old_slskd(self):
|
||||
client = self._client()
|
||||
with (
|
||||
patch.object(client._session, "request", return_value=_make_response(status=400)),
|
||||
pytest.raises(requests.exceptions.HTTPError),
|
||||
):
|
||||
client.enqueue_batch("peer", [{"filename": "a.epub", "size": 7}], "d")
|
||||
|
||||
def test_get_batch_returns_the_batch_and_none_when_it_is_gone(self):
|
||||
client = self._client()
|
||||
body = {"id": "b-1", "transfers": [{"id": "t1"}]}
|
||||
with patch.object(
|
||||
client._session,
|
||||
"request",
|
||||
side_effect=[_make_response(body), _make_response(status=404)],
|
||||
) as request:
|
||||
found = client.get_batch("b-1")
|
||||
gone = client.get_batch("b-2")
|
||||
|
||||
assert request.call_args_list[0].args[1].endswith("/transfers/downloads/batches/b-1")
|
||||
assert found == body
|
||||
assert gone is None
|
||||
|
||||
def test_get_events_reads_the_newest_events(self):
|
||||
client = self._client()
|
||||
events = [{"type": "DownloadFileComplete", "data": "{}"}]
|
||||
with patch.object(
|
||||
client._session, "request", return_value=_make_response(events)
|
||||
) as request:
|
||||
got = client.get_events(limit=50)
|
||||
|
||||
assert got == events
|
||||
assert request.call_args.args[1].endswith("/api/v0/events")
|
||||
assert request.call_args.kwargs["params"] == {"offset": 0, "limit": 50}
|
||||
|
||||
def test_get_events_tolerates_a_slskd_without_the_endpoint(self):
|
||||
client = self._client()
|
||||
with patch.object(client._session, "request", return_value=_make_response(status=404)):
|
||||
assert client.get_events() == []
|
||||
@@ -0,0 +1,742 @@
|
||||
"""Unit tests for the slskd download handler."""
|
||||
|
||||
from pathlib import Path
|
||||
from threading import Event
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from shelfmark.core.models import DownloadTask
|
||||
from shelfmark.release_sources import get_handler
|
||||
from shelfmark.release_sources.slskd.api import SlskdError
|
||||
from shelfmark.release_sources.slskd.handler import (
|
||||
SlskdDownloadSpec,
|
||||
SlskdHandler,
|
||||
_snapshot,
|
||||
local_path_for,
|
||||
resolve_local_download_root,
|
||||
spec_from_release_data,
|
||||
spec_from_task,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _original_enqueue_path(monkeypatch):
|
||||
"""These tests cover the original enqueue-and-poll-the-list path; isolated mode has its own
|
||||
tests in test_handler_isolated.py."""
|
||||
from shelfmark.release_sources.slskd.handler import SlskdHandler
|
||||
|
||||
monkeypatch.setattr(SlskdHandler, "_isolate_enabled", lambda self: False)
|
||||
|
||||
|
||||
# ── helpers ────────────────────────────────────────────────────────────────────
|
||||
|
||||
REMOTE_DIR = "@@share\\Books\\Author"
|
||||
FILE_A = f"{REMOTE_DIR}\\Book.epub"
|
||||
FILE_B = f"{REMOTE_DIR}\\Book.mobi"
|
||||
|
||||
|
||||
class ProgressRecorder:
|
||||
def __init__(self):
|
||||
self.progress_values: list[float] = []
|
||||
self.status_updates: list[tuple[str, str | None]] = []
|
||||
|
||||
def progress_callback(self, v: float):
|
||||
self.progress_values.append(v)
|
||||
|
||||
def status_callback(self, status: str, message: str | None):
|
||||
self.status_updates.append((status, message))
|
||||
|
||||
@property
|
||||
def last_status(self) -> str | None:
|
||||
return self.status_updates[-1][0] if self.status_updates else None
|
||||
|
||||
@property
|
||||
def last_message(self) -> str | None:
|
||||
return self.status_updates[-1][1] if self.status_updates else None
|
||||
|
||||
@property
|
||||
def statuses(self) -> list[str]:
|
||||
return [s[0] for s in self.status_updates]
|
||||
|
||||
|
||||
class FakeSlskd:
|
||||
"""In-memory stand-in for SlskdClient that scripts transfer state per poll."""
|
||||
|
||||
def __init__(self, *, download_dir="/downloads/complete", polls=None):
|
||||
self.download_dir = download_dir
|
||||
# Each poll returns the next list of transfer dicts; the last repeats forever.
|
||||
self.polls = list(polls or [])
|
||||
self.enqueued: list[tuple[str, list[dict]]] = []
|
||||
self.cancelled: list[tuple[str, str, bool]] = []
|
||||
self.enqueue_error: Exception | None = None
|
||||
self.poll_count = 0
|
||||
|
||||
def enqueue_downloads(self, username, files):
|
||||
if self.enqueue_error:
|
||||
raise self.enqueue_error
|
||||
self.enqueued.append((username, files))
|
||||
return [
|
||||
{"id": f"t{i}", "filename": f["filename"], "state": "Queued, Locally"}
|
||||
for i, f in enumerate(files)
|
||||
]
|
||||
|
||||
def list_downloads(self, username):
|
||||
self.poll_count += 1
|
||||
if not self.polls:
|
||||
return []
|
||||
index = min(self.poll_count - 1, len(self.polls) - 1)
|
||||
return list(self.polls[index])
|
||||
|
||||
def cancel_transfer(self, username, transfer_id, *, remove):
|
||||
"""Mimic slskd: a live transfer is cancelled (record kept), a complete one removed."""
|
||||
self.cancelled.append((username, transfer_id, remove))
|
||||
for poll in self.polls:
|
||||
for transfer in poll:
|
||||
if transfer.get("id") != transfer_id:
|
||||
continue
|
||||
if transfer["state"].startswith("Completed"):
|
||||
if remove:
|
||||
transfer["removed"] = True
|
||||
else:
|
||||
transfer["state"] = "Completed, Cancelled"
|
||||
return True
|
||||
|
||||
def get_download_directory(self):
|
||||
return self.download_dir
|
||||
|
||||
|
||||
def _transfer(filename, state, *, transferred=0, size=1000, tid=None, speed=0, **extra):
|
||||
return {
|
||||
"id": tid or f"id-{filename}",
|
||||
"filename": filename,
|
||||
"state": state,
|
||||
"bytesTransferred": transferred,
|
||||
"size": size,
|
||||
"averageSpeed": speed,
|
||||
"removed": False,
|
||||
**extra,
|
||||
}
|
||||
|
||||
|
||||
def _release_data(files=None, username="peer", directory=REMOTE_DIR):
|
||||
files = files if files is not None else [{"filename": FILE_A, "size": 1000}]
|
||||
return {
|
||||
"source": "slskd",
|
||||
"source_id": "slskd:abc",
|
||||
"title": "Book",
|
||||
"extra": {"username": username, "directory": directory, "files": files},
|
||||
}
|
||||
|
||||
|
||||
def _task(files=None, task_id="slskd:abc", username="peer"):
|
||||
files = files if files is not None else [{"filename": FILE_A, "size": 1000}]
|
||||
return DownloadTask(
|
||||
task_id=task_id,
|
||||
source="slskd",
|
||||
title="Book",
|
||||
retry_source_context={"username": username, "directory": REMOTE_DIR, "files": files},
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fast_handler(monkeypatch):
|
||||
"""Handler with no sleeps between polls / path checks."""
|
||||
monkeypatch.setattr(SlskdHandler, "_poll_interval", lambda self: 0)
|
||||
monkeypatch.setattr(SlskdHandler, "_completed_path_retry_interval", lambda self: 0)
|
||||
monkeypatch.setattr(SlskdHandler, "_completed_path_timeout_seconds", lambda self: 0)
|
||||
monkeypatch.setattr(SlskdHandler, "_queue_timeout_seconds", lambda self: 0)
|
||||
monkeypatch.setattr(SlskdHandler, "_cancel_remove_wait", lambda self: None)
|
||||
return SlskdHandler()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def config_values(monkeypatch):
|
||||
import shelfmark.release_sources.slskd.handler as mod
|
||||
|
||||
values: dict = {}
|
||||
monkeypatch.setattr(mod.config, "get", lambda k, d=None: values.get(k, d))
|
||||
return values
|
||||
|
||||
|
||||
def _mount(fast_handler, fake):
|
||||
fast_handler._get_client = lambda: fake # type: ignore[method-assign]
|
||||
|
||||
|
||||
def _write_completed(root: Path, *filenames: str) -> list[Path]:
|
||||
paths = []
|
||||
for filename in filenames:
|
||||
path = local_path_for(root, filename)
|
||||
assert path is not None
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_bytes(b"x" * 10)
|
||||
paths.append(path)
|
||||
return paths
|
||||
|
||||
|
||||
# ── spec parsing ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestSpec:
|
||||
def test_from_release_data(self):
|
||||
spec = spec_from_release_data(_release_data())
|
||||
assert spec == SlskdDownloadSpec(
|
||||
username="peer", directory=REMOTE_DIR, files=[{"filename": FILE_A, "size": 1000}]
|
||||
)
|
||||
assert spec.filenames == [FILE_A]
|
||||
assert spec.total_size == 1000
|
||||
|
||||
def test_from_release_data_without_extra(self):
|
||||
assert spec_from_release_data({"source": "slskd"}) is None
|
||||
assert spec_from_release_data({"extra": {"username": "peer", "files": []}}) is None
|
||||
assert (
|
||||
spec_from_release_data({"extra": {"username": "", "files": [{"filename": "x"}]}})
|
||||
is None
|
||||
)
|
||||
|
||||
def test_directory_derived_when_missing(self):
|
||||
spec = spec_from_release_data(_release_data(directory=""))
|
||||
assert spec is not None
|
||||
assert spec.directory == REMOTE_DIR
|
||||
|
||||
def test_from_task_round_trip(self):
|
||||
handler = SlskdHandler()
|
||||
fields = handler.build_retry_resolution_fields(_release_data())
|
||||
assert fields["retry_download_protocol"] == "soulseek"
|
||||
task = DownloadTask(task_id="x", source="slskd", title="t", **fields)
|
||||
assert spec_from_task(task) == spec_from_release_data(_release_data())
|
||||
|
||||
def test_from_task_without_context(self):
|
||||
assert spec_from_task(DownloadTask(task_id="x", source="slskd", title="t")) is None
|
||||
|
||||
def test_bad_sizes_become_zero(self):
|
||||
spec = spec_from_release_data(_release_data(files=[{"filename": FILE_A, "size": "n/a"}]))
|
||||
assert spec is not None
|
||||
assert spec.files[0]["size"] == 0
|
||||
|
||||
def test_build_retry_fields_empty_without_spec(self):
|
||||
assert SlskdHandler().build_retry_resolution_fields({"extra": {}}) == {}
|
||||
|
||||
def test_list_files(self):
|
||||
files = SlskdHandler().list_files(
|
||||
_release_data(files=[{"filename": FILE_A, "size": 10}, {"filename": FILE_B, "size": 0}])
|
||||
)
|
||||
assert files is not None
|
||||
assert [(f.path, f.size) for f in files] == [("Book.epub", 10), ("Book.mobi", None)]
|
||||
|
||||
def test_list_files_none_without_spec(self):
|
||||
assert SlskdHandler().list_files({"extra": {}}) is None
|
||||
|
||||
|
||||
# ── local path resolution ──────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestLocalPaths:
|
||||
def test_local_path_uses_last_folder_only(self):
|
||||
assert local_path_for(Path("/dl"), FILE_A) == Path("/dl/Author/Book.epub")
|
||||
|
||||
def test_local_path_forward_slashes(self):
|
||||
assert local_path_for(Path("/dl"), "share/Music/Album/01.mp3") == Path("/dl/Album/01.mp3")
|
||||
|
||||
def test_local_path_bare_file(self):
|
||||
assert local_path_for(Path("/dl"), "Book.epub") == Path("/dl/Book.epub")
|
||||
|
||||
def test_local_path_refuses_traversal(self):
|
||||
assert local_path_for(Path("/dl"), "a\\..\\Book.epub") is None
|
||||
assert local_path_for(Path("/dl"), "a\\Author\\..") is None
|
||||
assert local_path_for(Path("/dl"), "a\\Author\\") is None
|
||||
|
||||
def test_root_override_wins(self, config_values):
|
||||
config_values["SLSKD_DOWNLOAD_PATH"] = "/mnt/slskd "
|
||||
client = MagicMock()
|
||||
root, error = resolve_local_download_root(client)
|
||||
assert (root, error) == (Path("/mnt/slskd"), None)
|
||||
client.get_download_directory.assert_not_called()
|
||||
|
||||
def test_root_from_slskd_options(self, config_values):
|
||||
client = MagicMock()
|
||||
client.get_download_directory.return_value = "/downloads/complete"
|
||||
assert resolve_local_download_root(client) == (Path("/downloads/complete"), None)
|
||||
|
||||
def test_root_remapped_by_path_mapping(self, config_values):
|
||||
config_values["PROWLARR_REMOTE_PATH_MAPPINGS"] = [
|
||||
{"host": "slskd", "remotePath": "/downloads", "localPath": "/mnt/slskd"},
|
||||
{"host": "qbittorrent", "remotePath": "/downloads", "localPath": "/mnt/qb"},
|
||||
]
|
||||
client = MagicMock()
|
||||
client.get_download_directory.return_value = "/downloads/complete"
|
||||
assert resolve_local_download_root(client) == (Path("/mnt/slskd/complete"), None)
|
||||
|
||||
def test_root_error_when_slskd_reports_nothing(self, config_values):
|
||||
client = MagicMock()
|
||||
client.get_download_directory.return_value = None
|
||||
root, error = resolve_local_download_root(client)
|
||||
assert root is None
|
||||
assert error and "Downloads Path" in error
|
||||
|
||||
def test_root_error_when_slskd_unreachable(self, config_values):
|
||||
client = MagicMock()
|
||||
client.get_download_directory.side_effect = SlskdError("down")
|
||||
root, error = resolve_local_download_root(client)
|
||||
assert root is None
|
||||
assert error and "down" in error
|
||||
|
||||
|
||||
# ── snapshot aggregation ───────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestSnapshot:
|
||||
def _spec(self):
|
||||
return SlskdDownloadSpec(
|
||||
username="peer",
|
||||
directory=REMOTE_DIR,
|
||||
files=[{"filename": FILE_A, "size": 1000}, {"filename": FILE_B, "size": 1000}],
|
||||
)
|
||||
|
||||
def test_all_succeeded(self):
|
||||
snap = _snapshot(
|
||||
[
|
||||
_transfer(FILE_A, "Completed, Succeeded", transferred=1000),
|
||||
_transfer(FILE_B, "Completed, Succeeded", transferred=1000),
|
||||
],
|
||||
self._spec(),
|
||||
)
|
||||
assert snap.all_succeeded
|
||||
assert snap.bytes_transferred == 2000
|
||||
assert snap.failed_state is None
|
||||
|
||||
def test_partial_progress(self):
|
||||
snap = _snapshot(
|
||||
[
|
||||
_transfer(FILE_A, "InProgress", transferred=500, speed=100.0),
|
||||
_transfer(FILE_B, "Queued, Remotely", placeInQueue=3),
|
||||
],
|
||||
self._spec(),
|
||||
)
|
||||
assert not snap.all_succeeded
|
||||
assert snap.bytes_transferred == 500
|
||||
assert snap.in_progress == 1
|
||||
assert snap.queued_remotely == 1
|
||||
assert snap.queue_positions == [3]
|
||||
assert snap.speed == 100.0
|
||||
|
||||
def test_failure_detected(self):
|
||||
snap = _snapshot(
|
||||
[
|
||||
_transfer(FILE_A, "Completed, Succeeded", transferred=1000),
|
||||
_transfer(FILE_B, "Completed, Errored"),
|
||||
],
|
||||
self._spec(),
|
||||
)
|
||||
assert snap.failed_state == "Completed, Errored"
|
||||
assert snap.failed_filename == FILE_B
|
||||
|
||||
def test_ignores_unrelated_and_removed(self):
|
||||
snap = _snapshot(
|
||||
[
|
||||
_transfer("other\\thing.epub", "InProgress"),
|
||||
_transfer(FILE_A, "InProgress", removed=True),
|
||||
],
|
||||
self._spec(),
|
||||
)
|
||||
assert snap.found == 0
|
||||
|
||||
def test_prefers_live_record_over_old_failure(self):
|
||||
# slskd keeps a failed attempt around after a retry re-enqueues the file.
|
||||
snap = _snapshot(
|
||||
[
|
||||
_transfer(FILE_A, "Completed, Errored", requestedAt="2026-01-01T00:00:00"),
|
||||
_transfer(FILE_A, "InProgress", transferred=10, requestedAt="2026-01-01T00:01:00"),
|
||||
_transfer(FILE_B, "InProgress"),
|
||||
],
|
||||
self._spec(),
|
||||
)
|
||||
assert snap.failed_state is None
|
||||
assert snap.in_progress == 2
|
||||
|
||||
|
||||
# ── download lifecycle ─────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestDownload:
|
||||
def test_registered(self):
|
||||
assert isinstance(get_handler("slskd"), SlskdHandler)
|
||||
|
||||
def test_single_file_success_returns_client_path(self, fast_handler, config_values, tmp_path):
|
||||
config_values["SLSKD_DOWNLOAD_PATH"] = str(tmp_path)
|
||||
fake = FakeSlskd(
|
||||
polls=[
|
||||
[_transfer(FILE_A, "Queued, Remotely")],
|
||||
[_transfer(FILE_A, "InProgress", transferred=400, speed=2 * 1024 * 1024)],
|
||||
[_transfer(FILE_A, "Completed, Succeeded", transferred=1000, tid="done")],
|
||||
]
|
||||
)
|
||||
_mount(fast_handler, fake)
|
||||
(expected,) = _write_completed(tmp_path, FILE_A)
|
||||
rec = ProgressRecorder()
|
||||
|
||||
result = fast_handler.download(_task(), Event(), rec.progress_callback, rec.status_callback)
|
||||
|
||||
assert result == str(expected)
|
||||
assert fake.enqueued == [("peer", [{"filename": FILE_A, "size": 1000}])]
|
||||
assert rec.statuses[0] == "resolving"
|
||||
assert ("downloading", "Queued at peer") in rec.status_updates
|
||||
assert any(m and m.startswith("40% (2.0 MB/s)") for _, m in rec.status_updates)
|
||||
assert rec.progress_values[-1] == 100.0
|
||||
assert "error" not in rec.statuses
|
||||
assert fast_handler._cleanup_refs[_task().task_id][2] == ["done"]
|
||||
|
||||
def test_multi_file_success_stages_into_tmp(
|
||||
self, fast_handler, config_values, tmp_path, monkeypatch
|
||||
):
|
||||
import shelfmark.download.staging as staging
|
||||
|
||||
root = tmp_path / "slskd"
|
||||
staging_root = tmp_path / "staging"
|
||||
staging_root.mkdir()
|
||||
monkeypatch.setattr(staging, "get_staging_dir", lambda: staging_root)
|
||||
config_values["SLSKD_DOWNLOAD_PATH"] = str(root)
|
||||
files = [{"filename": FILE_A, "size": 10}, {"filename": FILE_B, "size": 10}]
|
||||
fake = FakeSlskd(
|
||||
polls=[
|
||||
[
|
||||
_transfer(FILE_A, "Completed, Succeeded", transferred=10),
|
||||
_transfer(FILE_B, "Completed, Succeeded", transferred=10),
|
||||
]
|
||||
]
|
||||
)
|
||||
_mount(fast_handler, fake)
|
||||
originals = _write_completed(root, FILE_A, FILE_B)
|
||||
# A stranger's file in the same per-folder directory must not come along.
|
||||
stranger = originals[0].parent / "stranger.epub"
|
||||
stranger.write_bytes(b"nope")
|
||||
rec = ProgressRecorder()
|
||||
|
||||
result = fast_handler.download(
|
||||
_task(files=files), Event(), rec.progress_callback, rec.status_callback
|
||||
)
|
||||
|
||||
assert result is not None
|
||||
staged = Path(result)
|
||||
assert staged.parent == staging_root
|
||||
assert sorted(p.name for p in staged.iterdir()) == ["Book.epub", "Book.mobi"]
|
||||
assert all(not p.exists() for p in originals)
|
||||
assert stranger.exists()
|
||||
|
||||
def test_failed_transfer_reports_outcome_and_removes(self, fast_handler, config_values):
|
||||
fake = FakeSlskd(polls=[[_transfer(FILE_A, "Completed, Rejected", tid="x")]])
|
||||
_mount(fast_handler, fake)
|
||||
rec = ProgressRecorder()
|
||||
|
||||
result = fast_handler.download(_task(), Event(), rec.progress_callback, rec.status_callback)
|
||||
|
||||
assert result is None
|
||||
assert rec.last_status == "error"
|
||||
assert rec.last_message == "Peer transfer failed: Rejected"
|
||||
assert fake.cancelled == [("peer", "x", True)]
|
||||
|
||||
def test_failed_file_named_in_multi_file_release(self, fast_handler, config_values):
|
||||
files = [{"filename": FILE_A, "size": 10}, {"filename": FILE_B, "size": 10}]
|
||||
fake = FakeSlskd(
|
||||
polls=[
|
||||
[
|
||||
_transfer(FILE_A, "Completed, Succeeded", transferred=10),
|
||||
_transfer(FILE_B, "Completed, TimedOut"),
|
||||
]
|
||||
]
|
||||
)
|
||||
_mount(fast_handler, fake)
|
||||
rec = ProgressRecorder()
|
||||
|
||||
result = fast_handler.download(
|
||||
_task(files=files), Event(), rec.progress_callback, rec.status_callback
|
||||
)
|
||||
|
||||
assert result is None
|
||||
assert rec.last_message == "Peer transfer failed: TimedOut (Book.mobi)"
|
||||
|
||||
def test_cancel_flag_cancels_transfers(self, fast_handler, config_values):
|
||||
fake = FakeSlskd(polls=[[_transfer(FILE_A, "InProgress", transferred=10, tid="live")]])
|
||||
_mount(fast_handler, fake)
|
||||
cancel = Event()
|
||||
rec = ProgressRecorder()
|
||||
|
||||
def status_callback(status, message):
|
||||
rec.status_callback(status, message)
|
||||
if status == "downloading":
|
||||
cancel.set()
|
||||
|
||||
result = fast_handler.download(_task(), cancel, rec.progress_callback, status_callback)
|
||||
|
||||
assert result is None
|
||||
assert rec.last_status == "cancelled"
|
||||
# First pass cancels the live transfer, second pass removes the leftover record.
|
||||
assert fake.cancelled == [("peer", "live", True), ("peer", "live", True)]
|
||||
assert all(t["removed"] for poll in fake.polls for t in poll)
|
||||
|
||||
def test_cancelled_before_start(self, fast_handler, config_values):
|
||||
fake = FakeSlskd()
|
||||
_mount(fast_handler, fake)
|
||||
cancel = Event()
|
||||
cancel.set()
|
||||
rec = ProgressRecorder()
|
||||
|
||||
assert (
|
||||
fast_handler.download(_task(), cancel, rec.progress_callback, rec.status_callback)
|
||||
is None
|
||||
)
|
||||
assert rec.last_status == "cancelled"
|
||||
assert fake.enqueued == []
|
||||
|
||||
def test_missing_spec_errors(self, fast_handler, config_values):
|
||||
_mount(fast_handler, FakeSlskd())
|
||||
rec = ProgressRecorder()
|
||||
task = DownloadTask(task_id="slskd:none", source="slskd", title="t")
|
||||
|
||||
assert (
|
||||
fast_handler.download(task, Event(), rec.progress_callback, rec.status_callback) is None
|
||||
)
|
||||
assert rec.last_status == "error"
|
||||
assert "missing" in (rec.last_message or "")
|
||||
|
||||
def test_unconfigured_client_errors(self, fast_handler, config_values):
|
||||
fast_handler._get_client = lambda: None # type: ignore[method-assign]
|
||||
rec = ProgressRecorder()
|
||||
|
||||
assert (
|
||||
fast_handler.download(_task(), Event(), rec.progress_callback, rec.status_callback)
|
||||
is None
|
||||
)
|
||||
assert rec.last_status == "error"
|
||||
assert rec.last_message == "slskd is not configured"
|
||||
|
||||
def test_enqueue_failure_errors(self, fast_handler, config_values):
|
||||
fake = FakeSlskd()
|
||||
fake.enqueue_error = SlskdError("slskd refused 1 file(s): Not shared")
|
||||
_mount(fast_handler, fake)
|
||||
rec = ProgressRecorder()
|
||||
|
||||
assert (
|
||||
fast_handler.download(_task(), Event(), rec.progress_callback, rec.status_callback)
|
||||
is None
|
||||
)
|
||||
assert rec.last_status == "error"
|
||||
assert "Not shared" in (rec.last_message or "")
|
||||
|
||||
def test_transfer_vanishing_errors_after_grace(self, fast_handler, config_values, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.handler as mod
|
||||
|
||||
monkeypatch.setattr(mod, "MAX_MISSING_POLLS", 3)
|
||||
fake = FakeSlskd(polls=[[]])
|
||||
_mount(fast_handler, fake)
|
||||
rec = ProgressRecorder()
|
||||
|
||||
assert (
|
||||
fast_handler.download(_task(), Event(), rec.progress_callback, rec.status_callback)
|
||||
is None
|
||||
)
|
||||
assert rec.last_status == "error"
|
||||
assert rec.last_message == "Transfer disappeared from slskd"
|
||||
# Three polls to give up, then the cleanup pass looks the transfers up again.
|
||||
assert fake.poll_count >= 3
|
||||
|
||||
def test_a_file_vanishing_from_a_multi_file_release_cancels_the_others(
|
||||
self, fast_handler, config_values, monkeypatch
|
||||
):
|
||||
import shelfmark.release_sources.slskd.handler as mod
|
||||
|
||||
monkeypatch.setattr(mod, "MAX_MISSING_POLLS", 3)
|
||||
files = [{"filename": FILE_A, "size": 1000}, {"filename": FILE_B, "size": 1000}]
|
||||
# FILE_B is never listed again; FILE_A keeps downloading.
|
||||
fake = FakeSlskd(polls=[[_transfer(FILE_A, "InProgress", transferred=10, tid="live")]])
|
||||
_mount(fast_handler, fake)
|
||||
rec = ProgressRecorder()
|
||||
|
||||
result = fast_handler.download(
|
||||
_task(files=files), Event(), rec.progress_callback, rec.status_callback
|
||||
)
|
||||
|
||||
assert result is None
|
||||
assert rec.last_message == "Transfer disappeared from slskd"
|
||||
assert ("peer", "live", True) in fake.cancelled
|
||||
|
||||
def test_queue_timeout_errors(self, fast_handler, config_values, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.handler as mod
|
||||
|
||||
monkeypatch.setattr(SlskdHandler, "_queue_timeout_seconds", lambda self: 120)
|
||||
clock = iter([0, 0, 200, 200, 200])
|
||||
monkeypatch.setattr(mod.time, "monotonic", lambda: next(clock))
|
||||
fake = FakeSlskd(polls=[[_transfer(FILE_A, "Queued, Remotely", tid="q")]])
|
||||
_mount(fast_handler, fake)
|
||||
rec = ProgressRecorder()
|
||||
|
||||
assert (
|
||||
fast_handler.download(_task(), Event(), rec.progress_callback, rec.status_callback)
|
||||
is None
|
||||
)
|
||||
assert rec.last_status == "error"
|
||||
assert rec.last_message == "Peer did not start the transfer within 2 minutes"
|
||||
assert fake.cancelled == [("peer", "q", True), ("peer", "q", True)]
|
||||
assert all(t["removed"] for poll in fake.polls for t in poll)
|
||||
|
||||
def test_completed_file_missing_errors(self, fast_handler, config_values, tmp_path):
|
||||
config_values["SLSKD_DOWNLOAD_PATH"] = str(tmp_path)
|
||||
fake = FakeSlskd(polls=[[_transfer(FILE_A, "Completed, Succeeded", transferred=1000)]])
|
||||
_mount(fast_handler, fake)
|
||||
rec = ProgressRecorder()
|
||||
|
||||
assert (
|
||||
fast_handler.download(_task(), Event(), rec.progress_callback, rec.status_callback)
|
||||
is None
|
||||
)
|
||||
assert rec.last_status == "error"
|
||||
assert "Completed file not found" in (rec.last_message or "")
|
||||
assert str(tmp_path / "Author" / "Book.epub") in (rec.last_message or "")
|
||||
|
||||
def test_completed_file_appears_after_wait(
|
||||
self, fast_handler, config_values, tmp_path, monkeypatch
|
||||
):
|
||||
config_values["SLSKD_DOWNLOAD_PATH"] = str(tmp_path)
|
||||
monkeypatch.setattr(SlskdHandler, "_completed_path_retry_interval", lambda self: 1)
|
||||
monkeypatch.setattr(SlskdHandler, "_completed_path_timeout_seconds", lambda self: 2)
|
||||
fake = FakeSlskd(polls=[[_transfer(FILE_A, "Completed, Succeeded", transferred=1000)]])
|
||||
_mount(fast_handler, fake)
|
||||
rec = ProgressRecorder()
|
||||
expected = local_path_for(tmp_path, FILE_A)
|
||||
assert expected is not None
|
||||
|
||||
cancel = MagicMock(spec=Event)
|
||||
cancel.is_set.return_value = False
|
||||
|
||||
def wait(timeout=None):
|
||||
_write_completed(tmp_path, FILE_A)
|
||||
return False
|
||||
|
||||
cancel.wait.side_effect = wait
|
||||
|
||||
result = fast_handler.download(_task(), cancel, rec.progress_callback, rec.status_callback)
|
||||
|
||||
assert result == str(expected)
|
||||
assert ("locating", "Waiting for completed files...") in rec.status_updates
|
||||
|
||||
def test_unsafe_remote_name_errors(self, fast_handler, config_values, tmp_path):
|
||||
config_values["SLSKD_DOWNLOAD_PATH"] = str(tmp_path)
|
||||
evil = "a\\..\\Book.epub"
|
||||
fake = FakeSlskd(polls=[[_transfer(evil, "Completed, Succeeded", transferred=10)]])
|
||||
_mount(fast_handler, fake)
|
||||
rec = ProgressRecorder()
|
||||
|
||||
task = _task(files=[{"filename": evil, "size": 10}])
|
||||
assert (
|
||||
fast_handler.download(task, Event(), rec.progress_callback, rec.status_callback) is None
|
||||
)
|
||||
assert rec.last_status == "error"
|
||||
assert "unsafe" in (rec.last_message or "")
|
||||
|
||||
def test_status_check_errors_are_retried(self, fast_handler, config_values, tmp_path):
|
||||
config_values["SLSKD_DOWNLOAD_PATH"] = str(tmp_path)
|
||||
fake = FakeSlskd(polls=[[_transfer(FILE_A, "Completed, Succeeded", transferred=1000)]])
|
||||
calls = {"n": 0}
|
||||
original = fake.list_downloads
|
||||
|
||||
def flaky(username):
|
||||
calls["n"] += 1
|
||||
if calls["n"] == 1:
|
||||
raise SlskdError("hiccup")
|
||||
return original(username)
|
||||
|
||||
fake.list_downloads = flaky
|
||||
_mount(fast_handler, fake)
|
||||
_write_completed(tmp_path, FILE_A)
|
||||
rec = ProgressRecorder()
|
||||
|
||||
result = fast_handler.download(_task(), Event(), rec.progress_callback, rec.status_callback)
|
||||
|
||||
assert result is not None
|
||||
assert ("resolving", "Waiting for slskd...") in rec.status_updates
|
||||
|
||||
|
||||
# ── post-processing cleanup ────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestCleanup:
|
||||
def _completed(self, fast_handler, config_values, tmp_path):
|
||||
config_values["SLSKD_DOWNLOAD_PATH"] = str(tmp_path)
|
||||
fake = FakeSlskd(
|
||||
polls=[[_transfer(FILE_A, "Completed, Succeeded", transferred=1000, tid="t")]]
|
||||
)
|
||||
_mount(fast_handler, fake)
|
||||
(path,) = _write_completed(tmp_path, FILE_A)
|
||||
rec = ProgressRecorder()
|
||||
task = _task()
|
||||
assert fast_handler.download(task, Event(), rec.progress_callback, rec.status_callback)
|
||||
return fake, task, path
|
||||
|
||||
def test_success_removes_transfer_and_empty_folder(self, fast_handler, config_values, tmp_path):
|
||||
fake, task, path = self._completed(fast_handler, config_values, tmp_path)
|
||||
path.unlink() # orchestrator moved it into the library
|
||||
|
||||
fast_handler.post_process_cleanup(task, success=True)
|
||||
|
||||
assert fake.cancelled == [("peer", "t", True)]
|
||||
assert not path.parent.exists()
|
||||
assert task.task_id not in fast_handler._cleanup_refs
|
||||
|
||||
def test_success_keeps_folder_with_other_files(self, fast_handler, config_values, tmp_path):
|
||||
_fake, task, path = self._completed(fast_handler, config_values, tmp_path)
|
||||
path.unlink()
|
||||
(path.parent / "other.epub").write_bytes(b"x")
|
||||
|
||||
fast_handler.post_process_cleanup(task, success=True)
|
||||
|
||||
assert path.parent.exists()
|
||||
|
||||
def test_success_respects_remove_setting(self, fast_handler, config_values, tmp_path):
|
||||
fake, task, _path = self._completed(fast_handler, config_values, tmp_path)
|
||||
config_values["SLSKD_REMOVE_COMPLETED"] = False
|
||||
|
||||
fast_handler.post_process_cleanup(task, success=True)
|
||||
|
||||
assert fake.cancelled == []
|
||||
|
||||
def test_failure_leaves_everything(self, fast_handler, config_values, tmp_path):
|
||||
fake, task, path = self._completed(fast_handler, config_values, tmp_path)
|
||||
|
||||
fast_handler.post_process_cleanup(task, success=False)
|
||||
|
||||
assert fake.cancelled == []
|
||||
assert path.exists()
|
||||
assert task.task_id not in fast_handler._cleanup_refs
|
||||
|
||||
def test_cleanup_without_refs_is_noop(self, fast_handler):
|
||||
fast_handler.post_process_cleanup(_task(), success=True)
|
||||
|
||||
def test_cancel_drops_refs(self, fast_handler, config_values, tmp_path):
|
||||
_fake, task, _path = self._completed(fast_handler, config_values, tmp_path)
|
||||
assert fast_handler.cancel(task.task_id) is True
|
||||
assert task.task_id not in fast_handler._cleanup_refs
|
||||
|
||||
|
||||
class TestCancelWhileLocatingFiles:
|
||||
def test_a_cancel_after_the_transfers_finished_still_removes_their_records(
|
||||
self, fast_handler, config_values, tmp_path
|
||||
):
|
||||
config_values["SLSKD_DOWNLOAD_PATH"] = str(tmp_path)
|
||||
cancel = Event()
|
||||
finished = [_transfer(FILE_A, "Completed, Succeeded", transferred=1000, tid="t0")]
|
||||
|
||||
class Fake(FakeSlskd):
|
||||
def list_downloads(self, username):
|
||||
result = super().list_downloads(username)
|
||||
cancel.set() # the cancel lands as the last transfer is reported finished
|
||||
return result
|
||||
|
||||
fake = Fake(polls=[finished])
|
||||
_mount(fast_handler, fake)
|
||||
rec = ProgressRecorder()
|
||||
|
||||
result = fast_handler.download(_task(), cancel, rec.progress_callback, rec.status_callback)
|
||||
|
||||
assert result is None
|
||||
assert rec.last_status == "cancelled"
|
||||
assert ("peer", "t0", True) in fake.cancelled
|
||||
@@ -0,0 +1,538 @@
|
||||
"""The handler's isolated mode: one folder per download, polled by id, confirmed on disk.
|
||||
|
||||
Several apps can share one slskd, and any of them can clear its list of finished transfers.
|
||||
A download that watched the list would then "lose" a file that had actually arrived. These
|
||||
tests pin the behaviour that does not: each download gets its own folder, each transfer is
|
||||
read by its id, the files on disk decide whether it finished, and the finished files stay in
|
||||
slskd's folder (still shared) while the orchestrator copies them.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from threading import Event
|
||||
|
||||
import pytest
|
||||
|
||||
import shelfmark.release_sources.slskd.handler as mod
|
||||
from shelfmark.core.models import DownloadTask
|
||||
from shelfmark.download.activity import ACTIVITY_GRACE_STATUS
|
||||
from shelfmark.release_sources.slskd.api import SlskdBatchUnsupportedError
|
||||
from shelfmark.release_sources.slskd.handler import SlskdHandler
|
||||
|
||||
REMOTE_DIR = "@@share\\Books\\Author"
|
||||
FILE_A = f"{REMOTE_DIR}\\Book One.epub"
|
||||
FILE_B = f"{REMOTE_DIR}\\Book Two.epub"
|
||||
FILES = [{"filename": FILE_A, "size": 5}, {"filename": FILE_B, "size": 7}]
|
||||
OK = "Completed, Succeeded"
|
||||
|
||||
|
||||
class Recorder:
|
||||
def __init__(self) -> None:
|
||||
self.progress: list[float] = []
|
||||
self.status: list[tuple[str, str | None]] = []
|
||||
|
||||
def progress_callback(self, value: float) -> None:
|
||||
self.progress.append(value)
|
||||
|
||||
def status_callback(self, status: str, message: str | None) -> None:
|
||||
self.status.append((status, message))
|
||||
|
||||
@property
|
||||
def real(self) -> list[tuple[str, str | None]]:
|
||||
"""Statuses a user would see, without the stall-grace signals."""
|
||||
return [(s, m) for s, m in self.status if s != ACTIVITY_GRACE_STATUS]
|
||||
|
||||
@property
|
||||
def statuses(self) -> list[str]:
|
||||
return [s for s, _ in self.real]
|
||||
|
||||
@property
|
||||
def graces(self) -> list[float]:
|
||||
return [float(m) for s, m in self.status if s == ACTIVITY_GRACE_STATUS]
|
||||
|
||||
|
||||
class FakeSlskd:
|
||||
"""slskd as the handler sees it: batches, transfers by id, a list, events."""
|
||||
|
||||
def __init__(self, root, *, script=None, write_on_success=True):
|
||||
self.root = root
|
||||
# script[filename] = list of states, one per get_transfer call; the last repeats.
|
||||
# A state of None means slskd no longer has the transfer (cleared from its list).
|
||||
self.script = script or {}
|
||||
self.write_on_success = write_on_success
|
||||
self.batches: list[dict] = []
|
||||
self.cancelled: list[tuple[str, str, bool]] = []
|
||||
self.list_calls = 0
|
||||
self.events: list[dict] = []
|
||||
self.refuse: list[dict] = []
|
||||
self.batch_error: Exception | None = None
|
||||
self._calls: dict[str, int] = {}
|
||||
self._written: set[str] = set()
|
||||
self._ids: dict[str, str] = {}
|
||||
self.on_get = None
|
||||
|
||||
def enqueue_batch(self, username, files, destination):
|
||||
if self.batch_error:
|
||||
raise self.batch_error
|
||||
self.batches.append({"username": username, "files": files, "destination": destination})
|
||||
self._destination = destination
|
||||
transfers = []
|
||||
for i, f in enumerate(files):
|
||||
if any(r["filename"] == f["filename"] for r in self.refuse):
|
||||
continue
|
||||
tid = f"t{i}"
|
||||
self._ids[tid] = f["filename"]
|
||||
transfers.append({"id": tid, "filename": f["filename"], "state": "Requested"})
|
||||
return {"batch_id": "b-1", "transfers": transfers, "failures": list(self.refuse)}
|
||||
|
||||
def get_transfer(self, username, transfer_id):
|
||||
filename = self._ids[transfer_id]
|
||||
count = self._calls.get(filename, 0)
|
||||
self._calls[filename] = count + 1
|
||||
states = self.script.get(filename, [OK])
|
||||
state = states[min(count, len(states) - 1)]
|
||||
if self.on_get:
|
||||
self.on_get(self, filename, state)
|
||||
if state is None:
|
||||
return None
|
||||
size = next(f["size"] for f in self.batches[-1]["files"] if f["filename"] == filename)
|
||||
if state == OK and self.write_on_success and filename not in self._written:
|
||||
folder = self.root / self._destination
|
||||
folder.mkdir(parents=True, exist_ok=True)
|
||||
(folder / filename.rsplit("\\", 1)[-1]).write_bytes(b"x" * size)
|
||||
self._written.add(filename)
|
||||
done = size if state == OK else 0
|
||||
return {
|
||||
"id": transfer_id,
|
||||
"filename": filename,
|
||||
"state": state,
|
||||
"size": size,
|
||||
"bytesTransferred": done,
|
||||
"averageSpeed": 1000.0,
|
||||
"placeInQueue": 3 if "Remotely" in state else None,
|
||||
}
|
||||
|
||||
def list_downloads(self, username):
|
||||
self.list_calls += 1
|
||||
return [
|
||||
{"id": tid, "filename": fn, "state": "InProgress", "removed": False}
|
||||
for tid, fn in self._ids.items()
|
||||
]
|
||||
|
||||
def cancel_transfer(self, username, transfer_id, *, remove):
|
||||
self.cancelled.append((username, transfer_id, remove))
|
||||
return True
|
||||
|
||||
def get_events(self, *, limit=200, offset=0):
|
||||
return list(self.events)
|
||||
|
||||
def get_download_directory(self):
|
||||
return str(self.root)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def cfg(monkeypatch, tmp_path):
|
||||
values: dict = {"SLSKD_DOWNLOAD_PATH": str(tmp_path)}
|
||||
monkeypatch.setattr(mod.config, "get", lambda k, d=None: values.get(k, d))
|
||||
return values
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def handler(monkeypatch):
|
||||
monkeypatch.setattr(SlskdHandler, "_poll_interval", lambda self: 0)
|
||||
monkeypatch.setattr(SlskdHandler, "_completed_path_retry_interval", lambda self: 0)
|
||||
monkeypatch.setattr(SlskdHandler, "_completed_path_timeout_seconds", lambda self: 0)
|
||||
monkeypatch.setattr(SlskdHandler, "_queue_timeout_seconds", lambda self: 0)
|
||||
monkeypatch.setattr(SlskdHandler, "_cancel_remove_wait", lambda self: None)
|
||||
monkeypatch.setattr(mod, "MAX_MISSING_POLLS", 3)
|
||||
return SlskdHandler()
|
||||
|
||||
|
||||
def _task(files=None, task_id="slskd:abc"):
|
||||
files = files if files is not None else FILES
|
||||
return DownloadTask(
|
||||
task_id=task_id,
|
||||
source="slskd",
|
||||
title="Book",
|
||||
retry_source_context={"username": "peer", "directory": REMOTE_DIR, "files": files},
|
||||
)
|
||||
|
||||
|
||||
def _run(handler, fake, task=None, cancel=None):
|
||||
handler._get_client = lambda: fake # type: ignore[method-assign]
|
||||
rec = Recorder()
|
||||
task = task or _task()
|
||||
result = handler.download(task, cancel or Event(), rec.progress_callback, rec.status_callback)
|
||||
return result, task, rec
|
||||
|
||||
|
||||
class TestHappyPath:
|
||||
def test_the_task_gets_its_own_folder_and_it_is_what_the_orchestrator_receives(
|
||||
self, handler, cfg, tmp_path
|
||||
):
|
||||
fake = FakeSlskd(
|
||||
tmp_path, script={FILE_A: ["Queued, Remotely", "InProgress", OK], FILE_B: [OK]}
|
||||
)
|
||||
|
||||
result, _task_obj, rec = _run(handler, fake)
|
||||
|
||||
folder = tmp_path / "shelfmark" / "slskd_abc"
|
||||
assert result == str(folder)
|
||||
assert sorted(p.name for p in folder.iterdir()) == ["Book One.epub", "Book Two.epub"]
|
||||
assert fake.batches[0]["destination"] == "shelfmark/slskd_abc"
|
||||
assert rec.progress[-1] == 100.0
|
||||
assert "downloading" in rec.statuses
|
||||
|
||||
def test_the_orchestrator_is_told_the_folder_is_the_clients_so_it_copies_not_moves(
|
||||
self, handler, cfg, tmp_path
|
||||
):
|
||||
fake = FakeSlskd(tmp_path)
|
||||
|
||||
result, task, _ = _run(handler, fake)
|
||||
|
||||
assert task.original_download_path == result
|
||||
|
||||
def test_transfers_are_read_by_id_never_by_the_list(self, handler, cfg, tmp_path):
|
||||
fake = FakeSlskd(tmp_path, script={FILE_A: ["InProgress", OK], FILE_B: ["InProgress", OK]})
|
||||
|
||||
_run(handler, fake)
|
||||
|
||||
assert fake.list_calls == 0
|
||||
|
||||
def test_a_failure_to_name_a_safe_folder_never_escapes_the_downloads_root(
|
||||
self, handler, cfg, tmp_path
|
||||
):
|
||||
fake = FakeSlskd(tmp_path)
|
||||
|
||||
_run(handler, fake, task=_task(task_id="../../etc/passwd"))
|
||||
|
||||
destination = fake.batches[0]["destination"]
|
||||
assert ".." not in destination and not destination.startswith("/")
|
||||
assert destination.startswith("shelfmark/")
|
||||
|
||||
def test_the_destination_prefix_is_a_setting_and_a_bad_one_is_ignored(
|
||||
self, handler, cfg, tmp_path
|
||||
):
|
||||
cfg["SLSKD_DESTINATION_PREFIX"] = "books/incoming"
|
||||
fake = FakeSlskd(tmp_path)
|
||||
_run(handler, fake)
|
||||
assert fake.batches[0]["destination"] == "books/incoming/slskd_abc"
|
||||
|
||||
cfg["SLSKD_DESTINATION_PREFIX"] = "../../elsewhere"
|
||||
fake2 = FakeSlskd(tmp_path)
|
||||
_run(handler, fake2, task=_task(task_id="slskd:def"))
|
||||
assert fake2.batches[0]["destination"] == "shelfmark/slskd_def"
|
||||
|
||||
|
||||
class TestAnotherAppClearsSlskdsList:
|
||||
def test_a_transfer_slskd_no_longer_lists_is_confirmed_by_the_files_on_disk(
|
||||
self, handler, cfg, tmp_path
|
||||
):
|
||||
# Both files arrive, then SoulSync (say) clears the finished transfers.
|
||||
fake = FakeSlskd(tmp_path, script={FILE_A: [OK, None], FILE_B: [OK, None]})
|
||||
|
||||
result, _, rec = _run(handler, fake)
|
||||
|
||||
assert result == str(tmp_path / "shelfmark" / "slskd_abc")
|
||||
assert "error" not in rec.statuses
|
||||
|
||||
def test_a_file_that_arrived_before_the_clear_is_not_lost(self, handler, cfg, tmp_path):
|
||||
folder = tmp_path / "shelfmark" / "slskd_abc"
|
||||
|
||||
def write_a_then_clear(fake, filename, state):
|
||||
if filename == FILE_A:
|
||||
folder.mkdir(parents=True, exist_ok=True)
|
||||
(folder / "Book One.epub").write_bytes(b"x" * 5)
|
||||
|
||||
fake = FakeSlskd(tmp_path, script={FILE_A: [None], FILE_B: ["InProgress", OK]})
|
||||
fake.on_get = write_a_then_clear
|
||||
|
||||
result, _, rec = _run(handler, fake)
|
||||
|
||||
assert result == str(folder)
|
||||
assert "error" not in rec.statuses
|
||||
|
||||
def test_a_transfer_that_vanishes_with_no_file_is_an_error_not_a_hang(
|
||||
self, handler, cfg, tmp_path
|
||||
):
|
||||
fake = FakeSlskd(tmp_path, script={FILE_A: [None], FILE_B: [None]}, write_on_success=False)
|
||||
|
||||
result, task, rec = _run(handler, fake)
|
||||
|
||||
assert result is None
|
||||
assert rec.real[-1][0] == "error"
|
||||
assert "slskd" in (rec.real[-1][1] or "").lower()
|
||||
assert task.original_download_path is None
|
||||
|
||||
def test_slskds_events_confirm_a_vanished_transfer_whose_file_is_still_arriving(
|
||||
self, handler, cfg, tmp_path
|
||||
):
|
||||
folder = tmp_path / "shelfmark" / "slskd_abc"
|
||||
fake = FakeSlskd(tmp_path, script={FILE_A: [None], FILE_B: [None]}, write_on_success=False)
|
||||
fake.events = [
|
||||
{"type": "DownloadFileComplete", "data": json.dumps({"remoteFilename": FILE_A})},
|
||||
{"type": "DownloadFileComplete", "data": json.dumps({"remoteFilename": FILE_B})},
|
||||
]
|
||||
# The files land a moment after slskd reported the events.
|
||||
calls = {"n": 0}
|
||||
real_get_events = fake.get_events
|
||||
|
||||
def events_then_files(**kw):
|
||||
folder.mkdir(parents=True, exist_ok=True)
|
||||
(folder / "Book One.epub").write_bytes(b"x" * 5)
|
||||
(folder / "Book Two.epub").write_bytes(b"x" * 7)
|
||||
calls["n"] += 1
|
||||
return real_get_events(**kw)
|
||||
|
||||
fake.get_events = events_then_files # type: ignore[method-assign]
|
||||
|
||||
result, _, rec = _run(handler, fake)
|
||||
|
||||
assert calls["n"] >= 1
|
||||
assert result == str(folder)
|
||||
assert "error" not in rec.statuses
|
||||
|
||||
|
||||
class TestFailuresAndCancel:
|
||||
def test_a_failed_transfer_fails_the_task_and_stops_the_rest(self, handler, cfg, tmp_path):
|
||||
fake = FakeSlskd(tmp_path, script={FILE_A: ["Completed, Errored"], FILE_B: ["InProgress"]})
|
||||
|
||||
result, task, rec = _run(handler, fake)
|
||||
|
||||
assert result is None
|
||||
assert rec.real[-1][0] == "error"
|
||||
assert "Errored" in (rec.real[-1][1] or "")
|
||||
assert {tid for _, tid, _ in fake.cancelled} == {"t0", "t1"}
|
||||
assert task.original_download_path is None
|
||||
|
||||
def test_cancelling_cancels_and_removes_every_transfer(self, handler, cfg, tmp_path):
|
||||
cancel = Event()
|
||||
fake = FakeSlskd(tmp_path, script={FILE_A: ["InProgress"], FILE_B: ["InProgress"]})
|
||||
fake.on_get = lambda f, name, state: cancel.set()
|
||||
|
||||
result, _, rec = _run(handler, fake, cancel=cancel)
|
||||
|
||||
assert result is None
|
||||
assert rec.real[-1][0] == "cancelled"
|
||||
assert {(tid, remove) for _, tid, remove in fake.cancelled} == {("t0", True), ("t1", True)}
|
||||
|
||||
def test_a_peer_that_never_starts_gives_up_after_the_queue_timeout(
|
||||
self, handler, cfg, tmp_path, monkeypatch
|
||||
):
|
||||
monkeypatch.setattr(SlskdHandler, "_queue_timeout_seconds", lambda self: 600.0)
|
||||
clock = iter([0.0, 1.0, 2.0, 700.0, 701.0, 702.0, 703.0, 704.0, 705.0])
|
||||
monkeypatch.setattr(SlskdHandler, "_clock", lambda self: next(clock), raising=False)
|
||||
fake = FakeSlskd(
|
||||
tmp_path,
|
||||
script={FILE_A: ["Queued, Remotely"], FILE_B: ["Queued, Remotely"]},
|
||||
write_on_success=False,
|
||||
)
|
||||
|
||||
result, _, rec = _run(handler, fake)
|
||||
|
||||
assert result is None
|
||||
assert "minutes" in (rec.real[-1][1] or "")
|
||||
assert fake.cancelled
|
||||
|
||||
|
||||
class TestPeerQueue:
|
||||
def test_a_queued_transfer_asks_the_orchestrator_not_to_stall_it(self, handler, cfg, tmp_path):
|
||||
fake = FakeSlskd(
|
||||
tmp_path,
|
||||
script={
|
||||
FILE_A: ["Queued, Remotely", "Queued, Remotely", OK],
|
||||
FILE_B: ["Queued, Remotely", "Queued, Remotely", OK],
|
||||
},
|
||||
)
|
||||
|
||||
_, _, rec = _run(handler, fake)
|
||||
|
||||
assert rec.graces, "no activity grace was requested while the peer queued the files"
|
||||
assert rec.graces[0] == mod.QUEUE_GRACE_SECONDS
|
||||
# and it is released once the transfers start, so the normal stall timer applies
|
||||
assert rec.graces[-1] == 0.0
|
||||
|
||||
|
||||
class TestRejoinAfterARestart:
|
||||
def test_files_already_on_disk_are_not_downloaded_again(self, handler, cfg, tmp_path):
|
||||
folder = tmp_path / "shelfmark" / "slskd_abc"
|
||||
folder.mkdir(parents=True)
|
||||
(folder / "Book One.epub").write_bytes(b"x" * 5)
|
||||
(folder / "Book Two.epub").write_bytes(b"x" * 7)
|
||||
fake = FakeSlskd(tmp_path)
|
||||
|
||||
result, task, rec = _run(handler, fake)
|
||||
|
||||
assert result == str(folder)
|
||||
assert fake.batches == []
|
||||
assert task.original_download_path == str(folder)
|
||||
assert rec.progress[-1] == 100.0
|
||||
|
||||
def test_files_slskd_says_are_already_queued_are_found_by_name_and_rejoined(
|
||||
self, handler, cfg, tmp_path
|
||||
):
|
||||
fake = FakeSlskd(tmp_path, script={FILE_A: [OK], FILE_B: [OK]})
|
||||
fake.refuse = [{"filename": FILE_A, "message": "already queued"}]
|
||||
fake._ids["t0"] = FILE_A # slskd still lists the earlier transfer for it
|
||||
|
||||
result, _, rec = _run(handler, fake)
|
||||
|
||||
assert result is not None
|
||||
assert "error" not in rec.statuses
|
||||
|
||||
def test_a_refusal_with_nothing_to_rejoin_is_reported(self, handler, cfg, tmp_path):
|
||||
fake = FakeSlskd(tmp_path)
|
||||
fake.refuse = [
|
||||
{"filename": FILE_A, "message": "denied"},
|
||||
{"filename": FILE_B, "message": "denied"},
|
||||
]
|
||||
|
||||
result, _, rec = _run(handler, fake)
|
||||
|
||||
assert result is None
|
||||
assert rec.real[-1][0] == "error"
|
||||
assert "denied" in (rec.real[-1][1] or "")
|
||||
|
||||
|
||||
class TestFallbackAndSettings:
|
||||
def test_an_old_slskd_without_batches_uses_the_original_enqueue(self, handler, cfg, tmp_path):
|
||||
class Old(FakeSlskd):
|
||||
def __init__(self, root):
|
||||
super().__init__(root)
|
||||
self.batch_error = SlskdBatchUnsupportedError("no batches")
|
||||
self.enqueued: list = []
|
||||
|
||||
def enqueue_downloads(self, username, files):
|
||||
self.enqueued.append(files)
|
||||
return [
|
||||
{"id": f"o{i}", "filename": f["filename"], "state": "Queued, Locally"}
|
||||
for i, f in enumerate(files)
|
||||
]
|
||||
|
||||
fake = Old(tmp_path)
|
||||
# the original path polls the list, so give it finished transfers and the files
|
||||
fake.list_downloads = lambda username: [ # type: ignore[method-assign]
|
||||
{
|
||||
"id": f"o{i}",
|
||||
"filename": f["filename"],
|
||||
"state": OK,
|
||||
"bytesTransferred": f["size"],
|
||||
"size": f["size"],
|
||||
"removed": False,
|
||||
}
|
||||
for i, f in enumerate(FILES)
|
||||
]
|
||||
for name in ("Book One.epub", "Book Two.epub"):
|
||||
target = tmp_path / "Author" / name
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_bytes(b"x" * 5)
|
||||
|
||||
result, _, rec = _run(handler, fake)
|
||||
|
||||
assert fake.enqueued, "the original enqueue was not used"
|
||||
assert result is not None
|
||||
assert "error" not in rec.statuses
|
||||
|
||||
def test_isolation_can_be_switched_off(self, handler, cfg, tmp_path):
|
||||
cfg["SLSKD_ISOLATE_DOWNLOADS"] = False
|
||||
|
||||
class NoBatch(FakeSlskd):
|
||||
def enqueue_batch(self, *a, **kw):
|
||||
raise AssertionError("the batch endpoint was used with isolation off")
|
||||
|
||||
def enqueue_downloads(self, username, files):
|
||||
return []
|
||||
|
||||
fake = NoBatch(tmp_path)
|
||||
fake.list_downloads = lambda username: [] # type: ignore[method-assign]
|
||||
|
||||
result, _, rec = _run(handler, fake)
|
||||
|
||||
assert result is None # the original path found nothing to wait for
|
||||
assert rec.real[-1][0] == "error"
|
||||
|
||||
|
||||
class TestCleanup:
|
||||
def _done(self, handler, cfg, tmp_path):
|
||||
fake = FakeSlskd(tmp_path)
|
||||
result, task, _ = _run(handler, fake)
|
||||
return fake, result, task
|
||||
|
||||
def test_by_default_finished_files_and_records_are_left_in_slskd_so_they_stay_shared(
|
||||
self, handler, cfg, tmp_path
|
||||
):
|
||||
fake, _result, task = self._done(handler, cfg, tmp_path)
|
||||
|
||||
handler.post_process_cleanup(task, success=True)
|
||||
|
||||
assert fake.cancelled == []
|
||||
assert (tmp_path / "shelfmark" / "slskd_abc").is_dir()
|
||||
|
||||
def test_with_keep_off_the_transfers_and_the_folder_are_removed_after_import(
|
||||
self, handler, cfg, tmp_path
|
||||
):
|
||||
cfg["SLSKD_KEEP_COMPLETED"] = False
|
||||
fake, _result, task = self._done(handler, cfg, tmp_path)
|
||||
|
||||
handler.post_process_cleanup(task, success=True)
|
||||
|
||||
assert {(tid, remove) for _, tid, remove in fake.cancelled} == {("t0", True), ("t1", True)}
|
||||
assert not (tmp_path / "shelfmark" / "slskd_abc").exists()
|
||||
|
||||
def test_a_failed_import_leaves_everything_for_a_retry(self, handler, cfg, tmp_path):
|
||||
cfg["SLSKD_KEEP_COMPLETED"] = False
|
||||
fake, _result, task = self._done(handler, cfg, tmp_path)
|
||||
|
||||
handler.post_process_cleanup(task, success=False)
|
||||
|
||||
assert fake.cancelled == []
|
||||
assert (tmp_path / "shelfmark" / "slskd_abc").is_dir()
|
||||
|
||||
|
||||
class TestFolderComplete:
|
||||
def _folder(self, tmp_path, files):
|
||||
folder = tmp_path / "f"
|
||||
folder.mkdir()
|
||||
for name, size in files.items():
|
||||
(folder / name).write_bytes(b"x" * size)
|
||||
return folder
|
||||
|
||||
def test_the_files_asked_for_are_complete(self, tmp_path):
|
||||
folder = self._folder(tmp_path, {"a.epub": 5, "b.epub": 7})
|
||||
|
||||
assert mod._folder_complete(folder, [5, 7]) is True
|
||||
|
||||
def test_a_file_of_the_wrong_size_is_not_complete(self, tmp_path):
|
||||
# the right number of files, but one is a partial copy
|
||||
folder = self._folder(tmp_path, {"a.epub": 5, "b.epub": 3})
|
||||
|
||||
assert mod._folder_complete(folder, [5, 7]) is False
|
||||
|
||||
def test_names_do_not_matter_because_slskd_may_rename_a_duplicate(self, tmp_path):
|
||||
folder = self._folder(tmp_path, {"a_1.epub": 5, "totally different.epub": 7})
|
||||
|
||||
assert mod._folder_complete(folder, [7, 5]) is True
|
||||
|
||||
def test_too_few_or_too_many_files_is_not_complete(self, tmp_path):
|
||||
assert mod._folder_complete(self._folder(tmp_path, {"a.epub": 5}), [5, 7]) is False
|
||||
other = tmp_path / "g"
|
||||
other.mkdir()
|
||||
for name, size in {"a": 5, "b": 7, "c": 1}.items():
|
||||
(other / name).write_bytes(b"x" * size)
|
||||
assert mod._folder_complete(other, [5, 7]) is False
|
||||
|
||||
def test_hidden_and_missing_folders_hold_nothing(self, tmp_path):
|
||||
folder = self._folder(tmp_path, {".part": 5})
|
||||
|
||||
assert mod._folder_complete(folder, [5]) is False
|
||||
assert mod._folder_complete(tmp_path / "nope", [5]) is False
|
||||
|
||||
def test_an_unknown_size_is_checked_by_count_only(self, tmp_path):
|
||||
folder = self._folder(tmp_path, {"a.epub": 5, "b.epub": 9})
|
||||
|
||||
assert mod._folder_complete(folder, [0, 0]) is True
|
||||
assert mod._folder_complete(folder, [0]) is False
|
||||
|
||||
def test_nothing_asked_for_is_never_complete(self, tmp_path):
|
||||
assert mod._folder_complete(self._folder(tmp_path, {"a": 1}), []) is False
|
||||
@@ -0,0 +1,80 @@
|
||||
"""Live checks against a real slskd instance.
|
||||
|
||||
Skipped unless ``SLSKD_TEST_URL`` and ``SLSKD_TEST_API_KEY`` are set, e.g. after
|
||||
``docker compose -f docker-compose.test-clients.yml up -d slskd``.
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from shelfmark.release_sources.slskd.api import SlskdAuthError, SlskdClient
|
||||
|
||||
pytestmark = pytest.mark.integration
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def live_client() -> SlskdClient:
|
||||
url = os.environ.get("SLSKD_TEST_URL", "").strip()
|
||||
api_key = os.environ.get("SLSKD_TEST_API_KEY", "").strip()
|
||||
if not url or not api_key:
|
||||
pytest.skip("SLSKD_TEST_URL / SLSKD_TEST_API_KEY not set")
|
||||
client = SlskdClient(url, api_key, timeout=15)
|
||||
try:
|
||||
client.get_application()
|
||||
except Exception as e: # pragma: no cover - environment dependent
|
||||
pytest.skip(f"slskd not reachable: {e}")
|
||||
return client
|
||||
|
||||
|
||||
def test_connection_reports_login_state(live_client):
|
||||
ok, message = live_client.test_connection()
|
||||
assert "slskd" in message
|
||||
if ok:
|
||||
assert "logged in" in message.lower()
|
||||
else:
|
||||
assert "not logged in" in message
|
||||
|
||||
|
||||
def test_bad_api_key_is_rejected(live_client):
|
||||
bad = SlskdClient(live_client.base_url, "definitely-not-the-key", timeout=15)
|
||||
with pytest.raises(SlskdAuthError):
|
||||
bad.get_application()
|
||||
|
||||
|
||||
def test_download_directory_is_reported(live_client):
|
||||
directory = live_client.get_download_directory()
|
||||
assert directory
|
||||
assert directory.startswith("/")
|
||||
|
||||
|
||||
def test_search_lifecycle(live_client):
|
||||
search_id = live_client.start_search(
|
||||
"zzzz-shelfmark-nonexistent-qqqq", timeout_ms=1000, response_limit=5
|
||||
)
|
||||
state = live_client.get_search(search_id)
|
||||
assert state["id"] == search_id
|
||||
assert isinstance(live_client.get_search_responses(search_id), list)
|
||||
live_client.delete_search(search_id)
|
||||
|
||||
|
||||
def test_missing_transfer_is_none(live_client):
|
||||
assert (
|
||||
live_client.get_transfer("nobody-shelfmark-test", "00000000-0000-0000-0000-000000000000")
|
||||
is None
|
||||
)
|
||||
assert live_client.list_downloads("nobody-shelfmark-test") == []
|
||||
|
||||
|
||||
def test_network_search_returns_ebooks_when_logged_in(live_client):
|
||||
if not live_client.is_logged_in():
|
||||
pytest.skip("slskd is not logged in to Soulseek")
|
||||
from shelfmark.release_sources.slskd.source import build_releases
|
||||
|
||||
responses = live_client.search("sherlock holmes epub", wait_seconds=10, response_limit=20)
|
||||
releases = build_releases(responses, content_type="ebook", formats={"epub"})
|
||||
# The network is live, so only assert on shape when something answered.
|
||||
for release in releases[:5]:
|
||||
assert release.source == "slskd"
|
||||
assert release.format == "epub"
|
||||
assert release.extra["files"][0]["filename"].lower().endswith(".epub")
|
||||
@@ -0,0 +1,127 @@
|
||||
"""slskd releases survive queueing, retry serialization and a full handler round-trip."""
|
||||
|
||||
from pathlib import Path
|
||||
from threading import Event
|
||||
|
||||
import pytest
|
||||
|
||||
from shelfmark.core.models import DownloadTask
|
||||
from shelfmark.download import orchestrator
|
||||
from shelfmark.release_sources import get_handler
|
||||
from shelfmark.release_sources.slskd.handler import SlskdHandler, spec_from_task
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _original_enqueue_path(monkeypatch):
|
||||
"""These tests cover the original enqueue-and-poll-the-list path; isolated mode has its own
|
||||
tests in test_handler_isolated.py."""
|
||||
from shelfmark.release_sources.slskd.handler import SlskdHandler
|
||||
|
||||
monkeypatch.setattr(SlskdHandler, "_isolate_enabled", lambda self: False)
|
||||
|
||||
|
||||
FILE = "@@share\\Books\\Author\\Book.epub"
|
||||
|
||||
|
||||
def _release_data():
|
||||
return {
|
||||
"source": "slskd",
|
||||
"source_id": "slskd:deadbeef",
|
||||
"title": "Book",
|
||||
"format": "epub",
|
||||
"protocol": "soulseek",
|
||||
"content_type": "ebook",
|
||||
"extra": {
|
||||
"username": "peer",
|
||||
"directory": "@@share\\Books\\Author",
|
||||
"files": [{"filename": FILE, "size": 1000}],
|
||||
"upload_speed": 1,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def test_queue_release_persists_soulseek_spec(monkeypatch):
|
||||
captured: dict[str, DownloadTask] = {}
|
||||
|
||||
def fake_add(task: DownloadTask) -> bool:
|
||||
captured["task"] = task
|
||||
return True
|
||||
|
||||
monkeypatch.setattr(orchestrator.config, "get", lambda _key, default=None, **_kw: default)
|
||||
monkeypatch.setattr(orchestrator, "_source_unavailable_message", lambda _source: None)
|
||||
monkeypatch.setattr(orchestrator.book_queue, "add", fake_add)
|
||||
monkeypatch.setattr(orchestrator, "ws_manager", None)
|
||||
|
||||
ok, error = orchestrator.queue_release(_release_data(), 0)
|
||||
|
||||
assert ok, error
|
||||
task = captured["task"]
|
||||
assert task.source == "slskd"
|
||||
assert task.task_id == "slskd:deadbeef"
|
||||
assert task.retry_download_protocol == "soulseek"
|
||||
assert task.retry_download_url is None
|
||||
spec = spec_from_task(task)
|
||||
assert spec is not None
|
||||
assert spec.username == "peer"
|
||||
assert spec.filenames == [FILE]
|
||||
|
||||
|
||||
def test_retry_payload_round_trips_soulseek_spec():
|
||||
handler = get_handler("slskd")
|
||||
fields = handler.build_retry_resolution_fields(_release_data())
|
||||
task = DownloadTask(task_id="slskd:deadbeef", source="slskd", title="Book", **fields)
|
||||
|
||||
payload = orchestrator.serialize_task_for_retry(task)
|
||||
restored = orchestrator._restore_task_from_retry_payload(payload)
|
||||
|
||||
assert restored is not None
|
||||
assert spec_from_task(restored) == spec_from_task(task)
|
||||
|
||||
|
||||
def test_handler_round_trip_from_queued_task(monkeypatch, tmp_path):
|
||||
"""Queue → task → handler download using the persisted context only (no cache)."""
|
||||
import shelfmark.release_sources.slskd.handler as mod
|
||||
|
||||
handler = get_handler("slskd")
|
||||
assert isinstance(handler, SlskdHandler)
|
||||
fields = handler.build_retry_resolution_fields(_release_data())
|
||||
task = DownloadTask(task_id="slskd:deadbeef", source="slskd", title="Book", **fields)
|
||||
|
||||
completed = tmp_path / "Author" / "Book.epub"
|
||||
completed.parent.mkdir(parents=True)
|
||||
completed.write_bytes(b"epub")
|
||||
|
||||
class Client:
|
||||
def enqueue_downloads(self, username, files):
|
||||
assert username == "peer"
|
||||
return [{"id": "t1", "filename": files[0]["filename"]}]
|
||||
|
||||
def list_downloads(self, username):
|
||||
return [
|
||||
{
|
||||
"id": "t1",
|
||||
"filename": FILE,
|
||||
"state": "Completed, Succeeded",
|
||||
"bytesTransferred": 1000,
|
||||
"removed": False,
|
||||
}
|
||||
]
|
||||
|
||||
def cancel_transfer(self, username, transfer_id, *, remove):
|
||||
return True
|
||||
|
||||
def get_download_directory(self):
|
||||
return str(tmp_path)
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", lambda k, d=None: d)
|
||||
monkeypatch.setattr(SlskdHandler, "_get_client", lambda self: Client())
|
||||
monkeypatch.setattr(SlskdHandler, "_poll_interval", lambda self: 0)
|
||||
monkeypatch.setattr(SlskdHandler, "_completed_path_timeout_seconds", lambda self: 0)
|
||||
|
||||
statuses: list[tuple[str, str | None]] = []
|
||||
result = handler.download(task, Event(), lambda p: None, lambda s, m: statuses.append((s, m)))
|
||||
|
||||
assert result == str(completed)
|
||||
assert Path(result).exists()
|
||||
assert "error" not in {s for s, _ in statuses}
|
||||
assert statuses[0] == ("resolving", "Sending to slskd")
|
||||
@@ -0,0 +1,189 @@
|
||||
"""Tests for the slskd settings tab and its Test Connection action."""
|
||||
|
||||
from shelfmark.core.settings_registry import (
|
||||
ActionButton,
|
||||
CheckboxField,
|
||||
NumberField,
|
||||
PasswordField,
|
||||
TextField,
|
||||
)
|
||||
from shelfmark.release_sources.slskd.settings import (
|
||||
_test_slskd_connection,
|
||||
slskd_config_settings,
|
||||
)
|
||||
|
||||
|
||||
def _field(key: str):
|
||||
return next(f for f in slskd_config_settings() if getattr(f, "key", None) == key)
|
||||
|
||||
|
||||
def test_settings_expose_expected_fields():
|
||||
assert isinstance(_field("SLSKD_ENABLED"), CheckboxField)
|
||||
assert isinstance(_field("SLSKD_URL"), TextField)
|
||||
assert isinstance(_field("SLSKD_API_KEY"), PasswordField)
|
||||
assert isinstance(_field("test_slskd"), ActionButton)
|
||||
assert isinstance(_field("SLSKD_DOWNLOAD_PATH"), TextField)
|
||||
assert isinstance(_field("SLSKD_SEARCH_TIMEOUT"), NumberField)
|
||||
assert isinstance(_field("SLSKD_RESPONSE_LIMIT"), NumberField)
|
||||
assert isinstance(_field("SLSKD_QUEUE_TIMEOUT_MINUTES"), NumberField)
|
||||
assert isinstance(_field("SLSKD_REMOVE_COMPLETED"), CheckboxField)
|
||||
|
||||
|
||||
def test_dependent_fields_hide_until_enabled():
|
||||
for key in ("SLSKD_URL", "SLSKD_API_KEY", "test_slskd", "SLSKD_DOWNLOAD_PATH"):
|
||||
assert _field(key).show_when == {"field": "SLSKD_ENABLED", "value": True}
|
||||
|
||||
|
||||
def test_defaults():
|
||||
assert _field("SLSKD_ENABLED").default is False
|
||||
assert _field("SLSKD_SEARCH_TIMEOUT").default == 15
|
||||
assert _field("SLSKD_RESPONSE_LIMIT").default == 100
|
||||
assert _field("SLSKD_QUEUE_TIMEOUT_MINUTES").default == 60
|
||||
assert _field("SLSKD_REMOVE_COMPLETED").default is True
|
||||
|
||||
|
||||
def test_connection_action_uses_form_values(monkeypatch):
|
||||
import shelfmark.release_sources.slskd.api as api_module
|
||||
|
||||
created: list[tuple[str, str]] = []
|
||||
|
||||
class FakeClient:
|
||||
def __init__(self, url, api_key):
|
||||
created.append((url, api_key))
|
||||
|
||||
def test_connection(self):
|
||||
return True, "Connected to slskd 0.26.0 (logged in to Soulseek)"
|
||||
|
||||
monkeypatch.setattr(api_module, "SlskdClient", FakeClient)
|
||||
|
||||
result = _test_slskd_connection({"SLSKD_URL": "slskd:5030/", "SLSKD_API_KEY": " key "})
|
||||
|
||||
assert result == {
|
||||
"success": True,
|
||||
"message": "Connected to slskd 0.26.0 (logged in to Soulseek)",
|
||||
}
|
||||
assert created == [("http://slskd:5030", "key")]
|
||||
|
||||
|
||||
def test_connection_action_requires_url(monkeypatch):
|
||||
import shelfmark.release_sources.slskd.settings as mod
|
||||
|
||||
monkeypatch.setattr(mod, "normalize_http_url", lambda url: url)
|
||||
from shelfmark.core.config import config
|
||||
|
||||
monkeypatch.setattr(config, "get", lambda k, d=None: d)
|
||||
result = _test_slskd_connection({"SLSKD_URL": "", "SLSKD_API_KEY": "k"})
|
||||
assert result == {"success": False, "message": "slskd URL is required"}
|
||||
|
||||
|
||||
def test_connection_action_requires_api_key(monkeypatch):
|
||||
from shelfmark.core.config import config
|
||||
|
||||
monkeypatch.setattr(config, "get", lambda k, d=None: d)
|
||||
result = _test_slskd_connection({"SLSKD_URL": "http://slskd:5030", "SLSKD_API_KEY": ""})
|
||||
assert result == {"success": False, "message": "slskd API key is required"}
|
||||
|
||||
|
||||
def test_connection_action_reports_failure(monkeypatch):
|
||||
import shelfmark.release_sources.slskd.api as api_module
|
||||
|
||||
class FakeClient:
|
||||
def __init__(self, url, api_key):
|
||||
pass
|
||||
|
||||
def test_connection(self):
|
||||
return False, "Invalid API key"
|
||||
|
||||
monkeypatch.setattr(api_module, "SlskdClient", FakeClient)
|
||||
result = _test_slskd_connection({"SLSKD_URL": "http://slskd:5030", "SLSKD_API_KEY": "bad"})
|
||||
assert result == {"success": False, "message": "Invalid API key"}
|
||||
|
||||
|
||||
def test_connection_action_wraps_unexpected_errors(monkeypatch):
|
||||
import shelfmark.release_sources.slskd.api as api_module
|
||||
|
||||
class FakeClient:
|
||||
def __init__(self, url, api_key):
|
||||
raise RuntimeError("kaboom")
|
||||
|
||||
monkeypatch.setattr(api_module, "SlskdClient", FakeClient)
|
||||
result = _test_slskd_connection({"SLSKD_URL": "http://slskd:5030", "SLSKD_API_KEY": "k"})
|
||||
assert result["success"] is False
|
||||
assert "kaboom" in result["message"]
|
||||
|
||||
|
||||
def test_remote_path_mapping_offers_slskd_host():
|
||||
from shelfmark.config.settings import advanced_settings
|
||||
|
||||
table = next(
|
||||
f for f in advanced_settings() if getattr(f, "key", None) == "PROWLARR_REMOTE_PATH_MAPPINGS"
|
||||
)
|
||||
host_column = next(c for c in table.columns if c["key"] == "host")
|
||||
assert {"value": "slskd", "label": "slskd"} in host_column["options"]
|
||||
|
||||
|
||||
class TestIsolatedDownloadSettings:
|
||||
"""The options for isolated downloads (fork additions)."""
|
||||
|
||||
def _fields(self):
|
||||
from shelfmark.core.settings_registry import get_settings_field_map
|
||||
from shelfmark.release_sources.slskd import settings as _settings # noqa: F401
|
||||
|
||||
return get_settings_field_map("slskd_config")
|
||||
|
||||
def test_isolation_is_on_by_default_with_its_own_folder_prefix(self):
|
||||
fields = self._fields()
|
||||
|
||||
isolate, _ = fields["SLSKD_ISOLATE_DOWNLOADS"]
|
||||
prefix, _ = fields["SLSKD_DESTINATION_PREFIX"]
|
||||
keep, _ = fields["SLSKD_KEEP_COMPLETED"]
|
||||
assert isolate.default is True
|
||||
assert prefix.default == "shelfmark"
|
||||
assert keep.default is True
|
||||
|
||||
def test_the_options_that_only_matter_when_isolated_are_hidden_otherwise(self):
|
||||
fields = self._fields()
|
||||
|
||||
for key in ("SLSKD_DESTINATION_PREFIX", "SLSKD_KEEP_COMPLETED"):
|
||||
assert fields[key][0].show_when == {"field": "SLSKD_ISOLATE_DOWNLOADS", "value": True}
|
||||
# the original "remove completed" option is the one for the non-isolated path
|
||||
assert fields["SLSKD_REMOVE_COMPLETED"][0].show_when == {
|
||||
"field": "SLSKD_ISOLATE_DOWNLOADS",
|
||||
"value": False,
|
||||
}
|
||||
|
||||
def test_a_cleared_url_in_the_form_is_not_replaced_by_the_saved_one(self, monkeypatch):
|
||||
from shelfmark.core import config as core_config
|
||||
from shelfmark.release_sources.slskd.settings import _test_slskd_connection
|
||||
|
||||
monkeypatch.setattr(
|
||||
core_config.config,
|
||||
"get",
|
||||
lambda key, default=None: {
|
||||
"SLSKD_URL": "http://saved:5030",
|
||||
"SLSKD_API_KEY": "saved",
|
||||
}.get(key, default),
|
||||
)
|
||||
|
||||
result = _test_slskd_connection({"SLSKD_URL": "", "SLSKD_API_KEY": "typed"})
|
||||
|
||||
assert result["success"] is False
|
||||
assert "URL is required" in result["message"]
|
||||
|
||||
def test_a_cleared_key_in_the_form_is_not_replaced_by_the_saved_one(self, monkeypatch):
|
||||
from shelfmark.core import config as core_config
|
||||
from shelfmark.release_sources.slskd.settings import _test_slskd_connection
|
||||
|
||||
monkeypatch.setattr(
|
||||
core_config.config,
|
||||
"get",
|
||||
lambda key, default=None: {
|
||||
"SLSKD_URL": "http://saved:5030",
|
||||
"SLSKD_API_KEY": "saved",
|
||||
}.get(key, default),
|
||||
)
|
||||
|
||||
result = _test_slskd_connection({"SLSKD_URL": "http://typed:5030", "SLSKD_API_KEY": ""})
|
||||
|
||||
assert result["success"] is False
|
||||
assert "API key is required" in result["message"]
|
||||
@@ -0,0 +1,617 @@
|
||||
"""Unit tests for the slskd release source."""
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from shelfmark.core.search_plan import ReleaseSearchPlan, ReleaseSearchVariant
|
||||
from shelfmark.metadata_providers import BookMetadata
|
||||
from shelfmark.release_sources import ReleaseProtocol, get_source
|
||||
from shelfmark.release_sources.slskd.source import (
|
||||
SlskdSource,
|
||||
_plan_queries,
|
||||
build_releases,
|
||||
file_extension,
|
||||
get_supported_formats,
|
||||
iter_peer_files,
|
||||
make_source_id,
|
||||
rank_key,
|
||||
remote_directory_name,
|
||||
split_remote_path,
|
||||
)
|
||||
|
||||
# ── fixtures / helpers ─────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _make_book(**kwargs) -> BookMetadata:
|
||||
defaults = {
|
||||
"provider": "hardcover",
|
||||
"provider_id": "1",
|
||||
"title": "Dune",
|
||||
"authors": ["Frank Herbert"],
|
||||
}
|
||||
defaults.update(kwargs)
|
||||
return BookMetadata(**defaults)
|
||||
|
||||
|
||||
def _make_plan(
|
||||
*,
|
||||
variants: list[tuple[str, str]] | None = None,
|
||||
manual_query: str | None = None,
|
||||
) -> ReleaseSearchPlan:
|
||||
variants = variants if variants is not None else [("Dune", "Frank Herbert")]
|
||||
title_variants = [ReleaseSearchVariant(title=t, author=a) for t, a in variants]
|
||||
return ReleaseSearchPlan(
|
||||
languages=["en"],
|
||||
isbn_candidates=[],
|
||||
author="Frank Herbert",
|
||||
title_variants=title_variants,
|
||||
grouped_title_variants=title_variants,
|
||||
manual_query=manual_query,
|
||||
)
|
||||
|
||||
|
||||
def _file(filename: str, size: int = 1000, **extra) -> dict:
|
||||
return {"filename": filename, "size": size, "isLocked": False, **extra}
|
||||
|
||||
|
||||
def _response(username: str = "peer", files: list[dict] | None = None, **extra) -> dict:
|
||||
base = {
|
||||
"username": username,
|
||||
"hasFreeUploadSlot": True,
|
||||
"queueLength": 0,
|
||||
"uploadSpeed": 500000,
|
||||
"fileCount": len(files or []),
|
||||
"files": files or [],
|
||||
}
|
||||
base.update(extra)
|
||||
return base
|
||||
|
||||
|
||||
EBOOK_FORMATS = {"epub", "mobi", "azw3", "pdf"}
|
||||
AUDIO_FORMATS = {"mp3", "m4b", "flac", "zip"}
|
||||
|
||||
|
||||
# ── path helpers ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestPathHelpers:
|
||||
def test_split_backslash_path(self):
|
||||
assert split_remote_path("@@a\\Books\\Author\\Dune.epub") == (
|
||||
"@@a\\Books\\Author",
|
||||
"Dune.epub",
|
||||
)
|
||||
|
||||
def test_split_forward_slash_path(self):
|
||||
assert split_remote_path("share/Books/Dune.epub") == ("share/Books", "Dune.epub")
|
||||
|
||||
def test_split_bare_filename(self):
|
||||
assert split_remote_path("Dune.epub") == ("", "Dune.epub")
|
||||
|
||||
def test_split_mixed_separators_uses_last(self):
|
||||
assert split_remote_path("a\\b/c\\d.epub") == ("a\\b/c", "d.epub")
|
||||
|
||||
def test_remote_directory_name(self):
|
||||
assert remote_directory_name("@@a\\Books\\Author") == "Author"
|
||||
assert remote_directory_name("Author") == "Author"
|
||||
assert remote_directory_name("") == ""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("name", "ext"),
|
||||
[
|
||||
("Dune.EPUB", "epub"),
|
||||
("Dune.part1.mp3", "mp3"),
|
||||
("no-extension", ""),
|
||||
("trailing.", ""),
|
||||
],
|
||||
)
|
||||
def test_file_extension(self, name, ext):
|
||||
assert file_extension(name) == ext
|
||||
|
||||
def test_source_id_is_stable_and_namespaced(self):
|
||||
a = make_source_id("peer", "x\\y.epub")
|
||||
assert a == make_source_id("peer", "x\\y.epub")
|
||||
assert a.startswith("slskd:")
|
||||
assert a != make_source_id("other", "x\\y.epub")
|
||||
|
||||
|
||||
# ── iter_peer_files ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestIterPeerFiles:
|
||||
def test_extracts_files(self):
|
||||
response = _response(files=[_file("@@a\\Books\\Dune.epub", 2048)])
|
||||
(pf,) = iter_peer_files(response)
|
||||
assert pf.username == "peer"
|
||||
assert pf.filename == "@@a\\Books\\Dune.epub"
|
||||
assert pf.directory == "@@a\\Books"
|
||||
assert pf.basename == "Dune.epub"
|
||||
assert pf.extension == "epub"
|
||||
assert pf.size == 2048
|
||||
|
||||
def test_skips_locked_files(self):
|
||||
response = _response(files=[_file("a\\locked.epub", isLocked=True), _file("a\\ok.epub")])
|
||||
assert [pf.basename for pf in iter_peer_files(response)] == ["ok.epub"]
|
||||
|
||||
def test_skips_garbage(self):
|
||||
response = _response(files=[None, {"filename": ""}, {"filename": "dir\\"}, "junk"])
|
||||
assert iter_peer_files(response) == []
|
||||
|
||||
def test_missing_username_yields_nothing(self):
|
||||
assert iter_peer_files({"files": [_file("a.epub")]}) == []
|
||||
|
||||
def test_size_coercion(self):
|
||||
response = _response(files=[_file("a.epub", size="12"), _file("b.epub", size=None)])
|
||||
assert [pf.size for pf in iter_peer_files(response)] == [12, 0]
|
||||
|
||||
|
||||
# ── build_releases ─────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestBuildReleases:
|
||||
def test_ebook_one_release_per_file(self):
|
||||
response = _response(
|
||||
files=[
|
||||
_file("@@a\\Books\\Dune.epub", 1000),
|
||||
_file("@@a\\Books\\Dune.mobi", 2000),
|
||||
_file("@@a\\Books\\cover.jpg", 50),
|
||||
]
|
||||
)
|
||||
releases = build_releases([response], content_type="ebook", formats=EBOOK_FORMATS)
|
||||
assert [r.title for r in releases] == ["Dune", "Dune"]
|
||||
assert [r.format for r in releases] == ["epub", "mobi"]
|
||||
assert releases[0].source == "slskd"
|
||||
assert releases[0].protocol == ReleaseProtocol.SOULSEEK
|
||||
assert releases[0].indexer == "peer"
|
||||
assert releases[0].content_type == "ebook"
|
||||
assert releases[0].size_bytes == 1000
|
||||
assert releases[0].size == "1000 B"
|
||||
assert releases[0].extra["username"] == "peer"
|
||||
assert releases[0].extra["files"] == [{"filename": "@@a\\Books\\Dune.epub", "size": 1000}]
|
||||
assert releases[0].extra["directory"] == "@@a\\Books"
|
||||
assert releases[0].extra["directory_name"] == "Books"
|
||||
assert releases[0].extra["availability"] == "Free slot"
|
||||
assert releases[0].extra["formats"] == ["epub"]
|
||||
|
||||
def test_format_filter_respected(self):
|
||||
response = _response(files=[_file("a\\x.pdf"), _file("a\\x.epub")])
|
||||
releases = build_releases([response], content_type="ebook", formats={"epub"})
|
||||
assert [r.format for r in releases] == ["epub"]
|
||||
|
||||
def test_bare_files_with_no_folder_are_not_grouped_into_one_audiobook(self):
|
||||
response = _response(
|
||||
files=[_file("01 Intro.mp3", 100), _file("Other Book.mp3", 200), _file("c.mp3", 300)]
|
||||
)
|
||||
|
||||
releases = build_releases([response], content_type="audiobook", formats=AUDIO_FORMATS)
|
||||
|
||||
assert sorted(r.extra["file_count"] for r in releases) == [1, 1, 1]
|
||||
|
||||
def test_audiobook_groups_directory(self):
|
||||
response = _response(
|
||||
files=[
|
||||
_file("@@a\\Audio\\Dune\\01.mp3", 100),
|
||||
_file("@@a\\Audio\\Dune\\02.mp3", 200),
|
||||
_file("@@a\\Audio\\Dune\\03.mp3", 300),
|
||||
_file("@@a\\Audio\\Dune\\cover.jpg", 5),
|
||||
_file("@@a\\Audio\\Other\\single.m4b", 999),
|
||||
]
|
||||
)
|
||||
releases = build_releases([response], content_type="audiobook", formats=AUDIO_FORMATS)
|
||||
assert len(releases) == 2
|
||||
grouped = next(r for r in releases if r.extra["file_count"] == 3)
|
||||
single = next(r for r in releases if r.extra["file_count"] == 1)
|
||||
assert grouped.title == "Dune (3 files)"
|
||||
assert grouped.format == "mp3"
|
||||
assert grouped.size_bytes == 600
|
||||
assert grouped.content_type == "audiobook"
|
||||
assert [f["filename"] for f in grouped.extra["files"]] == [
|
||||
"@@a\\Audio\\Dune\\01.mp3",
|
||||
"@@a\\Audio\\Dune\\02.mp3",
|
||||
"@@a\\Audio\\Dune\\03.mp3",
|
||||
]
|
||||
assert single.title == "single"
|
||||
assert single.format == "m4b"
|
||||
|
||||
def test_audiobook_archives_stay_separate(self):
|
||||
response = _response(
|
||||
files=[
|
||||
_file("@@a\\Dune\\Dune.zip", 5000),
|
||||
_file("@@a\\Dune\\01.mp3", 100),
|
||||
_file("@@a\\Dune\\02.mp3", 100),
|
||||
]
|
||||
)
|
||||
releases = build_releases([response], content_type="audiobook", formats=AUDIO_FORMATS)
|
||||
formats = sorted((r.format, r.extra["file_count"]) for r in releases)
|
||||
assert formats == [("mp3", 2), ("zip", 1)]
|
||||
|
||||
def test_dominant_format_wins_for_group(self):
|
||||
response = _response(
|
||||
files=[
|
||||
_file("@@a\\Dune\\01.flac"),
|
||||
_file("@@a\\Dune\\02.mp3"),
|
||||
_file("@@a\\Dune\\03.mp3"),
|
||||
]
|
||||
)
|
||||
(release,) = build_releases([response], content_type="audiobook", formats=AUDIO_FORMATS)
|
||||
assert release.format == "mp3"
|
||||
assert release.extra["formats"] == ["mp3", "flac"]
|
||||
|
||||
def test_dedupes_identical_files_across_responses(self):
|
||||
response = _response(files=[_file("a\\x.epub")])
|
||||
releases = build_releases([response, response], content_type="ebook", formats=EBOOK_FORMATS)
|
||||
assert len(releases) == 1
|
||||
|
||||
def test_same_file_from_two_peers_is_two_releases(self):
|
||||
a = _response(username="alice", files=[_file("a\\x.epub")])
|
||||
b = _response(username="bob", files=[_file("a\\x.epub")])
|
||||
releases = build_releases([a, b], content_type="ebook", formats=EBOOK_FORMATS)
|
||||
assert {r.indexer for r in releases} == {"alice", "bob"}
|
||||
assert len({r.source_id for r in releases}) == 2
|
||||
|
||||
def test_availability_labels(self):
|
||||
busy = _response(hasFreeUploadSlot=False, queueLength=0, files=[_file("a\\x.epub")])
|
||||
queued = _response(hasFreeUploadSlot=False, queueLength=4, files=[_file("a\\x.epub")])
|
||||
(busy_release,) = build_releases([busy], content_type="ebook", formats=EBOOK_FORMATS)
|
||||
(queued_release,) = build_releases([queued], content_type="ebook", formats=EBOOK_FORMATS)
|
||||
assert busy_release.extra["availability"] == "Busy"
|
||||
assert queued_release.extra["availability"] == "Queue 4"
|
||||
|
||||
|
||||
# ── ranking ────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestRanking:
|
||||
def test_free_slot_then_queue_then_speed(self):
|
||||
responses = [
|
||||
_response(username="slow-free", uploadSpeed=10, files=[_file("a\\x.epub")]),
|
||||
_response(
|
||||
username="fast-queued",
|
||||
hasFreeUploadSlot=False,
|
||||
queueLength=2,
|
||||
uploadSpeed=10**7,
|
||||
files=[_file("a\\x.epub")],
|
||||
),
|
||||
_response(username="fast-free", uploadSpeed=10**6, files=[_file("a\\x.epub")]),
|
||||
_response(
|
||||
username="short-queue",
|
||||
hasFreeUploadSlot=False,
|
||||
queueLength=1,
|
||||
uploadSpeed=1,
|
||||
files=[_file("a\\x.epub")],
|
||||
),
|
||||
]
|
||||
releases = build_releases(responses, content_type="ebook", formats=EBOOK_FORMATS)
|
||||
releases.sort(key=rank_key)
|
||||
assert [r.indexer for r in releases] == [
|
||||
"fast-free",
|
||||
"slow-free",
|
||||
"short-queue",
|
||||
"fast-queued",
|
||||
]
|
||||
|
||||
|
||||
# ── supported formats ──────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestSupportedFormats:
|
||||
def test_reads_ebook_formats(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(
|
||||
mod.config,
|
||||
"get",
|
||||
lambda k, d=None: ["EPUB", ".mobi"] if k == "SUPPORTED_FORMATS" else d,
|
||||
)
|
||||
assert get_supported_formats("ebook") == {"epub", "mobi"}
|
||||
|
||||
def test_reads_audiobook_formats(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(
|
||||
mod.config,
|
||||
"get",
|
||||
lambda k, d=None: ["m4b"] if k == "SUPPORTED_AUDIOBOOK_FORMATS" else d,
|
||||
)
|
||||
assert get_supported_formats("audiobook") == {"m4b"}
|
||||
|
||||
def test_falls_back_to_defaults(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", lambda k, d=None: d)
|
||||
assert "epub" in get_supported_formats("ebook")
|
||||
assert "mp3" in get_supported_formats("audiobook")
|
||||
assert "zip" in get_supported_formats("audiobook")
|
||||
|
||||
def test_accepts_comma_string(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", lambda k, d=None: "epub, pdf")
|
||||
assert get_supported_formats("ebook") == {"epub", "pdf"}
|
||||
|
||||
|
||||
# ── query planning ─────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestPlanQueries:
|
||||
def test_title_author_then_bare_title(self):
|
||||
assert _plan_queries(_make_plan()) == ["Dune Frank Herbert", "Dune"]
|
||||
|
||||
def test_manual_query_first(self):
|
||||
plan = _make_plan(manual_query="dune messiah herbert")
|
||||
assert _plan_queries(plan)[0] == "dune messiah herbert"
|
||||
|
||||
def test_dedupes_and_caps(self):
|
||||
plan = _make_plan(
|
||||
variants=[("Dune", "Frank Herbert"), ("dune", "frank herbert"), ("Dune Messiah", "X")]
|
||||
)
|
||||
queries = _plan_queries(plan)
|
||||
assert queries == ["Dune Frank Herbert", "Dune", "Dune Messiah X"]
|
||||
|
||||
def test_empty_plan(self):
|
||||
assert _plan_queries(_make_plan(variants=[])) == []
|
||||
|
||||
|
||||
# ── SlskdSource ────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestSlskdSource:
|
||||
def _config(self, **overrides):
|
||||
values = {"SLSKD_ENABLED": True, "SLSKD_URL": "http://slskd:5030", "SLSKD_API_KEY": "k"}
|
||||
values.update(overrides)
|
||||
return lambda k, default=None: values.get(k, default)
|
||||
|
||||
def test_registered(self):
|
||||
assert isinstance(get_source("slskd"), SlskdSource)
|
||||
|
||||
def test_available_when_enabled_and_url_set(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", self._config())
|
||||
assert SlskdSource().is_available() is True
|
||||
|
||||
def test_unavailable_when_disabled(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", self._config(SLSKD_ENABLED=False))
|
||||
assert SlskdSource().is_available() is False
|
||||
|
||||
def test_unavailable_without_url(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", self._config(SLSKD_URL=""))
|
||||
assert SlskdSource().is_available() is False
|
||||
|
||||
def test_column_config_serializes(self):
|
||||
from shelfmark.release_sources import serialize_column_config
|
||||
|
||||
data = serialize_column_config(SlskdSource().get_column_config())
|
||||
assert [c["key"] for c in data["columns"]] == [
|
||||
"extra.username",
|
||||
"extra.availability",
|
||||
"format",
|
||||
"size",
|
||||
]
|
||||
assert data["supported_filters"] == ["format"]
|
||||
assert data["leading_cell"]["type"] == "none"
|
||||
assert {o["sort_key"] for o in data["extra_sort_options"]} == {
|
||||
"extra.upload_speed",
|
||||
"extra.queue_length",
|
||||
}
|
||||
|
||||
def test_search_without_client_returns_empty(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", self._config(SLSKD_URL=""))
|
||||
assert SlskdSource().search(_make_book(), _make_plan()) == []
|
||||
|
||||
def test_search_stops_after_first_query_with_results(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", self._config())
|
||||
client = MagicMock()
|
||||
client.search.return_value = [_response(files=[_file("a\\Dune.epub")])]
|
||||
monkeypatch.setattr(SlskdSource, "_get_client", lambda self: client)
|
||||
|
||||
releases = SlskdSource().search(_make_book(), _make_plan())
|
||||
|
||||
assert len(releases) == 1
|
||||
client.search.assert_called_once()
|
||||
assert client.search.call_args.args == ("Dune Frank Herbert",)
|
||||
kwargs = client.search.call_args.kwargs
|
||||
assert kwargs["wait_seconds"] == 15
|
||||
assert kwargs["response_limit"] == 100
|
||||
|
||||
def test_search_falls_back_to_bare_title(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", self._config())
|
||||
client = MagicMock()
|
||||
client.search.side_effect = [[], [_response(files=[_file("a\\Dune.epub")])]]
|
||||
monkeypatch.setattr(SlskdSource, "_get_client", lambda self: client)
|
||||
|
||||
releases = SlskdSource().search(_make_book(), _make_plan())
|
||||
|
||||
assert len(releases) == 1
|
||||
assert [c.args[0] for c in client.search.call_args_list] == ["Dune Frank Herbert", "Dune"]
|
||||
|
||||
def test_expand_search_runs_every_query(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", self._config())
|
||||
client = MagicMock()
|
||||
client.search.side_effect = [
|
||||
[_response(files=[_file("a\\Dune.epub")])],
|
||||
[_response(username="other", files=[_file("b\\Dune.epub")])],
|
||||
]
|
||||
monkeypatch.setattr(SlskdSource, "_get_client", lambda self: client)
|
||||
|
||||
releases = SlskdSource().search(_make_book(), _make_plan(), expand_search=True)
|
||||
|
||||
assert len(releases) == 2
|
||||
assert client.search.call_count == 2
|
||||
|
||||
def test_search_survives_client_errors(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", self._config())
|
||||
client = MagicMock()
|
||||
client.search.side_effect = [RuntimeError("boom"), [_response(files=[_file("a\\x.epub")])]]
|
||||
monkeypatch.setattr(SlskdSource, "_get_client", lambda self: client)
|
||||
|
||||
assert len(SlskdSource().search(_make_book(), _make_plan())) == 1
|
||||
|
||||
def test_search_uses_configured_limits(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(
|
||||
mod.config, "get", self._config(SLSKD_SEARCH_TIMEOUT="30", SLSKD_RESPONSE_LIMIT=5000)
|
||||
)
|
||||
client = MagicMock()
|
||||
client.search.return_value = []
|
||||
monkeypatch.setattr(SlskdSource, "_get_client", lambda self: client)
|
||||
|
||||
SlskdSource().search(_make_book(), _make_plan())
|
||||
|
||||
kwargs = client.search.call_args_list[0].kwargs
|
||||
assert kwargs["wait_seconds"] == 30
|
||||
assert kwargs["response_limit"] == 1000 # clamped
|
||||
|
||||
def test_search_audiobook_uses_audio_formats(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", self._config())
|
||||
client = MagicMock()
|
||||
client.search.return_value = [
|
||||
_response(
|
||||
files=[_file("a\\Dune\\01.mp3"), _file("a\\Dune\\02.mp3"), _file("a\\Dune.epub")]
|
||||
)
|
||||
]
|
||||
monkeypatch.setattr(SlskdSource, "_get_client", lambda self: client)
|
||||
|
||||
releases = SlskdSource().search(_make_book(), _make_plan(), content_type="audiobook")
|
||||
|
||||
assert len(releases) == 1
|
||||
assert releases[0].extra["file_count"] == 2
|
||||
|
||||
|
||||
# ── fork additions: containers, merged variants, strict-match signals ───────────
|
||||
|
||||
|
||||
class TestWholeBookContainers:
|
||||
def test_an_m4b_next_to_loose_chapters_is_its_own_release(self):
|
||||
response = _response(
|
||||
files=[
|
||||
_file("@@a\\Dune\\Dune.m4b", 900),
|
||||
_file("@@a\\Dune\\01.mp3", 100),
|
||||
_file("@@a\\Dune\\02.mp3", 100),
|
||||
_file("@@a\\Dune\\03.mp3", 100),
|
||||
]
|
||||
)
|
||||
|
||||
releases = build_releases([response], content_type="audiobook", formats=AUDIO_FORMATS)
|
||||
|
||||
assert sorted((r.format, r.extra["file_count"]) for r in releases) == [
|
||||
("m4b", 1),
|
||||
("mp3", 3),
|
||||
]
|
||||
|
||||
def test_two_m4b_files_in_one_folder_are_two_releases(self):
|
||||
response = _response(files=[_file("@@a\\Dune\\A.m4b", 5), _file("@@a\\Dune\\B.m4b", 6)])
|
||||
|
||||
releases = build_releases([response], content_type="audiobook", formats=AUDIO_FORMATS)
|
||||
|
||||
assert sorted(r.extra["file_count"] for r in releases) == [1, 1]
|
||||
|
||||
def test_mixed_chapter_formats_stay_one_release(self):
|
||||
response = _response(files=[_file("@@a\\Dune\\01.flac"), _file("@@a\\Dune\\02.mp3")])
|
||||
|
||||
releases = build_releases([response], content_type="audiobook", formats=AUDIO_FORMATS)
|
||||
|
||||
assert len(releases) == 1
|
||||
|
||||
|
||||
class TestMergedQueryVariants:
|
||||
def test_chapters_found_by_different_queries_are_one_release(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", TestSlskdSource()._config())
|
||||
client = MagicMock()
|
||||
client.search.side_effect = [
|
||||
[_response(files=[_file("@@a\\Dune\\01.mp3"), _file("@@a\\Dune\\02.mp3")])],
|
||||
[_response(files=[_file("@@a\\Dune\\02.mp3"), _file("@@a\\Dune\\03.mp3")])],
|
||||
]
|
||||
monkeypatch.setattr(SlskdSource, "_get_client", lambda self: client)
|
||||
|
||||
releases = SlskdSource().search(
|
||||
_make_book(), _make_plan(), expand_search=True, content_type="audiobook"
|
||||
)
|
||||
|
||||
assert len(releases) == 1
|
||||
assert [f["filename"] for f in releases[0].extra["files"]] == [
|
||||
"@@a\\Dune\\01.mp3",
|
||||
"@@a\\Dune\\02.mp3",
|
||||
"@@a\\Dune\\03.mp3",
|
||||
]
|
||||
|
||||
def test_merging_keeps_each_peers_own_slot_details(self):
|
||||
from shelfmark.release_sources.slskd.source import merge_responses
|
||||
|
||||
first = _response(username="a", files=[_file("x\\1.mp3")], hasFreeUploadSlot=True)
|
||||
second = _response(
|
||||
username="a", files=[_file("x\\2.mp3")], hasFreeUploadSlot=False, queueLength=9
|
||||
)
|
||||
other = _response(username="b", files=[_file("y\\1.mp3")])
|
||||
|
||||
merged = merge_responses([[first], [second, other]])
|
||||
|
||||
assert sorted(r["username"] for r in merged) == ["a", "b"]
|
||||
a = next(r for r in merged if r["username"] == "a")
|
||||
assert sorted(f["filename"] for f in a["files"]) == ["x\\1.mp3", "x\\2.mp3"]
|
||||
assert a["hasFreeUploadSlot"] is True # the first answer's details win
|
||||
|
||||
|
||||
class TestStrictMatchSignals:
|
||||
"""Auto-download looks for the author in the title, the peer name and extra['author'], and
|
||||
for bundle names in extra['title_raw']. A Soulseek result has the author only in its folder."""
|
||||
|
||||
def test_the_folder_path_is_offered_as_the_author_text(self):
|
||||
response = _response(files=[_file("@@a\\Books\\Frank Herbert\\Dune\\Dune.epub")])
|
||||
|
||||
(release,) = build_releases([response], content_type="ebook", formats=EBOOK_FORMATS)
|
||||
|
||||
assert "Frank Herbert" in release.extra["author"]
|
||||
assert "Dune" in release.extra["author"]
|
||||
|
||||
def test_a_folder_release_also_offers_its_path_as_the_author_text(self):
|
||||
response = _response(
|
||||
files=[
|
||||
_file("@@a\\Audio\\Frank Herbert\\Dune\\01.mp3"),
|
||||
_file("@@a\\Audio\\Frank Herbert\\Dune\\02.mp3"),
|
||||
]
|
||||
)
|
||||
|
||||
(release,) = build_releases([response], content_type="audiobook", formats=AUDIO_FORMATS)
|
||||
|
||||
assert "Frank Herbert" in release.extra["author"]
|
||||
|
||||
def test_the_folder_name_is_kept_for_bundle_detection(self):
|
||||
response = _response(
|
||||
files=[
|
||||
_file("@@a\\Herbert\\Dune Complete Series\\01.mp3"),
|
||||
_file("@@a\\Herbert\\Dune Complete Series\\02.mp3"),
|
||||
]
|
||||
)
|
||||
|
||||
(release,) = build_releases([response], content_type="audiobook", formats=AUDIO_FORMATS)
|
||||
|
||||
assert "Complete Series" in release.extra["title_raw"]
|
||||
|
||||
|
||||
class TestAvailability:
|
||||
def test_unavailable_without_an_api_key(self, monkeypatch):
|
||||
import shelfmark.release_sources.slskd.source as mod
|
||||
|
||||
monkeypatch.setattr(mod.config, "get", TestSlskdSource()._config(SLSKD_API_KEY=""))
|
||||
|
||||
assert SlskdSource().is_available() is False
|
||||
Reference in new issue
Block a user