mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-11-15 17:38:16 +00:00
## Add Soulseek (slskd) as a release source, building on #1335 This is @dskvr's work from #1335 rebased onto current `main`, with my changes on top. #1335 has been quiet since 2026-09-13 and I needed the changes below, so I carried it forward instead of starting over. Their two commits are the first two in this branch with their authorship intact, and the source plugin, handler, API client, settings tab, tests, `ReleaseProtocol.SOULSEEK`, test client compose file and frontend bits are all theirs. I haven't changed how a search turns into releases beyond what is listed here. If @dskvr would rather keep #1335 alive and take these as commits on their branch, I can do that instead and close this. ### What I changed **Downloads survive other apps using the same slskd.** The handler found its files by listing slskd's downloads, and another app on the same instance (SoulSync here) sends `DELETE /transfers/downloads/all/completed` after each of its batches. Once that ran, a finished download no longer appeared in the list and the task never saw it complete, even though the files were on disk. With the new `SLSKD_ISOLATE_DOWNLOADS` setting (on by default) each task: - enqueues one batch (`POST /transfers/downloads/batches`) into its own folder, `<SLSKD_DESTINATION_PREFIX>/<task id>` - polls each transfer by id, which slskd still answers after the list has been cleared - decides it is done from the files on disk (sizes, not names), with the `/events` log as a fallback when the transfers are gone and the files aren't visible yet - copies out of that folder, and removes it only if `SLSKD_KEEP_COMPLETED` is off and the import worked With isolation off the handler behaves as it did in #1335. If the batch endpoint isn't there (404/405) it falls back to the original per-file enqueue. **Source changes.** m4b/m4a files are their own release next to loose chapter files, so auto-download can prefer one file over a chapter set. Results from the query variants are merged per peer. The folder text is exposed as `extra["author"]` and `extra["title_raw"]`, which is what auto-download's strict match and pack detection read. `is_available()` also needs an API key now. **Smaller fixes.** Cancelling while files are still being located now removes the transfer records it created. In the settings tab, a cleared form field is no longer replaced by the saved value when testing the connection. The new settings are in `docs/environment-variables.md`. ### Testing `tests/slskd` has 192 passing and 6 live tests that skip without an instance. I wrote the new tests first and checked the handler and source rules by breaking them and confirming a test failed. Ruff is clean. On the whole suite the only failures are 4 in `tests/download` that import `seleniumbase`, which isn't installed in my environment, and they fail the same on `main`. I've been running this on my own deployment against slskd 0.26.0 behind a VPN, with SoulSync active on a second instance. A search returned a couple hundred results from about 70 peers, an epub came down and was imported, and auto-download fetched five audiobooks as single m4b files. That is a day of use, so I don't know how it behaves on other slskd versions or on Windows path mappings. --------- Co-authored-by: sandwich <dskvr@users.noreply.github.com> Co-authored-by: sandwich <299465+dskvr@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
22 files changed
+5434
-2
No files matched your search
@@ -47,6 +47,14 @@ First-Time Setup:
|
||||
- Complete the setup wizard at http://localhost:8085
|
||||
- API key will be auto-detected by this script
|
||||
- In cwabd, copy API key from SABnzbd Config > General
|
||||
|
||||
slskd (Soulseek):
|
||||
- Before the first `up`, write .local/test-clients/slskd/config/slskd.yml.
|
||||
Run `uv run python scripts/test_clients.py --write-slskd-config` to create
|
||||
one with a throwaway Soulseek login and a fresh API key, then create the
|
||||
download folders it names (this script does both).
|
||||
- In Shelfmark: Settings > Soulseek (slskd), URL http://slskd:5030 and the
|
||||
API key from that file; Downloads Path /downloads/slskd/complete.
|
||||
"""
|
||||
|
||||
import sys
|
||||
@@ -84,8 +92,38 @@ CONFIG = {
|
||||
"rtorrent": {
|
||||
"url": "http://localhost:8000/RPC2",
|
||||
},
|
||||
# Soulseek
|
||||
"slskd": {
|
||||
"url": "http://localhost:5030",
|
||||
"config_file": Path(".local/test-clients/slskd/config/slskd.yml"),
|
||||
},
|
||||
}
|
||||
|
||||
SLSKD_CONFIG_TEMPLATE = """# slskd test-stack config (generated by scripts/test_clients.py; not committed).
|
||||
# Soulseek creates the account on first login, so any unused username works.
|
||||
web:
|
||||
port: 5030
|
||||
authentication:
|
||||
disabled: false
|
||||
username: slskd
|
||||
password: slskdslskd
|
||||
api_keys:
|
||||
shelfmark:
|
||||
key: {api_key}
|
||||
role: readwrite
|
||||
cidr: 0.0.0.0/0,::/0
|
||||
soulseek:
|
||||
username: {soulseek_username}
|
||||
password: {soulseek_password}
|
||||
description: shelfmark test-stack client
|
||||
directories:
|
||||
downloads: /downloads/slskd/complete
|
||||
incomplete: /downloads/slskd/incomplete
|
||||
shares:
|
||||
directories:
|
||||
- /shares
|
||||
"""
|
||||
|
||||
# Test magnet link (Ubuntu ISO - legal, small metadata)
|
||||
TEST_MAGNET = "magnet:?xt=urn:btih:3b245504cf5f11bbdbe1201cea6a6bf45aee1bc0&dn=ubuntu-22.04.3-live-server-amd64.iso"
|
||||
|
||||
@@ -522,7 +560,85 @@ def test_rtorrent() -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _read_slskd_api_key() -> str | None:
|
||||
"""Pull the shelfmark API key out of the generated slskd.yml (no YAML dependency)."""
|
||||
config_file = CONFIG["slskd"]["config_file"]
|
||||
if not config_file.exists():
|
||||
return None
|
||||
for line in config_file.read_text().splitlines():
|
||||
stripped = line.strip()
|
||||
if stripped.startswith("key:"):
|
||||
return stripped.split(":", 1)[1].strip()
|
||||
return None
|
||||
|
||||
|
||||
def write_slskd_config() -> Path:
|
||||
"""Write a fresh slskd.yml with a throwaway Soulseek login and API key."""
|
||||
import secrets
|
||||
|
||||
config_file = CONFIG["slskd"]["config_file"]
|
||||
config_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
suffix = secrets.token_hex(2)
|
||||
config_file.write_text(
|
||||
SLSKD_CONFIG_TEMPLATE.format(
|
||||
api_key=secrets.token_hex(24),
|
||||
soulseek_username=f"shelfmark-dev-{suffix}",
|
||||
soulseek_password=secrets.token_urlsafe(12),
|
||||
)
|
||||
)
|
||||
for folder in (
|
||||
"shares/TestBooks",
|
||||
"../downloads/slskd/complete",
|
||||
"../downloads/slskd/incomplete",
|
||||
):
|
||||
(config_file.parent.parent / folder).resolve().mkdir(parents=True, exist_ok=True)
|
||||
return config_file
|
||||
|
||||
|
||||
def test_slskd() -> bool:
|
||||
"""Test slskd connection and Soulseek login state."""
|
||||
print("\n" + "=" * 50)
|
||||
print("Testing slskd (Soulseek)")
|
||||
print("=" * 50)
|
||||
|
||||
# The slskd client needs the app's log/config dirs; keep them out of the way.
|
||||
import os
|
||||
import tempfile
|
||||
|
||||
scratch = tempfile.mkdtemp(prefix="shelfmark-test-clients-")
|
||||
for var, sub in (("LOG_ROOT", ""), ("CONFIG_DIR", "config"), ("TMP_DIR", "tmp")):
|
||||
os.environ.setdefault(var, str(Path(scratch, sub)))
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
from shelfmark.release_sources.slskd.api import SlskdClient
|
||||
|
||||
api_key = _read_slskd_api_key()
|
||||
if not api_key:
|
||||
print(f" ERROR: no API key found in {CONFIG['slskd']['config_file']}")
|
||||
print(" Run: uv run python scripts/test_clients.py --write-slskd-config")
|
||||
return False
|
||||
|
||||
client = SlskdClient(CONFIG["slskd"]["url"], api_key, timeout=10)
|
||||
success, message = client.test_connection()
|
||||
print(f" {message}")
|
||||
if not success:
|
||||
print(" Is the container running? docker ps | grep slskd")
|
||||
return False
|
||||
|
||||
print(f" Downloads directory (in slskd): {client.get_download_directory()}")
|
||||
search_id = client.start_search(
|
||||
"shelfmark-connectivity-probe", timeout_ms=1000, response_limit=1
|
||||
)
|
||||
print(f" Search API ok (id {search_id[:8]}...)")
|
||||
client.delete_search(search_id)
|
||||
print(" SUCCESS: slskd is working!")
|
||||
return True
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if "--write-slskd-config" in sys.argv:
|
||||
print(f"Wrote {write_slskd_config()}")
|
||||
return 0
|
||||
|
||||
print("Download Client Test Suite")
|
||||
print("=" * 50)
|
||||
print("Make sure containers are running:")
|
||||
@@ -546,6 +662,12 @@ def main() -> int:
|
||||
results["deluge"] = test_deluge()
|
||||
results["rtorrent"] = test_rtorrent()
|
||||
|
||||
# Test Soulseek
|
||||
print("\n" + "=" * 50)
|
||||
print("SOULSEEK")
|
||||
print("=" * 50)
|
||||
results["slskd"] = test_slskd()
|
||||
|
||||
# Summary
|
||||
print("\n" + "=" * 50)
|
||||
print("SUMMARY")
|
||||
|
||||
Reference in new issue
Block a user