starred/shelfmark
1
0 Fork 0
mirror of https://github.com/calibrain/shelfmark.git synced 2026-11-10 12:38:32 +00:00
Code Issues Packages Projects Releases 10 Wiki Activity
Files
e575ff0fcb586bc0aa0600a2ea198d5b8c05dd5d
shelfmark/docker-compose.test-clients.yml
T
splitsec2sandwichsandwichCopilot Autofix powered by AI
 bb1655238a feat(slskd): add Soulseek (slskd) release source, building on #1335 (#1428) 
## Add Soulseek (slskd) as a release source, building on #1335

This is @dskvr's work from #1335 rebased onto current `main`, with my
changes on top. #1335 has been quiet since 2026-09-13 and I needed the
changes below, so I carried it forward instead of starting over. Their
two commits are the first two in this branch with their authorship
intact, and the source plugin, handler, API client, settings tab, tests,
`ReleaseProtocol.SOULSEEK`, test client compose file and frontend bits
are all theirs. I haven't changed how a search turns into releases
beyond what is listed here.

If @dskvr would rather keep #1335 alive and take these as commits on
their branch, I can do that instead and close this.

### What I changed

**Downloads survive other apps using the same slskd.** The handler found
its files by listing slskd's downloads, and another app on the same
instance (SoulSync here) sends `DELETE
/transfers/downloads/all/completed` after each of its batches. Once that
ran, a finished download no longer appeared in the list and the task
never saw it complete, even though the files were on disk. With the new
`SLSKD_ISOLATE_DOWNLOADS` setting (on by default) each task:

- enqueues one batch (`POST /transfers/downloads/batches`) into its own
folder, `<SLSKD_DESTINATION_PREFIX>/<task id>`
- polls each transfer by id, which slskd still answers after the list
has been cleared
- decides it is done from the files on disk (sizes, not names), with the
`/events` log as a fallback when the transfers are gone and the files
aren't visible yet
- copies out of that folder, and removes it only if
`SLSKD_KEEP_COMPLETED` is off and the import worked

With isolation off the handler behaves as it did in #1335. If the batch
endpoint isn't there (404/405) it falls back to the original per-file
enqueue.

**Source changes.** m4b/m4a files are their own release next to loose
chapter files, so auto-download can prefer one file over a chapter set.
Results from the query variants are merged per peer. The folder text is
exposed as `extra["author"]` and `extra["title_raw"]`, which is what
auto-download's strict match and pack detection read. `is_available()`
also needs an API key now.

**Smaller fixes.** Cancelling while files are still being located now
removes the transfer records it created. In the settings tab, a cleared
form field is no longer replaced by the saved value when testing the
connection. The new settings are in `docs/environment-variables.md`.

### Testing

`tests/slskd` has 192 passing and 6 live tests that skip without an
instance. I wrote the new tests first and checked the handler and source
rules by breaking them and confirming a test failed. Ruff is clean. On
the whole suite the only failures are 4 in `tests/download` that import
`seleniumbase`, which isn't installed in my environment, and they fail
the same on `main`.

I've been running this on my own deployment against slskd 0.26.0 behind
a VPN, with SoulSync active on a second instance. A search returned a
couple hundred results from about 70 peers, an epub came down and was
imported, and auto-download fetched five audiobooks as single m4b files.
That is a day of use, so I don't know how it behaves on other slskd
versions or on Windows path mappings.

---------

Co-authored-by: sandwich <dskvr@users.noreply.github.com>
Co-authored-by: sandwich <299465+dskvr@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-10-20 01:33:12 -04:00

221 lines
6.9 KiB
YAML
Raw Blame History

# Test stack for download client development
# Includes shelfmark + all download clients on same network with shared volumes
#
# Usage:
# docker compose -f docker-compose.test-clients.yml up -d
# # Access shelfmark at http://localhost:8084
# # Configure clients in Settings > Prowlarr > Download Clients
#
# Web UIs:
# - shelfmark: http://localhost:8084
# - Prowlarr: http://localhost:9696 (no auth by default)
# - qBittorrent: http://localhost:8080 (check container logs for temp password)
# - Transmission: http://localhost:9091 (admin / admin)
# - Deluge: http://localhost:8112 (password: deluge)
# - NZBGet: http://localhost:6789 (nzbget / tegbzn6789)
# - NZBHydra: http://localhost:5076 (no auth by default)
# - SABnzbd: http://localhost:8085 (complete setup wizard for API key)
# - rTorrent: http://localhost:8000 (admin / admin - if auth enabled)
# - slskd: http://localhost:5030 (slskd / slskdslskd; API key in .local/test-clients/slskd/config/slskd.yml)
#
services:
shelfmark:
build:
context: .
dockerfile: Dockerfile
target: shelfmark
container_name: test-shelfmark
cap_add:
- SYS_PTRACE
environment:
TZ: UTC
DEBUG: "true"
# All client configuration is done via Settings UI
# Use Docker service names for URLs:
# - qBittorrent: http://qbittorrent:8080
# - Transmission: http://transmission:9091
# - Deluge Web UI: http://deluge:8112
# - NZBGet: http://nzbget:6789
# - NZBHydra: http://nzbhydra:5076
# - SABnzbd: http://sabnzbd:8080
# - rTorrent: http://rtorrent:80 (XMLRPC via HTTP) or rtorrent (port 5000 for SCGI)
# - slskd: http://slskd:5030 (downloads land in /downloads/slskd/complete)
ports:
- "8084:8084"
volumes:
# Config and state
- ./.local/test-clients/shelfmark/config:/config
- ./.local/test-clients/shelfmark/log:/var/log/shelfmark
# Book destination directory (where completed books go)
- ./.local/test-clients/books:/books
# Staging directory
- ./.local/test-clients/tmp:/tmp/shelfmark
# CRITICAL: Mount client download directories so shelfmark can access completed files
- ./.local/test-clients/downloads:/downloads
# Mount source code for hot-reload (no rebuild needed for Python changes)
- ./shelfmark:/app/shelfmark:ro
# Mount tests for running pytest in container
- ./tests:/app/tests:ro
- ./pyproject.toml:/app/pyproject.toml:ro
- ./uv.lock:/app/uv.lock:ro
# Mount client configs for integration tests to read credentials
- ./.local/test-clients/qbittorrent/config:/qbittorrent-config:ro
- ./.local/test-clients/sabnzbd/config:/sabnzbd-config:ro
depends_on:
- nzbget
- nzbhydra
- sabnzbd
- qbittorrent
- transmission
- deluge
- rtorrent
- slskd
restart: unless-stopped
prowlarr:
image: lscr.io/linuxserver/prowlarr:latest
container_name: test-prowlarr
environment:
- PUID=1000
- PGID=1000
- TZ=UTC
volumes:
- ./.local/test-clients/prowlarr/config:/config
ports:
- "9696:9696"
restart: unless-stopped
nzbget:
image: lscr.io/linuxserver/nzbget:latest
container_name: test-nzbget
environment:
- PUID=1000
- PGID=1000
- TZ=UTC
volumes:
- ./.local/test-clients/nzbget/config:/config
- ./.local/test-clients/downloads:/downloads
- ./.local/test-clients/nzbget/custom-cont-init.d:/custom-cont-init.d:ro
ports:
- "6789:6789" # Web UI / JSON-RPC
restart: unless-stopped
sabnzbd:
image: lscr.io/linuxserver/sabnzbd:latest
container_name: test-sabnzbd
environment:
- PUID=1000
- PGID=1000
- TZ=UTC
volumes:
- ./.local/test-clients/sabnzbd/config:/config
- ./.local/test-clients/downloads:/downloads
ports:
- "8085:8080" # Web UI (external:internal)
restart: unless-stopped
qbittorrent:
image: lscr.io/linuxserver/qbittorrent:latest
container_name: test-qbittorrent
environment:
- PUID=1000
- PGID=1000
- TZ=UTC
- WEBUI_PORT=8080
volumes:
- ./.local/test-clients/qbittorrent/config:/config
- ./.local/test-clients/downloads:/downloads
- ./.local/test-clients/qbittorrent/custom-cont-init.d:/custom-cont-init.d:ro
ports:
- "8080:8080" # Web UI / API
- "6882:6881"
- "6882:6881/udp"
restart: unless-stopped
transmission:
image: lscr.io/linuxserver/transmission:latest
container_name: test-transmission
environment:
- PUID=1000
- PGID=1000
- TZ=UTC
- USER=admin
- PASS=admin
volumes:
- ./.local/test-clients/transmission/config:/config
- ./.local/test-clients/downloads:/downloads
ports:
- "9091:9091" # Web UI / RPC
- "51413:51413"
- "51413:51413/udp"
restart: unless-stopped
deluge:
image: lscr.io/linuxserver/deluge:latest
container_name: test-deluge
environment:
- PUID=1000
- PGID=1000
- TZ=UTC
- DELUGE_LOGLEVEL=error
volumes:
- ./.local/test-clients/deluge/config:/config
- ./.local/test-clients/downloads:/downloads
ports:
- "8112:8112" # Web UI
- "58846:58846" # Daemon RPC
- "6881:6881"
- "6881:6881/udp"
restart: unless-stopped
rtorrent:
image: crazymax/rtorrent-rutorrent:latest # linuxserver has deprecated their rtorrent image
container_name: test-rtorrent
environment:
- PUID=1000
- PGID=1000
- TZ=UTC
volumes:
- ./.local/test-clients/rtorrent/config:/config
- ./.local/test-clients/downloads:/downloads
ports:
- "8000:8000" # XMLRPC
- "8089:8080" # ruTorrent Web UI
- "9000:9000" # SCGI port
- "50000:50000" # Incoming connections
- "6881:6881/udp"
restart: unless-stopped
nzbhydra:
image: lscr.io/linuxserver/nzbhydra2:latest
container_name: nzbhydra
environment:
- PUID=1000
- PGID=1000
- TZ=Europe/London
volumes:
- ./.local/test-clients/nzbhydra/config:/config
- ./.local/test-clients/downloads:/downloads
ports:
- 5076:5076
restart: unless-stopped
slskd:
image: slskd/slskd:latest
container_name: test-slskd
environment:
- TZ=UTC
# Bootstrap: write .local/test-clients/slskd/config/slskd.yml first (see
# scripts/test_clients.py for a template). It holds the Soulseek login, the
# API key Shelfmark uses, and the download directories below.
- SLSKD_CONFIG=/app/slskd.yml
volumes:
- ./.local/test-clients/slskd/config:/app
- ./.local/test-clients/slskd/shares:/shares:ro
- ./.local/test-clients/downloads:/downloads
ports:
- "5030:5030" # Web UI / API
- "50300:50300" # Soulseek peer listen port
restart: unless-stopped
Reference in new issue View Git Blame Copy Permalink