mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-11-18 00:00:33 +00:00
feat(download): sweep dead torrents and Soulseek leftovers on their own timer (#1619)
This adds two cleanups that nobody notices until they matter, and puts them on their own timer. Both do nothing until their setting and client are configured. The dead-torrent sweep removes a torrent Shelfmark added once it has been stalled, or stuck fetching metadata, for `STALE_TORRENT_HOURS` with no running download owning it, and its partial data goes with it. It only touches torrents this instance added, tracked in a ledger of hashes, whatever category they are in, and only on qBittorrent. The default is 6 hours (3 to 72, and 0 turns it off). I am not sure 6 is right for everyone, so say if you would rather ship it off. The Soulseek sweep covers what a restart leaves in slskd's downloads folder. A finished folder nothing imported gets imported, a partial folder is removed after `SLSKD_CLEAN_LEFTOVERS_DAYS`, and unfinished transfers no running download owns are picked up again when their request still waits for them. Both run from a small background thread every 15 minutes, after a 30 second startup delay. I first ran them from the Hardcover scheduler, but that only runs when Hardcover sync or auto-download is on, so anyone using neither would never get the cleanup. A separate thread keeps them independent, and the interval is fixed so there is no new setting. The new tests mock the sweeps. I also started a scratch instance, and the thread starts and the first cycle is a no-op when nothing is configured. The timer has not run against a real qBittorrent or slskd, though the two sweeps themselves have been in my fork's builds since October. Full Python suite, ruff and basedpyright pass. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Shelfmark periodically recovers eligible complete Soulseek downloads and reattaches unfinished transfers. It can also clean up eligible old folders while protecting active, wanted, and already-imported downloads. * Shelfmark can remove tracked qBittorrent torrents that have stalled or exceeded the configured inactivity limit, while preserving torrents associated with active tasks. Set the limit to 0 to disable cleanup. * **Bug Fixes** * Isolated Soulseek downloads reject task IDs that could produce unsafe or mismatched destination folders. * **Documentation** * Added configuration guidance for Soulseek recovery and cleanup, and qBittorrent stale-torrent cleanup, including defaults and eligibility rules. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
1 parent
e575ff0fcb
commit
bb13fae0bb
20 files changed
+2611
-6
No files matched your search
@@ -1904,6 +1904,8 @@ Automatically retry search without category filtering if no results are found
|
||||
| `SLSKD_ISOLATE_DOWNLOADS` | Download into a folder of its own inside slskd's downloads folder (needs slskd 0.26 or newer). Other apps using the same slskd cannot mix files with it or lose track of it when they clear slskd's list of finished transfers, and Shelfmark confirms a download by the files on disk. | boolean | `true` |
|
||||
| `SLSKD_DESTINATION_PREFIX` | The folder, inside slskd's downloads folder, that holds one folder per download. It must not climb out of slskd's downloads folder. | string | `shelfmark` |
|
||||
| `SLSKD_KEEP_COMPLETED` | Leave the finished files (and their transfer records) in slskd after Shelfmark copies them to your library, so they stay shared with other Soulseek users. Turn off to delete them once imported. | boolean | `true` |
|
||||
| `SLSKD_FINISH_LEFTOVERS` | A restart can land after the last file arrives and before it is copied to your library. Checked every 15 minutes, independent of the Hardcover sync, a download folder that holds every file asked for, whose download never completed and is still wanted, is imported once without downloading anything again (never one someone cancelled). If that import fails, the folder stays until you retry the download, or, with clean-up on, until it is a day old and its request has the book or was given up on. A transfer Shelfmark queued that no download is watching any more is picked up again if its request still needs it, a few per check, and cancelled otherwise, or when it cannot be picked up. Transfers queued by other apps or by hand in slskd are never touched, including the same file queued again after Shelfmark's download of it ended. | boolean | `true` |
|
||||
| `SLSKD_CLEAN_LEFTOVERS_DAYS` | Remove the folder of a failed or cancelled download, or one nothing records, once the download is this old, or one day old when its request already has the book or was given up on. A folder that holds every file asked for is never removed while the download is still wanted (you started it yourself, or its request is still open): it is imported once, and after that it waits for you to retry the download. The folder of a running download is never removed. 0 keeps every leftover folder. Folders already imported follow Keep finished files in slskd, not this setting. | number | `0` |
|
||||
| `SLSKD_SEARCH_TIMEOUT` | How long a Soulseek search collects peer responses. Longer waits find more peers but make every search slower. | number | `15` |
|
||||
| `SLSKD_RESPONSE_LIMIT` | Stop a search early once this many peers have answered | number | `100` |
|
||||
| `SLSKD_QUEUE_TIMEOUT_MINUTES` | Give up when the peer has not started sending within this long. Set to 0 to wait indefinitely. | number | `60` |
|
||||
@@ -1976,6 +1978,25 @@ Leave the finished files (and their transfer records) in slskd after Shelfmark c
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
|
||||
#### `SLSKD_FINISH_LEFTOVERS`
|
||||
|
||||
**Import finished folders an interruption left behind**
|
||||
|
||||
A restart can land after the last file arrives and before it is copied to your library. Checked every 15 minutes, independent of the Hardcover sync, a download folder that holds every file asked for, whose download never completed and is still wanted, is imported once without downloading anything again (never one someone cancelled). If that import fails, the folder stays until you retry the download, or, with clean-up on, until it is a day old and its request has the book or was given up on. A transfer Shelfmark queued that no download is watching any more is picked up again if its request still needs it, a few per check, and cancelled otherwise, or when it cannot be picked up. Transfers queued by other apps or by hand in slskd are never touched, including the same file queued again after Shelfmark's download of it ended.
|
||||
|
||||
- **Type:** boolean
|
||||
- **Default:** `true`
|
||||
|
||||
#### `SLSKD_CLEAN_LEFTOVERS_DAYS`
|
||||
|
||||
**Remove leftover download folders after (days)**
|
||||
|
||||
Remove the folder of a failed or cancelled download, or one nothing records, once the download is this old, or one day old when its request already has the book or was given up on. A folder that holds every file asked for is never removed while the download is still wanted (you started it yourself, or its request is still open): it is imported once, and after that it waits for you to retry the download. The folder of a running download is never removed. 0 keeps every leftover folder. Folders already imported follow Keep finished files in slskd, not this setting.
|
||||
|
||||
- **Type:** number
|
||||
- **Default:** `0`
|
||||
- **Constraints:** min: 0, max: 365
|
||||
|
||||
#### `SLSKD_SEARCH_TIMEOUT`
|
||||
|
||||
**Search Wait (seconds)**
|
||||
@@ -2353,6 +2374,7 @@ How long to keep cached search results before they expire.
|
||||
| `TORRENT_MIN_SEEDERS` | Seeds the client must see for a motionless torrent to keep waiting. 0 also turns the no-seed check off. Only qBittorrent reports seed counts. | number | `1` |
|
||||
| `PROWLARR_TORRENT_POST_IMPORT_CATEGORY` | Category or label to assign after a successful import | string | _empty string_ |
|
||||
| `TORRENT_STALL_TIMEOUT_MINUTES` | How long a torrent can go without any progress before Shelfmark cancels it. Applies from when the torrent starts and restarts each time it moves. Raise it for slow swarms or magnets that take a while to find peers, or lower it to give up on dead torrents sooner. | number | `15` |
|
||||
| `STALE_TORRENT_HOURS` | Remove a torrent Shelfmark added when it is stalled or cannot fetch its metadata, has had no activity for this many hours, and no running download owns it. Its partial data goes with it. Only torrents Shelfmark itself added are ever considered, whatever category they are in; torrents still downloading, queued, or finished are never touched. 3 to 72 hours (a value outside that is held to it); 0 keeps them. Checked every 15 minutes, independent of the Hardcover sync; only qBittorrent is supported. | number | `6` |
|
||||
| `PROWLARR_USENET_CLIENT` | Choose which usenet client to use | string (choice) | _empty string_ |
|
||||
| `NZBGET_URL` | URL of your NZBGet instance | string | _none_ |
|
||||
| `NZBGET_USERNAME` | NZBGet control username | string | `nzbget` |
|
||||
@@ -2814,6 +2836,16 @@ How long a torrent can go without any progress before Shelfmark cancels it. Appl
|
||||
- **Default:** `15`
|
||||
- **Constraints:** min: 5, max: 60
|
||||
|
||||
#### `STALE_TORRENT_HOURS`
|
||||
|
||||
**Remove dead torrents after (hours)**
|
||||
|
||||
Remove a torrent Shelfmark added when it is stalled or cannot fetch its metadata, has had no activity for this many hours, and no running download owns it. Its partial data goes with it. Only torrents Shelfmark itself added are ever considered, whatever category they are in; torrents still downloading, queued, or finished are never touched. 3 to 72 hours (a value outside that is held to it); 0 keeps them. Checked every 15 minutes, independent of the Hardcover sync; only qBittorrent is supported.
|
||||
|
||||
- **Type:** number
|
||||
- **Default:** `6`
|
||||
- **Constraints:** min: 0, max: 72
|
||||
|
||||
#### `PROWLARR_USENET_CLIENT`
|
||||
|
||||
**Usenet Client**
|
||||
|
||||
Reference in new issue
Block a user