Commit Graph
5 Commits
Author SHA1 Message Date
Rupraj Datta ed0777a2e0 Spool large debrid downloads to disk (#1632)
Fixes #1630.

Debrid downloads currently buffer entire files in RAM. The four clients
now opt into a spooled temporary file: downloads up to 8 MiB stay in
memory, and larger downloads roll to disk. The clients copy from the
file in chunks and close it after use. Other download_url callers retain
the existing BytesIO behavior.

Temporary files are closed on unsuccessful attempts and cancellation;
successful resumed downloads remain open for their caller. Added offline
coverage for disk rollover, timeout cleanup, and resumed content.
Updated the Debrid-Link test doubles for the new opt-in argument.

Validation: 113 tests passed across test_http_download_url,
test_debridlink_client, and test_torbox_client; Ruff check and format
check passed for changed files; git diff --check passed. No live debrid
services were exercised. AI assistance was used in implementing this
change.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Improvements**
* Large downloads are handled more efficiently, reducing memory use
while preserving download and resume behavior.
* **Bug Fixes**
* Temporary download data is cleaned up when a download is cancelled or
fails, helping prevent unnecessary resource usage.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-01 22:54:46 -05:00
splitsec2 2668705bfa perf: reuse connections and cache Anna's Archive download counts (#1546)
Two small changes that stop Shelfmark from opening a new HTTPS
connection for requests that go to the same host over and over.

Anna's Archive download counts. Every search result gets its count from
`/dyn/md5/inline_info/<md5>`, and each of those was a bare
`requests.get`, so a search with 50 results opened 50 TCP and TLS
connections to the mirror. `search_books` also runs once per title
variant, and the variants of one book mostly find the same files, so the
same md5 was looked up again for every variant that found it, and again
when the release list was reopened. The lookups now share one
`requests.Session`, and a successful count is kept for five minutes in a
bounded `CacheService` (the same class the metadata cache uses, as its
own instance, so it does not depend on the metadata cache toggle). A
lookup that fails or comes back without a count is not cached, so it is
retried on the next search just like today. The URL, headers and 5
second timeout are unchanged.

Debrid clients. Real-Debrid, AllDebrid, Debrid-Link and TorBox also used
the bare `requests` helpers. The download handler polls `get_status`
every two seconds for each active job, so every poll was a fresh TLS
handshake with the debrid API. Each of those modules now keeps one
`requests.Session` and sends its API calls through it, the way the
qBittorrent, Deluge, Prowlarr and Newznab clients already do. Headers,
timeouts, `verify` and error handling are still passed per call exactly
as before. The existing tests that patched
`<module>.requests.get/post/request` now patch
`<module>._session.<verb>` instead; nothing else in them changed.

How I checked it: each change has a test that fails on main and passes
here. They count the requests that reach `requests.Session.request` and
the sessions built while doing it, with no network. On main, three polls
of each debrid client build three sessions, and two overlapping batches
of AA results look `bbb` up twice. There are also tests for the count
cache expiring (with a faked clock) and for a failed lookup not being
cached. Full suite: 4432 passed, 48 skipped. ruff and basedpyright are
clean on the changed files.

I also looked at caching `/api/releases` on the server and left it out.
The release modal already caches each book's results per source and
content type in the browser for five minutes (longer for IRC and slskd
through `cache_ttl_seconds`), and that cache survives closing and
reopening the modal, so reopening it does not run a new search today. A
server cache would only help after a page reload, and it would quietly
undo the modal's "apply filters" and manual search refetches, which drop
the browser cache on purpose, unless the frontend also learned to send a
bypass. That did not seem worth it for this PR.
2026-10-22 20:24:40 -04:00
splitsec2 be2653bdb7 fix(debridlink): reuse the file list from the status check (#1431)
Kept small on purpose so it's easy to review.

When a torrent is ready, `get_status` has already fetched the full
record with `/seedbox/list?ids=` and `_handle_torrent_info` has checked
that every file is at 100%. The retrieval thread was then started
without that list, so `_process_and_download` called `_fetch_file_list`,
which asks the same endpoint again. That second call runs inside the
thread's generic `except`, without the flood and transient error
handling the status path has, so a rate limit or a hiccup there sets the
phase to error for a torrent that finished fine.

This passes the files already in hand into the thread.
`_process_and_download` already took an optional `files` argument. The
new test starts the thread from a ready status and checks it uses the
given list and never calls `_fetch_file_list`.

Separately, `download_url` returns a `BytesIO`, so a whole file sits in
memory. AllDebrid, TorBox and Real-Debrid do the same, so I left that
for a streaming variant of the shared helper instead of fixing it for
one client here.

Ruff and basedpyright are clean and `tests/prowlarr` passes.

The one red check is the existing clock-dependent test
`test_a_queued_torrent_asks_for_a_grace_once`, which fails on any runner
that booted recently. It is fixed in #1426 and is not caused by this
change.
2026-10-20 00:59:00 -04:00
CaliBrainandClaude Opus 5.5 9321096032 fix(clients): correct Debrid-Link file listing and rate-limit handling (#1425)
Follow-up to #1380.

- Fetch the file list with ?ids=, the documented parameter. There is no
  ?id=, so a many-file torrent was not expanded and the call could fail
  with badArguments at the last step of every download.
- Back off status checks for 60s after floodDetected, doubling to a 240s
cap, instead of an hour. The orchestrator cancels a download after five
minutes without a change, so the hour-long pause cancelled every active
  download. Only a flood on the status check starts it, and a successful
  check resets it.
- Drop the client-side one-hour stall timer. The orchestrator stall
timer
  always fires first.
- Keep progress at 50% when file retrieval starts instead of dropping to
  0% until the first file arrives.
- Report queued, paused and verifying torrents as QUEUED, PAUSED and
  CHECKING, so a queued torrent gets the handler queue grace.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 08:08:32 -04:00
splitsec2 cad18b8019 feat(clients): add Debrid-Link as a torrent client (#1380)
calibrain said in #1315 that he'd review and merge a Debrid-Link client
if someone took a stab at it, so here it is. It's the fourth debrid
service alongside AllDebrid, Real-Debrid and TorBox and it uses the same
DownloadClient interface as the other three. No frontend changes, the
settings UI picks it up from the backend field definitions.

**I can't test this against the live API.** I don't use debrid and I
don't have a Debrid-Link key. Everything here is built from their v2 API
docs and cross-checked against two independent reference
implementations, so the request shapes should be right, but nobody has
run a real magnet through it yet.

@Tatsu941 you opened the issue and mentioned they give developers free
premium. Would you be able to try it? The things most worth confirming
are the field names on a seedbox torrent, that `premiumLeft` is the
right premium check, and that an API error comes back the way the client
expects. Happy to fix whatever turns up.

Two things about their API shaped how this is written.

Every v2 endpoint wraps its payload in a `{success, value}` envelope and
a failure can arrive with an HTTP 200, so `_request_value` decides the
outcome from the body instead of the status code.

A completed seedbox torrent already carries a direct `downloadUrl` on
every file. The other three clients each need a separate unrestrict or
link-request call per file and this one doesn't, which is why it came
out shorter than them despite doing the same job.

Endpoints used:

| Endpoint | Used for |
|---|---|
| `GET account/infos` | test the key, confirm premium time remains |
| `POST seedbox/add` | magnet as JSON, .torrent as multipart |
| `GET seedbox/list` | poll progress |
| `DELETE seedbox/{id}/remove` | clean up |

Ruff and basedpyright are clean and the 36 new tests pass. The two
failures in `tests/e2e/test_proxy_auth_middleware.py` are already on
main and fail the same way on a pristine checkout. They assume the first
proxy-authenticated user becomes an admin, which only holds while the
user table has no admin in it, so they depend on how many users earlier
tests created.

One thing for review: the download loop uses `download_url` like the
other three clients, which holds each file in memory. I kept the
existing pattern rather than do something different in a new-provider
PR, but say so if you'd rather it streamed here.

Closes #1315
2026-10-03 07:55:36 -04:00