Fixes#1685
`_fetch_download_count_inline` fetches `/dyn/md5/inline_info/<md5>` for
every search result (5 workers) through a module-level
`requests.Session`, with no `proxies=`. `get_proxies()` reads the app
config rather than the environment, so with the proxy set in Settings →
Network the search page and the bypasser used it while these requests
left from the host's own IP, about 50 per search. It now passes
`network.get_proxies(url)` on each request, as the Libgen fast path in
the same file already does. The session stays module-level for
keep-alive, and proxies are resolved per request, so a settings change
applies without a restart.
Verification (macOS, `uv run --frozen pytest`):
- New `tests/direct_download/test_download_count_proxy.py` passes. With
the source change reverted it fails (`proxies` is `None`).
- `tests/direct_download`: 135 passed, including
`test_download_count_reuse.py`.
- `ruff check` and `ruff format --check` on the changed files pass.
- Not re-run live with this branch built. The behaviour before the
change was observed on a running v1.6.2 container: 5 sockets from the
`ThreadPoolExecutor` threads straight to the AA mirror's IP on :443, and
1 to the proxy.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Anna’s Archive download counts are now retrieved using the configured
network proxies. This improves access to download-count information in
environments where requests need a proxy, while preserving the existing
response handling and timeout behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This adds two cleanups that nobody notices until they matter, and puts
them on their own timer. Both do nothing until their setting and client
are configured.
The dead-torrent sweep removes a torrent Shelfmark added once it has
been stalled, or stuck fetching metadata, for `STALE_TORRENT_HOURS` with
no running download owning it, and its partial data goes with it. It
only touches torrents this instance added, tracked in a ledger of
hashes, whatever category they are in, and only on qBittorrent. The
default is 6 hours (3 to 72, and 0 turns it off). I am not sure 6 is
right for everyone, so say if you would rather ship it off.
The Soulseek sweep covers what a restart leaves in slskd's downloads
folder. A finished folder nothing imported gets imported, a partial
folder is removed after `SLSKD_CLEAN_LEFTOVERS_DAYS`, and unfinished
transfers no running download owns are picked up again when their
request still waits for them.
Both run from a small background thread every 15 minutes, after a 30
second startup delay. I first ran them from the Hardcover scheduler, but
that only runs when Hardcover sync or auto-download is on, so anyone
using neither would never get the cleanup. A separate thread keeps them
independent, and the interval is fixed so there is no new setting.
The new tests mock the sweeps. I also started a scratch instance, and
the thread starts and the first cycle is a no-op when nothing is
configured. The timer has not run against a real qBittorrent or slskd,
though the two sweeps themselves have been in my fork's builds since
October. Full Python suite, ruff and basedpyright pass.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **New Features**
* Shelfmark periodically recovers eligible complete Soulseek downloads
and reattaches unfinished transfers. It can also clean up eligible old
folders while protecting active, wanted, and already-imported downloads.
* Shelfmark can remove tracked qBittorrent torrents that have stalled or
exceeded the configured inactivity limit, while preserving torrents
associated with active tasks. Set the limit to 0 to disable cleanup.
* **Bug Fixes**
* Isolated Soulseek downloads reject task IDs that could produce unsafe
or mismatched destination folders.
* **Documentation**
* Added configuration guidance for Soulseek recovery and cleanup, and
qBittorrent stale-torrent cleanup, including defaults and eligibility
rules.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
A batch of activity list changes from running it as an admin on my own
instance. They are separate commits that build on each other, so they
can be reviewed in order.
The cards read better. Every card gets a format pill (violet for audio,
orange for ebooks), the requester is named once, and finished cards say
how long a download took and how long ago it ended ("took 5m · 35m
ago"). A download that picked up work already under way shows a
Reattached pill.
Admins can tell what is automatic and what needs a person. A declined
request shows who declined it and when, anything with an `[auto]` note
reads "Declined automatically", and "Needs Review" becomes "No Match
Yet" with a line saying the search repeats on every sync.
There is a Rejected view next to History for admins, with Reopen on each
card and a way to hide the dead ones.
The part to read most carefully is the backend in `requests_service.py`
and `user_db.py`. An admin can now reopen a fulfilled request whose
download failed or was cancelled, and the reopen a failed download
triggers remembers which release failed, so the next search doesn't pick
it again.
I ran it on a scratch instance (when pulling them together to move from
my fork to upstream) as an admin with seeded requests and looked at the
pills, the durations, the wording and the Rejected view in a browser.
Reopening a fulfilled request I only exercised in the tests. The
frontend and Python suites, type check, ruff and basedpyright pass.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **New Features**
* Admins can review rejected requests in a dedicated view, hide or
restore cards, and reopen requests.
* Fulfilled requests with failed or cancelled deliveries can be
reopened; the failed release is skipped in the next automatic search.
* Activity cards now show format badges, download timing, and when a
download resumes existing work.
* **Improvements**
* Activity views clarify automatic retries and actions, and show
reviewer names for admins.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Documentation**
* Added a Sponsors section to the README recognizing CodeRabbit and
linking to its logo.
* **Chores**
* Removed the automated pull-request review workflows for repository and
fork pull requests. These workflows no longer trigger review runs for
the pull-request events they previously handled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
When a request's download fails in a way the queue can't resume, the
request is reopened and its Retry button opens the release list. The
release that just failed was drawn there as a red, disabled "Failed"
button for as long as the failed task stayed in the queue status, so you
couldn't pick it again even when the release was fine and the download
client was the problem. A bad qBittorrent password is the easy way to
hit it. Thanks @joshft91 for the report and the steps.
What changes:
- In the release list, a failed request download now shows Retry, the
same as a failure the queue can retry already does. A direct download
with nothing to retry from still shows Failed. The rule lives in one
small helper used by the release modal and both button-state paths in
`useDownloadTracking`.
- A client that refuses the login is no longer held against the release.
qbittorrent-api raises `LoginFailed` with no message for a wrong
password, and only "Forbidden" or "Unauthorized" for a bad session or
API key, so none of it matched the unreachable-client markers from #1655
and the release went into `failed_releases`. The qBittorrent client now
says "qBittorrent authentication failed (...)" for these.
- The request reopen and the auto-download retry used two copies of the
same marker list. They now share one in `status_messages.py`, which also
counts an explicit authentication failure as the client's fault. A bare
403 or 401 is not on it, since a dead .torrent link fails that way too
and that one is the release's. Genuine release failures (no seeders, a
rejected torrent, a dead .torrent link) are still remembered as #1655
intended.
Tested with a scratch instance against a mock Prowlarr and a fake
qBittorrent that rejects the login: before, the request's Retry opened
the list with the failed release as a red X; after, it shows Retry,
picking it fulfils the request again, and the request reopens without
the release in `failed_releases`. New tests cover the qBittorrent
messages, both reopen paths and the button state.
Fixes#1674
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Failed releases are no longer skipped after download-client connection
or login failures; download failures and stall cancellations remain
eligible to be skipped on the next search.
* Failed requests with an associated request can now show a retry
option, while direct downloads without retries remain marked as failed.
* qBittorrent login and authorization errors are reported more clearly.
* **Documentation**
* Clarified which failures cause a release to be skipped on a later
search.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Bumps [publicsuffixlist](https://github.com/ko-zu/psl) from
1.1.2.20261101 to 1.1.2.20261105.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/ko-zu/psl/commits">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Chores**
* Updated the minimum required version of a dependency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Some Soulseek uploaders run a bot that blocks your downloads until you
type a word to them in a private message ("please type 'download' in
this chat"). The files sit queued until someone answers, and nobody
looking at Shelfmark would know why.
While a slskd download is polling, this checks slskd's unacknowledged
private messages about once a minute, replies with the word the message
asks for and marks the conversation read. It only matches three known
phrasings, the word has to fit `[A-Za-z0-9_-]{1,24}`, and each user gets
at most one reply per 12 hours (recorded after a successful send). Every
reply is logged at INFO.
The challenge patterns and the one-reply-per-12-hours idea come from
@Nezreka's [SoulSync](https://github.com/Nezreka/SoulSync)
(`core/chat_autoprove.py`, commit 445473b, MIT). I reworked it for
slskd's client and this download handler. The credit is in the module
docstring and the commit message too.
This sends messages to other people on your behalf without asking, so
the setting `SLSKD_CHAT_AUTOPROVE` is off by default here. Say if you'd
rather it not exist at all, or want it somewhere other than the slskd
settings.
It has run on my instance since October 22 without seeing a real
challenge, so the reply path is covered by tests but not by a live one.
The same code works in nezreka/soulsync with the same slskd client so I
am confident in it. Ruff, basedpyright and the slskd tests pass, and
`docs/environment-variables.md` and the readme line for slskd are
updated.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **New Features**
* Added an optional setting to automatically answer supported Soulseek
private-message challenges while a download is running. When enabled,
Shelfmark checks unread messages, replies to each person at most once
every 12 hours, logs replies, and marks answered conversations as read.
The setting is off by default.
* **Documentation**
* Updated the environment-variable and Soulseek setup documentation with
details about the setting and its behavior, including a reminder to
leave it off if you read slskd messages yourself.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Automatic downloads fulfil pending Hardcover requests as an admin, and
they picked up every pending request with provider `hardcover`. A
request a user makes from the search UI carries that provider too, so
with automatic downloads on, a request from a user whose policy needs
approval was approved by the pass instead of waiting for an admin. I
added that behaviour in #1516 and #1535 and missed this case.
The pass now runs the same check the download route makes. A request is
searched only on the release sources where its owner's resolved policy
(global settings plus their own overrides) is Download. Requests owned
by an admin, installs without login, and users with requests turned off
are not limited, since all of them can already download directly. A
request left with no source stays pending and untouched: no search, no
backoff state, no note. The 7-day and quick retries only reopen a failed
download on the same terms, so a request the pass can't fulfil isn't
moved back to pending for nothing.
The pass summary gains an `awaiting_approval` count. The
`AUTO_DOWNLOAD_ENABLED` description, the env var reference and the
Hardcover sync docs say which requests it approves.
Tests cover approval needed, direct download allowed, admin owned,
blocked, a per-source rule, a per-user override, requests turned off, no
login, and the retry reopen.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Automatic downloads now respect each request owner’s policy and
permitted sources. Requests that require approval or are blocked by
policy remain pending rather than being searched.
* Failed downloads are reopened for retry only when the owner’s policy
allows direct fulfillment.
* Admin-owned requests and requests in unauthenticated mode retain their
existing automatic fulfillment behavior.
* **Documentation**
* Clarified how request ownership, download policies, approval
requirements, and retries affect automatic downloads.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
In Download mode there is no way to ask for a book that has nothing to
download yet. The release modal has an "Add to requests" chip for
admins, which sends a book-level request, and when the default mode is
Download the server answers it with "Download policy requires a concrete
release". That is #995, where @expostarv2 used to bookmark unreleased
books this way. #1019 is the other side of it: @punkincognito wants a
less technical user to be able to download from one source and, when
that fails, ask for the book without choosing a release.
A book-level request in Download mode now becomes a normal pending book
request, stored with the mode `request_book`, when the actor is an admin
or when a new setting allows it: **Allow book requests alongside
downloads** (`REQUESTS_ALLOW_BOOK_REQUEST_WITH_DOWNLOAD`, off by
default, per-user overridable). A request that names a release still
goes straight to the download queue as before, and with the setting off
a non-admin gets the same error as today, so nothing changes for an
existing install unless it turns the setting on.
The policy endpoint reports the setting, and the release modal shows
"Add to requests" to a non-admin when requests are on and the setting
allows it (admins keep it as before, and it stays off while fulfilling a
request). The small `canRequestBookFromReleaseModal` helper holds that
rule so it can be tested.
I chose a setting over making this the default for everyone because it
changes what Download mode means, which is yours to decide. The other
way to read #1019 is a per-source rule, "Request Book" for Prowlarr and
Download for the direct sources. That is not here: the rules matrix
excludes Request Book today, the release modal shows such a source as
unavailable, and it needs more UI work than I can check without trying
it in a browser. This PR covers the half that works with the existing
flow, and the rule version can follow if you want it.
Tests: four route tests (a user with the setting on, an admin with it
off, a release request that must still queue a download, and the policy
endpoint reporting the setting), and a frontend test for the helper. The
route tests that depend on the change fail on current main. Full backend
and frontend suites pass, with ruff, oxlint and tsc clean. I have not
tried it in a browser, so this is a draft until someone has looked at
the modal.
Closes#995. #1019 stays open for the per-source rule.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **New Features**
* Added an off-by-default, user-overridable setting that lets users with
download access request a book without selecting a release. These
requests remain pending for admin action, and admins can always make
book-level requests.
* Eligible users can make book-level requests under the download policy
when the setting is enabled. This option is unavailable in browse-fulfil
mode.
* Requests that include release details continue to queue downloads
rather than create book-level requests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
A request that auto-download closes because the book is already in the
library never gets a download, so Clear Completed skipped it and those
requests piled up under Complete. The button also disappeared once the
last finished download was cleared, leaving no way to clear them.
ActivitySidebar.tsx: hasTerminalDownloadItems is dropped; the button
shows whenever getClearCompletedTargets returns anything, and on the
Requests tab as well as Downloads and All.
clearCompleted.ts: a request card with no download of its own is a
target when its request is fulfilled with no active delivery_state
(queued, resolving, locating, downloading), or cancelled. Pending and
rejected requests are left alone. The server already treats any
fulfilled request as terminal in _request_terminal_status, so the client
rule is the stricter of the two.
docs/users-and-requests.md: a line under Clearing the activity list
saying what the button clears.
Tests: two new cases in clearCompleted.test.ts, one for a closed request
and a cancelled one with no download, one that pending, rejected and
still-downloading requests are not touched.
Builds on #1438.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **New Features**
* **Clear Completed** can appear on the Requests tab, as well as
Downloads and All, when there are items to clear.
* Clearing completed items removes finished downloads and their
requests, fulfilled requests that are no longer in progress, and
cancelled requests without downloads. Pending and rejected requests
remain.
* **Documentation**
* Updated the help documentation to explain what **Clear Completed**
removes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
A failed or cancelled card showed the log line, things like `Peer
transfer failed: Errored` or a 500 with a URL in it, so a red card meant
going to the logs to find out whether anything was coming next. They now
say what happened and what happens next.
For a download that belongs to a request, the card says another copy
will be tried in about N minutes when the quick retry from #1535 is on,
and points at Retry or reopening the request when it isn't. A manual
download is never promised an automatic retry. The Soulseek cases each
get their own sentence: the peer dropped the transfer, the peer could
not be reached, slskd refused to queue it, and the peer never answered.
The "tried twice" wording leans on the retry from #1590.
The saved message is left alone because auto-download reads it to tell a
failed release from an interrupted one. Only what the activity list
shows is reworded, on a copy, and anything the new code doesn't
recognise comes back as it was.
It matches on how the message starts, so it is only as current as the
wording the rest of the code uses. A message that gets reworded
elsewhere falls back to the original line instead of breaking, which
seemed fine for something that only changes the display, but it is the
part I would look at first.
It has been running on my instance since mid-October. Ten new tests
cover each message and the next step, and the activity route tests still
pass. Ruff and basedpyright are clean on the changed files.
Bumps [publicsuffixlist](https://github.com/ko-zu/psl) from
1.1.2.20261029 to 1.1.2.20261101.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/ko-zu/psl/commits">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Chores**
* Updated the minimum required version of a dependency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the python-deps group with 1 update:
[flask-socketio](https://code.miguelgrinberg.com/miguelgrinberg/flask-socketio).
Updates `flask-socketio` from 5.7.1 to 5.7.2
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Why
Saving a Transmission address as HTTPS and leaving the port blank makes
the connection test fail. The client opens port 9091 and reports an SSL
wrong-version error, so a daemon published through a reverse proxy on
the usual HTTPS port never answers.
parse_transmission_url filled in 9091 whenever the URL omitted a port,
and that default ignored the scheme. An https address therefore started
TLS against 9091, where the peer is not speaking TLS.
## Summary
An omitted port now follows the scheme: 443 for https, and 9091 for
http, which still covers the empty-URL default. A port written in the
URL is left as given. The parser tests cover
https://transmission.myserver.org and expect https, that host, port 443,
and the path /transmission/rpc.
## Testing
The targeted tests for the shelfmark parse_transmission_url change pass
locally. The tests passed with the shelfmark parse_transmission_url
change applied; a run with it reverted gave no clear result, so no test
is shown to catch this bug.
Closes#1667
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* HTTPS Transmission URLs without an explicitly specified port now use
port 443. HTTP URLs continue to default to port 9091, and the default
RPC path is unchanged.
* **Documentation**
* Clarified the default ports for each URL scheme and noted that HTTPS
instances on port 9091 require `:9091` in the URL.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Bumps [oauthlib](https://github.com/oauthlib/oauthlib) from 4.0.0 to
4.0.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/oauthlib/oauthlib/releases">oauthlib's
releases</a>.</em></p>
<blockquote>
<h2>4.0.1</h2>
<h2>Security</h2>
<ul>
<li><strong>CVE-2026-49272</strong> / GHSA-3jp2-3mrg-2vxc: The device
code flow token endpoint did not validate the <code>device_code</code>
parameter. A request with a known <code>client_id</code> and a bogus
<code>device_code</code> could obtain a bearer token.
<code>DeviceCodeGrant.validate_token_request</code> now calls
<code>DeviceCodeGrant.validate_device_code()</code>, which rejects
missing or unknown device codes (<code>invalid_request</code> /
<code>invalid_grant</code>) and delegates the code's authorization
status to the new <code>RequestValidator.validate_device_code()</code>
hook, per RFC 8628 Section 3.5. (cherry-picked from <a
href="https://redirect.github.com/oauthlib/oauthlib/issues/954">#954</a>)</li>
</ul>
<h2>Upgrade note for device flow integrators</h2>
<p>Custom <code>RequestValidator</code> implementations using the device
flow must now implement <code>validate_device_code(client_id, code,
request)</code>. Return one of the
<code>DeviceCodeGrant.DEVICE_CODE_*</code> statuses for known codes, or
<code>None</code> for an unknown code. When returning
<code>AUTHORIZED</code>, the validator must also populate
<code>request.user</code> and <code>request.scopes</code> from the
stored authorization.</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/oauthlib/oauthlib/blob/v4.0.1/CHANGELOG.rst">oauthlib's
changelog</a>.</em></p>
<blockquote>
<h2>4.0.1 (2026-11-04):</h2>
<p>OAuth2.0 Provider (Security):</p>
<ul>
<li>
<p><strong>Security</strong>: CVE-2026-49272 / GHSA-3jp2-3mrg-2vxc: The
device code flow
token endpoint did not validate the <code>device_code</code> parameter.
A request
with a known <code>client_id</code> and a bogus <code>device_code</code>
could obtain a
bearer token. <code>DeviceCodeGrant.validate_token_request</code> now
calls a new
<code>DeviceCodeGrant.validate_device_code()</code> which rejects
missing or
unknown device codes (<code>400 invalid_request</code> / <code>400
invalid_grant</code>)
and delegates the code's authorization status to a new required
<code>RequestValidator.validate_device_code()</code> hook, per RFC 8628
Section 3.5. (cherry-picked from <a
href="https://redirect.github.com/oauthlib/oauthlib/issues/954">#954</a>,
merged as 1c91d64)</p>
<p><strong>Upgrade note for device flow integrators</strong>: custom
<code>RequestValidator</code> implementations using the device flow must
now
implement <code>validate_device_code(client_id, code, request)</code>.
Return one
of the <code>DeviceCodeGrant.DEVICE_CODE_*</code> statuses
(<code>AUTHORIZED</code>,
<code>PENDING</code>, <code>SLOW_DOWN</code> or <code>EXPIRED</code>)
for known codes, or <code>None</code>
for an unknown code. When returning <code>AUTHORIZED</code>, the
validator must
also populate <code>request.user</code> and <code>request.scopes</code>
from the stored
authorization.</p>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/oauthlib/oauthlib/commit/6e3aa05af92978755939a18cb4dd655ced156c1e"><code>6e3aa05</code></a>
Release 4.0.1: fix CVE-2026-49272 (device_code validation in device
flow)</li>
<li><a
href="https://github.com/oauthlib/oauthlib/commit/77a5f1c8ce00da2b0b8c5f6e9cc129920085f114"><code>77a5f1c</code></a>
Merge pull request <a
href="https://redirect.github.com/oauthlib/oauthlib/issues/954">#954</a>
from hekhuisk/improve-device-workflow</li>
<li>See full diff in <a
href="https://github.com/oauthlib/oauthlib/compare/v4.0.0...v4.0.1">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/calibrain/shelfmark/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the python-deps group with 3 updates:
[flask-socketio](https://code.miguelgrinberg.com/miguelgrinberg/flask-socketio),
[python-socketio](https://github.com/sponsors/miguelgrinberg) and
[authlib](https://github.com/authlib/authlib).
Updates `flask-socketio` from 5.7.0 to 5.7.1
Updates `python-socketio` from 5.17.1 to 5.18.1
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/sponsors/miguelgrinberg/commits">compare
view</a></li>
</ul>
</details>
<br />
Updates `authlib` from 1.9.1 to 1.9.2
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/authlib/authlib/releases">authlib's
releases</a>.</em></p>
<blockquote>
<h2>v1.9.2</h2>
<p>Changelog: <a
href="https://docs.authlib.org/en/stable/upgrades/changelog.html#changelog-1-9-2">https://docs.authlib.org/en/stable/upgrades/changelog.html#changelog-1-9-2</a></p>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/authlib/authlib/compare/v1.9.1...v1.9.2">https://github.com/authlib/authlib/compare/v1.9.1...v1.9.2</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/authlib/authlib/commit/d74dc90ea0de493858f987ddba47e1fccfb0c2f3"><code>d74dc90</code></a>
chore: release 1.9.2</li>
<li><a
href="https://github.com/authlib/authlib/commit/44c96176dc903d21efea5fcafe9dc18cb3eb0517"><code>44c9617</code></a>
fix(oauth): limit ID-JAG access tokens to the scope claim</li>
<li>See full diff in <a
href="https://github.com/authlib/authlib/compare/v1.9.1...v1.9.2">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
</details>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Release Notes**
* No end-user-facing changes are included in this update.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Fixes#1662, thanks @mietu111 for the report and the screenshot.
The source tabs share a row with up to five icons, which take 240px on a
phone. On a 393px screen that leaves 113px for the tabs, so Prowlarr
fits and Direct Download is down to a sliver. On a 360px screen not even
Prowlarr fits. The scrollbar is hidden, so there is also nothing to tell
you there is more to scroll to.
With more than one source, the tabs now get their own row below the sm
breakpoint and the icons sit on a row beneath them. Picking a tab also
scrolls it into view, for when there are too many sources to fit. This
does make the sticky header 36px taller on phones. One source and
anything sm or wider stay on a single row, with the same tab positions
and row height as before.
I checked this with screenshots of the dialog using mocked search
results, at 360 and 393px with two sources, 360px with five, 393px with
one, and at 640 and 1280px with two. The filter menu and manual query
panel still open correctly from the new row at 393px. There are no
layout tests for this dialog, so I did not add one.
Before, with Direct Download selected. The tab has 113px of space and
its label is cut off.
<img width="786" height="1704" alt="download"
src="https://github.com/user-attachments/assets/7fc46034-4faf-438b-aed3-7d9adbd41ac9"
/>
After, with Direct Download selected. Both tabs fit and the icons sit on
the row beneath.
<img width="786" height="1704" alt="download"
src="https://github.com/user-attachments/assets/e9e961e8-5f04-4b24-9483-89482891252a"
/>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Improvements**
* The active source tab is centered in the tab strip when selected.
* On narrow screens, multiple source tabs wrap onto their own row, and
the result count is hidden.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Follow-up to the route moves out of main.py. CodeRabbit's docstring
coverage check flagged #1637 to #1639 at 63%, so I went back over the
modules the moves created and filled in the 19 functions and nested
helpers that had none, in queue_routes, release_routes,
download_recording, system_routes, metadata_routes and auth_middleware.
Comments only, nothing else changes.
The route modules that were split out earlier (request_routes,
activity_routes and a few others) have more without docstrings, but I
left those alone since that code was not part of the moves.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Documentation**
* Expanded and clarified descriptions of error responses, download and
queue statuses, notification handling, queue ownership and visibility,
release-policy checks, and administrative access.
* No user-facing behavior changes are included in this update.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
When a download fails, the terminal hook reopens its request through
reopen_failed_request straight away, with no admin involved. That path
cleared the release without noting that it had failed, so the next
search could pick the same release again. Only the admin reopen and the
status sync remembered it.
Both paths now add the release to failed_releases before the reopen
clears it, the same list the unattended retry already skips, and a
release that is already listed is not added twice. A download that
failed because its client could not be reached (the same four markers
auto_download uses) is not held against the release, since the same one
is fine once the client is back.
This is the upstream shape of something I have been running on my own
instance since mid October. ruff, basedpyright and the requests_service
tests (87) pass, with five new tests for the two paths, the dedupe and
the unreachable client.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Failed requests are reopened after non-retryable delivery failures,
with the failed release recorded when appropriate.
* Connection failures caused by an unreachable client no longer mark the
release as failed.
* Reopening a request does not record the same release more than once or
alter book details when no new release can be recorded.
* Queue status updates apply the same release-tracking behavior, while
requests that are not failed remain unchanged.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
cleanup_old_lockouts collects the expired usernames, logs each one and
then deletes it. Under gevent the log write can yield, and a successful
login for the same user in that gap clears the entry first, so the del
raises KeyError in whatever request ran the cleanup. Both removals now
use pop(username, None), and the new test reproduces the KeyError on
main.
While in there I dropped the if db_user check in the builtin login path.
It is always true after the not db_user return just above it, and the
failed response after it could never be reached. The diff looks bigger
than it is because the block moves out one indent.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Login failures now follow a consistent response path when an account
is unavailable or its password data is invalid.
* Expired login lockouts can be cleaned up safely even if they are
cleared at the same time.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
load_or_create_secret_key writes the Flask secret with write_bytes and
then chmods it to 0600. Until the chmod the key sits in a file with
whatever the umask allows, and if the chmod fails it stays that way and
only a warning is logged. The file is now opened with mode 0600, an
existing one is fchmod'ed before anything is written, and then the key
goes in. The new test makes chmod fail and checks the file still ends up
private, which fails on main.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Secret-key files now receive restrictive permissions when created or
replaced, helping prevent unauthorized access to the stored key.
* Key creation maintains these permissions even when a later
permission-setting operation fails.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The Debug and Restart items in the menu show for every signed-in user
whenever Debug Mode is on, but `/api/debug` and `/api/restart` both need
an admin, so a regular user who clicks one gets a 403 page. The menu now
hides them unless the user is an admin.
It uses the auth check's `is_admin`, which is true when auth is off, so
a no-auth install keeps both items. The Debug chip in the footer is a
label and stays.
Frontend typecheck passes. I also ran a scratch instance with Debug Mode
on: the admin account sees Debug and Restart, a regular user sees only
Report a Bug and Settings, and with auth off both items are still there.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Debug information in the header is now shown only when debugging is
configured and the signed-in user is an administrator.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Closes#1646. In Firefox, hovering the Stop download button on an
in-progress download blanked the page with React error #185.
The Tooltip's layout effect moves a tooltip back inside the viewport and
then runs again on the new position, expecting to measure no overflow.
The Stop download tooltip runs about 10px past the right edge of the
unpinned Activity sidebar. After the move, Firefox measures its right
edge 0.0000153px past the limit (1594.0000153 against 1594), because it
works out a transformed box in single precision. The effect moves it by
that much, which is below what layout can represent, so the box stays
put
and the next pass measures the same overflow. After 50 passes React
throws #185 and unmounts the app. Chromium lands at 1017.996 against
1018 and stops after one pass.
The clamp now lives in getTooltipViewportNudge and treats overflow under
half a pixel as none. Tooltip calls it with the same inputs as before.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Tooltips are repositioned to stay within the viewport, including when
they are wider or taller than the available space.
* Minor overflow of less than half a pixel no longer triggers
unnecessary repositioning.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Bumps [publicsuffixlist](https://github.com/ko-zu/psl) from
1.1.2.20261023 to 1.1.2.20261029.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/ko-zu/psl/commits">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Chores**
* Updated project configuration. This maintenance update does not add or
remove visible features, controls, or workflows. Existing behavior
remains unchanged, and no new user actions are required. No other
user-facing changes are included in this release. These notes cover the
changes with a direct impact on the experience; no additional visible
updates are reported.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: splitsec2 <35583321+splitsec2@users.noreply.github.com>
Following the discussion on #1635, this drops the Chromium snapshot pin
and the apt preference, and installs whatever trixie-security currently
ships. Thanks @calibrain for suggesting we try it.
With no pin there is nothing left to go stale, so the scheduled check
that issue asked for isn't needed. Closes#1635.
I built the image from this branch and ran it as a scratch container. It
has Chromium 156.0.8078.79, and the bypass cleared DDoS-Guard on Anna's
Archive and returned search results. That challenge cleared on the
auto-wait step, so the click path wasn't exercised, same as the open
question in #1635.
One thing to know: the build cache is keyed on the Dockerfile text, so a
newer Debian Chromium only lands when that layer rebuilds, for example
on a base image bump.
The Python Typechecks job fails on main since #1632, which is why #1644
shows red too. `download_url` returns a `BytesIO`, or a
`SpooledTemporaryFile` when `spool_to_disk` is set, and both were typed
as `BinaryIO`. That has no `getbuffer()`, which four direct-download
callers use.
Two overloads on `spool_to_disk` give each caller the type it actually
gets, so nothing changes at runtime. Thanks @29rdx for the spooling
work, this only tightens the types around it.
basedpyright (source and tests), ruff and vulture are clean, and the
tests for the download, direct download and libgen code pass.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Developer Experience**
* Updated download API type information to distinguish between in-memory
buffers and disk-spooled buffers. The return type now reflects the
selected spooling option, making the API’s possible results clearer to
developers and improving type checking when working with downloaded
content.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Now that the route moves have landed, this takes care of the code
scanning alerts they brought along. Sixteen places in `queue_routes.py`,
`release_routes.py`, `metadata_routes.py` and `system_routes.py` handed
`str(e)` of an unexpected OSError, RuntimeError, TypeError or sqlite3
error back as the 500 body. It can carry file paths, URLs with tokens or
driver text. Same idea as #1558 and #1562.
Each now returns one fixed sentence ("Could not cancel the download",
"Could not read the queue order", "Metadata search failed" and so on)
and the full error and trace stay in the log. The routes covered are the
cover proxy, cancel, retry, priority, reorder, queue order and active
downloads; release download, search, source list and source record; the
metadata providers, config, search and book routes; and the debug
endpoint.
I left alone the messages that are written for the person using the app:
what a metadata provider raises as a ValueError or RuntimeError
("Hardcover API key is invalid"), `SourceUnavailableError`, and the
reason `queue_release` gives. CodeQL still flags those, as I read it
those are the ones that should be dismissed as intended, but that is
your call. The same pattern is in the settings and onboarding routes,
which aren't flagged, so I didn't touch them.
There are 15 new tests, one per fixed route except the debug endpoint,
which only exists in debug mode. All 15 fail without the change. Two
queue tests that asserted the old text now assert the fixed message. The
full suite, ruff and basedpyright are clean.
@calibrain I pushed through the refactors keeping them the same
functionality, this addresses "issues" flagged in them, I think some of
these were dismissed before, but I gathered them all up here in a PR
that you should have a say on as it is changing long standing code.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Error responses now show generic messages instead of operational
exception details across metadata, queue, release, and debug endpoints.
HTTP status codes and error logging remain unchanged.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#1630.
Debrid downloads currently buffer entire files in RAM. The four clients
now opt into a spooled temporary file: downloads up to 8 MiB stay in
memory, and larger downloads roll to disk. The clients copy from the
file in chunks and close it after use. Other download_url callers retain
the existing BytesIO behavior.
Temporary files are closed on unsuccessful attempts and cancellation;
successful resumed downloads remain open for their caller. Added offline
coverage for disk rollover, timeout cleanup, and resumed content.
Updated the Debrid-Link test doubles for the new opt-in argument.
Validation: 113 tests passed across test_http_download_url,
test_debridlink_client, and test_torbox_client; Ruff check and format
check passed for changed files; git diff --check passed. No live debrid
services were exercised. AI assistance was used in implementing this
change.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Improvements**
* Large downloads are handled more efficiently, reducing memory use
while preserving download and resume behavior.
* **Bug Fixes**
* Temporary download data is cleaned up when a download is cancelled or
fails, helping prevent unnecessary resource usage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Last of the planned moves is the download recording (row 9 in Discussion
#1536). The queue hooks that write a download to the history when it is
queued, finalize it on a terminal status, notify, push the activity
update and reopen a failed request move from `main.py` to
`core/download_recording.py`, registered with
`register_download_recording(...)`.
None of it was tested directly, so the first commit adds 38 tests: the
queued record for direct and request linked downloads, the terminal
snapshot for complete, error and cancelled, ignoring non terminal
statuses, who gets notified, reopening a failed request, and what
happens when a service is missing or a call fails. They pass on current
`main`. The second commit is the move.
`_resolve_status_scope` and the request update emitter stay in `main.py`
because other code uses them, so the emitter is passed in, and the user
database and history service are looked up per call as before. The hooks
are registered at the same spot. Some existing test lines changed with
the move and no assertion did: the `notify_admin`, `notify_user` and
`reopen_failed_request` patches point at the new module, the direct
calls to `_record_download_queued` go through the queue's hook, and two
tests import `QueueStatus` and `viewer_download_id` themselves now that
`main.py` no longer imports them.
The URL map is identical, all 77 rules, and the full suite, ruff and
basedpyright are clean.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Improvements**
* Download history and activity updates reflect queued downloads and
their completion statuses; activity updates are emitted after successful
finalization.
* Admins and eligible download owners receive notifications when
downloads reach a terminal status.
* Requests linked to downloads that fail and cannot be retried are
reopened with a failure reason.
* Activity updates use an opaque download identifier rather than
exposing the underlying task identifier.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The release routes are the largest piece left in `main.py`.
`/api/releases`, `/api/releases/download`, `/api/release-sources` and
the source record route move to `core/release_routes.py`, registered
with `register_release_routes(app, login_required, ...)`. They take the
content type inference, the request policy checks that gate listing and
download, and the release serializers with them. The inspect route is
registered from the same function since it shares the policy gate.
`main.py` is about 700 lines shorter.
The first commit adds 34 tests for what the policy, budget, direct
provider and title tests did not cover: content type inference on
download, the policy refusal on download, the error statuses of the
listing route, the source list and the record route. They pass on
current `main`. The second commit is the move.
The policy checks need the auth mode, the user database and the request
policy settings, so they come in as callables that `main.py` resolves
lazily, which keeps the existing tests that patch those on
`shelfmark.main` working. One test line changed, the patch of
`get_source_content_type_capabilities`, which now points at the new
module. Log lines from these routes now name
`shelfmark.core.release_routes`.
The URL map is identical, all 77 rules. The full suite, ruff and
basedpyright are clean, and I ran the app from the branch with the other
two route PRs merged in and hit the source list, the listing route's
validation, the record route and the download route, which answer as
before. The exception text a few of them return in a 500 is unchanged,
that is for the follow-up on exception text.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Release Discovery**
* Release searches can use metadata providers, manual searches, or
direct source browsing, with results and source details returned
together.
* Searches respect request policies and report unavailable sources or
partial-search errors.
* **Downloads**
* Download requests validate submitted details, infer a content type
when possible, and report whether the request was queued successfully.
* **Source Browsing**
* Available sources and individual source records can be retrieved,
subject to applicable access policies.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Row 8 of the plan in Discussion #1536 is the auth middleware. The API
key `before_request` hook, the proxy auth hook, the hook that strips the
session cookie from API key requests and `_proxy_default_is_admin` move
from `main.py` to `core/auth_middleware.py`, registered with
`register_auth_middleware(app, resolve_auth_mode=..., get_user_db=...)`.
This is the code that decides who a request is, so the first commit adds
41 tests before anything moves. They cover the hook order, the admin
group mapping (comma and pipe delimiters, a custom group header, the
WSGI fallback), proxy header handling in each auth mode, provisioning
once per session, the API key scope (exempt paths, a read only key only
on GET `/api/stats`, a wrong key) and the cookie stripping. They pass on
current `main`. The second commit is the move.
The call is made right after `register_security_headers`, so the
`before_request` and `after_request` order is the same as before. The
existing tests called `proxy_auth_middleware()` and
`api_key_auth_middleware()` on `shelfmark.main` directly, and those
names are gone, so 18 call lines in three test files now find the
registered hook by name through a small helper. No patch targets changed
since the auth mode and user database are still read from `main` at
request time.
The URL map and hook order are identical, and the full suite, ruff and
basedpyright are clean.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Authentication**
* API-key authentication supports admin and read-only access, with
read-only keys limited to statistics requests.
* Reverse-proxy authentication recognizes configured identity and group
information, and applies admin access according to the configured role
settings.
* Requests with invalid credentials, missing proxy identities, or
authentication errors receive appropriate error responses.
* API-key requests no longer retain session cookies.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Next from the route plan in Discussion #1536 are the queue routes:
status, cancel, retry, priority, reorder, queue order, active downloads,
the local file download and the cover proxy. They move from `main.py` to
`core/queue_routes.py`, registered with `register_queue_routes(app,
login_required, ...)`, and `main.py` is about 600 lines shorter.
The first commit adds 33 tests for what the guardrail tests did not
cover, mostly the local download in each shape, the admin paths, the
validation errors and the failure responses. They pass on current
`main`. The second commit is the move.
`resolve_status_scope` and the request update emitter stay in `main.py`
because the release routes and the request code still use them, so they
come in as parameters, with the user database and the history service as
callables evaluated per request. Two test lines changed: the patches of
`send_file` and of the delivery state sync now point at `queue_routes`,
since that is where they get looked up. Log lines from these routes now
name `shelfmark.core.queue_routes`.
The URL map is identical, all 77 rules. The full suite, ruff and
basedpyright are clean, and I ran the app from the branch and hit
status, queue order, active downloads, the local download and the cover
route, which answer as before. The exception text a few of them return
in a 500 is unchanged, that is for the follow-up on exception text.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Improvements**
* Queue status, downloads, and queue controls respect account ownership,
with administrative access retained.
* More specific errors clarify when queue items or files are missing or
unavailable.
* Queue actions support cancellation, retrying, prioritization, and
reordering.
* Cover requests can use cached images when caching is enabled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
`main.py` also builds the app itself, and this moves the parts of that
which need no route into `core/app_setup.py`: the persisted secret key
and the session cookie settings, the security headers, and the filter
that keeps status polling and the WebSocket upgrade noise out of the
log. It is the app setup item in the plan from Discussion #1536.
The first commit adds 14 tests for them, since none existed: the key
reused, created with mode 600, replaced when too short, and still
returned when the config directory cannot be written, the cookie
settings, the headers on a real response and that a header a route set
itself is kept, and each case the log filter drops or keeps. They pass
on current `main`. The second commit is the move.
The security headers hook is registered from the same spot in `main.py`
as before, so the `after_request` order is unchanged. The secret key
function takes the config directory as a parameter instead of reading
`CONFIG_DIR`, which is the one change in the move, and the key tests
call it with a temp directory.
The URL map and the hook order are identical, and the full suite, ruff
and basedpyright are clean. I also ran the app from the branch: the
secret file is created with mode 600 and the four security headers come
back on a normal response.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Sessions now remain consistent across app restarts, and session
cookies use more protective browser settings, including HTTP-only access
and a seven-day lifetime.
* Responses include baseline security headers while preserving any
headers already set by the app.
* Routine status-polling and connection-upgrade messages are filtered
from application logs, reducing unrelated log noise.
* If saving a session key fails, the app can still start with a
generated key.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This one is small. The three Socket.IO handlers, connect, disconnect and
request_status, move from `main.py` to `core/socket_handlers.py`,
registered with `register_socket_handlers(socketio, ...)`. It is the
Socket.IO item in the plan from Discussion #1536.
They had no tests, so the first commit adds 9 that go through
Flask-SocketIO's test client: what a client gets on connect with no
auth, as an admin, as a user and as a visitor with no session, a failure
reading the status, the manual status request, and the room bookkeeping
on disconnect. They pass on current `main`. The second commit is the
move.
`resolve_status_scope` stays in `main.py` since the HTTP routes use it
too, so it is passed in. One test line changed, the patch of
`redact_status`, which now points at the new module. Log lines from the
handlers now name `shelfmark.core.socket_handlers`.
The URL map and the registered socket events are identical before and
after, and the full suite, ruff and basedpyright are clean. I also
started the app from the branch and the Socket.IO handshake answers.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Status requests now return a consistent empty update when access is
denied and an error message when status retrieval fails.
* Status visibility follows the requester’s access level, with non-admin
results redacted.
* **Refactor**
* Connection, disconnection, and status-request handling now follows
consistent room and connection-tracking behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
ffmpeg is in the default image for one thing, the Debug mode screen
recording (`_start_ffmpeg_recording`). Nothing else calls it,
SeleniumBase included, and it brings 93 packages with it. On real builds
that's 406 Trivy findings (149 High) and 78 MB of the gzipped amd64
image. This takes it out of the image and installs it when Debug mode
needs it.
It came out of the base image discussion on #1616 and helps on any base,
though on Wolfi the install line would need `apk`.
## What changed
- **Dockerfile:** ffmpeg is gone from the full image's install.
- **Guard:** `_start_ffmpeg_recording` checks `shutil.which("ffmpeg")`
first. If it's missing it logs a warning and returns, and the bypass
carries on. Without this, Debug on with no ffmpeg fails every bypass
with `FileNotFoundError`. There's a new test for it.
- **Entrypoint:** when Debug is on, the internal bypasser is in use and
ffmpeg isn't already there, it installs ffmpeg in the background so the
app doesn't wait for it. The download and the install each run under a
120 s `timeout` with `DEBIAN_FRONTEND=noninteractive`. Downloading first
means a hung mirror can only be killed before dpkg starts, and the next
start runs `dpkg --configure -a` in case a restart cut dpkg off. It logs
when it starts, succeeds, times out or fails.
- **Docs:** a short paragraph in `docs/troubleshooting.md` covering the
install and the non-root limit.
## Debug has two sources
Debug can come from the `DEBUG` env var or from the Debug Mode setting
in the UI, which is saved to `/config/plugins/advanced.json`. The
existing block in `entrypoint.sh` only reads the env var, so an install
keyed on `$DEBUG` would miss anyone who turned it on in the UI.
The new check asks the app instead (`shelfmark.config.env.DEBUG`), which
reads both. The UI setting is `requires_restart`, so the restart it
already asks for is what brings it through the entrypoint.
## Non-root mode
A container started as `1000:1000` can't run `apt-get`, so the
entrypoint logs that it can't install ffmpeg, the guard warns on each
bypass, and there's no recording. Everything else works. The workaround
for a debugging session is to start as root with `PUID`/`PGID`.
## Startup cost
None that I could measure. With Debug on, `/api/health` answers in the
same time as main (3-4 s with Debug only in config, 9-10 s from env),
and ffmpeg was ready 6-8 s later on the runners.
The trade-off is that the first bypass after startup may be unrecorded
while the install runs. It logs the warning and passes. I saw both
outcomes on a bypass started right after health.
From reading the code: the install lives in the container, so recreating
the container installs it again. It also runs after `tor.sh` and
`wireguard.sh`, so apt goes through whichever egress is active.
## Testing
Real Docker builds on GitHub Actions on my fork, amd64 and arm64, with
the same result on both for every case below.
| Case | Result |
|---|---|
| Debug off | No install, `/api/health` in 3-4 s, same as main |
| Debug only in config | Health in 3-4 s, ffmpeg ready 6-8 s later, real
mp4 after a bypass |
| Debug from env | Health in 9-10 s (same as main), real mp4 after a
bypass |
| Started as root, app dropped to uid 1000 | Smoke and bypass pass |
| Started with `--user 1000`, Debug on | Logs both messages, bypass
passes, no mp4 |
| apt blocked (closed proxy) | Logs `Could not install ffmpeg`, bypass
passes, no mp4 |
| apt's proxy on a dropped address, so no TCP reply at all | Health in
9-10 s, a bypass during the install passes with the warning and no mp4.
apt gives up by itself in under a minute. Nothing left running, `dpkg
--audit` clean |
| DNS server on a dropped address | Health in 9-10 s, `Timed out
installing ffmpeg after 120s`. Nothing left running, `dpkg --audit`
clean |
| `--network none`, Debug on | No hang, health 200 from inside the
container |
| Container restarted while dpkg was running | Next start finishes the
install, `dpkg --audit` clean, real mp4 |
| Fresh named volume, Debug only in config | Recording made, uid 1000
can write |
| Existing uid-1000 volume, Debug only in config | Same, nothing
root-owned afterwards |
| `find /config /var/log/shelfmark -user root` | 0 files, same as main |
| `find /app -user root` | 245, same count as main |
| `tests/e2e/platform/run-e2e.sh` with `dns-blocked` and `baseline` |
Pass |
The bypass checks ran against example.com. A real Cloudflare challenge
was served by a demo page in the same runs, and no build solved it,
including main, so that part doesn't say anything about this change.
Locally: 5494 Python tests pass, and `ruff`, `basedpyright`, `vulture`
and `shellcheck` are clean.
The full results are in [the base image
report](https://github.com/splitsec2/shelfmark/blob/7afcecfb76727724a71561cbc078f091569b58dc/docs/research/2026-11-01-base-image-report.md).
The auth routes are the next group from the plan in Discussion #1536:
`/api/auth/login`, `/logout` and `/check`, plus the login lockout
helpers, move from `main.py` to `core/auth_routes.py`, registered with
`register_auth_routes(app, ...)`. `main.py` is about 370 lines shorter.
This is the login path, so I kept it strictly to a move.
Two commits. The first adds 59 tests that pin the current behaviour
before anything moves: login in each mode (builtin, none, CWA and the
OIDC password fallback), wrong passwords, the lockout (the warning when
five attempts are left, the 429 and its text on the tenth failure,
per-user isolation, expiry), the local-auth switches, logout in each
mode and `/check`. They pass on current `main`, and I ran the same file
against `main` and against this branch. The second commit is the move,
and no existing test changed.
The lockout state is one dict, now living in `auth_routes`, and
`main.py` imports the same object so its other users (the
password-attempt limiter for the self-user routes, and the tests) see
it. Everything else that comes from `main` (the user database, the auth
mode, the client IP, the CWA database path, the local-auth settings and
the password check) is passed in as a callable evaluated per request, so
patches on `main` still apply.
Two things to know. `main.py` re-exports the lockout names and keeps
three lines assigning the view functions (`api_login =
app.view_functions["api_login"]` and the other two), because the e2e
tests call `main_module.api_login()` directly and I did not change tests
in this PR. If you would rather drop both, I can edit those e2e tests
instead. The other is that the lockout dict is no longer reset by a
reload of `main`, which only matters in tests, and they clear it
themselves.
The URL map is identical, all 77 rules, and the full suite, ruff and
basedpyright are clean. I also ran the app with builtin auth on a
scratch instance and went through it over HTTP: a wrong password gets a
401, login works and `/check` shows the session, logout clears it, the
tenth failed attempt gets the 429, and the right password is refused
while the account is locked.
The system routes are the next group from the plan in Discussion #1536:
`/api/config`, `/api/health`, `/api/debug`, `/api/restart` and the
routes that serve the frontend (`/`, the SPA catch-all, `/assets/...`,
the logo, the favicons and `theme-init.js`) move from `main.py` to
`core/system_routes.py`, registered with `register_system_routes(app,
login_required, ...)`. `main.py` is about 250 lines shorter.
Same two-commit layout as before. The first commit adds 14 tests for the
frontend-serving routes and the `/api/config` error paths, which had
none, and they pass on current `main`. The second commit is the move,
and no existing test changed.
The helpers only these routes use moved with them (`admin_required`,
built on the `login_required` that is passed in, plus `_base_href` and
`_serve_index_html`). What `main.py` still owns, the base path, the
frontend folder, the auth mode and the session user id, is passed in as
callables evaluated per request, so the tests that patch them on `main`
still work. The debug routes are only registered when debug is on, as
before, with the stop-GUI and run-script hooks handed in.
The thing to look at is the SPA catch-all, which is now registered
earlier in `main.py` than it used to be (the old comment said it must be
last). Werkzeug picks the most specific rule whatever the registration
order, and I checked it: all 76 rule and method pairs in the URL map
still match their own endpoint, on `main` and on this branch, and an
unknown `/api/...` path still gets the JSON 404.
The Flask URL map is identical before and after, all 77 rules, and with
debug on (79). The full suite, ruff and basedpyright are clean. I also
ran the app from this branch: health, config, the logo, the favicon and
the theme script answer as before, a client-side path like
`/library/books` gets the index page, an unknown `/api/` path gets the
JSON 404, and the page loads in a browser. I did not exercise
`/api/restart` or a real debug script run, since nothing tests them.
The metadata routes are the next group from the plan in Discussion
#1536: the eight `/api/metadata/...` handlers (providers, config,
search, field-options, book and the three targets routes) move from
`main.py` to `core/metadata_routes.py`, registered with
`register_metadata_routes(app, ...)`. `main.py` is about 490 lines
shorter.
Same two-commit layout as the settings move (#1595). The first commit
adds 39 tests that pass on current `main` before anything moves. They
cover the 401 on all eight routes, the shape of the providers list and
the config, the search errors, the book lookup (200, 404, 400 and 503),
and how target errors map to 400, 502 and 500. The second commit is the
move, and no test changed in it.
Three helpers moved with the routes because nothing else uses them.
`_attach_library_ownership` stays in `main.py` since the releases routes
need it too, and is passed in along with the login decorator and the
auth mode, the way `activity_routes` takes them. The auth mode is a
callable evaluated per request, so the one test that patches it on
`main` still works.
The Flask URL map (every rule, its methods and its endpoint name) is
identical before and after, all 77 rules. The full suite, ruff and
basedpyright are clean. I also ran the app from this branch and hit the
routes over HTTP: providers and config answer 200, a book that doesn't
exist gets a 404, and search answers 503 with the "no metadata provider
configured" message on an instance with none enabled. I did not look at
it in a browser or run the e2e tests.
Log lines for these routes now come from
`shelfmark.core.metadata_routes`, and the error tuples are local copies
so the module doesn't import `main`. One thing to know when reading the
diff: the register call sits just after `_attach_library_ownership` and
not where the first route used to be, because that function has to be
defined before it can be passed in.
Nothing in the developer docs says where a new route goes, so the next
contributor adds it to `main.py` again. This writes down the convention
from Discussion #1536: a new route group gets its own
`register_<group>_routes` module under `shelfmark/core/`, and `main.py`
calls it once.
It is one paragraph on a new page, `docs/dev/routes.md`, with a line in
the developer index. It also says that a few older routes are still in
`main.py` and are being moved out one group at a time. The route moves
in that discussion will follow what this says, so this goes first.
Closes#1585. @joeRob2468 traced this one to the parser, and the answer
turned out to be sitting in the search response already.
A Hardcover search hit carries the book's featured series as
`featured_series`, with the same shape as the `featured_book_series` of
a full record: the position and a nested series with its id, name and
book count. `_parse_search_result` never read it, so a book requested
from a search card was stored with `series_name: null` and the
`{Series}` folder and the custom script payload came out empty. It now
reads it, through a small helper that `_parse_book` uses too, so there
is no extra Hardcover call per request (the issue's other option was
fetching the full record, which costs one against the rate limit).
I checked it against a live search for "Mistborn The Final Empire". The
top hit parses to series "The Mistborn Saga: The Original Trilogy",
position 1, id 10001, and its search title is still "Mistborn".
That last part is on purpose. I left `_compute_search_title` alone,
since passing it the series would change what gets searched for everyone
and the series name here doesn't equal the "Mistborn" prefix anyway.
Two things that behave differently. A search result now has
`series_name` set, so anything in the UI that shows it for a full record
(the series link, the bigger cover in the release modal) will show it on
search results too. I ran the branch and looked at the release modal for
the top Mistborn result: it shows "#1 of 3 in The Mistborn Saga: The
Original Trilogy" with the View series link. The modal fetches the full
record when it opens, so I can't say how much of that is new on this
path. Requests already stored keep the empty series they were saved
with.
Tests: four in a new file (a hit with a series, hits with none or an
incomplete one, a hit nested in a `document`, and the search title
staying the same), the first and third fail on current main. The
metadata and core suites, ruff and basedpyright are clean.
Came out of looking at what is in the image after @wrycu's #1606. The
selenium package ships prebuilt `selenium-manager` binaries for four
platforms (about 22MB uncompressed), and none of them are used here.
They only run when selenium has to find a webdriver, and the bypasser
drives the browser through SeleniumBase's Pure-CDP driver, which doesn't
use one.
This deletes them (and their SBOM file) in the same `RUN` as `uv sync`,
so the layer doesn't carry them. As a side effect image scanners stop
reporting on the Rust crates compiled into those binaries.
To check nothing needs them, I deleted the files in a throwaway
container from the current `dev` image and ran `internal_bypasser.get()`
against example.com. It started the browser, loaded the page and
returned it, same as without the deletion. I haven't run a full image
build of this branch, so CI is the first time that exact `RUN` line gets
executed.
TL;DR dropping an unused feature from an included package lowering size
and eliminating flags for security vulnerabilities that don't get
exercised.
@calibrain this one is optional too, but I don't see a downside, your
call.
Debian has a newer Chromium in trixie-security (156.0.8078.73), so this
moves the pin from .47 to .73 and the snapshot to 20261029T020000Z, the
first one that has it for both amd64 and arm64. Same shape as #1601.
To check the bypasser still works on it, I started a throwaway container
from the current `dev` image, upgraded Chromium to .73 the way the
Dockerfile does, and ran `internal_bypasser.get()` against example.com.
It starts, loads the page and returns it, the same as on .47. I also
pointed both versions at the Cloudflare demo challenge page at
scrapingcourse.com. They behave the same there, but neither one solves
it from where I ran it (same challenge detected 4 times, then abort), so
that only shows no difference, not a working solve.
I also ran it against an Anna's Archive search page on both versions.
Both returned the full results page, but the log shows no bypass
attempts, so that mirror didn't serve a challenge from where I ran it
and the solver itself wasn't exercised.
@calibrain your call on if we should update to this, it is minor bump to
what has already been pushed.
First of the route moves from Discussion #1536, following the plan table
there: the three onboarding routes (`GET` and `POST /api/onboarding`,
`POST /api/onboarding/skip`) leave `main.py` for
`core/onboarding_routes.py`, registered with
`register_onboarding_routes(app, login_required)` the way
`release_inspect_routes` takes it. `main.py` goes down by 72 lines and
the handler bodies are the same code (I diffed them after de-indenting:
the only difference is the local name of the error tuple).
There are two commits so the move can be reviewed on its own. The first
adds 15 tests for these routes, which had none, and they pass on current
`main` before anything moves. They pin the behaviour I most wanted to
keep: a 401 without a session when auth is on, a 403 for a signed-in
non-admin (that check lives in `login_required` and goes by path, which
is why the same decorator has to be passed in), the 400 on an empty
body, the result passed through on success and failure, the 500 with the
error text on an operational error, and open access when no auth is
configured. The second commit is the move, and no test changed in it.
What I checked on the move:
- The Flask URL map (every rule, its methods and its endpoint name) is
identical before and after: 77 rules, the three onboarding ones
included.
- Full suite, ruff and basedpyright are clean.
- I ran the app from this branch and stepped through the setup wizard in
a browser. It loads its steps from the moved route and goes from Search
Mode to Metadata Provider, and `POST /api/onboarding/skip` and the
empty-body 400 answer as before.
Two small things that do change. The log lines for these routes now come
from `shelfmark.core.onboarding_routes` instead of `main`, and the error
tuple is a local copy (`_ONBOARDING_ERRORS`, the same classes as
`_IMPORT_OPERATIONAL_ERRORS`) so the module does not import `main`.
Nothing else is different.
Second route move from the plan in Discussion #1536: the four settings
routes (`GET /api/settings`, `GET` and `PUT /api/settings/<tab>`, and
`POST /api/settings/<tab>/action/<key>`) go from `main.py` to
`core/settings_routes.py`, registered with
`register_settings_routes(app, login_required)`. `main.py` is 143 lines
shorter. These handlers call `settings_registry` directly, so nothing
else has to be passed in. The handler bodies are the same code, and the
only difference I found after de-indenting them is that `ruff format`
wrapped one signature that no longer fits on a line.
Same two-commit layout as the onboarding one (#1594). The first commit
adds 27 tests for these routes and they pass on current `main` before
anything moves. They pin the 404 for an unknown tab, the 400 when a
`PUT` body is not a JSON object (a list, a string, a number, or no body
at all), the "No changes to save" answer for an empty object, the
unsaved form values reaching an action, the result passed through with a
400 on failure, the 500 with the error text on an operational error, a
401 without a session, a 403 for a signed-in non-admin on all four
routes, and open access with no auth. The second commit is the move and
no test changed in it. The admin check for these paths lives in
`login_required`, so the same decorator is passed in.
What I checked:
- The Flask URL map (every rule, its methods and its endpoint name) is
identical before and after, all 77 rules.
- Full suite, ruff and basedpyright are clean.
- I ran the app from this branch and opened Settings in a browser: the
modal lists every tab and the General tab shows its values, which come
from the moved `GET`. Over HTTP, a `PUT` of a new search page title
saved and read back, a list body got the 400 and an unknown tab the 404.
As with #1594, log lines for these routes now come from
`shelfmark.core.settings_routes`, and the error tuple is a local copy
(`_SETTINGS_ERRORS`) so the module does not import `main`. The two PRs
touch neighbouring lines of `main.py` and I checked that they merge
together without a conflict, in either order.
Bumps python from 3.14.8-slim to 3.14.9-slim.
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
A wrong OIDC issuer, which is the usual IdP setup mistake, sends the
user back to the login page with a bare "Authentication failed". The
callback has a handler that explains it ("OIDC issuer validation failed.
Verify your discovery URL and IdP issuer/external URL configuration."),
and it never runs.
Authlib validates the ID token with joserfc now, so the error that comes
out of `authorize_access_token()` is `joserfc.errors.InvalidClaimError`.
The handler catches the one in `authlib.jose.errors`, which is a
different class with a different base, so the isinstance check is False.
I confirmed it by validating a real `CodeIDToken` with a wrong `iss` on
Authlib 1.9.1. I haven't run it against a live IdP.
The existing tests passed because they raise the old class from a mock.
The new one raises the error from a real `CodeIDToken`, and it fails on
the old import with `'Authentication failed'` in place of the guidance.
Two small things come with it:
- The new class names its attribute `claim` where the old one used
`claim_name`, so that read changes.
- `authlib.jose` is deprecated in favour of joserfc and Authlib's own
warning says it goes in 2.0. This was the only place Shelfmark used it,
so the import moving also takes us off the module that is going away.
joserfc is already installed as an Authlib requirement, so I added it to
`pyproject.toml` (`>=1.6.8`, Authlib's own floor) and the lock gets the
one entry. No new package.
Ruff, basedpyright and the backend suite pass (5424 passed).
The nightly client-deluge profile has failed since 61c7ad3 with "Failed
to
add to deluge: Failed to fetch torrent file: torrent fetch refused:
address
is not public". Shelfmark now fetches a .torrent only from the
configured
Prowlarr/Newznab/Jackett origin or from a public address, and the mock
Prowlarr's release linked straight to http://mock-aa/payload.torrent, a
private address on another origin. qBittorrent and Transmission hid this
by
handing the URL to the client; Deluge needs the file and failed.
Real Prowlarr rewrites indexer links to its own /{id}/download endpoint.
The
mock now does the same: the torznab link uses the host Shelfmark reached
it
on, and /{id}/download fetches the .torrent from mock-aa. Every client
now
gets the file through Shelfmark's own fetch.
tests/download/test_prowlarr_mock_torrent_fetch.py runs the real
extract_torrent_info against the mock in normal CI, so the link cannot
drift
off the Prowlarr origin again unnoticed; it fails on the old mock with
the
same error.
Bumps [publicsuffixlist](https://github.com/ko-zu/psl) from
1.1.2.20261021 to 1.1.2.20261023.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/ko-zu/psl/commits">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Two small things I found while going back over what has landed since
1.5.0.
`/api/releases` takes a `provider` and a `source`, and the policy check
only looked at `source`. A user whose policy blocks one release source
but not another could pass the blocked one as `provider` with the
allowed one as `source`, and get the blocked source's record back. The
record route already refuses that with a 403, so the listing route now
applies the same refusal when `provider` names a source that is not
being listed. The normal case, where provider and source are the same,
is already gated and skips the extra check.
The second commit is wording only. The Automatic Downloads setting and
the Hardcover sync doc said "synced requests", but the pass picks up
every pending Hardcover request, including ones a user made themselves
with Request Book. The download is still queued as the user who asked
(the admin is only the reviewer, same as a manual approval), and admins
already see every user's requests. The text now says that, and a new
test pins the behaviour for a user's own request.
I opened a scratch instance and looked at the Hardcover Sync tab: the
Automatic Downloads heading and the checkbox description read as
intended and nothing else on the tab moved. Full suite, ruff and
basedpyright pass, and the new policy test fails without the fix.
Draft until #1571 merges: this is stacked on it, so the first two
commits here are that PR's and only the last one is new. I will rebase
once #1571 is in.
Shelfmark strips credentials and links out of text in five places,
written in different PRs with different patterns: the log formatter,
`_safe_url` and `_redact_urls_in_text` in `torrent_utils.py`,
`redact_status_message` in `viewer_redaction.py`, the `sed` rules in
`genDebug.sh`, and the password handling in the settings registry.
Between them there were three regexes that find a URL in text, two
separate userinfo patterns and no shared list of what counts as a
secret. #1571 made the gap visible, because every pattern added to the
logger had to be remembered somewhere else too.
This moves the text and URL part into `shelfmark/core/redaction.py`,
which imports nothing from the app so the logger can depend on it:
- `redact_secrets` (mask in place, for logs) moved from
`core/logger.py`, patterns unchanged. The logger imports it.
- `safe_url` and `redact_urls_in_text` (keep scheme, host and path, drop
the query) moved from `torrent_utils.py` and are public now, so
`alldebrid.py` no longer imports a private name from another module.
- `strip_links` (remove links entirely) is what `redact_status_message`
now calls. It and `redact_urls_in_text` share one link pattern, the one
the viewer redaction already used.
Two small behaviour changes come from sharing that pattern. A closing
bracket or quote right after a URL is no longer swallowed into it when
scrubbing an exception message. And the scheme-less path that requests
prints after `url: ` now keeps only what is before its query in
`redact_urls_in_text`, where before it passed through as it was (the log
formatter masked the key, but the path and query stayed in the message).
Also here: the two `sed` rules in `genDebug.sh` that mask userinfo now
stop at `?` and `#` like the logger pattern does, with a test that fails
on the old script.
Not in this PR: the debug bundle zips the live log directory as it is,
so log lines written before #1571 are not redacted in it. Fixing that
means copying the logs to a staging folder and running them through this
module, which I would do as a separate change.
Tests: a new file covers `safe_url`, `redact_urls_in_text` and
`strip_links` directly, and two checks that the viewer and the logger
really use these. Full suite passes.
slskd answers a bare 500 when the peer doesn't reply to a queue request
inside its own 5 second wait (its log says "The wait timed out after
5000 milliseconds"). The peer often answers a moment later, so a
Soulseek download that would have worked failed on the first try.
The batch enqueue is now tried twice with 5 seconds between. A 4xx or a
connection failure is not retried, only a 5xx. While it waits the card
reads "The peer did not answer, trying again", and cancelling during the
wait stops without the second try. If the second try fails too, the
error message carries what slskd said (trimmed to 160 characters).
Before, that text was dropped and the card only showed the status line.
It's been running on my instance since mid-October, and the timeouts
still show up in slskd's log, so I can't say how many it saved. The five
new tests cover the retry that works, giving up after the second 500, a
400 and a connection error not being retried, and cancelling mid-wait.
Ruff, basedpyright and the slskd tests pass.
Newznab and Jackett fell back to the download URL sent with the queue
request when the release was no longer in the cache. Prowlarr already
returns no URL in that case, so this makes the other two match it: the
retry URL comes from the cache and nothing else.
A request for a release that has expired from the cache now fails with
"Release not found in cache" instead of being queued with whatever URL
came along.
The new tests cover both handlers on a cache miss and with no source_id,
and they fail on main. Full suite, ruff and basedpyright pass.
Code scanning alert 106 (stack trace exposure on the release search 503)
turned out to be worth a look. Jackett sends its API key as `?apikey=`
in the query string, and when Jackett is unreachable the source builds
its "unavailable" message from the requests exception, whose text
includes the full request URL. `/api/releases` returns that message to
whoever ran the search, so any signed-in user could read the Jackett key
while Jackett was down. I reproduced it with Jackett pointed at a closed
port: the 503 body had `apikey=<the key>` in it.
Jackett was added in #1521 and isn't in a release yet, so nothing
shipped with this.
`SourceUnavailableError` now masks credentials in its message with the
same `redact_secrets` the log formatter uses (query parameters like
`apikey=`, `user:pass@` in URLs and so on). I did it in the exception
rather than at the two Jackett sites that build the text, since its
message is written for the user and any other source that ends up with a
key in a URL would hit the same thing. The rest of the message stays
readable, for example "Max retries exceeded with url:
...?apikey=(removed)&t=book".
Prowlarr and slskd send their keys in headers so they didn't leak a key,
but their messages do include the internal host and port. I left that
alone here.
Tests: two Jackett ones (a connection error on the search and on the
indexer lookup, with the key in the exception text) and three on the
exception itself. The four that check masking fail on current main. The
full suite, ruff and basedpyright are clean.
On qBittorrent 5 a stopped torrent shows up as an unknown state. 5.0
renamed `pausedDL` and `pausedUP` to `stoppedDL` and `stoppedUP`, and
`get_status` only knows the old names, so the torrent comes back as
state `unknown` with the raw state name as its message. The paused
handling in the download handler never applies to it, for example the
grace it gives a torrent that has not started. `forcedMetaDL` is missing
from the same table, so a force-started magnet fetching its metadata
also shows up as unknown, although it is already in the
incomplete-states set and the metadata wait.
The three missing states are now mapped like their neighbours: the two
stopped states as paused, `forcedMetaDL` as downloading with "Fetching
metadata". The state names are the ones `qbittorrent-api` lists in its
`TorrentState` enum.
Tests: one parametrized test covers the old and new paused names and
`forcedMetaDL` (the two old paused names are there as controls). The
three new cases fail without the change. Full suite passes.
This is the UI side of #1581. A non-admin looking at their own
notification routes reads the same help text the admin does, including
"or mailtos://...?to={Email} in the admin default". They can't set that
default, a personal route can't use mailtos (personal routes are limited
to ntfy, Gotify, Discord, Slack, Telegram, Pushover and PushBullet), and
nothing on the page says whether an admin default exists. The default's
URL is hidden from them on purpose since it can carry credentials, so
the page just shows an empty All row, the same as when nothing is set
up.
What changes, on the user's own page only (the admin's view of a user
keeps the full text):
- The help text drops the admin default and mailtos part and lists the
services a personal route can use.
- The edit-context response now carries `adminDefaultRoutesSet`, a
boolean that is true when an admin default route with a URL exists. The
URL itself is still not sent, and the existing test that checks nothing
from the admin's values leaks still passes.
- When that is true, the help text adds a line saying a default applies
until the user adds their own, and once they have, that theirs replace
it.
How routes are resolved is unchanged. A user's own routes still replace
the default and don't merge with it. I didn't try to add mailtos for
personal routes, since keeping SMTP credentials in a user's own URL is
what the admin default avoids.
Tests: two route tests (flag true with a URL in the admin default, false
with an empty row), a vitest file for the description text, and the full
frontend suite, oxlint, oxfmt, tsc, ruff and basedpyright are clean. I
haven't looked at the page in a browser.
Fixes#1584. Thanks to @joeRob2468 for the write-up, which pinned this
down to the line.
Hardcover cuts a title like "Mistborn: The Final Empire" down to
"Mistborn" for searching when it can't tell the series prefix from a
title prefix, and search results don't carry the series. The library
check then matched the shelf title "Mistborn: The Final Empire" against
"Mistborn", saw "final empire" as extra words and took it for a longer,
different work (the "Dune" vs "Dune Messiah" rule), so the book got no
In library badge.
`match_entries` now tries the search title first as before and, if that
finds nothing and the full title is different, tries the full title too.
The full title is the more specific of the two so it should rarely add
false positives. Another book in the same series still doesn't match,
there is a test for that, and the Dune and Dune Messiah test passes
unchanged. The matching loop moved into a small helper to run it twice,
nothing in it changed.
This only touches the library check. The same short search title also
feeds release searches and auto-download matching, which I left alone
here because loosening or tightening those changes what gets searched
and downloaded for everyone.
Tests fail on current main without the change. The library tests, ruff
and basedpyright are clean.
- simplify row key serialization in TableField
- clean up userNotificationOverridesSection
Previously, after typing a single character, the browser would lose
focus due to the row keys changes on every new value. The fix makes the
row key independent of the URL’s changing text.
Additionally, fixed the per-user notification routes so that unchecking
"All" clears all event selections.
The torrent stall timer can be deferred for as long as the swarm keeps
changing. Since #1494 the torrent handler appends a note to its progress
message (`0% · 2 seeds`, and `· 0 seeds, giving up in 7m` while the
no-seeders clock runs). `update_download_status` treats any changed
message as activity, so a seed count that goes 2, 3, 1 between polls, or
the give-up countdown ticking down every minute, refreshes
`_last_activity` and the torrent never stalls. A torrent whose seed
count stays constant still stalls, which is why this is easy to miss.
`shelfmark/download/activity.py` already says what counts: a changed
status event or a changed progress value, and a message that changes on
a timer proves nothing. So `update_download_status` now compares
messages through a small `activity_key()` that drops that trailing seed
note. The card still gets every message and shows the live seed count,
only the stall clock ignores a change in the note. A new progress
percentage, a different client message or a status change still counts
as activity.
The handler builds the note in `_seed_note` and `activity_key` has to
match it, so one test builds the note for a range of seed counts and
countdowns and checks the key strips all of them. That also covers the
gap #1494 left, where no test connected a handler's message to the
orchestrator: the other two drive `update_download_status` with real
notes and a fake clock. Without the change the note test fails and the
key tests cannot even import `activity_key`. The other orchestrator test
is a control that a progress or status change still counts. Full suite
passes, and each new test also passes when run alone.
## What
A Webshare.cz download shows "Resolving Webshare link" for its entire
duration, so a large download (e.g. a 1.3 GB audiobook archive) looks
stuck.
## Why
After the link was resolved, the handler called
`status_callback("downloading", None)`. `update_download_status` treats
a `None` message as "keep the current message", so the "Resolving
Webshare link" detail stayed in the activity panel. The HTTP handler
sends `""` for exactly this reason.
## How
- Report `downloading` with an empty message, which clears the resolving
text.
- When the download response has no `Content-Length`, use the release's
size (`task.size`, parsed with `parse_size_string`) as the total for
progress. Without it, progress never moved, and stall detection would
flag a healthy long download after 5 minutes.
## Testing
- New tests in `tests/webshare/test_handler.py`: the downloading status
clears the message, and progress is computed from the release size when
there is no Content-Length.
- `uv run pytest tests/webshare` is green (34 passed), and `ruff check`
and `ruff format --check` are clean.
Follow-up to #1543.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A torrent that finishes with the wrong kind of files in it holds the
download for the whole completed-path timeout (60 seconds by default, up
to an hour) and then fails with "Completed download has no stable
supported files yet." An ebook task whose torrent holds only an `.m4b`,
an audiobook task that got an `.epub`, or a `.pdf`-only release with the
default format list all land here. In v1.4.0 these failed with the
specific format error from post-processing, but the wait for the sync to
settle (#1464) only counts files of the task's own type, so it keeps
polling for files that are never going to appear.
If the directory has no supported files, `_wait_for_completed_path` now
looks for finished book or audio files that this task does not accept
(the audiobook formats, the default ebook formats and pdf). When it
finds some and they are the same on two polls in a row, it returns the
path so post-processing can say what is wrong with the formats. Cover
art and notes alone do not count, because that is what a directory looks
like while the real files are still syncing over a remote mount, so
those keep waiting as before. A wrong-type file that is still growing
keeps waiting too. With no wait configured there is only one sample, and
a wrong-type file is handed on straight away.
The existing test that pins "no book files, no wait configured" uses a
jpg, so it stays as it is. I extended the test helper to take a content
type, which the wait tests never passed before. Four new cases fail
without the change (a wrong-type directory for ebook, audiobook and pdf
tasks), and two more check that sidecar-only and still-growing
directories keep waiting. Full suite passes.
Logs get pasted into issues, and the formatter from #1488 only masks
`?key=value` query parameters. I went through what else commonly ends up
in a log line and found these unmasked:
- credentials in a URL's userinfo, like a proxy's
`socks5://user:pass@host:1080` or `http://admin:secret@qbittorrent:8080`
- a passkey as a path segment, like `/rss/<key>` or
`/passkey/<key>/get.torrent`, which private trackers use
- the token in a Telegram bot, Discord webhook or Slack webhook URL,
which is what a notification URL looks like when it gets logged
All three are masked now, in `redact_secrets`, so they apply to every
handler. The path rule only fires after a segment that names a secret
(`passkey`, `apikey`, `rsskey`, `authkey`, `token`, `secret`, `rss`,
`torznab`) followed by 16 or more token characters. I left out generic
words like `download` on purpose, because `/api/download/<md5>/cancel`
appears in our own request log and the ids there are useful when
debugging. That is the tradeoff: a passkey in a `/download/<passkey>/`
path still gets through.
Separately, four modules created their logger with plain
`logging.getLogger`, so nothing they logged went through the formatter
at all: `api/websocket.py`, `download/clients/__init__.py`, and the
Jackett, Newznab and Prowlarr package inits. They use `setup_logger` now
like everything else.
Tests: ten cases, seven for the new shapes and three that must stay
readable (an md5 route id, a short `/rss/latest`, an email address),
plus two that check the stray modules now carry the redacting formatter.
Nine of them fail without the change. Full suite passes.
Not covered here: `/api/debug` still zips the existing log files as they
are on disk, so anything logged before this change rides along in a
bundle. That is a separate change in `genDebug.sh`.
If the download coordinator thread dies, every task sits at "Queued" and
nothing says why. The rest of the app keeps answering, `/api/health`
stays green and the container looks fine. #823 and #1166 were both that,
and the loop's `except Exception` cannot catch everything that can end a
thread.
`/api/health` now reports it. The response gets a `coordinator` field,
`running` or `stopped`, and when it is stopped the status is `error` and
the HTTP code is 503. The Docker healthcheck in the Dockerfile and the
one in the readme both use `curl -f`, so the container shows as
unhealthy and anything watching health (Uptime Kuma, autoheal) can act
on it. It needs no login, same as before, and shows no queue contents.
That 503 is the one decision here. Keeping the 200 and only adding a
`degraded` entry would not trip a status-code health check, which is the
whole point, so I went with 503. Easy to flip if you would rather not.
The orchestrator gets a small `coordinator_running()` that asks whether
the thread is alive. `start()` already knows how to restart a dead
thread, so restarting it automatically would be the true fix, but I
would rather the failure be visible first and decide on that separately.
Tests: three for the route (ok, 503, no login needed) and three for
`coordinator_running` (never started, alive, dead). They fail without
the change. One existing test hits `/api/health` through an app that
never starts the coordinator, so it now patches that. The readme's
health section has a sentence about the 503. Full suite passes, ruff and
basedpyright are clean.
`test_cache_hit_flushes_the_index_once_the_window_has_passed` (from
#1544) failed on the first attempt of two of my PRs' Python Tests runs
today (#1576, #1579) and passed on rerun, with nothing in either PR
touching the image cache.
The test sets the clock to exactly `last_saved + ACCESS_FLUSH_SECONDS`
and expects a flush. `(now + 30) - now` isn't always 30.0 in floating
point. For example `105.65207077218265 + 30 - 105.65207077218265` is
`29.999999999999986`, so the `>=` check is false and the flush never
happens. That only shows up when the monotonic value is small, which
fits a freshly booted runner (on this box the uptime is in the millions
of seconds and it never fails). I haven't been able to see the runner's
actual uptime in the CI logs, so that part is an inference, but pinning
`_last_saved` to that value reproduces the exact failure (`assert [] ==
[1]`) locally.
The test now puts the clock one second past the window. The production
check is fine, real elapsed time doesn't have this edge, so only the
test changes.
Fixes#1561, with thanks to @tmussche for the write-up, which had the
cause and the fix. When the files reach Shelfmark through a sync
(Resilio in the report), the wait for the completed path accepts the
folder once two samples a few seconds apart look the same. A sync that
copies in batches can pause longer than that, so 10 of 23 files were
imported and some of those were still partial copies.
For qBittorrent the client can say what the finished download holds, so
the wait now checks that. `DownloadClient` gets an optional
`get_expected_files()` that returns `ExpectedFile(name, size)` entries,
or None when the client cannot say, which is the default and keeps every
other client as it was. qBittorrent answers it from
`/api/v2/torrents/files`, leaving out files the torrent does not
download (priority 0), zero-byte entries and BEP 47 padding files.
`_wait_for_completed_path` asks once, and while any of those files is
missing or smaller than its final size it keeps waiting, using the same
timeout as before. When the wait runs out the error now says how far it
got, for example "Completed download is not fully on disk yet (10 of 23
files)." Once everything is there the existing stability check still
runs, so nothing is accepted that was accepted before.
Names from the client start with the torrent's top-level folder, so each
file is looked up with and without it, which also covers a torrent saved
without a subfolder. A single-file torrent is checked by size too, since
a file that is still syncing exists at a smaller size.
Two things I would like a second look at: the comparison is on exact
size, so a setup where something rewrites the files after the torrent
finishes would never match and would run into the timeout; and the file
list is read once per wait, not on every poll. If the client does not
answer, nothing changes.
I also moved the nested authenticated GET in the path derivation into a
`_get_with_auth` method so both calls share it, and added a sentence
about this to `docs/configuration.md`. Tests: eight for the wait
(partial folder, truncated file, files arriving during the wait,
complete folder, no top-level folder, single file both ways, and a
client without a list), and six for the qBittorrent file list. The wait
tests that depend on the new behaviour fail on current main. Full suite
passes.
Six places read the Supported Book Formats and Supported Audiobook
Formats settings on their own: post-processing, the post-process scan,
IRC, slskd, Prowlarr and direct download. For ebooks they already
agreed, since they all fall back to `DEFAULT_EBOOK_FORMATS`. For
audiobooks they did not. The settings registry and slskd fall back to
every audio format plus zip and rar, while post-processing and IRC fell
back to m4b and mp3 only.
They now share `shelfmark/core/formats.py`, with
`get_supported_formats`, `get_supported_audiobook_formats` and a
content-type aware `get_supported_formats_for`, and a shared
`DEFAULT_AUDIOBOOK_FORMATS` in `core/utils.py` next to the ebook one.
`policy.py` re-exports the same functions, so nothing that imports them
from there changes. It is not in `policy.py` itself because importing
the `postprocess` package pulls in the router, which imports the release
sources, so a source could not import from it without a cycle.
One behaviour difference, and only in the fallback: with the audiobook
setting unset, or emptied, post-processing and IRC used to accept m4b
and mp3 and now accept the full default list like slskd and the settings
page already did. A configured list is read exactly as before. Direct
download's own helper used the raw configured list and now reads the
same normalized one, which is identical for any list the settings page
can save.
Also public now: `parse_size` in the Prowlarr source (slskd and Newznab
imported it with the underscore) and
`coerce_completed_path_timeout_seconds` in the client base handler
(slskd imported that too).
Not changed on purpose: slskd's `_coerce_int` looks like a copy of
`coerce_int` but is not. It reads `"12.0"` as 12 and `True` as 0, where
`coerce_int` gives 0 and 1. And the `_parse_size` in the IRC source is a
different function, string to bytes.
Tests: a new set checks that post-processing, IRC and slskd all fall
back to the shared defaults and that the settings default is the shared
one, and the scan-archive and IRC tests now patch the shared config
instead of the per-module copies. Full suite passes.
The two seed settings, Give Up With No Seeders and Minimum Seeders, show
up for every torrent client, but only qBittorrent reports a seed count.
With Transmission, Deluge or rTorrent you can set both and nothing
happens. The first one says "Only qBittorrent reports seed counts" at
the end of its description, the second says nothing about it.
Both are now shown only when qBittorrent is the selected client, the
same `show_when` the other qBittorrent fields use, and Minimum Seeders
gets the same sentence. A value someone already saved for another client
stays in their config, it just is not shown, and it had no effect before
either.
`docs/environment-variables.md` is regenerated for the changed
description, which is two lines. One test checks that both fields are
tied to qBittorrent and say so, and it fails without the change.
These are the follow-ups from the review notes on #1516 and #1534 that I
said I would do once the series had merged.
The scheduler waits for the next cycle outside the try that guards the
cycle itself, so an interval setting that raised would end the thread
and nothing would sync again until a restart. It now logs the error and
tries again after six hours. `docs/hardcover-sync.md` says so.
The history check in the sync opened a new SQLite connection for every
book on the shelf. It opens one per sync now and shares it, which adds
up on a slow disk or a small board.
The two test nits from #1534 are in as well: a `parametrize` with a
single case, and an unused argument on a helper.
The thread surviving a failed wait and the shared connection each have a
test that fails without the change. Ruff, basedpyright and the backend
suite pass (5297 passed).
Sorry about this one. The environment variable references had also
fallen well behind, and again I was a big part of that. A bunch of the
gaps come from my own PRs (Hardcover sync, auto-download, the
Audiobookshelf library check) where I charged ahead with the
functionality and skipped the boring documentation part (did I mention
that documentation is boring?).
I again had Claude compare the docs to the code and generate the
updates, so it would be good to have another human look this over before
it merges. The doc diff is just the generator's output. The part worth a
real read is the handful of new entries in
`scripts/generate_env_docs.py`, since I wrote those descriptions from
`entrypoint.sh` and `tor.sh` and not from a settings definition.
What changed:
- Regenerated `docs/environment-variables.md`. It had 287 variables and
the generator now produces 311. The 21 that were missing are the
Hardcover Sync tab (6), auto-download (11) and the Audiobookshelf
library check (4). Nothing was removed and no existing default, type or
option changed.
- `SHELFMARK_API_KEY_READONLY` was only ever hand-edited into the doc,
so the generator did not know it and a plain regeneration would have
dropped the row. It is in the generator now.
- Added `UMASK`, `TOR_CHECK_INTERVAL` and `TOR_MAX_FAILURES`. The code
reads all three and no doc mentioned them.
- The table of contents built its anchors by lowercasing and swapping
spaces, so the Webshare.cz and Moly.hu links were already broken (GitHub
drops the dots) and Soulseek (slskd) would have broken with the
parentheses. They now follow GitHub's rule.
Two tests cover the new entries and the anchors, and both fail without
the generator change. To regenerate: `python
scripts/generate_env_docs.py` on Python 3.14.
A test that fails when the committed file differs from the generator
output would have caught this drift, but then every PR that adds a
setting has to regenerate the doc. I left that out, your call if you
want it.
This is the biggest of the docs batch, and this time I don't think it
was my PRs that caused it. The plugin guides fell behind as the plugin
API changed over several releases, and nobody updated them along the
way.
Same arrangement as the others: Claude compared the guides to the code
and wrote the changes, so another human look before this merges would be
good. Someone who knows the plugin API well would catch things I would
not. These are the parts I would read first: the search plan table, the
`SourceUnavailableError` advice in the error handling section, and the
`locating` status row.
What was wrong and is now fixed, all checked against the code:
Release sources guide
- `search` takes `(book, plan, *, expand_search, content_type)`. Every
sample had the old one-argument signature, so a source written from the
guide raised a TypeError on its first search. There is now a short table
of the search plan fields.
- A source is registered by adding its module to
`_BUILTIN_SOURCE_MODULES`. The guide said to add an import at the bottom
of `__init__.py`.
- `get_column_config` is an instance method, not a classmethod, and
`ColumnRenderType.SEEDERS` does not exist (the real members are listed).
- `download` can return a `HandoffResult`. `list_available_sources`
lists every source with an `enabled` flag, it does not filter.
- `group="sources"` is gone, since only `direct_download` and
`metadata_providers` exist.
- Field lists for `Release`, `DownloadTask`, `BookMetadata`,
`ColumnSchema` and `ReleaseColumnConfig` are brought up to date, with
the protocol enum and the `locating` status.
- The optional hooks that were not mentioned (`post_process_cleanup`,
`build_retry_resolution_fields`, `list_authors`, `seed_complaints`,
`get_record`, `get_destination_override`, `search_results_are_releases`)
are documented.
Plugin settings guide
- The examples imported from `shelfmark.metadata_providers.base` and
`shelfmark.release_sources.base`, which do not exist. The metadata
example also skipped the abstract `search_by_isbn` and the kwargs
factory.
- Value priority now starts with the environment variable, and
`search_mode` is listed next to `general` as using `settings.json`. The
`metadata_providers` group order is 12, not 50.
- Documented `TagListField`, `OrderableListField`, `TableField`,
`env_var`, `env_supported`, `user_overridable`, `universal_only`,
`filter_by_field`, `register_on_save` and the `current_values` argument
for action callbacks.
Also in here: `docs/dev/index.md` was a TODO stub and is now a short
index linked from `docs/index.md`, a dead link in the metadata providers
README is fixed, and the README table gains Databáze knih and Moly.hu.
To check the samples were not just plausible, I ran the quick start and
the full BookNet example against the real registry. Both register,
instantiate and return their column config. The old BookNet example
failed on import with a NameError, because the Test Connection callback
was defined below the settings function that references it. The
decorator calls that function straight away, so both settings snippets
now define the callback first.
These are the leftovers from the docs check, the fixes that did not
belong in the other three.
Claude did the comparison and wrote the changes here too, so I would
still like a human look before it merges. The one new piece of prose
worth a read is `troubleshooting.md`. It was a TODO page that two other
pages link to, so I filled it with only what I could confirm in the
code: where the logs go, `LOG_LEVEL`, the `DEBUG=true` bundle from the
bug report template, and reporting an issue. Happy to drop it back to a
stub if you would rather write that one yourself.
- `configuration.md`, `custom-scripts.md`, `users-and-requests.md`:
Booklore is Grimmory in the UI now. The stored mode value is still
`booklore`, and the custom script doc says so. The hardlink heading now
matches the "Hardlink Book Torrents" and "Hardlink Audiobook Torrents"
checkboxes.
- `custom-scripts.md`: the email output runs scripts too
(`phase=post_email`, `output.mode=email`, details under
`output.details.email`). It was missing from the phase list.
- `users-and-requests.md`: a delivery that fails with no retry available
goes back to pending on its own, and the admin reopen action is for
rejected requests. The page said an admin reopens a failed delivery.
Also noted that rules cannot use Request Book.
- `hardcover-sync.md`: added the missing
`AUTO_DOWNLOAD_AUDIOBOOK_MIN_SIZE_PERCENT` row and a sentence in the
ranking paragraph.
- `api-access.md`: an admin session also reads `/api/stats`, and what
the 403 says.
- `installation.md`: points at the Lite, Tor and WireGuard variants.
Docs only, no code touched.
The sample extraction from Discussion #1536: `/api/stats` moves out of
`main.py` into `shelfmark/core/stats_routes.py`, registered with
`register_stats_routes(app, ...)` like the other route modules. The
route logic is the same, and no test changed.
`main.py` passes two callables instead of values, one for the user
database and one for the auth mode. They are called per request, so the
tests that patch `main.user_db` and `main.get_auth_mode` after import
still work. The read-only key allowlist in the middleware now uses the
path constant from the new module, so the path lives in one place.
This is only the shape. If it looks right I'll move the route groups
from the 1.5.0 PRs the same way, one PR each. If you'd rather leave
`main.py` as it is, close this and no harm done.
Sorry, this one is overdue. The readme still describes Shelfmark as it
was a few releases ago. Part of that is on me: Hardcover Sync, Library
Check and a few other things I added came in through PRs where I charged
ahead with the functionality and skipped the boring documentation part.
The rest are features other people added (Jackett, Webshare.cz, DNB and
so on) that never made it into the readme either.
I had Claude compare the readme to the code and generate the updates, so
it would be good to have another human look this over before it merges.
The scope section in particular is your call, since it states what
Shelfmark is for.
What changed:
- **Project Scope** said Shelfmark doesn't know what you own, doesn't
integrate with library software and has no background automation.
Library Check and Hardcover Sync ship exactly those things, so the list
now says what is still true (it never edits your library, and Hardcover
Sync is the one optional background job) and links both docs. I kept the
four bullets and the "not missing features" line.
- **Sources and providers** now include Jackett, Webshare.cz and
OceanofPDF, and DNB, Databáze knih and Moly.hu.
- **Features** gain Library Check and Hardcover Sync, both linking to
their existing docs pages.
- **Settings list** gets the download clients (including the four debrid
services), Jackett and Webshare.cz, and the Library Link bullet mentions
the audiobook library URL.
Docs only, no code touched. Every name and link was checked against the
code and the docs folder.
Same idea as #1558, for three more places that hand `str(e)` to whoever
is logged in. These are code scanning alerts 106, 142 and 143.
`/api/status` and `/api/config` returned the raw exception text of an
OSError, RuntimeError and so on with the 500. They now return "Could not
read the download status" and "Could not load the configuration". The
full error is still logged with its trace.
For `/api/releases`, a source failing with an unexpected error put
`source: <exception text>` into the error list that goes back to the
caller. `search_source_releases` takes a new `include_error_detail`
flag. The route passes True for admins (and when auth is off, where
everyone is treated as an admin) and False for everyone else, who get
`source: search failed`. The message of a `SourceUnavailableError` is
written for the user, so it is kept either way, and the background
auto-download caller keeps the default and sees the full text as before.
Left alone on purpose: the `reason` in the release inspect response,
since a test covers that it carries the error text and it looks
intentional.
Tests: two for the routes and two for the helper. All four fail without
the change. Full suite passes (5295), ruff and basedpyright are clean.
## Summary
Adds Jackett as a separate release source (`jackett`), as requested in
#1443.
- New `shelfmark/release_sources/jackett/` package: API client, settings
tab, source and handler.
- Searches Jackett's Torznab endpoints: the `all` aggregate by default,
or each selected indexer individually. Failed or timed-out searches
raise `SourceUnavailableError` instead of reporting "no releases found".
- Reuses Prowlarr code rather than duplicating it: the Torznab parser
(now also reads `<jackettindexer>`), `_prowlarr_result_to_release` (new
optional `source`/`source_id` args), the release cache, URL/protocol
helpers, the timeout clamp and size-limit helper (new optional
config-key args), and the handler's `list_files` / retry-restore logic.
- Downloads use the existing shared torrent client settings.
- `JACKETT_URL` is a trusted origin for `.torrent` fetching (it is
usually on the LAN). Prowlarr's `X-Api-Key` is not sent to it.
- Added a Jackett section to `docs/environment-variables.md`.
## Screenshots
<img width="1117" height="936" alt="Screenshot 2026-10-21 at 13-45-34
Shelfmark"
src="https://github.com/user-attachments/assets/00339999-72c7-4226-bd9a-df8b325a7e22"
/>
<img width="1117" height="938" alt="Screenshot 2026-10-21 at 13-50-38
Shelfmark"
src="https://github.com/user-attachments/assets/3c9cd2b4-932a-4b5c-9e10-6ec22b5f987d"
/>
<img width="1117" height="937" alt="Screenshot 2026-10-21 at 13-50-57
Shelfmark"
src="https://github.com/user-attachments/assets/9c4d0ed9-da2f-4323-b188-341b45066efd"
/>
## Not included
- First-run onboarding does not offer Jackett yet.
- No Jackett-specific category settings; it uses the standard 7000 /
3030 categories.
## Testing
- New unit tests in `tests/jackett/` (client, source, handler) using
mocked responses.
- Full suite passes except 11 `tests/e2e` tests that also fail on a
clean tree here (cannot write `/var/log/shelfmark`).
- Tested against a live Jackett instance.
Closes#1443🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
CodeQL scans `tests/`, which only runs in CI and the e2e docker stack.
That is where the open `py/full-ssrf` (critical) and the two
`py/reflective-xss` alerts come from, all in the e2e mock server, plus
four `py/incomplete-url-substring-sanitization` alerts on URL asserts.
None of them can reach a running instance.
This adds `.github/codeql/codeql-config.yml` with `paths-ignore: tests`
and points the init step at it. It closes those seven and hides nothing
in `shelfmark/` or the frontend. The rest of the open alerts are
unchanged.
@calibrain I know I have ability to merge this, but wanted you to be the
one for this. It clears the test alerts and leaves the rest, and
prevents you from having to clear them manually. Just a little QoL.
Dependabot has four open npm alerts and no PRs for them, since they are
all transitive. This updates the lockfiles only, no package.json change.
| Package | From | To | Alert |
|---|---|---|---|
| socket.io-parser | 4.2.6 | 4.2.7 | zero-attachment memory exhaustion
(high) |
| ws | 8.18.3 | 8.21.3 | memory exhaustion DoS (high) and uninitialized
memory disclosure (medium) |
| engine.io-client | 6.6.4 | 6.6.7 | needed to get ws past its `~8.18`
range |
| baseline-browser-mapping | 2.9.19 | 2.11.31 | process termination on
invalid input (medium, dev only) |
I think the practical risk was low to begin with, since ws is only used
by engine.io-client under Node and the built bundle doesn't include it.
Still, it clears the alerts and the bumps are small.
Done with npm 11 so the `libc` fields in the lockfile stay as they were.
A clean `npm ci` from the new lockfile gives 0 vulnerabilities, and
`tsc`, `vitest` (241 tests) and `vite build` all pass.
Two log fixes in the same area, both found while going through the open
code scanning alerts.
AllDebrid: the unlock path logged the file link in full. The last-resort
fallback appends `apikey=` to the link and returns it, and the "Failed
to download from ..." error carries that same link into
`logger.exception`, so the key could land in the log and the traceback.
The links are now logged through `_safe_url` (scheme, host and path
only). The GET unlock fallback also logged the exception text at debug
level, and requests puts the request URL in that message, which includes
`apikey=`. That one now logs the exception type only. This covers code
scanning alert 124.
Newznab: `logger.debug("Newznab API: GET %s params=%s", url, *params)`
star-expands the dict keys into the log arguments, so with debug logging
on it raises `TypeError: not all arguments converted during string
formatting` and the line never prints. Logging `params` as is would
print the API key, so it now logs the sorted parameter names.
Tests: the Newznab one formats the logged line and checks the key is
absent. The AllDebrid ones check the fallback and failure paths log no
query string and that the fallback still returns the link with the key.
All three fail without the change. Ruff and basedpyright are clean,
`tests/newznab` and `tests/prowlarr` pass.
Two small leaks from going through the open CodeQL alerts (code scanning
alerts 100 and 123).
The local download route returned `str(e)` on an OSError, and for a
missing or unreadable file that text is the file's path on the server
(`[Errno 2] No such file or directory:
'/audiobooks/Author/Title.epub'`). #1422 cuts a non-admin's download
path down to the file name everywhere else, so this put it back. The
route now answers "Could not read the file" and the detail stays in the
log.
`should_bypass_proxy` logged the whole request URL at debug when it
could not be parsed. URLs can carry credentials, so it logs only the
exception now.
Each change has a test that fails without it. The full non-e2e suite,
ruff and basedpyright pass.
A request whose book turns out to be in the library is closed with the note `[auto] Already in the library.` instead of sitting in Needs Review. A request an admin handled by hand while a pass was running is skipped. A series label in front of a Calibre title no longer hides an owned book, and a Calibre title marked as another edition (full cast, dramatized, GraphicAudio) is not the book.
When no source matches, up to three simpler forms of the title are tried on each source under the same strict match. Optionally (off by default), an unmatched audiobook request is looked up on Audible US and UK through Audiobookshelf's metadata search and the request says what the store answered. It never declines anything.
A Soulseek peer whose recent transfers all failed, and an AudiobookBay post whose comments say it has no seeders, are tried after the other matches and never excluded. A failed or stalled release is kept out for AUTO_DOWNLOAD_RETRY_FAILED_RELEASE_DAYS and then tried again, and a torrent that died for one request is not offered to another. A request stuck on queued or downloading is settled from the download history. Optionally (off by default) a failed download is retried with another release within minutes.
A synced request whose download failed, was interrupted by a restart, or was cancelled by the stall timer goes back to pending after AUTO_DOWNLOAD_RETRY_DAYS (7 by default, 0 turns it off), a few per pass, oldest first. Only what never arrived is retried, the failed release is remembered, and a request is given up on after three failed retries. Downloads cancelled by a person and rejected requests are never retried.
Errors count as errors: a search that failed in one source no longer reads as no match, a throttled Hardcover lookup is an error and not a skip, and a request that blows up doesn't abort the pass. Both summaries carry the error count.
A request for the same Hardcover book and format is skipped when the history already has a finished or running download for it. A request that finds nothing is searched once a day after three no-match passes, then every two days, up to AUTO_DOWNLOAD_BACKOFF_MAX_DAYS (3 by default, 0 turns it off).
A shelf book can be requested as an ebook, an audiobook or both (one request per format), and each request is checked against its own library, source priority list and format guard. Ranking within a source prefers the most downloaded copy, then size, and one m4b is preferred to a set of chapter files unless it is much smaller than the other copies. Soulseek peers are ranked by free slot, queue length and speed.
Automatic downloads refuse multi-book packs, sequels and other works named in the title, study guides, abridged or dramatised versions, two-in-one titles, releases in a language the requester didn't ask for, and an archive for an audiobook request.
Matching folds accents on both sides, drops an alternate title for the search and the match, and reads small numbers the same in any form. AudiobookBay releases with no author in the listing title are judged on the authors credited on the detail page.
Adds an Audiobookshelf provider for the library check, next to the Calibre one. It reads the server over its REST API (title, author, ISBN and ASIN per item), answers for audiobooks, and has its settings in General with a test button that uses the unsaved form values. Read only.
The Hardcover sync and automatic downloads now fail closed when an enabled library cannot be read: the sync skips its cycle and an automatic download holds the request. A library that listed items at the last index and now lists none counts as a failed read. Badges still fail open.
## feat(metadata): add Deutsche Nationalbibliothek (DNB) metadata
provider
This adds the Deutsche Nationalbibliothek (DNB) as a metadata provider.
It covers German editions far better than the existing providers.
No API key, disabled by default, and the same pattern as the other
providers.
It uses DNB's public SRU interface and parses MARC21-XML. Thats one
reason why this needs more code than the Google Books or Open Library
providers. The others are the search (DNB has no relevance ranking) and
the language filter, see below.
### Notes
- **Search box.** Search-box text goes from precise to loose until a
stage has hits: the complete title, then the title as a phrase, then
every word in a title or a person's name, last without German articles.
- **"Only show editions in my book languages"** (optional, on by
default). DNB also catalogs a lot of translations. DNB filters by the
searching user's Default Book Languages, else the system default.
- **No covers.** DNB's cover service is behind Anubis bot protection, so
the provider returns no cover URLs.
- **Proxy support** already included like Open Library since #1526
### Changes outside the provider
- DNB is in the onboarding dropdown.
- Calibre's `dnb-idn` identifier is mapped, so DNB results show library
ownership
- In the env-var docs I added only the DNB section. Regenerating the
whole file on main currently also reorders unrelated sections and drops
`SHELFMARK_API_KEY_READONLY` (and the Databáze knih section sits inside
the Moly.hu block).
### Ideas
- **Deduplication.** DNB lists every edition. Kafka's "Die Verwandlung"
has around 200 German editions of the same text. Grouping them would
touch the API, the frontend and release search.
- **Language-aware metadata search** for all providers. The other
providers ignore Default Book Languages in metadata search. The
`user_id` added here could serve them too.
## What
Adds [Webshare.cz](https://webshare.cz), the Czech file host, as a
release source for ebooks and audiobooks. It follows the Libgen source:
disabled by default, with a settings tab for credentials and a **Test
Login** button that reports whether the account is VIP.
## Why
Webshare is one of the main places Czech and Slovak ebooks and
audiobooks are shared, and none of the existing sources reach it.
Together with the Databáze knih metadata provider (#1479), this lets
Universal mode find Czech and Slovak titles end to end.
## How
- **API client** (`webshare/api.py`): Webshare's XML API (`POST
/api/<endpoint>/`). Login is the salted exchange the API documents:
fetch the salt, send `SHA1(MD5_CRYPT(password, salt))` plus an MD5
digest. Python dropped the `crypt` module in 3.13, so md5-crypt is a
small pure-Python port, checked against `openssl passwd -1`. The session
token is cached per account and refreshed once when an authenticated
call fails. File-level errors (`FILE_LINK_FATAL_*`) are not retried. XML
is parsed with `defusedxml`.
- **Search** (`webshare/source.py`): Webshare matches file names only,
so the file name becomes the release title and its extension the format.
Queries are title + author, then title only, or the manual query.
Results are filtered to the configured `SUPPORTED_FORMATS` /
`SUPPORTED_AUDIOBOOK_FORMATS`, plus `zip`/`rar` for audiobooks, since
they are usually uploaded as archives. Password-protected files are
skipped. The language is guessed from tags in the name (`CZ`, `SK`,
`ENG`, ...), and the vote balance is a sortable column. API failures
raise `SourceUnavailableError`.
- **Download** (`webshare/handler.py`): resolves a time-limited link per
attempt with `file_link`, then streams it into `TMP_DIR` with progress
and cancel support. Large audiobook archives are not buffered in memory.
- **Settings**: `WEBSHARE_ENABLED`, `WEBSHARE_USERNAME`,
`WEBSHARE_PASSWORD` (a `PasswordField`, already covered by the
`_PASSWORD` redaction in `genDebug.sh`), and `WEBSHARE_MAX_RESULTS`. The
`docs/environment-variables.md` section was added.
## Testing
- `tests/webshare/`: 31 offline tests. They cover md5-crypt vectors,
password hashing, XML parsing, token reuse and re-login, non-retried
file errors, availability gating, format filtering, query fallback,
release mapping, language guessing, and the handler's streaming,
small-file rejection, link errors and cancel.
- `ruff check` and `ruff format --check` are clean. The rest of the unit
suite fails the same way with and without this change: Windows-only
tests on my machine.
- Verified live against webshare.cz: search parsing and the salt
endpoint's error path.
- Still a draft: I haven't yet verified login and download with a real
account.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
This replaces #1047, which @InfiniteAvenger opened and calibrain offered
to re-review. It is Caleb's feature commit with his authorship kept,
rebased onto current main, with small fixes, tests and a docs page on
top. Everything is opt-in and off by default, so nothing changes for
anyone who doesn't turn it on.
What it adds is a **Settings, Hardcover Sync** tab with three parts that
can be switched on separately:
- **Shelf sync**: a Hardcover shelf (Want to Read by default) is pulled
into requests on a schedule, or with a Sync now button. Requests carry
the Hardcover metadata and cover.
- **Automatic downloads**: synced requests are approved and a release is
picked from a source priority list. A strict title, author and format
match (plus a seeder minimum for torrents) decides, so anything without
a confident match stays pending for manual review.
- **Skip books you already own**: sync and automatic downloads ask
whichever libraries are enabled under Library Check (Calibre today) and
skip a book that is already there.
On top of Caleb's commit I made the request owner a lookup instead of
assuming user id 1, sent synced cover URLs through the proxy when the
request is created, got ruff and basedpyright clean, and added tests and
`docs/hardcover-sync.md`. His commit also had its own Audiobookshelf
settings and a Test library connection button, written before the
library check was split into providers. The button would have raised a
TypeError against the current `test_connection`, so those settings are
gone from this tab and an Audiobookshelf provider comes as a separate
PR, which is the part of #1047 that is missing here (for now
anyways...).
I've run this daily for about a couple of weeks. Ruff, basedpyright and
the backend suite pass (4426 passed).
This is 1 of 5. As calibrain suggested, the rest are stacked on this
one, each saying which number it is.
Closes
https://github.com/calibrain/shelfmark/pull/1047#issuecomment-6302071588
---------
Co-authored-by: InfiniteAvenger <calebewest02@gmail.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
Two small cuts to work that repeats on every poll or request.
The delivery-state sync behind /api/status and the activity snapshot
listed every fulfilled request (SELECT * plus a JSON parse of book_data
and release_data per row) before it looked at the queue. That history is
never purged, so an idle instance paid for it on every poll. It now
returns straight away when no queue bucket holds a task, since nothing
could map to a request then. Results and emitted updates are unchanged.
The activity snapshot gains less from this, because its status merges
download history rows, so it is usually not empty.
In proxy auth mode the before-request hook looked the user up by name
and by session id on every API call, cover images included, each on a
new SQLite connection. UserDB.get_user_cached answers repeat lookups
from memory for up to five seconds, bounded at 256 entries. create_user,
update_user and delete_user drop the cache as soon as they commit, and a
lookup that overlaps a write is not stored, so a role change or deletion
made in the app applies on the next request. Changes made outside that
UserDB instance (another process, or the separate instances some helpers
open) show within the five seconds. Only the middleware uses the cached
path; every other get_user caller is unchanged.
Each change has a test that failed before it: one counts list_requests
calls for an empty queue, the others count get_user calls across
repeated lookups and middleware runs, check expiry with a patched clock,
and check that writes and caller mutations do not leave a stale row
behind. The full suite passes (4432 passed, 48 skipped, e2e excluded),
as do ruff and basedpyright on the changed files.
I also looked at throttling the full status_update broadcast when only
the progress message changes during torrent and usenet polling, and left
it out. Those handlers poll every 2 seconds and
DOWNLOAD_PROGRESS_UPDATE_INTERVAL defaults to 1 second, so a throttle on
that interval would skip nothing at the default setting. A longer window
would slow the speed and ETA text in the UI, and without a trailing
flush it would hide the last message of a quick burst, such as "Waiting
for bot response" arriving just after "Requesting file from bot", for
the whole wait that follows.
---------
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
#1422 hid `source_id` and the download path from non-admins but left the
status message alone. A failed add stores the client's exception text
there, for example `401 Client Error: Unauthorized for url: <full url>`,
and for a requests error that URL carries the path and query: an indexer
key, a tracker passkey, a peer name. It reached the owner through
`/api/status`, `/api/activity` and the `status_update` socket event.
`redact_download_payload` now replaces every link in `status_message`
with `(link removed)`. That covers the live queue, the socket push and
the history snapshot, since all three go through it. Messages without a
link are unchanged and admins still see the full text.
Tests cover the payload, a whole status snapshot, and messages that must
stay as they are. They fail on main and pass here.
Found in a Claude pass over 1.5.0.
---------
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
I went through what Shelfmark does per search and per cover and measured
the pieces that showed up. Four of them are cheap to fix, so this is one
PR. Shelfmark runs on a single gevent worker, so any of these blocks
every other request while it runs, and I'm assuming a lot of people run
it on small hardware (a Pi, say).
Cover cache. Every cache hit rewrote the whole `cache_index.json` just
to bump the entry's last-access time. On my box that is 3.7 ms of CPU
per hit at 500 entries and about 100 ms at 15,000 (2.2 MB written each
time), and a results page asks for around 40 covers. The access time now
stays in memory and the index is written at most every 30 seconds, or
whenever something else saves it. A crash inside that window loses some
access times, which only changes eviction order.
Settings reload. `config.refresh()` rebuilt the cache by reading and
parsing a tab's JSON file once per field, so about 250 parses per
refresh, 100 ms of CPU, and the metadata search, providers and config
routes each trigger it. It now reads each file once per rebuild (several
tabs share `settings.json`, so the memo is keyed on the file path).
Library match. When the library check is on, each search result is
compared against every library entry, and the title was split into words
again for every entry. The words are now computed once per result. I
measured 33 ms per result at 5,000 library entries and 106 ms at 20,000,
which is a second or more per page of 40 results. The matching itself is
unchanged.
Cover URL probe. `is_covers_cache_enabled()` creates and deletes a file
in the config directory to check it is writable, and it runs for every
cover, queue task and history row. The answer is remembered for 60
seconds. The probe is 0.045 ms on its own, but it is called a few
hundred times per request in the worst case.
Each change has a test that fails without it (call counts on the index
saves, file opens during a refresh, tokenisation calls, and probe
calls). Ruff, basedpyright on the changed files and the non-e2e suite
pass. The timings are from a dev machine with an empty config, so the
real numbers depend on cache and library size.
Two small changes that stop Shelfmark from opening a new HTTPS
connection for requests that go to the same host over and over.
Anna's Archive download counts. Every search result gets its count from
`/dyn/md5/inline_info/<md5>`, and each of those was a bare
`requests.get`, so a search with 50 results opened 50 TCP and TLS
connections to the mirror. `search_books` also runs once per title
variant, and the variants of one book mostly find the same files, so the
same md5 was looked up again for every variant that found it, and again
when the release list was reopened. The lookups now share one
`requests.Session`, and a successful count is kept for five minutes in a
bounded `CacheService` (the same class the metadata cache uses, as its
own instance, so it does not depend on the metadata cache toggle). A
lookup that fails or comes back without a count is not cached, so it is
retried on the next search just like today. The URL, headers and 5
second timeout are unchanged.
Debrid clients. Real-Debrid, AllDebrid, Debrid-Link and TorBox also used
the bare `requests` helpers. The download handler polls `get_status`
every two seconds for each active job, so every poll was a fresh TLS
handshake with the debrid API. Each of those modules now keeps one
`requests.Session` and sends its API calls through it, the way the
qBittorrent, Deluge, Prowlarr and Newznab clients already do. Headers,
timeouts, `verify` and error handling are still passed per call exactly
as before. The existing tests that patched
`<module>.requests.get/post/request` now patch
`<module>._session.<verb>` instead; nothing else in them changed.
How I checked it: each change has a test that fails on main and passes
here. They count the requests that reach `requests.Session.request` and
the sessions built while doing it, with no network. On main, three polls
of each debrid client build three sessions, and two overlapping batches
of AA results look `bbb` up twice. There are also tests for the count
cache expiring (with a faked clock) and for a failed lookup not being
cached. Full suite: 4432 passed, 48 skipped. ruff and basedpyright are
clean on the changed files.
I also looked at caching `/api/releases` on the server and left it out.
The release modal already caches each book's results per source and
content type in the browser for five minutes (longer for IRC and slskd
through `cache_ttl_seconds`), and that cache survives closing and
reopening the modal, so reopening it does not run a new search today. A
server cache would only help after a page reload, and it would quietly
undo the modal's "apply filters" and manual search refetches, which drop
the browser cache on purpose, unless the frontend also learned to send a
bypass. That did not seem worth it for this PR.
The start-up reconcile runs at import, before the coordinator starts,
inside a try that named five exception types and left out `TypeError`.
`main.py` already defines `_IMPORT_OPERATIONAL_ERRORS` for this shape
and the other module-level guards use it, so this switches to it. An
unexpected `TypeError` from the reconcile would otherwise fail the
import of `shelfmark.main`, and the worker would fail to boot on every
restart.
I didn't use `except Exception` because BLE001 is on and nothing in
main.py does.
The test reloads main with the reconcile raising `TypeError`, and with
`OSError` as the control (that one already passes on main). Full non-e2e
suite passes, ruff and basedpyright are clean.
Found in a Claude pass over 1.5.0.
The users, download history and activity view-state databases open their
connections with SQLite's default busy timeout of 5 seconds. A start-up
migration, the restart reconcile, or any statement that overlaps a
backup or a second process raised "database is locked" after 5 seconds,
and the reconcile finished only part of its work.
One shared `DB_BUSY_TIMEOUT_SECONDS` (30) in `core/utils.py`, used by
all three. The test reads SQLite's `busy_timeout` pragma back from each
connection, so it doesn't need a real lock fight.
I left `UserDB.initialize()` without a guard on purpose. With 30 seconds
the wait covers a backup, and swallowing a schema failure there would
hide a worker that can't run.
Found in a Claude pass over 1.5.0.
With Max Concurrent Downloads Per Source above 0 the download pool is
sized per source and a task is admitted when its own source is below the
limit. Max Concurrent Downloads isn't read at all in that mode, so 3 per
source across 3 busy sources runs 9 at once. Neither description said
so.
The per-source text now says it replaces the global limit and what the
total can be, and the global one says it isn't used while the per-source
limit is set. Text only, no behaviour change.
I changed the two rows in `docs/environment-variables.md` by hand.
Running `scripts/generate_env_docs.py` on main today also drops the
`SHELFMARK_API_KEY_READONLY` entries and changes the slskd anchor, so
the file looks hand-edited since it was last generated, and I didn't
want to fold that into this. The two rows match what the generator
produces for them.
Found in a Claude pass over 1.5.0.
#1429 made the first terminal status win, but the guard only refused
terminal to terminal changes. A worker's poll that reports DOWNLOADING
after a cancel still overwrote CANCELLED (the poll loop calls the status
callback before it waits on the cancel flag), and the COMPLETE or ERROR
that followed was then accepted as a first terminal status, so the
terminal hook ran a second time.
This refuses any status change away from a terminal status. `add()` and
`enqueue_existing()` reset the status themselves, so retries are
unaffected.
The new test cancels a downloading task, sends DOWNLOADING and then
COMPLETE, and expects CANCELLED and one hook call. It fails on main and
passes here. Full non-e2e suite passes, ruff and basedpyright are clean.
Found in a Claude pass over 1.5.0, same as my other small fixes from it.
Fixes#1531.
All Direct Download providers share the one search budget
(`RELEASE_SEARCH_TIMEOUT`, 300s). Anna's Archive runs first, and when
its bypass keeps failing the retries use up the whole budget. OceanofPDF
then sees an expired deadline, skips its queries, and the search raises
Anna's Archive's error even though OceanofPDF never got a turn.
I didn't split the budget evenly, since the deadline module itself says
a cold-start Anna's Archive solve can legitimately take 60 to 120s and a
150s or 37s slice would break setups that work today. Instead
`search_deadline.unit_deadline(waiting=n)` opens a child deadline for
one unit of work. It gets the parent's remaining time minus 30s for each
unit still waiting, never less than 30s and never more than the parent
has left. A single provider still gets everything, a fast provider's
unused time rolls on to the next, and a stuck one stops at its own limit
with the reserve intact.
It's used in the provider loop in `DirectDownloadSource.search`, and in
the source loop in `/api/releases` since it's the same shape and four
lines. With several sources enabled the first one now gets 30s less per
source behind it, which is still well above a cold solve for the usual
two or three.
Tests: a stuck first provider no longer starves the second (fails on
main, passes here), a lone provider gets the full budget, and a child
never outlives its parent. Two existing tests in
`test_releases_api_budget.py` tripped the deadline in force from inside
the source, which is now the child, so they use a short request budget
instead. ruff, basedpyright on the changed files and the non-e2e suite
are clean.
OceanofPDF is inactive on my own instance (no mirror URL set), so I
haven't reproduced this live. It's verified by reading the code and by
the tests. I had Claude do the pass on this one.
The default for Supported Book Formats (`epub, mobi, azw3, fb2, djvu,
cbz, cbr`) was written out in five places: the settings registry,
`postprocess/policy.py`, `irc/parser.py`, Prowlarr's `source.py` (added
in #1490) and slskd's `source.py` (added in #1428). Both of those two
arrived in 1.5.0, and both are mine, so this is cleaning up after myself
as much as anything.
The setting was also read three different ways. Post-processing split on
commas only. slskd split on commas and spaces and stripped a leading
dot. IRC split on commas and kept the dot. Prowlarr split on commas and
kept `""` as a format when the value was an empty string. So `.epub` or
`epub mobi` worked in one source and not another.
This puts `DEFAULT_EBOOK_FORMATS` and `normalize_format_list` in
`core/utils.py` next to `AUDIOBOOK_FORMATS`, and the registry,
post-processing, IRC, Prowlarr and slskd all use them. I took the most
lenient reading of the three (comma or space split, leading dot dropped,
empties ignored, default when nothing is left), so no existing config
gets stricter. I put the reader in `core/utils.py` and not in
`policy.py` because the release sources can't import `policy` without
pulling in the whole post-processing package.
A config with a sensible list reads exactly as before. The only
differences are for malformed values: an emptied setting or an empty
string now falls back to the default in every place, where IRC used to
filter everything out and Prowlarr treated `""` as a format.
The audiobook default is a separate story and I left it alone: the
registry and slskd default to every processable audiobook format plus
the archives, while post-processing and IRC fall back to `m4b, mp3`.
They only differ when the setting is unset or emptied, but they do
differ, and I'd rather ask which one you want than pick one here.
Ruff, basedpyright on the changed files and the backend suite pass (4400
passed, e2e not run). New tests cover the reader and the two fallbacks.
Closes#1130.
Per-user notification route URLs can now use `{Email}` and `{User}`,
filled in from each user's account at send time, the same way `{User}`
already works in destination paths. One default route like
`mailtos://user:pass@smtp.example.com?to={Email}` now reaches every user
at their own address, with no per-user setup.
- Values are URL-encoded (`alice+books@example.com` keeps its `+`);
placeholders are case-insensitive.
- A route that needs a value the user doesn't have (e.g. no email on
file) is skipped for that user; other routes still send.
- The user lookup only runs when a route contains a placeholder.
- The personal "Test notification" button expands the placeholders too,
so testing a `{Email}` route works.
- Added to the routes setting help, `docs/users-and-requests.md`, and
tests.
@calibrain ready for review.
---------
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
Bumps [publicsuffixlist](https://github.com/ko-zu/psl) from
1.1.0.20261017 to 1.1.0.20261018.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/ko-zu/psl/commits">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
jules-pr-reviewer v0.0.2 reviewed only before...head on synchronize.
After
"Update branch" merged main into fork PRs, that range was main's
commits, so
of their own changes. v0.0.3 of the calibrain fork always reviews the
PR's
full diff. Also correct the fork workflow comment: jules-forks has no
required reviewers by design, so runs don't wait for approval.
Fixes#1524
Open Library metadata searches and Prowlarr API calls used
`get_ssl_verify` but never `get_proxies`, so the HTTP proxy set in
Settings → Network was not applied to their HTTPS requests. Both now
pass the configured proxies. Relying on `requests`' `trust_env` with
only `HTTP_PROXY` set leaves the same gap for `https://` destinations.
Verification (Linux, `uv run --frozen pytest`):
- New tests `tests/metadata/test_openlibrary_proxy.py` and
`tests/prowlarr/test_api_proxy.py`: 4 passed. With the source change
reverted they fail (2 failed, 2 errors).
- `tests/metadata` + `tests/prowlarr`: 949 passed, 42 skipped.
- The full suite in my environment showed 61 failures and 26 errors
outside these areas. I did not compare them against `main`, so I'm not
claiming the full suite is green here; CI will show it.
- `ruff check` and `ruff format --check` on the changed files pass.
Prepared with AI assistance; reviewed and tested by me before
submitting.
Co-authored-by: dbdefault <246079841+dbdefault@users.noreply.github.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
Fork PRs get no secrets under pull_request and Dependabot runs get only
Dependabot secrets, so JULES_API_KEY was empty and every such run failed
at
config. The same-repo workflow now skips them, and a new
pull_request_target
workflow reviews fork PRs behind maintainer approval on the jules-forks
environment, without checking out PR code. Both use calibrain's fork of
jules-pr-reviewer, since upstream refuses pull_request_target.
A reverse proxy whose read timeout is shorter than the release search
budget
(nginx defaults to 60s) answers a slow search with its own 504 page.
describeSearchFailure found no server message there and fell back to
"Unable to reach download source. Network may be restricted or mirrors
blocked.", which sent users off changing mirrors (discussion #1520).
- 504 and Cloudflare's 524 now say the proxy stopped waiting and that
its
read timeout should be at least the release search timeout.
- A 502/503 with a non-JSON body keeps fetchJSON's "Server unavailable"
wording instead of the mirrors line.
- docs: the root-path nginx example sets proxy_read_timeout and
proxy_send_timeout like the subpath ones, and a new troubleshooting
section covers searches that time out behind a proxy.
This PR adds the Advanced Jules PR Reviewer workflow to automatically
review PRs. It is configured to run on forks automatically as requested
by setting `skip_forks: false`. JULES_API_KEY and GITHUB_TOKEN are
mapped correctly as per documentation.
---
*PR created automatically by Jules for task
[10601196597817548410](https://jules.google.com/task/10601196597817548410)
started by @calibrain*
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: calibrain <191882101+calibrain@users.noreply.github.com>
2026-10-21 14:25:27 -04:00
303 changed files with 36477 additions and 5229 deletions
Set `SHELFMARK_API_KEY_READONLY` to a second secret for dashboards and monitors. It can read one endpoint,
`GET /api/stats`, and nothing else. Writes and every other path return 403, and the key never gets a session.
The admin key reaches `/api/stats` too.
The admin key reaches `/api/stats` too, and so does a browser session signed in as an admin. A refused request gets `{"error": "This API key is read-only"}`.
- Point `/books` to your library ingest folder (Calibre-Web, Booklore, Audiobookshelf, etc) for automatic import.
- If you set Books Output Mode to Booklore (API), books are uploaded via API instead of written to `/books`. Audiobooks still use a destination folder.
- Point `/books` to your library ingest folder (Calibre-Web, Grimmory, Audiobookshelf, etc) for automatic import.
- If you set Output Mode to Grimmory (API), books are uploaded via API instead of written to `/books`. Audiobooks still use a destination folder.
- Ensure `PUID`/`PGID` (or legacy `UID`/`GID`) match the owner of the host directories.
- For non-root mode, start the container as `1000:1000`.
- On Kubernetes, set `runAsUser: 1000`, `runAsGroup: 1000`, and `runAsNonRoot: true` together.
@@ -95,13 +95,16 @@ If the files are copied or synced into Shelfmark on a delay, increase **Complete
in Settings -> Advanced. The default is 60 seconds; seedbox or remote-sync setups may need a value
longer than the sync interval.
With qBittorrent, Shelfmark also reads the torrent's own file list and imports only once every file
the torrent downloads is there at its full size, so a sync that copies files in batches and pauses
between them does not cause a partial import. The same wait applies, and files the torrent skips
are not counted.
## File Processing Options
### Transfer Method (Torrent / Usenet Only)
### Hardlink Torrents (Torrent / Usenet Only)
Available methods:
- Copy (default). Works everywhere.
- Hardlink. Preserves seeding without duplicating files.
Imported files are copied by default, which works everywhere. Turn on **Hardlink Book Torrents** (and **Hardlink Audiobook Torrents** for audiobooks) to hardlink instead, which preserves seeding without duplicating files.
Hardlink requirements and behavior:
- Source and destination must be on the same filesystem.
Shelfmark can run an executable you provide after a download task completes successfully. The script runs after the selected output has finished (for example: transfer to the folder destination, or upload to Booklore).
Shelfmark can run an executable you provide after a download task completes successfully. The script runs after the selected output has finished (for example: transfer to the folder destination, upload to Grimmory, or send by email).
## Quick Start (Recommended)
@@ -71,7 +71,8 @@ Shelfmark chooses a "best single path" for the task:
What the target path refers to depends on the output mode:
- Folder output (`output.mode=folder`, `phase=post_transfer`): the final imported file or folder inside your destination.
-Booklore output (`output.mode=booklore`, `phase=post_upload`): the local file or folder that was uploaded (the destination is remote).
-Grimmory output (`output.mode=booklore`, `phase=post_upload`): the local file or folder that was uploaded (the destination is remote).
- Email output (`output.mode=email`, `phase=post_email`): the file that was sent, or the folder holding the files when there were several.
By default, `$1` is an absolute path inside the Shelfmark container (or on your host, if you are not using Docker).
@@ -79,12 +80,12 @@ By default, `$1` is an absolute path inside the Shelfmark container (or on your
When enabled, Shelfmark sends a versioned JSON payload to your script via stdin (and still passes `$1`). This is the recommended way to write robust scripts, especially for multi-file imports (audiobooks) and output-specific context (like Booklore).
When enabled, Shelfmark sends a versioned JSON payload to your script via stdin (and still passes `$1`). This is the recommended way to write robust scripts, especially for multi-file imports (audiobooks) and output-specific context (like Grimmory).
- The JSON payload always includes absolute paths in `paths.*`, even if you set Custom Script Path Mode to `relative` for `$1`.
-`output.mode` tells you which output ran.
-`output.details` is output-specific. For Booklore output, `output.details.booklore` includes connection details such as `base_url`, `library_id`, and `path_id`.
-`phase` indicates when the script is running. Current values: `post_transfer` (folder output), `post_upload` (Booklore output).
-`output.details` is output-specific. For Grimmory output (the mode value is still `booklore`), `output.details.booklore` includes connection details such as `base_url`, `library_id`, and `path_id`. For email output, `output.details.email` has `to`, `label`, `host`, `port`, and `security`.
-`phase` indicates when the script is running. Current values: `post_transfer` (folder output), `post_upload` (Grimmory output), `post_email` (email output).
-`transfer` is only included for outputs that do a local transfer (for example the folder output).
If JSON payload is disabled, stdin is empty (EOF). Don't `cat` stdin unless you've enabled the payload.
@@ -171,7 +172,7 @@ This setting controls what gets passed as `$1`:
-`absolute` (default): pass an absolute path.
-`relative`: pass a path relative to the output's "destination root", and run the script with `$PWD` set to that root.
For folder output, the destination root is your configured destination folder. For Booklore output, it's the local upload folder.
For folder output, the destination root is your configured destination folder. For Grimmory output, it's the local upload folder. For email output, it's the folder holding the files that were sent.
Example (folder destination is `/data/library/books`, and the imported file ended up in `Isaac Asimov/Foundation/Foundation.epub`):
@@ -192,4 +193,5 @@ Note: if the target is the destination folder itself, `relative` mode may pass `
## Notes And Caveats
- **Hardlinks and torrents:** if you use hardlinking to keep seeding, avoid scripts that modify file contents, since hardlinked files share data with the seeding copy.
- **Booklore output mode:** scripts run after upload. `$1` will point at the local uploaded file (or staging folder).
- **Grimmory output mode:** scripts run after upload. `$1` will point at the local uploaded file (or staging folder).
- **Email output mode:** scripts run after the message is sent. `$1` will point at the file that was sent (or the folder holding the files).
`SelectField` also takes `filter_by_field`, the key of another field whose value filters this field's options through each option's `childOf` property. `options` can be a list or a function that returns one, evaluated when the schema is built.
### MultiSelectField
Multi-choice selection from a list of options.
Multi-choice selection from a list of options.`variant` is `"pills"` (default) or `"dropdown"` for a checkbox dropdown.
```python
MultiSelectField(
@@ -156,6 +158,56 @@ MultiSelectField(
)
```
### TagListField
Editable list of free-form strings, shown as tags. The value is a list of strings.
```python
TagListField(
key="MY_PLUGIN_MIRROR_URLS",
label="Mirror URLs",
description="Type a URL and press Enter to add it",
placeholder="https://mirror.example",
normalize_urls=True,# default: entries are cleaned up as URLs
)
```
### OrderableListField
A list of options that can be toggled on and off and reordered. The value is a list of `{"id": ..., "enabled": ...}` objects in priority order.
```python
OrderableListField(
key="MY_PLUGIN_SOURCE_PRIORITY",
label="Source priority",
options=[
{"id":"a","label":"Source A"},
{"id":"b","label":"Source B","isLocked":True,"disabledReason":"Needs an API key"},
A callback that declares a `current_values` parameter receives the form's current, unsaved values as a dict, so a "Test connection" button can test what the user just typed instead of what was last saved:
-`direct_download` (order=20): For download-related settings
-`metadata_providers` (order=50): For metadata provider plugins
-`metadata_providers` (order=12): For metadata provider plugins
## Value Resolution Priority
Settings values are resolved in this order (highest priority first):
1.**Config File** - Stored in `CONFIG_DIR/plugins/<tab_name>.json`
2.**Field Default** - Value specified in the field definition
1.**Environment variable** - The field's key, or `env_var` when set. Skipped for a field with `env_supported=False`. A value that comes from an environment variable is shown locked in the UI
2.**Config File** - Stored in `CONFIG_DIR/plugins/<tab_name>.json`
3.**Field Default** - Value specified in the field definition
The `general` tab uses`CONFIG_DIR/settings.json` instead of the plugins subdirectory.
The `general` and `search_mode` tabs use `CONFIG_DIR/settings.json` instead of the plugins subdirectory.
## Validating or Transforming on Save
`register_on_save(tab_name, handler)` runs a function each time the tab is saved. The handler receives a copy of the values being saved and returns a dict. Return `{"error": True, "message": "..."}` to refuse the save and show the message, or `{"values": {...}}` to save changed values.
All four methods above are abstract, so a provider that leaves one out cannot be instantiated. The provider module also has to be imported once so its decorators run: add it to the list of imports at the bottom of `shelfmark/metadata_providers/__init__.py`.
## Complete Example: Release Source
Here's a complete example for a release source plugin:
@@ -457,11 +556,11 @@ Here's a complete example for a release source plugin:
```python
# shelfmark/release_sources/my_source.py
fromshelfmark.release_sources.baseimport(
ReleaseSource,
fromshelfmark.release_sourcesimport(
DownloadHandler,
register_source,
ReleaseSource,
register_handler,
register_source,
)
fromshelfmark.core.settings_registryimport(
register_settings,
@@ -559,22 +658,25 @@ class MySource(ReleaseSource):
The built-in sources are packages (`jackett/`, `webshare/`, `slskd/` and so on) with the source, handler and settings in separate modules that the package `__init__.py` imports. A single module is fine for a small source.
---
## ReleaseSource Interface
@@ -139,7 +153,8 @@ The `ReleaseSource` abstract base class defines the search interface:
can_be_default:bool=True# Whether it can be picked as the default source in settings
@abstractmethod
defsearch(self,book:BookMetadata)->List[Release]:
defsearch(
self,
book:BookMetadata,
plan:ReleaseSearchPlan,
*,
expand_search:bool=False,
content_type:str="ebook",
)->List[Release]:
"""Search for releases of a book.
Args:
book: Book metadata including title, authors, ISBN, language
plan: Pre-computed search inputs shared by all sources (see below)
expand_search: True when the user asked for a wider search
content_type: "ebook" or "audiobook"
Returns:
List of Release objects found from this source
"""
pass
@abstractmethod
defis_available(self)->bool:
@@ -169,10 +195,8 @@ class ReleaseSource(ABC):
Returns:
True if source can be used for searching
"""
pass
@classmethod
defget_column_config(cls)->ReleaseColumnConfig:
defget_column_config(self)->ReleaseColumnConfig:
"""Get the column configuration for this source's release list UI.
Override to customize how releases are displayed in the modal.
@@ -181,17 +205,39 @@ class ReleaseSource(ABC):
return_default_column_config()
```
Optional hooks, all with working defaults:
| Method | Purpose |
|--------|---------|
| `search_results_are_releases()` | Return `True` when the source's browse results are already concrete releases. Request policy then treats "request book" as "request release" for it |
| `get_record(record_id, *, fetch_download_count=True)` | Resolve a source-native record for browse flows. Raises `NotImplementedError` by default |
| `get_destination_override(task)` | Return a source-specific destination `Path` for a queued download, or `None` |
### The search plan
`plan` is a `ReleaseSearchPlan` (`shelfmark/core/search_plan.py`) that Shelfmark builds once per search, so each source does not redo the same work:
| Field | Description |
|-------|-------------|
| `languages` | Language codes to filter by, or `None` |
| `isbn_candidates` | ISBNs to try, best first |
| `author` | Author string to search with |
| `title_variants`, `grouped_title_variants` | Title queries, best first. `plan.primary_query` is the first one |
| `manual_query` | Text the user typed into the search box, when they did |
| `indexers` | Indexer names for indexer-based sources |
| `source_filters` | Filters from the source's own search form |
### Search Method
The `search` method receives complete book metadata and returns standardized releases:
The `search` method receives the book metadata and the search plan, and returns standardized releases:
Prowlarr parses the `.torrent` it already fetches (magnet-only releases return
`None`), and AudiobookBay reads the file table off its detail page.
### Optional: Other Handler Hooks
| Method | Purpose |
|--------|---------|
| `post_process_cleanup(task, *, success)` | Runs after Shelfmark has post-processed the download. External clients use it to clean up on their side only once the files are safely imported |
| `build_retry_resolution_fields(release_data)` | Return private queue-time fields the handler needs to retry after a restart |
| `list_authors(release_data)` | Return authors read from a source page, for sources whose search results can leave the author out. `None` when there is nothing more to say |
| `seed_complaints(release_data)` | How many comments on the release's page say nobody is seeding it, for sources with no seeder count of their own. `None` when there is no such signal |
### Download Method Parameters
| Parameter | Type | Description |
@@ -318,12 +376,13 @@ Prowlarr parses the `.torrent` it already fetches (magnet-only releases return
|--------|-------------|-------------|
| `"queued"` | Waiting in queue | Set by orchestrator, rarely needed in handler |
| `"locating"` | Waiting for the finished files to show up | Set by the handler with `status_callback` once an external client or peer reports done, as the torrent, usenet and Soulseek handlers do |
| `"downloading"` | Active download | File transfer in progress |
| `"complete"` | Successfully finished | Set by orchestrator when handler returns a file path |
| `"error"` | Failed | Any unrecoverable error (handler should set this) |
| `"cancelled"` | User cancelled | Set by orchestrator when `cancel_flag` is triggered |
**Note**: The orchestrator automatically sets `"complete"` when your handler returns a valid file path, and `"cancelled"` when cancellation is detected. Your handler should primarily use `"resolving"`, `"downloading"`, and `"error"`.
**Note**: The orchestrator automatically sets `"complete"` when your handler returns a valid file path, and `"cancelled"` when cancellation is detected. Your handler should primarily use `"resolving"`, `"downloading"`, and `"error"`, plus `"locating"` if it waits on an external client.
**Critical**: Your plugin must be imported at module load time. Add to `release_sources/__init__.py`:
**Critical**: Your plugin must be imported for its decorators to run. Built-in sources are listed in `_BUILTIN_SOURCE_MODULES` in `release_sources/__init__.py`, and `_ensure_builtin_sources_registered()` imports each one the first time a source or handler is looked up:
"shelfmark.release_sources.my_source",# add your module here
)
```
The `noqa` comments suppress linter warnings about unused imports.
Use the dotted path of the module (or package) that imports your source, handler and settings.
---
## Settings Integration
Register plugin settings using the settings registry decorator:
Register plugin settings using the settings registry decorator. The decorator calls your function as soon as the module is imported, so define anything the fields refer to, such as a button callback, above it:
```python
fromshelfmark.core.settings_registryimport(
@@ -551,12 +655,39 @@ from shelfmark.core.settings_registry import (
return{"success":False,"message":f"API returned {response.status_code}"}
exceptExceptionase:
return{"success":False,"message":str(e)}
```
A source's settings tab is normally top level, as Jackett and Webshare.cz are. The optional `group=` argument files a tab under a sidebar group registered with `register_group()`. The only groups today are `direct_download` and `metadata_providers`.
### Field Types Reference
| Field Type | Purpose | Key Properties |
@@ -688,8 +798,7 @@ from shelfmark.release_sources import (
| `hide_mobile` | `bool` | Hide on small screens |
| `color_hint` | `ColumnColorHint` | For `BADGE` type coloring |
| `fallback` | `str` | Shown when data missing (default: `"-"`) |
| `uppercase` | `bool` | Force uppercase text |
| `sortable` | `bool` | Offer this column in the sort dropdown (opt-in) |
| `sort_key` | `str` | Field to sort by, defaults to `key` |
| `setting_key` | `str` | Name of a boolean setting that shows or hides the column |
| `content_types` | `tuple[str, ...]` | Only show the column for these content types (`"ebook"`, `"audiobook"`) |
`ReleaseColumnConfig` takes more than `columns`, `grid_template` and `leading_cell`:
| Property | Description |
|----------|-------------|
| `supported_filters` | Which filters the source supports: `["format", "language", "indexer"]` |
| `extra_sort_options` | `SortOption(label, sort_key, default_direction)` entries for sorts not tied to a column |
| `action_button` | `SourceActionButton(label, action)` that replaces the default "expand search" button |
| `cache_ttl_seconds` | How long results are cached (default 5 minutes) |
| `online_servers` | IRC: nicks of the servers currently online |
| `available_indexers`, `default_indexers` | Indexer sources: every enabled indexer, and the ones selected in settings |
### Color Hints
@@ -841,16 +965,20 @@ error → "All download sources failed"
### In Search Methods
Return empty list on errors, log for debugging:
Return an empty list when the search simply found nothing or failed in a way the user cannot act on, and log for debugging. When the source is configured but unreachable, raise `SourceUnavailableError` with a message written for the user, so the UI can say why there are no results instead of showing an empty list:
New API routes go in their own module under `shelfmark/core/`, named for the group (`stats_routes.py`, `settings_routes.py`), as a `register_<group>_routes(app, ...)` function that `main.py` calls once. Anything the routes need that lives in `main.py` or is shared with another group, such as `login_required`, the auth mode or the user database, is passed in as a parameter, so a route module never imports `main`. When a test patches one of those, pass a callable that is evaluated per request, the way `register_stats_routes` does. Helpers that belong to one group stay in its module. Please don't add `@app.route` handlers to `main.py` itself: two PRs that each add a route there land in the same stretch of the file and conflict, where two route modules only share the one register call. `shelfmark/core/stats_routes.py` is the smallest example to copy. A few older routes are still in `main.py` and are being moved out one group at a time, see [Discussion #1536](https://github.com/calibrain/shelfmark/discussions/1536).
@@ -52,8 +56,9 @@ These environment variables are used at startup before the settings system loads
| `HIDE_LOCAL_AUTH` | Hide the username/password login form when OIDC is active. | boolean | `false` |
| `DISABLE_LOCAL_AUTH` | Disable username/password login and remove the local-admin prerequisite for OIDC. Implies HIDE_LOCAL_AUTH; with AUTH_METHOD=builtin, everyone is locked out until auth env vars are changed. | boolean | `false` |
| `SHELFMARK_API_KEY` | Optional static API key. When set, requests carrying it as 'Authorization: Bearer <key>' (or X-Api-Key) are authenticated as an admin; browser sessions keep working. Unset = off. | string | `unset` |
| `SHELFMARK_API_KEY_READONLY` | Optional read-only API key. Can only GET `/api/stats`; never an admin. Unset = off. | string | `unset` |
| `SHELFMARK_API_KEY_READONLY` | Optional second API key for dashboards and monitors. Send it like SHELFMARK_API_KEY. It can only GET /api/stats (counters only, no titles or user names). Every other request, and any write, is refused with 403, and it never gets a session. Unset = off. | string | `unset` |
| `OIDC_AUTO_REDIRECT` | Automatically redirect to the OIDC provider instead of showing the login page. | boolean | `false` |
| `UMASK` | File creation mask for the app process, read by entrypoint.sh. The default 0022 gives files 644 and directories 755. | string (octal) | `0022` |
| `DOCKERMODE` | Indicates the application is running inside a Docker container. | boolean | `false` |
| `ONBOARDING` | Show the onboarding wizard on first run. Set to false to skip (useful for ephemeral storage). | boolean | `true` |
@@ -146,7 +151,7 @@ Optional static API key. When set, requests carrying it as 'Authorization: Beare
#### `SHELFMARK_API_KEY_READONLY`
Optional second API key for dashboards. Send it like `SHELFMARK_API_KEY`. It can only `GET /api/stats` (counters only, no titles or user names). Every other request, and any write, is refused with 403, and it never gets a session. Unset = off.
Optional second API key for dashboards and monitors. Send it like SHELFMARK_API_KEY. It can only GET /api/stats (counters only, no titles or user names). Every other request, and any write, is refused with 403, and it never gets a session. Unset = off.
- **Type:** string
- **Default:** `unset`
@@ -158,6 +163,13 @@ Automatically redirect to the OIDC provider instead of showing the login page.
- **Type:** boolean
- **Default:** `false`
#### `UMASK`
File creation mask for the app process, read by entrypoint.sh. The default 0022 gives files 644 and directories 755.
- **Type:** string (octal)
- **Default:** `0022`
#### `DOCKERMODE`
Indicates the application is running inside a Docker container.
@@ -176,7 +188,7 @@ Show the onboarding wizard on first run. Set to false to skip (useful for epheme
## Egress / VPN Routing
These startup-only variables are consumed by `entrypoint.sh` / `wireguard.sh` to select and configure the WireGuard transparent-egress kill-switch. `USING_WIREGUARD` and [`USING_TOR`](#using_tor) (documented under Network) are mutually exclusive; both require root startup.
These startup-only variables are consumed by `entrypoint.sh` / `wireguard.sh` to select and configure the WireGuard transparent-egress kill-switch. `USING_WIREGUARD` and [`USING_TOR`](#using_tor) (documented under Network) are mutually exclusive; both require root startup.`TOR_CHECK_INTERVAL` and `TOR_MAX_FAILURES` tune the Tor watchdog in `tor.sh`.
| Variable | Description | Type | Default |
|----------|-------------|------|---------|
@@ -190,6 +202,8 @@ These startup-only variables are consumed by `entrypoint.sh` / `wireguard.sh` to
| `WIREGUARD_ALLOW_IPV6_LEAK` | Escape hatch: continue startup even when an IPv6 kill-switch cannot be installed AND IPv6 cannot be disabled. Only set when the container has no IPv6 connectivity, as IPv6 egress may otherwise bypass the tunnel. | boolean | `false` |
| `WIREGUARD_ALLOW_WEBUI_OFFTUNNEL` | When false (default) the kill-switch is strictly fail-closed: the only off-tunnel egress permitted is loopback, the tunnel device and the LAN allowlist. Set true only if a NON-LAN client (e.g. a public reverse proxy on a different segment) must reach the WebUI; it permits app-server REPLY packets (--sport FLASK_PORT, conntrack REPLY) to leave off-tunnel. Server replies only, never client-initiated egress, so it cannot leak outbound browsing/downloads or the real IP for outbound requests, but it is still an off-tunnel path while the tunnel is down, hence opt-in. LAN WebUI clients never need it (covered by LAN_NETWORK). | boolean | `false` |
| `WIREGUARD_STALE_AFTER` | Seconds since the last WireGuard handshake before the healthcheck bounces the tunnel. | number | `180` |
| `TOR_CHECK_INTERVAL` | Seconds between Tor watchdog probes. The watchdog resets the container when Tor stops being usable, so the restart policy rebuilds the Tor setup. | integer (seconds) | `30` |
| `TOR_MAX_FAILURES` | Consecutive failed Tor watchdog probes before the container is reset. 2 tolerates one failed probe. | integer | `2` |
<details>
<summary>Detailed descriptions</summary>
@@ -264,6 +278,20 @@ Seconds since the last WireGuard handshake before the healthcheck bounces the tu
- **Type:** number
- **Default:** `180`
#### `TOR_CHECK_INTERVAL`
Seconds between Tor watchdog probes. The watchdog resets the container when Tor stops being usable, so the restart policy rebuilds the Tor setup.
- **Type:** integer (seconds)
- **Default:** `30`
#### `TOR_MAX_FAILURES`
Consecutive failed Tor watchdog probes before the container is reset. 2 tolerates one failed probe.
- **Type:** integer
- **Default:** `2`
</details>
## General
@@ -275,6 +303,10 @@ Seconds since the last WireGuard handshake before the healthcheck bounces the tu
| `AUDIOBOOK_LIBRARY_URL` | Adds a separate navigation button for your audiobook library (Audiobookshelf, Plex, etc). When both URLs are set, icons are shown instead of text. | string | _none_ |
| `LIBRARY_CHECK_CALIBRE_ENABLED` | Read the Calibre metadata.db and mark search results you already own, so you do not download a second copy. Read only, nothing is written to the library. | boolean | `false` |
| `CALIBRE_LIBRARY_DB_PATH` | Path to metadata.db as seen from inside the Shelfmark container. Mount the Calibre library folder read-only, e.g. /path/to/calibre-library:/calibre-library:ro. | string | `/calibre-library/metadata.db` |
| `LIBRARY_CHECK_AUDIOBOOKSHELF_ENABLED` | Ask your Audiobookshelf server which audiobooks you already own, so synced and automatic downloads skip them. Read only, nothing is written to Audiobookshelf. | boolean | `false` |
| `AUDIOBOOKSHELF_URL` | Base URL reachable from the Shelfmark container (not the public URL). Often the host LAN IP, e.g. http://10.0.0.91:13378. | string | _none_ |
| `AUDIOBOOKSHELF_TOKEN` | API token from Audiobookshelf (Settings > Users > your user > API Token). | string (secret) | _none_ |
| `AUDIOBOOKSHELF_LIBRARY_IDS` | Comma-separated library IDs to check. Leave blank for all book libraries. | string | _none_ |
| `SUPPORTED_FORMATS` | Book formats to include in search results. ZIP/RAR archives are extracted automatically and book files are used if found. | string (comma-separated) | `epub,mobi,azw3,fb2,djvu,cbz,cbr` |
| `SUPPORTED_AUDIOBOOK_FORMATS` | Audiobook formats to include in search results. ZIP/RAR archives are extracted automatically and audiobook files are used if found. | string (comma-separated) | `m4b,mp3,m4a,mp4,flac,ogg,wma,aac,wav,opus,zip,rar` |
@@ -326,6 +358,42 @@ Path to metadata.db as seen from inside the Shelfmark container. Mount the Calib
- **Type:** string
- **Default:** `/calibre-library/metadata.db`
#### `LIBRARY_CHECK_AUDIOBOOKSHELF_ENABLED`
**Mark audiobooks already in your Audiobookshelf library**
Ask your Audiobookshelf server which audiobooks you already own, so synced and automatic downloads skip them. Read only, nothing is written to Audiobookshelf.
- **Type:** boolean
- **Default:** `false`
#### `AUDIOBOOKSHELF_URL`
**Audiobookshelf URL**
Base URL reachable from the Shelfmark container (not the public URL). Often the host LAN IP, e.g. http://10.0.0.91:13378.
- **Type:** string
- **Default:** _none_
#### `AUDIOBOOKSHELF_TOKEN`
**Audiobookshelf API Token**
API token from Audiobookshelf (Settings > Users > your user > API Token).
- **Type:** string (secret)
- **Default:** _none_
#### `AUDIOBOOKSHELF_LIBRARY_IDS`
**Audiobookshelf Library IDs (optional)**
Comma-separated library IDs to check. Leave blank for all book libraries.
- **Type:** string
- **Default:** _none_
#### `SUPPORTED_FORMATS`
**Supported Book Formats**
@@ -549,8 +617,8 @@ Audiobooks only. AudiobookBay lists it for most releases. In Prowlarr results it
| `HARDLINK_TORRENTS_AUDIOBOOK` | Create hardlinks instead of copying. Preserves seeding but archives won't be extracted. Don't use if destination is a library ingest folder. | boolean | `true` |
| `AUTO_OPEN_DOWNLOADS_SIDEBAR` | Automatically open the downloads sidebar when a new download is queued. | boolean | `false` |
| `DOWNLOAD_TO_BROWSER_CONTENT_TYPES` | Automatically download completed files to your browser for the selected content types. | string (comma-separated) | _empty list_ |
| `MAX_CONCURRENT_DOWNLOADS` | Maximum number of simultaneous downloads. | number | `3` |
| `MAX_CONCURRENT_DOWNLOADS_PER_SOURCE` | Simultaneous downloads allowed from each source (Soulseek, AudiobookBay, Direct Download and so on). A source at its limit does not hold the others back, so a Soulseek download waiting in a peer's queue cannot stop a torrent from starting. 0 uses the single limit above for all sources together. | number | `0` |
| `MAX_CONCURRENT_DOWNLOADS` | Maximum number of simultaneous downloads. Not used while Max Concurrent Downloads Per Source is above 0. | number | `3` |
| `MAX_CONCURRENT_DOWNLOADS_PER_SOURCE` | Simultaneous downloads allowed from each source (Soulseek, AudiobookBay, Direct Download and so on). A source at its limit does not hold the others back, so a Soulseek download waiting in a peer's queue cannot stop a torrent from starting. Above 0 this replaces Max Concurrent Downloads: the total can be this number for every source that has downloads at once. 0 uses the single limit above for all sources together. | number | `0` |
| `STATUS_TIMEOUT` | How long to keep completed/failed downloads in the queue display. | number | `3600` |
<details>
@@ -854,7 +922,7 @@ Automatically download completed files to your browser for the selected content
**Max Concurrent Downloads**
Maximum number of simultaneous downloads.
Maximum number of simultaneous downloads. Not used while Max Concurrent Downloads Per Source is above 0.
- **Type:** number
- **Default:** `3`
@@ -865,7 +933,7 @@ Maximum number of simultaneous downloads.
**Max Concurrent Downloads Per Source**
Simultaneous downloads allowed from each source (Soulseek, AudiobookBay, Direct Download and so on). A source at its limit does not hold the others back, so a Soulseek download waiting in a peer's queue cannot stop a torrent from starting. 0 uses the single limit above for all sources together.
Simultaneous downloads allowed from each source (Soulseek, AudiobookBay, Direct Download and so on). A source at its limit does not hold the others back, so a Soulseek download waiting in a peer's queue cannot stop a torrent from starting. Above 0 this replaces Max Concurrent Downloads: the total can be this number for every source that has downloads at once. 0 uses the single limit above for all sources together.
- **Type:** number
- **Default:** `0`
@@ -1049,6 +1117,197 @@ Custom label for the OIDC sign-in button on the login page.
</details>
## Hardcover Sync
| Variable | Description | Type | Default |
|----------|-------------|------|---------|
| `HARDCOVER_SYNC_ENABLED` | Periodically sync the selected Hardcover shelves into requests. | boolean | `false` |
| `HARDCOVER_SYNC_TOKEN` | Bearer token for your Hardcover account. Leave blank to reuse the token from the Hardcover metadata provider. | string (secret) | _none_ |
| `HARDCOVER_SYNC_STATUSES` | Which Hardcover reading shelf to pull from. | string (choice) | `1` |
| `HARDCOVER_SYNC_CONTENT_TYPE` | Content type for synced requests. "Ebooks and audiobooks" creates one request of each per shelf book; every request is checked against its own library and downloaded with its own format guard and source priority. | string (choice) | `audiobook` |
| `HARDCOVER_SYNC_INTERVAL` | How often the scheduled sync runs (minimum 1 minute). | number | `6` |
| `HARDCOVER_SYNC_INTERVAL_UNIT` | Whether the sync interval is measured in minutes or hours. | string (choice) | `hours` |
| `AUTO_DOWNLOAD_ENABLED` | Auto-approve and auto-download matching releases for Hardcover requests. Another user's request is only approved this way where their request policy already lets them download directly; the rest wait for an admin. | boolean | `false` |
| `AUTO_DOWNLOAD_SOURCE_PRIORITY` | Drag to set which release sources are tried first for audiobook requests. The first source with a strict match wins. Disabled sources are skipped. | JSON array | _empty list_ |
| `AUTO_DOWNLOAD_EBOOK_SOURCE_PRIORITY` | Drag to set which release sources are tried first for ebook requests. The first source with a strict match wins. Disabled sources are skipped. | JSON array | _empty list_ |
| `AUTO_DOWNLOAD_MIN_SEEDERS` | Skip torrent releases with fewer seeders than this. | number | `1` |
| `AUTO_DOWNLOAD_RETRY_DAYS` | A synced request whose download failed, or was interrupted (a restart, say), goes back to pending after this many days and is tried again, a few per pass. A release that failed is not picked again, and a request is given up on after three failed retries. A download the stall timer cancelled counts as failed; one cancelled by a person, and rejected requests, are never retried. 0 turns retries off. | number | `7` |
| `AUTO_DOWNLOAD_FAILURE_RETRY_MINUTES` | A download that failed, or that was given up on (stalled, no seeders), goes back to pending this many minutes later and another release is chosen, without waiting for the retry days above or the next sync cycle. The release that failed is not picked again, and a request is given up on after three failed attempts. 0 turns this off. | number | `0` |
| `AUTO_DOWNLOAD_RETRY_FAILED_RELEASE_DAYS` | A release that failed or stalled is kept out of the search for this many days, then it may be picked again. A torrent that stalled can be revived by new seeds, so a failure should not shut a book out for good. A request is still given up on after three failed attempts. | number | `7` |
| `AUTO_DOWNLOAD_BACKOFF_MAX_DAYS` | A request that no source has a match for is searched every cycle at first. After three passes in a row with no match it is searched once a day, then every two days, and at most this often apart. A match, or Sync now, starts it over. 0 searches every unmatched request every cycle. | number | `3` |
| `AUTO_DOWNLOAD_EXISTENCE_CHECK_ENABLED` | An audiobook request that stays unmatched is looked up on Audible (US and UK) through your Audiobookshelf server. Either way it stays pending and its reason says what the store answered (with the ASIN when listed), for you to decline or keep; it is checked again after a week. A lookup that fails decides nothing. Needs the Audiobookshelf URL and token below. | boolean | `false` |
| `AUTO_DOWNLOAD_EXISTENCE_CHECK_AFTER_PASSES` | How many consecutive passes with no strict match a request goes through before the store is asked. A pass that matched, or a reopen, starts the count over. | number | `3` |
| `AUTO_DOWNLOAD_AUDIOBOOK_MIN_SIZE_PERCENT` | A copy of an audiobook smaller than this share of the largest copy found for the same book is probably a lower-bitrate copy. It is tried after the others, and a single m4b that small is not preferred over a fuller set of chapter files. 0 turns the check off. | number | `67` |
<details>
<summary>Detailed descriptions</summary>
#### `HARDCOVER_SYNC_ENABLED`
**Enable scheduled sync**
Periodically sync the selected Hardcover shelves into requests.
- **Type:** boolean
- **Default:** `false`
#### `HARDCOVER_SYNC_TOKEN`
**Hardcover API Token**
Bearer token for your Hardcover account. Leave blank to reuse the token from the Hardcover metadata provider.
- **Type:** string (secret)
- **Default:** _none_
#### `HARDCOVER_SYNC_STATUSES`
**Shelves to sync**
Which Hardcover reading shelf to pull from.
- **Type:** string (choice)
- **Default:** `1`
- **Options:** `1` (Want to Read), `2` (Currently Reading), `3` (Read), `5` (Did Not Finish)
#### `HARDCOVER_SYNC_CONTENT_TYPE`
**Request as**
Content type for synced requests. "Ebooks and audiobooks" creates one request of each per shelf book; every request is checked against its own library and downloaded with its own format guard and source priority.
- **Type:** string (choice)
- **Default:** `audiobook`
- **Options:** `audiobook` (Audiobooks), `ebook` (Ebooks), `both` (Ebooks and audiobooks)
#### `HARDCOVER_SYNC_INTERVAL`
**Sync interval**
How often the scheduled sync runs (minimum 1 minute).
- **Type:** number
- **Default:** `6`
- **Constraints:** min: 1, max: 10000
#### `HARDCOVER_SYNC_INTERVAL_UNIT`
**Interval unit**
Whether the sync interval is measured in minutes or hours.
Auto-approve and auto-download matching releases for Hardcover requests. Another user's request is only approved this way where their request policy already lets them download directly; the rest wait for an admin.
- **Type:** boolean
- **Default:** `false`
#### `AUTO_DOWNLOAD_SOURCE_PRIORITY`
**Audiobook source priority**
Drag to set which release sources are tried first for audiobook requests. The first source with a strict match wins. Disabled sources are skipped.
- **Type:** JSON array
- **Default:** _empty list_
#### `AUTO_DOWNLOAD_EBOOK_SOURCE_PRIORITY`
**Ebook source priority**
Drag to set which release sources are tried first for ebook requests. The first source with a strict match wins. Disabled sources are skipped.
- **Type:** JSON array
- **Default:** _empty list_
#### `AUTO_DOWNLOAD_MIN_SEEDERS`
**Minimum seeders (torrents)**
Skip torrent releases with fewer seeders than this.
- **Type:** number
- **Default:** `1`
- **Constraints:** min: 0, max: 1000
#### `AUTO_DOWNLOAD_RETRY_DAYS`
**Retry failed downloads after (days)**
A synced request whose download failed, or was interrupted (a restart, say), goes back to pending after this many days and is tried again, a few per pass. A release that failed is not picked again, and a request is given up on after three failed retries. A download the stall timer cancelled counts as failed; one cancelled by a person, and rejected requests, are never retried. 0 turns retries off.
- **Type:** number
- **Default:** `7`
- **Constraints:** min: 0, max: 365
#### `AUTO_DOWNLOAD_FAILURE_RETRY_MINUTES`
**Try another release after a failed download (minutes)**
A download that failed, or that was given up on (stalled, no seeders), goes back to pending this many minutes later and another release is chosen, without waiting for the retry days above or the next sync cycle. The release that failed is not picked again, and a request is given up on after three failed attempts. 0 turns this off.
- **Type:** number
- **Default:** `0`
- **Constraints:** min: 0, max: 1440
#### `AUTO_DOWNLOAD_RETRY_FAILED_RELEASE_DAYS`
**Try a failed release again after (days)**
A release that failed or stalled is kept out of the search for this many days, then it may be picked again. A torrent that stalled can be revived by new seeds, so a failure should not shut a book out for good. A request is still given up on after three failed attempts.
- **Type:** number
- **Default:** `7`
- **Constraints:** min: 1, max: 365
#### `AUTO_DOWNLOAD_BACKOFF_MAX_DAYS`
**Longest wait between searches for an unmatched request (days)**
A request that no source has a match for is searched every cycle at first. After three passes in a row with no match it is searched once a day, then every two days, and at most this often apart. A match, or Sync now, starts it over. 0 searches every unmatched request every cycle.
- **Type:** number
- **Default:** `3`
- **Constraints:** min: 0, max: 30
#### `AUTO_DOWNLOAD_EXISTENCE_CHECK_ENABLED`
**Mark audiobooks no store sells**
An audiobook request that stays unmatched is looked up on Audible (US and UK) through your Audiobookshelf server. Either way it stays pending and its reason says what the store answered (with the ASIN when listed), for you to decline or keep; it is checked again after a week. A lookup that fails decides nothing. Needs the Audiobookshelf URL and token below.
- **Type:** boolean
- **Default:** `false`
#### `AUTO_DOWNLOAD_EXISTENCE_CHECK_AFTER_PASSES`
**Look the book up after (unmatched passes)**
How many consecutive passes with no strict match a request goes through before the store is asked. A pass that matched, or a reopen, starts the count over.
- **Type:** number
- **Default:** `3`
- **Constraints:** min: 1, max: 100
#### `AUTO_DOWNLOAD_AUDIOBOOK_MIN_SIZE_PERCENT`
**Smallest acceptable audiobook copy (%)**
A copy of an audiobook smaller than this share of the largest copy found for the same book is probably a lower-bitrate copy. It is tried after the others, and a single m4b that small is not preferred over a fuller set of chapter files. 0 turns the check off.
- **Type:** number
- **Default:** `67`
- **Constraints:** min: 0, max: 100
</details>
## Network
| Variable | Description | Type | Default |
@@ -1460,6 +1719,100 @@ The mam_id value MyAnonamouse shows when you create a session. MAM locks each se
</details>
## Jackett
| Variable | Description | Type | Default |
|----------|-------------|------|---------|
| `JACKETT_ENABLED` | Enable searching for books via Jackett indexers | boolean | `false` |
| `JACKETT_URL` | Base URL of your Jackett instance | string | _none_ |
| `JACKETT_API_KEY` | Shown at the top right of the Jackett dashboard | string (secret) | _none_ |
| `JACKETT_INDEXERS` | Select which indexers to search. Leave empty to search all configured indexers. | string (comma-separated) | _empty list_ |
| `JACKETT_INDEXER_TIMEOUT` | How long to wait for Jackett to answer a search. Searching all indexers waits for the slowest one, and indexers behind FlareSolverr can need 90 seconds or more on a cold Cloudflare challenge. | number | `90` |
| `JACKETT_EBOOK_MAX_SIZE_MB` | Hide ebook results larger than this. 0 shows everything, and a result with no reported size is always shown. Audiobook searches are not limited. | number | `0` |
| `JACKETT_AUTO_EXPAND` | Automatically retry search without category filtering if no results are found | boolean | `false` |
| `JACKETT_HIDE_UNSUPPORTED_FORMATS` | Hide a result whose title ends in a file extension that is not in Supported Book Formats, such as a .doc or .rtf file. Titles without an extension, archives and audiobook searches are never hidden. | boolean | `true` |
<details>
<summary>Detailed descriptions</summary>
#### `JACKETT_ENABLED`
**Enable Jackett source**
Enable searching for books via Jackett indexers
- **Type:** boolean
- **Default:** `false`
#### `JACKETT_URL`
**Jackett URL**
Base URL of your Jackett instance
- **Type:** string
- **Default:** _none_
- **Required:** Yes
#### `JACKETT_API_KEY`
**API Key**
Shown at the top right of the Jackett dashboard
- **Type:** string (secret)
- **Default:** _none_
- **Required:** Yes
#### `JACKETT_INDEXERS`
**Indexers to Search**
Select which indexers to search. Leave empty to search all configured indexers.
- **Type:** string (comma-separated)
- **Default:** _empty list_
#### `JACKETT_INDEXER_TIMEOUT`
**Indexer Search Timeout (seconds)**
How long to wait for Jackett to answer a search. Searching all indexers waits for the slowest one, and indexers behind FlareSolverr can need 90 seconds or more on a cold Cloudflare challenge.
- **Type:** number
- **Default:** `90`
- **Constraints:** min: 5, max: 300
#### `JACKETT_EBOOK_MAX_SIZE_MB`
**Maximum Ebook Size (MB)**
Hide ebook results larger than this. 0 shows everything, and a result with no reported size is always shown. Audiobook searches are not limited.
- **Type:** number
- **Default:** `0`
- **Constraints:** min: 0, max: 100000
#### `JACKETT_AUTO_EXPAND`
**Auto-expand search on no results**
Automatically retry search without category filtering if no results are found
- **Type:** boolean
- **Default:** `false`
#### `JACKETT_HIDE_UNSUPPORTED_FORMATS`
**Hide results in formats you did not enable**
Hide a result whose title ends in a file extension that is not in Supported Book Formats, such as a .doc or .rtf file. Titles without an extension, archives and audiobook searches are never hidden.
- **Type:** boolean
- **Default:** `true`
</details>
## Newznab
| Variable | Description | Type | Default |
@@ -1551,9 +1904,12 @@ Automatically retry search without category filtering if no results are found
| `SLSKD_ISOLATE_DOWNLOADS` | Download into a folder of its own inside slskd's downloads folder (needs slskd 0.26 or newer). Other apps using the same slskd cannot mix files with it or lose track of it when they clear slskd's list of finished transfers, and Shelfmark confirms a download by the files on disk. | boolean | `true` |
| `SLSKD_DESTINATION_PREFIX` | The folder, inside slskd's downloads folder, that holds one folder per download. It must not climb out of slskd's downloads folder. | string | `shelfmark` |
| `SLSKD_KEEP_COMPLETED` | Leave the finished files (and their transfer records) in slskd after Shelfmark copies them to your library, so they stay shared with other Soulseek users. Turn off to delete them once imported. | boolean | `true` |
| `SLSKD_FINISH_LEFTOVERS` | A restart can land after the last file arrives and before it is copied to your library. Checked every 15 minutes, independent of the Hardcover sync, a download folder that holds every file asked for, whose download never completed and is still wanted, is imported once without downloading anything again (never one someone cancelled). If that import fails, the folder stays until you retry the download, or, with clean-up on, until it is a day old and its request has the book or was given up on. A transfer Shelfmark queued that no download is watching any more is picked up again if its request still needs it, a few per check, and cancelled otherwise, or when it cannot be picked up. Transfers queued by other apps or by hand in slskd are never touched, including the same file queued again after Shelfmark's download of it ended. | boolean | `true` |
| `SLSKD_CLEAN_LEFTOVERS_DAYS` | Remove the folder of a failed or cancelled download, or one nothing records, once the download is this old, or one day old when its request already has the book or was given up on. A folder that holds every file asked for is never removed while the download is still wanted (you started it yourself, or its request is still open): it is imported once, and after that it waits for you to retry the download. The folder of a running download is never removed. 0 keeps every leftover folder. Folders already imported follow Keep finished files in slskd, not this setting. | number | `0` |
| `SLSKD_SEARCH_TIMEOUT` | How long a Soulseek search collects peer responses. Longer waits find more peers but make every search slower. | number | `15` |
| `SLSKD_RESPONSE_LIMIT` | Stop a search early once this many peers have answered | number | `100` |
| `SLSKD_QUEUE_TIMEOUT_MINUTES` | Give up when the peer has not started sending within this long. Set to 0 to wait indefinitely. | number | `60` |
| `SLSKD_CHAT_AUTOPROVE` | Some uploaders block downloads until you type a word in a private message ("please type 'download' in this chat"). Shelfmark replies with the word the message asks for, at most once per user every 12 hours, and logs each reply. It reads every unread private message in slskd, not only those from the peer of a running download, and marks a conversation it answered as read in slskd. Checked while a download is running. | boolean | `false` |
| `SLSKD_REMOVE_COMPLETED` | Clear the transfer from slskd's download list after Shelfmark imports it. Applies when each download does not get its own folder. | boolean | `true` |
<details>
@@ -1622,6 +1978,25 @@ Leave the finished files (and their transfer records) in slskd after Shelfmark c
- **Type:** boolean
- **Default:** `true`
#### `SLSKD_FINISH_LEFTOVERS`
**Import finished folders an interruption left behind**
A restart can land after the last file arrives and before it is copied to your library. Checked every 15 minutes, independent of the Hardcover sync, a download folder that holds every file asked for, whose download never completed and is still wanted, is imported once without downloading anything again (never one someone cancelled). If that import fails, the folder stays until you retry the download, or, with clean-up on, until it is a day old and its request has the book or was given up on. A transfer Shelfmark queued that no download is watching any more is picked up again if its request still needs it, a few per check, and cancelled otherwise, or when it cannot be picked up. Transfers queued by other apps or by hand in slskd are never touched, including the same file queued again after Shelfmark's download of it ended.
- **Type:** boolean
- **Default:** `true`
#### `SLSKD_CLEAN_LEFTOVERS_DAYS`
**Remove leftover download folders after (days)**
Remove the folder of a failed or cancelled download, or one nothing records, once the download is this old, or one day old when its request already has the book or was given up on. A folder that holds every file asked for is never removed while the download is still wanted (you started it yourself, or its request is still open): it is imported once, and after that it waits for you to retry the download. The folder of a running download is never removed. 0 keeps every leftover folder. Folders already imported follow Keep finished files in slskd, not this setting.
- **Type:** number
- **Default:** `0`
- **Constraints:** min: 0, max: 365
#### `SLSKD_SEARCH_TIMEOUT`
**Search Wait (seconds)**
@@ -1652,6 +2027,15 @@ Give up when the peer has not started sending within this long. Set to 0 to wait
- **Default:** `60`
- **Constraints:** min: 0, max: 1440
#### `SLSKD_CHAT_AUTOPROVE`
**Answer anti-leech chat challenges**
Some uploaders block downloads until you type a word in a private message ("please type 'download' in this chat"). Shelfmark replies with the word the message asks for, at most once per user every 12 hours, and logs each reply. It reads every unread private message in slskd, not only those from the peer of a running download, and marks a conversation it answered as read in slskd. Checked while a download is running.
- **Type:** boolean
- **Default:** `false`
#### `SLSKD_REMOVE_COMPLETED`
**Remove completed transfers from slskd**
@@ -1779,6 +2163,59 @@ Maximum number of results to request per search (1-100).
</details>
## Webshare.cz
| Variable | Description | Type | Default |
|----------|-------------|------|---------|
| `WEBSHARE_ENABLED` | Search Webshare.cz by file name and download matching ebooks and audiobooks. Requires a Webshare account; a VIP account is recommended for full download speed. | boolean | `false` |
| `WEBSHARE_MAX_RESULTS` | Maximum number of files to request per search (1-200). Results are then filtered to your supported ebook or audiobook formats. | number | `50` |
<details>
<summary>Detailed descriptions</summary>
#### `WEBSHARE_ENABLED`
**Enable Webshare.cz**
Search Webshare.cz by file name and download matching ebooks and audiobooks. Requires a Webshare account; a VIP account is recommended for full download speed.
- **Type:** boolean
- **Default:** `false`
#### `WEBSHARE_USERNAME`
**Username or Email**
Your Webshare.cz login.
- **Type:** string
- **Default:** _none_
- **Required:** Yes
#### `WEBSHARE_PASSWORD`
**Password**
Your Webshare.cz password.
- **Type:** string (secret)
- **Default:** _none_
- **Required:** Yes
#### `WEBSHARE_MAX_RESULTS`
**Max Results**
Maximum number of files to request per search (1-200). Results are then filtered to your supported ebook or audiobook formats.
- **Type:** number
- **Default:** `50`
- **Constraints:** min: 1, max: 200
</details>
## IRC
| Variable | Description | Type | Default |
@@ -1894,49 +2331,50 @@ How long to keep cached search results before they expire.
| `DEBRIDLINK_API_KEY` | Debrid-Link API Key from your Debrid-Link account settings | string (secret) | _none_ |
| `REALDEBRID_API_KEY` | Real-Debrid API Key (Secret Token) from your Real-Debrid account settings | string (secret) | _none_ |
| `TORBOX_API_KEY` | TorBox API Key from your TorBox account settings | string (secret) | _none_ |
| `QBITTORRENT_DOWNLOAD_DIR_BOOK` | Server-side directory where book torrents are downloaded (optional, uses qBittorrent default if not specified) | string | _none_ |
| `QBITTORRENT_DOWNLOAD_DIR_AUDIOBOOK` | Server-side directory where audiobook torrents are downloaded (optional, uses qBittorrent default if not specified) | string | _none_ |
| `QBITTORRENT_URL` | Web UI URL of your qBittorrent instance | string | _none_ |
| `QBITTORRENT_API_KEY` | Found in qBittorrent: Options > Web UI > API Key (qBittorrent 5.2.0+). Used instead of the username and password when set. | string (secret) | _none_ |
| `QBITTORRENT_CATEGORY` | Category to assign to book downloads in qBittorrent | string | `books` |
| `QBITTORRENT_CATEGORY_AUDIOBOOK` | Category for audiobook downloads. Leave empty to use the book category. | string | _empty string_ |
| `TRANSMISSION_SEPARATE_DOWNLOAD_DIRS` | Download books and audiobooks to different directories instead of one shared directory. | boolean | `false` |
| `QBITTORRENT_SEPARATE_DOWNLOAD_DIRS` | Download books and audiobooks to different directories instead of one shared directory. | boolean | `false` |
| `TRANSMISSION_DOWNLOAD_DIR_BOOK` | Server-side directory where book torrents are downloaded (optional, uses Transmission default if not specified) | string | _none_ |
| `TRANSMISSION_DOWNLOAD_DIR_AUDIOBOOK` | Server-side directory where audiobook torrents are downloaded (optional, uses Transmission default if not specified) | string | _none_ |
| `QBITTORRENT_DOWNLOAD_DIR` | Server-side directory where torrents are downloaded (optional, uses qBittorrent default if not specified) | string | _none_ |
| `QBITTORRENT_DOWNLOAD_DIR_BOOK` | Server-side directory where book torrents are downloaded (optional, uses qBittorrent default if not specified) | string | _none_ |
| `QBITTORRENT_DOWNLOAD_DIR_AUDIOBOOK` | Server-side directory where audiobook torrents are downloaded (optional, uses qBittorrent default if not specified) | string | _none_ |
| `QBITTORRENT_TAG` | Tag(s) to assign to qBittorrent downloads. Leave empty for no tags. | string (comma-separated) | _empty list_ |
| `TRANSMISSION_URL` | URL of your Transmission instance (use https:// for TLS) | string | _none_ |
| `TRANSMISSION_URL` | URL of your Transmission instance (use https:// for TLS). Without a port, https uses 443 and http uses 9091, so add :9091 to an https URL served on 9091 | string | _none_ |
| `DELUGE_SEPARATE_DOWNLOAD_DIRS` | Download books and audiobooks to different directories instead of one shared directory. | boolean | `false` |
| `TRANSMISSION_CATEGORY` | Label to assign to book downloads in Transmission | string | `books` |
| `DELUGE_DOWNLOAD_DIR_BOOK` | Server-side directory where book torrents are downloaded (optional, uses Deluge default if not specified) | string | _none_ |
| `DELUGE_DOWNLOAD_DIR_AUDIOBOOK` | Server-side directory where audiobook torrents are downloaded (optional, uses Deluge default if not specified) | string | _none_ |
| `TRANSMISSION_CATEGORY_AUDIOBOOK` | Label for audiobook downloads. Leave empty to use the book label. | string | _empty string_ |
| `TRANSMISSION_SEPARATE_DOWNLOAD_DIRS` | Download books and audiobooks to different directories instead of one shared directory. | boolean | `false` |
| `TRANSMISSION_DOWNLOAD_DIR` | Server-side directory where torrents are downloaded (optional, uses Transmission default if not specified) | string | _none_ |
| `TRANSMISSION_DOWNLOAD_DIR_BOOK` | Server-side directory where book torrents are downloaded (optional, uses Transmission default if not specified) | string | _none_ |
| `TRANSMISSION_DOWNLOAD_DIR_AUDIOBOOK` | Server-side directory where audiobook torrents are downloaded (optional, uses Transmission default if not specified) | string | _none_ |
| `DELUGE_HOST` | Hostname/IP or full URL of your Deluge Web UI (deluge-web) | string | `localhost` |
| `DELUGE_PORT` | Deluge Web UI port (default: 8112) | string | `8112` |
| `RTORRENT_SEPARATE_DOWNLOAD_DIRS` | Download books and audiobooks to different directories instead of one shared directory. | boolean | `false` |
| `DELUGE_CATEGORY` | Label to assign to book downloads in Deluge | string | `books` |
| `RTORRENT_DOWNLOAD_DIR_BOOK` | Server-side directory where book torrents are downloaded (optional, uses rTorrent default if not specified) | string | _none_ |
| `RTORRENT_DOWNLOAD_DIR_AUDIOBOOK` | Server-side directory where audiobook torrents are downloaded (optional, uses rTorrent default if not specified) | string | _none_ |
| `DELUGE_CATEGORY_AUDIOBOOK` | Label for audiobook downloads. Leave empty to use the book label. | string | _empty string_ |
| `DELUGE_SEPARATE_DOWNLOAD_DIRS` | Download books and audiobooks to different directories instead of one shared directory. | boolean | `false` |
| `DELUGE_DOWNLOAD_DIR` | Server-side directory where torrents are downloaded (optional, uses Deluge default if not specified) | string | _none_ |
| `DELUGE_DOWNLOAD_DIR_BOOK` | Server-side directory where book torrents are downloaded (optional, uses Deluge default if not specified) | string | _none_ |
| `DELUGE_DOWNLOAD_DIR_AUDIOBOOK` | Server-side directory where audiobook torrents are downloaded (optional, uses Deluge default if not specified) | string | _none_ |
| `RTORRENT_URL` | XML-RPC URL of your rTorrent instance | string | _none_ |
| `RTORRENT_LABEL` | Label to assign to ebook downloads in rTorrent | string | `cwabd` |
| `RTORRENT_AUDIOBOOK_LABEL` | Label to assign to audiobook downloads in rTorrent (falls back to Book Label if not set) | string | _none_ |
| `RTORRENT_SEPARATE_DOWNLOAD_DIRS` | Download books and audiobooks to different directories instead of one shared directory. | boolean | `false` |
| `RTORRENT_DOWNLOAD_DIR` | Server-side directory where torrents are downloaded (optional, uses rTorrent default if not specified) | string | _none_ |
| `RTORRENT_DOWNLOAD_DIR_BOOK` | Server-side directory where book torrents are downloaded (optional, uses rTorrent default if not specified) | string | _none_ |
| `RTORRENT_DOWNLOAD_DIR_AUDIOBOOK` | Server-side directory where audiobook torrents are downloaded (optional, uses rTorrent default if not specified) | string | _none_ |
| `PROWLARR_TORRENT_ACTION` | After a successful import, Remove keeps downloaded files in qBittorrent, Transmission, and Deluge; Remove & Delete Files also deletes them. rTorrent cannot delete download data, so this action leaves its torrent untouched. Blackhole does not support removal. Debrid clients delete temporary local files for either Remove action. | string (choice) | `keep` |
| `TORRENT_NO_SEEDERS_WAIT` | Give up on a torrent when its client sees no seed at all for this long, even while it is still downloading, or fewer seeds than the minimum below while nothing is downloading. The torrent is removed and the download marked failed, so another release can be tried. The card shows the seed count and a countdown while it waits. 0 keeps the default behaviour (only the stall timer ends it). Only qBittorrent reports seed counts. | number | `0` |
| `TORRENT_MIN_SEEDERS` | Seeds the client must see for a motionless torrent to keep waiting. 0 also turns the no-seed check off. | number | `1` |
| `TORRENT_MIN_SEEDERS` | Seeds the client must see for a motionless torrent to keep waiting. 0 also turns the no-seed check off. Only qBittorrent reports seed counts. | number | `1` |
| `PROWLARR_TORRENT_POST_IMPORT_CATEGORY` | Category or label to assign after a successful import | string | _empty string_ |
| `TORRENT_STALL_TIMEOUT_MINUTES` | How long a torrent can go without any progress before Shelfmark cancels it. Applies from when the torrent starts and restarts each time it moves. Raise it for slow swarms or magnets that take a while to find peers, or lower it to give up on dead torrents sooner. | number | `15` |
| `STALE_TORRENT_HOURS` | Remove a torrent Shelfmark added when it is stalled or cannot fetch its metadata, has had no activity for this many hours, and no running download owns it. Its partial data goes with it. Only torrents Shelfmark itself added are ever considered, whatever category they are in; torrents still downloading, queued, or finished are never touched. 3 to 72 hours (a value outside that is held to it); 0 keeps them. Checked every 15 minutes, independent of the Hardcover sync; only qBittorrent is supported. | number | `6` |
| `PROWLARR_USENET_CLIENT` | Choose which usenet client to use | string (choice) | _empty string_ |
| `NZBGET_URL` | URL of your NZBGet instance | string | _none_ |
@@ -2053,6 +2491,14 @@ Category to assign to book downloads in qBittorrent
- **Default:** `books`
#### `QBITTORRENT_CATEGORY_AUDIOBOOK`
**Audiobook Category**
Category for audiobook downloads. Leave empty to use the book category.
- **Type:** string
- **Default:** _empty string_
#### `QBITTORRENT_SEPARATE_DOWNLOAD_DIRS`
**Separate Book and Audiobook Directories**
@@ -2062,16 +2508,7 @@ Download books and audiobooks to different directories instead of one shared dir
- **Type:** boolean
- **Default:** `false`
**Audiobook Category**
Category for audiobook downloads. Leave empty to use the book category.
- **Type:** string
- **Default:** _empty string_
#### `QBITTORRENT_DOWNLOAD_DIR`
#### `QBITTORRENT_DOWNLOAD_DIR_BOOK`
**Download Directory**
@@ -2080,6 +2517,7 @@ Server-side directory where torrents are downloaded (optional, uses qBittorrent
- **Type:** string
- **Default:** _none_
#### `QBITTORRENT_DOWNLOAD_DIR_BOOK`
**Book Download Directory**
@@ -2110,7 +2548,7 @@ Tag(s) to assign to qBittorrent downloads. Leave empty for no tags.
**Transmission URL**
URL of your Transmission instance (use https:// for TLS)
URL of your Transmission instance (use https:// for TLS). Without a port, https uses 443 and http uses 9091, so add :9091 to an https URL served on 9091
- **Type:** string
- **Default:** _none_
@@ -2143,6 +2581,14 @@ Label to assign to book downloads in Transmission
- **Default:** `books`
#### `TRANSMISSION_CATEGORY_AUDIOBOOK`
**Audiobook Label**
Label for audiobook downloads. Leave empty to use the book label.
- **Type:** string
- **Default:** _empty string_
#### `TRANSMISSION_SEPARATE_DOWNLOAD_DIRS`
**Separate Book and Audiobook Directories**
@@ -2152,16 +2598,7 @@ Download books and audiobooks to different directories instead of one shared dir
- **Type:** boolean
- **Default:** `false`
**Audiobook Label**
Label for audiobook downloads. Leave empty to use the book label.
- **Type:** string
- **Default:** _empty string_
#### `TRANSMISSION_DOWNLOAD_DIR`
#### `TRANSMISSION_DOWNLOAD_DIR_BOOK`
**Download Directory**
@@ -2170,6 +2607,7 @@ Server-side directory where torrents are downloaded (optional, uses Transmission
- **Type:** string
- **Default:** _none_
#### `TRANSMISSION_DOWNLOAD_DIR_BOOK`
**Book Download Directory**
@@ -2224,6 +2662,14 @@ Label to assign to book downloads in Deluge
- **Default:** `books`
#### `DELUGE_CATEGORY_AUDIOBOOK`
**Audiobook Label**
Label for audiobook downloads. Leave empty to use the book label.
- **Type:** string
- **Default:** _empty string_
#### `DELUGE_SEPARATE_DOWNLOAD_DIRS`
**Separate Book and Audiobook Directories**
@@ -2233,16 +2679,7 @@ Download books and audiobooks to different directories instead of one shared dir
- **Type:** boolean
- **Default:** `false`
**Audiobook Label**
Label for audiobook downloads. Leave empty to use the book label.
- **Type:** string
- **Default:** _empty string_
#### `DELUGE_DOWNLOAD_DIR`
#### `DELUGE_DOWNLOAD_DIR_BOOK`
**Download Directory**
@@ -2251,6 +2688,7 @@ Server-side directory where torrents are downloaded (optional, uses Deluge defau
- **Type:** string
- **Default:** _none_
#### `DELUGE_DOWNLOAD_DIR_BOOK`
**Book Download Directory**
@@ -2305,6 +2743,14 @@ Label to assign to ebook downloads in rTorrent
- **Default:** `cwabd`
#### `RTORRENT_AUDIOBOOK_LABEL`
**Audiobook Label**
Label to assign to audiobook downloads in rTorrent (falls back to Book Label if not set)
- **Type:** string
- **Default:** _none_
#### `RTORRENT_SEPARATE_DOWNLOAD_DIRS`
**Separate Book and Audiobook Directories**
@@ -2314,16 +2760,7 @@ Download books and audiobooks to different directories instead of one shared dir
- **Type:** boolean
- **Default:** `false`
**Audiobook Label**
Label to assign to audiobook downloads in rTorrent (falls back to Book Label if not set)
- **Type:** string
- **Default:** _none_
#### `RTORRENT_DOWNLOAD_DIR`
#### `RTORRENT_DOWNLOAD_DIR_BOOK`
**Download Directory**
@@ -2332,6 +2769,7 @@ Server-side directory where torrents are downloaded (optional, uses rTorrent def
- **Type:** string
- **Default:** _none_
#### `RTORRENT_DOWNLOAD_DIR_BOOK`
**Book Download Directory**
@@ -2373,7 +2811,7 @@ Give up on a torrent when its client sees no seed at all for this long, even whi
**Minimum Seeders**
Seeds the client must see for a motionless torrent to keep waiting. 0 also turns the no-seed check off.
Seeds the client must see for a motionless torrent to keep waiting. 0 also turns the no-seed check off. Only qBittorrent reports seed counts.
- **Type:** number
- **Default:** `1`
@@ -2398,6 +2836,16 @@ How long a torrent can go without any progress before Shelfmark cancels it. Appl
- **Default:** `15`
- **Constraints:** min: 5, max: 60
#### `STALE_TORRENT_HOURS`
**Remove dead torrents after (hours)**
Remove a torrent Shelfmark added when it is stalled or cannot fetch its metadata, has had no activity for this many hours, and no running download owns it. Its partial data goes with it. Only torrents Shelfmark itself added are ever considered, whatever category they are in; torrents still downloading, queued, or finished are never touched. 3 to 72 hours (a value outside that is held to it); 0 keeps them. Checked every 15 minutes, independent of the Hardcover sync; only qBittorrent is supported.
- **Type:** number
- **Default:** `6`
- **Constraints:** min: 0, max: 72
#### `PROWLARR_USENET_CLIENT`
**Usenet Client**
@@ -2660,6 +3108,14 @@ Default sort order for Google Books search results.
#### `MOLY_ENABLED`
**Enable Moly.hu**
Enable Moly.hu as a metadata provider for book searches
- **Type:** boolean
- **Default:** `false`
</details>
### Metadata Providers: Databáze knih
| Variable | Description | Type | Default |
@@ -2680,12 +3136,45 @@ Enable Databáze knih as a metadata provider for book searches
</details>
### Metadata Providers: DNB
Enable Moly.hu as a metadata provider for book searches
| Variable | Description | Type | Default |
|----------|-------------|------|---------|
| `DNB_ENABLED` | Enable DNB as a metadata provider for book searches | boolean | `false` |
| `DNB_DEFAULT_SORT` | Default sort order for DNB search results. | string (choice) | `relevance` |
| `DNB_USE_BOOK_LANGUAGES` | Filter DNB results by Default Book Languages: each user's own setting, else the system default. DNB also catalogs translations into many other languages. | boolean | `true` |
<details>
<summary>Detailed descriptions</summary>
#### `DNB_ENABLED`
**Enable DNB**
Enable DNB as a metadata provider for book searches
Filter DNB results by Default Book Languages: each user's own setting, else the system default. DNB also catalogs translations into many other languages.
Hardcover Sync pulls books from your [Hardcover](https://hardcover.app) reading shelves into Shelfmark as requests, optionally downloads them without an admin picking a release, and can skip books you already own in a library you have set up under [Library Check](library-check.md). Every part is opt-in and off by default; the tab does nothing until you enable it.
## Capabilities
### Shelf sync
On a schedule, Shelfmark reads the selected Hardcover shelf (by default **Want to Read**) through the Hardcover metadata provider and creates a **pending** request for each book that is not already known. Requests carry Hardcover's metadata, including the cover, and are owned by the lowest-id admin account.
A book is skipped when:
- a request with the same Hardcover book id already exists, in any status
- a request for the same Hardcover book and format, by any user, has a download in the history that finished or is still running (an unfinished download older than a day no longer counts as running)
- the library check (below) says you already own it
- a pending request with the same title, author, and content type already exists
### Automatic downloads
When enabled, each pending request that came from Hardcover is searched across your release sources in the priority order configured for its content type (audiobook requests use the audiobook list, ebook requests the ebook list). The first source that yields a **strict match** wins, and the best candidate from that source is queued through the normal request fulfilment path, so it appears in the activity view and download history exactly like a request an admin approved.
Automatic downloads act for an admin, so they only approve what the request's owner could have downloaded without asking. Synced requests belong to an admin and are always searched. A request another user made is searched only on the release sources where their request policy (including any per-user override) resolves to **Download**, and only while requests are turned on for them; with requests off they download directly anyway. A request whose policy needs approval, or blocks that content type, is left pending for an admin, untouched, like any other request awaiting review. A failed download is only reopened for a retry on the same terms.
A release is a strict match only when all of the following hold:
- at least 85% of the significant words in the book's title appear in the release title. Numbers from 1 to 20 count as the same word however they are written, so *Part II*, *Part Two* and *Part 2* agree, while *Part One* never stands in for *Part Two*;
- the author's surname appears in the release title, indexer name or author field (a book with no author metadata matches on title alone). Some AudiobookBay listings carry no author in their title (*The Mummy or Ramses the Damned*). When a source turns up no strict match, the detail pages of up to three such releases that pass every other rule are read, and the author they credit (a narrator does not count) is checked like any other. A page that credits nobody, or someone else, leaves the release rejected;
- the release title doesn't name a different book that contains the requested one. A "Dune" request won't take *Dune Messiah* or *Children of Dune*, and a bare number after the title (*Mistborn 2*, *Mistborn Two*, *Dune Chronicles Book II*) names a sequel. A title with a number of its own (*Ready Player One*) takes no other number after it, 1 included. Only the parts of the release name that hold the title are judged, so narrators, uploader tags and the book's own series, author and subtitle words don't count against it; a part that is just the series name (*Alex Cross 25*) does not vouch for a one-word title, so a "Cross" request won't take *Alex Cross: Cross Justice*. Two slash-separated names that both hold the title (*Dune / Dune Messiah*) are two books;
- the release is not a derivative of the work: *summary*, *analysis*, *study guide*, *workbook*, *abridged*, *dramatized*, *adaptation* or *graphic novel* anywhere in the name disqualifies it, unless the requested title itself carries the word. *Unabridged* is a full recording and is fine;
- the release is not a multi-book pack: ranges like `1-24` or `Books 1 to 17`, or words like *complete*, *collection*, *box set*, *omnibus*, *trilogy*, *megapack*, *+ short stories*, unless the requested title itself carries them. "Complete and Unabridged" describes one recording and doesn't count. The library check uses the same word list;
- the format fits the request — an audiobook request needs an audiobook signal (`m4b`/`mp3`, a release from an audiobook-only source, or "audiobook"/"unabridged" in the title) and is never an ebook-only file; an ebook request needs one of your **Supported Book Formats** (or an `.epub`-style marker in the title) and no audiobook signal;
- for torrents that report a seeder count, it meets **Minimum seeders** — sources that report none (AudiobookBay) are not judged on seeders.
AudiobookBay requires every word of a query, and its posts for one volume of a multi-part book often spell the part differently from the request ("Part Two" for "Part II"), so a query carrying the part finds only the packs that list it. When no row a query returns carries the requested title, the source tries the next query, ending with the title minus its trailing *, Part N* or *Volume N*. The strict match above still decides which part was found.
When no source yields a strict match for the request's own search title, up to three simpler titles are tried in turn, each across the sources in priority order and all within the request's search budget: the title without a subtitle after `:` or ` - `, the title without an alternate title (*The Hobbit, or There and Back Again* becomes *The Hobbit*), and the title with its numbers written as digits (*Part II* becomes *Part 2*). The first one that yields a strict match wins, and the log names it. A simpler title is matched by exactly the rules above, so a release has to name the shorter title, the right author, and no sequel, edition, derivative or pack. The subtitle is only dropped when the book's metadata says it is one (a series name is known, or the book's subtitle is what follows the separator), and never when what is left is only the series or author name: *Star Wars: Thrawn* is not searched as *Star Wars*.
Among matching releases from the winning source, format ranks first (`m4b` over `mp3` for audiobooks; `epub` over `azw3` over `mobi` for ebooks), then the copy other people chose — more downloads (direct-download sources report them), then more seeders — then larger size. The exception is an audiobook copy far smaller than the others for the same book, which is probably a lower-bitrate copy: it is tried after the others (see **Smallest acceptable audiobook copy** below). If no source produces a confident match the request stays **pending** for manual review; nothing is guessed.
A request whose book turns out to be in your library already is closed, not left pending: it is marked fulfilled with the admin note `[auto] Already in the library.`, so it stops being looked up every pass. Before this it sat in Needs Review indefinitely. **Clear Completed** clears these too.
Hardcover allows a short burst of lookups and then answers 429. Each pass paces its book lookups and backs off when told to, following `Retry-After`. A lookup that still fails counts as an error for that request, not as "book not found", and the request is tried again next pass.
A download that fails, or is interrupted (a restart while it was queued, say), does not stay that way forever. Once the request has sat there for **Retry failed downloads after (days)** (7 by default), it goes back to pending and the next pass searches again. Each pass reopens at most five, oldest first, so a backlog is spread out instead of hitting your sources at once. An interrupted download may pick the same release again, since a torrent already in your client is joined instead of added twice. A download that itself failed, or that Shelfmark's stall timer cancelled, has its release remembered and skipped next time (not when the download client could not be reached or refused Shelfmark's login: that says nothing about the release), and the request is given up on after three failed retries. Requests you or an admin cancelled are never retried, and neither are rejected ones: both are decisions. Set the value to 0 to turn retries off.
A request that no source has a match for is not searched every cycle for ever. After three passes in a row with no match it is searched once a day, then every two days, and no further apart than **Longest wait between searches for an unmatched request (days)** (3 by default, 0 searches every unmatched request every cycle). A match, a search that failed (which says nothing about the book) or **Sync now** starts it over.
**Trying another release sooner.** Set **Try another release after a failed download (minutes)** (off by default) and a download that fails, or that is given up on (stalled, or no seeders when **Give Up With No Seeders** is on), does not wait for the days above. Between sync cycles Shelfmark looks for such requests, reopens them after that many minutes and searches again, for those requests only. The release that failed is skipped, and a request is still given up on after three failed attempts.
**Sources known to be dead are tried last.** When choosing among matching releases, a Soulseek peer whose transfers failed in the last three days and who delivered nothing in that time comes after the others. AudiobookBay shows no seed counts, so the comments on its five best posts are read, and a post where comments say it has no seeders comes after the others. Nothing is excluded: if a dead-looking release is the only match, it is still tried.
### Audiobooks no store sells
Some requested books were never recorded, so no source will ever match them and the request waits in Needs Review for good. With **Mark audiobooks no store sells** on, an audiobook request that has gone **Look the book up after (unmatched passes)** consecutive passes (3 by default) without a strict match is looked up on Audible's US and UK stores, through the metadata search of the Audiobookshelf server set up for the library check (its URL and token are needed; the library check itself can stay off).
- **On neither store:** the request stays pending and its failure reason reads `Not found on Audible (US/UK) after <N> searches, <date>`. It is not declined: Audible search misses real recordings (novellas sold only in a collection, books retitled before release, older or fan recordings), so the call is left to you. The store is asked again a week later.
- **On a store:** the request stays pending and its failure reason reads `Exists on Audible (<ASIN>), not on our sources yet`. The store is asked again a week later, not every pass.
- **Lookup failed:** nothing changes and the next pass asks again. A failed lookup never marks a request. Audiobookshelf answers a failed Audible search with an empty list, which looks like "no such book", so before marking a request as not found Shelfmark also looks up a book both stores are known to sell (*The Hobbit*). If that comes back missing too, the answer is treated as a failure.
A store result counts as the book when its title matches the request's once both are normalised (accents, punctuation, "Part II"/"Part Two"/"Part 2", a subtitle after `:` or `, or`, bracketed notes like "(Unabridged)" and a leading series label are all ignored, and a title that begins the other with at least two words also counts) and the request author's surname is in its author field. The matching leans towards "found": a wrong "found" only leaves the request pending. The pass counter is reset when a source matches the book and when the request is rejected, so a reopened request counts afresh. Store lookups come out of the request's search time; when too little is left, the lookup waits for the next pass. Ebook requests are not checked.
### Library check
Both shelf sync and automatic downloads ask the libraries you have enabled under [Library Check](library-check.md) whether the book is already there, for the content type being requested, and skip it if so. There is no separate switch on this tab. A library that is not enabled, or that does not hold the requested content type, never causes a skip.
The check **fails closed** here: if an enabled library cannot be read and has not been read before, the sync cycle is skipped and an automatic download is held, and the request stays pending until the next cycle. Skipping is the safe side, because carrying on would request or download books the library already holds. Badges in search results still fail open.
## Settings
| Key | Label | Default | Notes |
|-----|-------|---------|-------|
| `HARDCOVER_SYNC_ENABLED` | Enable scheduled sync | Off | Runs shelf sync on the interval below |
| `HARDCOVER_SYNC_TOKEN` | Hardcover API Token | — | Leave blank to reuse the Hardcover metadata provider's token |
| `HARDCOVER_SYNC_STATUSES` | Shelves to sync | Want to Read | Hardcover reading status to pull from |
| `HARDCOVER_SYNC_CONTENT_TYPE` | Request as | Audiobooks | `Audiobooks`, `Ebooks`, or `Ebooks and audiobooks` — the last creates one request of each per shelf book, each checked against its own library |
| `HARDCOVER_SYNC_INTERVAL` | Sync interval | 6 | Combined with the unit below |
| `HARDCOVER_SYNC_INTERVAL_UNIT` | Interval unit | Hours | Minutes or hours. The effective interval never drops below 60 seconds |
| `AUTO_DOWNLOAD_ENABLED` | Enable automatic downloads | Off | Auto-approve and queue strict matches for pending Hardcover requests: synced ones, and a user's own only on the sources where their request policy already lets them download directly; the rest wait for an admin. The download belongs to the user who asked, and admins see every user's requests |
| `AUTO_DOWNLOAD_SOURCE_PRIORITY` | Audiobook source priority | All usable sources | Drag to order; used for audiobook requests. Sources you disable here, or that are not configured, are skipped |
| `AUTO_DOWNLOAD_EBOOK_SOURCE_PRIORITY` | Ebook source priority | All usable sources | Same, for ebook requests |
| `AUTO_DOWNLOAD_MIN_SEEDERS` | Minimum seeders (torrents) | 1 | Torrent releases below this are ignored |
| `AUTO_DOWNLOAD_AUDIOBOOK_MIN_SIZE_PERCENT` | Smallest acceptable audiobook copy (%) | 67 | A copy smaller than this share of the largest copy found for the same book is tried after the others, and a single m4b that small is not preferred over a fuller set of chapter files. 0 turns the check off |
| `AUTO_DOWNLOAD_RETRY_DAYS` | Retry failed downloads after (days) | 7 | A failed or interrupted download goes back to pending after this long. 0 turns retries off |
| `AUTO_DOWNLOAD_BACKOFF_MAX_DAYS` | Longest wait between searches for an unmatched request (days) | 3 | 0 searches every unmatched request every cycle |
| `AUTO_DOWNLOAD_FAILURE_RETRY_MINUTES` | Try another release after a failed download (minutes) | 0 (off) | Reopens failed or given-up downloads and picks another release this long after, between sync cycles |
| `AUTO_DOWNLOAD_RETRY_FAILED_RELEASE_DAYS` | Try a failed release again after (days) | 7 | A release that failed or stalled is kept out of the search for this long, then may be picked again; a request is still given up on after three failed attempts |
| `AUTO_DOWNLOAD_EXISTENCE_CHECK_ENABLED` | Mark audiobooks no store sells | Off | See [Audiobooks no store sells](#audiobooks-no-store-sells). Needs the Audiobookshelf URL and token |
| `AUTO_DOWNLOAD_EXISTENCE_CHECK_AFTER_PASSES` | Look the book up after (unmatched passes) | 3 | Consecutive passes without a strict match before the stores are asked |
One action button sits at the bottom of the tab:
- **Sync now** runs a sync and automatic-download pass immediately using the saved settings. Sync runs even when scheduled sync is off; automatic downloads still require their own toggle.
## Request Flow
1. Shelf sync creates a **pending** request per new Hardcover book, with cover, year, and subtitle where Hardcover has them.
2. If automatic downloads are enabled, the request is searched source by source and the first strict match is queued through the normal fulfil path. The request becomes **fulfilled** and its delivery state is tracked like any other.
3. If no source yields a confident match, the request stays **pending** in the admin queue for manual review, exactly as if a user had requested the book.
Only requests whose provider is Hardcover are considered for automatic downloads, and requests that have already been dispatched are left alone.
## Scheduler
The scheduler starts with the app and waits **60 seconds** before its first cycle. Each cycle runs a shelf sync (when scheduled sync is enabled) followed by an automatic-download pass (when that is enabled), then sleeps for the configured interval. Intervals shorter than 60 seconds are raised to 60 seconds so a typo cannot hammer the Hardcover API. If the interval cannot be read at all, the scheduler logs the error and tries again after six hours instead of stopping. When neither toggle is on the scheduler idles.
Only one run happens at a time. If a scheduled cycle is in progress, **Sync now** reports that a sync is already running.
## Troubleshooting
**Sync adds nothing and the log says "no Hardcover token configured"** — set **Hardcover API Token** on this tab, or configure the token on the Hardcover metadata provider. A sync without a usable token counts as an error and creates no requests.
**Requests appear but nothing downloads** — check **Enable automatic downloads** is on, that the source-priority list for the request's content type contains at least one configured source, and look for `no strict match` lines in the log. A request from another user is only searched on the sources where their policy allows a direct download (or, with requests turned off for them, on every source); with no such source it stays pending for an admin and is not searched. Lowering **Minimum seeders** helps on quiet trackers; loosening the title or author match is deliberately not an option.
**Books you own keep being added** — enable the library check for the format you request (see [Library Check](library-check.md)) and use its test button. A library that cannot be read shows up in the log as a skipped cycle.
**Too many requests at once** — the first sync of a large shelf creates one request per book, and the per-user pending limit from **Users & Requests** does not apply to synced requests. Start with a shelf you are happy to see in the queue in full.
@@ -21,6 +21,26 @@ database directly. No HTTP call, no API token, and nothing is ever written back.
A result that matches the library then carries an **In library** badge in the card, list
and compact views, and in the details dialog.
## Audiobookshelf
Shelfmark asks the Audiobookshelf server over its REST API for the title, author, ISBN and
ASIN of every item, and nothing is written back.
1. In **Settings, General**, turn on **Mark audiobooks already in your Audiobookshelf
library**.
2. Set **Audiobookshelf URL** to an address the Shelfmark container can reach (often the
host's LAN IP, not the public URL), for example `http://10.0.0.91:13378`.
3. Paste an **Audiobookshelf API Token** (Audiobookshelf **Settings, Users, your user, API
Token**). Read access is enough.
4. Leave **Audiobookshelf Library IDs** blank to check every book library, or list the IDs
to check (comma separated).
5. Press **Test Audiobookshelf library**. It reports how many items it indexed, using the
values on the form, so it works before you save.
Audiobookshelf titles are often raw folder names such as `Alex Cross 02: Kiss the Girls`.
When the server reports a series for the item, the leading label and number are read as the
series, not as a different book, so a search for `Kiss the Girls` still matches.
## How a match is decided
In order of confidence:
@@ -32,12 +52,20 @@ In order of confidence:
## Behaviour worth knowing
- **It fails open.** If the database cannot be read, Shelfmark logs a warning, reuses the
last successful read if it has one, and otherwise treats the book as not owned. A broken
path degrades the badge, it never blocks a search.
Sequels and other near-miss titles fail the title rule; the same title by a different author fails the surname rule. A series label in front of a library title ("Alex Cross 02: Kiss the Girls", "HHG 5: Mostly Harmless") is read as the series, not as part of the title, when the entry is in a series, so renaming a shelf to show reading order doesn't make its books look unowned. A shelf title whose bracketed words name another edition ("Full Cast Edition", "Dramatized Adaptation") is not the book either: owning the adaptation is not owning the recording asked for. Printing words ("Illustrated Edition", "Unabridged") change nothing, and in a Calibre title a numbered parenthetical ("(Alex Cross Series #11)") is read as the series, not as an edition.
- **It fails open for badges.** If a library cannot be read, Shelfmark logs a warning, reuses
the last successful read if it has one, and otherwise treats the book as not owned. A
broken path degrades the badge, it never blocks a search.
- **It fails closed for anything that acts on the answer.** The Hardcover sync and automatic
downloads would otherwise request or download books the library already holds, so when an
enabled library cannot be read and has not been read before, the sync cycle is skipped and
the request stays pending until the next one. A library that listed items a moment ago and
now lists none is treated as unreadable too, not as an emptied shelf.
- **Results are cached** for ten minutes, and refreshed early when the database file
changes, so a large library costs one read rather than one per search.
- **Ebooks only.** A Calibre library holds ebooks, so the badge answers for ebooks. The
provider interface in `shelfmark/core/library_providers/` takes more libraries: add a
- **Each library answers for its own format.** Calibre holds ebooks and Audiobookshelf holds
audiobooks, so an ebook result is only checked against Calibre and an audiobook against
Audiobookshelf. The provider interface in `shelfmark/core/library_providers/` takes more libraries: add a
module with the `LibraryProvider` shape and list it in `all_providers()`. Nothing above
Keep the read timeout at least as high as `RELEASE_SEARCH_TIMEOUT` (300 seconds by default). Nginx's own default is 60 seconds, which cuts off any release search that has to solve a protection challenge first. See [Searches that time out behind a proxy](#searches-that-time-out-behind-a-proxy).
## Subpath setup
Running Shelfmark under a subpath like `/shelfmark` is supported without extra rewrite rules.
@@ -164,6 +168,14 @@ If login, settings saves, or downloads appear to fail in the browser but the act
This is especially relevant for Nginx Proxy Manager or custom advanced config snippets that add websocket headers globally.
## Searches that time out behind a proxy
A release search can legitimately take minutes. When a source's protection cookies expire, the next search has to solve a browser challenge before it can run. Shelfmark gives a search up to `RELEASE_SEARCH_TIMEOUT` seconds (300 by default) and says why when it gives up, but a proxy with a shorter read timeout cuts the request off first. The search then fails with a message about the reverse proxy, or, on older versions, with "Unable to reach download source. Network may be restricted or mirrors blocked." A retry often works because the solve finished on the server after the proxy stopped waiting.
- Set the proxy's read timeout to at least `RELEASE_SEARCH_TIMEOUT`. In Nginx and Nginx Proxy Manager (the **Advanced** tab of the proxy host), add the `proxy_read_timeout` and `proxy_send_timeout` lines from the examples above.
- Cloudflare's proxy gives up after 100 seconds with a `524` error, and only Enterprise plans can raise that. If searches outlast it, reach Shelfmark without going through Cloudflare's proxy, for example on your LAN or over a VPN.
- To confirm the proxy is the cause, run the same search against Shelfmark directly at `http://<host>:8084`. If it succeeds there, or fails with a specific reason, the proxy was cutting it off.
Start with the logs, then attach a debug bundle if you open an issue.
## Logs
Shelfmark writes its log to the container output, so `docker logs shelfmark` shows it. With `ENABLE_LOGGING` on (the default) it also writes log files under `LOG_ROOT/shelfmark/` (`/var/log/shelfmark/` by default), including `shelfmark.log` and the startup logs.
`LOG_LEVEL` sets how much is written (`INFO` by default). Set it to `DEBUG` to see what a search or download is doing. See the [Environment Variables](environment-variables.md) reference for both settings.
## Debug bundle
Set `DEBUG=true` on the container and restart it. A **DEBUG** button then appears in the interface. It generates a zip with logs, configuration and environment details, with sensitive values redacted. Have a look through it before you post it publicly, then attach it when you open an issue.
The full image also records the browser's screen during a bypass while Debug mode is on. ffmpeg isn't in the image, so the container installs it in the background at startup, which needs network access. Bypasses that run before it finishes aren't recorded. The log says when the install succeeds, fails or gives up after 120 seconds. In non-root mode it can't install anything, so the log warns that bypasses won't be recorded and everything else works as normal.
## Reporting an issue
Open an issue on [GitHub](https://github.com/calibrain/shelfmark/issues) and fill in the bug report template. A description of what you expected, the steps to reproduce it and the debug bundle are what make a problem findable.
@@ -61,12 +61,18 @@ Override how a user searches, on top of the global search defaults:
Users can configure personal notification routes, separate from the global notification settings. Each route targets a URL (e.g. an Apprise-compatible endpoint) and can be scoped to specific event types or all events.
Route URLs can use `{User}` and `{Email}` in their path or query, filled in from each user's account when a notification is sent. A route is skipped for users without the value it needs, or whose value contains spaces or any of `, ; / \ "` (or is only dots, such as `..`), since those would split it into several targets or change the URL path. The admin-set default routes apply to every user who hasn't set routes of their own and can use any service, so one default such as `mailtos://user:pass@smtp.example.com?to={Email}` reaches each user at their own address. The service a default points to then receives each user's email address, so only use one you trust with it. Routes a user sets themselves stay limited to the personal-route services, e.g. `ntfys://ntfy.sh/shelfmark-{User}`.
### Request Policy (admin-only)
Admins can override the default ebook/audiobook modes and request rules for individual users. See [Per-User Overrides](#per-user-overrides) below.
---
## Clearing the activity list
**Clear Completed** in the activity list clears finished downloads and their requests, and also finished requests that never had a download, such as a request closed because the book was already in the library. Pending and rejected requests are left alone.
## Requests
The request system controls whether users can download directly or need admin approval first.
@@ -92,6 +98,7 @@ Each content type (ebook, audiobook) has a default mode that sets the baseline:
| Max Pending Requests Per User | Open request limit per user | 20 |
| Allow Notes on Requests | Let users attach a note when submitting | On |
| Allow book requests alongside downloads | Let users who can download also request a book without choosing a release, for a book nobody has released yet or one no source has. The request waits for an admin, and the user gets the Requests tab to follow or cancel it. Admins can always do this | Off |
### Request Rules
@@ -99,6 +106,8 @@ The rules matrix lets you override the mode for specific source + content type c
For example, if the default ebook mode is "Download", a rule can restrict a specific source to "Request Release" or "Blocked", but not the other way around. If no rule matches, the content-type default applies.
Rules cannot use "Request Book". Sources whose browse results are already concrete releases are also treated as "Request Release" when the default mode is "Request Book", since there is no book-level request to make for them.
### Per-User Overrides
Admins can override the default ebook/audiobook modes and request rules for individual users. Per-user rules are overlaid on the global rules, not replacing them.
@@ -109,6 +118,16 @@ Admins can override the default ebook/audiobook modes and request rules for indi
2. Request appears in the admin request queue as **pending**
3. Admin either **fulfils** (queues a download) or **rejects** the request
4. For fulfilled requests, delivery state is tracked through the download pipeline
5. If delivery fails, an admin can reopen the request to try a different release
5. If delivery fails and the download cannot be retried, the request goes back to pending automatically so an admin can approve it again with a different release. An admin can also reopen a rejected request
6. Users can cancel their own pending requests
7. An admin can **reopen** a rejected request, from the Requests tab or from History. It goes back to pending with what the user originally asked for, and shows in the activity list again even if the user had dismissed it.
7. An admin can **reopen** a rejected request, from the Requests tab or from History. It goes back to pending with what the user originally asked for, and shows in the activity list again even if the user had dismissed it. A fulfilled request whose download failed or was cancelled can be reopened the same way: it goes back to pending without its release, and the next automatic search skips the release that failed (an admin can still pick it by hand).
An admin sees who declined a request and when on its card, with the admin note if one was given.
### Rejected view (this fork)
A button next to History opens **Rejected**: every rejected request, including ones cleared from the activity list, each with **Reopen**. A card can be hidden from this view, and **Show hidden (N)** in its header brings hidden ones back, so a request is never out of reach. Hiding here is separate from clearing a card in Activity.
Every card shows the format as a coloured pill: **Audiobook** or **Ebook** on a request, or the file format (EPUB, M4B, MP3) once a release is chosen or downloaded. Audio is violet and ebooks are orange.
Admin actions taken by scripts or automation carry an admin note starting `[auto]`, for example `[auto] Already in the library.`, so they can be told apart from decisions a person made.
@@ -17,11 +17,13 @@ Works great alongside the following library tools, with support for automatic im
## ✨ Features
- **One-Stop Interface** - A clean, modern UI to search, browse, and download from multiple configured sources in one place
- **Multiple Sources** - Configurable web, torrent, usenet, IRC, and Soulseek (slskd) source support
- **Multiple Sources** - Configurable web (including OceanofPDF), torrent, usenet, IRC, Soulseek (slskd), and Webshare.cz source support, with indexers through Prowlarr or Jackett
- **Audiobook Support** - Full audiobook search and download with dedicated processing
- **Flexible Search** - Search metadata providers (Hardcover, Open Library, Google Books) for rich book and audiobook discovery, or query configured sources directly
- **Flexible Search** - Search metadata providers (Hardcover, Open Library, Google Books, DNB, Databáze knih, Moly.hu) for rich book and audiobook discovery, or query configured sources directly
- **Multi-User & Requests** - Share your instance with others, let users browse and request books, and manage approvals with configurable notifications
- **Authentication** - Built-in login, OIDC single sign-on, proxy auth, and Calibre-Web database support
- **Library Check** - Optionally mark search results you already own in a Calibre or Audiobookshelf library, so you don't download a second copy. See [Library Check](docs/library-check.md)
- **Hardcover Sync** - Optionally pull a Hardcover shelf in as requests and download the books without an admin picking a release. Off by default. See [Hardcover Sync](docs/hardcover-sync.md)
- **Real-Time Progress** - Unified download queue with live status updates across all sources
- **Network Flexibility** - Configurable proxy support, DNS settings, and optional Cloudflare handling for protected sources
@@ -91,7 +93,7 @@ volumes:
- Queries configured sources directly
**Universal** (recommended)
- Search via metadata providers (Hardcover, Open Library, Google Books) for richer results
- Search via metadata providers (Hardcover, Open Library, Google Books, DNB, Databáze knih, Moly.hu) for richer results
- Aggregates releases from multiple configured sources
- Full audiobook support
@@ -145,16 +147,18 @@ Environment variables work for initial setup and Docker deployments. They serve
See the full [Environment Variables Reference](docs/environment-variables.md) for all available options.
Some of the additional options available in Settings:
- **Prowlarr** - Configure indexers and download clients to download books and audiobooks
- **Prowlarr / Jackett** - Configure indexers to download books and audiobooks through your torrent and usenet clients
- **Download clients** - qBittorrent, Transmission, Deluge, rTorrent, NZBGet, SABnzbd, a blackhole folder, or a debrid service (AllDebrid, Real-Debrid, TorBox, Debrid-Link)
- **Webshare.cz** - Search Webshare.cz by file name and download ebooks and audiobooks with your Webshare account (a VIP account is recommended for full speed)
- **Direct Download mirrors** - Supply your own Anna's Archive mirror URLs; Auto mode tries them in the order listed. The `annas-archive.is` domain does not currently work as a source — use `annas-archive.gl` instead (checked August 2026; mirror availability changes)
- **IRC** - Add details for IRC book sources and download directly from the UI. Most networks serve audiobooks from the same channel as ebooks (on `irc.irchighway.net` that's `#ebooks`, while `#bookz` is effectively inactive), so leave the separate audiobook channel blank unless your network actually indexes one. IRC audiobooks usually arrive as ZIP/RAR archives — keep those enabled under Supported Audiobook Formats or the releases are filtered out of results
- **Soulseek (slskd)** - Search the Soulseek network and download through a [slskd](https://github.com/slskd/slskd) instance. Point Shelfmark at slskd's URL with an API key (`web.authentication.api_keys` in `slskd.yml`, role `readwrite`), and mount slskd's downloads folder into the Shelfmark container. If the folder sits at a different path inside Shelfmark, set **Downloads Path** in the slskd settings or add a Remote Path Mapping for client `slskd`. Ebook results are single files; audiobook results group a peer's folder of audio files into one release
- **Library Link** - Add a link to your Calibre-Web or Grimmory instance in the UI header
- **Soulseek (slskd)** - Search the Soulseek network and download through a [slskd](https://github.com/slskd/slskd) instance. Point Shelfmark at slskd's URL with an API key (`web.authentication.api_keys` in `slskd.yml`, role `readwrite`), and mount slskd's downloads folder into the Shelfmark container. If the folder sits at a different path inside Shelfmark, set **Downloads Path** in the slskd settings or add a Remote Path Mapping for client `slskd`. Ebook results are single files; audiobook results group a peer's folder of audio files into one release. Some uploaders block downloads until you type a word to them in a private message; turn on **Answer anti-leech chat challenges** and Shelfmark types it for you, at most once per user every 12 hours. It reads every unread private message in slskd and marks a conversation it answered as read, so leave it off if you also read slskd's messages yourself
- **Library Link** - Add links to your book library (Calibre-Web or Grimmory) and your audiobook library (Audiobookshelf, Plex) in the UI header
- **File processing** - Customiseable download paths, file renaming and directory creation with template-based renaming
- **Network Settings** - Custom proxy support (SOCKS5 + HTTP/S) and configurable DNS
- **Format & Language** - Filter downloads by preferred formats, languages and sorting order
- **Metadata Providers** - Configure API keys for Hardcover, Open Library, etc.
- **Metadata Providers** - Configure API keys for Hardcover, Google Books and the other metadata providers
## 🐳 Docker Variants
@@ -253,14 +257,14 @@ See [API Access](docs/api-access.md) to call the API with a static key from scri
## Project Scope
Shelfmark is a manual search and download tool, the entry point to your book library, not a library manager. It finds books, downloads them, and sends them to a configured destination. That's the full scope.
Shelfmark is a manual search and download tool, the entry point to your book library, not a library manager. It finds books, downloads them, and sends them to a configured destination. That's the core scope.
Shelfmark intentionally does not:
- **Track or manage your library** - it doesn't know or care what you already own
- **Integrate with library software** - what happens after delivery is up to your library tool
- **Monitor authors, series, or new releases** - there is no background automation
- **Queue future downloads** - if a book isn't available now, Shelfmark won't watch for it
- **Manage your library** - it never edits, tags or reorganises what you already own. The optional [Library Check](docs/library-check.md) only reads your Calibre or Audiobookshelf library to mark results you already have
- **Integrate with library software beyond delivery** - what happens after delivery is up to your library tool
- **Monitor authors, series, or new releases** - there is no watching for new books. The optional [Hardcover Sync](docs/hardcover-sync.md) is the one background job: it turns a Hardcover shelf you choose into requests, and it is off by default
- **Queue future downloads** - if a book isn't available now, Shelfmark won't watch for it. Hardcover Sync retries a request on a schedule, which is the closest it gets
These are non-goals, not missing features.
@@ -284,6 +288,8 @@ healthcheck:
retries: 3
```
The endpoint answers 503 when the download coordinator, the part that picks queued downloads up, is not running. Without it every download stays queued, so a container that reports unhealthy for this reason needs a restart.
## Logging
Logs are available via:
@@ -318,6 +324,20 @@ make restart # Restart container
The frontend dev server proxies to the backend on port 8084.
## Sponsors
Code review on Shelfmark's pull requests is sponsored by [CodeRabbit](https://coderabbit.link/calibrain).
"description":"Optional second API key for dashboards and monitors. Send it like SHELFMARK_API_KEY. It can only GET /api/stats (counters only, no titles or user names). Every other request, and any write, is refused with 403, and it never gets a session. Unset = off.",
"type":"string",
"default":"unset",
},
{
"name":"OIDC_AUTO_REDIRECT",
"description":"Automatically redirect to the OIDC provider instead of showing the login page.",
"type":"boolean",
"default":"false",
},
{
"name":"UMASK",
"description":"File creation mask for the app process, read by entrypoint.sh. The default 0022 gives files 644 and directories 755.",
"type":"string (octal)",
"default":"0022",
},
{
"name":"DOCKERMODE",
"description":"Indicates the application is running inside a Docker container.",
"description":"Seconds between Tor watchdog probes. The watchdog resets the container when Tor stops being usable, so the restart policy rebuilds the Tor setup.",
"type":"integer (seconds)",
"default":"30",
},
{
"name":"TOR_MAX_FAILURES",
"description":"Consecutive failed Tor watchdog probes before the container is reset. 2 tolerates one failed probe.",
"type":"integer",
"default":"2",
},
]
lines=[
"## Egress / VPN Routing",
"",
"These startup-only variables are consumed by `entrypoint.sh` / `wireguard.sh` to select and configure the WireGuard transparent-egress kill-switch. `USING_WIREGUARD` and [`USING_TOR`](#using_tor) (documented under Network) are mutually exclusive; both require root startup.",
"These startup-only variables are consumed by `entrypoint.sh` / `wireguard.sh` to select and configure the WireGuard transparent-egress kill-switch. `USING_WIREGUARD` and [`USING_TOR`](#using_tor) (documented under Network) are mutually exclusive; both require root startup. `TOR_CHECK_INTERVAL` and `TOR_MAX_FAILURES` tune the Tor watchdog in `tor.sh`.",
"UPDATE torrent_ledger SET task_id = ? WHERE hash = ?",
(task_id,torrent_hash.lower()),
)
except(sqlite3.Error,OSError)asexc:
logger.warning("Could not update torrent %s in the ledger: %s",torrent_hash,exc)
defentries()->dict[str,str]:
"""Every recorded hash with the id of the task that last owned it."""
try:
withclosing(_connect())asconn:
return{
str(row[0]):str(row[1])
forrowinconn.execute("SELECT hash, task_id FROM torrent_ledger")
}
except(sqlite3.Error,OSError)asexc:
logger.warning("Could not read the torrent ledger: %s",exc)
return{}
defforget(*torrent_hashes:str)->None:
"""Drop rows, once their torrent has been removed or is gone from the client."""
hashes=[(h.lower(),)forhintorrent_hashesifh]
ifnothashes:
return
try:
withclosing(_connect())asconn,conn:
conn.executemany("DELETE FROM torrent_ledger WHERE hash = ?",hashes)
except(sqlite3.Error,OSError)asexc:
logger.warning("Could not update the torrent ledger: %s",exc)
Loaded 100 of 303 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Kris AndersonandClaude Sonnet 5.5
splitsec2
CaliBrain
dependabot[bot]
Matt Van HornandMatt Van Horn
Rupraj Datta
Josh Theisen
misko903andClaude Opus 5.5
ProcyonX1
Amalkrishna A SandCaliBrain
