fix: review regressions in metadata, notifications, activity redaction, search options and genDebug (#1510)

- **Open Library (#1451): keep the work unless the search matched a
translation.** Open Library shows an English edition whenever one
matches the query. So "Mistborn" returned the 2023 "Mistborn" reissue
(2023) instead of "The Final Empire" (2001), and "Children of Dune"
returned "Children of Dune (Dune Chronicles, Book 3)" (1987). The
matched edition's title, ISBNs, cover, language and year now win only
when the edition is in another language and the query doesn't name the
work's own title. Checked against live Open Library: "Children of Dune"
and "Mistborn" give the v1.4.0 results again, and "Der Weg der Könige"
and "Kinder des Nebels" still return the German editions.
- **Google Books (#1434): the ISBN cache decorator is back on
`search_by_isbn`.** It had moved to the new `search_paginated`. Failed
searches were cached as empty pages, and ISBN lookups lost their cache.
- **Notifications: Test notification shows the plugin's failure reason
again.** Apprise 2 delivers on its own worker threads, even for a single
service when that service has a deadline. `_AppriseLogCapture` filtered
records by thread id and dropped all of them, so the UI only said "json:
delivery failed". Apprise runs those workers in a copy of the caller's
context, so the capture now filters on a context variable. That still
keeps concurrent dispatches apart. `AppriseAsset(async_mode=False)`
would not fix this, because the sequential path also uses the worker
pool when a service has a deadline.
- **Activity: `activity_update` no longer sends the raw task id to
non-admins.** The owner's `user_<id>` room got the raw task id. For a
request-linked download that id can hold a private tracker's passkey. It
now goes through `viewer_download_id`, the same mapping `/api/status`
and `download_progress` use. The admins room still gets the raw id.
- **SearchBar (#1476): Escape and an outside click close the homepage's
inline options panel again,** as they did in v1.4.0.
- **genDebug.sh: packages are listed through `importlib.metadata`,**
since #1404 removed pip from the image. The script also now redacts:
  - `PROWLARR_MAM_ID`
- `user:pass@` credentials in URLs (proxies), in both the environment
and the config files
- notification route URLs, which carry their service's token (Telegram,
Discord and others)

- `make python-checks`, `make python-test`, `make frontend-checks` and
`make frontend-test` pass on top of current `main`.
- New tests cover:
  - Open Library edition selection
  - Google Books caching
- log capture against a real Apprise 2 delivery to a local server that
answers 500, plus a concurrent dispatch on another thread
  - the `activity_update` payload sent to each room
  - genDebug redaction and the package listing

  Each one fails on the old code.
- I checked the SearchBar fix in the running app, because the frontend
has no DOM test setup: Escape closes the panel, an outside click closes
it, and a click inside keeps it open.
- I ran the genDebug redaction under GNU sed (Debian image) and BSD sed.
The package listing ran in the pip-less image.
This commit is contained in:
CaliBrain authored and GitHub committed 2026-10-21 01:58:41 -04:00
1 parent a00f3d28bf
commit c269230ef3
11 files changed
+580 -54

No files matched your search

+22 -8
View File
@@ -91,8 +91,14 @@ echo | openssl s_client -connect cloudflare.com:443 2>&1 | grep -E "Verify retur
echo "" >> "$LOG_DIR/network_info.txt"
# Add installed packages
# The image ships without pip, so ask the app's interpreter directly
echo "=== Installed Python Packages ===" > "$LOG_DIR/packages.txt"
pip list >> "$LOG_DIR/packages.txt" 2>&1 || echo "pip not found" >> "$LOG_DIR/packages.txt"
python - >> "$LOG_DIR/packages.txt" 2>&1 <<'PY' || echo "Could not list Python packages" >> "$LOG_DIR/packages.txt"
import importlib.metadata as metadata
for dist in sorted(metadata.distributions(), key=lambda dist: (dist.name or "").lower()):
print(f"{dist.name}=={dist.version}")
PY
echo "" >> "$LOG_DIR/packages.txt"
# Check Permissions
@@ -133,8 +139,19 @@ else
echo "Not running in Docker container" >> "$LOG_DIR/container_info.txt"
fi
# Add environment variables (redacting sensitive info)
env | grep -v -E "^[^=]*(_KEY|_SECRET|_PASSWORD|_TOKEN)[^=]*=" | sort > "$LOG_DIR/environment.txt"
# Add environment variables (redacting sensitive info and the credentials in URLs
# such as a proxy's http://user:pass@host:port)
env | grep -v -E "^[^=]*(_KEY|_SECRET|_PASSWORD|_TOKEN|_MAM_ID)[^=]*=" | sed -E 's#(://)[^/@[:space:]]+@#\1[REDACTED]@#g' | sort > "$LOG_DIR/environment.txt"
# Redact sensitive fields (API keys, passwords, tokens, the MAM session id), the
# credentials in URLs, and notification URLs, which carry their service's token
redact_config_json() {
sed -E \
-e 's/("(AA_DONATOR_KEY|HARDCOVER_API_KEY|[^"]*_KEY|[^"]*_SECRET|[^"]*_PASSWORD|[^"]*_TOKEN|[^"]*_MAM_ID)"[[:space:]]*:[[:space:]]*")[^"]+"/\1[REDACTED]"/g' \
-e 's#(://)[^/@[:space:]"]+@#\1[REDACTED]@#g' \
-e 's#("url"[[:space:]]*:[[:space:]]*"([[:alnum:]+.-]+://)?)[^"]+"#\1[REDACTED]"#g' \
"$1"
}
# Add configuration files (redacting sensitive values)
CONFIG_DIR=${CONFIG_DIR:-"/config"}
@@ -143,9 +160,7 @@ if [ -d "$CONFIG_DIR" ]; then
# Copy and redact main settings file
if [ -f "$CONFIG_DIR/settings.json" ]; then
# Redact sensitive fields (API keys, passwords, tokens)
sed -E 's/("(AA_DONATOR_KEY|HARDCOVER_API_KEY|[^"]*_KEY|[^"]*_SECRET|[^"]*_PASSWORD|[^"]*_TOKEN)"[[:space:]]*:[[:space:]]*")[^"]+"/\1[REDACTED]"/g' \
"$CONFIG_DIR/settings.json" > "$LOG_DIR/config/settings.json" 2>/dev/null
redact_config_json "$CONFIG_DIR/settings.json" > "$LOG_DIR/config/settings.json" 2>/dev/null
fi
# Copy and redact plugin config files
@@ -154,8 +169,7 @@ if [ -d "$CONFIG_DIR" ]; then
for config_file in "$CONFIG_DIR/plugins"/*.json; do
if [ -f "$config_file" ]; then
filename=$(basename "$config_file")
sed -E 's/("(AA_DONATOR_KEY|HARDCOVER_API_KEY|[^"]*_KEY|[^"]*_SECRET|[^"]*_PASSWORD|[^"]*_TOKEN)"[[:space:]]*:[[:space:]]*")[^"]+"/\1[REDACTED]"/g' \
"$config_file" > "$LOG_DIR/config/plugins/$filename" 2>/dev/null
redact_config_json "$config_file" > "$LOG_DIR/config/plugins/$filename" 2>/dev/null
fi
done
fi
+15 -4
View File
@@ -8,6 +8,7 @@ import socket
import threading
from concurrent.futures import ThreadPoolExecutor
from contextlib import contextmanager, suppress
from contextvars import ContextVar
from dataclasses import dataclass
from enum import StrEnum
from typing import TYPE_CHECKING, Any, Protocol, TypeGuard
@@ -135,14 +136,22 @@ def _extract_url_schemes(urls: Iterable[str]) -> list[str]:
return schemes
# The capture active in the current context. Apprise 2 delivers on its own worker
# threads but runs them in a copy of the caller's context, so a context variable
# (unlike the thread id) still tells this dispatch's records apart from those of
# a dispatch running concurrently on another thread.
_active_apprise_capture: ContextVar[_AppriseLogCapture | None] = ContextVar(
"active_apprise_capture", default=None
)
class _AppriseLogCapture(logging.Handler):
def __init__(self, *, thread_id: int) -> None:
def __init__(self) -> None:
super().__init__(level=logging.INFO)
self.records: list[tuple[int, str, str, str]] = []
self._thread_id = thread_id
def emit(self, record: logging.LogRecord) -> None:
if record.thread != self._thread_id:
if _active_apprise_capture.get() is not self:
return
message = record.getMessage()
@@ -164,7 +173,8 @@ def _capture_apprise_logs(
) -> Iterator[list[tuple[int, str, str, str]]]:
apprise_logger = logging.getLogger(_APPRISE_LOGGER_NAME)
previous_level = apprise_logger.level
handler = _AppriseLogCapture(thread_id=threading.get_ident())
handler = _AppriseLogCapture()
token = _active_apprise_capture.set(handler)
apprise_logger.addHandler(handler)
if previous_level == logging.NOTSET or previous_level > min_level:
@@ -175,6 +185,7 @@ def _capture_apprise_logs(
finally:
apprise_logger.removeHandler(handler)
apprise_logger.setLevel(previous_level)
_active_apprise_capture.reset(token)
def _log_apprise_records(records: Iterable[tuple[int, str, str, str]]) -> None:
+8
View File
@@ -1622,6 +1622,14 @@ def _emit_activity_update_for_task(*, payload: dict[str, Any], task: Any) -> Non
)
if owner_user_id is None:
return
# The owner knows a request-linked download by its opaque id, as /api/status
# shows it, since the task id can carry a private tracker's passkey (#1418).
task_id = payload.get("task_id")
if isinstance(task_id, str):
payload = {
**payload,
"task_id": viewer_download_id(task_id, getattr(task, "request_id", None)),
}
emit_ws_event(
ws_manager,
event_name="activity_update",
+4 -4
View File
@@ -214,15 +214,15 @@ class GoogleBooksProvider(MetadataProvider):
logger.exception("Google Books get_book error")
return None
def search_paginated(self, options: MetadataSearchOptions) -> SearchResult:
"""Google returns at most 40 per page, so a full page is 40 whatever limit asked for."""
return super().search_paginated(replace(options, limit=min(options.limit, 40)))
@cacheable(
ttl_key="METADATA_CACHE_BOOK_TTL",
ttl_default=600,
key_prefix="googlebooks:isbn",
)
def search_paginated(self, options: MetadataSearchOptions) -> SearchResult:
"""Google returns at most 40 per page, so a full page is 40 whatever limit asked for."""
return super().search_paginated(replace(options, limit=min(options.limit, 40)))
def search_by_isbn(self, isbn: str) -> BookMetadata | None:
"""Search for a book by ISBN-10 or ISBN-13."""
# Clean ISBN (remove hyphens and spaces)
+69 -33
View File
@@ -3,6 +3,7 @@
import re
import threading
import time
import unicodedata
from collections import deque
from http import HTTPStatus
from typing import Any, ClassVar
@@ -10,6 +11,7 @@ from typing import Any, ClassVar
import requests
from shelfmark.core.cache import cacheable
from shelfmark.core.languages import normalize_language
from shelfmark.core.logger import setup_logger
from shelfmark.core.settings_registry import (
ActionButton,
@@ -19,6 +21,7 @@ from shelfmark.core.settings_registry import (
SettingsField,
register_settings,
)
from shelfmark.core.text_match import STOPWORDS
from shelfmark.download.network import get_ssl_verify
from shelfmark.metadata_providers import (
BookMetadata,
@@ -103,6 +106,41 @@ SORT_MAPPING: dict[str, str | None] = {
}
def _title_words(text: str) -> set[str]:
"""Return the significant words of a title, folded for comparison."""
decomposed = unicodedata.normalize("NFKD", text.casefold())
folded = "".join(ch for ch in decomposed if not unicodedata.combining(ch))
return {word for word in re.findall(r"\w+", folded) if len(word) >= 2} - STOPWORDS
def _is_translation_match(work_title: str, edition: dict, query: str) -> bool:
"""Return whether a search's matched edition is a translation of the work.
Open Library shows the English edition of a work whenever one satisfies the
query, so an English edition says nothing about which edition the user meant:
searching "Mistborn" picks the 2023 "Mistborn" reissue of "The Final Empire",
and "Children of Dune" may pick "Children of Dune (Dune Chronicles, Book 3)".
An edition in another language comes back when only that language's edition
matched, which is what searching a translated title looks like. A `lang`
preference also brings one back for a search that names the work itself
("Dune-Zyklus" for "Children of Dune"), so such a search keeps the work.
"""
edition_title = edition.get("title")
if not edition_title or _title_words(edition_title) == _title_words(work_title):
return False
edition_languages = {normalize_language(code) for code in edition.get("language") or []}
if not edition_languages - {None} or "en" in edition_languages:
return False
query_words = _title_words(query)
work_words = _title_words(work_title)
names_work = bool(query_words and work_words) and (
work_words <= query_words or query_words <= work_words
)
return not names_work
@register_provider("openlibrary")
class OpenLibraryProvider(MetadataProvider):
"""Open Library metadata provider using REST API."""
@@ -209,7 +247,7 @@ class OpenLibraryProvider(MetadataProvider):
data = response.json()
for doc in data.get("docs", []):
book = self._parse_search_doc(doc)
book = self._parse_search_doc(doc, title_value or options.query)
if book:
books.append(book)
@@ -331,8 +369,12 @@ class OpenLibraryProvider(MetadataProvider):
logger.exception("Open Library ISBN search parsing error")
return None
def _parse_search_doc(self, doc: dict) -> BookMetadata | None:
"""Parse a search document into BookMetadata."""
def _parse_search_doc(self, doc: dict, query: str = "") -> BookMetadata | None:
"""Parse a search document into BookMetadata.
``query`` is what the user searched for (the title field when given), used
to tell a search for a translated title apart from one for the work itself.
"""
try:
# Extract work ID from key
key = doc.get("key", "")
@@ -372,44 +414,38 @@ class OpenLibraryProvider(MetadataProvider):
# matched the search query (e.g. querying "Der Weg der Könige" matches
# the German edition of "The Way of Kings", but the work's title field
# still says "The Way of Kings"). The nested `editions` block, requested
# above, is populated with the specific edition whose own fields
# satisfied the query, so prefer its title/isbn/cover/language when one
# is present and differs from the work - otherwise both the result shown
# to the user and the release search query (which uses `search_title` or
# else falls back to `title`, see search_plan._pick_search_title) end up
# in the wrong language even though the user searched for a specific
# translation by name.
# above, holds the edition Open Library picked to show for the query.
# When that edition is a translation, prefer its title/isbn/cover/
# language - otherwise both the result shown to the user and the
# release search query (which uses `search_title` or else falls back to
# `title`, see search_plan._pick_search_title) end up in the wrong
# language even though the user searched for a specific translation by
# name. Any other edition keeps the work's own title and first year.
title = doc["title"]
search_title = None
publish_year = doc.get("first_publish_year")
matched_edition = next(iter(doc.get("editions", {}).get("docs", [])), None)
if matched_edition and matched_edition.get("title"):
edition_title = matched_edition["title"]
if edition_title != title:
search_title = edition_title
title = edition_title
if matched_edition and _is_translation_match(title, matched_edition, query):
search_title = matched_edition["title"]
title = search_title
edition_isbns = matched_edition.get("isbn", [])
edition_isbn_10 = next(
(i for i in edition_isbns if len(i) == ISBN_10_LENGTH), None
)
edition_isbn_13 = next(
(i for i in edition_isbns if len(i) == ISBN_13_LENGTH), None
)
isbn_10 = edition_isbn_10 or isbn_10
isbn_13 = edition_isbn_13 or isbn_13
edition_isbns = matched_edition.get("isbn", [])
edition_isbn_10 = next((i for i in edition_isbns if len(i) == ISBN_10_LENGTH), None)
edition_isbn_13 = next((i for i in edition_isbns if len(i) == ISBN_13_LENGTH), None)
isbn_10 = edition_isbn_10 or isbn_10
isbn_13 = edition_isbn_13 or isbn_13
cover_id = matched_edition.get("cover_i") or cover_id
cover_id = matched_edition.get("cover_i") or cover_id
edition_languages = matched_edition.get("language") or []
if edition_languages:
language = edition_languages[0]
edition_languages = matched_edition.get("language") or []
if edition_languages:
language = edition_languages[0]
# A translation is published later than the work it translates,
# e.g. 2015 for "Der Weg der Könige" against 2010 for the work.
edition_years = matched_edition.get("publish_year") or []
if edition_years:
publish_year = min(edition_years)
# A translation is published later than the work it translates,
# e.g. 2015 for "Der Weg der Könige" against 2010 for the work.
edition_years = matched_edition.get("publish_year") or []
if edition_years:
publish_year = min(edition_years)
cover_url = f"{COVERS_BASE_URL}/b/id/{cover_id}-L.jpg" if cover_id else None
+4 -2
View File
@@ -241,6 +241,7 @@ export const SearchBar = forwardRef<SearchBarHandle, SearchBarProps>(
const selectPanelRef = useRef<HTMLDivElement>(null);
const autocompletePanelRef = useRef<HTMLDivElement>(null);
const optionsPanelId = useId();
const optionsPanelRef = useRef<HTMLDivElement>(null);
const isPopover = optionsLayout === 'popover';
const layoutClasses = OPTIONS_LAYOUT_CLASSES[optionsLayout];
const [isOptionsOpen, setIsOptionsOpen] = useState(false);
@@ -270,7 +271,8 @@ export const SearchBar = forwardRef<SearchBarHandle, SearchBarProps>(
const combinedSelectorLocked = combinedModeLocked && !isManualTarget;
const combinedToggleAvailable = !!onCombinedModeChange && !isManualTarget;
useDismiss(isOptionsOpen && isPopover, [selectorRef], closeOptions);
// The inline panel renders below the bar, outside selectorRef.
useDismiss(isOptionsOpen, [selectorRef, optionsPanelRef], closeOptions);
useDismiss(isSelectOpen, [selectPanelRef, selectTriggerRef], () => setIsSelectOpen(false));
useDismiss(isAutocompleteOpen, [autocompletePanelRef, inputRef], () =>
setIsAutocompleteOpen(false),
@@ -628,7 +630,7 @@ export const SearchBar = forwardRef<SearchBarHandle, SearchBarProps>(
) : null;
const optionsPanel = (
<div id={optionsPanelId} className={layoutClasses.panel}>
<div id={optionsPanelId} ref={optionsPanelRef} className={layoutClasses.panel}>
{showContentTypeSelector && (
<div className={layoutClasses.group}>
<OptionsGroupHeader label="Content" action={optionsButton} />
@@ -11,6 +11,7 @@ import pytest
from shelfmark.core.models import DownloadTask, QueueStatus
from shelfmark.core.notifications import NotificationEvent
from shelfmark.core.viewer_redaction import public_download_id
@pytest.fixture(scope="module")
@@ -39,6 +40,32 @@ def _read_download_history_row(main_module, task_id: str):
conn.close()
def _create_release_request(main_module, user: dict, task_id: str) -> dict:
return main_module.user_db.create_request(
user_id=user["id"],
content_type="ebook",
request_level="release",
policy_mode="request_release",
book_data={
"title": "Requested Activity",
"author": "Activity Author",
"provider": "openlibrary",
"provider_id": "activity-req",
},
release_data={"source": "prowlarr", "source_id": task_id, "title": "Requested.epub"},
status="fulfilled",
delivery_state="queued",
)
def _activity_update_payloads(mock_emit) -> dict[str, dict]:
return {
call.kwargs["to"]: call.args[1]
for call in mock_emit.call_args_list
if call.args[0] == "activity_update"
}
class TestTerminalSnapshotCapture:
def test_complete_transition_records_direct_snapshot(self, main_module):
user = _create_user(main_module, prefix="snap-direct")
@@ -193,6 +220,35 @@ class TestTerminalSnapshotCapture:
finally:
main_module.backend.book_queue.cancel_download(task_id)
def test_activity_update_hides_request_linked_task_id_from_owner(self, main_module):
user = _create_user(main_module, prefix="snap-activity-redact")
task_id = f"7:https://tracker.example/download/1?passkey={uuid.uuid4().hex[:8]}"
request_row = _create_release_request(main_module, user, task_id)
task = DownloadTask(
task_id=task_id,
source="prowlarr",
title="Requested Activity",
user_id=user["id"],
username=user["username"],
request_id=request_row["id"],
)
assert main_module.backend.book_queue.add(task) is True
try:
with patch.object(main_module.ws_manager, "is_enabled", return_value=True):
with patch.object(main_module.ws_manager.socketio, "emit") as mock_emit:
main_module.backend.book_queue.update_status(task_id, QueueStatus.COMPLETE)
payloads = _activity_update_payloads(mock_emit)
assert payloads["admins"]["task_id"] == task_id
assert payloads[f"user_{user['id']}"] == {
"kind": "download_terminal",
"task_id": public_download_id(task_id),
"status": "complete",
}
finally:
main_module.backend.book_queue.cancel_download(task_id)
def test_error_transition_triggers_download_failed_notification(self, main_module):
user = _create_user(main_module, prefix="snap-notify-error")
task_id = f"notify-error-{uuid.uuid4().hex[:8]}"
@@ -420,6 +476,40 @@ class TestTerminalSnapshotCapture:
to=f"user_{user['id']}",
)
def test_queue_hook_reset_hides_request_linked_task_id_from_owner(self, main_module):
user = _create_user(main_module, prefix="snap-reset-redact")
task_id = f"7:https://tracker.example/download/2?passkey={uuid.uuid4().hex[:8]}"
request_row = _create_release_request(main_module, user, task_id)
main_module.activity_view_state_service.dismiss(
viewer_scope=f"user:{user['id']}",
item_type="download",
item_key=f"download:{task_id}",
)
task = SimpleNamespace(
user_id=user["id"],
username=user["username"],
request_id=request_row["id"],
source="prowlarr",
title="Requested Activity",
author="Activity Author",
format="epub",
size="1 MB",
preview=None,
content_type="ebook",
)
with patch.object(main_module.ws_manager, "is_enabled", return_value=True):
with patch.object(main_module.ws_manager.socketio, "emit") as mock_emit:
main_module._record_download_queued(task_id, task)
payloads = _activity_update_payloads(mock_emit)
assert payloads["admins"] == {"kind": "activity_reset", "task_id": task_id}
assert payloads[f"user_{user['id']}"] == {
"kind": "activity_reset",
"task_id": public_download_id(task_id),
}
def test_cancelled_transition_does_not_trigger_notification(self, main_module):
user = _create_user(main_module, prefix="snap-notify-cancel")
task_id = f"notify-cancel-{uuid.uuid4().hex[:8]}"
+84 -2
View File
@@ -4,14 +4,16 @@ People attach the debug bundle to public issues, so the filter is tested by runn
real pipeline line from the script against a controlled environment.
"""
import json
import re
import subprocess
import sys
from pathlib import Path
SCRIPT = Path(__file__).resolve().parents[2] / "genDebug.sh"
def _filtered_env(env: dict[str, str]) -> set[str]:
def _filtered_env_lines(env: dict[str, str]) -> list[str]:
line = next(
line for line in SCRIPT.read_text().splitlines() if line.startswith("env | grep -v")
)
@@ -23,7 +25,25 @@ def _filtered_env(env: dict[str, str]) -> set[str]:
text=True,
check=True,
).stdout
return {entry.split("=", 1)[0] for entry in out.splitlines()}
return out.splitlines()
def _filtered_env(env: dict[str, str]) -> set[str]:
return {entry.split("=", 1)[0] for entry in _filtered_env_lines(env)}
def _redacted_config(tmp_path: Path, settings: dict[str, object]) -> dict[str, object]:
match = re.search(r"^redact_config_json\(\) \{\n.*?^\}\n", SCRIPT.read_text(), re.M | re.S)
assert match is not None
config_file = tmp_path / "settings.json"
config_file.write_text(json.dumps(settings, indent=2))
out = subprocess.run(
["bash", "-c", f'{match.group(0)}redact_config_json "$1"', "bash", str(config_file)],
capture_output=True,
text=True,
check=True,
).stdout
return json.loads(out)
def test_secret_variables_are_left_out() -> None:
@@ -37,6 +57,7 @@ def test_secret_variables_are_left_out() -> None:
"SOME_SECRET": "e",
"DB_PASSWORD_FILE": "f",
"SERVICE_TOKEN_VALUE": "g",
"PROWLARR_MAM_ID": "h",
}
)
@@ -49,6 +70,7 @@ def test_secret_variables_are_left_out() -> None:
"SOME_SECRET",
"DB_PASSWORD_FILE",
"SERVICE_TOKEN_VALUE",
"PROWLARR_MAM_ID",
}
)
@@ -59,3 +81,63 @@ def test_ordinary_variables_are_kept() -> None:
)
assert {"TZ", "PUID", "FLASK_PORT"} <= names
def test_proxy_credentials_are_redacted() -> None:
lines = _filtered_env_lines(
{
"PATH": "/usr/bin:/bin",
"HTTP_PROXY": "http://alice:s3cret@proxy.local:3128",
"HTTPS_PROXY": "socks5://bob@10.0.0.1:1080",
"NO_PROXY": "localhost",
}
)
assert "HTTP_PROXY=http://[REDACTED]@proxy.local:3128" in lines
assert "HTTPS_PROXY=socks5://[REDACTED]@10.0.0.1:1080" in lines
assert "NO_PROXY=localhost" in lines
def test_config_secrets_are_redacted(tmp_path: Path) -> None:
redacted = _redacted_config(
tmp_path,
{
"PROWLARR_API_KEY": "key",
"PROWLARR_MAM_ID": "mam-session",
"QBITTORRENT_PASSWORD": "password",
"HTTP_PROXY": "http://alice:s3cret@proxy.local:3128",
"PROWLARR_URL": "http://prowlarr:9696",
"ADMIN_NOTIFICATION_ROUTES": [
{"event": "all", "url": "tgram://123456:bot-token/987"},
{"event": "download_failed", "url": "discord://webhook-id/webhook-token"},
],
"BOOK_LANGUAGE": ["en"],
},
)
assert redacted == {
"PROWLARR_API_KEY": "[REDACTED]",
"PROWLARR_MAM_ID": "[REDACTED]",
"QBITTORRENT_PASSWORD": "[REDACTED]",
"HTTP_PROXY": "http://[REDACTED]@proxy.local:3128",
"PROWLARR_URL": "http://prowlarr:9696",
"ADMIN_NOTIFICATION_ROUTES": [
{"event": "all", "url": "tgram://[REDACTED]"},
{"event": "download_failed", "url": "discord://[REDACTED]"},
],
"BOOK_LANGUAGE": ["en"],
}
def test_package_listing_does_not_need_pip() -> None:
"""The image ships without pip, so packages are listed through importlib.metadata."""
match = re.search(r"<<'PY'.*?\n(.*?)^PY$", SCRIPT.read_text(), re.M | re.S)
assert match is not None
out = subprocess.run(
[sys.executable, "-I", "-c", match.group(1)],
capture_output=True,
text=True,
check=True,
).stdout
assert any(line.startswith("pytest==") for line in out.splitlines())
+75
View File
@@ -1,6 +1,11 @@
"""Tests for core notification rendering and dispatch helpers."""
import contextvars
import logging
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import apprise
from shelfmark.core import notifications as notifications_module
@@ -366,6 +371,76 @@ def test_dispatch_to_apprise_logs_captured_apprise_warning_messages(monkeypatch)
)
class _ServerErrorHandler(BaseHTTPRequestHandler):
def do_POST(self):
self.send_response(500)
self.end_headers()
def log_message(self, format, *args): # noqa: A002
return
def test_dispatch_to_apprise_reports_failure_logged_on_apprise_worker_thread():
"""Apprise 2 delivers on its own worker threads, which log the plugin's reason."""
server = ThreadingHTTPServer(("127.0.0.1", 0), _ServerErrorHandler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
result = notifications_module._dispatch_to_apprise(
[f"json://127.0.0.1:{server.server_address[1]}/hook"],
title="Test",
body="Body",
notify_type=apprise.NotifyType.INFO,
)
finally:
server.shutdown()
server.server_close()
thread.join(timeout=5)
assert result["success"] is False
assert len(result["details"]) == 1
assert result["details"][0].startswith("json: ")
assert "error=500" in result["details"][0]
def test_dispatch_to_apprise_ignores_apprise_records_of_other_dispatches(monkeypatch):
fake_apprise = _FakeAppriseModule()
monkeypatch.setattr(notifications_module, "apprise", fake_apprise)
def _log_warning(message):
logging.getLogger("apprise.plugins.json").warning(message)
def _notify(**kwargs):
_ = kwargs
# A dispatch running concurrently on another thread, in its own context.
other = threading.Thread(
target=_log_warning,
args=("Another dispatch failed",),
context=contextvars.Context(),
)
other.start()
other.join()
# Apprise's worker thread, running in a copy of this dispatch's context.
worker = threading.Thread(
target=contextvars.copy_context().run,
args=(_log_warning, "This dispatch failed"),
)
worker.start()
worker.join()
return False
monkeypatch.setattr(fake_apprise.client, "notify", _notify)
result = notifications_module._dispatch_to_apprise(
["json://example.invalid/hook"],
title="Test",
body="Body",
notify_type=_FakeNotifyType.INFO,
)
assert result["details"] == ["json: apprise.plugins.json: This dispatch failed"]
def test_dispatch_to_apprise_logs_add_exception_at_debug_with_trace(monkeypatch):
fake_apprise = _FakeAppriseModule()
fake_apprise.client.instantiate_exceptions_by_url = {
+30
View File
@@ -63,6 +63,36 @@ def test_googlebooks_search_does_not_cache_request_failures():
assert [book.title for book in result] == ["Recovered Book"]
def test_googlebooks_paginated_search_does_not_cache_request_failures():
get_metadata_cache().clear()
provider = GoogleBooksProvider(api_key="test-key")
session = _FlakyGoogleBooksSession()
provider.session = session
options = MetadataSearchOptions(query="Recovered Book")
assert provider.search_paginated(options).books == []
result = provider.search_paginated(options)
assert session.calls == 2
assert [book.title for book in result.books] == ["Recovered Book"]
def test_googlebooks_isbn_lookup_is_cached():
get_metadata_cache().clear()
provider = GoogleBooksProvider(api_key="test-key")
session = _FlakyGoogleBooksSession()
session.calls = 1 # skip the simulated timeout
provider.session = session
first = provider.search_by_isbn("978-0-00-000000-2")
second = provider.search_by_isbn("978-0-00-000000-2")
assert first is not None
assert second == first
assert session.calls == 2
def test_googlebooks_pages_tile_when_limit_exceeds_api_maximum():
get_metadata_cache().clear()
provider = GoogleBooksProvider(api_key="test-key")
+179 -1
View File
@@ -61,7 +61,13 @@ class TestOpenLibraryParseSearchDoc:
"author_name": ["Brandon Sanderson"],
"first_publish_year": 2010,
"editions": {
"docs": [{"key": "/books/OL32771812M", "title": "Der Weg der Könige"}]
"docs": [
{
"key": "/books/OL32771812M",
"title": "Der Weg der Könige",
"language": ["ger"],
}
]
},
}
)
@@ -116,3 +122,175 @@ class TestOpenLibraryParseSearchDoc:
assert result is not None
assert result.search_title is None
assert result.isbn_13 == "9780765326355"
def test_keeps_work_for_an_english_edition_with_another_title(self):
"""Open Library shows an English edition whenever one matches the query.
Searching "Mistborn" picks the 2023 "Mistborn" reissue of "The Final
Empire", which is the same book under a different English title, so the
result keeps the work's title, ISBNs and first publication year.
"""
provider = OpenLibraryProvider()
result = provider._parse_search_doc(
{
"key": "/works/OL5738148W",
"title": "The Final Empire",
"author_name": ["Brandon Sanderson"],
"isbn": ["9780765311788"],
"cover_i": 14658160,
"language": ["fre", "ger", "eng"],
"first_publish_year": 2001,
"editions": {
"docs": [
{
"key": "/books/OL62594701M",
"title": "Mistborn",
"language": ["eng"],
"isbn": ["9781250318541"],
"cover_i": 1,
"publish_year": [2023],
}
]
},
},
"Mistborn",
)
assert result is not None
assert result.title == "The Final Empire"
assert result.search_title is None
assert result.publish_year == 2001
assert result.isbn_13 == "9780765311788"
assert result.cover_url == "https://covers.openlibrary.org/b/id/14658160-L.jpg"
def test_keeps_work_for_a_subtitled_reprint(self):
provider = OpenLibraryProvider()
result = provider._parse_search_doc(
{
"key": "/works/OL893516W",
"title": "Children of Dune",
"author_name": ["Frank Herbert"],
"first_publish_year": 1976,
"editions": {
"docs": [
{
"key": "/books/OL1M",
"title": "Children of Dune (Dune Chronicles, Book 3)",
"language": ["eng"],
"publish_year": [1987],
}
]
},
},
"Children of Dune",
)
assert result is not None
assert result.title == "Children of Dune"
assert result.search_title is None
assert result.publish_year == 1976
def test_keeps_work_when_the_query_names_the_work(self):
"""A `lang` preference shows a translation even for the work's own title."""
provider = OpenLibraryProvider()
result = provider._parse_search_doc(
{
"key": "/works/OL893516W",
"title": "Children of Dune",
"author_name": ["Frank Herbert"],
"first_publish_year": 1976,
"editions": {
"docs": [
{
"key": "/books/OL2M",
"title": "Dune-Zyklus",
"language": ["ger"],
"publish_year": [1985],
}
]
},
},
"children of dune",
)
assert result is not None
assert result.title == "Children of Dune"
assert result.search_title is None
assert result.publish_year == 1976
def test_keeps_work_for_an_edition_of_unknown_language(self):
provider = OpenLibraryProvider()
result = provider._parse_search_doc(
{
"key": "/works/OL5738148W",
"title": "The Final Empire",
"author_name": ["Brandon Sanderson"],
"first_publish_year": 2001,
"editions": {
"docs": [{"key": "/books/OL3M", "title": "Mistborn", "publish_year": [2023]}]
},
},
"Mistborn",
)
assert result is not None
assert result.title == "The Final Empire"
assert result.publish_year == 2001
def test_ignores_edition_differing_only_in_case_and_accents(self):
provider = OpenLibraryProvider()
result = provider._parse_search_doc(
{
"key": "/works/OL1W",
"title": "L’étranger",
"author_name": ["Albert Camus"],
"first_publish_year": 1942,
"editions": {
"docs": [
{
"key": "/books/OL4M",
"title": "L'Etranger",
"language": ["fre"],
"publish_year": [1995],
}
]
},
}
)
assert result is not None
assert result.title == "L’étranger"
assert result.publish_year == 1942
def test_prefers_translation_matched_by_query(self):
provider = OpenLibraryProvider()
result = provider._parse_search_doc(
{
"key": "/works/OL5738148W",
"title": "The Final Empire",
"author_name": ["Brandon Sanderson"],
"first_publish_year": 2001,
"editions": {
"docs": [
{
"key": "/books/OL5M",
"title": "Kinder des Nebels",
"language": ["ger"],
"publish_year": [2018],
}
]
},
},
"Kinder des Nebels",
)
assert result is not None
assert result.title == "Kinder des Nebels"
assert result.search_title == "Kinder des Nebels"
assert result.publish_year == 2018