mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-11-14 20:38:30 +00:00
CaliBrain
c269230ef3
- **Open Library (#1451): keep the work unless the search matched a translation.** Open Library shows an English edition whenever one matches the query. So "Mistborn" returned the 2023 "Mistborn" reissue (2023) instead of "The Final Empire" (2001), and "Children of Dune" returned "Children of Dune (Dune Chronicles, Book 3)" (1987). The matched edition's title, ISBNs, cover, language and year now win only when the edition is in another language and the query doesn't name the work's own title. Checked against live Open Library: "Children of Dune" and "Mistborn" give the v1.4.0 results again, and "Der Weg der Könige" and "Kinder des Nebels" still return the German editions. - **Google Books (#1434): the ISBN cache decorator is back on `search_by_isbn`.** It had moved to the new `search_paginated`. Failed searches were cached as empty pages, and ISBN lookups lost their cache. - **Notifications: Test notification shows the plugin's failure reason again.** Apprise 2 delivers on its own worker threads, even for a single service when that service has a deadline. `_AppriseLogCapture` filtered records by thread id and dropped all of them, so the UI only said "json: delivery failed". Apprise runs those workers in a copy of the caller's context, so the capture now filters on a context variable. That still keeps concurrent dispatches apart. `AppriseAsset(async_mode=False)` would not fix this, because the sequential path also uses the worker pool when a service has a deadline. - **Activity: `activity_update` no longer sends the raw task id to non-admins.** The owner's `user_<id>` room got the raw task id. For a request-linked download that id can hold a private tracker's passkey. It now goes through `viewer_download_id`, the same mapping `/api/status` and `download_progress` use. The admins room still gets the raw id. - **SearchBar (#1476): Escape and an outside click close the homepage's inline options panel again,** as they did in v1.4.0. - **genDebug.sh: packages are listed through `importlib.metadata`,** since #1404 removed pip from the image. The script also now redacts: - `PROWLARR_MAM_ID` - `user:pass@` credentials in URLs (proxies), in both the environment and the config files - notification route URLs, which carry their service's token (Telegram, Discord and others) - `make python-checks`, `make python-test`, `make frontend-checks` and `make frontend-test` pass on top of current `main`. - New tests cover: - Open Library edition selection - Google Books caching - log capture against a real Apprise 2 delivery to a local server that answers 500, plus a concurrent dispatch on another thread - the `activity_update` payload sent to each room - genDebug redaction and the package listing Each one fails on the old code. - I checked the SearchBar fix in the running app, because the frontend has no DOM test setup: Escape closes the panel, an outside click closes it, and a click inside keeps it open. - I ran the genDebug redaction under GNU sed (Debian image) and BSD sed. The package listing ran in the pip-less image.
216 lines
9.0 KiB
Bash
Executable File
216 lines
9.0 KiB
Bash
Executable File
#!/bin/bash
|
|
|
|
# Set up log paths
|
|
LOG_ROOT=${LOG_ROOT:-"/var/log"}
|
|
LOG_DIR="$LOG_ROOT/shelfmark"
|
|
OUTPUT_FILE_NAME="shelfmark-debug_BUILD-${BUILD_VERSION:-local}_RELEASE-${RELEASE_VERSION:-NA}_$(date +%Y%m%d-%H%M%S)"
|
|
OUTPUT_FILE="/tmp/$OUTPUT_FILE_NAME.zip"
|
|
|
|
# Create LOG_DIR if it doesn't exist
|
|
mkdir -p "$LOG_DIR"
|
|
|
|
# Add system information directly to LOG_DIR
|
|
echo "=== System Information ===" > "$LOG_DIR/system_info.txt"
|
|
echo "Date: $(date)" >> "$LOG_DIR/system_info.txt"
|
|
echo "Hostname: $(hostname)" >> "$LOG_DIR/system_info.txt"
|
|
echo "Kernel: $(uname -a)" >> "$LOG_DIR/system_info.txt"
|
|
echo "" >> "$LOG_DIR/system_info.txt"
|
|
|
|
# Add disk usage
|
|
echo "=== Disk Usage ===" >> "$LOG_DIR/system_info.txt"
|
|
df -h >> "$LOG_DIR/system_info.txt" 2>&1
|
|
echo "" >> "$LOG_DIR/system_info.txt"
|
|
|
|
# Add memory info
|
|
echo "=== Memory Info ===" >> "$LOG_DIR/system_info.txt"
|
|
free -h >> "$LOG_DIR/system_info.txt" 2>&1
|
|
echo "" >> "$LOG_DIR/system_info.txt"
|
|
|
|
# Add running processes
|
|
echo "=== Running Processes ===" >> "$LOG_DIR/system_info.txt"
|
|
ps aux >> "$LOG_DIR/system_info.txt" 2>&1
|
|
echo "" >> "$LOG_DIR/system_info.txt"
|
|
|
|
# Add network information using basic commands
|
|
echo "=== Network Information ===" > "$LOG_DIR/network_info.txt"
|
|
|
|
# Try to get basic connectivity information
|
|
echo "=== Basic Connectivity ===" >> "$LOG_DIR/network_info.txt"
|
|
echo "Hostname resolution:" >> "$LOG_DIR/network_info.txt"
|
|
cat /etc/hosts >> "$LOG_DIR/network_info.txt" 2>&1 || echo "Unable to read /etc/hosts" >> "$LOG_DIR/network_info.txt"
|
|
echo "" >> "$LOG_DIR/network_info.txt"
|
|
|
|
echo "DNS configuration:" >> "$LOG_DIR/network_info.txt"
|
|
cat /etc/resolv.conf >> "$LOG_DIR/network_info.txt" 2>&1 || echo "Unable to read /etc/resolv.conf" >> "$LOG_DIR/network_info.txt"
|
|
echo "" >> "$LOG_DIR/network_info.txt"
|
|
|
|
# Try to get interface information from /proc
|
|
echo "=== Network Interfaces (/proc) ===" >> "$LOG_DIR/network_info.txt"
|
|
if [ -f "/proc/net/dev" ]; then
|
|
cat /proc/net/dev >> "$LOG_DIR/network_info.txt" 2>&1
|
|
else
|
|
echo "Not available: /proc/net/dev not found" >> "$LOG_DIR/network_info.txt"
|
|
fi
|
|
echo "" >> "$LOG_DIR/network_info.txt"
|
|
|
|
# Try connectivity tests
|
|
echo "=== Internet Connectivity ===" >> "$LOG_DIR/network_info.txt"
|
|
ping -c 3 1.1.1.1 >> "$LOG_DIR/network_info.txt" 2>&1 || echo "Ping command failed or not available" >> "$LOG_DIR/network_info.txt"
|
|
echo "" >> "$LOG_DIR/network_info.txt"
|
|
ping -c 3 one.one.one.one >> "$LOG_DIR/network_info.txt" 2>&1 || echo "DNS resolution test failed" >> "$LOG_DIR/network_info.txt"
|
|
echo "" >> "$LOG_DIR/network_info.txt"
|
|
|
|
# Test IPv6 connectivity
|
|
echo "=== IPv6 Connectivity ===" >> "$LOG_DIR/network_info.txt"
|
|
# Check if IPv6 is enabled
|
|
if [ -f "/proc/sys/net/ipv6/conf/all/disable_ipv6" ]; then
|
|
IPV6_DISABLED=$(cat /proc/sys/net/ipv6/conf/all/disable_ipv6)
|
|
if [ "$IPV6_DISABLED" = "0" ]; then
|
|
echo "IPv6 is enabled in the kernel" >> "$LOG_DIR/network_info.txt"
|
|
else
|
|
echo "IPv6 is disabled in the kernel" >> "$LOG_DIR/network_info.txt"
|
|
fi
|
|
else
|
|
echo "Unable to determine IPv6 kernel status" >> "$LOG_DIR/network_info.txt"
|
|
fi
|
|
echo "" >> "$LOG_DIR/network_info.txt"
|
|
|
|
# Try IPv6 connectivity test using Cloudflare's IPv6 DNS
|
|
echo "Testing IPv6 connectivity to Cloudflare DNS:" >> "$LOG_DIR/network_info.txt"
|
|
ping6 -c 3 2606:4700:4700::1111 >> "$LOG_DIR/network_info.txt" 2>&1 || echo "IPv6 ping failed or not available" >> "$LOG_DIR/network_info.txt"
|
|
echo "" >> "$LOG_DIR/network_info.txt"
|
|
|
|
# Test SSL connectivity
|
|
echo "=== SSL Connectivity Tests ===" >> "$LOG_DIR/network_info.txt"
|
|
echo "Testing SSL connection to Cloudflare (1.1.1.1):" >> "$LOG_DIR/network_info.txt"
|
|
echo | openssl s_client -connect 1.1.1.1:443 2>&1 | grep -E "Verify return code:|subject=|issuer=" >> "$LOG_DIR/network_info.txt" || echo "SSL test to Cloudflare failed" >> "$LOG_DIR/network_info.txt"
|
|
echo "" >> "$LOG_DIR/network_info.txt"
|
|
|
|
echo "Testing SSL connection to cloudflare.com:" >> "$LOG_DIR/network_info.txt"
|
|
echo | openssl s_client -connect cloudflare.com:443 2>&1 | grep -E "Verify return code:|subject=|issuer=" >> "$LOG_DIR/network_info.txt" || echo "SSL test to cloudflare.com failed" >> "$LOG_DIR/network_info.txt"
|
|
echo "" >> "$LOG_DIR/network_info.txt"
|
|
|
|
# Add installed packages
|
|
# The image ships without pip, so ask the app's interpreter directly
|
|
echo "=== Installed Python Packages ===" > "$LOG_DIR/packages.txt"
|
|
python - >> "$LOG_DIR/packages.txt" 2>&1 <<'PY' || echo "Could not list Python packages" >> "$LOG_DIR/packages.txt"
|
|
import importlib.metadata as metadata
|
|
|
|
for dist in sorted(metadata.distributions(), key=lambda dist: (dist.name or "").lower()):
|
|
print(f"{dist.name}=={dist.version}")
|
|
PY
|
|
echo "" >> "$LOG_DIR/packages.txt"
|
|
|
|
# Check Permissions
|
|
echo "=== Permissions ===" > "$LOG_DIR/permissions.txt"
|
|
echo "ls -all /app" >> "$LOG_DIR/permissions.txt"
|
|
ls -all /app >> "$LOG_DIR/permissions.txt" 2>&1
|
|
echo "" >> "$LOG_DIR/permissions.txt"
|
|
echo "ls -all ${INGEST_DIR:-/books}" >> "$LOG_DIR/permissions.txt"
|
|
ls -all ${INGEST_DIR:-/books} >> "$LOG_DIR/permissions.txt" 2>&1
|
|
echo "" >> "$LOG_DIR/permissions.txt"
|
|
echo "ls -all /var/log/shelfmark" >> "$LOG_DIR/permissions.txt"
|
|
ls -all /var/log/shelfmark >> "$LOG_DIR/permissions.txt" 2>&1
|
|
echo "" >> "$LOG_DIR/permissions.txt"
|
|
echo "ls -all /tmp/shelfmark" >> "$LOG_DIR/permissions.txt"
|
|
ls -all /tmp/shelfmark >> "$LOG_DIR/permissions.txt" 2>&1
|
|
echo "" >> "$LOG_DIR/permissions.txt"
|
|
|
|
# Check Iptables (NAT)
|
|
echo "=== IPtables NAT Rules ===" > "$LOG_DIR/iptables_nat.txt"
|
|
iptables -t nat -L -v -n >> "$LOG_DIR/iptables_nat.txt" 2>&1
|
|
|
|
# Check DNS Resolution details
|
|
echo "=== DNS Resolution Test ===" > "$LOG_DIR/dns_test.txt"
|
|
echo "Resolving google.com:" >> "$LOG_DIR/dns_test.txt"
|
|
nslookup google.com >> "$LOG_DIR/dns_test.txt" 2>&1
|
|
echo "" >> "$LOG_DIR/dns_test.txt"
|
|
echo "Resolving check.torproject.org:" >> "$LOG_DIR/dns_test.txt"
|
|
nslookup check.torproject.org >> "$LOG_DIR/dns_test.txt" 2>&1
|
|
|
|
|
|
# Check if running in Docker
|
|
echo "=== Container Info ===" > "$LOG_DIR/container_info.txt"
|
|
if [ -f /.dockerenv ]; then
|
|
echo "Running in Docker container (found /.dockerenv)" >> "$LOG_DIR/container_info.txt"
|
|
elif grep -q docker /proc/1/cgroup 2>/dev/null; then
|
|
echo "Running in Docker container (detected from cgroups)" >> "$LOG_DIR/container_info.txt"
|
|
else
|
|
echo "Not running in Docker container" >> "$LOG_DIR/container_info.txt"
|
|
fi
|
|
|
|
# Add environment variables (redacting sensitive info and the credentials in URLs
|
|
# such as a proxy's http://user:pass@host:port)
|
|
env | grep -v -E "^[^=]*(_KEY|_SECRET|_PASSWORD|_TOKEN|_MAM_ID)[^=]*=" | sed -E 's#(://)[^/@[:space:]]+@#\1[REDACTED]@#g' | sort > "$LOG_DIR/environment.txt"
|
|
|
|
# Redact sensitive fields (API keys, passwords, tokens, the MAM session id), the
|
|
# credentials in URLs, and notification URLs, which carry their service's token
|
|
redact_config_json() {
|
|
sed -E \
|
|
-e 's/("(AA_DONATOR_KEY|HARDCOVER_API_KEY|[^"]*_KEY|[^"]*_SECRET|[^"]*_PASSWORD|[^"]*_TOKEN|[^"]*_MAM_ID)"[[:space:]]*:[[:space:]]*")[^"]+"/\1[REDACTED]"/g' \
|
|
-e 's#(://)[^/@[:space:]"]+@#\1[REDACTED]@#g' \
|
|
-e 's#("url"[[:space:]]*:[[:space:]]*"([[:alnum:]+.-]+://)?)[^"]+"#\1[REDACTED]"#g' \
|
|
"$1"
|
|
}
|
|
|
|
# Add configuration files (redacting sensitive values)
|
|
CONFIG_DIR=${CONFIG_DIR:-"/config"}
|
|
if [ -d "$CONFIG_DIR" ]; then
|
|
mkdir -p "$LOG_DIR/config"
|
|
|
|
# Copy and redact main settings file
|
|
if [ -f "$CONFIG_DIR/settings.json" ]; then
|
|
redact_config_json "$CONFIG_DIR/settings.json" > "$LOG_DIR/config/settings.json" 2>/dev/null
|
|
fi
|
|
|
|
# Copy and redact plugin config files
|
|
if [ -d "$CONFIG_DIR/plugins" ]; then
|
|
mkdir -p "$LOG_DIR/config/plugins"
|
|
for config_file in "$CONFIG_DIR/plugins"/*.json; do
|
|
if [ -f "$config_file" ]; then
|
|
filename=$(basename "$config_file")
|
|
redact_config_json "$config_file" > "$LOG_DIR/config/plugins/$filename" 2>/dev/null
|
|
fi
|
|
done
|
|
fi
|
|
|
|
echo "Configuration files copied (sensitive values redacted)" >> "$LOG_DIR/container_info.txt"
|
|
else
|
|
echo "Config directory not found at $CONFIG_DIR" >> "$LOG_DIR/container_info.txt"
|
|
fi
|
|
|
|
echo "--- HTTPBin ---" >> $LOG_DIR/network_info.txt
|
|
curl -s https://httpbin.org/get >> $LOG_DIR/network_info.txt 2>&1
|
|
echo "" >> $LOG_DIR/network_info.txt
|
|
echo "--- HowsMySSL ---" >> $LOG_DIR/network_info.txt
|
|
curl -s https://www.howsmyssl.com/a/check >> $LOG_DIR/network_info.txt 2>&1
|
|
echo "" >> $LOG_DIR/network_info.txt
|
|
echo "--- IPInfo ---" >> $LOG_DIR/network_info.txt
|
|
curl -s https://ipinfo.io >> $LOG_DIR/network_info.txt 2>&1
|
|
echo "" >> $LOG_DIR/network_info.txt
|
|
echo "--- Cloudflare Trace ---" >> $LOG_DIR/network_info.txt
|
|
curl -s https://1.1.1.1/cdn-cgi/trace >> $LOG_DIR/network_info.txt 2>&1
|
|
|
|
# Copy Tor logs if they exist
|
|
if [ -f "/var/log/tor/notices.log" ]; then
|
|
cp "/var/log/tor/notices.log" "$LOG_DIR/tor_notices.log"
|
|
fi
|
|
|
|
# Copy Supervisor logs if they exist
|
|
if [ -d "/var/log/supervisor" ]; then
|
|
cp -rf "/var/log/supervisor/" "$LOG_DIR/supervisor/"
|
|
fi
|
|
|
|
# Create the zip file directly from LOG_DIR
|
|
ln -s "$LOG_DIR" /tmp/$OUTPUT_FILE_NAME
|
|
(cd /tmp && zip -r "$OUTPUT_FILE" $OUTPUT_FILE_NAME > /dev/null 2>&1)
|
|
rm -f /tmp/$OUTPUT_FILE_NAME
|
|
|
|
if [ -f "$OUTPUT_FILE" ]; then
|
|
echo "$OUTPUT_FILE"
|
|
exit 0
|
|
else
|
|
echo "Failed to create debug archive"
|
|
exit 1
|
|
fi
|