starred/shelfmark
1
0 Fork 0
mirror of https://github.com/calibrain/shelfmark.git synced 2026-11-14 09:38:28 +00:00
Code Issues Packages Projects Releases 10 Wiki Activity
Files
86fff322976699dc6c27c23fdf8f413dfd00be76
shelfmark/genDebug.sh
T
CaliBrain c269230ef3 fix: review regressions in metadata, notifications, activity redaction, search options and genDebug (#1510) 
- **Open Library (#1451): keep the work unless the search matched a
translation.** Open Library shows an English edition whenever one
matches the query. So "Mistborn" returned the 2023 "Mistborn" reissue
(2023) instead of "The Final Empire" (2001), and "Children of Dune"
returned "Children of Dune (Dune Chronicles, Book 3)" (1987). The
matched edition's title, ISBNs, cover, language and year now win only
when the edition is in another language and the query doesn't name the
work's own title. Checked against live Open Library: "Children of Dune"
and "Mistborn" give the v1.4.0 results again, and "Der Weg der Könige"
and "Kinder des Nebels" still return the German editions.
- **Google Books (#1434): the ISBN cache decorator is back on
`search_by_isbn`.** It had moved to the new `search_paginated`. Failed
searches were cached as empty pages, and ISBN lookups lost their cache.
- **Notifications: Test notification shows the plugin's failure reason
again.** Apprise 2 delivers on its own worker threads, even for a single
service when that service has a deadline. `_AppriseLogCapture` filtered
records by thread id and dropped all of them, so the UI only said "json:
delivery failed". Apprise runs those workers in a copy of the caller's
context, so the capture now filters on a context variable. That still
keeps concurrent dispatches apart. `AppriseAsset(async_mode=False)`
would not fix this, because the sequential path also uses the worker
pool when a service has a deadline.
- **Activity: `activity_update` no longer sends the raw task id to
non-admins.** The owner's `user_<id>` room got the raw task id. For a
request-linked download that id can hold a private tracker's passkey. It
now goes through `viewer_download_id`, the same mapping `/api/status`
and `download_progress` use. The admins room still gets the raw id.
- **SearchBar (#1476): Escape and an outside click close the homepage's
inline options panel again,** as they did in v1.4.0.
- **genDebug.sh: packages are listed through `importlib.metadata`,**
since #1404 removed pip from the image. The script also now redacts:
  - `PROWLARR_MAM_ID`
- `user:pass@` credentials in URLs (proxies), in both the environment
and the config files
- notification route URLs, which carry their service's token (Telegram,
Discord and others)

- `make python-checks`, `make python-test`, `make frontend-checks` and
`make frontend-test` pass on top of current `main`.
- New tests cover:
  - Open Library edition selection
  - Google Books caching
- log capture against a real Apprise 2 delivery to a local server that
answers 500, plus a concurrent dispatch on another thread
  - the `activity_update` payload sent to each room
  - genDebug redaction and the package listing

  Each one fails on the old code.
- I checked the SearchBar fix in the running app, because the frontend
has no DOM test setup: Escape closes the panel, an outside click closes
it, and a click inside keeps it open.
- I ran the genDebug redaction under GNU sed (Debian image) and BSD sed.
The package listing ran in the pip-less image.
2026-10-21 01:58:41 -04:00

216 lines
9.0 KiB
Bash
Executable File
Raw Blame History

#!/bin/bash
# Set up log paths
LOG_ROOT=${LOG_ROOT:-"/var/log"}
LOG_DIR="$LOG_ROOT/shelfmark"
OUTPUT_FILE_NAME="shelfmark-debug_BUILD-${BUILD_VERSION:-local}_RELEASE-${RELEASE_VERSION:-NA}_$(date +%Y%m%d-%H%M%S)"
OUTPUT_FILE="/tmp/$OUTPUT_FILE_NAME.zip"
# Create LOG_DIR if it doesn't exist
mkdir -p "$LOG_DIR"
# Add system information directly to LOG_DIR
echo "=== System Information ===" > "$LOG_DIR/system_info.txt"
echo "Date: $(date)" >> "$LOG_DIR/system_info.txt"
echo "Hostname: $(hostname)" >> "$LOG_DIR/system_info.txt"
echo "Kernel: $(uname -a)" >> "$LOG_DIR/system_info.txt"
echo "" >> "$LOG_DIR/system_info.txt"
# Add disk usage
echo "=== Disk Usage ===" >> "$LOG_DIR/system_info.txt"
df -h >> "$LOG_DIR/system_info.txt" 2>&1
echo "" >> "$LOG_DIR/system_info.txt"
# Add memory info
echo "=== Memory Info ===" >> "$LOG_DIR/system_info.txt"
free -h >> "$LOG_DIR/system_info.txt" 2>&1
echo "" >> "$LOG_DIR/system_info.txt"
# Add running processes
echo "=== Running Processes ===" >> "$LOG_DIR/system_info.txt"
ps aux >> "$LOG_DIR/system_info.txt" 2>&1
echo "" >> "$LOG_DIR/system_info.txt"
# Add network information using basic commands
echo "=== Network Information ===" > "$LOG_DIR/network_info.txt"
# Try to get basic connectivity information
echo "=== Basic Connectivity ===" >> "$LOG_DIR/network_info.txt"
echo "Hostname resolution:" >> "$LOG_DIR/network_info.txt"
cat /etc/hosts >> "$LOG_DIR/network_info.txt" 2>&1 || echo "Unable to read /etc/hosts" >> "$LOG_DIR/network_info.txt"
echo "" >> "$LOG_DIR/network_info.txt"
echo "DNS configuration:" >> "$LOG_DIR/network_info.txt"
cat /etc/resolv.conf >> "$LOG_DIR/network_info.txt" 2>&1 || echo "Unable to read /etc/resolv.conf" >> "$LOG_DIR/network_info.txt"
echo "" >> "$LOG_DIR/network_info.txt"
# Try to get interface information from /proc
echo "=== Network Interfaces (/proc) ===" >> "$LOG_DIR/network_info.txt"
if [ -f "/proc/net/dev" ]; then
cat /proc/net/dev >> "$LOG_DIR/network_info.txt" 2>&1
else
echo "Not available: /proc/net/dev not found" >> "$LOG_DIR/network_info.txt"
fi
echo "" >> "$LOG_DIR/network_info.txt"
# Try connectivity tests
echo "=== Internet Connectivity ===" >> "$LOG_DIR/network_info.txt"
ping -c 3 1.1.1.1 >> "$LOG_DIR/network_info.txt" 2>&1 || echo "Ping command failed or not available" >> "$LOG_DIR/network_info.txt"
echo "" >> "$LOG_DIR/network_info.txt"
ping -c 3 one.one.one.one >> "$LOG_DIR/network_info.txt" 2>&1 || echo "DNS resolution test failed" >> "$LOG_DIR/network_info.txt"
echo "" >> "$LOG_DIR/network_info.txt"
# Test IPv6 connectivity
echo "=== IPv6 Connectivity ===" >> "$LOG_DIR/network_info.txt"
# Check if IPv6 is enabled
if [ -f "/proc/sys/net/ipv6/conf/all/disable_ipv6" ]; then
IPV6_DISABLED=$(cat /proc/sys/net/ipv6/conf/all/disable_ipv6)
if [ "$IPV6_DISABLED" = "0" ]; then
echo "IPv6 is enabled in the kernel" >> "$LOG_DIR/network_info.txt"
else
echo "IPv6 is disabled in the kernel" >> "$LOG_DIR/network_info.txt"
fi
else
echo "Unable to determine IPv6 kernel status" >> "$LOG_DIR/network_info.txt"
fi
echo "" >> "$LOG_DIR/network_info.txt"
# Try IPv6 connectivity test using Cloudflare's IPv6 DNS
echo "Testing IPv6 connectivity to Cloudflare DNS:" >> "$LOG_DIR/network_info.txt"
ping6 -c 3 2606:4700:4700::1111 >> "$LOG_DIR/network_info.txt" 2>&1 || echo "IPv6 ping failed or not available" >> "$LOG_DIR/network_info.txt"
echo "" >> "$LOG_DIR/network_info.txt"
# Test SSL connectivity
echo "=== SSL Connectivity Tests ===" >> "$LOG_DIR/network_info.txt"
echo "Testing SSL connection to Cloudflare (1.1.1.1):" >> "$LOG_DIR/network_info.txt"
echo | openssl s_client -connect 1.1.1.1:443 2>&1 | grep -E "Verify return code:|subject=|issuer=" >> "$LOG_DIR/network_info.txt" || echo "SSL test to Cloudflare failed" >> "$LOG_DIR/network_info.txt"
echo "" >> "$LOG_DIR/network_info.txt"
echo "Testing SSL connection to cloudflare.com:" >> "$LOG_DIR/network_info.txt"
echo | openssl s_client -connect cloudflare.com:443 2>&1 | grep -E "Verify return code:|subject=|issuer=" >> "$LOG_DIR/network_info.txt" || echo "SSL test to cloudflare.com failed" >> "$LOG_DIR/network_info.txt"
echo "" >> "$LOG_DIR/network_info.txt"
# Add installed packages
# The image ships without pip, so ask the app's interpreter directly
echo "=== Installed Python Packages ===" > "$LOG_DIR/packages.txt"
python - >> "$LOG_DIR/packages.txt" 2>&1 <<'PY' || echo "Could not list Python packages" >> "$LOG_DIR/packages.txt"
import importlib.metadata as metadata
for dist in sorted(metadata.distributions(), key=lambda dist: (dist.name or "").lower()):
print(f"{dist.name}=={dist.version}")
PY
echo "" >> "$LOG_DIR/packages.txt"
# Check Permissions
echo "=== Permissions ===" > "$LOG_DIR/permissions.txt"
echo "ls -all /app" >> "$LOG_DIR/permissions.txt"
ls -all /app >> "$LOG_DIR/permissions.txt" 2>&1
echo "" >> "$LOG_DIR/permissions.txt"
echo "ls -all ${INGEST_DIR:-/books}" >> "$LOG_DIR/permissions.txt"
ls -all ${INGEST_DIR:-/books} >> "$LOG_DIR/permissions.txt" 2>&1
echo "" >> "$LOG_DIR/permissions.txt"
echo "ls -all /var/log/shelfmark" >> "$LOG_DIR/permissions.txt"
ls -all /var/log/shelfmark >> "$LOG_DIR/permissions.txt" 2>&1
echo "" >> "$LOG_DIR/permissions.txt"
echo "ls -all /tmp/shelfmark" >> "$LOG_DIR/permissions.txt"
ls -all /tmp/shelfmark >> "$LOG_DIR/permissions.txt" 2>&1
echo "" >> "$LOG_DIR/permissions.txt"
# Check Iptables (NAT)
echo "=== IPtables NAT Rules ===" > "$LOG_DIR/iptables_nat.txt"
iptables -t nat -L -v -n >> "$LOG_DIR/iptables_nat.txt" 2>&1
# Check DNS Resolution details
echo "=== DNS Resolution Test ===" > "$LOG_DIR/dns_test.txt"
echo "Resolving google.com:" >> "$LOG_DIR/dns_test.txt"
nslookup google.com >> "$LOG_DIR/dns_test.txt" 2>&1
echo "" >> "$LOG_DIR/dns_test.txt"
echo "Resolving check.torproject.org:" >> "$LOG_DIR/dns_test.txt"
nslookup check.torproject.org >> "$LOG_DIR/dns_test.txt" 2>&1
# Check if running in Docker
echo "=== Container Info ===" > "$LOG_DIR/container_info.txt"
if [ -f /.dockerenv ]; then
echo "Running in Docker container (found /.dockerenv)" >> "$LOG_DIR/container_info.txt"
elif grep -q docker /proc/1/cgroup 2>/dev/null; then
echo "Running in Docker container (detected from cgroups)" >> "$LOG_DIR/container_info.txt"
else
echo "Not running in Docker container" >> "$LOG_DIR/container_info.txt"
fi
# Add environment variables (redacting sensitive info and the credentials in URLs
# such as a proxy's http://user:pass@host:port)
env | grep -v -E "^[^=]*(_KEY|_SECRET|_PASSWORD|_TOKEN|_MAM_ID)[^=]*=" | sed -E 's#(://)[^/@[:space:]]+@#\1[REDACTED]@#g' | sort > "$LOG_DIR/environment.txt"
# Redact sensitive fields (API keys, passwords, tokens, the MAM session id), the
# credentials in URLs, and notification URLs, which carry their service's token
redact_config_json() {
sed -E \
-e 's/("(AA_DONATOR_KEY|HARDCOVER_API_KEY|[^"]*_KEY|[^"]*_SECRET|[^"]*_PASSWORD|[^"]*_TOKEN|[^"]*_MAM_ID)"[[:space:]]*:[[:space:]]*")[^"]+"/\1[REDACTED]"/g' \
-e 's#(://)[^/@[:space:]"]+@#\1[REDACTED]@#g' \
-e 's#("url"[[:space:]]*:[[:space:]]*"([[:alnum:]+.-]+://)?)[^"]+"#\1[REDACTED]"#g' \
"$1"
}
# Add configuration files (redacting sensitive values)
CONFIG_DIR=${CONFIG_DIR:-"/config"}
if [ -d "$CONFIG_DIR" ]; then
mkdir -p "$LOG_DIR/config"
# Copy and redact main settings file
if [ -f "$CONFIG_DIR/settings.json" ]; then
redact_config_json "$CONFIG_DIR/settings.json" > "$LOG_DIR/config/settings.json" 2>/dev/null
fi
# Copy and redact plugin config files
if [ -d "$CONFIG_DIR/plugins" ]; then
mkdir -p "$LOG_DIR/config/plugins"
for config_file in "$CONFIG_DIR/plugins"/*.json; do
if [ -f "$config_file" ]; then
filename=$(basename "$config_file")
redact_config_json "$config_file" > "$LOG_DIR/config/plugins/$filename" 2>/dev/null
fi
done
fi
echo "Configuration files copied (sensitive values redacted)" >> "$LOG_DIR/container_info.txt"
else
echo "Config directory not found at $CONFIG_DIR" >> "$LOG_DIR/container_info.txt"
fi
echo "--- HTTPBin ---" >> $LOG_DIR/network_info.txt
curl -s https://httpbin.org/get >> $LOG_DIR/network_info.txt 2>&1
echo "" >> $LOG_DIR/network_info.txt
echo "--- HowsMySSL ---" >> $LOG_DIR/network_info.txt
curl -s https://www.howsmyssl.com/a/check >> $LOG_DIR/network_info.txt 2>&1
echo "" >> $LOG_DIR/network_info.txt
echo "--- IPInfo ---" >> $LOG_DIR/network_info.txt
curl -s https://ipinfo.io >> $LOG_DIR/network_info.txt 2>&1
echo "" >> $LOG_DIR/network_info.txt
echo "--- Cloudflare Trace ---" >> $LOG_DIR/network_info.txt
curl -s https://1.1.1.1/cdn-cgi/trace >> $LOG_DIR/network_info.txt 2>&1
# Copy Tor logs if they exist
if [ -f "/var/log/tor/notices.log" ]; then
cp "/var/log/tor/notices.log" "$LOG_DIR/tor_notices.log"
fi
# Copy Supervisor logs if they exist
if [ -d "/var/log/supervisor" ]; then
cp -rf "/var/log/supervisor/" "$LOG_DIR/supervisor/"
fi
# Create the zip file directly from LOG_DIR
ln -s "$LOG_DIR" /tmp/$OUTPUT_FILE_NAME
(cd /tmp && zip -r "$OUTPUT_FILE" $OUTPUT_FILE_NAME > /dev/null 2>&1)
rm -f /tmp/$OUTPUT_FILE_NAME
if [ -f "$OUTPUT_FILE" ]; then
echo "$OUTPUT_FILE"
exit 0
else
echo "Failed to create debug archive"
exit 1
fi
Reference in new issue View Git Blame Copy Permalink