mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-11-10 12:38:32 +00:00
ffmpeg is in the default image for one thing, the Debug mode screen recording (`_start_ffmpeg_recording`). Nothing else calls it, SeleniumBase included, and it brings 93 packages with it. On real builds that's 406 Trivy findings (149 High) and 78 MB of the gzipped amd64 image. This takes it out of the image and installs it when Debug mode needs it. It came out of the base image discussion on #1616 and helps on any base, though on Wolfi the install line would need `apk`. ## What changed - **Dockerfile:** ffmpeg is gone from the full image's install. - **Guard:** `_start_ffmpeg_recording` checks `shutil.which("ffmpeg")` first. If it's missing it logs a warning and returns, and the bypass carries on. Without this, Debug on with no ffmpeg fails every bypass with `FileNotFoundError`. There's a new test for it. - **Entrypoint:** when Debug is on, the internal bypasser is in use and ffmpeg isn't already there, it installs ffmpeg in the background so the app doesn't wait for it. The download and the install each run under a 120 s `timeout` with `DEBIAN_FRONTEND=noninteractive`. Downloading first means a hung mirror can only be killed before dpkg starts, and the next start runs `dpkg --configure -a` in case a restart cut dpkg off. It logs when it starts, succeeds, times out or fails. - **Docs:** a short paragraph in `docs/troubleshooting.md` covering the install and the non-root limit. ## Debug has two sources Debug can come from the `DEBUG` env var or from the Debug Mode setting in the UI, which is saved to `/config/plugins/advanced.json`. The existing block in `entrypoint.sh` only reads the env var, so an install keyed on `$DEBUG` would miss anyone who turned it on in the UI. The new check asks the app instead (`shelfmark.config.env.DEBUG`), which reads both. The UI setting is `requires_restart`, so the restart it already asks for is what brings it through the entrypoint. ## Non-root mode A container started as `1000:1000` can't run `apt-get`, so the entrypoint logs that it can't install ffmpeg, the guard warns on each bypass, and there's no recording. Everything else works. The workaround for a debugging session is to start as root with `PUID`/`PGID`. ## Startup cost None that I could measure. With Debug on, `/api/health` answers in the same time as main (3-4 s with Debug only in config, 9-10 s from env), and ffmpeg was ready 6-8 s later on the runners. The trade-off is that the first bypass after startup may be unrecorded while the install runs. It logs the warning and passes. I saw both outcomes on a bypass started right after health. From reading the code: the install lives in the container, so recreating the container installs it again. It also runs after `tor.sh` and `wireguard.sh`, so apt goes through whichever egress is active. ## Testing Real Docker builds on GitHub Actions on my fork, amd64 and arm64, with the same result on both for every case below. | Case | Result | |---|---| | Debug off | No install, `/api/health` in 3-4 s, same as main | | Debug only in config | Health in 3-4 s, ffmpeg ready 6-8 s later, real mp4 after a bypass | | Debug from env | Health in 9-10 s (same as main), real mp4 after a bypass | | Started as root, app dropped to uid 1000 | Smoke and bypass pass | | Started with `--user 1000`, Debug on | Logs both messages, bypass passes, no mp4 | | apt blocked (closed proxy) | Logs `Could not install ffmpeg`, bypass passes, no mp4 | | apt's proxy on a dropped address, so no TCP reply at all | Health in 9-10 s, a bypass during the install passes with the warning and no mp4. apt gives up by itself in under a minute. Nothing left running, `dpkg --audit` clean | | DNS server on a dropped address | Health in 9-10 s, `Timed out installing ffmpeg after 120s`. Nothing left running, `dpkg --audit` clean | | `--network none`, Debug on | No hang, health 200 from inside the container | | Container restarted while dpkg was running | Next start finishes the install, `dpkg --audit` clean, real mp4 | | Fresh named volume, Debug only in config | Recording made, uid 1000 can write | | Existing uid-1000 volume, Debug only in config | Same, nothing root-owned afterwards | | `find /config /var/log/shelfmark -user root` | 0 files, same as main | | `find /app -user root` | 245, same count as main | | `tests/e2e/platform/run-e2e.sh` with `dns-blocked` and `baseline` | Pass | The bypass checks ran against example.com. A real Cloudflare challenge was served by a demo page in the same runs, and no build solved it, including main, so that part doesn't say anything about this change. Locally: 5494 Python tests pass, and `ruff`, `basedpyright`, `vulture` and `shellcheck` are clean. The full results are in [the base image report](https://github.com/splitsec2/shelfmark/blob/7afcecfb76727724a71561cbc078f091569b58dc/docs/research/2026-11-01-base-image-report.md).
646 lines
22 KiB
Bash
646 lines
22 KiB
Bash
#!/bin/bash
|
|
|
|
set -e
|
|
|
|
is_truthy() {
|
|
case "${1,,}" in
|
|
true|yes|1|y) return 0 ;;
|
|
*) return 1 ;;
|
|
esac
|
|
}
|
|
|
|
ENABLE_LOGGING_VALUE="${ENABLE_LOGGING:-true}"
|
|
LOG_PIPE_DIR=""
|
|
LOG_PIPE=""
|
|
TEE_PID=""
|
|
FILE_LOGGING_ENABLED="false"
|
|
CURRENT_UID=$(id -u)
|
|
CURRENT_GID=$(id -g)
|
|
RUN_AS_NON_ROOT="false"
|
|
RUNTIME_TMP_DIR="${TMP_DIR:-/tmp/shelfmark}"
|
|
DEFAULT_RUNTIME_HOME="${RUNTIME_TMP_DIR}/home"
|
|
|
|
if [ "$CURRENT_UID" != "0" ]; then
|
|
RUN_AS_NON_ROOT="true"
|
|
fi
|
|
|
|
start_file_logging() {
|
|
local logfile="$1"
|
|
|
|
LOG_PIPE_DIR="$(mktemp -d)"
|
|
LOG_PIPE="${LOG_PIPE_DIR}/shelfmark-log.pipe"
|
|
mkfifo "$LOG_PIPE"
|
|
|
|
tee -a "$logfile" < "$LOG_PIPE" &
|
|
TEE_PID=$!
|
|
|
|
exec 3>&1 4>&2
|
|
exec > "$LOG_PIPE" 2>&1
|
|
}
|
|
|
|
stop_file_logging() {
|
|
if [ -z "${TEE_PID:-}" ]; then
|
|
return 0
|
|
fi
|
|
|
|
exec 1>&3 2>&4
|
|
exec 3>&- 4>&-
|
|
|
|
rm -f "$LOG_PIPE"
|
|
rmdir "$LOG_PIPE_DIR" 2>/dev/null || true
|
|
|
|
wait "$TEE_PID" 2>/dev/null || true
|
|
TEE_PID=""
|
|
}
|
|
|
|
if is_truthy "$ENABLE_LOGGING_VALUE"; then
|
|
LOG_DIR=${LOG_ROOT:-/var/log/}/shelfmark
|
|
if mkdir -p "$LOG_DIR" 2>/dev/null; then
|
|
LOG_FILE="${LOG_DIR}/shelfmark_entrypoint.log"
|
|
# Keep the previous entrypoint log instead of deleting all history on boot.
|
|
rotation_ok="true"
|
|
if [ -f "${LOG_FILE}.prev" ] && ! rm -f "${LOG_FILE}.prev"; then
|
|
echo "Warning: could not remove previous entrypoint log ${LOG_FILE}.prev, continuing without file logging" >&2
|
|
rotation_ok="false"
|
|
fi
|
|
if [ "$rotation_ok" = "true" ] && [ -f "$LOG_FILE" ] && ! mv "$LOG_FILE" "${LOG_FILE}.prev"; then
|
|
echo "Warning: could not rotate entrypoint log $LOG_FILE, continuing without file logging" >&2
|
|
rotation_ok="false"
|
|
fi
|
|
|
|
if [ "$rotation_ok" = "true" ]; then
|
|
FILE_LOGGING_ENABLED="true"
|
|
else
|
|
ENABLE_LOGGING_VALUE="false"
|
|
export ENABLE_LOGGING="false"
|
|
fi
|
|
else
|
|
echo "Warning: could not create log directory $LOG_DIR, continuing without file logging" >&2
|
|
ENABLE_LOGGING_VALUE="false"
|
|
export ENABLE_LOGGING="false"
|
|
fi
|
|
fi
|
|
|
|
# Egress modes are mutually exclusive. Check this BEFORE starting either one so
|
|
# we never run tor.sh and then abort, leaving a half-configured network stack.
|
|
if [ "$USING_TOR" = "true" ] && [ "$USING_WIREGUARD" = "true" ]; then
|
|
echo "USING_TOR and USING_WIREGUARD are mutually exclusive; enable only one egress mode." >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [ "$USING_TOR" = "true" ]; then
|
|
if [ "$RUN_AS_NON_ROOT" = "true" ]; then
|
|
echo "USING_TOR=true requires the container to start as root." >&2
|
|
echo "Non-root mode skips the privileged filesystem and network setup Tor depends on." >&2
|
|
exit 1
|
|
fi
|
|
./tor.sh
|
|
fi
|
|
|
|
if [ "$USING_WIREGUARD" = "true" ]; then
|
|
if [ "$RUN_AS_NON_ROOT" = "true" ]; then
|
|
echo "USING_WIREGUARD=true requires the container to start as root." >&2
|
|
echo "Non-root mode skips the privileged network setup WireGuard depends on." >&2
|
|
exit 1
|
|
fi
|
|
./wireguard.sh
|
|
fi
|
|
|
|
if [ "$FILE_LOGGING_ENABLED" = "true" ]; then
|
|
start_file_logging "$LOG_FILE"
|
|
fi
|
|
|
|
echo "Starting entrypoint script"
|
|
if [ "$FILE_LOGGING_ENABLED" = "true" ]; then
|
|
echo "Log file: $LOG_FILE"
|
|
else
|
|
echo "File logging disabled (ENABLE_LOGGING=$ENABLE_LOGGING_VALUE)"
|
|
fi
|
|
|
|
PYTHON_BIN="/app/.venv/bin/python"
|
|
if [ ! -x "$PYTHON_BIN" ]; then
|
|
PYTHON_BIN="python3"
|
|
fi
|
|
|
|
# Defensive: some orchestrators (e.g. Unraid Dockhand templates) inject a default
|
|
# PATH that drops the venv bin directory baked in by the Dockerfile. Prepend it
|
|
# so subprocesses launched without an absolute path still resolve correctly.
|
|
case ":${PATH}:" in
|
|
*":/app/.venv/bin:"*) ;;
|
|
*) export PATH="/app/.venv/bin:${PATH}" ;;
|
|
esac
|
|
GUNICORN_BIN="/app/.venv/bin/gunicorn"
|
|
if [ ! -x "$GUNICORN_BIN" ]; then
|
|
GUNICORN_BIN="gunicorn"
|
|
fi
|
|
|
|
# Print build version
|
|
echo "Build version: $BUILD_VERSION"
|
|
echo "Release version: $RELEASE_VERSION"
|
|
|
|
# Configure timezone
|
|
if [ "$TZ" ]; then
|
|
if [ "$RUN_AS_NON_ROOT" = "true" ]; then
|
|
echo "TZ is set to $TZ (non-root mode leaves /etc/localtime unchanged)"
|
|
else
|
|
echo "Setting timezone to $TZ"
|
|
ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
|
|
fi
|
|
fi
|
|
|
|
if [ "$RUN_AS_NON_ROOT" = "true" ]; then
|
|
RUN_UID="$CURRENT_UID"
|
|
RUN_GID="$CURRENT_GID"
|
|
USERNAME=$(getent passwd "$RUN_UID" 2>/dev/null | cut -d: -f1 || true)
|
|
if [ -z "$USERNAME" ]; then
|
|
USERNAME="$RUN_UID"
|
|
echo "No passwd entry found for UID $RUN_UID; using numeric identity"
|
|
fi
|
|
TARGET_USER_SPEC="${RUN_UID}:${RUN_GID}"
|
|
else
|
|
# Determine user ID with proper precedence:
|
|
# 1. PUID (LinuxServer.io standard - recommended)
|
|
# 2. UID (legacy, for backward compatibility with existing installs)
|
|
# 3. Default to 1000
|
|
#
|
|
# Note: $UID is a bash builtin that's always set. We use `printenv` to detect
|
|
# if UID was explicitly set as an environment variable (e.g., via docker-compose).
|
|
if [ -n "$PUID" ]; then
|
|
RUN_UID="$PUID"
|
|
echo "Using PUID=$RUN_UID"
|
|
elif printenv UID >/dev/null 2>&1; then
|
|
RUN_UID="$(printenv UID)"
|
|
echo "Using UID=$RUN_UID (legacy - consider migrating to PUID)"
|
|
else
|
|
RUN_UID=1000
|
|
echo "Using default UID=$RUN_UID"
|
|
fi
|
|
|
|
# Determine group ID with proper precedence:
|
|
# 1. PGID (LinuxServer.io standard - recommended)
|
|
# 2. GID (legacy, for backward compatibility with existing installs)
|
|
# 3. Default to 1000
|
|
if [ -n "$PGID" ]; then
|
|
RUN_GID="$PGID"
|
|
echo "Using PGID=$RUN_GID"
|
|
elif [ -n "$GID" ]; then
|
|
RUN_GID="$GID"
|
|
echo "Using GID=$RUN_GID (legacy - consider migrating to PGID)"
|
|
else
|
|
RUN_GID=1000
|
|
echo "Using default GID=$RUN_GID"
|
|
fi
|
|
|
|
if ! getent group "$RUN_GID" >/dev/null; then
|
|
echo "Adding group $RUN_GID with name appuser"
|
|
groupadd -g "$RUN_GID" appuser
|
|
fi
|
|
|
|
# Create user if it doesn't exist for this UID yet.
|
|
if ! getent passwd "$RUN_UID" >/dev/null; then
|
|
echo "Adding user $RUN_UID with name appuser"
|
|
useradd -u "$RUN_UID" -g "$RUN_GID" -d "$DEFAULT_RUNTIME_HOME" -s /sbin/nologin appuser
|
|
fi
|
|
|
|
# Get username for the UID (whether we just created it or it existed)
|
|
USERNAME=$(getent passwd "$RUN_UID" | cut -d: -f1)
|
|
if [ -z "$USERNAME" ]; then
|
|
USERNAME="$RUN_UID"
|
|
fi
|
|
TARGET_USER_SPEC="${RUN_UID}:${RUN_GID}"
|
|
fi
|
|
|
|
# Avoid unnecessary gosu hops when we're already running as the target user.
|
|
# Some nested LXC setups spin on root-to-root gosu invocations.
|
|
needs_user_switch() {
|
|
local current_uid
|
|
local current_gid
|
|
|
|
current_uid=$(id -u)
|
|
current_gid=$(id -g)
|
|
|
|
[ "$current_uid" != "$RUN_UID" ] || [ "$current_gid" != "$RUN_GID" ]
|
|
}
|
|
|
|
run_as_target_user() {
|
|
if needs_user_switch; then
|
|
gosu "$TARGET_USER_SPEC" "$@"
|
|
return $?
|
|
fi
|
|
|
|
"$@"
|
|
}
|
|
|
|
exec_as_target_user() {
|
|
if needs_user_switch; then
|
|
exec gosu "$TARGET_USER_SPEC" "$@"
|
|
fi
|
|
|
|
exec "$@"
|
|
}
|
|
|
|
test_write() {
|
|
local folder=$1
|
|
local test_file="$folder/shelfmark_TEST_WRITE"
|
|
local FILE_CONTENT
|
|
local result
|
|
local result_text
|
|
|
|
if ! mkdir -p "$folder"; then
|
|
echo "Failed to create directory for write test: $folder"
|
|
return 1
|
|
fi
|
|
|
|
# This is a probe: a failure here is expected (e.g. a fresh root-owned bind
|
|
# mount) and is recovered by the caller via change_ownership + re-probe. Hide
|
|
# the shell's "Permission denied"/"Read-only file system" stderr so a handled
|
|
# probe miss doesn't masquerade as a real boot failure in the logs.
|
|
if ! run_as_target_user sh -c 'echo 0123456789_TEST 2>/dev/null > "$1"' _ "$test_file"; then
|
|
echo "Failed to write test file in $folder as $USERNAME"
|
|
return 1
|
|
fi
|
|
|
|
FILE_CONTENT=$(cat "$test_file" 2>/dev/null || echo "")
|
|
# A folder can be writable but not deletable (e.g. a Synology share without
|
|
# "Delete subfolders and files"). That is not a boot failure - the app writes
|
|
# files in place on such shares - so don't let a failed cleanup print an
|
|
# alarming error or fail the probe.
|
|
run_as_target_user rm -f "$test_file" 2>/dev/null || \
|
|
echo "Note: could not remove test file in $folder (folder is writable but not deletable)"
|
|
[ "$FILE_CONTENT" = "0123456789_TEST" ]
|
|
result=$?
|
|
if [ $result -eq 0 ]; then
|
|
result_text="true"
|
|
else
|
|
result_text="false"
|
|
fi
|
|
echo "Test write to $folder by $USERNAME: $result_text"
|
|
return $result
|
|
}
|
|
|
|
make_writable() {
|
|
local folder="$1"
|
|
local mode="${2:-tree}"
|
|
local did_full_chown=0
|
|
local is_writable
|
|
set +e
|
|
test_write "$folder"
|
|
is_writable=$?
|
|
set -e
|
|
if [ $is_writable -eq 0 ]; then
|
|
echo "Folder $folder is writable, no need to change ownership"
|
|
else
|
|
if [ "$mode" = "root" ]; then
|
|
echo "Folder $folder is not writable, fixing top-level ownership and permissions"
|
|
mkdir -p "$folder"
|
|
chown "${RUN_UID}:${RUN_GID}" "$folder" || echo "Failed to change ownership for ${folder}, continuing..."
|
|
chmod u+rwx "$folder" || echo "Failed to change owner permissions for ${folder}, continuing..."
|
|
else
|
|
echo "Folder $folder is not writable, changing ownership"
|
|
change_ownership "$folder"
|
|
chmod -R g+r,g+w "$folder" || echo "Failed to change group permissions for ${folder}, continuing..."
|
|
fi
|
|
did_full_chown=1
|
|
fi
|
|
# Fix any misowned subdirectories/files (e.g., from previous runs as root)
|
|
if [ "$mode" = "tree" ] && [ "$did_full_chown" -eq 0 ] && [ -d "$folder" ]; then
|
|
echo "Checking for misowned files/directories in $folder"
|
|
# Stay on the same filesystem to avoid traversing mounted subpaths
|
|
# (for example read-only bind mounts under /app in dev setups).
|
|
find "$folder" -xdev -mindepth 1 \( ! -user "$RUN_UID" -o ! -group "$RUN_GID" \) \
|
|
-exec chown "$RUN_UID:$RUN_GID" {} + 2>/dev/null || true
|
|
fi
|
|
test_write "$folder" || echo "Failed to test write to ${folder}, continuing..."
|
|
}
|
|
|
|
fix_misowned() {
|
|
local folder="$1"
|
|
mkdir -p "$folder"
|
|
echo "Checking for misowned files/directories in $folder"
|
|
# Stay on the same filesystem to avoid traversing mounted subpaths
|
|
# (for example read-only bind mounts under /app in dev setups).
|
|
find "$folder" -xdev \( ! -user "$RUN_UID" -o ! -group "$RUN_GID" \) \
|
|
-exec chown "$RUN_UID:$RUN_GID" {} + 2>/dev/null || true
|
|
}
|
|
|
|
# Ensure proper ownership of application directories
|
|
change_ownership() {
|
|
local folder="$1"
|
|
mkdir -p "$folder"
|
|
echo "Changing ownership of $folder to $USERNAME:$RUN_GID"
|
|
chown -R "${RUN_UID}:${RUN_GID}" "${folder}" || echo "Failed to change ownership for ${folder}, continuing..."
|
|
}
|
|
|
|
require_writable_dir() {
|
|
local folder="$1"
|
|
local label="${2:-Directory}"
|
|
|
|
if ! mkdir -p "$folder"; then
|
|
echo "Failed to create ${label} directory: $folder"
|
|
exit 1
|
|
fi
|
|
|
|
if ! test_write "$folder"; then
|
|
echo "${label} directory is not writable in non-root mode: $folder"
|
|
echo "Prepare ownership outside the container (for example with a pre-owned volume or Kubernetes fsGroup)."
|
|
exit 1
|
|
fi
|
|
}
|
|
|
|
fail_unwritable_config_dir() {
|
|
local folder="$1"
|
|
local owner
|
|
|
|
owner=$(stat -c '%u:%g' "$folder" 2>/dev/null || echo "unknown")
|
|
|
|
echo ""
|
|
echo "========================================================"
|
|
echo "ERROR: Config directory is not writable!"
|
|
echo ""
|
|
echo "Config directory: $folder"
|
|
echo "Current owner: $owner"
|
|
echo "Configured runtime identity: ${RUN_UID}:${RUN_GID}"
|
|
echo ""
|
|
echo "To fix this permanently, run on your HOST machine:"
|
|
echo " chown -R $RUN_UID:$RUN_GID /path/to/config"
|
|
echo "========================================================"
|
|
echo ""
|
|
exit 1
|
|
}
|
|
|
|
resolve_runtime_home() {
|
|
local runtime_home
|
|
|
|
runtime_home=$(getent passwd "$RUN_UID" 2>/dev/null | cut -d: -f6 || true)
|
|
case "$runtime_home" in
|
|
""|/|/app|/nonexistent)
|
|
runtime_home="$DEFAULT_RUNTIME_HOME"
|
|
;;
|
|
esac
|
|
|
|
printf '%s\n' "$runtime_home"
|
|
}
|
|
|
|
ensure_tree_writable() {
|
|
local folder="$1"
|
|
|
|
make_writable "$folder"
|
|
if [ -d "$folder" ]; then
|
|
chmod -R u+rwX,g+rwX "$folder" || echo "Failed to relax permissions for ${folder}, continuing..."
|
|
fi
|
|
}
|
|
|
|
ensure_symlinked_dir() {
|
|
local link_path="$1"
|
|
local target_path="$2"
|
|
|
|
ensure_tree_writable "$target_path"
|
|
|
|
if [ -L "$link_path" ]; then
|
|
local current_target
|
|
current_target=$(readlink "$link_path" 2>/dev/null || echo "")
|
|
if [ "$current_target" = "$target_path" ]; then
|
|
echo "$link_path already points to $target_path"
|
|
return 0
|
|
fi
|
|
echo "Replacing symlink $link_path -> $current_target with $target_path"
|
|
rm -f "$link_path" || echo "Failed to replace symlink ${link_path}, continuing..."
|
|
elif [ -d "$link_path" ]; then
|
|
echo "Moving existing scratch files from $link_path to $target_path"
|
|
find "$link_path" -xdev -mindepth 1 -maxdepth 1 -exec mv -t "$target_path" {} + 2>/dev/null || true
|
|
ensure_tree_writable "$target_path"
|
|
|
|
if ! rmdir "$link_path" 2>/dev/null; then
|
|
echo "Could not replace $link_path with symlink, leaving existing directory in place"
|
|
ensure_tree_writable "$link_path"
|
|
return 0
|
|
fi
|
|
elif [ -e "$link_path" ]; then
|
|
echo "$link_path exists and is not a directory, leaving it in place"
|
|
return 0
|
|
fi
|
|
|
|
if [ ! -e "$link_path" ]; then
|
|
ln -s "$target_path" "$link_path" || echo "Failed to create symlink ${link_path}, continuing..."
|
|
fi
|
|
}
|
|
|
|
if [ "$RUN_AS_NON_ROOT" = "true" ]; then
|
|
require_writable_dir /tmp/shelfmark "Temporary"
|
|
|
|
if [ "${USING_EXTERNAL_BYPASSER}" != "true" ]; then
|
|
require_writable_dir /tmp/shelfmark/seleniumbase/downloaded_files "SeleniumBase downloads"
|
|
require_writable_dir /tmp/shelfmark/seleniumbase/archived_files "SeleniumBase archive"
|
|
fi
|
|
|
|
require_writable_dir "${CONFIG_DIR:-/config}" "Config"
|
|
else
|
|
fix_misowned /var/log/shelfmark
|
|
fix_misowned /tmp/shelfmark
|
|
|
|
# Keep SeleniumBase on its default /app-based paths, but redirect the scratch
|
|
# directories into /tmp so bypasser startup doesn't depend on image-layer writes.
|
|
if [ "${USING_EXTERNAL_BYPASSER}" != "true" ]; then
|
|
ensure_symlinked_dir /app/downloaded_files /tmp/shelfmark/seleniumbase/downloaded_files
|
|
ensure_symlinked_dir /app/archived_files /tmp/shelfmark/seleniumbase/archived_files
|
|
|
|
# Keep SeleniumBase's bundled drivers directory writable as well for
|
|
# compatibility with legacy UC code paths that still probe bundled assets.
|
|
set +e
|
|
SELENIUMBASE_DRIVERS_DIR=$("$PYTHON_BIN" -c "import pathlib, seleniumbase; print(pathlib.Path(seleniumbase.__file__).resolve().parent / 'drivers')" 2>/dev/null)
|
|
set -e
|
|
|
|
if [ -n "$SELENIUMBASE_DRIVERS_DIR" ] && [ -d "$SELENIUMBASE_DRIVERS_DIR" ]; then
|
|
change_ownership "$SELENIUMBASE_DRIVERS_DIR"
|
|
|
|
# If the legacy driver already exists, ensure it's executable for the runtime user.
|
|
if [ -f "${SELENIUMBASE_DRIVERS_DIR}/uc_driver" ]; then
|
|
chmod +x "${SELENIUMBASE_DRIVERS_DIR}/uc_driver" || echo "Failed to chmod uc_driver, continuing..."
|
|
fi
|
|
fi
|
|
fi
|
|
|
|
# Config is Shelfmark-owned state, so it keeps the thorough repair path.
|
|
make_writable "${CONFIG_DIR:-/config}" tree
|
|
|
|
# Refuse to continue if the config directory is still not writable after repair.
|
|
CONFIG_PATH=${CONFIG_DIR:-/config}
|
|
set +e
|
|
test_write "$CONFIG_PATH" >/dev/null 2>&1
|
|
config_ok=$?
|
|
set -e
|
|
|
|
if [ $config_ok -ne 0 ]; then
|
|
fail_unwritable_config_dir "$CONFIG_PATH"
|
|
fi
|
|
|
|
# The ingest/destination library (default /books) is user data and may be a
|
|
# bind mount owned by another uid; downloads fail with "Destination not
|
|
# writable" if the runtime user can't write there. Fix the top-level dir only
|
|
# (root mode) so we don't recursively chown a potentially huge library.
|
|
make_writable "${INGEST_DIR:-/books}" root
|
|
fi
|
|
|
|
# Always run Gunicorn (even when DEBUG=true) to ensure Socket.IO WebSocket
|
|
# upgrades work reliably on customer machines.
|
|
# Map app LOG_LEVEL (often DEBUG/INFO/...) to gunicorn's --log-level (lowercase).
|
|
# Gunicorn rejects anything outside its own list, so normalize and fall back to
|
|
# info rather than letting a typo stop the container from booting.
|
|
if [ "$DEBUG" = "true" ]; then
|
|
gunicorn_loglevel=debug
|
|
else
|
|
gunicorn_loglevel=$(echo "${LOG_LEVEL:-info}" | tr '[:upper:]' '[:lower:]')
|
|
[ "$gunicorn_loglevel" = "warn" ] && gunicorn_loglevel=warning
|
|
case "$gunicorn_loglevel" in
|
|
debug|info|warning|error|critical) ;;
|
|
*) gunicorn_loglevel=info ;;
|
|
esac
|
|
fi
|
|
command="${GUNICORN_BIN} --log-level ${gunicorn_loglevel} --access-logfile - --error-logfile - --worker-class geventwebsocket.gunicorn.workers.GeventWebSocketWorker --workers 1 -t 300 -b ${FLASK_HOST:-0.0.0.0}:${FLASK_PORT:-8084} shelfmark.main:app"
|
|
|
|
# Debug mode's screen recording needs ffmpeg, which the image leaves out because nothing
|
|
# else uses it. This runs in the background so a slow or silent mirror can't hold up
|
|
# startup. A mirror can only hang the download, and killing that leaves nothing half
|
|
# installed. dpkg then works from what's on disk, and the next start's --configure -a
|
|
# clears up if a restart or the time limit cuts it off.
|
|
install_ffmpeg() {
|
|
local limit="${FFMPEG_INSTALL_TIMEOUT:-120}" status=0
|
|
export DEBIAN_FRONTEND=noninteractive
|
|
umask 0022
|
|
timeout -k 10 "$limit" dpkg --configure -a || status=$?
|
|
if [ "$status" = 0 ]; then
|
|
# No && after update: a partial index failure should not rule out the install.
|
|
timeout -k 10 "$limit" sh -c 'apt-get update -o Acquire::Retries=3
|
|
apt-get install -y --no-install-recommends --download-only ffmpeg' || status=$?
|
|
fi
|
|
if [ "$status" = 0 ]; then
|
|
timeout -k 10 "$limit" apt-get install -y --no-install-recommends --no-download ffmpeg || status=$?
|
|
fi
|
|
case "$status" in
|
|
0)
|
|
if ffmpeg -version >/dev/null 2>&1; then
|
|
echo "ffmpeg installed; bypasses will be recorded from now on."
|
|
else
|
|
echo "Could not install ffmpeg (apt finished but ffmpeg does not run); bypasses will not be recorded."
|
|
fi
|
|
;;
|
|
124 | 137) echo "Timed out installing ffmpeg after ${limit}s; bypasses will not be recorded." ;;
|
|
*) echo "Could not install ffmpeg (exit $status); bypasses will not be recorded." ;;
|
|
esac
|
|
}
|
|
|
|
# Ask the app rather than reading $DEBUG: the Debug Mode setting lives in the config file
|
|
# and needs a restart, so this is where it takes effect.
|
|
INSTALL_FFMPEG="false"
|
|
if [ "$USING_EXTERNAL_BYPASSER" != "true" ] && ! ffmpeg -version >/dev/null 2>&1 &&
|
|
"$PYTHON_BIN" -c 'from shelfmark.config.env import DEBUG; raise SystemExit(0 if DEBUG else 1)' 2>/dev/null; then
|
|
if [ "$RUN_AS_NON_ROOT" = "true" ]; then
|
|
echo "Debug mode is on but ffmpeg is not installed and non-root mode cannot install it; bypasses will not be recorded."
|
|
else
|
|
echo "Debug mode is on: installing ffmpeg in the background; bypasses before it finishes will not be recorded."
|
|
INSTALL_FFMPEG="true"
|
|
fi
|
|
fi
|
|
|
|
# If DEBUG and not using an external bypass
|
|
if [ "$DEBUG" = "true" ] && [ "$USING_EXTERNAL_BYPASSER" != "true" ]; then
|
|
set +e
|
|
set -x
|
|
echo "vvvvvvvvvvvv DEBUG MODE vvvvvvvvvvvv"
|
|
echo "Starting Xvfb for debugging"
|
|
"$PYTHON_BIN" -c "from pyvirtualdisplay import Display; Display(visible=False, size=(1440,1880)).start()"
|
|
id
|
|
free -h
|
|
uname -a
|
|
ulimit -a
|
|
df -h /tmp
|
|
env | sort
|
|
mount
|
|
cat /proc/cpuinfo
|
|
echo "==========================================="
|
|
echo "Debugging Chrome itself"
|
|
chromium --version
|
|
mkdir -p /tmp/chrome_crash_dumps
|
|
timeout --preserve-status 5s chromium \
|
|
--headless=new \
|
|
--no-sandbox \
|
|
--disable-gpu \
|
|
--enable-logging --v=1 --log-level=0 \
|
|
--log-file=/tmp/chrome_entrypoint_test.log \
|
|
--crash-dumps-dir=/tmp/chrome_crash_dumps \
|
|
< /dev/null
|
|
EXIT_CODE=$?
|
|
echo "Chrome exit code: $EXIT_CODE"
|
|
ls -lh /tmp/chrome_entrypoint_test.log
|
|
ls -lh /tmp/chrome_crash_dumps
|
|
if [[ "$EXIT_CODE" -ne 0 && "$EXIT_CODE" -le 127 ]]; then
|
|
echo "Chrome failed to start. Lets trace it"
|
|
apt-get update && apt-get install -y strace
|
|
timeout --preserve-status 10s strace -f -o "/tmp/chrome_strace.log" chromium \
|
|
--headless=new \
|
|
--no-sandbox \
|
|
--version \
|
|
< /dev/null
|
|
EXIT_CODE=$?
|
|
echo "Strace exit code: $EXIT_CODE"
|
|
echo "Strace log:"
|
|
cat /tmp/chrome_strace.log
|
|
fi
|
|
|
|
pkill -9 -f Xvfb
|
|
pkill -9 -f chromium
|
|
sleep 1
|
|
ps aux
|
|
set +x
|
|
set -e
|
|
echo "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^"
|
|
fi
|
|
|
|
# Verify /tmp has at least 1MB of space and is writable/readable
|
|
echo "Verifying /tmp has enough space"
|
|
rm -f /tmp/test.shelfmark
|
|
if dd if=/dev/zero of=/tmp/test.shelfmark bs=1M count=1 2>/dev/null && \
|
|
[ "$(wc -c < /tmp/test.shelfmark)" -eq 1048576 ]; then
|
|
rm -f /tmp/test.shelfmark
|
|
echo "Success: /tmp is writable and readable"
|
|
else
|
|
echo "Failure: /tmp is not writable or has insufficient space"
|
|
exit 1
|
|
fi
|
|
|
|
RUNTIME_HOME=$(resolve_runtime_home)
|
|
if [ "$RUN_AS_NON_ROOT" = "true" ]; then
|
|
require_writable_dir "$RUNTIME_HOME" "Home"
|
|
else
|
|
mkdir -p "$RUNTIME_HOME"
|
|
make_writable "$RUNTIME_HOME" tree
|
|
fi
|
|
|
|
if [ "$RUN_AS_NON_ROOT" = "true" ]; then
|
|
echo "Startup mode: non-root"
|
|
elif [ "$RUN_UID" = "0" ] && [ "$RUN_GID" = "0" ]; then
|
|
echo "Startup mode: root"
|
|
else
|
|
echo "Startup mode: root bootstrap with privilege drop"
|
|
fi
|
|
echo "Runtime identity: $USERNAME (${RUN_UID}:${RUN_GID})"
|
|
|
|
echo "Running command: '$command' as '$USERNAME' (debug=${DEBUG:-false})"
|
|
|
|
# Set umask for file permissions (default: 0022 = files 644, dirs 755)
|
|
UMASK_VALUE=${UMASK:-0022}
|
|
echo "Setting umask to $UMASK_VALUE"
|
|
umask $UMASK_VALUE
|
|
|
|
stop_file_logging
|
|
# Started only now because stop_file_logging waits for the log pipe to close, which a
|
|
# background job holding it would delay. The outer subshell leaves it to dumb-init to reap.
|
|
if [ "$INSTALL_FFMPEG" = "true" ]; then
|
|
if [ "$FILE_LOGGING_ENABLED" = "true" ]; then
|
|
( (install_ffmpeg 2>&1 | tee -a "$LOG_FILE") </dev/null & )
|
|
else
|
|
( (install_ffmpeg 2>&1) </dev/null & )
|
|
fi
|
|
fi
|
|
exec_as_target_user env HOME="$RUNTIME_HOME" $command
|