Commit Graph
3 Commits
Author SHA1 Message Date
splitsec2 c6685feba5 fix(docker): drop ffmpeg from the default image; install on demand in Debug mode (#1631)
ffmpeg is in the default image for one thing, the Debug mode screen
recording (`_start_ffmpeg_recording`). Nothing else calls it,
SeleniumBase included, and it brings 93 packages with it. On real builds
that's 406 Trivy findings (149 High) and 78 MB of the gzipped amd64
image. This takes it out of the image and installs it when Debug mode
needs it.

It came out of the base image discussion on #1616 and helps on any base,
though on Wolfi the install line would need `apk`.

## What changed

- **Dockerfile:** ffmpeg is gone from the full image's install.
- **Guard:** `_start_ffmpeg_recording` checks `shutil.which("ffmpeg")`
first. If it's missing it logs a warning and returns, and the bypass
carries on. Without this, Debug on with no ffmpeg fails every bypass
with `FileNotFoundError`. There's a new test for it.
- **Entrypoint:** when Debug is on, the internal bypasser is in use and
ffmpeg isn't already there, it installs ffmpeg in the background so the
app doesn't wait for it. The download and the install each run under a
120 s `timeout` with `DEBIAN_FRONTEND=noninteractive`. Downloading first
means a hung mirror can only be killed before dpkg starts, and the next
start runs `dpkg --configure -a` in case a restart cut dpkg off. It logs
when it starts, succeeds, times out or fails.
- **Docs:** a short paragraph in `docs/troubleshooting.md` covering the
install and the non-root limit.

## Debug has two sources

Debug can come from the `DEBUG` env var or from the Debug Mode setting
in the UI, which is saved to `/config/plugins/advanced.json`. The
existing block in `entrypoint.sh` only reads the env var, so an install
keyed on `$DEBUG` would miss anyone who turned it on in the UI.

The new check asks the app instead (`shelfmark.config.env.DEBUG`), which
reads both. The UI setting is `requires_restart`, so the restart it
already asks for is what brings it through the entrypoint.

## Non-root mode

A container started as `1000:1000` can't run `apt-get`, so the
entrypoint logs that it can't install ffmpeg, the guard warns on each
bypass, and there's no recording. Everything else works. The workaround
for a debugging session is to start as root with `PUID`/`PGID`.

## Startup cost

None that I could measure. With Debug on, `/api/health` answers in the
same time as main (3-4 s with Debug only in config, 9-10 s from env),
and ffmpeg was ready 6-8 s later on the runners.

The trade-off is that the first bypass after startup may be unrecorded
while the install runs. It logs the warning and passes. I saw both
outcomes on a bypass started right after health.

From reading the code: the install lives in the container, so recreating
the container installs it again. It also runs after `tor.sh` and
`wireguard.sh`, so apt goes through whichever egress is active.

## Testing

Real Docker builds on GitHub Actions on my fork, amd64 and arm64, with
the same result on both for every case below.

| Case | Result |
|---|---|
| Debug off | No install, `/api/health` in 3-4 s, same as main |
| Debug only in config | Health in 3-4 s, ffmpeg ready 6-8 s later, real
mp4 after a bypass |
| Debug from env | Health in 9-10 s (same as main), real mp4 after a
bypass |
| Started as root, app dropped to uid 1000 | Smoke and bypass pass |
| Started with `--user 1000`, Debug on | Logs both messages, bypass
passes, no mp4 |
| apt blocked (closed proxy) | Logs `Could not install ffmpeg`, bypass
passes, no mp4 |
| apt's proxy on a dropped address, so no TCP reply at all | Health in
9-10 s, a bypass during the install passes with the warning and no mp4.
apt gives up by itself in under a minute. Nothing left running, `dpkg
--audit` clean |
| DNS server on a dropped address | Health in 9-10 s, `Timed out
installing ffmpeg after 120s`. Nothing left running, `dpkg --audit`
clean |
| `--network none`, Debug on | No hang, health 200 from inside the
container |
| Container restarted while dpkg was running | Next start finishes the
install, `dpkg --audit` clean, real mp4 |
| Fresh named volume, Debug only in config | Recording made, uid 1000
can write |
| Existing uid-1000 volume, Debug only in config | Same, nothing
root-owned afterwards |
| `find /config /var/log/shelfmark -user root` | 0 files, same as main |
| `find /app -user root` | 245, same count as main |
| `tests/e2e/platform/run-e2e.sh` with `dns-blocked` and `baseline` |
Pass |

The bypass checks ran against example.com. A real Cloudflare challenge
was served by a demo page in the same runs, and no build solved it,
including main, so that part doesn't say anything about this change.

Locally: 5494 Python tests pass, and `ruff`, `basedpyright`, `vulture`
and `shellcheck` are clean.

The full results are in [the base image
report](https://github.com/splitsec2/shelfmark/blob/7afcecfb76727724a71561cbc078f091569b58dc/docs/research/2026-11-01-base-image-report.md).
2026-11-01 18:09:11 -07:00
CaliBrain e1c3f057ab fix: bypass recordings, welib wrong-md5 links, footer build sha (#1364) (#1373)
Debug screen recordings never started. Every bypass logged "Capturearea
1540x1050 at position 0.0 outside the screen size 1440x1880".We ask
ffmpeg for the fingerprint screen size plus margin, the size wealso pass
SeleniumBase as xvfb_metrics. SeleniumBase builds thatdisplay with
use_xauth=True, the image ships no xauth binary, so itfalls back to a
fixed 1440x1880 Xvfb and the requested size neverexists. Drop
-video_size so x11grab records the whole screen, whateversize it turned
out to be.

welib could hand back a link for a different book. Welib
answers/md5/<md5> with a search for that md5; when it does not have the
file,the resolver took the first "Download" on the results page
(md5a2c1dc0c... resolved to auto_download/9c8cf85d...). On an
/md5/<md5>page a GET/Download link is now only taken when its href names
thatmd5; otherwise the source is reported as not having the file.
Alsoremoves _get_download_urls_from_welib and _is_source_enabled:
themd5-template branch in _get_urls_for_source always handles welib
first,so that resolver could never run.

The footer showed the build date instead of the commit. CI
stampsBUILD_VERSION as <yyyy-mm-dd>-<sha> (pr-<sha> for PR images) and
thefooter kept its first seven characters, so dev images read
"Shelfmarkmain (2026-09)". Take the trailing commit sha instead: "main
(1a5b37d)".The full BUILD_VERSION stays in the hover title
2026-09-21 00:15:41 -04:00
CaliBrain 97e289ae13 fix: search, Prowlarr and qBittorrent follow-ups (#1276, #1283) (#1284) 2026-08-30 03:09:13 -04:00