From fcdc2dfb690d4465164e05c0993c32077f750502 Mon Sep 17 00:00:00 2001 From: splitsec2 <35583321+splitsec2@users.noreply.github.com> Date: Fri, 2 Oct 2026 14:30:10 -0600 Subject: [PATCH] perf(docker): stop shipping build-only content in the runtime image (#1404) While checking why each build took so much disk on my server, I looked at what's actually in the runtime image. A good share of it is there for the build and never used after. **The C toolchain.** The base stage installs `gcc`, `g++`, `libffi-dev` and `python3-dev` for building C extensions, and they stay in both the full and lite images, which is 290 MB installed. Nothing gets compiled any more. Every compiled dependency in `uv.lock` (cffi, gevent, greenlet, zope-interface) ships a cp314 manylinux wheel for both amd64 and arm64, and the packages that only have an sdist are pure Python. `python3-dev` was also pulling Debian's `libpython3.13` into a 3.14 image. **The build context.** `COPY . .` puts the whole context into `/app`, so `tests/`, `docs/` and the frontend source ship too. `.dockerignore` now leaves out the trees nothing reads at runtime. `src/` can't go in `.dockerignore` because the frontend-builder stage needs it, so both final stages remove it after the built dist is copied. The venv's own `pip` goes as well, since uv seeds one and nothing installs at runtime. This part is @DrNgo's work from his fork, and the commit carries his name. Measured by building both targets on native amd64 and arm64 runners: | | amd64 full | amd64 lite | arm64 full | arm64 lite | |---|---|---|---|---| | before | 1,532 MB | 589 MB | 1,528 MB | 619 MB | | after | 1,238 MB | 295 MB | 1,251 MB | 342 MB | Compressed, the full image goes from 617 MB to 504 MB on amd64. On both architectures the image builds, reaches healthy, and Chromium starts the same way the bypasser starts it (xvfb, `_get_browser_args()`). The Python suite passes too. It's also running on my own install now. If a dependency ever needs compiling again, the clean fix is a builder stage that builds the wheel and copies it in, rather than putting the toolchain back in the runtime image. Happy to add that now if you'd rather have it in place. --------- Co-authored-by: Michael Ngo --- .dockerignore | 19 +++++++++++++++++++ Dockerfile | 27 ++++++++++++++++++++++----- 2 files changed, 41 insertions(+), 5 deletions(-) diff --git a/.dockerignore b/.dockerignore index df06f328..8491d13e 100644 --- a/.dockerignore +++ b/.dockerignore @@ -48,3 +48,22 @@ src/frontend/.vite/ templates/ static/css/ static/js/ + +# Developer-only trees. `COPY . .` puts the whole context into /app, so anything +# left here ships to production: the CodeGraph index alone was 38MB of the image. +# NOTE: this file applies to EVERY stage, so `src/` cannot be listed — the +# frontend-builder stage needs it. The final stage drops it after the copy. +# Deliberately NOT listed: `data/` (languages.py:19 reads +# data/book-languages.json at runtime) and `genDebug.sh` (the final stage +# chmod +x's it, so excluding it fails the build). +.codegraph/ +.ruff_cache/ +.claude/ +tests/ +docs/ +scripts/ +compose/ +downloaded_files/ +frontend-dist/ +SESSION_STATE.md +Makefile diff --git a/Dockerfile b/Dockerfile index 1b2395a4..cb01304d 100644 --- a/Dockerfile +++ b/Dockerfile @@ -57,11 +57,11 @@ ENV FLASK_PORT=8084 # Configure locale, timezone, and perform initial cleanup in a single layer RUN apt-get update && \ apt-get install -y --no-install-recommends \ - # For building C-extensions (cffi, gevent, etc.) - gcc \ - g++ \ - libffi-dev \ - python3-dev \ + # No C toolchain: every compiled dependency (cffi, gevent, greenlet, + # zope-interface) ships a cp314 manylinux wheel for amd64 and arm64, and + # the sdist-only ones are pure Python. The toolchain was 290MB of the + # runtime image, and python3-dev pulled Debian's libpython3.13 into a 3.14 + # image. If a dependency ever needs compiling, build it in a separate stage. # For locale locales tzdata \ # For healthcheck @@ -209,6 +209,13 @@ RUN --mount=type=cache,target=/root/.cache/uv \ uv pip install --python /app/.venv/bin/python --reinstall python-xlib==0.33 && \ /app/.venv/bin/python -c "import Xlib.X; assert hasattr(Xlib.X, 'FamilyServerInterpreted'), 'Xlib.X.FamilyServerInterpreted missing after fix'; print('Xlib namespace OK:', Xlib.__version__)" +# The venv's own pip goes too. uv seeded a copy into /app/.venv and nothing +# installs at runtime. setuptools deliberately STAYS: several deps still import +# pkg_resources on the hot path. +RUN rm -rf /app/.venv/bin/pip /app/.venv/bin/pip3 /app/.venv/bin/pip3.* \ + /app/.venv/lib/python*/site-packages/pip \ + /app/.venv/lib/python*/site-packages/pip-*.dist-info + # Keep SeleniumBase's bundled driver cache writable for the fixed non-root user. RUN SELENIUMBASE_DRIVERS_DIR=$(/app/.venv/bin/python -c "import pathlib, seleniumbase; print(pathlib.Path(seleniumbase.__file__).resolve().parent / 'drivers')") && \ chown -R 1000:1000 "${SELENIUMBASE_DRIVERS_DIR}" && \ @@ -224,6 +231,11 @@ COPY . . COPY --from=frontend-builder /frontend/dist /app/frontend-dist +# The frontend SOURCE cannot be excluded via .dockerignore — that file applies +# to every stage and frontend-builder needs src/. Only the built dist (copied +# above) is served at runtime, so drop the source here. +RUN rm -rf /app/src + # Image-owned runtime paths for the fixed non-root user. Root/PUID mode still # re-homes ownership at startup when needed. RUN mkdir -p \ @@ -262,6 +274,11 @@ COPY . . COPY --from=frontend-builder /frontend/dist /app/frontend-dist +# The frontend SOURCE cannot be excluded via .dockerignore — that file applies +# to every stage and frontend-builder needs src/. Only the built dist (copied +# above) is served at runtime, so drop the source here. +RUN rm -rf /app/src + # Image-owned runtime paths for the fixed non-root user. Root/PUID mode still # re-homes ownership at startup when needed. RUN mkdir -p \