Run trafilatura server-side on the rendered DOM (page.content()), so
JS-rendered pages stay fully visible to the extractor; fall back to
innerText when trafilatura cannot score any main content.
- Move OWUI_API_KEY into pydantic settings (src/consts.py); drop the
Dockerfile ENV entry so the key is only ever set at runtime
- Enforce auth before the browser is launched via dependency ordering
- Compare bearer tokens in constant time (hmac.compare_digest)
- Keep extracting when networkidle times out, matching /v1 behavior
- Type page as Page, drop redundant comments and docstrings
Add /load endpoint for Open WebUI's WEB_LOADER_ENGINE=external integration.
Uses document.body.innerText for content extraction.
Configure in Open WebUI:
WEB_LOADER_ENGINE=external
EXTERNAL_WEB_LOADER_URL=http://byparr:8191/load
EXTERNAL_WEB_LOADER_API_KEY=<OWUI_API_KEY env var>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
ThePhaseless
Alex Thomson
marchingphoenixandClaude Opus 4.5