A page whose network never goes idle (background analytics, websockets)
used to fail the whole request with a 408 once the networkidle wait
expired. Since the DOM is fully usable after domcontentloaded, treat a
networkidle timeout as non-fatal and return the loaded page instead.
Fatal timeouts during initial load or challenge solving still return 408.
Adds unit coverage for both paths using a fake page that fails
configured load-state waits.
The Firefox engine evaluates JS via eval(), which pages whose CSP
lacks 'unsafe-eval' block - every page.evaluate() then fails with
"call to eval() blocked by CSP". yggtorrent's search URL redirects to
a page with such a CSP, crashing the user-agent read and 500ing /v1.
Rewrite document responses without CSP headers via route.fetch +
fulfill. Juggler only routes the first request of a redirect chain,
so follow redirects inside the fetch and record the final URL
ourselves instead of relying on page.url.
Add a maxTimeout alias to LinkRequest.max_timeout for FlareSolverr
drop-in compatibility. Values of 1000 or more are treated as
milliseconds and normalized to seconds; smaller values keep the
native seconds semantics. Closes#382.
Install invisible-playwright from PyPI rather than by git URL
Conflict resolution: keep the >=0.6.1 floor set by the follow-up
version bump; uv.lock already resolves to 0.6.1.
On IPv6-enabled Docker networks, 'localhost' resolves to ::1 first,
but uvicorn binds to 0.0.0.0 by default, so the healthcheck can fail.
Using 127.0.0.1 avoids the IPv6/IPv4 mismatch.
Fixes#346
- Add pydantic-settings as direct dependency
- Replace os.getenv calls with typed Settings class
- Add BLOCK_MEDIA and RETURN_ONLY_COOKIES env defaults
- Catch both builtins.TimeoutError and playwright TimeoutError as 408
- Check challenge title before networkidle to avoid timeout on Cloudflare interstitial
- Add blockMedia and returnOnlyCookies request options
- Return raw PDF bytes as base64 with contentType application/pdf
- Skip tests on 408 timeouts; add PDF handling test
- Replace camoufox[geoip] with invisible_playwright git dependency
- Switch playwright-captcha framework from CAMOUFOX to PLAYWRIGHT
- Remove camoufox addon path from consts
- Add git to Docker base image; fetch invisible_playwright binary
- Make /cache writable for runtime USER 1000
cloverlabs-camoufox 0.6.0 was a confirmed regression (3/6 tests failed
with 'Cloudflare iframes not found' vs 6/6 passing on camoufox 0.4.11).
Revert to camoufox[geoip]==0.4.* and pin playwright==1.60.* (exact pin
to avoid the 1.61 protocol error).
ubuntu:latest rolled to 26.04 LTS on 2026-05-06, breaking the Docker
build for 50+ consecutive CI runs. Playwright 1.58.0 (pinned in uv.lock)
cannot install firefox deps for ubuntu26.04-x64 -- it prints 'Cannot
install dependencies for ubuntu26.04-x64 with Playwright 1.58.0!' and
installs nothing, leaving libgtk-3.so.0 absent. Camoufox's bundled
Firefox then fails to load XPCOM at runtime:
libgtk-3.so.0: cannot open shared object file: No such file or directory
Couldn't load XPCOM.
Pinning to 24.04 (the last-known-good base, supported by Playwright 1.58)
restores libgtk-3-0t64 and the rest of the GTK runtime. Adopted from PR #362
which independently diagnosed the same issue.
Verified locally:
- app stage: ldconfig shows libgtk-3.so.0 present (was absent)
- test target: 6/6 tests pass (was BrowserType.launch failure)
- runtime: POST /v1 returns 200 status:ok (was 500 libgtk-3 traceback)
Jakub Orchowski
Federico
renovate[bot]
CopilotandThePhaseless