The retry in read_settled_html covered page.content() racing a
navigation. #418 only showed that race on the early read taken right
after domcontentloaded, and this branch already removes that read. The
late read now matches the Cloudflare path, which reads after the idle
wait and has never retried.
When the retry fires at the end of the budget, remaining_ms() is 1 ms.
The second idle wait then times out at once, and the second read lands
in the same 60-150 ms window that failed. So the retry adds a helper and
threads timer into content.py for a case it does not cover.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When no Cloudflare interstitial was detected, _navigate_and_solve()
snapshotted page.content() right after domcontentloaded and
build_response_content() returned that snapshot, even though
_wait_for_networkidle() ran afterwards. A site with its own JS guard
(com-x.life: 302 to a proof-of-work page that sets cookies and calls
location.replace) finishes its redirect during that wait, so Byparr
returned the guard page while solution.url already named the target.
The snapshot came in with the localized-detection change (#388) only to
reuse HTML that had already been read; before it the body was always
read after the idle wait. Drop it, along with the challenge_detected and
page_html plumbing, and read the settled page on both branches.
The early read was also a liability of its own: a guard redirecting
within a few hundred milliseconds of domcontentloaded makes that read
fail with "Failed to find execution context", which escaped read_item
as a 500. The late read can still race a navigation that lands right at
the budget's edge, so if it fails, wait for network idle again and read
once more. The error is caught as invisible_playwright's Error, which is
what the browser actually raises.
Fixes#418. Thanks to c4uran for the analysis.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FlareSolverr hardcodes the solution status ("todo: fix, selenium not
provides this info"), so clients built against it never see anything else.
Byparr handed back the real navigation code on the branch without a
challenge and 200 on the branch with one, which is neither honest nor
compatible. Always report 200.
Move the challenge handling into src/challenge.py and the response bodies
into src/content.py, leaving endpoints.py with the routes and navigation.
That also confines the import of playwright_captcha's private detection
module to a single file, so a patch release can only break one import
instead of the app and the test module at once.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>