## Summary
Two failure modes on the same code path, both reported this week: Anna's
Archive `/search` is gated behind a DDoS-Guard cookie probe that the
manual redirect follower can never satisfy.
**#1202 — the cookie is dropped on every hop.** AA URLs set
`allow_redirects = False`, so `html_get_page` follows redirects by hand.
The 302 to `?check=1` carries a `Set-Cookie` (`__ddg*`) that has to come
back on the next request. Because cookies are passed per call and
`requests` keeps no jar across manual hops, it was discarded each time
and the server just re-issued the same redirect until `_MAX_REDIRECTS`
raised `TooManyRedirects`. The file already had the right helper —
`_new_cookies()` — but only the 503 Z-Library handshake branch called
it.
**#1204 — the loop never reaches the bypasser.** `TooManyRedirects`
isn't in `_is_retryable_error` and carries no status code, so the 403
rescue path (`status == _HTTP_STATUS_FORBIDDEN`) never fired and all
attempts repeated the identical failure — ~2.5 min, surfacing as the
misleading "Network restricted or mirrors are blocked".
These interact, which is why #1202's fix alone isn't enough. Requests
merge as `cookies={**handshake_cookies, **cookies}`, so **stale bypasser
cookies override the fresh handshake ones** — once `_cf_cookies` holds
an expired `__ddg*`, the probe can never clear no matter how faithfully
we echo. Hence one search per restart, exactly as #1204 describes.
## Changes
1. Harvest cookies in the same-host redirect branch, the way the 503
branch already does. `_new_cookies()` returns only *new* values, so a
server re-sending an identical cookie yields an empty dict and a genuine
redirect loop still terminates at `_MAX_REDIRECTS`.
2. Treat a redirect loop as a detected challenge: purge the stored
cookies for that host and switch to the bypasser, instead of burning the
retry budget. Gated on `allow_bypasser_fallback` and
`_is_cf_bypass_enabled()`, and skipped when already bypassing, so
AudiobookBay (`allow_bypasser_fallback=False`) and external-bypasser
setups are unaffected.
The broader point in #1204 stands — the fallback would be better gated
on "challenge detected" than on specific status codes, since DDoS-Guard
presents at least three faces (403 js-challenge, 429, and this redirect
loop). This PR fixes the two live exits without that refactor.
## Tests
Two regression tests, both failing before and passing after:
- `test_html_get_page_echoes_cookies_across_same_host_redirects` — the
fake server only returns results if `__ddg2_` comes back on the
`?check=1` hop.
- `test_html_get_page_redirect_loop_purges_cookies_and_bypasses` —
asserts the stored cookies are cleared, the bypasser runs, and the loop
is cut short rather than repeated per attempt.
`ruff check` and `ruff format` clean. `tests/download/` passes except
`test_download_url_ignores_zlib_cookie_refresh_failure`, which fails
identically on unmodified `main` in my environment (no `seleniumbase` —
the `browser` extra isn't installed).
## Verification
Applied on a live v1.3.7 install (Debian LXC, internal CDP bypasser).
Before: every search timed out through 10 retries with
`TooManyRedirects`, zero results. After:
```
http.py:455 - Redirect loop detected; switching to bypasser
internal_bypasser.py:756 - Bypass successful using _bypass_method_cdp_gui_click
internal_bypasser.py:322 - Extracted 9 protection cookies for annas-archive.pk
direct_download.py:1865 - Found 24 releases via ISBN
```
~25 s per search, results render. Note the second search still re-solves
the challenge, since the freshly stored cookies go stale immediately —
the design issue #1204 raises, left for the broader fix.
Fixes#1202Fixes#1204🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_012Ln3yVj3sWHG2c6T78W1we
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
