The default audiobook formats include `zip` and `rar`.
`scan_directory_tree` checks the supported-format list before checking
for archives, so a downloaded archive lands in `book_files` and is
imported as-is. The extraction branch in `collect_directory_files` is
never reached.
This keeps archives out of `book_files`, so they always take the archive
path: extracted when extraction is allowed, imported as-is when it isn't
(unchanged).
Tests added in `tests/download/test_postprocess_scan_archives.py`; three
of the four fail without the change.
## Problem
Two defects in `html_get_page`, either of which is enough to make an
Anna's Archive search fail *without the bypasser ever running*. Found
while chasing why AA search returned nothing on v1.3.7 even with
`USE_CF_BYPASS` on and a working bypasser.
### 1. An AA redirect loop is treated as a network fault
AA serves its DDoS-Guard handshake as a same-host redirect loop:
`/search?…` redirects to `/search?…&check=1`, which redirects back,
indefinitely. The manual redirect follower counts those against
`_MAX_REDIRECTS` and raises `TooManyRedirects`:
```python
redirects_followed += 1
if redirects_followed > _MAX_REDIRECTS:
_raise_too_many_redirects(f"Too many redirects for {current_url}")
```
That lands in the retry path, so every one of the `MAX_RETRY` attempts
re-runs the same 6-redirect loop and the URL is never offered to the
bypasser — which is the only thing that can clear the challenge. With
the default `MAX_RETRY=10` that's ~60 requests to AA per search, all of
which can only fail:
```
Retry 5/10 for https://annas-archive.pk/search?…&check=1: TooManyRedirects
Retry 6/10 for https://annas-archive.pk/search?…&check=1: TooManyRedirects
…
Giving up after 10 attempts: https://annas-archive.pk/search?…&check=1
```
Surfaced to the user as `Unable to reach download source. Network
restricted or mirrors are blocked.`
### 2. Both bypasser handoffs are a no-op at `MAX_RETRY=1`
The existing 403 handoff — and the new redirect one — set a flag and
`continue`:
```python
logger.info("403 detected; switching to bypasser: %s", current_url)
use_bypasser_now = True
continue
```
The branch that acts on `use_bypasser_now` sits at the top of the
**next** retry attempt. With `MAX_RETRY=1` there is no next attempt, so
a 403 simply ends the search and the bypasser never runs. `MAX_RETRY` is
user-configurable down to 1, so this is reachable in normal use.
The redirect handoff had an additional problem: it sits inside the inner
redirect `while`, so a `continue` there re-enters *that* loop rather
than reaching the retry branch at all.
## Change
Both handoffs now invoke the bypasser directly, through a shared
`_run_bypasser()` closure extracted from the existing branch body. No
behaviour change to the bypass itself — same grace handling, same error
reporting, same `finally`.
The redirect handoff also honours `allow_bypasser_fallback`, for the
same reason the 403 path does: callers such as the `/dyn/md5/summary/…`
fetch behind the details modal pass `False` precisely so a best-effort
request fails fast instead of holding the UI open for a minutes-long
browser solve.
## Result
Measured against `/api/releases` for the same book, internal bypasser,
default `MAX_RETRY`:
| | searches returning results |
|---|---|
| before | 4 / 9 |
| after | 3 / 3, then 7 / 7 |
Zero `TooManyRedirects` give-ups after, and the new path is visible in
the logs:
```
redirect loop on https://annas-archive.gl/search?…&check=1; switching to bypasser
Bypass successful using _bypass_method_cdp_gui_click
```
The request volume drop is the other half of the win — a failing search
no longer emits ~60 requests to AA before giving up.
## Notes
- Only `shelfmark/download/http.py` changes; no config or API surface.
- `use_bypasser_now` is still set before each direct call, so the guard
against double-invocation is unchanged.
- Tested with the internal bypasser (seleniumbase). The
external-bypasser path goes through the same `get_bypassed_page()` call
and is unaffected by the control-flow change, though I have not measured
it against DDoS-Guard specifically — in my testing
FlareSolverr-compatible solvers do not clear that challenge regardless.
Co-authored-by: D <d@e>
David Young
