mirror of
https://github.com/calibrain/shelfmark.git
synced 2026-11-15 20:28:14 +00:00
Jorge Lima
97d1bb0df4
## What Add `aa_ddg_check` to the cookie-store allowlist. One name, one test. ## Why Every replay of stored clearance ends in the `?check=1` redirect loop, so each search pays a fresh browser solve. On this instance (v1.3.15, WireGuard egress, 0 VPN restarts across the traces) not one replay was accepted in three days of DEBUG logs. The `__ddg*` cookies are stored and replayed correctly. Anna's Archive also sets a cookie of its own, `aa_ddg_check`, and its `?check=1` hop only answers with the page when that cookie is present too. The allowlist keeps `cf_*` and `__ddg*` names, so this one was never stored. ## Measured, same egress IP, cookies taken from one solve | replayed | plain `requests` | `curl_cffi`, Chrome TLS fingerprint | |---|---|---| | filtered `__ddg*` only (current behaviour) | 302 → 302 → 302 … loop | 302 → 302 → 302 … loop | | filtered + `__ddg8_/9_/10_` | loop | loop | | filtered + `aa_ddg_check` | **302 → 200, real search page** | 302 → 200 | | `aa_ddg_check` alone | 302 → 403 | — | So the TLS fingerprint is not the problem, the per-check trio is not the answer, and the cookie needs the `__ddg*` clearance next to it. Cookie attributes as issued: domain `.annas-archive.gl`, path `/`, expiry 90 days. It is not bound to the query, and it is accepted with a stock Python User-Agent. ## Through the real fetch path Same process, `html_get_page`, the name allowlisted, three different queries: ``` 1st: solve expected 25.8s bypass_calls=1 title='frankenstein shelley - search - an' md5=True 2nd: other query 9.5s bypass_calls=0 title='pride and prejudice austen - searc' md5=True 3rd: third query 4.8s bypass_calls=0 title='dracula stoker - search - anna's a' md5=True ``` ## Notes - `tests/bypass/test_ddg_cookie_reuse.py` gains `test_aa_check_cookie_is_stored`; its docstring table gains the row. The bypass tests need seleniumbase to import and do not run on my macOS host, so this leans on CI. `ruff check` and `ruff format --check` pass. The logic was checked directly against `cookie_store` with the settings registry stubbed. - `__ddgmark_` carries a 24 h expiry, so the store's clearance is good for about a day before the next solve, which is what a browser would see too. - Follow-up to #1286. Same instance, same method: DEBUG trace, then a probe script inside the container.