12 Commits
Author SHA1 Message Date
Alex 4472fbe8cf Download overhaul - DNS fallback, bypasser enhancements, revamped error handling, better frontend UX (#336)
## Changelog

### 🌐 Network Resilience

- **Auto DNS rotation**: New `CUSTOM_DNS=auto` mode (now default) starts
with system DNS and automatically rotates through Cloudflare, Google,
Quad9, and OpenDNS when failures are detected. DNS results are cached to
improve performance.
- **Mirror failover**: Anna's Archive requests automatically fail over
between mirrors (.org, .se, .li) when one is unreachable
- **Round-robin source distribution**: Concurrent downloads are
distributed across different AA partner servers to avoid rate limiting

### 📥 Download Reliability

- **Much more reliable downloads**: Improved parsing of Anna's Archive
pages, smarter source prioritization, and better retry logic with
exponential backoff
- **Download resume support**: Interrupted downloads can now resume from
where they left off (if the server supports Range requests)
- **Cookie sharing**: Cloudflare bypass cookies are extracted and shared
with subsequent requests, often avoiding the need for re-bypass entirely
- **Stall detection**: Downloads with no progress for 5 minutes are
automatically cancelled and retried
- **Staggered concurrent downloads**: Small delays between starting
concurrent downloads to avoid hitting rate limits
- **Source failure tracking**: After multiple failures from the same
source type (e.g., Libgen), that source is temporarily skipped
- **Lazy welib loading**: Welib sources are fetched as a fallback only
when primary sources fail (unless `PRIORITIZE_WELIB` is enabled)

### 🛡️ Cloudflare & Protection Bypass

- **DDOS-Guard support**: Internal bypasser now detects and handles
DDOS-Guard challenges with dedicated bypass strategies
- **Cancellation support**: Bypass operations can now be cancelled
mid-operation when user cancels a download
- **Smart warmup**: Chrome driver is pre-warmed when first client
connects (controlled by `BYPASS_WARMUP_ON_CONNECT` env var) and shuts
down after periods of inactivity

### 🔌 External Bypasser (FlareSolverr)

- **Improved resilience**: Retry with exponential backoff, mirror/DNS
rotation on failure, and proper timeout handling
- **Cancellation support**: External bypasser operations respect
cancellation flags

### 🖥️ Web UI Improvements

- **Simplified download status**: Removed intermediate states
(bypassing, verifying, ingesting) — now just shows Queued → Resolving →
Downloading → Complete
- **Status messages**: Downloads show detailed status like "Trying
Anna's Archive (Server 3)" or "Server busy, trying next...", or live
waitlist countdowns.
- **Improved download sidebar**:
  - Downloads sorted by add time (newest first)
  - X button moved to top-right corner for better UX
  - Wave animation on in-progress items
  - Error messages shown directly on failed items
  - X button on completed/errored items clears them from the list

### ⚙️ Configuration Changes

- **`CUSTOM_DNS=auto`** is now the default (previously empty/system DNS)
- **`DOWNLOAD_PROGRESS_UPDATE_INTERVAL`** default changed from 5s to 1s
for smoother progress
- **`BYPASS_WARMUP_ON_CONNECT`** (default: true) — warm up Chrome when
first client connects

### 🐛 Bug Fixes

- **Download cancellation actually works**: Fixed issue where cancelling
downloads didn't properly stop in-progress operations
- **WELIB prioritization**: Fixed `PRIORITIZE_WELIB` not being respected
- **File exists handling**: Downloads to same filename now get `_1`,
`_2` suffix instead of overwriting
- **Empty search results**: "No books found" now returns empty list
instead of throwing exception
- **Search unavailable error**: Network/mirror failures during search
now return proper 503 error to client
2025-12-14 21:18:05 -05:00
CaliBrain 78c61e88b3 Fix kwargs bug in tracing log (#327) 2025-11-23 01:06:00 -05:00
CaliBrain 712f71a466 Add DEBUG button (#147) 2025-04-20 05:32:11 -04:00
CaliBrain 00d0a2e7c3 more debug logging (#135) 2025-04-15 10:52:25 -04:00
CaliBrain 57f15b4d95 Tor support, use SeleniumBase instead of DrissionPage (#123)
Refactor: Improve Docker build, add Tor support, use SeleniumBase

- Overhauled Dockerfile:
    - Switched to python:3.10-slim base.
    - Implemented multi-stage builds (base, standard, tor).
    - Consolidated RUN layers for efficiency.
    - Added locale/timezone setup.
- Added Tor setup and iptables configuration in dedicated stage/script.
- Replaced DrissionPage with SeleniumBase for Cloudflare bypassing.
- Added Tor support via `docker-compose.tor.yml` and `tor.sh` script.
- Updated `docker-compose.yml` to build locally and changed default port
to 8083.
- Added `docker-compose.dev.yml` and `docker-compose.tor.dev.yml`.
- Updated `entrypoint.sh` for timezone and sudo usage.
- Added/Updated environment variables (`TZ`, `USING_TOR`, etc.).
- Improved `.dockerignore`.
- Updated `readme.md` to reflect port changes, build process, document
`TZ`, and add details about the new Tor variant.
2025-04-13 02:34:39 -04:00
CaliBrain 785a1cd7c4 Fix order of imports (#111) 2025-03-29 11:52:45 -04:00
CaliBrain 5b2b640937 DNS (#108)
This should help fix #106 #103
2025-03-29 03:36:34 -04:00
CaliBrain 93a3481750 Fix errors and improve logging (#102)
Fix #100
Fix #101
2025-03-22 17:35:12 -04:00
CaliBrain c2e7acd473 Use a tmp name while downloading files (#99)
Should help with #98
2025-03-20 00:56:54 -04:00
CaliBrain 2b80de35a8 [FIX] Actually log to a file (#44) 2025-01-07 14:04:50 -05:00
CaliBrain b8a7247538 [FIX] Refactor code to fix typing (#23)
This also fixes some bug with the CF bypasser and the file checker
2024-12-22 22:11:35 -05:00
CaliBrain 18b146af64 Innitial Release 2024-12-16 22:48:26 +00:00