Compare commits

...
43 Commits
Author SHA1 Message Date
Alex 0cac541c0b Update Readme with new changes (#344) 2025-12-15 10:40:02 -05:00
CaliBrain 85c8c9151d Fix tor timeout (#343)
Fix for #340
2025-12-14 22:18:18 -05:00
Alex 4472fbe8cf Download overhaul - DNS fallback, bypasser enhancements, revamped error handling, better frontend UX (#336)
## Changelog

### 🌐 Network Resilience

- **Auto DNS rotation**: New `CUSTOM_DNS=auto` mode (now default) starts
with system DNS and automatically rotates through Cloudflare, Google,
Quad9, and OpenDNS when failures are detected. DNS results are cached to
improve performance.
- **Mirror failover**: Anna's Archive requests automatically fail over
between mirrors (.org, .se, .li) when one is unreachable
- **Round-robin source distribution**: Concurrent downloads are
distributed across different AA partner servers to avoid rate limiting

### 📥 Download Reliability

- **Much more reliable downloads**: Improved parsing of Anna's Archive
pages, smarter source prioritization, and better retry logic with
exponential backoff
- **Download resume support**: Interrupted downloads can now resume from
where they left off (if the server supports Range requests)
- **Cookie sharing**: Cloudflare bypass cookies are extracted and shared
with subsequent requests, often avoiding the need for re-bypass entirely
- **Stall detection**: Downloads with no progress for 5 minutes are
automatically cancelled and retried
- **Staggered concurrent downloads**: Small delays between starting
concurrent downloads to avoid hitting rate limits
- **Source failure tracking**: After multiple failures from the same
source type (e.g., Libgen), that source is temporarily skipped
- **Lazy welib loading**: Welib sources are fetched as a fallback only
when primary sources fail (unless `PRIORITIZE_WELIB` is enabled)

### 🛡️ Cloudflare & Protection Bypass

- **DDOS-Guard support**: Internal bypasser now detects and handles
DDOS-Guard challenges with dedicated bypass strategies
- **Cancellation support**: Bypass operations can now be cancelled
mid-operation when user cancels a download
- **Smart warmup**: Chrome driver is pre-warmed when first client
connects (controlled by `BYPASS_WARMUP_ON_CONNECT` env var) and shuts
down after periods of inactivity

### 🔌 External Bypasser (FlareSolverr)

- **Improved resilience**: Retry with exponential backoff, mirror/DNS
rotation on failure, and proper timeout handling
- **Cancellation support**: External bypasser operations respect
cancellation flags

### 🖥️ Web UI Improvements

- **Simplified download status**: Removed intermediate states
(bypassing, verifying, ingesting) — now just shows Queued → Resolving →
Downloading → Complete
- **Status messages**: Downloads show detailed status like "Trying
Anna's Archive (Server 3)" or "Server busy, trying next...", or live
waitlist countdowns.
- **Improved download sidebar**:
  - Downloads sorted by add time (newest first)
  - X button moved to top-right corner for better UX
  - Wave animation on in-progress items
  - Error messages shown directly on failed items
  - X button on completed/errored items clears them from the list

### ⚙️ Configuration Changes

- **`CUSTOM_DNS=auto`** is now the default (previously empty/system DNS)
- **`DOWNLOAD_PROGRESS_UPDATE_INTERVAL`** default changed from 5s to 1s
for smoother progress
- **`BYPASS_WARMUP_ON_CONNECT`** (default: true) — warm up Chrome when
first client connects

### 🐛 Bug Fixes

- **Download cancellation actually works**: Fixed issue where cancelling
downloads didn't properly stop in-progress operations
- **WELIB prioritization**: Fixed `PRIORITIZE_WELIB` not being respected
- **File exists handling**: Downloads to same filename now get `_1`,
`_2` suffix instead of overwriting
- **Empty search results**: "No books found" now returns empty list
instead of throwing exception
- **Search unavailable error**: Network/mirror failures during search
now return proper 503 error to client
2025-12-14 21:18:05 -05:00
CaliBrain b293bee5f4 Always run Gunicorn (even when DEBUG=true) to ensure Socket.IO WebSocket (#341) 2025-12-13 00:21:44 -05:00
Alex 122a3633c2 APP_ENV removal and secure cookie handling (#333)
Hey, made the tweaks we discussed, plus a couple related fixes :)

- Removed APP_ENV entirely. All dev-specific functionality is enabled
via `DEBUG: true` env var
- Set secure cookie handling to false by default, added to the readme to
enable if exclusively using HTTPS connection
- Fixed healthcheck potentially not working with auth enabled
- Removed APP_ENV from docker compose files and made sure app.db lines
are included in all versions.

APP_ENV in people's existing composes should get ignored entirely and
will be put on the default env, so no issues when updating.
2025-12-11 17:13:41 -05:00
CaliBrain 0e2580030b Change APP_ENV from 'prod' to 'dev' default (#331)
Fix for #330
2025-12-08 14:48:33 -05:00
AlexandCaliBrain 17057ecfbe WebUI - Mobile view tweaks (#329)
One set of changes I forgot to commit yesterday - a few minor
adjustments to improve experience on mobile, especially PWAs. Adjusted
search bar positioning and added explicit safe views for individual
components such as the header and footer.

Before / After  - test PWA on my iPhone: 
<img width="300" alt="IMG_1004"
src="https://github.com/user-attachments/assets/690b567e-a1a1-44c7-8e57-52ee8d896476"
/> - <img width="300" alt="IMG_1003"
src="https://github.com/user-attachments/assets/3f2b51cb-f408-47c8-9726-4b5d7d5a840c"
/>

@calibrain I think you can also close #31 #178 and #270 , should all be
covered off by the various WebUI PRs in the last week or so :)

---------

Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2025-11-23 17:12:48 -05:00
2b831dcfa5 [FEATURE] Separate download folders #122 (#297)
Re: Issue #122
Fetches content type from search results - displays it on thumbnails in
results grid;
Fetches content type from book id detail page (dfaults to "Other") and
uses it to construct the `final_path`.

---------

Co-authored-by: Patricia Ritter <pritter@events.com>
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2025-11-23 17:11:43 -05:00
CaliBrain 78c61e88b3 Fix kwargs bug in tracing log (#327) 2025-11-23 01:06:00 -05:00
CaliBrain 57d85d0748 fix format (#326)
- Fix function signature error
2025-11-23 00:10:00 -05:00
CaliBrain 6492bd6a3c Fix rare case where special character might break parsing (#325)
Actual fix for #322
2025-11-23 00:05:15 -05:00
Alex ed88aac5d5 WebUI - UI fixes, additional features and refactoring (#324)
One more on the frontend with some code cleanup, additional features and
bug fixes after testing this last week or so :)

* Various refactoring - removing reused code where possible and creating
new shared components (AdvancedFilters, DownloadButton, SearchBar,
buildSearchQuery, BookCard etc). Will hopefully help with further
features and improvements within the frontend. (cc @ZYancey)
* Added book descriptions to the details pane, grabbed alongside all
other info.
* Moved format selection into a dropdown list
* Added a toast notification if no results are found when searching
* Added “Clear search” button
* Added “Report bug” link in the header menu, linking to the issues page
* Fixed various UI bugs (Mouse hover colors, download badge)
2025-11-23 00:01:43 -05:00
Alex 5751910426 Fix for #322 - File extension fallback (#323)
Updated the fallback for file extension and size to work when
size/format details are missing but `_details` does still include some
information. If this was the case previously, the code wouldn't run and
files could be saved with no extension.

Fix for #322
2025-11-22 10:09:39 -05:00
CaliBrain b02ad7452c Remove deprecated /request route prefix support (#318)
This commit removes all references to the deprecated /request route
prefix
that was previously used for dual routing. The following changes were
made:

- Removed register_dual_routes() function that registered routes with
/request prefix
- Removed url_for_with_request() helper function for generating /request
URLs
- Removed call to register_dual_routes(app) at application startup
- Removed /request/ prefixed favicon routes
- Updated StatusEndpointFilter to remove /request/api/status log
filtering
- Removed unused flask_url_for import

All routes now only use the standard paths without the /request prefix.
2025-11-16 15:41:29 -05:00
CaliBrain 289666aeef Enhance tor.sh for hostname extraction and IP resolution (#317)
Updated the script to extract hostname and IP from EXT_BYPASSER_URL and
modify /etc/hosts accordingly. Replaced pyrequests with curl for network
requests.
2025-11-16 14:02:01 -05:00
Alex cc30d24144 HTTPS cookie handling (#315)
This is the one conflict from the other merge :)
2025-11-16 13:33:44 -05:00
CaliBrain 50e53a13b0 Fix Dockerfile for arm64 qemu crashes (#316)
Added build arguments for platform-specific builds and debug output.
2025-11-16 12:50:12 -05:00
CaliBrain a46d302ba8 Add iptables rules to bypass TOR for local networks (#314)
Added iptables rules to bypass TOR for local and private networks.

Tentative fix for #306
2025-11-16 00:15:36 -05:00
Alex c5d22e0f91 WebUI - Additional Search Features (#310)
### Main Points / To Do List

- [X] New Compact mode, with automatic and manual activation
- [X] New List mode, additional manual view 
- [X] Move sorting options to the main search results pane - dropdown
menu alongside view toggles
- [X] New language handling, including default language and multi-select
options.
- [x] New details view 
- [X] Various refactoring, including reuseable components for the three
search view components (Card, Compact & List), the download button, and
a reuseable dropdown list component.

---

### Card sizes: 

**Compact**
<img width="1246" height="612" alt="Screenshot 2025-11-15 at 15 35 55"
src="https://github.com/user-attachments/assets/445bceee-b876-4de7-880c-21c65f5f03eb"
/>

Mobile: Compact by default:
<img width="319" height="695" alt="Screenshot 2025-11-15 at 15 37 35"
src="https://github.com/user-attachments/assets/218361b3-326c-4e04-9b8a-03c503b28ae2"
/>


**List**
<img width="1263" height="623" alt="Screenshot 2025-11-15 at 15 35 24"
src="https://github.com/user-attachments/assets/7fcd2fb6-9b33-4f27-8b4b-c20247d92b16"
/>

Mobile: Optional
<img width="319" height="695" alt="Screenshot 2025-11-15 at 15 37 59"
src="https://github.com/user-attachments/assets/0e69069a-7e45-4499-b818-08e6d8dc2636"
/>

---
### Redesigned details pane: 
<img width="1487" height="729" alt="Screenshot 2025-11-15 at 15 39 50"
src="https://github.com/user-attachments/assets/bdfc61ae-4550-4c31-9bbb-80acb815bc72"
/>

Mobile: 
<img width="314" height="691" alt="Screenshot 2025-11-15 at 15 40 41"
src="https://github.com/user-attachments/assets/1dac3baa-ec9b-4da4-8e4a-d7d381d9bee2"
/>

--- 
### Multi-select languages
<img width="245" height="345" alt="Screenshot 2025-11-15 at 15 41 29"
src="https://github.com/user-attachments/assets/49ce7b96-06a7-4473-857a-ccfb52c2676f"
/>
2025-11-16 00:10:06 -05:00
CaliBrain 03321a5435 Improve book metadata handling in book_manager.py (#313)
Refactor book metadata extraction and add helper function.
Fix #300 and #307
2025-11-15 03:42:40 -05:00
CaliBrain 6aed906dfe Skip ad rows in search result parsing (#312)
AA started injection an ad banner into their code, for now we start
skipping this.
I am expecting later we will need to revisit this code for when they
actually start injecting the add
2025-11-15 01:21:44 -05:00
742da1c43a WebUI - Frontend Refactor (#302)
This PR was coauthored by alexhb1 and davidemarcoli. It builds on the FE
rework created by alex, but adds a myriad of additional tweaks and
optimizations to make the frontend feel modern, fast, and responsive.
The summary of the changes is as follows:

### Architecture Changes
React/TypeScript Migration: Refactored frontend from template/JS
structure to React/TypeScript application for better maintainability and
scalability
WebSocket Integration: Implemented real-time updates for download status
and progress with automatic fallback to polling
Gevent Worker: Configured production WebSocket support

### UI/UX Improvements
<img width="1502" height="890" alt="Screenshot 2025-11-10 at 10 02
59 AM"
src="https://github.com/user-attachments/assets/86bf8649-623f-413c-b8e5-656e687e55a8"
/>

Downloads Sidebar: Replaced bottom downloads section with sidebar
interface for better organization
<img width="201" height="450" alt="Screenshot 2025-11-10 at 10 07 52 AM"
src="https://github.com/user-attachments/assets/92b98e7c-c3bc-4b7e-80f1-252c3a760e33"
/>

Status Badges: Color-coded download status indicators instead of plain
text
Pinned Header: Fixed header position for consistent navigation
Enhanced Book Cards: Improved layout and hover states with info modal
button
<img width="1474" height="899" alt="Screenshot 2025-11-10 at 10 08
18 AM"
src="https://github.com/user-attachments/assets/9216d8a3-f662-434d-80e6-2a69b96abc31"
/>

Download Progress: Circular progress indicator on download buttons
Toast Notifications: Added user feedback for actions
Spinner Feedback: Loading indicators on search and download buttons
Animations: Smooth transitions and fluid progress updates

### Mobile & Responsive Design
Mobile-friendly Layouts: Optimized book cards and search interface for
mobile
<img width="225" height="450" alt="Screenshot 2025-11-10 at 10 05 49 AM"
src="https://github.com/user-attachments/assets/c8236c1c-5837-4309-9577-46db7292a54b"
/>

Keyboard Handling: Improved mobile keyboard behavior with proper input
types
PWA Improvements: Enhanced progressive web app functionality
Responsive Search: Better search box width and positioning across
devices

### Developer Experience
Development Mode: Separate frontend dev server that works with existing
backend container
Makefile: Added build automation and development commands
Documentation: Updated README with frontend architecture details

### Bug Fixes
Fixed "Clear completed" functionality
Fixed dark mode toggle text
Fixed sticky header behavior
Fixed mobile search box positioning
Removed active downloads requirement for initial state view

### Additional Features
ESC Key: Close downloads sidebar with ESC key
Calibre-Web Button: Direct link to Calibre-Web instance
<img width="282" height="83" alt="Screenshot 2025-11-11 at 9 38 05 AM"
src="https://github.com/user-attachments/assets/273075be-9743-4e13-9e48-5bf498f6c067"
/>
Granular Status Tracking: More detailed download progress information
obtained via websockets

---------

Co-authored-by: Alex <alex.bilbie1@gmail.com>
Co-authored-by: Zack Yancey <yanceyz@proton.me>
Co-authored-by: davidemarcoli <davide@marcoli.ch>
2025-11-14 15:48:44 -05:00
CaliBrain 8ea2fee0bb Fixing the title and book details from AA (#289)
Should fix #288
2025-10-04 14:44:10 -04:00
John Cocula 1c24312eb0 Update book_manager.py to fix #286 (#287)
Implement the fix mentioned in
https://github.com/calibrain/calibre-web-automated-book-downloader/issues/286

Note however that I have 0% success with downloads with 0.2.2 even with
this change.
2025-10-02 19:26:29 -04:00
CaliBrain 98e3a2f114 Add all supported format as default (#283) 2025-09-16 11:22:19 -04:00
CaliBrain cd16f09f2e Fix local download (#282) 2025-09-16 11:19:09 -04:00
CaliBrain 527c5d495d Fix formats in the HTML (read from config) (#279)
Fix #277
2025-09-09 08:15:44 -04:00
CaliBrain f5de2ab143 Fix AA extension parsing (#275)
Fix #274
2025-09-07 13:57:43 -04:00
RHDevandRyan Hults 4e5c9b788f Display book covers at full height (#266)
# Why
Book covers in the UI are currently cut off on the top and bottom,
making it hard to see.

# How
doubled the height of the book cover image div so the covers are not cut
off. I found that setting it to a specific size (rather than `h-full`)
resulted in better handling of small images and made for a more
consistent look.

# Before
<img width="488" height="520" alt="Screenshot from 2025-09-02 11-44-49"
src="https://github.com/user-attachments/assets/cf94e5f7-3981-40b6-a148-2a847f565c41"
/>
<img width="488" height="520" alt="Screenshot from 2025-09-02 11-45-06"
src="https://github.com/user-attachments/assets/5849eb34-57e8-4c36-af26-cb2f9647605f"
/>



# After
<img width="488" height="520" alt="Screenshot from 2025-09-02 11-41-41"
src="https://github.com/user-attachments/assets/6ffdde1f-ba36-4253-8092-12afe1c8f84e"
/>
<img width="488" height="520" alt="Screenshot from 2025-09-02 11-44-38"
src="https://github.com/user-attachments/assets/e44b57b7-ee03-4e8b-a186-444e8a5bf5aa"
/>

---------

Co-authored-by: Ryan Hults <contact@ryanthults.com>
2025-09-02 13:13:10 -04:00
CaliBrain 199d8453eb Adding Release version (#263) 2025-08-30 03:10:15 -04:00
BMillerCodesandBMillerCodes a9854b1a5c UI Overhaul - Tailwind CSS (#259)
New Homepage
<img width="2559" height="1388" alt="image"
src="https://github.com/user-attachments/assets/787668c6-61a9-4a2d-9878-9daaee8ae114"
/>
New Card Layout:
<img width="2547" height="1250" alt="image"
src="https://github.com/user-attachments/assets/44d015f9-c29d-4cac-a8b4-1c4ff7d40009"
/>
Light-Mode:
<img width="2547" height="1253" alt="image"
src="https://github.com/user-attachments/assets/7f81f6d7-ec69-4680-8ddc-4737601d00bb"
/>
New Download Queue and Status:
<img width="1260" height="146" alt="image"
src="https://github.com/user-attachments/assets/4acbb5a3-e985-4b23-8527-392e21151fa4"
/>

Haven't contributed to open-source before, but figured I could try and
help out on the UI side of the house. Appreciate everything you've done
this far!

Wanted to get something that was a tad bit more mobile-friendly.

Open to any feedback/comments/questions/concerns. :)

---------

Co-authored-by: BMillerCodes <BMillerCodes@users.noreply.github.com>
2025-08-29 22:48:25 -04:00
CaliBrain e4d3a372c8 Add retry logic for failed file copy (#261) 2025-08-29 22:32:04 -04:00
CaliBrain ff44881415 Pyautogui bug fix (#260) 2025-08-29 21:14:06 -04:00
CaliBrain 9ffedc1fc0 Several Bug fixes (#256)
Emoji check fix Fix multi language books
Fix DNS in Chromium Headless
Fix DNS IPv6 address by un-abreviating them
Fix typo in Quad9 DNS
2025-08-29 12:47:05 -04:00
CaliBrain 00370818f0 Fix eager cloudflare check (#247) 2025-08-28 17:42:50 -04:00
CaliBrain 7d9a82bfea Add default flaresolverr values (#253) 2025-08-28 17:41:15 -04:00
Federico Della Rovere 207cff96d3 External CloudFlare resolver (#245)
Adding support for an external CloudFlare bypasser service and
introducing a new Docker image build with a dedicated target.

Key Changes
- Added `cloudflare_bypasser_external.py` for external bypasser
integration.
- Updated Docker Compose files to support the new service.
- Introduced a new Docker target for building a separate image for the
external bypasser.
- Refactored relevant modules to utilize the external bypasser when
configured.
- Documentation and configuration updates to reflect new options and
Docker targets.

Impact
- Users can now choose between internal and external CloudFlare
bypassing.
- New Docker image and target streamline deployment of the external
bypasser.
- Improved modularity and maintainability.
- No breaking changes for existing workflows.

Testing
- Manual and E2E tests performed for both bypasser modes.
- Docker Compose setups and new image build verified for development and
production.

Notes
Please review the new configuration options and Docker targets. Update
your environment and deployment scripts as needed. Feedback and
suggestions are welcome!
2025-08-28 17:37:59 -04:00
CaliBrain c8f21b8f8d Fix progression in download (#248) 2025-08-25 23:59:05 -04:00
Stephon ParkerandCaliBrain 5e04b6bfb8 Add Download Queue Management & Concurrent Downloads (#231)
## Summary

This PR completely overhauls the download system to support concurrent
downloads, cancellation, and advanced queue
management. No more waiting for stuck downloads to block the entire
queue!

  ## New Features

  ###  **Concurrent Downloads**
  - **3 simultaneous downloads** by default (configurable)
- ThreadPoolExecutor-based architecture replaces single-threaded
bottleneck
  - Downloads no longer block each other

  ###  **Download Cancellation**
  - **Cancel button** for active/queued downloads
  - Clean cancellation with partial file cleanup
  - Thread-safe cancellation flags

  ### **Queue Priority & Reordering**
  - **Priority-based queue** (lower number = higher priority)
  - **Editable priority inputs** in UI for queued items
  - **Bulk reordering** API support

  ## UI Enhancements

- **Enhanced status table** with Priority, Progress, and Actions columns
  - **Progress bars** for active downloads
  - **Cancel buttons** for downloads
- **Queue management controls**: Refresh, Clear Completed, Active
Counter
  - **Real-time updates** every 60 seconds

## Implementation

  ### New Architecture
  - `queue.PriorityQueue` replaces simple `set()` for proper ordering
  - `concurrent.futures.ThreadPoolExecutor` handles worker threads
  - `threading.Event` flags enable clean cancellation
  - Thread-safe operations with proper locking

  ### New API Endpoints
  DELETE /api/download/{id}/cancel     # Cancel downloads
PUT /api/queue/{id}/priority # Set priorityPOST /api/queue/reorder #
Bulk reorder
  GET /api/downloads/active          # Active tracking
  DELETE /api/queue/clear            # Clear completed

  ### New Environment Variables
  ```bash
  MAX_CONCURRENT_DOWNLOADS=3          # Worker thread count
  DOWNLOAD_PROGRESS_UPDATE_INTERVAL=5  # Progress frequency
  ```

### Fully backward compatible - existing functionality unchanged

### Fixes

  - Resolves: Downloads blocking when one gets stuck
  - Resolves: No way to cancel problematic downloads
  - Resolves: No queue management or prioritization
  
  #183

---------

Co-authored-by: CaliBrain <calibrain@l4n.xyz>
2025-08-25 13:28:35 -04:00
CaliBrain 09bd5ae9f0 Bug fixes for Login and WELIB (#242)
Fix login (when using latest CWA in dev)
Add PRIORITIZE_WELIB option
libgen seems to be back, revert the url rewrite

Should fix #242 and #226
2025-08-22 18:21:55 -04:00
CaliBrain 5f6a81d97d Update requirements.txt (#239)
#229
2025-08-20 20:55:30 -04:00
CaliBrain cf3d4d9352 Use matrix build for main and tor github action (#237) 2025-08-20 09:03:42 -04:00
CaliBrain b1673904c5 fix build latest (#236)
- Fix branch tagging and use :dev for main and :latest for tagged
release
- Fix latest tag check for release
2025-08-20 08:50:51 -04:00
83 changed files with 13173 additions and 2493 deletions
+10
View File
@@ -37,3 +37,13 @@ dist/
venv/
.venv/
env/
# Frontend build artifacts (built in separate stage)
src/frontend/node_modules/
src/frontend/dist/
src/frontend/.vite/
# Old frontend code (replaced by src/frontend)
templates/
static/css/
static/js/
@@ -1,5 +1,4 @@
name: Create and publish Docker images
on:
push:
branches:
@@ -7,11 +6,9 @@ on:
tags:
- 'v*'
workflow_dispatch:
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-push-images:
runs-on: ubuntu-latest
@@ -20,12 +17,26 @@ jobs:
packages: write
attestations: write
id-token: write
strategy:
matrix:
include:
- suffix: ""
target: cwa-bd
image_name_suffix: ""
- suffix: "-tor"
target: cwa-bd-tor
image_name_suffix: "-tor"
- suffix: "-extbp"
target: cwa-bd-extbp
image_name_suffix: "-extbp"
steps:
- name: Get current date
id: date
run: echo "date=$(date +'%Y-%m-%d')" >> $GITHUB_OUTPUT
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
@@ -33,15 +44,14 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Build and push main image
- name: Extract metadata for main image
id: meta-main
- name: Extract metadata for ${{ matrix.target }} image
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}${{ matrix.image_name_suffix }}
tags: |
type=raw,value=dev,enable={{is_default_branch}}
type=raw,value=latest,enable={{is_tag}}
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/') }}
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha
@@ -50,59 +60,24 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build and push main Docker image
id: push-main
- name: Build and push ${{ matrix.target }} Docker image
id: push
uses: docker/build-push-action@v5
with:
platforms: linux/amd64,linux/arm64
context: .
target: cwa-bd
target: ${{ matrix.target }}
push: ${{ github.event_name != 'pull_request' }}
build-args: |
BUILD_VERSION=${{ steps.date.outputs.date }}-${{ github.sha }}
tags: ${{ steps.meta-main.outputs.tags }}
labels: ${{ steps.meta-main.outputs.labels }}
- name: Generate artifact attestation for main image
RELEASE_VERSION=${{ github.ref_name }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Generate artifact attestation for ${{ matrix.target }} image
if: github.event_name != 'pull_request'
uses: actions/attest-build-provenance@v2
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
subject-digest: ${{ steps.push-main.outputs.digest }}
push-to-registry: true
# Build and push tor image
- name: Extract metadata for tor image
id: meta-tor
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-tor
tags: |
type=raw,value=dev,enable={{is_default_branch}}
type=raw,value=latest,enable={{is_tag}}
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha
type=ref,event=tag
- name: Build and push tor Docker image
id: push-tor
uses: docker/build-push-action@v5
with:
platforms: linux/amd64,linux/arm64
context: .
target: cwa-bd-tor
push: ${{ github.event_name != 'pull_request' }}
build-args: |
BUILD_VERSION=${{ steps.date.outputs.date }}-${{ github.sha }}
tags: ${{ steps.meta-tor.outputs.tags }}
labels: ${{ steps.meta-tor.outputs.labels }}
- name: Generate artifact attestation for tor image
if: github.event_name != 'pull_request'
uses: actions/attest-build-provenance@v2
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-tor
subject-digest: ${{ steps.push-tor.outputs.digest }}
push-to-registry: true
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}${{ matrix.image_name_suffix }}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true
+2 -1
View File
@@ -15,7 +15,8 @@
"LOG_ROOT": "/tmp/cwa-book-downloader",
"ENABLE_LOGGING": "true",
"DOCKERMODE": "false",
"DEBUG": "true"
"DEBUG": "true",
"CUSTOM_DNS": "google",
},
},
{
+68 -22
View File
@@ -1,9 +1,36 @@
ARG TARGETPLATFORM
ARG TARGETARCH
ARG BUILDPLATFORM
ARG BUILDARCH
# Frontend build stage.
FROM --platform=$BUILDPLATFORM node:20-alpine AS frontend-builder
# Helpful debug output to see what platforms BuildKit thinks it's using
RUN echo "BUILDPLATFORM=$BUILDPLATFORM BUILDARCH=$BUILDARCH TARGETPLATFORM=$TARGETPLATFORM TARGETARCH=$TARGETARCH"
WORKDIR /frontend
# Copy frontend package files
COPY src/frontend/package*.json ./
# Install dependencies
RUN npm ci
# Copy frontend source
COPY src/frontend/ ./
# Build the frontend
RUN npm run build
# Use python-slim as the base image
FROM python:3.10-slim AS base
# Add build argument for version
ARG BUILD_VERSION
ENV BUILD_VERSION=${BUILD_VERSION}
ARG RELEASE_VERSION
ENV RELEASE_VERSION=${RELEASE_VERSION}
# Set shell to bash with pipefail option
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
@@ -22,8 +49,7 @@ ENV DEBIAN_FRONTEND=noninteractive \
# UID/GID will be handled by entrypoint script, but TZ/Locale are still needed
LANG=en_US.UTF-8 \
LANGUAGE=en_US:en \
LC_ALL=en_US.UTF-8 \
APP_ENV=prod
LC_ALL=en_US.UTF-8
# Set ARG for build-time expansion (FLASK_PORT), ENV for runtime access
ENV FLASK_PORT=8084
@@ -38,18 +64,10 @@ RUN apt-get update && \
curl \
# For entrypoint
dumb-init \
# For dumb display
xvfb \
# For screen recording
ffmpeg \
# For debug
zip iputils-ping \
# For user switching
sudo \
# --- Chromium Browser ---
chromium-driver \
# For tkinter (pyautogui)
python3-tk && \
sudo && \
# Cleanup APT cache *after* all installs in this layer
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false && \
apt-get clean && \
@@ -67,20 +85,18 @@ WORKDIR /app
# Install Python dependencies using pip
# Upgrade pip first, then copy requirements and install
# Copying requirements.txt separately leverages build cache
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt && \
# Copying requirements-base.txt separately leverages build cache
COPY requirements-base.txt .
RUN pip install --no-cache-dir -r requirements-base.txt && \
# Clean root's pip cache
rm -rf /root/.cache
# Add this line to grant read/execute permissions to others
RUN chmod -R o+rx /usr/bin/chromium && \
chmod -R o+rx /usr/bin/chromedriver && \
chmod -R o+w /usr/local/lib/python3.10/site-packages/seleniumbase/drivers/
# Copy application code *after* dependencies are installed
COPY . .
# Copy built frontend from frontend-builder stage
COPY --from=frontend-builder /frontend/dist /app/frontend-dist
# Final setup: permissions and directories in one layer
# Only creating directories and setting executable bits.
# Ownership will be handled by the entrypoint script.
@@ -91,9 +107,9 @@ RUN mkdir -p /var/log/cwa-book-downloader /cwa-book-ingest && \
EXPOSE ${FLASK_PORT}
# Add healthcheck for container status
# This will run as root initially, but check localhost which should work if the app binds correctly.
# Uses /api/health which doesn't require authentication
HEALTHCHECK --interval=60s --timeout=60s --start-period=60s --retries=3 \
CMD curl -s http://localhost:${FLASK_PORT}/request/api/status > /dev/null || exit 1
CMD curl -s http://localhost:${FLASK_PORT}/api/health > /dev/null || exit 1
# Use dumb-init as the entrypoint to handle signals properly
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
@@ -101,10 +117,34 @@ ENTRYPOINT ["/usr/bin/dumb-init", "--"]
FROM base AS cwa-bd
RUN apt-get update && \
apt-get install -y --no-install-recommends \
# For dumb display
xvfb \
# For screen recording
ffmpeg \
# --- Chromium ---
chromium \
# --- ChromeDriver ---
chromium-driver \
# For tkinter (pyautogui)
python3-tk
# install additional dependencies
COPY requirements-cwa-bd.txt .
RUN pip install --no-cache-dir -r requirements-cwa-bd.txt && \
# Clean root's pip cache
rm -rf /root/.cache
# Add this line to grant read/execute permissions to others
RUN chmod -R o+rx /usr/bin/chromium && \
chmod -R o+rx /usr/bin/chromedriver && \
chmod -R o+w /usr/local/lib/python3.10/site-packages/seleniumbase/drivers/
# Default command to run the application entrypoint script
CMD ["/app/entrypoint.sh"]
FROM base AS cwa-bd-tor
FROM cwa-bd AS cwa-bd-tor
ENV USING_TOR=true
@@ -124,3 +164,9 @@ RUN apt-get update && \
# Override the default command to run Tor
CMD ["/app/entrypoint.sh"]
FROM base AS cwa-bd-extbp
ENV USING_EXTERNAL_BYPASSER=true
CMD ["/app/entrypoint.sh"]
+78
View File
@@ -0,0 +1,78 @@
.PHONY: help install dev build preview typecheck clean up down docker-build refresh
# Frontend directory
FRONTEND_DIR := src/frontend
# Docker compose file
COMPOSE_FILE := docker-compose.dev.yml
# Default target
help:
@echo "Available targets:"
@echo ""
@echo "Frontend:"
@echo " install - Install frontend dependencies"
@echo " dev - Start development server"
@echo " build - Build frontend for production"
@echo " preview - Preview production build"
@echo " typecheck - Run TypeScript type checking"
@echo " clean - Remove node_modules and build artifacts"
@echo ""
@echo "Backend (Docker):"
@echo " up - Start backend services"
@echo " down - Stop backend services"
@echo " docker-build - Build Docker image"
@echo " refresh - Rebuild and restart backend services"
# Install dependencies
install:
@echo "Installing frontend dependencies..."
cd $(FRONTEND_DIR) && npm install
# Start development server
dev:
@echo "Starting development server..."
cd $(FRONTEND_DIR) && npm run dev
# Build for production
build:
@echo "Building frontend for production..."
cd $(FRONTEND_DIR) && npm run build
# Preview production build
preview:
@echo "Previewing production build..."
cd $(FRONTEND_DIR) && npm run preview
# Type checking
typecheck:
@echo "Running TypeScript type checking..."
cd $(FRONTEND_DIR) && npm run typecheck
# Clean build artifacts and dependencies
clean:
@echo "Cleaning build artifacts and dependencies..."
rm -rf $(FRONTEND_DIR)/node_modules
rm -rf $(FRONTEND_DIR)/dist
# Start backend services
up:
@echo "Starting backend services..."
docker compose -f $(COMPOSE_FILE) up -d
# Stop backend services
down:
@echo "Stopping backend services..."
docker compose -f $(COMPOSE_FILE) down
# Build Docker image
docker-build:
@echo "Building Docker image..."
docker compose -f $(COMPOSE_FILE) build
# Rebuild and restart backend services
refresh:
@echo "Rebuilding and restarting backend services..."
docker compose -f $(COMPOSE_FILE) down
docker compose -f $(COMPOSE_FILE) build
docker compose -f $(COMPOSE_FILE) up -d
Binary file not shown.

Before

Width:  |  Height:  |  Size: 874 KiB

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 233 KiB

After

Width:  |  Height:  |  Size: 2.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 110 KiB

After

Width:  |  Height:  |  Size: 160 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 419 KiB

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 244 KiB

After

Width:  |  Height:  |  Size: 371 KiB

+599 -116
View File
@@ -1,22 +1,31 @@
"""Flask web application for book download service with URL rewrite support."""
import io
import logging
import io, re, os
import os
import sqlite3
import time
from datetime import datetime, timedelta
from functools import wraps
from flask import Flask, request, jsonify, render_template, send_file, send_from_directory
from typing import Any, Dict, Tuple, Union
from flask import Flask, jsonify, request, send_file, send_from_directory, session
from flask_cors import CORS
from flask_socketio import SocketIO, emit
from werkzeug.middleware.proxy_fix import ProxyFix
from werkzeug.security import check_password_hash
from werkzeug.wrappers import Response
from flask import url_for as flask_url_for
import typing
from logger import setup_logger
from config import _SUPPORTED_BOOK_LANGUAGE, BOOK_LANGUAGE
from env import FLASK_HOST, FLASK_PORT, APP_ENV, CWA_DB_PATH, DEBUG
import backend
from book_manager import SearchUnavailable
from config import BOOK_LANGUAGE, SUPPORTED_FORMATS, _SUPPORTED_BOOK_LANGUAGE
from env import (
BUILD_VERSION, CALIBRE_WEB_URL, CWA_DB_PATH, DEBUG, FLASK_HOST, FLASK_PORT,
RELEASE_VERSION, USING_EXTERNAL_BYPASSER,
)
from logger import setup_logger
from models import SearchFilters
from websocket_manager import ws_manager
logger = setup_logger(__name__)
app = Flask(__name__)
@@ -24,6 +33,143 @@ app.wsgi_app = ProxyFix(app.wsgi_app) # type: ignore
app.config['SEND_FILE_MAX_AGE_DEFAULT'] = 0 # Disable caching
app.config['APPLICATION_ROOT'] = '/'
# Socket.IO async mode.
# We run this app under Gunicorn with a gevent websocket worker (even when DEBUG=true),
# so Socket.IO should always use gevent here.
async_mode = 'gevent'
# Initialize Flask-SocketIO with reverse proxy support
socketio = SocketIO(
app,
cors_allowed_origins="*",
async_mode=async_mode,
logger=False,
engineio_logger=False,
# Reverse proxy / Traefik compatibility settings
path='/socket.io',
ping_timeout=60, # Time to wait for pong response
ping_interval=25, # Send ping every 25 seconds
# Allow both websocket and polling for better compatibility
transports=['websocket', 'polling'],
# Enable CORS for all origins (you can restrict this in production)
allow_upgrades=True,
# Important for proxies that buffer
http_compression=True
)
# Initialize WebSocket manager
ws_manager.init_app(app, socketio)
logger.info(f"Flask-SocketIO initialized with async_mode='{async_mode}'")
# Rate limiting for login attempts
# Structure: {username: {'count': int, 'lockout_until': datetime}}
failed_login_attempts: Dict[str, Dict[str, Any]] = {}
MAX_LOGIN_ATTEMPTS = 10
LOCKOUT_DURATION_MINUTES = 30
def cleanup_old_lockouts() -> None:
"""Remove expired lockout entries to prevent memory buildup."""
current_time = datetime.now()
expired_users = [
username for username, data in failed_login_attempts.items()
if 'lockout_until' in data and data['lockout_until'] < current_time
]
for username in expired_users:
logger.info(f"Lockout expired for user: {username}")
del failed_login_attempts[username]
def is_account_locked(username: str) -> bool:
"""Check if an account is currently locked due to failed login attempts."""
cleanup_old_lockouts()
if username not in failed_login_attempts:
return False
lockout_until = failed_login_attempts[username].get('lockout_until')
if lockout_until and datetime.now() < lockout_until:
return True
return False
def record_failed_login(username: str, ip_address: str) -> bool:
"""
Record a failed login attempt and lock account if threshold is reached.
Returns True if account is now locked, False otherwise.
"""
if username not in failed_login_attempts:
failed_login_attempts[username] = {'count': 0}
failed_login_attempts[username]['count'] += 1
count = failed_login_attempts[username]['count']
logger.warning(f"Failed login attempt {count}/{MAX_LOGIN_ATTEMPTS} for user '{username}' from IP {ip_address}")
if count >= MAX_LOGIN_ATTEMPTS:
lockout_until = datetime.now() + timedelta(minutes=LOCKOUT_DURATION_MINUTES)
failed_login_attempts[username]['lockout_until'] = lockout_until
logger.warning(f"Account locked for user '{username}' until {lockout_until.strftime('%Y-%m-%d %H:%M:%S')} due to {count} failed login attempts")
return True
return False
def clear_failed_logins(username: str) -> None:
"""Clear failed login attempts for a user after successful login."""
if username in failed_login_attempts:
del failed_login_attempts[username]
logger.debug(f"Cleared failed login attempts for user: {username}")
# Enable CORS in development mode for local frontend development
if DEBUG:
CORS(app, resources={
r"/*": {
"origins": ["http://localhost:5173", "http://127.0.0.1:5173"],
"supports_credentials": True,
"allow_headers": ["Content-Type", "Authorization"],
"methods": ["GET", "POST", "PUT", "DELETE", "OPTIONS"]
}
})
# Custom log filter to exclude routine status endpoint polling and WebSocket noise
class StatusEndpointFilter(logging.Filter):
"""Filter out routine status endpoint requests and WebSocket upgrade errors to reduce log noise."""
def filter(self, record):
if hasattr(record, 'getMessage'):
message = record.getMessage()
# Exclude GET /api/status requests (polling noise)
if 'GET /api/status' in message:
return False
# Exclude WebSocket upgrade errors (benign - falls back to polling)
if 'write() before start_response' in message:
return False
# Exclude the Error on request line that precedes WebSocket errors
if 'Error on request:' in message and record.levelno == logging.ERROR:
return False
return True
class WebSocketErrorFilter(logging.Filter):
"""Filter out WebSocket upgrade errors that occur in Werkzeug dev server.
These errors are benign - Flask-SocketIO automatically falls back to polling transport.
The error occurs because Werkzeug's built-in server doesn't fully support WebSocket upgrades.
"""
def filter(self, record):
# Filter out the AssertionError traceback for WebSocket upgrades
if record.levelno == logging.ERROR:
message = record.getMessage() if hasattr(record, 'getMessage') else str(record.msg)
# Filter out the full traceback that includes the WebSocket assertion error
if 'write() before start_response' in message:
return False
# Also filter the "Error on request" header that precedes it
if hasattr(record, 'exc_info') and record.exc_info:
exc_type = record.exc_info[0]
if exc_type and exc_type.__name__ == 'AssertionError':
# Check if it's the WebSocket-related assertion
exc_value = record.exc_info[1]
if exc_value and 'write() before start_response' in str(exc_value):
return False
return True
# Flask logger
app.logger.handlers = logger.handlers
app.logger.setLevel(logger.level)
@@ -31,14 +177,28 @@ app.logger.setLevel(logger.level)
werkzeug_logger = logging.getLogger('werkzeug')
werkzeug_logger.handlers = logger.handlers
werkzeug_logger.setLevel(logger.level)
# Add filters to suppress routine status endpoint polling logs and WebSocket upgrade errors
werkzeug_logger.addFilter(StatusEndpointFilter())
werkzeug_logger.addFilter(WebSocketErrorFilter())
# Set up authentication defaults
# The secret key will reset every time we restart, which will
# require users to authenticate again
# Session cookie security - set to 'true' if exclusively using HTTPS
session_cookie_secure_env = os.getenv('SESSION_COOKIE_SECURE', 'false').lower()
SESSION_COOKIE_SECURE = session_cookie_secure_env in ['true', 'yes', '1']
app.config.update(
SECRET_KEY = os.urandom(64)
SECRET_KEY = os.urandom(64),
SESSION_COOKIE_HTTPONLY = True,
SESSION_COOKIE_SAMESITE = 'Lax',
SESSION_COOKIE_SECURE = SESSION_COOKIE_SECURE,
PERMANENT_SESSION_LIFETIME = 604800 # 7 days in seconds
)
logger.info(f"Session cookie secure setting: {SESSION_COOKIE_SECURE} (from env: {session_cookie_secure_env})")
def login_required(f):
@wraps(f)
def decorated_function(*args, **kwargs):
@@ -46,88 +206,78 @@ def login_required(f):
# path, return a server error
if CWA_DB_PATH is not None and not os.path.isfile(CWA_DB_PATH):
logger.error(f"CWA_DB_PATH is set to {CWA_DB_PATH} but this is not a valid path")
return Response("Internal Server Error", 500)
if not authenticate():
return Response(
response="Unauthorized",
status=401,
headers={
"WWW-Authenticate": 'Basic realm="Calibre-Web-Automated-Book-Downloader"',
},
)
return jsonify({"error": "Internal Server Error"}), 500
# If no database is configured, allow access
if not CWA_DB_PATH:
return f(*args, **kwargs)
# Check if user has a valid session
if 'user_id' not in session:
return jsonify({"error": "Unauthorized"}), 401
return f(*args, **kwargs)
return decorated_function
def register_dual_routes(app : Flask) -> None:
"""
Register each route both with and without the /request prefix.
This function should be called after all routes are defined.
"""
# Store original url_map rules
rules = list(app.url_map.iter_rules())
# Add /request prefix to each rule
for rule in rules:
if rule.rule != '/request/' and rule.rule != '/request': # Skip if it's already a request route
# Create new routes with /request prefix, both with and without trailing slash
base_rule = rule.rule[:-1] if rule.rule.endswith('/') else rule.rule
if base_rule == '': # Special case for root path
app.add_url_rule('/request', f"root_request",
view_func=app.view_functions[rule.endpoint],
methods=rule.methods)
app.add_url_rule('/request/', f"root_request_slash",
view_func=app.view_functions[rule.endpoint],
methods=rule.methods)
else:
app.add_url_rule(f"/request{base_rule}",
f"{rule.endpoint}_request",
view_func=app.view_functions[rule.endpoint],
methods=rule.methods)
app.add_url_rule(f"/request{base_rule}/",
f"{rule.endpoint}_request_slash",
view_func=app.view_functions[rule.endpoint],
methods=rule.methods)
app.jinja_env.globals['url_for'] = url_for_with_request
def url_for_with_request(endpoint : str, **values : typing.Any) -> str:
"""Generate URLs with /request prefix by default."""
if endpoint == 'static':
# For static files, add /request prefix
url = flask_url_for(endpoint, **values)
return f"/request{url}"
return flask_url_for(endpoint, **values)
# Serve frontend static files
@app.route('/assets/<path:filename>')
def serve_frontend_assets(filename: str) -> Response:
"""
Serve static assets from the built frontend.
"""
return send_from_directory(os.path.join(app.root_path, 'frontend-dist', 'assets'), filename)
@app.route('/')
@login_required
def index() -> str:
def index() -> Response:
"""
Render main page with search and status table.
Serve the React frontend application.
Authentication is handled by the React app itself.
"""
return render_template('index.html', book_languages=_SUPPORTED_BOOK_LANGUAGE, default_language=BOOK_LANGUAGE, debug=DEBUG)
return send_from_directory(os.path.join(app.root_path, 'frontend-dist'), 'index.html')
@app.route('/logo.png')
def logo() -> Response:
"""
Serve logo from built frontend assets.
"""
return send_from_directory(os.path.join(app.root_path, 'frontend-dist'),
'logo.png', mimetype='image/png')
@app.route('/favicon.ico')
@app.route('/favico<path:_>')
@app.route('/request/favico<path:_>')
@app.route('/request/static/favico<path:_>')
def favicon(_ : typing.Any) -> Response:
return send_from_directory(os.path.join(app.root_path, 'static', 'media'),
def favicon(_: Any = None) -> Response:
"""
Serve favicon from built frontend assets.
"""
return send_from_directory(os.path.join(app.root_path, 'frontend-dist'),
'favicon.ico', mimetype='image/vnd.microsoft.icon')
from typing import Union, Tuple
# Register bypasser warmup callback for when first WebSocket client connects
# and shutdown callback for when all clients disconnect
if not USING_EXTERNAL_BYPASSER:
from cloudflare_bypasser import warmup as bypasser_warmup, shutdown_if_idle as bypasser_shutdown
ws_manager.register_on_first_connect(bypasser_warmup)
ws_manager.register_on_all_disconnect(bypasser_shutdown)
logger.info("Registered Cloudflare bypasser warmup/shutdown on WebSocket connect/disconnect")
if DEBUG:
import subprocess
import time
from cloudflare_bypasser import _reset_driver as STOP_GUI
@app.route('/debug', methods=['GET'])
if USING_EXTERNAL_BYPASSER:
STOP_GUI = lambda: None
else:
from cloudflare_bypasser import _reset_driver as STOP_GUI
@app.route('/api/debug', methods=['GET'])
@login_required
def debug() -> Union[Response, Tuple[Response, int]]:
"""
This will run the /app/debug.sh script, which will generate a debug zip with all the logs
This will run the /app/genDebug.sh script, which will generate a debug zip with all the logs
The file will be named /tmp/cwa-book-downloader-debug.zip
And then return it to the user
"""
try:
# Run the debug script
logger.info("Debug endpoint called, stopping GUI and generating debug info...")
STOP_GUI()
time.sleep(1)
result = subprocess.run(['/app/genDebug.sh'], capture_output=True, text=True, check=True)
@@ -136,9 +286,10 @@ if DEBUG:
logger.info(f"Debug script executed: {result.stdout}")
debug_file_path = result.stdout.strip().split('\n')[-1]
if not os.path.exists(debug_file_path):
logger.error("Debug zip file not found after running debug script")
logger.error(f"Debug zip file not found at: {debug_file_path}")
return jsonify({"error": "Failed to generate debug information"}), 500
logger.info(f"Sending debug file: {debug_file_path}")
# Return the file to the user
return send_file(
debug_file_path,
@@ -199,6 +350,9 @@ def api_search() -> Union[Response, Tuple[Response, int]]:
try:
books = backend.search_books(query, filters)
return jsonify(books)
except SearchUnavailable as e:
logger.warning(f"Search unavailable: {e}")
return jsonify({"error": str(e)}), 503
except Exception as e:
logger.error_trace(f"Search error: {e}")
return jsonify({"error": str(e)}), 500
@@ -245,14 +399,47 @@ def api_download() -> Union[Response, Tuple[Response, int]]:
return jsonify({"error": "No book ID provided"}), 400
try:
success = backend.queue_book(book_id)
priority = int(request.args.get('priority', 0))
success = backend.queue_book(book_id, priority)
if success:
return jsonify({"status": "queued"})
return jsonify({"status": "queued", "priority": priority})
return jsonify({"error": "Failed to queue book"}), 500
except Exception as e:
logger.error_trace(f"Download error: {e}")
return jsonify({"error": str(e)}), 500
@app.route('/api/config', methods=['GET'])
@login_required
def api_config() -> Union[Response, Tuple[Response, int]]:
"""
Get application configuration for frontend.
"""
try:
config = {
"calibre_web_url": CALIBRE_WEB_URL,
"debug": DEBUG,
"build_version": BUILD_VERSION,
"release_version": RELEASE_VERSION,
"book_languages": _SUPPORTED_BOOK_LANGUAGE,
"default_language": BOOK_LANGUAGE,
"supported_formats": SUPPORTED_FORMATS
}
return jsonify(config)
except Exception as e:
logger.error_trace(f"Config error: {e}")
return jsonify({"error": str(e)}), 500
@app.route('/api/health', methods=['GET'])
def api_health() -> Union[Response, Tuple[Response, int]]:
"""
Health check endpoint for container orchestration.
No authentication required.
Returns:
flask.Response: JSON with status "ok".
"""
return jsonify({"status": "ok"})
@app.route('/api/status', methods=['GET'])
@login_required
def api_status() -> Union[Response, Tuple[Response, int]]:
@@ -290,15 +477,12 @@ def api_local_download() -> Union[Response, Tuple[Response, int]]:
if file_data is None:
# Book data not found or not available
return jsonify({"error": "File not found"}), 404
# Santize the file name
file_name = book_info.title
file_name = re.sub(r'[\\/:*?"<>|]', '_', file_name.strip())[:245]
file_extension = book_info.format
file_name = book_info.get_filename()
# Prepare the file for sending to the client
data = io.BytesIO(file_data)
return send_file(
data,
download_name=f"{file_name}.{file_extension}",
download_name=file_name,
as_attachment=True
)
@@ -306,6 +490,147 @@ def api_local_download() -> Union[Response, Tuple[Response, int]]:
logger.error_trace(f"Local download error: {e}")
return jsonify({"error": str(e)}), 500
@app.route('/api/download/<book_id>/cancel', methods=['DELETE'])
@login_required
def api_cancel_download(book_id: str) -> Union[Response, Tuple[Response, int]]:
"""
Cancel a download.
Path Parameters:
book_id (str): Book identifier to cancel
Returns:
flask.Response: JSON status indicating success or failure.
"""
try:
success = backend.cancel_download(book_id)
if success:
return jsonify({"status": "cancelled", "book_id": book_id})
return jsonify({"error": "Failed to cancel download or book not found"}), 404
except Exception as e:
logger.error_trace(f"Cancel download error: {e}")
return jsonify({"error": str(e)}), 500
@app.route('/api/queue/<book_id>/priority', methods=['PUT'])
@login_required
def api_set_priority(book_id: str) -> Union[Response, Tuple[Response, int]]:
"""
Set priority for a queued book.
Path Parameters:
book_id (str): Book identifier
Request Body:
priority (int): New priority level (lower number = higher priority)
Returns:
flask.Response: JSON status indicating success or failure.
"""
try:
data = request.get_json()
if not data or 'priority' not in data:
return jsonify({"error": "Priority not provided"}), 400
priority = int(data['priority'])
success = backend.set_book_priority(book_id, priority)
if success:
return jsonify({"status": "updated", "book_id": book_id, "priority": priority})
return jsonify({"error": "Failed to update priority or book not found"}), 404
except ValueError:
return jsonify({"error": "Invalid priority value"}), 400
except Exception as e:
logger.error_trace(f"Set priority error: {e}")
return jsonify({"error": str(e)}), 500
@app.route('/api/queue/reorder', methods=['POST'])
@login_required
def api_reorder_queue() -> Union[Response, Tuple[Response, int]]:
"""
Bulk reorder queue by setting new priorities.
Request Body:
book_priorities (dict): Mapping of book_id to new priority
Returns:
flask.Response: JSON status indicating success or failure.
"""
try:
data = request.get_json()
if not data or 'book_priorities' not in data:
return jsonify({"error": "book_priorities not provided"}), 400
book_priorities = data['book_priorities']
if not isinstance(book_priorities, dict):
return jsonify({"error": "book_priorities must be a dictionary"}), 400
# Validate all priorities are integers
for book_id, priority in book_priorities.items():
if not isinstance(priority, int):
return jsonify({"error": f"Invalid priority for book {book_id}"}), 400
success = backend.reorder_queue(book_priorities)
if success:
return jsonify({"status": "reordered", "updated_count": len(book_priorities)})
return jsonify({"error": "Failed to reorder queue"}), 500
except Exception as e:
logger.error_trace(f"Reorder queue error: {e}")
return jsonify({"error": str(e)}), 500
@app.route('/api/queue/order', methods=['GET'])
@login_required
def api_queue_order() -> Union[Response, Tuple[Response, int]]:
"""
Get current queue order for display.
Returns:
flask.Response: JSON array of queued books with their order and priorities.
"""
try:
queue_order = backend.get_queue_order()
return jsonify({"queue": queue_order})
except Exception as e:
logger.error_trace(f"Queue order error: {e}")
return jsonify({"error": str(e)}), 500
@app.route('/api/downloads/active', methods=['GET'])
@login_required
def api_active_downloads() -> Union[Response, Tuple[Response, int]]:
"""
Get list of currently active downloads.
Returns:
flask.Response: JSON array of active download book IDs.
"""
try:
active_downloads = backend.get_active_downloads()
return jsonify({"active_downloads": active_downloads})
except Exception as e:
logger.error_trace(f"Active downloads error: {e}")
return jsonify({"error": str(e)}), 500
@app.route('/api/queue/clear', methods=['DELETE'])
@login_required
def api_clear_completed() -> Union[Response, Tuple[Response, int]]:
"""
Clear all completed, errored, or cancelled books from tracking.
Returns:
flask.Response: JSON with count of removed books.
"""
try:
removed_count = backend.clear_completed()
# Broadcast status update after clearing
if ws_manager:
ws_manager.broadcast_status_update(backend.queue_status())
return jsonify({"status": "cleared", "removed_count": removed_count})
except Exception as e:
logger.error_trace(f"Clear completed error: {e}")
return jsonify({"error": str(e)}), 500
@app.errorhandler(404)
def not_found_error(error: Exception) -> Union[Response, Tuple[Response, int]]:
"""
@@ -334,56 +659,214 @@ def internal_error(error: Exception) -> Union[Response, Tuple[Response, int]]:
logger.error_trace(f"500 error: {error}")
return jsonify({"error": "Internal server error"}), 500
def authenticate() -> bool:
@app.route('/api/auth/login', methods=['POST'])
def api_login() -> Union[Response, Tuple[Response, int]]:
"""
Helper function that validates Basic credentials
against a Calibre-Web app.db SQLite database
Database structure:
- Table 'user' with columns: 'name' (username), 'password'
Login endpoint that validates credentials and creates a session.
Includes rate limiting: 10 failed attempts = 30 minute lockout.
Request Body:
username (str): Username
password (str): Password
remember_me (bool): Whether to extend session duration
Returns:
flask.Response: JSON with success status or error message.
"""
# If the database doesn't exist, the user is always authenticated
if not CWA_DB_PATH:
return True
# If no authorization object exists, return false to prompt
# a request to the user
if not request.authorization:
return False
username = request.authorization.get("username")
password = request.authorization.get("password")
# Validate credentials against database
try:
conn = sqlite3.connect(CWA_DB_PATH)
cur = conn.cursor()
cur.execute("SELECT password FROM user WHERE name = ?", (username,))
row = cur.fetchone()
conn.close()
# Check if user exists and password is correct
if not row or not row[0] or not check_password_hash(row[0], password):
logger.error("User not found or password check failed")
return False
# Get client IP address (handles reverse proxy forwarding)
ip_address = request.headers.get('X-Forwarded-For', request.remote_addr)
if ip_address and ',' in ip_address:
# X-Forwarded-For can contain multiple IPs, take the first one
ip_address = ip_address.split(',')[0].strip()
data = request.get_json()
if not data:
return jsonify({"error": "No data provided"}), 400
username = data.get('username', '').strip()
password = data.get('password', '')
remember_me = data.get('remember_me', False)
if not username or not password:
return jsonify({"error": "Username and password are required"}), 400
# Check if account is locked due to failed login attempts
if is_account_locked(username):
lockout_until = failed_login_attempts[username].get('lockout_until')
remaining_time = (lockout_until - datetime.now()).total_seconds() / 60
logger.warning(f"Login attempt blocked for locked account '{username}' from IP {ip_address}")
return jsonify({
"error": f"Account temporarily locked due to multiple failed login attempts. Try again in {int(remaining_time)} minutes."
}), 429
# If the database doesn't exist, authentication always succeeds
if not CWA_DB_PATH:
session['user_id'] = username
session.permanent = remember_me
clear_failed_logins(username)
logger.info(f"Login successful for user '{username}' from IP {ip_address} (no DB configured)")
return jsonify({"success": True})
# If the CWA_DB_PATH variable exists, but isn't a valid path, return error
if not os.path.isfile(CWA_DB_PATH):
logger.error(f"CWA_DB_PATH is set to {CWA_DB_PATH} but this is not a valid path")
return jsonify({"error": "Database configuration error"}), 500
# Validate credentials against database
try:
db_path = os.fspath(CWA_DB_PATH)
db_uri = f"file:{db_path}?mode=ro&immutable=1"
conn = sqlite3.connect(db_uri, uri=True)
cur = conn.cursor()
cur.execute("SELECT password FROM user WHERE name = ?", (username,))
row = cur.fetchone()
conn.close()
# Check if user exists and password is correct
if not row or not row[0] or not check_password_hash(row[0], password):
# Record failed login attempt
is_now_locked = record_failed_login(username, ip_address)
if is_now_locked:
return jsonify({
"error": f"Account locked due to {MAX_LOGIN_ATTEMPTS} failed login attempts. Try again in {LOCKOUT_DURATION_MINUTES} minutes."
}), 429
else:
attempts_remaining = MAX_LOGIN_ATTEMPTS - failed_login_attempts[username]['count']
# Only show attempts remaining when 5 or fewer attempts remain (after 6+ failed attempts)
if attempts_remaining <= 5:
return jsonify({
"error": f"Invalid username or password. {attempts_remaining} attempts remaining."
}), 401
else:
return jsonify({
"error": "Invalid username or password."
}), 401
# Successful authentication - create session and clear failed attempts
session['user_id'] = username
session.permanent = remember_me
clear_failed_logins(username)
logger.info(f"Login successful for user '{username}' from IP {ip_address} (remember_me={remember_me})")
return jsonify({"success": True})
except Exception as e:
logger.error_trace(f"Database error during login: {e}")
return jsonify({"error": "Authentication system error"}), 500
except Exception as e:
logger.error_trace(f"CWA DB or authentication send_from_directory: {e}")
return False
logger.error_trace(f"Login error: {e}")
return jsonify({"error": "Login failed"}), 500
logger.info(f"Authentication successful for user {username}")
return True
@app.route('/api/auth/logout', methods=['POST'])
def api_logout() -> Union[Response, Tuple[Response, int]]:
"""
Logout endpoint that clears the session.
Returns:
flask.Response: JSON with success status.
"""
try:
# Get client IP address (handles reverse proxy forwarding)
ip_address = request.headers.get('X-Forwarded-For', request.remote_addr)
if ip_address and ',' in ip_address:
ip_address = ip_address.split(',')[0].strip()
username = session.get('user_id', 'unknown')
session.clear()
logger.info(f"Logout successful for user '{username}' from IP {ip_address}")
return jsonify({"success": True})
except Exception as e:
logger.error_trace(f"Logout error: {e}")
return jsonify({"error": "Logout failed"}), 500
# Register all routes with /request prefix
register_dual_routes(app)
@app.route('/api/auth/check', methods=['GET'])
def api_auth_check() -> Union[Response, Tuple[Response, int]]:
"""
Check if user has a valid session.
Returns:
flask.Response: JSON with authentication status and whether auth is required.
"""
try:
# If no database is configured, authentication is not required
if not CWA_DB_PATH:
return jsonify({
"authenticated": True,
"auth_required": False
})
# Check if user has a valid session
is_authenticated = 'user_id' in session
return jsonify({
"authenticated": is_authenticated,
"auth_required": True
})
except Exception as e:
logger.error_trace(f"Auth check error: {e}")
return jsonify({
"authenticated": False,
"auth_required": True
})
# Catch-all route for React Router (must be last)
# This handles client-side routing by serving index.html for any unmatched routes
@app.route('/<path:path>')
def catch_all(path: str) -> Response:
"""
Serve the React app for any route not matched by API endpoints.
This allows React Router to handle client-side routing.
Authentication is handled by the React app itself.
"""
# If the request is for an API endpoint or static file, let it 404
if path.startswith('api/') or path.startswith('assets/'):
return jsonify({"error": "Resource not found"}), 404
# Otherwise serve the React app
return send_from_directory(os.path.join(app.root_path, 'frontend-dist'), 'index.html')
# WebSocket event handlers
@socketio.on('connect')
def handle_connect():
"""Handle client connection."""
logger.info("WebSocket client connected")
# Track the connection (triggers warmup callbacks on first connect)
ws_manager.client_connected()
# Send initial status to the newly connected client
try:
status = backend.queue_status()
emit('status_update', status)
except Exception as e:
logger.error(f"Error sending initial status: {e}")
@socketio.on('disconnect')
def handle_disconnect():
"""Handle client disconnection."""
logger.info("WebSocket client disconnected")
# Track the disconnection
ws_manager.client_disconnected()
@socketio.on('request_status')
def handle_status_request():
"""Handle manual status request from client."""
try:
status = backend.queue_status()
emit('status_update', status)
except Exception as e:
logger.error(f"Error handling status request: {e}")
emit('error', {'message': 'Failed to get status'})
logger.log_resource_usage()
if __name__ == '__main__':
logger.info(f"Starting Flask application on {FLASK_HOST}:{FLASK_PORT} IN {APP_ENV} mode")
app.run(
logger.info(f"Starting Flask application with WebSocket support on {FLASK_HOST}:{FLASK_PORT} (debug={DEBUG})")
socketio.run(
app,
host=FLASK_HOST,
port=FLASK_PORT,
debug=DEBUG
debug=DEBUG,
allow_unsafe_werkzeug=True # For development only
)
+388 -66
View File
@@ -1,24 +1,41 @@
"""Backend logic for the book download application."""
import threading, time
import shutil
from pathlib import Path
from typing import Dict, List, Optional, Any, Tuple
import subprocess
import os
import random
import shutil
import subprocess
import threading
import time
from concurrent.futures import Future, ThreadPoolExecutor
from pathlib import Path
from threading import Event, Lock
from typing import Any, Dict, List, Optional, Tuple
from logger import setup_logger
from config import CUSTOM_SCRIPT
from env import INGEST_DIR, TMP_DIR, MAIN_LOOP_SLEEP_TIME, USE_BOOK_TITLE
from models import book_queue, BookInfo, QueueStatus, SearchFilters
import book_manager
from book_manager import SearchUnavailable
from config import CUSTOM_SCRIPT
from env import (
DOWNLOAD_PATHS, DOWNLOAD_PROGRESS_UPDATE_INTERVAL, INGEST_DIR,
MAIN_LOOP_SLEEP_TIME, MAX_CONCURRENT_DOWNLOADS, TMP_DIR, USE_BOOK_TITLE,
)
from logger import setup_logger
from models import BookInfo, QueueStatus, SearchFilters, book_queue
logger = setup_logger(__name__)
def _sanitize_filename(filename: str) -> str:
"""Sanitize a filename by replacing spaces with underscores and removing invalid characters."""
keepcharacters = (' ','.','_')
return "".join(c for c in filename if c.isalnum() or c in keepcharacters).rstrip()
# WebSocket manager (initialized by app.py)
try:
from websocket_manager import ws_manager
except ImportError:
ws_manager = None
# Progress update throttling - track last broadcast time per book
_progress_last_broadcast: Dict[str, float] = {}
_progress_lock = Lock()
# Stall detection - track last activity time per download
_last_activity: Dict[str, float] = {}
STALL_TIMEOUT = 300 # 5 minutes without progress/status update = stalled
def search_books(query: str, filters: SearchFilters) -> List[Dict[str, Any]]:
"""Search for books matching the query.
@@ -33,6 +50,9 @@ def search_books(query: str, filters: SearchFilters) -> List[Dict[str, Any]]:
try:
books = book_manager.search_books(query, filters)
return [_book_info_to_dict(book) for book in books]
except SearchUnavailable as e:
logger.warning(f"Search unavailable: {e}")
raise
except Exception as e:
logger.error_trace(f"Error searching books: {e}")
return []
@@ -53,19 +73,25 @@ def get_book_info(book_id: str) -> Optional[Dict[str, Any]]:
logger.error_trace(f"Error getting book info: {e}")
return None
def queue_book(book_id: str) -> bool:
"""Add a book to the download queue.
def queue_book(book_id: str, priority: int = 0) -> bool:
"""Add a book to the download queue with specified priority.
Args:
book_id: Book identifier
priority: Priority level (lower number = higher priority)
Returns:
bool: True if book was successfully queued
"""
try:
book_info = book_manager.get_book_info(book_id)
book_queue.add(book_id, book_info)
logger.info(f"Book queued: {book_info.title}")
book_queue.add(book_id, book_info, priority)
logger.info(f"Book queued with priority {priority}: {book_info.title}")
# Broadcast status update via WebSocket
if ws_manager:
ws_manager.broadcast_status_update(queue_status())
return True
except Exception as e:
logger.error_trace(f"Error queueing book: {e}")
@@ -75,12 +101,21 @@ def queue_status() -> Dict[str, Dict[str, Any]]:
"""Get current status of the download queue.
Returns:
Dict: Queue status organized by status type
Dict: Queue status organized by status type with serialized book data
"""
status = book_queue.get_status()
# Convert Enum keys to strings and properly format the response
for _, books in status.items():
for _, book_info in books.items():
if book_info.download_path:
if not os.path.exists(book_info.download_path):
book_info.download_path = None
# Convert Enum keys to strings and BookInfo objects to dicts for JSON serialization
return {
status_type.value: books
status_type.value: {
book_id: _book_info_to_dict(book_info)
for book_id, book_info in books.items()
}
for status_type, books in status.items()
}
@@ -100,8 +135,9 @@ def get_book_data(book_id: str) -> Tuple[Optional[bytes], BookInfo]:
return f.read(), book_info
except Exception as e:
logger.error_trace(f"Error getting book data: {e}")
book_info.download_path = None
return None, ""
if book_info:
book_info.download_path = None
return None, book_info if book_info else BookInfo(id=book_id, title="Unknown")
def _book_info_to_dict(book: BookInfo) -> Dict[str, Any]:
"""Convert BookInfo object to dictionary representation."""
@@ -110,82 +146,368 @@ def _book_info_to_dict(book: BookInfo) -> Dict[str, Any]:
if value is not None
}
def _download_book(book_id: str) -> Optional[str]:
"""Download and process a book.
def _prepare_download_folder(book_info: BookInfo) -> Path:
"""Prepare final content-type subdir"""
content = book_info.content
content_dir = DOWNLOAD_PATHS.get(content) if content and content in DOWNLOAD_PATHS else INGEST_DIR
os.makedirs(content_dir, exist_ok=True)
return content_dir
def _download_book_with_cancellation(book_id: str, cancel_flag: Event) -> Optional[str]:
"""Download and process a book with cancellation support.
Args:
book_id: Book identifier
cancel_flag: Threading event to signal cancellation
Returns:
str: Path to the downloaded book if successful, None otherwise
"""
try:
# Check for cancellation before starting
if cancel_flag.is_set():
logger.info(f"Download cancelled before starting: {book_id}")
return None
book_info = book_queue._book_data[book_id]
logger.info(f"Starting download: {book_info.title}")
if USE_BOOK_TITLE:
book_name = _sanitize_filename(book_info.title)
else:
book_name = book_id
book_name += f".{book_info.format}"
if not book_info.download_urls:
raise ValueError(f"No download URLs available for {book_id}")
# get_filename() resolves format as side effect
full_name = book_info.get_filename()
book_name = full_name if USE_BOOK_TITLE else f"{book_id}.{book_info.format or 'bin'}"
book_path = TMP_DIR / book_name
success = book_manager.download_book(book_info, book_path)
if not success:
raise Exception("Unkown error downloading book")
# Check cancellation before download
if cancel_flag.is_set():
logger.info(f"Download cancelled before book manager call: {book_id}")
return None
progress_callback = lambda progress: update_download_progress(book_id, progress)
status_callback = lambda status, message=None: update_download_status(book_id, status, message)
# Set status to resolving immediately when processing starts
update_download_status(book_id, "resolving")
success_download_url = book_manager.download_book(book_info, book_path, progress_callback, cancel_flag, status_callback)
# Stop progress updates
cancel_flag.wait(0.1) # Brief pause for progress thread cleanup
if cancel_flag.is_set():
logger.info(f"Download cancelled during download: {book_id}")
# Clean up partial download
if book_path.exists():
book_path.unlink()
return None
if not success_download_url:
raise Exception("Unknown error downloading book")
# Check cancellation before post-processing
if cancel_flag.is_set():
logger.info(f"Download cancelled before post-processing: {book_id}")
if book_path.exists():
book_path.unlink()
return None
logger.debug(f"Post-processing download: {book_info.title}")
if CUSTOM_SCRIPT:
logger.info(f"Running custom script: {CUSTOM_SCRIPT}")
subprocess.run([CUSTOM_SCRIPT, book_path])
intermediate_path = INGEST_DIR / f"{book_id}.crdownload"
final_path = INGEST_DIR / book_name
# Regenerate filename with fallback to successful download URL for format
full_name = book_info.get_filename(success_download_url)
book_name = full_name if USE_BOOK_TITLE else f"{book_id}.{book_info.format or 'bin'}"
final_dir = _prepare_download_folder(book_info)
intermediate_path = final_dir / f"{book_id}.crdownload"
final_path = final_dir / book_name
# Handle file already exists - add suffix to avoid overwrite
if final_path.exists():
base = final_path.stem
ext = final_path.suffix
counter = 1
while final_path.exists():
final_path = final_dir / f"{base}_{counter}{ext}"
counter += 1
logger.info(f"File already exists, saving as: {final_path.name}")
if os.path.exists(book_path):
logger.info(f"Moving book to ingest directory then renaming: {book_path} -> {intermediate_path} -> {final_path}")
logger.info(f"Moving book to ingest directory: {book_path} -> {final_path}")
try:
shutil.move(book_path, intermediate_path)
except Exception as e:
logger.debug(f"Error moving book: {e}, will try copying instead")
shutil.copy(book_path, intermediate_path)
try:
logger.debug(f"Error moving book: {e}, will try copying instead")
shutil.move(book_path, intermediate_path)
except Exception as e:
logger.debug(f"Error copying book: {e}, will try copying without permissions instead")
shutil.copyfile(book_path, intermediate_path)
os.remove(book_path)
logger.info(f"Renaming book: {intermediate_path} -> {final_path}")
# Final cancellation check before completing
if cancel_flag.is_set():
logger.info(f"Download cancelled before final rename: {book_id}")
if intermediate_path.exists():
intermediate_path.unlink()
return None
os.rename(intermediate_path, final_path)
logger.info(f"Download completed successfully: {book_info.title}")
return str(final_path)
except Exception as e:
logger.error_trace(f"Error downloading book: {e}")
if cancel_flag.is_set():
logger.info(f"Download cancelled during error handling: {book_id}")
else:
logger.error_trace(f"Error downloading book: {e}")
return None
def download_loop() -> None:
"""Background thread for processing download queue."""
logger.info("Starting download loop")
def update_download_progress(book_id: str, progress: float) -> None:
"""Update download progress with throttled WebSocket broadcasts.
Progress is always stored in the queue, but WebSocket broadcasts are
throttled to avoid flooding clients with updates. Broadcasts occur:
- At most once per DOWNLOAD_PROGRESS_UPDATE_INTERVAL seconds
- Always at 0% (start) and 100% (complete)
- On significant progress jumps (>10%)
"""
book_queue.update_progress(book_id, progress)
# Track activity for stall detection
with _progress_lock:
_last_activity[book_id] = time.time()
while True:
book_id = book_queue.get_next()
if not book_id:
time.sleep(MAIN_LOOP_SLEEP_TIME)
continue
# Broadcast progress via WebSocket with throttling
if ws_manager:
current_time = time.time()
should_broadcast = False
with _progress_lock:
last_broadcast = _progress_last_broadcast.get(book_id, 0)
last_progress = _progress_last_broadcast.get(f"{book_id}_progress", 0)
time_elapsed = current_time - last_broadcast
try:
book_queue.update_status(book_id, QueueStatus.DOWNLOADING)
download_path = _download_book(book_id)
if download_path:
book_queue.update_download_path(book_id, download_path)
# Always broadcast at start (0%) or completion (>=99%)
if progress <= 1 or progress >= 99:
should_broadcast = True
# Broadcast if enough time has passed (convert interval from seconds)
elif time_elapsed >= DOWNLOAD_PROGRESS_UPDATE_INTERVAL:
should_broadcast = True
# Broadcast on significant progress jumps (>10%)
elif progress - last_progress >= 10:
should_broadcast = True
if should_broadcast:
_progress_last_broadcast[book_id] = current_time
_progress_last_broadcast[f"{book_id}_progress"] = progress
if should_broadcast:
ws_manager.broadcast_download_progress(book_id, progress, 'downloading')
new_status = (
QueueStatus.AVAILABLE if download_path else QueueStatus.ERROR
)
book_queue.update_status(book_id, new_status)
def update_download_status(book_id: str, status: str, message: Optional[str] = None) -> None:
"""Update download status with optional detailed message.
Args:
book_id: Book identifier
status: Status string (e.g., 'resolving', 'downloading')
message: Optional detailed status message for UI display
"""
# Map string status to QueueStatus enum
status_map = {
'queued': QueueStatus.QUEUED,
'resolving': QueueStatus.RESOLVING,
'downloading': QueueStatus.DOWNLOADING,
'complete': QueueStatus.COMPLETE,
'available': QueueStatus.AVAILABLE,
'error': QueueStatus.ERROR,
'done': QueueStatus.DONE,
'cancelled': QueueStatus.CANCELLED,
}
queue_status_enum = status_map.get(status.lower())
if queue_status_enum:
book_queue.update_status(book_id, queue_status_enum)
# Track activity for stall detection
with _progress_lock:
_last_activity[book_id] = time.time()
# Update status message if provided (empty string clears the message)
if message is not None:
book_queue.update_status_message(book_id, message)
# Broadcast status update via WebSocket
if ws_manager:
ws_manager.broadcast_status_update(queue_status())
def cancel_download(book_id: str) -> bool:
"""Cancel a download.
Args:
book_id: Book identifier to cancel
Returns:
bool: True if cancellation was successful
"""
result = book_queue.cancel_download(book_id)
# Broadcast status update via WebSocket
if result and ws_manager and ws_manager.is_enabled():
ws_manager.broadcast_status_update(queue_status())
return result
def set_book_priority(book_id: str, priority: int) -> bool:
"""Set priority for a queued book.
Args:
book_id: Book identifier
priority: New priority level (lower = higher priority)
Returns:
bool: True if priority was successfully changed
"""
return book_queue.set_priority(book_id, priority)
def reorder_queue(book_priorities: Dict[str, int]) -> bool:
"""Bulk reorder queue.
Args:
book_priorities: Dict mapping book_id to new priority
Returns:
bool: True if reordering was successful
"""
return book_queue.reorder_queue(book_priorities)
def get_queue_order() -> List[Dict[str, any]]:
"""Get current queue order for display."""
return book_queue.get_queue_order()
def get_active_downloads() -> List[str]:
"""Get list of currently active downloads."""
return book_queue.get_active_downloads()
def clear_completed() -> int:
"""Clear all completed downloads from tracking."""
return book_queue.clear_completed()
def _cleanup_progress_tracking(book_id: str) -> None:
"""Clean up progress tracking data for a completed/cancelled download."""
with _progress_lock:
_progress_last_broadcast.pop(book_id, None)
_progress_last_broadcast.pop(f"{book_id}_progress", None)
_last_activity.pop(book_id, None)
def _process_single_download(book_id: str, cancel_flag: Event) -> None:
"""Process a single download job."""
try:
# Status will be updated through callbacks during download process
# (resolving -> downloading -> complete)
download_path = _download_book_with_cancellation(book_id, cancel_flag)
# Clean up progress tracking
_cleanup_progress_tracking(book_id)
if cancel_flag.is_set():
book_queue.update_status(book_id, QueueStatus.CANCELLED)
# Broadcast cancellation
if ws_manager:
ws_manager.broadcast_status_update(queue_status())
return
logger.info(
f"Book {book_id} download {'successful' if download_path else 'failed'}"
)
if download_path:
book_queue.update_download_path(book_id, download_path)
new_status = QueueStatus.COMPLETE
else:
new_status = QueueStatus.ERROR
except Exception as e:
logger.error_trace(f"Error in download loop: {e}")
book_queue.update_status(book_id, new_status)
# Broadcast final status (completed or error)
if ws_manager:
ws_manager.broadcast_status_update(queue_status())
except Exception as e:
# Clean up progress tracking even on error
_cleanup_progress_tracking(book_id)
if not cancel_flag.is_set():
logger.error_trace(f"Error in download processing: {e}")
book_queue.update_status(book_id, QueueStatus.ERROR)
# Set error message if not already set by download_book()
if book_id in book_queue._book_data and not book_queue._book_data[book_id].status_message:
book_queue.update_status_message(book_id, f"Download failed: {type(e).__name__}: {str(e)}")
else:
logger.info(f"Download cancelled: {book_id}")
book_queue.update_status(book_id, QueueStatus.CANCELLED)
# Broadcast error/cancelled status
if ws_manager:
ws_manager.broadcast_status_update(queue_status())
# Start download loop in background thread
download_thread = threading.Thread(
target=download_loop,
daemon=True
def concurrent_download_loop() -> None:
"""Main download coordinator using ThreadPoolExecutor for concurrent downloads."""
logger.info(f"Starting concurrent download loop with {MAX_CONCURRENT_DOWNLOADS} workers")
with ThreadPoolExecutor(max_workers=MAX_CONCURRENT_DOWNLOADS, thread_name_prefix="BookDownload") as executor:
active_futures: Dict[Future, str] = {} # Track active download futures
while True:
# Clean up completed futures
completed_futures = [f for f in active_futures if f.done()]
for future in completed_futures:
book_id = active_futures.pop(future)
try:
future.result() # This will raise any exceptions from the worker
except Exception as e:
logger.error_trace(f"Future exception for {book_id}: {e}")
# Check for stalled downloads (no activity in STALL_TIMEOUT seconds)
current_time = time.time()
with _progress_lock:
for future, book_id in list(active_futures.items()):
last_active = _last_activity.get(book_id, current_time)
if current_time - last_active > STALL_TIMEOUT:
logger.warning(f"Download stalled for {book_id}, cancelling")
book_queue.cancel_download(book_id)
book_queue.update_status_message(book_id, f"Download stalled (no activity for {STALL_TIMEOUT}s)")
# Start new downloads if we have capacity
while len(active_futures) < MAX_CONCURRENT_DOWNLOADS:
next_download = book_queue.get_next()
if not next_download:
break
# Stagger concurrent downloads to avoid rate limiting on shared download servers
# Only delay if other downloads are already active
if active_futures:
stagger_delay = random.uniform(2, 5)
logger.debug(f"Staggering download start by {stagger_delay:.1f}s")
time.sleep(stagger_delay)
book_id, cancel_flag = next_download
# Submit download job to thread pool
future = executor.submit(_process_single_download, book_id, cancel_flag)
active_futures[future] = book_id
# Brief sleep to prevent busy waiting
time.sleep(MAIN_LOOP_SLEEP_TIME)
# Start concurrent download coordinator
download_coordinator_thread = threading.Thread(
target=concurrent_download_loop,
daemon=True,
name="DownloadCoordinator"
)
download_thread.start()
download_coordinator_thread.start()
logger.info(f"Download system initialized with {MAX_CONCURRENT_DOWNLOADS} concurrent workers")
+524 -117
View File
@@ -1,19 +1,38 @@
"""Book download manager handling search and retrieval operations."""
import time, json, re
import itertools
import json
import re
import time
from pathlib import Path
from threading import Event
from typing import Callable, Dict, List, Optional
from urllib.parse import quote
from typing import List, Optional, Dict, Union
from bs4 import BeautifulSoup, Tag, NavigableString, ResultSet
from bs4 import BeautifulSoup, NavigableString, Tag
import downloader
import network
from config import BOOK_LANGUAGE, SUPPORTED_FORMATS
from env import AA_DONATOR_KEY, ALLOW_USE_WELIB, DEBUG_SKIP_SOURCES, DOWNLOAD_PATHS, PRIORITIZE_WELIB, USE_CF_BYPASS
from logger import setup_logger
from config import SUPPORTED_FORMATS, BOOK_LANGUAGE, AA_BASE_URL
from env import AA_DONATOR_KEY, USE_CF_BYPASS
from models import BookInfo, SearchFilters
logger = setup_logger(__name__)
# Round-robin counter for AA slow download source rotation
# Distributes concurrent downloads across different partner mirrors
_aa_slow_rotation = itertools.count()
if DEBUG_SKIP_SOURCES:
logger.warning("DEBUG_SKIP_SOURCES active: skipping sources %s", DEBUG_SKIP_SOURCES)
class SearchUnavailable(Exception):
"""Raised when Anna's Archive cannot be reached via any mirror/DNS."""
pass
def search_books(query: str, filters: SearchFilters) -> List[BookInfo]:
"""Search for books matching the query.
@@ -61,8 +80,10 @@ def search_books(query: str, filters: SearchFilters) -> List[BookInfo]:
)
index += 1
selector = network.AAMirrorSelector()
url = (
f"{AA_BASE_URL}"
f"{network.get_aa_base_url()}"
f"/search?index=&page=1&display=table"
f"&acc=aa_download&acc=external_download"
f"&ext={'&ext='.join(formats_to_use)}"
@@ -70,13 +91,14 @@ def search_books(query: str, filters: SearchFilters) -> List[BookInfo]:
f"{filters_query}"
)
html = downloader.html_get_page(url)
html = downloader.html_get_page(url, selector=selector)
if not html:
raise Exception("Failed to fetch search results")
# Network/mirror exhaustion path bubbles up so API can notify clients
raise SearchUnavailable("Unable to reach Anna's Archive. Network restricted or mirrors are blocked.")
if "No files found." in html:
logger.info(f"No books found for query: {query}")
raise Exception("No books found. Please try another query.")
return []
soup = BeautifulSoup(html, "html.parser")
tbody: Tag | NavigableString | None = soup.find("table")
@@ -109,6 +131,9 @@ def search_books(query: str, filters: SearchFilters) -> List[BookInfo]:
def _parse_search_result_row(row: Tag) -> Optional[BookInfo]:
"""Parse a single search result row into a BookInfo object."""
try:
# Skip ad rows
if row.text.strip().lower().startswith("your ad here"):
return None
cells = row.find_all("td")
preview_img = cells[0].find("img")
preview = preview_img["src"] if preview_img else None
@@ -121,6 +146,7 @@ def _parse_search_result_row(row: Tag) -> Optional[BookInfo]:
publisher=cells[3].find("span").next,
year=cells[4].find("span").next,
language=cells[7].find("span").next,
content=cells[8].find("span").next.lower(),
format=cells[9].find("span").next.lower(),
size=cells[10].find("span").next,
)
@@ -138,8 +164,9 @@ def get_book_info(book_id: str) -> BookInfo:
Returns:
BookInfo: Detailed book information
"""
url = f"{AA_BASE_URL}/md5/{book_id}"
html = downloader.html_get_page(url)
url = f"{network.get_aa_base_url()}/md5/{book_id}"
selector = network.AAMirrorSelector()
html = downloader.html_get_page(url, selector=selector)
if not html:
raise Exception(f"Failed to fetch book info for ID: {book_id}")
@@ -168,72 +195,127 @@ def _parse_book_info_page(soup: BeautifulSoup, book_id: str) -> BookInfo:
data = soup.find_all("div", {"class": "main-inner"})[0].find_next("div")
divs = list(data.children)
format = divs[13].text.split(" · ")[1].strip().lower()
size = divs[13].text.split(" · ")[2].strip().lower()
every_url = soup.find_all("a")
slow_urls_no_waitlist = set()
slow_urls_with_waitlist = set()
external_urls_libgen = set()
external_urls_z_lib = set()
external_urls_welib = set()
# Collect download URLs by source type (lists preserve page order, dedup inline)
slow_urls_no_waitlist: list[str] = []
slow_urls_with_waitlist: list[str] = []
external_urls_libgen: list[str] = []
external_urls_z_lib: list[str] = []
for url in every_url:
def _append_unique(lst: list[str], href: str) -> None:
if href and href not in lst:
lst.append(href)
for anchor in soup.find_all("a"):
try:
if url.text.strip().lower().startswith("slow partner server"):
if (
url.next is not None
and url.next.next is not None
and "waitlist" in url.next.next.strip().lower()
):
internal_text = url.next.next.strip().lower()
if "no waitlist" in internal_text:
slow_urls_no_waitlist.add(url["href"])
else:
slow_urls_with_waitlist.add(url["href"])
elif (
url.next is not None
and url.next.next is not None
and "click “GET” at the top" in url.next.next.text.strip()
):
libgen_url = url["href"]
# TODO : Temporary fix ? Maybe get URLs from https://open-slum.org/ ?
libgen_url = libgen_url = re.sub(r'libgen\.(\w+)', 'libgen.bz', url["href"])
external_urls_libgen.add(libgen_url)
elif url.text.strip().lower().startswith("z-lib"):
if ".onion/" not in url["href"]:
external_urls_z_lib.add(url["href"])
text = anchor.text.strip().lower()
href = anchor.get("href", "")
next_text = ""
if anchor.next and anchor.next.next:
next_text = getattr(anchor.next.next, 'text', str(anchor.next.next)).strip().lower()
if text.startswith("slow partner server") and "waitlist" in next_text:
if "no waitlist" in next_text:
_append_unique(slow_urls_no_waitlist, href)
else:
_append_unique(slow_urls_with_waitlist, href)
elif 'libgen.li' in href:
# Normalize libgen domains
libgen_url = re.sub(r'libgen\.(li|lc|is|bz|st)', 'libgen.gl', href)
_append_unique(external_urls_libgen, libgen_url)
elif text.startswith("z-lib") and ".onion/" not in href:
_append_unique(external_urls_z_lib, href)
except:
pass
logger.debug(
"Source inventory for %s -> aa_no_wait=%d, aa_wait=%d, libgen=%d, zlib=%d",
book_id,
len(slow_urls_no_waitlist),
len(slow_urls_with_waitlist),
len(external_urls_libgen),
len(external_urls_z_lib),
)
urls = []
urls += list(slow_urls_no_waitlist) if USE_CF_BYPASS else []
urls += list(external_urls_libgen)
urls += list( _get_download_urls_from_welib(book_id)) if USE_CF_BYPASS else []
urls += list(slow_urls_with_waitlist) if USE_CF_BYPASS else []
urls += list(external_urls_z_lib)
# Priority: reliable sources first, then external fallbacks
# 1. AA slow (no waitlist) - instant but can be slow
# 2. Libgen - instant, external
# 3. AA slow (waitlist) - has countdown timer but faster once started
# Note: Z-Library disabled - download tokens are session-bound
urls += slow_urls_no_waitlist if USE_CF_BYPASS else []
urls += external_urls_libgen
urls += slow_urls_with_waitlist if USE_CF_BYPASS else []
for i in range(len(urls)):
urls[i] = downloader.get_absolute_url(AA_BASE_URL, urls[i])
urls[i] = downloader.get_absolute_url(network.get_aa_base_url(), urls[i])
# Remove empty urls
urls = [url for url in urls if url != ""]
# Tag AA slow URLs with detailed source type for skip/retry tracking
base_url = network.get_aa_base_url()
for rel_url in slow_urls_no_waitlist:
abs_url = downloader.get_absolute_url(base_url, rel_url)
if abs_url:
_url_source_types[abs_url] = "aa-slow-nowait"
for rel_url in slow_urls_with_waitlist:
abs_url = downloader.get_absolute_url(base_url, rel_url)
if abs_url:
_url_source_types[abs_url] = "aa-slow-wait"
# Filter out divs that are not text
original_divs = divs
divs = [div for div in divs if div.text.strip() != ""]
all_details = _find_in_divs(divs, " · ")
format = ""
size = ""
content = ""
for _details in all_details:
_details = _details.split(" · ")
for f in _details:
if format == "" and f.strip().lower() in SUPPORTED_FORMATS:
format = f.strip().lower()
if size == "" and any(u in f.strip().lower() for u in ["mb", "kb", "gb"]):
# Preserve original case but uppercase the unit (e.g., "5.2 mb" -> "5.2 MB")
size = re.sub(r'(kb|mb|gb|tb)', lambda m: m.group(1).upper(), f.strip(), flags=re.IGNORECASE)
if content == "":
for ct in DOWNLOAD_PATHS.keys():
if ct in f.strip().lower():
content = ct
break
if format == "" or size == "":
for f in _details:
stripped = f.strip().lower()
if format == "" and stripped and " " not in stripped:
format = stripped
if size == "" and "." in stripped:
# Uppercase any size units
size = re.sub(r'(kb|mb|gb|tb)', lambda m: m.group(1).upper(), f.strip(), flags=re.IGNORECASE)
book_title = _find_in_divs(divs, "🔍")[0].strip("🔍").strip()
# Extract basic information
description = _extract_book_description(soup)
book_info = BookInfo(
id=book_id,
preview=preview,
title=divs[7].next.strip(),
publisher=divs[11].text.strip(),
author=divs[9].text.strip(),
title=book_title,
content=content,
publisher=_find_in_divs(divs, "icon-[mdi--company]", is_class=True)[0],
author=_find_in_divs(divs, "icon-[mdi--user-edit]", is_class=True)[0],
format=format,
size=size,
description=description,
download_urls=urls,
)
# Extract additional metadata
info = _extract_book_metadata(divs[-6])
info = _extract_book_metadata(original_divs[-6])
book_info.info = info
# Set language and year from metadata if available
@@ -242,24 +324,139 @@ def _parse_book_info_page(soup: BeautifulSoup, book_id: str) -> BookInfo:
if info.get("Year"):
book_info.year = info["Year"][0]
# TODO :
# Backfill missing metadata from original book
# To do this, we need to cache the results of search_books() in some kind of LRU
return book_info
def _get_download_urls_from_welib(book_id: str) -> List[str]:
"""Get download urls from welib.org."""
url = f"https://welib.org/md5/{book_id}"
html = downloader.html_get_page(url, use_bypasser=True)
if not html:
return []
soup = BeautifulSoup(html, "html.parser")
download_links = soup.find_all("a", href=True)
download_links = [link["href"] for link in download_links]
download_links = [link for link in download_links if "/slow_download/" in link]
download_links = [downloader.get_absolute_url(url, link) for link in download_links]
return set(download_links)
def _find_in_divs(divs: List, text: str, is_class: bool = False) -> List[str]:
"""Find divs containing text or having a specific class."""
results = []
for div in divs:
if is_class:
if div.find(class_=text):
results.append(div.text.strip())
elif text in div.text.strip():
results.append(div.text.strip())
return results
def _extract_book_metadata(
metadata_divs
) -> Dict[str, List[str]]:
# Download source definitions: (log_label, friendly_name, url_patterns)
_DOWNLOAD_SOURCES = [
("welib", "Welib", ["welib.org"]),
("aa-fast", "Anna's Archive (Fast)", ["/dyn/api/fast_download"]),
("aa-slow-wait", "Anna's Archive (Waitlist)", []), # Matched via _url_source_types
("aa-slow-nowait", "Anna's Archive", []), # Matched via _url_source_types
("aa-slow", "Anna's Archive", ["/slow_download/", "annas-"]), # Fallback for untagged AA URLs
("libgen", "Libgen", ["libgen"]),
("zlib", "Z-Library", ["z-lib", "zlibrary"]),
]
# Track detailed source types for AA slow URLs (populated during get_book_info)
_url_source_types: dict[str, str] = {}
def _get_source_info(link: str) -> tuple[str, str]:
"""Get source label and friendly name for a download link.
Args:
link: Download URL
Returns:
Tuple of (log_label, friendly_name)
"""
# Check detailed source type mapping first (for AA slow distinction)
if link in _url_source_types:
detailed_label = _url_source_types[link]
for log_label, friendly_name, _ in _DOWNLOAD_SOURCES:
if log_label == detailed_label:
return log_label, friendly_name
for log_label, friendly_name, patterns in _DOWNLOAD_SOURCES:
if patterns and any(pattern in link for pattern in patterns):
return log_label, friendly_name
return "unknown", "Mirror"
def _label_source(link: str) -> str:
"""Get lightweight source tag for logging/metrics."""
return _get_source_info(link)[0]
def _friendly_source_name(link: str) -> str:
"""Get user-friendly name for a download source."""
return _get_source_info(link)[1]
def _get_download_urls_from_welib(book_id: str, selector: Optional[network.AAMirrorSelector] = None, cancel_flag: Optional[Event] = None) -> list[str]:
"""Get download URLs from welib.org (bypasser required)."""
if not ALLOW_USE_WELIB:
return []
url = f"https://welib.org/md5/{book_id}"
logger.info(f"Fetching welib.org download URLs for {book_id}")
try:
html = downloader.html_get_page(url, use_bypasser=True, selector=selector or network.AAMirrorSelector(), cancel_flag=cancel_flag)
except Exception as exc:
logger.error_trace(f"Welib fetch failed for {book_id}: {exc}")
return []
if not html:
logger.warning(f"Welib page empty for {book_id}")
return []
soup = BeautifulSoup(html, "html.parser")
links = [
downloader.get_absolute_url(url, a["href"])
for a in soup.find_all("a", href=True)
if "/slow_download/" in a["href"]
]
return list(dict.fromkeys(links)) # Dedupe while preserving order
def _get_next_value_div(label_div: Tag) -> Optional[Tag]:
"""Find the next sibling div that holds the value for a metadata label."""
sibling = label_div.next_sibling
while sibling:
if isinstance(sibling, Tag) and sibling.name == "div":
return sibling
sibling = sibling.next_sibling
return None
def _extract_book_description(soup: BeautifulSoup) -> Optional[str]:
"""Extract the primary or alternative description from the book page."""
container = soup.select_one(".js-md5-top-box-description")
if not container:
return None
description: Optional[str] = None
alternative: Optional[str] = None
label_divs = container.select("div.text-xs.text-gray-500.uppercase")
for label_div in label_divs:
label_text = label_div.get_text(strip=True).lower()
value_div = _get_next_value_div(label_div)
if not value_div:
continue
value_text = value_div.get_text(separator=" ", strip=True)
if not value_text:
continue
if label_text == "description":
return value_text
if label_text == "alternative description" and not alternative:
alternative = value_text
if alternative:
return alternative
# Fallback to the first text block inside the description container
fallback_div = container.find("div", class_="mb-1")
if fallback_div:
fallback_text = fallback_div.get_text(separator=" ", strip=True)
if fallback_text:
return fallback_text
return None
def _extract_book_metadata(metadata_divs) -> Dict[str, List[str]]:
"""Extract metadata from book info divs."""
info: Dict[str, List[str]] = {}
@@ -297,82 +494,292 @@ def _extract_book_metadata(
}
def download_book(book_info: BookInfo, book_path: Path) -> bool:
# After N consecutive failures of the same source type, skip remaining sources of that type
SOURCE_FAILURE_THRESHOLD = 4
# Minimum valid file size in bytes (10KB) - anything smaller is likely an error page
MIN_VALID_FILE_SIZE = 10 * 1024
def download_book(book_info: BookInfo, book_path: Path, progress_callback: Optional[Callable[[float], None]] = None, cancel_flag: Optional[Event] = None, status_callback: Optional[Callable[[str, Optional[str]], None]] = None) -> Optional[str]:
"""Download a book from available sources.
Args:
book_id: Book identifier (MD5 hash)
title: Book title for logging
progress_callback: Optional callback for download progress updates
cancel_flag: Optional cancellation flag
status_callback: Optional callback for status updates (status, message)
Returns:
Optional[BytesIO]: Book content buffer if successful
str: Download URL if successful, None otherwise
"""
selector = network.AAMirrorSelector()
if len(book_info.download_urls) == 0:
book_info = get_book_info(book_info.id)
download_links = book_info.download_urls
download_links = list(book_info.download_urls)
# If AA_DONATOR_KEY is set, use the fast download URL. Else try other sources.
if AA_DONATOR_KEY != "":
download_links.insert(
0,
f"{AA_BASE_URL}/dyn/api/fast_download.json?md5={book_info.id}&key={AA_DONATOR_KEY}",
f"{network.get_aa_base_url()}/dyn/api/fast_download.json?md5={book_info.id}&key={AA_DONATOR_KEY}",
)
for link in download_links:
try:
download_url = _get_download_url(link, book_info.title)
if download_url != "":
logger.info(f"Downloading `{book_info.title}` from `{download_url}`")
data = downloader.download_url(download_url, book_info.size or "")
if not data:
raise Exception("No data received")
# Preserve order but drop duplicates to avoid retrying the same host
download_links = list(dict.fromkeys(download_links))
logger.info(f"Download finished. Writing to {book_path}")
with open(book_path, "wb") as f:
f.write(data.getbuffer())
logger.info(f"Writing `{book_info.title}` successfully")
return True
# Round-robin rotation for AA slow download URLs to distribute load across mirrors
# This prevents all concurrent downloads from hitting the same partner server first
# Rotate aa-slow-nowait and aa-slow-wait independently to preserve priority ordering
rotation_value = next(_aa_slow_rotation)
except Exception as e:
logger.error_trace(f"Failed to download from {link}: {e}")
def _rotate_category_in_place(links: list, source_type: str) -> int:
"""Rotate URLs of a specific source type within the list, preserving their positions."""
indices = [i for i, u in enumerate(links) if _url_source_types.get(u) == source_type]
if len(indices) <= 1:
return 0
rotation = rotation_value % len(indices)
if rotation == 0:
return 0
# Extract values, rotate, put back
values = [links[i] for i in indices]
rotated = values[rotation:] + values[:rotation]
for idx, val in zip(indices, rotated):
links[idx] = val
return rotation
nowait_rotation = _rotate_category_in_place(download_links, "aa-slow-nowait")
wait_rotation = _rotate_category_in_place(download_links, "aa-slow-wait")
if nowait_rotation or wait_rotation:
logger.info(f"AA source rotation: nowait={nowait_rotation}, wait={wait_rotation}")
links_queue = download_links
# Fetch welib URLs upfront when prioritized
welib_fallback_loaded = "welib" in DEBUG_SKIP_SOURCES # Skip welib entirely if in debug skip list
if USE_CF_BYPASS and PRIORITIZE_WELIB and ALLOW_USE_WELIB and not welib_fallback_loaded:
logger.info("Fetching welib.org download URLs (PRIORITIZE_WELIB enabled)")
if status_callback:
status_callback("resolving", "Fetching welib sources...")
welib_links = _get_download_urls_from_welib(book_info.id, selector=selector, cancel_flag=cancel_flag)
if welib_links:
links_queue = welib_links + [l for l in links_queue if l not in welib_links]
welib_fallback_loaded = True
total_sources = len(links_queue)
# Handle case where no download sources are available
if total_sources == 0:
logger.warning(f"No download sources available for: {book_info.title}")
if status_callback:
status_callback("error", "No download sources found")
return None
# Track consecutive failures per source type to skip after threshold
source_failures: dict[str, int] = {}
# Iterate with index so we can append welib links later
idx = 0
while idx < len(links_queue):
link = links_queue[idx]
source_label = _label_source(link)
friendly_name = _friendly_source_name(link)
# Debug: skip sources for testing fallback chains
if source_label in DEBUG_SKIP_SOURCES:
logger.info("DEBUG_SKIP_SOURCES: skipping %s (%s)", source_label, link)
idx += 1
continue
return False
# Skip source types that have failed too many times
if source_failures.get(source_label, 0) >= SOURCE_FAILURE_THRESHOLD:
logger.info("Skipping %s - source type '%s' failed %d times", link, source_label, SOURCE_FAILURE_THRESHOLD)
idx += 1
continue
try:
current_pos = idx + 1
# Update total if we added more sources
total_sources = len(links_queue)
logger.info("Trying download source [%s]: %s (%d/%d)", source_label, link, current_pos, total_sources)
# Build source context for status messages (e.g., "Welib (1/12)")
source_context = f"{friendly_name} (Server #{current_pos})"
# Update status with simple message showing which source we're trying
if status_callback:
status_callback("resolving", f"Trying {source_context}")
download_url = _get_download_url(link, book_info.title, cancel_flag, status_callback, selector, source_context)
if download_url == "":
raise Exception("No download URL resolved")
logger.info("Resolved download URL [%s]: %s", source_label, download_url)
# Pass source page as referer (required by some sites)
data = downloader.download_url(download_url, book_info.size or "", progress_callback, cancel_flag, selector, status_callback, referer=link)
if not data:
raise Exception("No data received from download")
# Validate file size - reject suspiciously small files
file_size = data.tell()
if file_size < MIN_VALID_FILE_SIZE:
logger.warning(f"Downloaded file too small ({file_size} bytes), likely an error page")
raise Exception(f"File too small ({file_size} bytes)")
logger.debug(f"Download finished ({file_size} bytes). Writing to {book_path}")
data.seek(0) # Reset buffer position before writing
with open(book_path, "wb") as f:
f.write(data.getbuffer())
return download_url
except Exception as e:
logger.warning(f"Failed to download from {link} (source={source_label}): {e}")
source_failures[source_label] = source_failures.get(source_label, 0) + 1
idx += 1
# If we exhausted primary links and haven't loaded welib yet, fetch them lazily
if (
idx >= len(links_queue)
and not welib_fallback_loaded
and USE_CF_BYPASS
and ALLOW_USE_WELIB
):
welib_selector = selector # reuse AA mirror selector for consistency
welib_links = _get_download_urls_from_welib(book_info.id, selector=welib_selector, cancel_flag=cancel_flag)
welib_fallback_loaded = True
if welib_links:
new_links = [wl for wl in welib_links if wl not in links_queue]
if new_links:
logger.info("Adding welib fallback links (%d)", len(new_links))
links_queue.extend(new_links)
# continue loop to try newly added links
continue
# All sources exhausted - report final error to UI
if status_callback:
status_callback("error", f"All {len(links_queue)} sources failed")
return None
def _get_download_url(link: str, title: str) -> str:
"""Extract actual download URL from various source pages."""
def _get_download_url(link: str, title: str, cancel_flag: Optional[Event] = None, status_callback: Optional[Callable[[str, Optional[str]], None]] = None, selector: Optional[network.AAMirrorSelector] = None, source_context: Optional[str] = None) -> str:
"""Extract actual download URL from various source pages.
Args:
link: URL to extract download link from
title: Book title for logging
cancel_flag: Optional cancellation flag
status_callback: Optional callback for status updates
selector: Optional AA mirror selector
source_context: Optional context string like "Welib (1/12)" for status messages
"""
sel = selector or network.AAMirrorSelector()
# AA fast download API (JSON response)
if link.startswith(f"{network.get_aa_base_url()}/dyn/api/fast_download.json"):
page = downloader.html_get_page(link, selector=sel, cancel_flag=cancel_flag)
return downloader.get_absolute_url(link, json.loads(page).get("download_url", ""))
html = downloader.html_get_page(link, selector=sel, cancel_flag=cancel_flag)
if not html:
return ""
soup = BeautifulSoup(html, "html.parser")
url = ""
if link.startswith(f"{AA_BASE_URL}/dyn/api/fast_download.json"):
page = downloader.html_get_page(link)
url = json.loads(page).get("download_url")
# Z-Library
if link.startswith("https://z-lib."):
dl = soup.find("a", href=True, class_="addDownloadedBook")
url = dl["href"] if dl else ""
# AA slow download / partner servers
elif "/slow_download/" in link:
url = _extract_slow_download_url(soup, link, title, cancel_flag, status_callback, sel, source_context)
# Libgen (GET button)
else:
html = downloader.html_get_page(link)
if html == "":
return ""
soup = BeautifulSoup(html, "html.parser")
if link.startswith("https://z-lib."):
download_link = soup.find_all("a", href=True, class_="addDownloadedBook")
if download_link:
url = download_link[0]["href"]
elif "/slow_download/" in link:
download_links = soup.find_all("a", href=True, string="📚 Download now")
if not download_links:
countdown = soup.find_all("span", class_="js-partner-countdown")
if countdown:
sleep_time = int(countdown[0].text)
logger.info(f"Waiting {sleep_time}s for {title}")
time.sleep(sleep_time)
url = _get_download_url(link, title)
else:
url = download_links[0]["href"]
else:
url = soup.find_all("a", string="GET")[0]["href"]
get_btn = soup.find("a", string="GET")
url = get_btn["href"] if get_btn else ""
return downloader.get_absolute_url(link, url)
def _extract_slow_download_url(soup: BeautifulSoup, link: str, title: str, cancel_flag: Optional[Event], status_callback, selector, source_context: Optional[str] = None) -> str:
"""Extract download URL from AA slow download pages."""
# Try "Download now" button variations
dl_link = soup.find("a", href=True, string="📚 Download now")
if not dl_link:
dl_link = soup.find("a", href=True, string=lambda s: s and "Download now" in s)
if dl_link:
return dl_link["href"]
# Try finding URL in gray background span (AA's copy URL format)
# The URL appears as plain text in <span class="bg-gray-200 ...">http://...</span>
for span in soup.find_all("span", class_=lambda c: c and "bg-gray-200" in c):
text = span.get_text(strip=True)
if text.startswith("http://") or text.startswith("https://"):
return text
# Try "copy this URL" pattern (legacy)
copy_text = soup.find(string=lambda s: s and "copy this url" in s.lower())
if copy_text and copy_text.parent:
parent = copy_text.parent
next_link = parent.find_next("a", href=True)
if next_link and next_link.get("href"):
return next_link["href"]
code_elem = parent.find_next("code")
if code_elem:
return code_elem.get_text(strip=True)
for sibling in parent.find_next_siblings():
text = sibling.get_text(strip=True) if hasattr(sibling, 'get_text') else str(sibling).strip()
if text.startswith("http"):
return text
# Check for countdown timer (waitlist)
countdown = soup.find("span", class_="js-partner-countdown")
if countdown:
# Cap countdown at 10 minutes to prevent malformed HTML from blocking indefinitely
MAX_COUNTDOWN_SECONDS = 600
try:
raw_countdown = int(countdown.text)
except (ValueError, TypeError):
logger.warning(f"Invalid countdown value '{countdown.text}', skipping wait")
raw_countdown = 0
sleep_time = min(raw_countdown, MAX_COUNTDOWN_SECONDS)
if raw_countdown > MAX_COUNTDOWN_SECONDS:
logger.warning(f"Countdown {raw_countdown}s exceeds max, capping at {MAX_COUNTDOWN_SECONDS}s")
logger.info(f"Waiting {sleep_time}s for {title}")
# Live countdown with status updates
remaining = sleep_time
while remaining > 0:
# Format countdown message with source context
if source_context:
wait_msg = f"{source_context} - Waiting {remaining}s"
else:
wait_msg = f"Waiting {remaining}s"
if status_callback:
status_callback("resolving", wait_msg)
# Wait 1 second (or until cancelled)
if cancel_flag and cancel_flag.wait(timeout=1):
logger.info(f"Cancelled wait for {title}")
return ""
remaining -= 1
# After countdown, update status and re-fetch
if status_callback and source_context:
status_callback("resolving", f"{source_context} - Fetching...")
return _get_download_url(link, title, cancel_flag, status_callback, selector, source_context)
# Debug fallback
link_texts = [a.get_text(strip=True)[:50] for a in soup.find_all("a", href=True)[:10]]
logger.warning(f"No download URL found. First 10 links: {link_texts}")
return ""
+856 -162
View File
File diff suppressed because it is too large Load Diff
+159
View File
@@ -0,0 +1,159 @@
from logger import setup_logger
from threading import Event
from typing import Optional, TYPE_CHECKING
import requests
import time
import random
if TYPE_CHECKING:
import network
class BypassCancelledException(Exception):
"""Raised when a bypass operation is cancelled."""
pass
try:
from env import EXT_BYPASSER_PATH, EXT_BYPASSER_TIMEOUT, EXT_BYPASSER_URL
except ImportError:
raise RuntimeError("Failed to import environment variables. Are you using an `extbp` image?")
logger = setup_logger(__name__)
# Connection timeout (seconds) - how long to wait for external bypasser to accept connection
CONNECT_TIMEOUT = 10
# Maximum read timeout cap (seconds) - hard limit regardless of EXT_BYPASSER_TIMEOUT
MAX_READ_TIMEOUT = 120
# Buffer added to bypasser's configured timeout (seconds) - accounts for processing overhead
READ_TIMEOUT_BUFFER = 15
# Retry settings for bypasser failures
MAX_RETRY = 5
BACKOFF_BASE = 1.0
BACKOFF_CAP = 10.0
def _fetch_via_bypasser(target_url: str) -> Optional[str]:
"""Make a single request to the external bypasser service.
Args:
target_url: The URL to fetch through the bypasser
Returns:
HTML content if successful, None otherwise
"""
if not EXT_BYPASSER_URL or not EXT_BYPASSER_PATH:
logger.error("External bypasser not configured. Check EXT_BYPASSER_URL and EXT_BYPASSER_PATH.")
return None
bypasser_endpoint = f"{EXT_BYPASSER_URL}{EXT_BYPASSER_PATH}"
headers = {"Content-Type": "application/json"}
payload = {
"cmd": "request.get",
"url": target_url,
"maxTimeout": EXT_BYPASSER_TIMEOUT
}
# Calculate read timeout: bypasser timeout (ms → s) + buffer, capped at max
read_timeout = min((EXT_BYPASSER_TIMEOUT / 1000) + READ_TIMEOUT_BUFFER, MAX_READ_TIMEOUT)
try:
response = requests.post(
bypasser_endpoint,
headers=headers,
json=payload,
timeout=(CONNECT_TIMEOUT, read_timeout)
)
response.raise_for_status()
result = response.json()
status = result.get('status', 'unknown')
message = result.get('message', '')
logger.debug(f"External bypasser response for '{target_url}': {status} - {message}")
# Check for error status (bypasser returns status="error" with solution=null on failure)
if status != 'ok':
logger.warning(f"External bypasser failed for '{target_url}': {status} - {message}")
return None
solution = result.get('solution')
if not solution:
logger.warning(f"External bypasser returned empty solution for '{target_url}'")
return None
html = solution.get('response', '')
if not html:
logger.warning(f"External bypasser returned empty response for '{target_url}'")
return None
return html
except requests.exceptions.Timeout:
logger.warning(f"External bypasser timed out for '{target_url}' (connect: {CONNECT_TIMEOUT}s, read: {read_timeout:.0f}s)")
return None
except requests.exceptions.RequestException as e:
logger.warning(f"External bypasser request failed for '{target_url}': {e}")
return None
except (KeyError, TypeError, ValueError) as e:
logger.warning(f"External bypasser returned malformed response for '{target_url}': {e}")
return None
def get_bypassed_page(url: str, selector: Optional["network.AAMirrorSelector"] = None, cancel_flag: Optional[Event] = None) -> Optional[str]:
"""Fetch HTML content from a URL using an external Cloudflare bypasser service.
Retries with exponential backoff and mirror/DNS rotation on failure.
Args:
url: Target URL to fetch
selector: Mirror selector for AA URL rewriting and rotation
cancel_flag: Optional threading Event to signal cancellation
Returns:
HTML content if successful, None otherwise
Raises:
BypassCancelledException: If cancel_flag is set during operation
"""
import network
sel = selector or network.AAMirrorSelector()
for attempt in range(1, MAX_RETRY + 1):
# Check for cancellation before each attempt
if cancel_flag and cancel_flag.is_set():
logger.info("External bypasser cancelled by user")
raise BypassCancelledException("Bypass cancelled")
attempt_url = sel.rewrite(url)
result = _fetch_via_bypasser(attempt_url)
if result:
return result
if attempt == MAX_RETRY:
break
# Check for cancellation before backoff wait
if cancel_flag and cancel_flag.is_set():
logger.info("External bypasser cancelled during retry")
raise BypassCancelledException("Bypass cancelled")
# Backoff with jitter before retry, checking cancellation during wait
delay = min(BACKOFF_CAP, BACKOFF_BASE * (2 ** (attempt - 1))) + random.random()
logger.info(f"External bypasser attempt {attempt}/{MAX_RETRY} failed, retrying in {delay:.1f}s")
# Check cancellation during delay (check every second)
for _ in range(int(delay)):
if cancel_flag and cancel_flag.is_set():
logger.info("External bypasser cancelled during backoff")
raise BypassCancelledException("Bypass cancelled")
time.sleep(1)
# Sleep remaining fraction
remaining = delay - int(delay)
if remaining > 0:
time.sleep(remaining)
# Rotate mirror/DNS for next attempt
new_base, action = sel.next_mirror_or_rotate_dns()
if action in ("mirror", "dns") and new_base:
logger.info(f"Rotated {action} for retry")
return None
+34 -11
View File
@@ -35,22 +35,34 @@ logger.info(f"CROSS_FILE_SYSTEM: {CROSS_FILE_SYSTEM}")
# Network settings
_custom_dns = env._CUSTOM_DNS.lower().strip()
_doh_server = ""
if _custom_dns == "google":
CUSTOM_DNS = ["8.8.8.8", "8.8.4.4", "2001:4860:4860::8888", "2001:4860:4860::8844"]
_doh_server = "https://dns.google/dns-query"
if _custom_dns == "auto" or _custom_dns == "":
# Auto mode - DNS provider rotation handled by network.py
# Starts with system DNS, switches to providers from DNS_PROVIDERS on failure
CUSTOM_DNS = []
_doh_server = ""
logger.info("CUSTOM_DNS: auto (starts with system DNS, rotates on failure)")
elif _custom_dns == "google":
CUSTOM_DNS = ["8.8.8.8", "8.8.4.4", "2001:4860:4860:0000:0000:0000:0000:8888", "2001:4860:4860:0000:0000:0000:0000:8844"]
_doh_server = "https://dns.google/resolve"
logger.info(f"CUSTOM_DNS: google {CUSTOM_DNS}")
elif _custom_dns == "quad9":
CUSTOM_DNS = ["9.9.9.9", "149.112.112.112", "2620:fe::fe", "26620:fe::9"]
CUSTOM_DNS = ["9.9.9.9", "149.112.112.112", "2620:00fe:0000:0000:0000:0000:0000:00fe", "2620:00fe:0000:0000:0000:0000:0000:0009"]
_doh_server = "https://dns.quad9.net/dns-query"
logger.info(f"CUSTOM_DNS: quad9 {CUSTOM_DNS}")
elif _custom_dns == "cloudflare":
CUSTOM_DNS = ["1.1.1.1", "1.0.0.1", "2606:4700:4700::1111", "2606:4700:4700::1001"]
CUSTOM_DNS = ["1.1.1.1", "1.0.0.1", "2606:4700:4700:0000:0000:0000:0000:1111", "2606:4700:4700:0000:0000:0000:0000:1001"]
_doh_server = "https://cloudflare-dns.com/dns-query"
logger.info(f"CUSTOM_DNS: cloudflare {CUSTOM_DNS}")
elif _custom_dns == "opendns":
CUSTOM_DNS = ["208.67.222.222", "208.67.220.220", "2620:119:35::35", "2620:119:53::53"]
CUSTOM_DNS = ["208.67.222.222", "208.67.220.220", "2620:0119:0035:0000:0000:0000:0000:0035", "2620:0119:0053:0000:0000:0000:0000:0053"]
_doh_server = "https://doh.opendns.com/dns-query"
logger.info(f"CUSTOM_DNS: opendns {CUSTOM_DNS}")
else:
# Custom DNS IPs provided by user
_custom_dns_ip = _custom_dns.split(",")
CUSTOM_DNS = [dns.strip() for dns in _custom_dns_ip if dns.replace(":", "").replace(".", "").strip().isdigit()]
logger.info(f"CUSTOM_DNS: {CUSTOM_DNS}")
logger.info(f"CUSTOM_DNS: custom {CUSTOM_DNS}")
DOH_SERVER = _doh_server
if env.USE_DOH:
DOH_SERVER = _doh_server
@@ -58,6 +70,15 @@ else:
DOH_SERVER = ""
logger.info(f"DOH_SERVER: {DOH_SERVER}")
# Warn about external bypasser DNS limitations
if env.USING_EXTERNAL_BYPASSER and env.USE_CF_BYPASS:
logger.warning(
"Using external bypasser (FlareSolverr). Note: FlareSolverr uses its own DNS resolution, "
"not this application's custom DNS settings. If you experience DNS-related blocks, "
"configure DNS at the Docker/system level for your FlareSolverr container, "
"or consider using the internal bypasser which integrates with the app's DNS system."
)
# Proxy settings
PROXIES = {}
if env.HTTP_PROXY:
@@ -93,7 +114,9 @@ if CUSTOM_SCRIPT:
CUSTOM_SCRIPT = ""
# Debugging settings
VIRTUAL_SCREEN_SIZE = (1024, 768)
RECORDING_DIR = env.LOG_DIR / "recording"
if env.DEBUG:
RECORDING_DIR.mkdir(parents=True, exist_ok=True)
if not env.USING_EXTERNAL_BYPASSER:
# Virtual display settings for debugging internal cloudflare bypasser
VIRTUAL_SCREEN_SIZE = (1024, 768)
RECORDING_DIR = env.LOG_DIR / "recording"
if env.DEBUG:
RECORDING_DIR.mkdir(parents=True, exist_ok=True)
-3
View File
@@ -9,9 +9,6 @@ services:
target: cwa-bd
environment:
DEBUG: true
APP_ENV: dev
USE_DOH: true
CUSTOM_DNS: cloudflare
volumes:
- /tmp/cwa-book-downloader:/tmp/cwa-book-downloader
- /tmp/cwa-book-downloader-log:/var/log/cwa-book-downloader
+34
View File
@@ -0,0 +1,34 @@
services:
calibre-web-automated-book-downloader-extbp-dev:
container_name: cwa-bd-extbp-dev
extends:
file: ./docker-compose.extbp.yml
service: calibre-web-automated-book-downloader-extbp
build:
context: .
dockerfile: Dockerfile
target: cwa-bd-extbp
environment:
DEBUG: true
USE_DOH: true
CUSTOM_DNS: cloudflare
USE_CF_BYPASS: true # Enable Cloudflare bypass (default: true)
# External Cloudflare Bypass environment variables
EXT_BYPASSER_URL: "http://flaresolverr:8191" # URL of the external Cloudflare resolver service (used FlareSolverr)
EXT_BYPASSER_PATH: "/v1" # Path for external Cloudflare resolver API (default: /v1)
EXT_BYPASSER_TIMEOUT: 60000 # Timeout for external Cloudflare resolver requests (default: 60000)
volumes:
#- /tmp/cwa-book-downloader:/tmp/cwa-book-downloader
#- /tmp/cwa-book-downloader-log:/var/log/cwa-book-downloader
- ./deploy/ingest:/cwa-book-ingest
- ./deploy/log:/var/log/cwa-book-downloader
- ./deploy/tmp:/tmp/cwa-book-downloader
flaresolverr: # External Cloudflare resolver service
image: ghcr.io/flaresolverr/flaresolverr:v3.3.22
container_name: flaresolverr
environment:
LOG_LEVEL: info
LOG_HTML: false
CAPTCHA_SOLVER: none
TZ: Europe/Rome
+27
View File
@@ -0,0 +1,27 @@
services:
calibre-web-automated-book-downloader-extbp:
image: ghcr.io/calibrain/calibre-web-automated-book-downloader-extbp:latest
environment:
FLASK_PORT: 8084
LOG_LEVEL: info
BOOK_LANGUAGE: en
USE_BOOK_TITLE: true
TZ: America/New_York
UID: 1000
GID: 100
# CWA_DB_PATH: /auth/app.db # Uncomment to enable authentication
# SESSION_COOKIE_SECURE: 'true' # Set to 'true' if accessing ONLY via HTTPS
# DEBUG: 'true' # Enable debug mode (debug button, verbose logging)
EXT_BYPASSER_URL: http://flaresolverr:8191
ports:
- 8084:8084
restart: unless-stopped
volumes:
# This is where the books will be downloaded to, usually it would be
# the same as whatever you gave in "calibre-web-automated"
- /tmp/data/calibre-web/ingest:/cwa-book-ingest
# This is the location of CWA's app.db, which contains authentication
# details. Uncomment to enable authentication (also uncomment CWA_DB_PATH above)
#- /cwa/config/path/app.db:/auth/app.db:ro
flaresolverr:
image: ghcr.io/flaresolverr/flaresolverr:latest
-1
View File
@@ -9,7 +9,6 @@ services:
target: cwa-bd-tor
environment:
DEBUG: true
APP_ENV: dev
volumes:
- /tmp/cwa-book-downloader:/tmp/cwa-book-downloader
- /tmp/cwa-book-downloader-log:/var/log/cwa-book-downloader
+8 -3
View File
@@ -8,7 +8,9 @@ services:
USE_BOOK_TITLE: true
TZ: America/New_York
USING_TOR: true
APP_ENV: prod
# CWA_DB_PATH: /auth/app.db # Uncomment to enable authentication
# SESSION_COOKIE_SECURE: 'true' # Set to 'true' if accessing ONLY via HTTPS
# DEBUG: 'true' # Enable debug mode (debug button, verbose logging)
cap_add:
- NET_ADMIN
- NET_RAW
@@ -16,6 +18,9 @@ services:
- 8084:8084
restart: unless-stopped
volumes:
# This is where the books will be downloaded to, usually it would be
# the same as whatever you gave in "calibre-web-automated"
# This is where the books will be downloaded to, usually it would be
# the same as whatever you gave in "calibre-web-automated"
- /tmp/data/calibre-web/ingest:/cwa-book-ingest
# This is the location of CWA's app.db, which contains authentication
# details. Uncomment to enable authentication (also uncomment CWA_DB_PATH above)
#- /cwa/config/path/app.db:/auth/app.db:ro
+11 -3
View File
@@ -1,6 +1,9 @@
services:
calibre-web-automated-book-downloader:
image: ghcr.io/calibrain/calibre-web-automated-book-downloader:latest
# Uncomment to build the image from the Dockerfile for local testing changes.
# Remember to comment out the image line above.
#build: .
container_name: calibre-web-automated-book-downloader
environment:
FLASK_PORT: 8084
@@ -8,10 +11,15 @@ services:
BOOK_LANGUAGE: en
USE_BOOK_TITLE: true
TZ: America/New_York
APP_ENV: prod
UID: 1000
GID: 100
CWA_DB_PATH: /auth/app.db
# CWA_DB_PATH: /auth/app.db # Uncomment to enable authentication (also uncomment volume below)
# CALIBRE_WEB_URL: http://localhost:8080 # Uncomment and add your custom library URL to enable "Go To Library" button in the Web UI
# SESSION_COOKIE_SECURE: 'true' # Set to 'true' if accessing ONLY via HTTPS
# DEBUG: 'true' # Enable debug mode (debug button, verbose logging)
# Queue management settings
MAX_CONCURRENT_DOWNLOADS: 3
DOWNLOAD_PROGRESS_UPDATE_INTERVAL: 5
ports:
- 8084:8084
restart: unless-stopped
@@ -20,5 +28,5 @@ services:
# the same as whatever you gave in "calibre-web-automated"
- /tmp/data/calibre-web/ingest:/cwa-book-ingest
# This is the location of CWA's app.db, which contains authentication
# details
# details. Uncomment to enable authentication (also uncomment CWA_DB_PATH above)
#- /cwa/config/path/app.db:/auth/app.db:ro
+340 -114
View File
@@ -1,131 +1,357 @@
"""Network operations manager for the book downloader application."""
import network
network.init()
import requests
import random
import time
from io import BytesIO
from typing import Optional
from threading import Event
from typing import Callable, Optional
from urllib.parse import urlparse
import requests
from tqdm import tqdm
from logger import setup_logger
import network
from config import PROXIES
from env import MAX_RETRY, DEFAULT_SLEEP, USE_CF_BYPASS
from env import DEFAULT_SLEEP, MAX_RETRY, USE_CF_BYPASS, USING_EXTERNAL_BYPASSER
from logger import setup_logger
# Import bypasser if enabled
if USE_CF_BYPASS:
import cloudflare_bypasser
if USING_EXTERNAL_BYPASSER:
from cloudflare_bypasser_external import get_bypassed_page
# External bypasser doesn't share cookies
get_cf_cookies_for_domain = lambda domain: {}
else:
from cloudflare_bypasser import get_bypassed_page, get_cf_cookies_for_domain
logger = setup_logger(__name__)
# Network settings
REQUEST_TIMEOUT = (5, 10) # (connect, read)
MAX_DOWNLOAD_RETRIES = 2
MAX_RESUME_ATTEMPTS = 3
RETRYABLE_CODES = (429, 500, 502, 503, 504)
CONNECTION_ERRORS = (requests.exceptions.ConnectionError, requests.exceptions.Timeout,
requests.exceptions.SSLError, requests.exceptions.ChunkedEncodingError)
DOWNLOAD_HEADERS = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'Accept-Language': 'en-US,en;q=0.5',
}
def html_get_page(url: str, retry: int = MAX_RETRY, use_bypasser: bool = False) -> str:
"""Fetch HTML content from a URL with retry mechanism.
Args:
url: Target URL
retry: Number of retry attempts
skip_404: Whether to skip 404 errors
Returns:
str: HTML content if successful, None otherwise
"""
response = None
def parse_size_string(size: str) -> Optional[float]:
"""Parse a human-readable size string (e.g., '10.5 MB') into bytes."""
if not size:
return None
try:
logger.debug(f"html_get_page: {url}, retry: {retry}, use_bypasser: {use_bypasser}")
if use_bypasser and USE_CF_BYPASS:
logger.info(f"GET Using Cloudflare Bypasser for: {url}")
response_html = cloudflare_bypasser.get(url)
logger.debug(f"Cloudflare Bypasser response length: {len(response_html)}")
if response_html.strip() != "":
return response_html
else:
raise requests.exceptions.RequestException("Failed to bypass Cloudflare")
else:
logger.info(f"GET: {url}")
response = requests.get(url, proxies=PROXIES)
response.raise_for_status()
logger.debug(f"Success getting: {url}")
time.sleep(1)
return str(response.text)
except Exception as e:
if retry == 0:
logger.error_trace(f"Failed to fetch page: {url}, error: {e}")
return ""
if use_bypasser and USE_CF_BYPASS:
logger.warning(f"Exception while using cloudflare bypass for URL: {url}")
logger.warning(f"Exception: {e}")
logger.warning(f"Response: {response}")
elif response is not None and response.status_code == 404:
logger.warning(f"404 error for URL: {url}")
return ""
elif response is not None and response.status_code == 403:
logger.warning(f"403 detected for URL: {url}. Should retry using cloudflare bypass.")
return html_get_page(url, retry - 1, True)
sleep_time = DEFAULT_SLEEP * (MAX_RETRY - retry + 1)
logger.warning(
f"Retrying GET {url} in {sleep_time} seconds due to error: {e}"
)
time.sleep(sleep_time)
return html_get_page(url, retry - 1, use_bypasser)
def download_url(link: str, size: str = "") -> Optional[BytesIO]:
"""Download content from URL into a BytesIO buffer.
Args:
link: URL to download from
Returns:
BytesIO: Buffer containing downloaded content if successful
"""
try:
logger.info(f"Downloading from: {link}")
response = requests.get(link, stream=True, proxies=PROXIES)
response.raise_for_status()
total_size : float = 0.0
try:
# we assume size is in MB
total_size = float(size.strip().replace(" ", "").replace(",", ".").upper()[:-2].strip()) * 1024 * 1024
except:
total_size = float(response.headers.get('content-length', 0))
buffer = BytesIO()
# Initialize the progress bar with your guess
pbar = tqdm(total=total_size, unit='B', unit_scale=True, desc='Downloading')
for chunk in response.iter_content(chunk_size=1000):
buffer.write(chunk)
pbar.update(len(chunk))
pbar.close()
if buffer.tell() * 0.1 < total_size * 0.9:
# Check the content of the buffer if its HTML or binary
if response.headers.get('content-type', '').startswith('text/html'):
logger.warn(f"Failed to download content for {link}. Found HTML content instead.")
return None
return buffer
except requests.exceptions.RequestException as e:
logger.error_trace(f"Failed to download from {link}: {e}")
normalized = size.strip().replace(" ", "").replace(",", ".").upper()
multipliers = {"GB": 1024**3, "MB": 1024**2, "KB": 1024}
for suffix, mult in multipliers.items():
if normalized.endswith(suffix):
return float(normalized[:-2]) * mult
return float(normalized)
except (ValueError, IndexError):
return None
def get_absolute_url(base_url: str, url: str) -> str:
"""Get absolute URL from relative URL and base URL.
def _backoff_delay(attempt: int, base: float = 0.25, cap: float = 3.0) -> float:
"""Exponential backoff with jitter."""
return min(cap, base * (2 ** (attempt - 1))) + random.random() * base
def _get_status_code(e: Exception) -> Optional[int]:
"""Extract HTTP status code from an exception, or None if not applicable."""
if isinstance(e, requests.exceptions.HTTPError) and e.response is not None:
return e.response.status_code
return None
def _is_retryable_error(e: Exception) -> bool:
"""Check if error is retryable (connection error or retryable HTTP status)."""
if isinstance(e, CONNECTION_ERRORS):
return True
status = _get_status_code(e)
return status in RETRYABLE_CODES if status else False
def _try_rotation(original_url: str, current_url: str, selector: network.AAMirrorSelector) -> Optional[str]:
"""Try mirror/DNS rotation. Returns new URL or None."""
if current_url.startswith(network.get_aa_base_url()):
new_base, action = selector.next_mirror_or_rotate_dns()
if action in ("mirror", "dns") and new_base:
new_url = selector.rewrite(original_url)
logger.info(f"[{action}] switching to: {new_url}")
return new_url
elif network.should_rotate_dns_for_url(current_url) and network.rotate_dns_provider():
logger.info(f"[dns-rotate] retrying: {original_url}")
return original_url
return None
def html_get_page(
url: str,
retry: int = MAX_RETRY,
use_bypasser: bool = False,
selector: Optional[network.AAMirrorSelector] = None,
cancel_flag: Optional[Event] = None,
) -> str:
"""Fetch HTML content from a URL with retry mechanism."""
selector = selector or network.AAMirrorSelector()
original_url = url
current_url = selector.rewrite(original_url)
use_bypasser_now = use_bypasser
for attempt in range(1, retry + 1):
# Check for cancellation before each attempt
if cancel_flag and cancel_flag.is_set():
logger.info(f"html_get_page cancelled before attempt {attempt}")
return ""
try:
if use_bypasser_now and USE_CF_BYPASS:
logger.info(f"GET (bypasser): {current_url}")
try:
result = get_bypassed_page(current_url, selector, cancel_flag)
return result or ""
except Exception as e:
logger.warning(f"Bypasser error: {type(e).__name__}: {e}")
return ""
logger.info(f"GET: {current_url}")
# Try with CF cookies if available (from previous bypass)
cookies = {}
if USE_CF_BYPASS:
parsed = urlparse(current_url)
cookies = get_cf_cookies_for_domain(parsed.hostname or "")
response = requests.get(current_url, proxies=PROXIES, timeout=REQUEST_TIMEOUT, cookies=cookies)
response.raise_for_status()
time.sleep(1)
return response.text
except Exception as e:
status = _get_status_code(e)
# 403 = Cloudflare/DDoS-Guard protection
if status == 403:
if USE_CF_BYPASS and not use_bypasser_now:
# Before switching to bypasser, check if cookies have become available
# (another concurrent download may have completed bypass and extracted cookies)
parsed = urlparse(current_url)
fresh_cookies = get_cf_cookies_for_domain(parsed.hostname or "")
if fresh_cookies and not cookies:
# Cookies are now available - retry with cookies before using bypasser
logger.debug(f"403 but cookies now available - retrying with cookies: {current_url}")
continue
logger.info(f"403 detected; switching to bypasser: {current_url}")
use_bypasser_now = True
continue
logger.warning(f"403 error, giving up: {current_url}")
return ""
# 404 = Not found
if status == 404:
logger.warning(f"404 error: {current_url}")
return ""
# Try mirror/DNS rotation on retryable errors
if _is_retryable_error(e):
new_url = _try_rotation(original_url, current_url, selector)
if new_url:
current_url = new_url
continue
# Retry with backoff
if attempt < retry:
logger.warning(f"Retry {attempt}/{retry} for {current_url}: {type(e).__name__}: {e}")
time.sleep(_backoff_delay(attempt))
else:
logger.error(f"Giving up after {retry} attempts: {current_url}")
return ""
def download_url(
link: str,
size: str = "",
progress_callback: Optional[Callable[[float], None]] = None,
cancel_flag: Optional[Event] = None,
_selector: Optional[network.AAMirrorSelector] = None,
status_callback: Optional[Callable[[str, Optional[str]], None]] = None,
referer: Optional[str] = None,
) -> Optional[BytesIO]:
"""Download content from URL with automatic retry and resume support."""
selector = _selector or network.AAMirrorSelector()
current_url = selector.rewrite(link)
# Build headers with optional referer
headers = DOWNLOAD_HEADERS.copy()
if referer:
headers['Referer'] = referer
total_size = parse_size_string(size) or 0
attempt = 0
while attempt < MAX_DOWNLOAD_RETRIES:
if cancel_flag and cancel_flag.is_set():
return None
buffer = BytesIO()
bytes_downloaded = 0
try:
if attempt > 0 and status_callback:
status_callback("resolving", f"Connecting (Attempt {attempt + 1}/{MAX_DOWNLOAD_RETRIES})")
logger.info(f"Downloading: {current_url} (attempt {attempt + 1}/{MAX_DOWNLOAD_RETRIES})")
# Try with CF cookies if available
cookies = {}
if USE_CF_BYPASS:
parsed = urlparse(current_url)
cookies = get_cf_cookies_for_domain(parsed.hostname or "")
if cookies:
logger.debug(f"Using {len(cookies)} cookies for {parsed.hostname}: {list(cookies.keys())}")
response = requests.get(current_url, stream=True, proxies=PROXIES, timeout=REQUEST_TIMEOUT, cookies=cookies, headers=headers)
response.raise_for_status()
if status_callback:
status_callback("downloading", "")
total_size = total_size or float(response.headers.get('content-length', 0))
pbar = tqdm(total=total_size, unit='B', unit_scale=True, desc='Downloading')
for chunk in response.iter_content(chunk_size=8192):
if chunk:
buffer.write(chunk)
bytes_downloaded += len(chunk)
pbar.update(len(chunk))
if progress_callback and total_size > 0:
progress_callback(bytes_downloaded * 100.0 / total_size)
if cancel_flag and cancel_flag.is_set():
pbar.close()
return None
pbar.close()
# Validate - check we didn't get HTML instead of file
if total_size > 0 and bytes_downloaded < total_size * 0.9:
if response.headers.get('content-type', '').startswith('text/html'):
logger.warning(f"Received HTML instead of file: {current_url}")
return None
logger.debug(f"Download completed: {bytes_downloaded} bytes")
return buffer
except requests.exceptions.RequestException as e:
status = _get_status_code(e)
retryable = _is_retryable_error(e)
# Non-retryable errors
if status in (403, 404):
logger.warning(f"Download failed ({status}): {current_url}")
return None
# Rate limited - skip to next source immediately
# (waiting doesn't help with concurrent downloads hitting the same server)
if status == 429:
logger.info(f"Rate limited (429) - trying next source")
if status_callback:
status_callback("resolving", "Server busy, trying next...")
return None
# Timeout - don't retry, server likely overloaded
if isinstance(e, requests.exceptions.Timeout):
logger.warning(f"Timeout: {current_url} - skipping to next source")
if status_callback:
status_callback("resolving", "Server timed out, trying next...")
return None
# Try to resume if we got some data
if bytes_downloaded > 0 and retryable:
resumed = _try_resume(current_url, buffer, bytes_downloaded, total_size, progress_callback, cancel_flag, headers)
if resumed:
return resumed
# Try mirror/DNS rotation if nothing downloaded yet
if bytes_downloaded == 0 and retryable:
new_url = _try_rotation(link, current_url, selector)
if new_url:
current_url = new_url
attempt += 1
continue
logger.warning(f"Download error: {type(e).__name__}: {e}")
if attempt < MAX_DOWNLOAD_RETRIES - 1:
time.sleep(_backoff_delay(attempt + 1))
attempt += 1
logger.error(f"Download failed after {MAX_DOWNLOAD_RETRIES} attempts: {link}")
return None
def _try_resume(
url: str,
buffer: BytesIO,
start_byte: int,
total_size: float,
progress_callback: Optional[Callable[[float], None]],
cancel_flag: Optional[Event],
base_headers: Optional[dict] = None,
) -> Optional[BytesIO]:
"""Try to resume an interrupted download."""
for attempt in range(MAX_RESUME_ATTEMPTS):
logger.info(f"Resuming from {start_byte} bytes (attempt {attempt + 1}/{MAX_RESUME_ATTEMPTS})")
time.sleep(_backoff_delay(attempt + 1, base=0.5, cap=5.0))
try:
# Try with CF cookies if available
cookies = {}
if USE_CF_BYPASS:
parsed = urlparse(url)
cookies = get_cf_cookies_for_domain(parsed.hostname or "")
resume_headers = {**(base_headers or DOWNLOAD_HEADERS), 'Range': f'bytes={start_byte}-'}
response = requests.get(
url, stream=True, proxies=PROXIES, timeout=REQUEST_TIMEOUT,
headers=resume_headers, cookies=cookies
)
# Check resume support
if response.status_code == 200: # Server doesn't support resume
logger.info("Server doesn't support resume")
return None
if response.status_code == 416: # Range not satisfiable
logger.warning("Range not satisfiable")
return None
if response.status_code != 206:
response.raise_for_status()
pbar = tqdm(total=total_size, initial=start_byte, unit='B', unit_scale=True, desc='Resuming')
for chunk in response.iter_content(chunk_size=8192):
if chunk:
buffer.write(chunk)
start_byte += len(chunk)
pbar.update(len(chunk))
if progress_callback and total_size > 0:
progress_callback(start_byte * 100.0 / total_size)
if cancel_flag and cancel_flag.is_set():
pbar.close()
return None
pbar.close()
logger.info(f"Resume completed: {start_byte} bytes")
return buffer
except requests.exceptions.RequestException as e:
logger.debug(f"Resume attempt {attempt + 1} failed: {e}")
Args:
base_url: Base URL
url: Relative URL
"""
if url.strip() == "":
return ""
if url.strip("#") == "":
return ""
if url.startswith("http"):
return url
parsed_url = urlparse(url)
parsed_base = urlparse(base_url)
if parsed_url.netloc == "" or parsed_url.scheme == "":
parsed_url = parsed_url._replace(netloc=parsed_base.netloc, scheme=parsed_base.scheme)
return parsed_url.geturl()
logger.warning(f"Resume failed after {MAX_RESUME_ATTEMPTS} attempts")
return None
def get_absolute_url(base_url: str, url: str) -> str:
"""Convert a relative URL to absolute using the base URL."""
url = url.strip()
if not url or url == "#" or url.startswith("http"):
return url if url.startswith("http") else ""
parsed = urlparse(url)
base = urlparse(base_url)
if not parsed.netloc or not parsed.scheme:
parsed = parsed._replace(netloc=base.netloc, scheme=base.scheme)
return parsed.geturl()
+9 -10
View File
@@ -20,6 +20,7 @@ set -e
# Print build version
echo "Build version: $BUILD_VERSION"
echo "Release version: $RELEASE_VERSION"
# Configure timezone
if [ "$TZ" ]; then
@@ -104,16 +105,14 @@ change_ownership /tmp/cwa-book-downloader
# Test write to all folders
make_writable /cwa-book-ingest
# Set the command to run based on the environment
is_prod=$(echo "$APP_ENV" | tr '[:upper:]' '[:lower:]')
if [ "$is_prod" = "prod" ]; then
command="gunicorn -t 300 -b ${FLASK_HOST:-0.0.0.0}:${FLASK_PORT:-8084} app:app"
else
command="python3 app.py"
fi
# Always run Gunicorn (even when DEBUG=true) to ensure Socket.IO WebSocket
# upgrades work reliably on customer machines.
# Map app LOG_LEVEL (often DEBUG/INFO/...) to gunicorn's --log-level (lowercase).
gunicorn_loglevel=$([ "$DEBUG" = "true" ] && echo debug || echo "${LOG_LEVEL:-info}" | tr '[:upper:]' '[:lower:]')
command="gunicorn --log-level ${gunicorn_loglevel} --access-logfile - --error-logfile - --worker-class geventwebsocket.gunicorn.workers.GeventWebSocketWorker --workers 1 -t 300 -b ${FLASK_HOST:-0.0.0.0}:${FLASK_PORT:-8084} app:app"
# IF DEBUG
if [ "$DEBUG" = "true" ]; then
# If DEBUG and not using an external bypass
if [ "$DEBUG" = "true" ] && [ "$USING_EXTERNAL_BYPASSER" != "true" ]; then
set +e
set -x
echo "vvvvvvvvvvvv DEBUG MODE vvvvvvvvvvvv"
@@ -174,7 +173,7 @@ sum=$(python3 -c "print(sum(int(l.strip()) for l in open('/tmp/test.cwa-bd').rea
[ "$sum" == 11250075000 ] && echo "Success: /tmp is writable" || (echo "Failure: /tmp is not writable" && exit 1)
rm /tmp/test.cwa-bd
echo "Running command: '$command' as '$USERNAME' in '$APP_ENV' mode"
echo "Running command: '$command' as '$USERNAME' (debug=$is_debug)"
# Stop logging
exec 1>&3 2>&4
+50 -3
View File
@@ -4,12 +4,36 @@ from pathlib import Path
def string_to_bool(s: str) -> bool:
return s.lower() in ["true", "yes", "1", "y"]
# Authentication and session settings
SESSION_COOKIE_SECURE_ENV = os.getenv("SESSION_COOKIE_SECURE", "false")
CWA_DB = os.getenv("CWA_DB_PATH")
CWA_DB_PATH = Path(CWA_DB) if CWA_DB else None
LOG_ROOT = Path(os.getenv("LOG_ROOT", "/var/log/"))
LOG_DIR = LOG_ROOT / "cwa-book-downloader"
TMP_DIR = Path(os.getenv("TMP_DIR", "/tmp/cwa-book-downloader"))
INGEST_DIR = Path(os.getenv("INGEST_DIR", "/cwa-book-ingest"))
INGEST_DIR_BOOK_FICTION = os.getenv("INGEST_DIR_BOOK_FICTION", "")
INGEST_DIR_BOOK_NON_FICTION = os.getenv("INGEST_DIR_BOOK_NON_FICTION", "")
INGEST_DIR_BOOK_UNKNOWN = os.getenv("INGEST_DIR_BOOK_UNKNOWN", "")
INGEST_DIR_MAGAZINE = os.getenv("INGEST_DIR_MAGAZINE", "")
INGEST_DIR_COMIC_BOOK = os.getenv("INGEST_DIR_COMIC_BOOK", "")
INGEST_DIR_AUDIOBOOK = os.getenv("INGEST_DIR_AUDIOBOOK", "")
INGEST_DIR_STANDARDS_DOCUMENT = os.getenv("INGEST_DIR_STANDARDS_DOCUMENT", "")
INGEST_DIR_MUSICAL_SCORE = os.getenv("INGEST_DIR_MUSICAL_SCORE", "")
INGEST_DIR_OTHER = os.getenv("INGEST_DIR_OTHER", "")
DOWNLOAD_PATHS = {
"book (fiction)": Path(INGEST_DIR_BOOK_FICTION) if INGEST_DIR_BOOK_FICTION else INGEST_DIR,
"book (non-fiction)": Path(INGEST_DIR_BOOK_NON_FICTION) if INGEST_DIR_BOOK_NON_FICTION else INGEST_DIR,
"book (unknown)": Path(INGEST_DIR_BOOK_UNKNOWN) if INGEST_DIR_BOOK_UNKNOWN else INGEST_DIR,
"magazine": Path(INGEST_DIR_MAGAZINE) if INGEST_DIR_MAGAZINE else INGEST_DIR,
"comic book": Path(INGEST_DIR_COMIC_BOOK) if INGEST_DIR_COMIC_BOOK else INGEST_DIR,
"audiobook": Path(INGEST_DIR_AUDIOBOOK) if INGEST_DIR_AUDIOBOOK else INGEST_DIR,
"standards document": Path(INGEST_DIR_STANDARDS_DOCUMENT) if INGEST_DIR_STANDARDS_DOCUMENT else INGEST_DIR,
"musical score": Path(INGEST_DIR_MUSICAL_SCORE) if INGEST_DIR_MUSICAL_SCORE else INGEST_DIR,
"other": Path(INGEST_DIR_OTHER) if INGEST_DIR_OTHER else INGEST_DIR,
}
STATUS_TIMEOUT = int(os.getenv("STATUS_TIMEOUT", "3600"))
USE_BOOK_TITLE = string_to_bool(os.getenv("USE_BOOK_TITLE", "false"))
MAX_RETRY = int(os.getenv("MAX_RETRY", "10"))
@@ -25,7 +49,18 @@ _BOOK_LANGUAGE = os.getenv("BOOK_LANGUAGE", "en").lower()
_CUSTOM_SCRIPT = os.getenv("CUSTOM_SCRIPT", "").strip()
FLASK_HOST = os.getenv("FLASK_HOST", "0.0.0.0")
FLASK_PORT = int(os.getenv("FLASK_PORT", "8084"))
DEBUG = string_to_bool(os.getenv("DEBUG", "False"))
DEBUG = string_to_bool(os.getenv("DEBUG", "false"))
# Debug: skip specific download sources for testing fallback chains
# Comma-separated values: aa-fast, aa-slow-nowait, aa-slow-wait, libgen, zlib, welib
_DEBUG_SKIP_SOURCES_RAW = os.getenv("DEBUG_SKIP_SOURCES", "").strip().lower()
DEBUG_SKIP_SOURCES = set(s.strip() for s in _DEBUG_SKIP_SOURCES_RAW.split(",") if s.strip())
PRIORITIZE_WELIB = string_to_bool(os.getenv("PRIORITIZE_WELIB", "false"))
ALLOW_USE_WELIB = string_to_bool(os.getenv("ALLOW_USE_WELIB", "true"))
# Version information from Docker build
BUILD_VERSION = os.getenv("BUILD_VERSION", "N/A")
RELEASE_VERSION = os.getenv("RELEASE_VERSION", "N/A")
# If debug is true, we want to log everything
if DEBUG:
LOG_LEVEL = "DEBUG"
@@ -33,14 +68,23 @@ else:
LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO").upper()
ENABLE_LOGGING = string_to_bool(os.getenv("ENABLE_LOGGING", "true"))
MAIN_LOOP_SLEEP_TIME = int(os.getenv("MAIN_LOOP_SLEEP_TIME", "5"))
MAX_CONCURRENT_DOWNLOADS = int(os.getenv("MAX_CONCURRENT_DOWNLOADS", "3"))
DOWNLOAD_PROGRESS_UPDATE_INTERVAL = int(os.getenv("DOWNLOAD_PROGRESS_UPDATE_INTERVAL", "1"))
DOCKERMODE = string_to_bool(os.getenv("DOCKERMODE", "false"))
_CUSTOM_DNS = os.getenv("CUSTOM_DNS", "").strip()
_CUSTOM_DNS = os.getenv("CUSTOM_DNS", "auto").strip()
USE_DOH = string_to_bool(os.getenv("USE_DOH", "false"))
BYPASS_RELEASE_INACTIVE_MIN = int(os.getenv("BYPASS_RELEASE_INACTIVE_MIN", "5"))
APP_ENV = os.getenv("APP_ENV", "prod").lower()
BYPASS_WARMUP_ON_CONNECT = string_to_bool(os.getenv("BYPASS_WARMUP_ON_CONNECT", "true"))
# Logging settings
LOG_FILE = LOG_DIR / "cwa-book-downloader.log"
USING_EXTERNAL_BYPASSER = string_to_bool(os.getenv("USING_EXTERNAL_BYPASSER", "false"))
if USING_EXTERNAL_BYPASSER:
EXT_BYPASSER_URL = os.getenv("EXT_BYPASSER_URL", "http://flaresolverr:8191").strip()
EXT_BYPASSER_PATH = os.getenv("EXT_BYPASSER_PATH", "/v1").strip()
EXT_BYPASSER_TIMEOUT = int(os.getenv("EXT_BYPASSER_TIMEOUT", "60000"))
USING_TOR = string_to_bool(os.getenv("USING_TOR", "false"))
# If using Tor, we don't need to set custom DNS, use DOH, or proxy
if USING_TOR:
@@ -48,4 +92,7 @@ if USING_TOR:
USE_DOH = False
HTTP_PROXY = ""
HTTPS_PROXY = ""
# Calibre-Web URL for navigation button
CALIBRE_WEB_URL = os.getenv("CALIBRE_WEB_URL", "").strip()
+9 -9
View File
@@ -3,7 +3,7 @@
# Set up log paths
LOG_ROOT=${LOG_ROOT:-"/var/log"}
LOG_DIR="$LOG_ROOT/cwa-book-downloader"
OUTPUT_FILE_NAME="cwa-book-downloader-debug_BUILD-${BUILD_VERSION:-local}_$(date +%Y%m%d-%H%M%S)"
OUTPUT_FILE_NAME="cwa-book-downloader-debug_BUILD-${BUILD_VERSION:-local}_RELEASE-${RELEASE_VERSION:-NA}_$(date +%Y%m%d-%H%M%S)"
OUTPUT_FILE="/tmp/$OUTPUT_FILE_NAME.zip"
# Create LOG_DIR if it doesn't exist
@@ -124,17 +124,17 @@ fi
# Add environment variables (redacting sensitive info)
env | grep -v -E "(AA_DONATOR_KEY)" | sort > "$LOG_DIR/environment.txt"
echo "--- HTTPBin ---" > $LOG_DIR/network_info.txt
pyrequests https://httpbin.org/get >> $LOG_DIR/network_info.txt
ehco ""
echo "--- HTTPBin ---" >> $LOG_DIR/network_info.txt
curl -s https://httpbin.org/get >> $LOG_DIR/network_info.txt
echo "" >> $LOG_DIR/network_info.txt
echo "--- HowsMySSL ---" >> $LOG_DIR/network_info.txt
pyrequests https://www.howsmyssl.com/a/check >> $LOG_DIR/network_info.txt
ehco ""
curl -s https://www.howsmyssl.com/a/check >> $LOG_DIR/network_info.txt
echo "" >> $LOG_DIR/network_info.txt
echo "--- IPInfo ---" >> $LOG_DIR/network_info.txt
pyrequests https://ipinfo.io >> $LOG_DIR/network_info.txt
ehco ""
curl -s https://ipinfo.io >> $LOG_DIR/network_info.txt
echo "" >> $LOG_DIR/network_info.txt
echo "--- Cloudflare Trace ---" >> $LOG_DIR/network_info.txt
pyrequests https://1.1.1.1/cdn-cgi/trace >> $LOG_DIR/network_info.txt
curl -s https://1.1.1.1/cdn-cgi/trace >> $LOG_DIR/network_info.txt
# Create the zip file directly from LOG_DIR
ln -s "$LOG_DIR" /tmp/$OUTPUT_FILE_NAME
+13 -7
View File
@@ -13,22 +13,28 @@ class CustomLogger(logging.Logger):
def error_trace(self, msg: Any, *args: Any, **kwargs: Any) -> None:
"""Log an error message with full stack trace."""
self.log_resource_usage()
kwargs.pop('exc_info', None)
self.error(msg, *args, exc_info=True, **kwargs)
def warning_trace(self, msg: Any, *args: Any, **kwargs: Any) -> None:
"""Log a warning message with full stack trace."""
self.log_resource_usage()
kwargs.pop('exc_info', None)
self.warning(msg, *args, exc_info=True, **kwargs)
def info_trace(self, msg: Any, *args: Any, **kwargs: Any) -> None:
"""Log an info message with full stack trace."""
self.log_resource_usage()
self.info(msg, *args, exc_info=True, **kwargs)
"""Log an info message (stack trace only if exception active)."""
kwargs.pop('exc_info', None)
# Only include exc_info if there's actually an exception
has_exception = sys.exc_info()[0] is not None
self.info(msg, *args, exc_info=has_exception, **kwargs)
def debug_trace(self, msg: Any, *args: Any, **kwargs: Any) -> None:
"""Log a debug message with full stack trace."""
self.log_resource_usage()
self.debug(msg, *args, exc_info=True, **kwargs)
"""Log a debug message (stack trace only if exception active)."""
kwargs.pop('exc_info', None)
# Only include exc_info if there's actually an exception
has_exception = sys.exc_info()[0] is not None
self.debug(msg, *args, exc_info=has_exception, **kwargs)
def log_resource_usage(self):
import psutil
+315 -14
View File
@@ -1,20 +1,39 @@
"""Data structures and models used across the application."""
from dataclasses import dataclass, field
from typing import Dict, List, Optional
from typing import Dict, List, Optional, Tuple
from enum import Enum
from datetime import datetime, timedelta
from threading import Lock
from threading import Lock, Event
from pathlib import Path
import queue
import re
import time
from env import INGEST_DIR, STATUS_TIMEOUT
class QueueStatus(str, Enum):
"""Enum for possible book queue statuses."""
QUEUED = "queued"
RESOLVING = "resolving"
DOWNLOADING = "downloading"
COMPLETE = "complete"
AVAILABLE = "available"
ERROR = "error"
DONE = "done"
CANCELLED = "cancelled"
@dataclass
class QueueItem:
"""Queue item with priority and metadata."""
book_id: str
priority: int
added_time: float
def __lt__(self, other):
"""Compare items for priority queue (lower priority number = higher precedence)."""
if self.priority != other.priority:
return self.priority < other.priority
return self.added_time < other.added_time
@dataclass
class BookInfo:
@@ -26,33 +45,113 @@ class BookInfo:
publisher: Optional[str] = None
year: Optional[str] = None
language: Optional[str] = None
content: Optional[str] = None
format: Optional[str] = None
size: Optional[str] = None
info: Optional[Dict[str, List[str]]] = None
description: Optional[str] = None
download_urls: List[str] = field(default_factory=list)
download_path: Optional[str] = None
priority: int = 0
progress: Optional[float] = None
status_message: Optional[str] = None # Detailed status message for UI display
added_time: Optional[float] = None # Timestamp when added to queue
def get_filename(self, fallback_url: Optional[str] = None) -> str:
"""Build sanitized filename: 'Author - Title (Year).format'
Resolves format from self.format, download_urls, or fallback_url.
Args:
fallback_url: URL to extract format from if not already known
Returns:
Sanitized filename safe for filesystem use
"""
# Resolve format if needed
if not self.format:
for url in (self.download_urls[0] if self.download_urls else None, fallback_url):
if url:
ext = url.split(".")[-1].lower()
if ext and len(ext) <= 5 and ext.isalnum():
self.format = ext
break
# Build filename
parts = []
if self.author:
parts.append(self.author)
parts.append(" - ")
parts.append(self.title)
if self.year:
parts.append(f" ({self.year})")
filename = "".join(parts)
filename = re.sub(r'[\\/:*?"<>|]', '_', filename.strip())[:245]
if self.format:
filename = f"{filename}.{self.format}"
return filename
class BookQueue:
"""Thread-safe book queue manager."""
"""Thread-safe book queue manager with priority support and cancellation."""
def __init__(self) -> None:
self._queue: set[str] = set()
self._queue: queue.PriorityQueue[QueueItem] = queue.PriorityQueue()
self._lock = Lock()
self._status: dict[str, QueueStatus] = {}
self._book_data: dict[str, BookInfo]= {}
self._book_data: dict[str, BookInfo] = {}
self._status_timestamps: dict[str, datetime] = {} # Track when each status was last updated
self._status_timeout = timedelta(seconds=STATUS_TIMEOUT) # 1 hour timeout
self._cancel_flags: dict[str, Event] = {} # Cancellation flags for active downloads
self._active_downloads: dict[str, bool] = {} # Track currently downloading books
def add(self, book_id: str, book_data: BookInfo) -> None:
"""Add a book to the queue."""
def add(self, book_id: str, book_data: BookInfo, priority: int = 0) -> None:
"""Add a book to the queue with specified priority.
Args:
book_id: Unique identifier for the book
book_data: Book information
priority: Priority level (lower number = higher priority)
"""
with self._lock:
self._queue.add(book_id)
# Don't add if already exists and not in error/done state
if book_id in self._status and self._status[book_id] not in [QueueStatus.ERROR, QueueStatus.DONE, QueueStatus.CANCELLED]:
return
added_time = time.time()
book_data.priority = priority
book_data.added_time = added_time
queue_item = QueueItem(book_id, priority, added_time)
self._queue.put(queue_item)
self._book_data[book_id] = book_data
self._update_status(book_id, QueueStatus.QUEUED)
def get_next(self) -> Optional[str]:
"""Get next book ID from queue."""
with self._lock:
return self._queue.pop() if self._queue else None
def get_next(self) -> Optional[Tuple[str, Event]]:
"""Get next book ID from queue with cancellation flag.
Returns:
Tuple of (book_id, cancel_flag) or None if queue is empty
"""
# Use iterative approach to avoid stack overflow if many items are cancelled
while True:
try:
queue_item = self._queue.get_nowait()
book_id = queue_item.book_id
with self._lock:
# Check if book was cancelled while in queue
if book_id in self._status and self._status[book_id] == QueueStatus.CANCELLED:
continue # Skip cancelled items, try next
# Create cancellation flag for this download
cancel_flag = Event()
self._cancel_flags[book_id] = cancel_flag
self._active_downloads[book_id] = True
return book_id, cancel_flag
except queue.Empty:
return None
def _update_status(self, book_id: str, status: QueueStatus) -> None:
"""Internal method to update status and timestamp."""
@@ -63,11 +162,29 @@ class BookQueue:
"""Update status of a book in the queue."""
with self._lock:
self._update_status(book_id, status)
# Clean up active download tracking when finished
if status in [QueueStatus.COMPLETE, QueueStatus.AVAILABLE, QueueStatus.ERROR, QueueStatus.DONE, QueueStatus.CANCELLED]:
self._active_downloads.pop(book_id, None)
self._cancel_flags.pop(book_id, None)
def update_download_path(self, book_id: str, download_path: str) -> None:
"""Update the download path of a book in the queue."""
with self._lock:
self._book_data[book_id].download_path = download_path
if book_id in self._book_data:
self._book_data[book_id].download_path = download_path
def update_progress(self, book_id: str, progress: float) -> None:
"""Update download progress for a book."""
with self._lock:
if book_id in self._book_data:
self._book_data[book_id].progress = progress
def update_status_message(self, book_id: str, message: str) -> None:
"""Update detailed status message for a book."""
with self._lock:
if book_id in self._book_data:
self._book_data[book_id].status_message = message
def get_status(self) -> Dict[QueueStatus, Dict[str, BookInfo]]:
"""Get current queue status."""
@@ -78,6 +195,190 @@ class BookQueue:
if book_id in self._book_data:
result[status][book_id] = self._book_data[book_id]
return result
def get_queue_order(self) -> List[Dict[str, any]]:
"""Get current queue order for display."""
with self._lock:
queue_items = []
# Get items from priority queue without removing them
temp_items = []
while not self._queue.empty():
try:
item = self._queue.get_nowait()
temp_items.append(item)
if item.book_id in self._book_data:
book_info = self._book_data[item.book_id]
queue_items.append({
'id': item.book_id,
'title': book_info.title,
'author': book_info.author,
'priority': item.priority,
'added_time': item.added_time,
'status': self._status.get(item.book_id, QueueStatus.QUEUED)
})
except queue.Empty:
break
# Put items back in queue
for item in temp_items:
self._queue.put(item)
return sorted(queue_items, key=lambda x: (x['priority'], x['added_time']))
def cancel_download(self, book_id: str) -> bool:
"""Cancel a download or clear a completed/errored item.
Args:
book_id: Book identifier to cancel or clear
Returns:
bool: True if cancellation/clearing was successful
"""
with self._lock:
current_status = self._status.get(book_id)
# Allow cancellation during any active state
if current_status in [QueueStatus.RESOLVING, QueueStatus.DOWNLOADING]:
# Signal active download to stop
if book_id in self._cancel_flags:
self._cancel_flags[book_id].set()
self._update_status(book_id, QueueStatus.CANCELLED)
return True
elif current_status == QueueStatus.QUEUED:
# Remove from queue and mark as cancelled
self._update_status(book_id, QueueStatus.CANCELLED)
return True
elif current_status in [QueueStatus.COMPLETE, QueueStatus.DONE, QueueStatus.AVAILABLE, QueueStatus.ERROR, QueueStatus.CANCELLED]:
# Clear completed/errored/cancelled items from tracking
self._status.pop(book_id, None)
self._status_timestamps.pop(book_id, None)
self._book_data.pop(book_id, None)
self._cancel_flags.pop(book_id, None)
self._active_downloads.pop(book_id, None)
return True
return False
def set_priority(self, book_id: str, new_priority: int) -> bool:
"""Change the priority of a queued book.
Args:
book_id: Book identifier
new_priority: New priority level (lower = higher priority)
Returns:
bool: True if priority was successfully changed
"""
with self._lock:
if book_id not in self._status or self._status[book_id] != QueueStatus.QUEUED:
return False
# Remove book from queue and re-add with new priority
temp_items = []
found = False
while not self._queue.empty():
try:
item = self._queue.get_nowait()
if item.book_id == book_id:
# Create new item with updated priority
new_item = QueueItem(book_id, new_priority, item.added_time)
temp_items.append(new_item)
found = True
# Update book data priority
if book_id in self._book_data:
self._book_data[book_id].priority = new_priority
else:
temp_items.append(item)
except queue.Empty:
break
# Put all items back
for item in temp_items:
self._queue.put(item)
return found
def reorder_queue(self, book_priorities: Dict[str, int]) -> bool:
"""Bulk reorder queue by setting new priorities.
Args:
book_priorities: Dict mapping book_id to new priority
Returns:
bool: True if reordering was successful
"""
with self._lock:
# Extract all items from queue
all_items = []
while not self._queue.empty():
try:
item = self._queue.get_nowait()
# Update priority if specified
if item.book_id in book_priorities:
new_priority = book_priorities[item.book_id]
item = QueueItem(item.book_id, new_priority, item.added_time)
# Update book data priority
if item.book_id in self._book_data:
self._book_data[item.book_id].priority = new_priority
all_items.append(item)
except queue.Empty:
break
# Put all items back with updated priorities
for item in all_items:
self._queue.put(item)
return True
def get_active_downloads(self) -> List[str]:
"""Get list of currently active download book IDs."""
with self._lock:
return list(self._active_downloads.keys())
def has_pending_work(self) -> bool:
"""Check if there are any active downloads or queued items.
This is useful for determining if the bypasser should stay active
even when the UI is closed.
Returns:
bool: True if there are active downloads or queued items
"""
with self._lock:
# Check for active downloads
if self._active_downloads:
return True
# Check for queued items (excluding cancelled ones)
for book_id, status in self._status.items():
if status == QueueStatus.QUEUED:
return True
return False
def clear_completed(self) -> int:
"""Remove all completed, errored, or cancelled books from tracking.
Returns:
int: Number of books removed
"""
with self._lock:
to_remove = []
for book_id, status in self._status.items():
if status in [QueueStatus.COMPLETE, QueueStatus.DONE, QueueStatus.AVAILABLE, QueueStatus.ERROR, QueueStatus.CANCELLED]:
to_remove.append(book_id)
removed_count = len(to_remove)
for book_id in to_remove:
self._status.pop(book_id, None)
self._status_timestamps.pop(book_id, None)
self._book_data.pop(book_id, None)
self._cancel_flags.pop(book_id, None)
self._active_downloads.pop(book_id, None)
return removed_count
def refresh(self) -> None:
"""Remove any books that are done downloading or have stale status."""
@@ -101,7 +402,7 @@ class BookQueue:
# Check for stale status entries
last_update = self._status_timestamps.get(book_id)
if last_update and (current_time - last_update) > self._status_timeout:
if status == QueueStatus.DONE or status == QueueStatus.ERROR or status == QueueStatus.AVAILABLE:
if status in [QueueStatus.COMPLETE, QueueStatus.DONE, QueueStatus.ERROR, QueueStatus.AVAILABLE, QueueStatus.CANCELLED]:
to_remove.append(book_id)
# Remove stale entries
+547 -64
View File
@@ -13,9 +13,147 @@ import ipaddress
from logger import setup_logger
from config import PROXIES, AA_BASE_URL, CUSTOM_DNS, AA_AVAILABLE_URLS, DOH_SERVER
import config
import env
from datetime import datetime, timedelta
# Try to use gevent locks if available (for gevent worker compatibility)
# Fall back to threading locks for non-gevent environments
try:
from gevent.lock import RLock as _RLock
_using_gevent_locks = True
except ImportError:
from threading import RLock as _RLock
_using_gevent_locks = False
logger = setup_logger(__name__)
# In-memory state (no disk persistence)
STATE_TTL_DAYS = 30
_initialized = False
_dns_initialized = False
_aa_initialized = False
state: dict[str, Any] = {}
# Locks for greenlet-safe initialization and DNS switching
# Use RLock (reentrant lock) since init() calls init_dns() and init_aa()
_init_lock = _RLock()
_dns_switch_lock = _RLock()
# DNS rotation callbacks - called when DNS provider switches in auto mode
# Callbacks receive (provider_name: str, servers: List[str], doh_url: str)
_dns_rotation_callbacks: List[Callable[[str, List[str], str], None]] = []
_dns_callback_lock = _RLock()
def register_dns_rotation_callback(callback: Callable[[str, List[str], str], None]) -> None:
"""Register a callback to be called when DNS provider rotates.
The callback receives (provider_name, servers, doh_url) as arguments.
Use this to restart components that cache DNS resolution (e.g., Chrome).
"""
with _dns_callback_lock:
if callback not in _dns_rotation_callbacks:
_dns_rotation_callbacks.append(callback)
logger.debug(f"Registered DNS rotation callback: {callback.__name__}")
def unregister_dns_rotation_callback(callback: Callable[[str, List[str], str], None]) -> None:
"""Unregister a previously registered DNS rotation callback."""
with _dns_callback_lock:
if callback in _dns_rotation_callbacks:
_dns_rotation_callbacks.remove(callback)
logger.debug(f"Unregistered DNS rotation callback: {callback.__name__}")
def _notify_dns_rotation(provider_name: str, servers: List[str], doh_url: str) -> None:
"""Notify all registered callbacks about DNS rotation."""
with _dns_callback_lock:
callbacks = _dns_rotation_callbacks.copy()
for callback in callbacks:
try:
logger.debug(f"Calling DNS rotation callback: {callback.__name__}")
callback(provider_name, servers, doh_url)
except Exception as e:
logger.warning(f"DNS rotation callback {callback.__name__} failed: {e}")
def _agent_debug_log(code: str, source: str, reason: str, meta: Optional[dict] = None) -> None:
"""Lightweight debug hook for automated runs; safe no-op on failure."""
try:
logger.debug(f"[agent] code={code} source={source} reason={reason} meta={meta or {}}")
except Exception as exc:
# Avoid raising inside debug logger
logger.debug(f"[agent] log failed: {exc}")
def _load_state():
"""Return current in-memory network state (no disk persistence)."""
if state.get('chosen_at'):
chosen = datetime.fromisoformat(state['chosen_at'])
if datetime.now() - chosen > timedelta(days=STATE_TTL_DAYS):
state.clear()
return state
def _save_state(aa_url=None, dns_provider=None):
"""Update in-memory network state (no disk persistence)."""
if aa_url:
state['aa_base_url'] = aa_url
if dns_provider:
state['dns_provider'] = dns_provider
state['chosen_at'] = datetime.now().isoformat()
# AA URL failover state
_current_aa_url_index = 0
_aa_urls = AA_AVAILABLE_URLS.copy()
def _ensure_initialized() -> None:
"""Lazy guard so runtime setup happens once and late calls still work."""
global _initialized
if _initialized:
return
with _init_lock:
# Double-check after acquiring lock
if not _initialized:
init()
# DNS provider definitions: (name, servers, doh_url)
# Note: Google uses /resolve endpoint for JSON API, others use /dns-query
DNS_PROVIDERS = [
("cloudflare", ["1.1.1.1", "1.0.0.1"], "https://cloudflare-dns.com/dns-query"),
("google", ["8.8.8.8", "8.8.4.4"], "https://dns.google/resolve"),
("quad9", ["9.9.9.9", "149.112.112.112"], "https://dns.quad9.net/dns-query"),
("opendns", ["208.67.222.222", "208.67.220.220"], "https://doh.opendns.com/dns-query"),
]
# Domain patterns that should trigger DNS rotation on failure
DNS_ROTATION_DOMAINS = [
"annas-archive",
]
def should_rotate_dns_for_url(url: str) -> bool:
"""Check if a URL matches a known source domain for DNS rotation."""
url_lower = url.lower()
return any(domain in url_lower for domain in DNS_ROTATION_DOMAINS)
# DNS state
_current_dns_index = -1 # -1 = system DNS
_dns_exhausted_logged = False
def _is_auto_dns_mode() -> bool:
"""Check if DNS is in auto-rotation mode."""
return env._CUSTOM_DNS.lower().strip() == "auto" and not env.USING_TOR
def _current_dns_label() -> str:
"""Readable label for the active DNS choice."""
if _current_dns_index >= 0:
return DNS_PROVIDERS[_current_dns_index][0]
if CUSTOM_DNS:
return f"custom {CUSTOM_DNS}"
return "system"
# Common helper functions for DNS resolution
def _decode_host(host: Union[str, bytes, None]) -> str:
"""Convert host to string, handling bytes and None cases."""
@@ -34,7 +172,6 @@ def _decode_port(port: Union[str, bytes, int, None]) -> int:
return int(port)
def _is_local_address(host_str: str) -> bool:
"""Check if an address is local and should bypass custom DNS."""
"""Check if an address is local or private and should bypass custom DNS."""
# Localhost checks
if (host_str == 'localhost' or
@@ -67,17 +204,34 @@ def _is_ip_address(host_str: str) -> bool:
except ValueError:
return False
def _aa_hostnames() -> List[str]:
"""Return hostname portions for all configured AA URLs."""
return [
parsed.hostname for parsed in (urllib.parse.urlparse(url) for url in _aa_urls)
if parsed.hostname
]
def _is_aa_hostname(host_str: str) -> bool:
"""Check if a hostname matches any configured AA mirror host."""
return any(host_str.endswith(hostname) for hostname in _aa_hostnames())
# Store the original getaddrinfo function
original_getaddrinfo = socket.getaddrinfo
class DoHResolver:
"""DNS over HTTPS resolver implementation."""
"""DNS over HTTPS resolver implementation with caching."""
# Cache TTL in seconds (5 minutes)
CACHE_TTL = 300
def __init__(self, provider_url: str, hostname: str, ip: str):
"""Initialize DoH resolver with specified provider."""
self.base_url = provider_url.lower().strip()
self.hostname = hostname # Store the hostname for hostname-based skipping
self.ip = ip # Store IP for direct connections
self.session = requests.Session()
# DNS cache: {(hostname, record_type): (ip_list, timestamp)}
self._cache: dict[tuple[str, str], tuple[List[str], datetime]] = {}
# Different headers based on provider
if 'google' in self.base_url:
@@ -89,6 +243,24 @@ class DoHResolver:
'Accept': 'application/dns-json',
})
def _get_cached(self, hostname: str, record_type: str) -> Optional[List[str]]:
"""Get cached DNS result if still valid."""
key = (hostname, record_type)
if key in self._cache:
ips, timestamp = self._cache[key]
if datetime.now() - timestamp < timedelta(seconds=self.CACHE_TTL):
logger.debug(f"DoH cache hit for {hostname}: {ips}")
return ips
else:
# Cache expired, remove it
del self._cache[key]
return None
def _set_cached(self, hostname: str, record_type: str, ips: List[str]) -> None:
"""Cache DNS result."""
if ips: # Only cache non-empty results
self._cache[(hostname, record_type)] = (ips, datetime.now())
def resolve(self, hostname: str, record_type: str) -> List[str]:
"""Resolve a hostname using DoH.
@@ -113,6 +285,11 @@ class DoHResolver:
if hostname == self.hostname:
logger.debug(f"Skipping DoH resolution for DoH server itself: {hostname}")
return [self.ip]
# Check cache first
cached = self._get_cached(hostname, record_type)
if cached is not None:
return cached
try:
params = {
@@ -124,7 +301,7 @@ class DoHResolver:
self.base_url,
params=params,
proxies=PROXIES,
timeout=5
timeout=10 # Increased from 5s to handle slow network conditions
)
response.raise_for_status()
@@ -136,35 +313,31 @@ class DoHResolver:
# Extract IP addresses from the response
answers = [answer['data'] for answer in data['Answer']
if answer.get('type') == (28 if record_type == 'AAAA' else 1)]
logger.debug(f"Resolved {hostname} to {len(answers)} addresses using DoH: {answers}")
# Cache the result
self._set_cached(hostname, record_type, answers)
# Don't log here - the caller (custom_getaddrinfo) will log the final result
return answers
except Exception as e:
logger.warning(f"DoH resolution failed for {hostname}: {e}")
return []
def create_custom_resolver():
"""Create a custom DNS resolver using the configured DNS servers."""
def create_custom_resolver(servers: Optional[List[str]] = None):
"""Create a custom DNS resolver using the specified or configured DNS servers."""
custom_resolver = dns.resolver.Resolver()
custom_resolver.nameservers = CUSTOM_DNS
custom_resolver.nameservers = servers if servers is not None else CUSTOM_DNS
return custom_resolver
def resolve_with_custom_dns(resolver, hostname: str, record_type: str) -> List[str]:
"""Resolve hostname using custom DNS resolver.
Args:
resolver: The DNS resolver to use
hostname: The hostname to resolve
record_type: The DNS record type (A or AAAA)
Returns:
List of resolved IP addresses
"""
"""Resolve hostname using custom DNS resolver."""
try:
answers = resolver.resolve(hostname, record_type)
return [str(answer) for answer in answers]
except Exception as e:
logger.debug(f"{record_type} resolution failed for {hostname}: {e}")
except Exception:
# Don't log here - let the caller handle it to prevent spam
# Don't trigger DNS switch here either - caller handles it
return []
def create_custom_getaddrinfo(
@@ -193,42 +366,63 @@ def create_custom_getaddrinfo(
host_str = _decode_host(host)
port_int = _decode_port(port)
def _log_results(source: str, provider_label: str, res: Sequence[Tuple[AddressFamily, SocketKind, int, str, Tuple[Any, ...]]], is_bypass: bool = False) -> None:
"""Emit a unified resolver log with the IPs returned.
Args:
source: Description of resolver source
provider_label: Label for the DNS provider
res: Resolution results
is_bypass: If True, log at DEBUG level (for local/IP addresses)
"""
# Skip logging entirely for localhost to reduce noise
if host_str in ('localhost', '127.0.0.1', '::1'):
return
try:
ips = [entry[4][0] for entry in res if len(entry) >= 5 and entry[4]]
msg = f"Resolved {host_str} via {source} [{provider_label}]: {ips}"
if is_bypass:
logger.debug(msg)
else:
logger.info(msg)
except Exception:
pass # Silently ignore logging failures
# Skip custom resolution for IP addresses, local addresses, or if skip check passes
if _is_ip_address(host_str) or _is_local_address(host_str) or (skip_check and skip_check(host_str)):
logger.debug(f"Using system DNS for IP address or local/private address: {host_str}")
return original_getaddrinfo(host, port, family, type, proto, flags)
# Quietly bypass custom resolution for IP/local targets
res = original_getaddrinfo(host, port, family, type, proto, flags)
_log_results("system resolver (bypass)", "system", res, is_bypass=True)
return res
results: list[Tuple[AddressFamily, SocketKind, int, str, Tuple[Any, ...]]] = []
try:
# Try IPv6 first if family allows it
if family == 0 or family == socket.AF_INET6:
logger.debug(f"Resolving IPv6 address for {host_str}")
ipv6_answers = resolve_ipv6(host_str)
for answer in ipv6_answers:
results.append((socket.AF_INET6, cast(SocketKind, type), proto, '', (answer, port_int, 0, 0)))
if ipv6_answers:
logger.debug(f"Found {len(ipv6_answers)} IPv6 addresses for {host_str}")
# Then try IPv4
# Try IPv4 (IPv6 disabled to avoid noisy AAAA failures)
if family == 0 or family == socket.AF_INET:
logger.debug(f"Resolving IPv4 address for {host_str}")
ipv4_answers = resolve_ipv4(host_str)
for answer in ipv4_answers:
results.append((socket.AF_INET, cast(SocketKind, type), proto, '', (answer, port_int)))
if ipv4_answers:
logger.debug(f"Found {len(ipv4_answers)} IPv4 addresses for {host_str}")
if results:
logger.debug(f"Resolved {host_str} to {len(results)} addresses")
_log_results("custom resolver", _current_dns_label(), results)
return results
except Exception as e:
logger.warning(f"Custom DNS resolution failed for {host_str}: {e}, falling back to system DNS")
# Trigger DNS switch on failure (if auto mode)
if _is_auto_dns_mode() and not _is_local_address(host_str) and not _is_ip_address(host_str):
# Only switch if we haven't exhausted all providers
if _current_dns_index < len(DNS_PROVIDERS):
logger.info(f"Requesting DNS provider switch after custom resolver failure for {host_str}")
switch_dns_provider()
# Fall back to system DNS if custom resolution fails
logger.info(f"Custom DNS returned no addresses for {host_str}; falling back to system resolver")
try:
return original_getaddrinfo(host, port, family, type, proto, flags)
res = original_getaddrinfo(host, port, family, type, proto, flags)
_log_results("system resolver (fallback)", "system", res)
return res
except Exception as e:
logger.error(f"System DNS resolution also failed for {host_str}: {e}")
# Last resort: Try to connect to the hostname directly
@@ -240,11 +434,45 @@ def create_custom_getaddrinfo(
return custom_getaddrinfo
def init_doh_resolver(doh_server: str = DOH_SERVER):
"""Initialize DNS over HTTPS resolver.
def create_system_failover_getaddrinfo():
"""Wrap system getaddrinfo to trigger DNS provider switch on failure."""
_switch_logged: set[str] = set()
def system_failover_getaddrinfo(
host: Union[str, bytes, None],
port: Union[str, bytes, int, None],
family: int = 0,
type: int = 0,
proto: int = 0,
flags: int = 0
) -> Sequence[Tuple[AddressFamily, SocketKind, int, str, Tuple[Any, ...]]]:
host_str = _decode_host(host)
try:
return original_getaddrinfo(host, port, family, type, proto, flags)
except Exception as e:
if host_str not in _switch_logged:
logger.warning(f"System DNS resolution failed for {host_str}: {e}")
# Trigger DNS switch only in auto mode for non-local targets
if _is_auto_dns_mode() and not _is_ip_address(host_str) and not _is_local_address(host_str):
if _current_dns_index + 1 < len(DNS_PROVIDERS):
if host_str not in _switch_logged:
logger.info(f"Switching DNS provider after system DNS failure for {host_str}")
_switch_logged.add(host_str)
if switch_dns_provider():
return socket.getaddrinfo(host, port, family, type, proto, flags)
raise
return system_failover_getaddrinfo
def _init_doh_resolver_internal(doh_server: str) -> DoHResolver:
"""Internal: Initialize DNS over HTTPS resolver with specified server.
Args:
doh_server: The DoH server URL
Returns:
Configured DoHResolver instance
"""
# Pre-resolve the DoH server hostname to prevent recursion
url = urllib.parse.urlparse(doh_server)
@@ -292,9 +520,14 @@ def init_doh_resolver(doh_server: str = DOH_SERVER):
logger.info("DoH resolver successfully configured and activated")
return doh_resolver
def init_custom_resolver():
"""Initialize custom DNS resolver using configured DNS servers."""
custom_resolver = create_custom_resolver()
def _init_custom_resolver_internal(servers: List[str]):
"""Internal: Initialize custom DNS resolver with specified servers.
Args:
servers: List of DNS server IPs to use
"""
custom_resolver = create_custom_resolver(servers)
# Create resolver functions
def resolve_ipv4(hostname: str) -> List[str]:
@@ -309,32 +542,286 @@ def init_custom_resolver():
logger.info("Custom DNS resolver successfully configured and activated")
return custom_resolver
# Initialize DNS resolvers based on configuration
def init_doh_resolver(doh_server: str = ""):
"""Initialize DNS over HTTPS resolver."""
server = doh_server or DOH_SERVER
if not server:
return None
return _init_doh_resolver_internal(server)
def init_custom_resolver():
"""Initialize custom DNS resolver using configured DNS servers."""
if not CUSTOM_DNS:
return None
return _init_custom_resolver_internal(CUSTOM_DNS)
def switch_dns_provider() -> bool:
"""Switch to next DNS provider (auto mode only)."""
global CUSTOM_DNS, DOH_SERVER, _current_dns_index, _dns_exhausted_logged
if not _is_auto_dns_mode():
return False
with _dns_switch_lock:
if _current_dns_index + 1 >= len(DNS_PROVIDERS):
if not _dns_exhausted_logged:
logger.warning("All DNS providers exhausted, staying with current")
_dns_exhausted_logged = True
return False
_current_dns_index += 1
name, servers, doh = DNS_PROVIDERS[_current_dns_index]
CUSTOM_DNS = servers
DOH_SERVER = doh
config.CUSTOM_DNS = servers
config.DOH_SERVER = doh
logger.warning(f"Switched DNS provider to: {name} (using DoH)")
_save_state(dns_provider=name)
init_dns_resolvers()
# Notify listeners (e.g., Chrome bypasser) to restart with new DNS
_notify_dns_rotation(name, servers, doh)
return True
def rotate_dns_provider() -> bool:
"""Rotate DNS provider (auto mode only), cycling back if exhausted."""
global _current_dns_index, _dns_exhausted_logged
if not _is_auto_dns_mode():
return False
if _current_dns_index + 1 >= len(DNS_PROVIDERS):
logger.warning("DNS rotation: cycling back to first provider")
_current_dns_index = -1
_dns_exhausted_logged = False
return switch_dns_provider()
def rotate_dns_and_reset_aa() -> bool:
"""
Switch DNS provider (auto mode) and reset AA URL list to the first entry.
Returns True if DNS switched; False if no providers left or not in auto mode.
Note: This function can be called during initialization, so we must NOT call
_ensure_initialized() here to avoid recursive init loops.
"""
if not rotate_dns_provider():
return False
# Reset AA URL to first available auto option if using auto AA
global AA_BASE_URL, _current_aa_url_index
if AA_BASE_URL == "auto" or AA_BASE_URL in _aa_urls:
_current_aa_url_index = 0
AA_BASE_URL = _aa_urls[0]
config.AA_BASE_URL = AA_BASE_URL
logger.info(f"After DNS switch, resetting AA URL to: {AA_BASE_URL}")
_save_state(aa_url=AA_BASE_URL)
return True
def init_dns_resolvers():
"""Initialize DNS resolvers based on configuration."""
if len(CUSTOM_DNS) > 0:
global CUSTOM_DNS, DOH_SERVER
if _is_auto_dns_mode():
if _current_dns_index >= 0:
name, servers, doh = DNS_PROVIDERS[_current_dns_index]
CUSTOM_DNS = servers
DOH_SERVER = doh
config.CUSTOM_DNS = servers
config.DOH_SERVER = doh
logger.info(f"Using DNS provider: {name} (DoH enabled)")
else:
CUSTOM_DNS = []
DOH_SERVER = ""
config.CUSTOM_DNS = []
config.DOH_SERVER = ""
logger.info("Using system DNS (auto mode - will switch on failure)")
socket.getaddrinfo = cast(Any, create_system_failover_getaddrinfo())
return
if CUSTOM_DNS:
init_custom_resolver()
if DOH_SERVER:
init_doh_resolver()
init_doh_resolver(DOH_SERVER)
# Initialize DNS resolvers
init_dns_resolvers()
# Check available AA_BASE_URLs if set to auto
if AA_BASE_URL == "auto":
logger.info(f"AA_BASE_URL: auto, checking available urls {AA_AVAILABLE_URLS}")
for url in AA_AVAILABLE_URLS:
try:
response = requests.get(url, proxies=PROXIES)
if response.status_code == 200:
AA_BASE_URL = url
break
except Exception as e:
logger.error_trace(f"Error checking {url}: {e}")
def _initialize_dns_state() -> None:
"""Restore persisted DNS choice or start fresh."""
global _current_dns_index
if _is_auto_dns_mode():
persisted = state.get('dns_provider') if state else None
if persisted:
for i, (name, _, _) in enumerate(DNS_PROVIDERS):
if name == persisted:
_current_dns_index = i
logger.info(f"Restored DNS provider from state: {name}")
return
_current_dns_index = -1
def _initialize_aa_state() -> None:
"""Restore or probe AA URL state."""
global AA_BASE_URL, _current_aa_url_index
if AA_BASE_URL == "auto":
AA_BASE_URL = AA_AVAILABLE_URLS[0]
config.AA_BASE_URL = AA_BASE_URL
logger.info(f"AA_BASE_URL: {AA_BASE_URL}")
if state.get('aa_base_url') and state['aa_base_url'] in _aa_urls:
_current_aa_url_index = _aa_urls.index(state['aa_base_url'])
AA_BASE_URL = state['aa_base_url']
else:
logger.info(f"AA_BASE_URL: auto, checking available urls {_aa_urls}")
for i, url in enumerate(_aa_urls):
try:
response = requests.get(url, proxies=PROXIES, timeout=3)
if response.status_code == 200:
_current_aa_url_index = i
AA_BASE_URL = url
_save_state(aa_url=AA_BASE_URL)
break
except Exception:
pass
if AA_BASE_URL == "auto":
AA_BASE_URL = _aa_urls[0]
_current_aa_url_index = 0
elif AA_BASE_URL not in _aa_urls:
logger.info(f"AA_BASE_URL set to custom value {AA_BASE_URL}; skipping auto-switch")
else:
_current_aa_url_index = _aa_urls.index(AA_BASE_URL)
config.AA_BASE_URL = AA_BASE_URL
logger.info(f"AA_BASE_URL: {AA_BASE_URL}")
def init_dns(force: bool = False) -> None:
"""Initialize DNS state and resolvers."""
global state, _dns_initialized
if _dns_initialized and not force:
return
with _init_lock:
# Double-check after acquiring lock
if _dns_initialized and not force:
return
# Set flag BEFORE doing work to prevent recursive calls during init
_dns_initialized = True
try:
logger.debug(f"Initializing DNS (using {'gevent' if _using_gevent_locks else 'threading'} locks)")
state = _load_state()
_initialize_dns_state()
init_dns_resolvers()
except Exception:
_dns_initialized = False
raise
def init_aa(force: bool = False) -> None:
"""Initialize AA mirror selection."""
global state, _aa_initialized
if _aa_initialized and not force:
return
with _init_lock:
# Double-check after acquiring lock
if _aa_initialized and not force:
return
# Set flag BEFORE doing work to prevent recursive calls during init
_aa_initialized = True
try:
state = _load_state()
_initialize_aa_state()
except Exception:
_aa_initialized = False
raise
def init(force: bool = False) -> None:
"""
Initialize network state (DNS resolvers and AA mirror selection).
Called lazily on first network operation. Safe to call repeatedly;
later calls no-op unless force=True.
"""
global _initialized
if _initialized and not force:
return
with _init_lock:
# Double-check after acquiring lock
if _initialized and not force:
return
# Set flag BEFORE doing work to prevent recursive calls during init
# (e.g., DNS failover handlers calling back into init)
_initialized = True
try:
init_dns(force=force)
init_aa(force=force)
except Exception:
# Reset flag on failure so retry is possible
_initialized = False
raise
def get_aa_base_url():
"""Get current AA base URL."""
_ensure_initialized()
return AA_BASE_URL
def get_available_aa_urls():
"""Get list of configured AA URLs (copy)."""
_ensure_initialized()
return _aa_urls.copy()
def set_aa_url_index(new_index: int) -> bool:
"""Set AA base URL by index in available list; returns True if applied."""
_ensure_initialized()
global AA_BASE_URL, _current_aa_url_index
if new_index < 0 or new_index >= len(_aa_urls):
return False
_current_aa_url_index = new_index
AA_BASE_URL = _aa_urls[_current_aa_url_index]
config.AA_BASE_URL = AA_BASE_URL
logger.info(f"Set AA URL to: {AA_BASE_URL}")
_save_state(aa_url=AA_BASE_URL)
return True
class AAMirrorSelector:
"""
Small helper to keep AA mirror switching consistent across call sites.
Tracks attempts per DNS cycle and rewrites URLs safely.
"""
def __init__(self) -> None:
self._ensure_fresh_state(reset_attempts=True)
def _ensure_fresh_state(self, reset_attempts: bool = False) -> None:
_ensure_initialized()
self.aa_urls = get_available_aa_urls()
self._index = self._safe_index(get_aa_base_url())
self.current_base = self.aa_urls[self._index] if self.aa_urls else ""
if reset_attempts:
self.attempts_this_dns = 0
def _safe_index(self, base: str) -> int:
if base in self.aa_urls:
return self.aa_urls.index(base)
return 0
def rewrite(self, url: str) -> str:
"""Replace any known AA base in url with current_base."""
for base in self.aa_urls:
if url.startswith(base):
return url.replace(base, self.current_base, 1)
return url
def next_mirror_or_rotate_dns(self, allow_dns: bool = True) -> tuple[Optional[str], str]:
"""
Advance to next mirror; if exhausted and allowed, rotate DNS and reset to first.
Returns (new_base, action) where action is 'mirror', 'dns', or 'exhausted'.
"""
self.attempts_this_dns += 1
if self.attempts_this_dns >= len(self.aa_urls):
if allow_dns and rotate_dns_and_reset_aa():
self._ensure_fresh_state(reset_attempts=True)
return self.current_base, "dns"
return None, "exhausted"
next_index = (self._index + 1) % len(self.aa_urls)
set_aa_url_index(next_index)
self._ensure_fresh_state(reset_attempts=False)
return self.current_base, "mirror"
# Configure urllib opener with appropriate headers
opener = urllib.request.build_opener()
@@ -344,7 +831,3 @@ opener.addheaders = [
'Chrome/129.0.0.0 Safari/537.3')
]
urllib.request.install_opener(opener)
# Need an empty function to be called by downloader.py
def init():
pass
+134 -24
View File
@@ -1,6 +1,6 @@
# 📚 Calibre-Web-Automated-Book-Downloader
![Calibre-Web Automated Book Downloader](static/media/logo.png 'Calibre-Web Automated Book Downloader')
<img src="src/frontend/public/logo.png" alt="Calibre-Web Automated Book Downloader" width="200">
An intuitive web interface for searching and requesting book downloads, designed to work seamlessly with [Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated). This project streamlines the process of downloading books and preparing them for integration into your Calibre library.
@@ -63,10 +63,15 @@ An intuitive web interface for searching and requesting book downloads, designed
| `CWA_DB_PATH` | Calibre-Web's database | None |
| `ENABLE_LOGGING` | Enable log file | `true` |
| `LOG_LEVEL` | Log level to use | `info` |
| `SESSION_COOKIE_SECURE` | Secure cookie enforcement - Use for HTTPS connections only | `false` |
| `CALIBRE_WEB_URL` | Custom WebUI library link | None |
| `BYPASS_WARMUP_ON_CONNECT` | Warm up Cloudflare bypasser when first client connects | `true` |
If you wish to enable authentication, you must set `CWA_DB_PATH` to point to Calibre-Web's `app.db`, in order to match the username and password.
If logging is enabld, log folder default location is `/var/log/cwa-book-downloader`
Set `CALIBRE_WEB_URL` to your Calibre-Web / Booklore base URL. A ‘Go to library’ button will appear in the Web UI for quick access while downloading, and it also provides library access when CWA-BD is installed as a mobile PWA.
If logging is enabled, log folder default location is `/var/log/cwa-book-downloader`
Available log levels: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`. Higher levels show fewer messages.
Note that if using TOR, the TZ will be calculated automatically based on IP.
@@ -82,15 +87,41 @@ Note that if using TOR, the TZ will be calculated automatically based on IP.
| `BOOK_LANGUAGE` | Preferred language for books | `en` |
| `AA_DONATOR_KEY` | Optional Donator key for Anna's Archive fast download API | `` |
| `USE_BOOK_TITLE` | Use book title as filename instead of ID | `false` |
| `PRIORITIZE_WELIB` | When downloading, download from WELIB first instead of AA | `false` |
| `ALLOW_USE_WELIB` | Allow usage of welib for downloading books if found there | `true` |
If you change `BOOK_LANGUAGE`, you can add multiple comma separated languages, such as `en,fr,ru` etc.
Use the following environment variables to set specific folders in which to download
different content types (Book, Magazine, Comic, etc.):
| Variable | Description | Default Value |
|---------------------------------|--------------------------------|---------------|
| `INGEST_DIR_BOOK_FICTION` | Book (fiction) folder name | `` |
| `INGEST_DIR_BOOK_NON_FICTION` | Book (non-fiction) folder name | `` |
| `INGEST_DIR_BOOK_UNKNOWN` | Book (unknown) folder name | `` |
| `INGEST_DIR_MAGAZINE` | Magazine folder name | `` |
| `INGEST_DIR_COMIC_BOOK` | Comic book folder name | `` |
| `INGEST_DIR_AUDIOBOOK` | Audiobook folder name | `` |
| `INGEST_DIR_STANDARDS_DOCUMENT` | Standards document folder name | `` |
| `INGEST_DIR_MUSICAL_SCORE` | Musical score folder name | `` |
If no specific path is set for a content type the default is `INGEST_DIR`.
Remember to map the specified paths to where your instance of Calibre-Web-Automated (CWA) will find them, e.g.:
```
volumes:
- /tmp/data/calibre-web/comicbook-ingest:/cwa-comicbook-ingest
```
if `INGEST_DIR_COMIC_BOOK=/cwa-comicbook-ingest` and your CWA is configured to use `/tmp/data/calibre-web/comicbook-ingest`
for comic books.
#### AA
| Variable | Description | Default Value |
| ---------------------- | --------------------------------------------------------- | --------------------------------- |
| `AA_BASE_URL` | Base URL of Annas-Archive (could be changed for a proxy) | `https://annas-archive.org` |
| `USE_CF_BYPASS` | Disable CF bypass and use alternative links instead | `true` |
| `USE_CF_BYPASS` | Disable CF bypass and use alternative links instead | `true` |
If you are a donator on AA, you can use your Key in `AA_DONATOR_KEY` to speed up downloads and bypass the wait times.
If disabling the cloudflare bypass, you will be using alternative download hosts, such as libgen or z-lib, but they usually have a delay before getting the more recent books and their collection is not as big as aa's. But this setting should work for the majority of books.
@@ -102,9 +133,11 @@ If disabling the cloudflare bypass, you will be using alternative download hosts
| `AA_ADDITIONAL_URLS` | Proxy URLs for AA (, separated) | `` |
| `HTTP_PROXY` | HTTP proxy URL | `` |
| `HTTPS_PROXY` | HTTPS proxy URL | `` |
| `CUSTOM_DNS` | Custom DNS IP | `` |
| `CUSTOM_DNS` | DNS configuration | `auto` |
| `USE_DOH` | Use DNS over HTTPS | `false` |
**Proxy Configuration**
For proxy configuration, you can specify URLs in the following format:
```bash
# Basic proxy
@@ -116,31 +149,44 @@ HTTP_PROXY=http://username:password@proxy.example.com:8080
HTTPS_PROXY=http://username:password@proxy.example.com:8080
```
**DNS Configuration**
The `CUSTOM_DNS` setting supports two formats:
The `CUSTOM_DNS` setting controls how DNS resolution works. By default, it is set to `auto` which provides automatic failover for reliable connectivity.
1. **Custom DNS Servers**: A comma-separated list of DNS server IP addresses
**Auto Mode (Default)**
When `CUSTOM_DNS=auto`, the application starts with your system's default DNS. If DNS resolution fails, it automatically rotates through alternative providers using DNS over HTTPS (DoH):
1. System DNS (initial)
2. Cloudflare (1.1.1.1)
3. Google (8.8.8.8)
4. Quad9 (9.9.9.9)
5. OpenDNS (208.67.222.222)
This automatic rotation helps bypass ISP-level blocks and DNS issues without any manual configuration.
**Manual DNS Configuration**
If you prefer to use a specific DNS configuration, you can override the auto behavior:
1. **Preset DNS Providers**: Use one of these predefined options:
- `google` - Google DNS (8.8.8.8, 8.8.4.4)
- `quad9` - Quad9 DNS (9.9.9.9, 149.112.112.112)
- `cloudflare` - Cloudflare DNS (1.1.1.1, 1.0.0.1)
- `opendns` - OpenDNS (208.67.222.222, 208.67.220.220)
2. **Custom DNS Servers**: A comma-separated list of DNS server IP addresses
- Example: `127.0.0.53,127.0.1.53` (useful for PiHole)
- Supports both IPv4 and IPv6 addresses in the same string
- Supports both IPv4 and IPv6 addresses
2. **Preset DNS Providers**: Use one of these predefined options:
- `google` - Google DNS
- `quad9` - Quad9 DNS
- `cloudflare` - Cloudflare DNS
- `opendns` - OpenDNS
For users experiencing ISP-level website blocks (such as Virgin Media in the UK), using alternative DNS providers like Cloudflare may help bypass these restrictions
If a `CUSTOM_DNS` is specified from the preset providers, you can also set a `USE_DOH=true` to force using DNS over HTTPS,
which might also help in certain network situations. Note that only `google`, `quad9`, `cloudflare` and `opendns` are
supported for now, and any other value in `CUSTOM_DNS` will make the `USE_DOH` flag ignored.
Try something like this :
When using preset providers, you can optionally enable DNS over HTTPS with `USE_DOH=true`:
```bash
CUSTOM_DNS=cloudflare
USE_DOH=true
```
Note: When using custom IP addresses, the `USE_DOH` flag is ignored since DoH requires a known provider endpoint.
#### Custom configuration
| Variable | Description | Default Value |
@@ -174,7 +220,9 @@ volumes:
Mount should align with your Calibre-Web-Automated ingest folder.
## 🧅 Tor Variant
## Variants:
### 🧅 Tor Variant
This application also offers a variant that routes all its traffic through the Tor network. This can be useful for enhanced privacy or bypassing network restrictions.
@@ -195,11 +243,73 @@ To use the Tor variant:
* **Timezone:** When running in Tor mode, the container will attempt to determine the timezone based on the Tor exit node's IP address and set it automatically. This will override the `TZ` environment variable if it is set.
* **Network Settings:** Custom DNS, DoH, and HTTP(S) proxy settings (`CUSTOM_DNS`, `USE_DOH`, `HTTP_PROXY`, `HTTPS_PROXY`) are ignored when using the Tor variant, as all traffic goes through Tor.
### External Cloudflare resolver variant
This variant allows the application to use an external service to bypass Cloudflare protection, instead of relying on the built-in bypasser. This is useful if you already have a dedicated Cloudflare resolver (such as [FlareSolverr](https://github.com/FlareSolverr/FlareSolverr) or compatible services like [ByParr](https://github.com/ThePhaseless/Byparr)) running elsewhere.
#### How it works:
- When enabled, all requests that require Cloudflare bypass are sent to your external resolver service.
- The application communicates with the resolver using its API.
#### Configuration
| Variable | Description | Default Value |
| ---------------------- | ----------------------------------------------------------- | ----------------------- |
| `EXT_BYPASSER_URL` | The full URL of your external resolver (required) | |
| `EXT_BYPASSER_PATH` | API path for the resolver (usually `/v1`) | `/v1` |
| `EXT_BYPASSER_TIMEOUT` | Timeout for page loading (in milliseconds) | `60000` |
#### Important
This feature follows the same configuration of the built-in Cloudflare bypasser, so you should turn on the `USE_CF_BYPASS` configuration to enable it.
#### To use the External Cloudflare resolver variant:
1. Get the extbp-specific docker-compose file:
```bash
curl -O https://raw.githubusercontent.com/calibrain/calibre-web-automated-book-downloader/refs/heads/main/docker-compose.extbp.yml
```
2. Start the service using this file:
```bash
docker compose -f docker-compose.extbp.yml up -d
```
#### Compatibility:
This feature is designed to work with any resolver that implements the `FlareSolverr` API schema, including `ByParr` and similar projects.
#### Internal vs External Bypasser
The **internal bypasser** (default) is custom-designed for this application's specific needs. It handles session management, cookie persistence, and retry logic optimized for book downloading workflows. For most users, this provides the most reliable experience out of the box.
The **external bypasser** is better suited if you:
- Already run FlareSolverr/ByParr for other services and want to consolidate
- Need to share bypass infrastructure across multiple applications
- Want to offload browser automation to a dedicated, more powerful container
If you're unsure which to use, start with the default internal bypasser.
## 🏗️ Architecture
The application consists of a single service:
The application consists of a Flask backend with a React-based frontend:
1. **calibre-web-automated-bookdownloader**: Main application providing web interface and download functionality
### Backend
- **Flask Application**: Python-based backend (`app.py`, `backend.py`) providing REST API and WebSocket support
- **Download Manager**: Handles book search, download requests, and queue management (`downloader.py`, `book_manager.py`)
- **Network Layer**: Cloudflare bypass and proxy support (`cloudflare_bypasser.py`, `network.py`)
### Frontend
- **React + TypeScript**: Modern web interface built with Vite (`src/frontend`)
- **Real-time Updates**: WebSocket integration for live download status
- **Responsive UI**: TailwindCSS-based design for mobile and desktop
For frontend development, use the provided Makefile:
```bash
make install # Install dependencies
make dev # Start development server
make build # Build for production
```
If you run the docker compose file, the frontend will be built and served automatically. But if you run the frontend dev server it will supercede the docker compose frontend.
## 🏥 Health Monitoring
@@ -213,7 +323,7 @@ Checks run every 30 seconds with a 30-second timeout and 3 retries.
You can enable by adding this to your compose :
```
HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
CMD pyrequests http://localhost:8084/request/api/status || exit 1
CMD curl -s http://localhost:8084/api/status || exit 1
```
## 📝 Logging
+13
View File
@@ -0,0 +1,13 @@
flask
flask-cors
flask-socketio
python-socketio
requests[socks]
beautifulsoup4
tqdm
dnspython
gunicorn
gevent
gevent-websocket
psutil
emoji
+4
View File
@@ -0,0 +1,4 @@
pyvirtualdisplay
pyautogui
seleniumbase>=4.41.1
python-xlib
-11
View File
@@ -1,11 +0,0 @@
flask
requests[socks]
beautifulsoup4
tqdm
pyvirtualdisplay
dnspython
pyautogui
seleniumbase==4.41
gunicorn
python-xlib
psutil
+99
View File
@@ -0,0 +1,99 @@
# Source Code Documentation
This directory contains the frontend application for Calibre-Web Automated Book Downloader.
## Structure
```
src/
└── frontend/ # React + TypeScript frontend application
├── public/ # Static assets (logo, favicon)
├── src/ # Source code
│ ├── components/ # React components
│ ├── App.tsx # Main application component
│ └── styles.css # Global styles
├── package.json # Dependencies and scripts
├── vite.config.ts # Vite configuration
└── tsconfig.json # TypeScript configuration
```
## Frontend Development
### Prerequisites
- Node.js (v16 or higher)
- npm or yarn
### Quick Start
From the project root:
```bash
# Install dependencies
make install
# Start development server (http://localhost:5173)
make dev
# Build for production
make build
# Preview production build
make preview
# Run type checking
make typecheck
```
Alternatively, from `src/frontend`:
```bash
npm install
npm run dev
npm run build
```
### Technology Stack
- **Framework**: React 18 with TypeScript
- **Build Tool**: Vite 5
- **Styling**: TailwindCSS 3
- **Communication**: WebSocket for real-time updates
### Key Features
- **Search Interface**: Real-time book search with filtering
- **Download Queue**: Live status updates via WebSocket
- **Details Modal**: Rich book information display
- **Responsive Design**: Mobile-first approach
## Development Tips
### Hot Module Replacement (HMR)
The development server supports HMR for instant feedback during development.
### API Integration
The frontend communicates with the Flask backend via:
- REST API endpoints (`/api/*`)
- WebSocket connection (`ws://localhost:8084/ws`)
### Building for Production
The production build is optimized and minified:
```bash
make build
```
Output is generated in `src/frontend/dist/`
### Type Safety
Run TypeScript checks without building:
```bash
make typecheck
```
## Debugging
### Development Server Issues
- Ensure port 5173 is available
- Check that the backend is running on port 8084
- Verify WebSocket connection in browser console
### Build Issues
- Clear `node_modules` and reinstall: `make clean && make install`
- Check Node.js version compatibility
- Verify TypeScript configuration
+26
View File
@@ -0,0 +1,26 @@
# Dependencies
node_modules
/.pnp
.pnp.js
# Production
/dist
# Misc
.DS_Store
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# IDE
.vscode/*
!.vscode/extensions.json
.idea
# Environment
.env
.env.local
.env.production.local
.env.development.local
.env.test.local
+42
View File
@@ -0,0 +1,42 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
<meta name="description" content="Calibre Web Book Downloader - Modern UI" />
<!-- Theme color with media queries for light/dark mode -->
<meta name="theme-color" content="#f8f8f8" media="(prefers-color-scheme: light)" />
<meta name="theme-color" content="#121212" media="(prefers-color-scheme: dark)" />
<!-- iOS PWA Meta Tags -->
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<meta name="apple-mobile-web-app-title" content="Book Downloader" />
<!-- App Icons -->
<link rel="icon" type="image/x-icon" href="/favicon.ico" />
<link rel="apple-touch-icon" href="/logo.png" />
<title>Book Downloader</title>
<script>
// Apply theme immediately before first paint to prevent flash
(function() {
const savedTheme = localStorage.getItem('preferred-theme') || 'auto';
let theme = savedTheme;
if (savedTheme === 'auto') {
theme = window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light';
}
document.documentElement.setAttribute('data-theme', theme);
// Add class to prevent transitions on initial load
document.documentElement.classList.add('preload');
})();
</script>
</head>
<body style="background: var(--bg); color: var(--text);">
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
+3299
View File
File diff suppressed because it is too large Load Diff
+29
View File
@@ -0,0 +1,29 @@
{
"name": "cwad-frontend",
"private": true,
"version": "1.0.0",
"type": "module",
"scripts": {
"dev": "vite --host 0.0.0.0",
"build": "tsc && vite build",
"preview": "vite preview",
"typecheck": "tsc --noEmit"
},
"dependencies": {
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-router-dom": "^6.30.2",
"socket.io-client": "^4.7.5"
},
"devDependencies": {
"@types/node": "^24.10.0",
"@types/react": "^18.3.3",
"@types/react-dom": "^18.3.0",
"@vitejs/plugin-react": "^4.3.1",
"autoprefixer": "^10.4.16",
"postcss": "^8.4.32",
"tailwindcss": "^3.4.0",
"typescript": "^5.5.3",
"vite": "^5.4.0"
}
}
+6
View File
@@ -0,0 +1,6 @@
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
}

Before

Width:  |  Height:  |  Size: 199 KiB

After

Width:  |  Height:  |  Size: 199 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

+557
View File
@@ -0,0 +1,557 @@
import { useState, useEffect, useCallback, useRef, CSSProperties } from 'react';
import { Navigate, Route, Routes, useNavigate } from 'react-router-dom';
import {
Book,
StatusData,
ButtonStateInfo,
AppConfig,
LoginCredentials,
AdvancedFilterState,
} from './types';
import { searchBooks, getBookInfo, downloadBook, cancelDownload, clearCompleted, getConfig, login, logout, checkAuth, AuthenticationError } from './services/api';
import { useToast } from './hooks/useToast';
import { useRealtimeStatus } from './hooks/useRealtimeStatus';
import { Header } from './components/Header';
import { SearchSection } from './components/SearchSection';
import { AdvancedFilters } from './components/AdvancedFilters';
import { ResultsSection } from './components/ResultsSection';
import { DetailsModal } from './components/DetailsModal';
import { DownloadsSidebar } from './components/DownloadsSidebar';
import { ToastContainer } from './components/ToastContainer';
import { Footer } from './components/Footer';
import { LoginPage } from './pages/LoginPage';
import { DEFAULT_LANGUAGES, DEFAULT_SUPPORTED_FORMATS } from './data/languages';
import { LANGUAGE_OPTION_DEFAULT } from './utils/languageFilters';
import { buildSearchQuery } from './utils/buildSearchQuery';
import './styles.css';
const DEFAULT_FORMAT_SELECTION = DEFAULT_SUPPORTED_FORMATS.filter(format => format !== 'pdf');
function App() {
// Authentication state
const [isAuthenticated, setIsAuthenticated] = useState<boolean>(false);
const [authRequired, setAuthRequired] = useState<boolean>(true);
const [authChecked, setAuthChecked] = useState<boolean>(false);
const [loginError, setLoginError] = useState<string | null>(null);
const [isLoggingIn, setIsLoggingIn] = useState<boolean>(false);
const navigate = useNavigate();
const [books, setBooks] = useState<Book[]>([]);
const [selectedBook, setSelectedBook] = useState<Book | null>(null);
const [isSearching, setIsSearching] = useState(false);
const [config, setConfig] = useState<AppConfig | null>(null);
const [searchInput, setSearchInput] = useState('');
const [showAdvanced, setShowAdvanced] = useState(false);
const [downloadsSidebarOpen, setDownloadsSidebarOpen] = useState(false);
const [lastSearchQuery, setLastSearchQuery] = useState('');
const [advancedFilters, setAdvancedFilters] = useState<AdvancedFilterState>({
isbn: '',
author: '',
title: '',
lang: [LANGUAGE_OPTION_DEFAULT],
sort: '',
content: '',
formats: DEFAULT_FORMAT_SELECTION,
});
const { toasts, showToast, removeToast } = useToast();
const updateAdvancedFilters = useCallback((updates: Partial<AdvancedFilterState>) => {
setAdvancedFilters(prev => ({ ...prev, ...updates }));
}, []);
// Determine WebSocket URL based on current location
// In production, use the same origin as the page; in dev, use localhost
const wsUrl = window.location.hostname === 'localhost' || window.location.hostname === '127.0.0.1'
? 'http://localhost:8084'
: window.location.origin;
// Use realtime status with WebSocket and polling fallback
const {
status: currentStatus,
isUsingWebSocket,
forceRefresh: fetchStatus
} = useRealtimeStatus({
wsUrl,
pollInterval: 5000,
reconnectAttempts: 3,
});
// Calculate status counts for header badges
const getStatusCounts = () => {
const ongoing = [
currentStatus.queued,
currentStatus.resolving,
currentStatus.downloading,
].reduce((sum, status) => sum + (status ? Object.keys(status).length : 0), 0);
const completed = currentStatus.complete
? Object.keys(currentStatus.complete).length
: 0;
const errored = currentStatus.error ? Object.keys(currentStatus.error).length : 0;
return { ongoing, completed, errored };
};
const statusCounts = getStatusCounts();
const activeCount = statusCounts.ongoing;
// Compute visibility states
const hasResults = books.length > 0;
const isInitialState = !hasResults;
// Detect status changes and show notifications
const detectChanges = useCallback((prev: StatusData, curr: StatusData) => {
if (!prev || Object.keys(prev).length === 0) return;
// Check for new items in queue
const prevQueued = prev.queued || {};
const currQueued = curr.queued || {};
Object.keys(currQueued).forEach(bookId => {
if (!prevQueued[bookId]) {
const book = currQueued[bookId];
showToast(`${book.title || 'Book'} added to queue`, 'info');
}
});
// Check for items that started downloading
const prevDownloading = prev.downloading || {};
const currDownloading = curr.downloading || {};
Object.keys(currDownloading).forEach(bookId => {
if (!prevDownloading[bookId]) {
const book = currDownloading[bookId];
showToast(`${book.title || 'Book'} started downloading`, 'info');
}
});
// Check for completed items
const prevDownloadingIds = new Set(Object.keys(prevDownloading));
const prevResolvingIds = new Set(Object.keys(prev.resolving || {}));
const prevQueuedIds = new Set(Object.keys(prevQueued));
const currComplete = curr.complete || {};
Object.keys(currComplete).forEach(bookId => {
if (prevDownloadingIds.has(bookId) || prevQueuedIds.has(bookId)) {
const book = currComplete[bookId];
showToast(`${book.title || 'Book'} completed`, 'success');
}
});
// Check for failed items
const currError = curr.error || {};
Object.keys(currError).forEach(bookId => {
if (prevDownloadingIds.has(bookId) || prevResolvingIds.has(bookId) || prevQueuedIds.has(bookId)) {
const book = currError[bookId];
const errorMsg = book.status_message || 'Download failed';
showToast(`${book.title || 'Book'}: ${errorMsg}`, 'error');
}
});
}, [showToast]);
// Track previous status for change detection
const prevStatusRef = useRef<StatusData>({});
// Check authentication on mount
useEffect(() => {
const verifyAuth = async () => {
try {
const response = await checkAuth();
const authenticated = response.authenticated || false;
const authIsRequired = response.auth_required !== false; // Default to true if undefined
setAuthRequired(authIsRequired);
setIsAuthenticated(authenticated);
} catch (error) {
console.error('Auth check failed:', error);
// On error, assume auth is required and user is not authenticated
setAuthRequired(true);
setIsAuthenticated(false);
} finally {
setAuthChecked(true);
}
};
verifyAuth();
}, []);
// Authentication handlers
const handleLogin = async (credentials: LoginCredentials) => {
setIsLoggingIn(true);
setLoginError(null);
try {
const response = await login(credentials);
if (response.success) {
setIsAuthenticated(true);
setLoginError(null);
navigate('/', { replace: true });
} else {
setLoginError(response.error || 'Login failed');
}
} catch (error) {
if (error instanceof Error) {
setLoginError(error.message || 'Login failed');
} else {
setLoginError('Login failed');
}
} finally {
setIsLoggingIn(false);
}
};
const handleLogout = async () => {
try {
await logout();
setIsAuthenticated(false);
// Clear application state
setBooks([]);
setSelectedBook(null);
setSearchInput('');
setLastSearchQuery('');
navigate('/login', { replace: true });
} catch (error) {
console.error('Logout failed:', error);
showToast('Logout failed', 'error');
}
};
// Detect status changes when currentStatus updates
useEffect(() => {
if (prevStatusRef.current && Object.keys(prevStatusRef.current).length > 0) {
detectChanges(prevStatusRef.current, currentStatus);
}
prevStatusRef.current = currentStatus;
}, [currentStatus, detectChanges]);
// Fetch config on mount and when authentication changes
useEffect(() => {
const loadConfig = async () => {
try {
const cfg = await getConfig();
setConfig(cfg);
// Update format selection to match supported formats from config
// This ensures PDF is auto-selected when added to SUPPORTED_FORMATS env var
if (cfg?.supported_formats) {
setAdvancedFilters(prev => ({
...prev,
formats: cfg.supported_formats,
}));
}
} catch (error) {
console.error('Failed to load config:', error);
// Use defaults if config fails to load
}
};
// Only fetch config if authenticated (or auth is not required)
if (isAuthenticated) {
loadConfig();
}
}, [isAuthenticated]);
// Log WebSocket connection status changes
useEffect(() => {
if (isUsingWebSocket) {
console.log('✅ Using WebSocket for real-time updates');
} else {
console.log('⏳ Using polling fallback (5s interval)');
}
}, [isUsingWebSocket]);
// Fetch status immediately on startup
useEffect(() => {
fetchStatus();
}, [fetchStatus]);
// Search handler
const handleSearch = async (query: string) => {
if (!query) {
setBooks([]);
setLastSearchQuery('');
return;
}
setIsSearching(true);
setLastSearchQuery(query);
try {
const results = await searchBooks(query);
setBooks(results);
if (results.length === 0) {
showToast('No results found', 'error');
}
} catch (error) {
if (error instanceof AuthenticationError) {
setIsAuthenticated(false);
if (authRequired) {
navigate('/login', { replace: true });
}
} else {
console.error('Search failed:', error);
setBooks([]);
const message = error instanceof Error ? error.message : 'Search failed';
const friendly = message.includes("Anna's Archive") || message.includes('Network restricted')
? message
: "Unable to reach Anna's Archive. Network may be restricted or mirrors blocked.";
showToast(friendly, 'error');
}
} finally {
setIsSearching(false);
}
};
// Show book details
const handleShowDetails = async (id: string): Promise<void> => {
try {
const book = await getBookInfo(id);
setSelectedBook(book);
} catch (error) {
console.error('Failed to load book details:', error);
showToast('Failed to load book details', 'error');
}
};
// Download book
const handleDownload = async (book: Book): Promise<void> => {
try {
await downloadBook(book.id);
// Fetch status to update button states (detectChanges will show toast)
await fetchStatus();
} catch (error) {
console.error('Download failed:', error);
showToast('Failed to queue download', 'error');
throw error; // Re-throw so button components can reset their queuing state
}
};
// Cancel download
const handleCancel = async (id: string) => {
try {
await cancelDownload(id);
await fetchStatus();
} catch (error) {
console.error('Cancel failed:', error);
}
};
// Clear completed
const handleClearCompleted = async () => {
try {
await clearCompleted();
await fetchStatus();
} catch (error) {
console.error('Clear completed failed:', error);
}
};
// Reset search state (clear books and search input)
const handleResetSearch = () => {
setBooks([]);
setSearchInput('');
setShowAdvanced(false);
setLastSearchQuery('');
// Use config's supported formats if available, otherwise fall back to default
const resetFormats = config?.supported_formats || DEFAULT_FORMAT_SELECTION;
setAdvancedFilters({
isbn: '',
author: '',
title: '',
lang: [LANGUAGE_OPTION_DEFAULT],
sort: '',
content: '',
formats: resetFormats,
});
};
const handleSortChange = (value: string) => {
updateAdvancedFilters({ sort: value });
if (!lastSearchQuery) return;
const params = new URLSearchParams(lastSearchQuery);
if (value) {
params.set('sort', value);
} else {
params.delete('sort');
}
const nextQuery = params.toString();
if (!nextQuery) return;
handleSearch(nextQuery);
};
// Get button state for a book - memoized to ensure proper re-renders when status changes
const getButtonState = useCallback((bookId: string): ButtonStateInfo => {
// Check error first
if (currentStatus.error && currentStatus.error[bookId]) {
return { text: 'Failed', state: 'error' };
}
// Check completed
if (currentStatus.complete && currentStatus.complete[bookId]) {
return { text: 'Downloaded', state: 'complete' };
}
// Check in-progress states
if (currentStatus.downloading && currentStatus.downloading[bookId]) {
const book = currentStatus.downloading[bookId];
return {
text: 'Downloading',
state: 'downloading',
progress: book.progress
};
}
if (currentStatus.resolving && currentStatus.resolving[bookId]) {
return { text: 'Resolving', state: 'resolving' };
}
if (currentStatus.queued && currentStatus.queued[bookId]) {
return { text: 'Queued', state: 'queued' };
}
return { text: 'Download', state: 'download' };
}, [currentStatus]);
const bookLanguages = config?.book_languages || DEFAULT_LANGUAGES;
const supportedFormats = config?.supported_formats || DEFAULT_SUPPORTED_FORMATS;
const defaultLanguageCodes =
config?.default_language && config.default_language.length > 0
? config.default_language
: [bookLanguages[0]?.code || 'en'];
const mainAppContent = (
<>
<Header
calibreWebUrl={config?.calibre_web_url || ''}
debug={config?.debug || false}
logoUrl="/logo.png"
showSearch={!isInitialState}
searchInput={searchInput}
onSearchChange={setSearchInput}
onDownloadsClick={() => setDownloadsSidebarOpen(true)}
statusCounts={statusCounts}
onLogoClick={handleResetSearch}
authRequired={authRequired}
isAuthenticated={isAuthenticated}
onLogout={handleLogout}
onSearch={() => {
const query = buildSearchQuery({
searchInput,
showAdvanced,
advancedFilters,
bookLanguages,
defaultLanguage: defaultLanguageCodes,
});
handleSearch(query);
}}
onAdvancedToggle={() => setShowAdvanced(!showAdvanced)}
isLoading={isSearching}
onShowToast={showToast}
onRemoveToast={removeToast}
/>
<AdvancedFilters
visible={showAdvanced && !isInitialState}
bookLanguages={bookLanguages}
defaultLanguage={defaultLanguageCodes}
supportedFormats={supportedFormats}
filters={advancedFilters}
onFiltersChange={updateAdvancedFilters}
/>
<main className="w-full max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-3 sm:py-6">
<SearchSection
onSearch={handleSearch}
isLoading={isSearching}
isInitialState={isInitialState}
bookLanguages={bookLanguages}
defaultLanguage={defaultLanguageCodes}
supportedFormats={config?.supported_formats || DEFAULT_SUPPORTED_FORMATS}
logoUrl="/logo.png"
searchInput={searchInput}
onSearchInputChange={setSearchInput}
showAdvanced={showAdvanced}
onAdvancedToggle={() => setShowAdvanced(!showAdvanced)}
advancedFilters={advancedFilters}
onAdvancedFiltersChange={updateAdvancedFilters}
/>
<ResultsSection
books={books}
visible={hasResults}
onDetails={handleShowDetails}
onDownload={handleDownload}
getButtonState={getButtonState}
sortValue={advancedFilters.sort}
onSortChange={handleSortChange}
/>
{selectedBook && (
<DetailsModal
book={selectedBook}
onClose={() => setSelectedBook(null)}
onDownload={handleDownload}
buttonState={getButtonState(selectedBook.id)}
/>
)}
</main>
<Footer
buildVersion={config?.build_version}
releaseVersion={config?.release_version}
debug={config?.debug}
/>
<ToastContainer toasts={toasts} />
{/* Downloads Sidebar */}
<DownloadsSidebar
isOpen={downloadsSidebarOpen}
onClose={() => setDownloadsSidebarOpen(false)}
status={currentStatus}
onRefresh={fetchStatus}
onClearCompleted={handleClearCompleted}
onCancel={handleCancel}
activeCount={activeCount}
/>
</>
);
const visuallyHiddenStyle: CSSProperties = {
position: 'absolute',
width: '1px',
height: '1px',
padding: 0,
margin: '-1px',
overflow: 'hidden',
clip: 'rect(0, 0, 0, 0)',
whiteSpace: 'nowrap',
border: 0,
};
if (!authChecked) {
return (
<div aria-live="polite" style={visuallyHiddenStyle}>
Checking authentication…
</div>
);
}
const shouldRedirectFromLogin = !authRequired || isAuthenticated;
const appElement = authRequired && !isAuthenticated ? (
<Navigate to="/login" replace />
) : (
mainAppContent
);
return (
<Routes>
<Route
path="/login"
element={
shouldRedirectFromLogin ? (
<Navigate to="/" replace />
) : (
<LoginPage
onLogin={handleLogin}
error={loginError}
isLoading={isLoggingIn}
/>
)
}
/>
<Route path="/*" element={appElement} />
</Routes>
);
}
export default App;
@@ -0,0 +1,165 @@
import { ReactNode } from 'react';
import { AdvancedFilterState, Language } from '../types';
import { normalizeLanguageSelection } from '../utils/languageFilters';
import { LanguageMultiSelect } from './LanguageMultiSelect';
import { DropdownList } from './DropdownList';
import { CONTENT_OPTIONS } from '../data/filterOptions';
const FORMAT_TYPES = ['pdf', 'epub', 'mobi', 'azw3', 'fb2', 'djvu', 'cbz', 'cbr'] as const;
interface AdvancedFiltersProps {
visible: boolean;
bookLanguages: Language[];
defaultLanguage: string[];
supportedFormats: string[];
filters: AdvancedFilterState;
onFiltersChange: (updates: Partial<AdvancedFilterState>) => void;
formClassName?: string;
renderWrapper?: (form: ReactNode) => ReactNode;
}
export const AdvancedFilters = ({
visible,
bookLanguages,
defaultLanguage,
supportedFormats,
filters,
onFiltersChange,
formClassName,
renderWrapper,
}: AdvancedFiltersProps) => {
const { isbn, author, title, lang, content, formats } = filters;
const handleLangChange = (next: string[]) => {
const normalized = normalizeLanguageSelection(next);
onFiltersChange({ lang: normalized });
};
const handleContentChange = (next: string[] | string) => {
const value = Array.isArray(next) ? next[0] ?? '' : next;
onFiltersChange({ content: value });
};
const handleFormatsChange = (next: string[] | string) => {
const nextFormats = Array.isArray(next) ? next : next ? [next] : [];
onFiltersChange({ formats: nextFormats });
};
const formatOptions = FORMAT_TYPES.map(format => ({
value: format,
label: format.toUpperCase(),
disabled: !supportedFormats.includes(format),
}));
if (!visible) return null;
const form = (
<form
id="search-filters"
className={
formClassName ??
'grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4 px-2 lg:ml-[calc(3rem+1rem)] lg:w-[50vw]'
}
>
<div>
<label htmlFor="isbn-input" className="block text-sm mb-1 opacity-80">
ISBN
</label>
<input
id="isbn-input"
type="text"
placeholder="ISBN"
autoComplete="off"
className="w-full px-3 py-2 rounded-md border"
style={{
background: 'var(--bg-soft)',
color: 'var(--text)',
borderColor: 'var(--border-muted)',
}}
value={isbn}
onChange={e => {
onFiltersChange({ isbn: e.target.value });
}}
/>
</div>
<div>
<label htmlFor="author-input" className="block text-sm mb-1 opacity-80">
Author
</label>
<input
id="author-input"
type="text"
placeholder="Author"
autoComplete="off"
className="w-full px-3 py-2 rounded-md border"
style={{
background: 'var(--bg-soft)',
color: 'var(--text)',
borderColor: 'var(--border-muted)',
}}
value={author}
onChange={e => {
onFiltersChange({ author: e.target.value });
}}
/>
</div>
<div>
<label htmlFor="title-input" className="block text-sm mb-1 opacity-80">
Title
</label>
<input
id="title-input"
type="text"
placeholder="Title"
autoComplete="off"
className="w-full px-3 py-2 rounded-md border"
style={{
background: 'var(--bg-soft)',
color: 'var(--text)',
borderColor: 'var(--border-muted)',
}}
value={title}
onChange={e => {
onFiltersChange({ title: e.target.value });
}}
/>
</div>
<LanguageMultiSelect
options={bookLanguages}
value={lang}
onChange={handleLangChange}
defaultLanguageCodes={defaultLanguage}
label="Language"
/>
<DropdownList
label="Content"
options={CONTENT_OPTIONS}
value={content}
onChange={handleContentChange}
placeholder="All"
/>
<div>
<DropdownList
label="Formats"
placeholder="Any"
options={formatOptions}
value={formats}
onChange={handleFormatsChange}
multiple
showCheckboxes
keepOpenOnSelect
/>
</div>
</form>
);
const wrappedForm = renderWrapper ? (
renderWrapper(form)
) : (
<div className="w-full border-b pt-6 pb-4 mb-4" style={{ borderColor: 'var(--border-muted)' }}>
<div className="w-full px-4 sm:px-6 lg:px-8">{form}</div>
</div>
);
return wrappedForm;
};
@@ -0,0 +1,254 @@
import { useEffect, useState, CSSProperties } from 'react';
import { ButtonStateInfo } from '../types';
interface CircularProgressProps {
progress?: number;
size?: number;
className?: string;
}
const CircularProgress = ({ progress, size = 16, className }: CircularProgressProps) => {
const radius = (size - 2) / 2;
const circumference = 2 * Math.PI * radius;
const progressValue = progress ?? 0;
const strokeDashoffset = circumference - (progressValue / 100) * circumference;
const svgClassName = className ? `transform -rotate-90 ${className}` : 'transform -rotate-90';
return (
<svg width={size} height={size} className={svgClassName}>
<circle
cx={size / 2}
cy={size / 2}
r={radius}
fill="none"
stroke="currentColor"
strokeWidth="2"
opacity="0.3"
/>
<circle
cx={size / 2}
cy={size / 2}
r={radius}
fill="none"
stroke="currentColor"
strokeWidth="2"
strokeDasharray={circumference}
strokeDashoffset={strokeDashoffset}
strokeLinecap="round"
style={{ transition: 'stroke-dashoffset 0.3s ease' }}
/>
</svg>
);
};
type ButtonSize = 'sm' | 'md';
type ButtonVariant = 'primary' | 'icon';
interface BookDownloadButtonProps {
buttonState: ButtonStateInfo;
onDownload: () => Promise<void>;
size?: ButtonSize;
fullWidth?: boolean;
className?: string;
showIcon?: boolean;
style?: CSSProperties;
variant?: ButtonVariant;
ariaLabel?: string;
}
const sizeClasses: Record<ButtonSize, string> = {
sm: 'px-2.5 py-1.5 text-xs',
md: 'px-4 py-2.5 text-sm',
};
const iconVariantSizeClasses: Record<ButtonSize, string> = {
sm: 'p-1 sm:p-1.5',
md: 'p-1.5 sm:p-2',
};
const primaryIconSizes: Record<ButtonSize, string> = {
sm: 'w-3.5 h-3.5',
md: 'w-4 h-4',
};
const iconVariantIconSizes: Record<ButtonSize, { mobile: string; desktop: string }> = {
sm: { mobile: 'w-3.5 h-3.5', desktop: 'w-4 h-4' },
md: { mobile: 'w-4 h-4', desktop: 'w-5 h-5' },
};
const iconVariantProgressSizes: Record<ButtonSize, { mobile: number; desktop: number }> = {
sm: { mobile: 14, desktop: 16 },
md: { mobile: 16, desktop: 20 },
};
export const BookDownloadButton = ({
buttonState,
onDownload,
size = 'md',
fullWidth = false,
className = '',
showIcon = false,
style,
variant = 'primary',
ariaLabel,
}: BookDownloadButtonProps) => {
const [isQueuing, setIsQueuing] = useState(false);
useEffect(() => {
if (isQueuing && buttonState.state !== 'download') {
setIsQueuing(false);
}
}, [buttonState.state, isQueuing]);
const isCompleted = buttonState.state === 'complete';
const hasError = buttonState.state === 'error';
const isInProgress = ['queued', 'resolving', 'downloading'].includes(buttonState.state);
const isDisabled = buttonState.state !== 'download' || isQueuing || isCompleted;
const displayText = isQueuing ? 'Queuing...' : buttonState.text;
const showCircularProgress = buttonState.state === 'downloading' && buttonState.progress !== undefined;
const showSpinner = (isInProgress && !showCircularProgress) || isQueuing;
const primaryStateClasses =
isCompleted
? 'bg-green-600 cursor-not-allowed'
: hasError
? 'bg-red-600 cursor-not-allowed opacity-75'
: isInProgress
? 'bg-gray-500 cursor-not-allowed opacity-75'
: 'bg-sky-700 hover:bg-sky-800';
const iconStateClasses =
isCompleted
? 'bg-green-600 text-white cursor-not-allowed'
: hasError
? 'bg-red-600 text-white cursor-not-allowed opacity-75'
: isInProgress
? 'bg-gray-500 text-white cursor-not-allowed opacity-75'
: 'text-gray-600 dark:text-gray-200 hover-action';
const stateClasses = variant === 'icon' ? iconStateClasses : primaryStateClasses;
const widthClasses = variant === 'primary' && fullWidth ? 'w-full' : '';
const baseClasses =
variant === 'icon'
? 'flex items-center justify-center rounded-full transition-all duration-200 disabled:opacity-80 disabled:cursor-not-allowed focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-offset-2 focus-visible:ring-sky-500'
: 'inline-flex items-center justify-center gap-1.5 rounded text-white transition-all duration-200 disabled:opacity-80 disabled:cursor-not-allowed focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-offset-2 focus-visible:ring-sky-500';
const sizeClass = variant === 'icon' ? iconVariantSizeClasses[size] : sizeClasses[size];
const iconSizes = variant === 'icon' ? iconVariantIconSizes[size] : undefined;
const handleDownload = async () => {
if (isDisabled) return;
setIsQueuing(true);
try {
await onDownload();
} catch (error) {
setIsQueuing(false);
}
};
const renderStatusIcon = () => {
if (isCompleted) {
if (variant === 'icon' && iconSizes) {
return (
<>
<svg className={`${iconSizes.mobile} sm:hidden`} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M5 13l4 4L19 7" />
</svg>
<svg className={`${iconSizes.desktop} hidden sm:block`} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M5 13l4 4L19 7" />
</svg>
</>
);
}
return (
<svg className={primaryIconSizes[size]} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M5 13l4 4L19 7" />
</svg>
);
}
if (hasError) {
if (variant === 'icon' && iconSizes) {
return (
<>
<svg className={`${iconSizes.mobile} sm:hidden`} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M6 18L18 6M6 6l12 12" />
</svg>
<svg className={`${iconSizes.desktop} hidden sm:block`} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M6 18L18 6M6 6l12 12" />
</svg>
</>
);
}
return (
<svg className={primaryIconSizes[size]} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M6 18L18 6M6 6l12 12" />
</svg>
);
}
if (showCircularProgress) {
if (variant === 'icon') {
const sizes = iconVariantProgressSizes[size];
return (
<>
<CircularProgress progress={buttonState.progress} size={sizes.mobile} className="block sm:hidden" />
<CircularProgress progress={buttonState.progress} size={sizes.desktop} className="hidden sm:block" />
</>
);
}
return <CircularProgress progress={buttonState.progress} size={size === 'sm' ? 12 : 16} />;
}
if (showSpinner) {
const spinnerClass =
variant === 'icon'
? size === 'sm'
? 'w-3.5 h-3.5 sm:w-4 h-4'
: 'w-4 h-4 sm:w-5 h-5'
: size === 'sm'
? 'w-3 h-3'
: 'w-4 h-4';
return <div className={`${spinnerClass} border-2 border-current border-t-transparent rounded-full animate-spin`} />;
}
if (variant === 'icon' && iconSizes) {
return (
<>
<svg className={`${iconSizes.mobile} sm:hidden`} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M3 16.5v2.25A2.25 2.25 0 0 0 5.25 21h13.5A2.25 2.25 0 0 0 21 18.75V16.5M16.5 12 12 16.5m0 0L7.5 12m4.5 4.5V3" />
</svg>
<svg className={`${iconSizes.desktop} hidden sm:block`} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={1.5} d="M3 16.5v2.25A2.25 2.25 0 0 0 5.25 21h13.5A2.25 2.25 0 0 0 21 18.75V16.5M16.5 12 12 16.5m0 0L7.5 12m4.5 4.5V3" />
</svg>
</>
);
}
return null;
};
return (
<button
className={`${baseClasses} ${sizeClass} ${stateClasses} ${widthClasses} ${className}`.trim()}
onClick={handleDownload}
disabled={isDisabled || isInProgress}
data-action="download"
style={style}
aria-label={ariaLabel ?? displayText}
>
{variant === 'primary' && showIcon && !isCompleted && !hasError && !showCircularProgress && !showSpinner && (
<svg className={primaryIconSizes[size]} fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 4v12m0 0l-4-4m4 4 4-4M6 20h12" />
</svg>
)}
{variant === 'primary' && <span className="download-button-text">{displayText}</span>}
{variant === 'icon' && <span className="sr-only">{ariaLabel ?? displayText}</span>}
{renderStatusIcon()}
</button>
);
};
@@ -0,0 +1,199 @@
import { useState, useEffect } from 'react';
import { Book, ButtonStateInfo } from '../types';
import { BookDownloadButton } from './BookDownloadButton';
interface DetailsModalProps {
book: Book | null;
onClose: () => void;
onDownload: (book: Book) => Promise<void>;
buttonState: ButtonStateInfo;
}
export const DetailsModal = ({ book, onClose, onDownload, buttonState }: DetailsModalProps) => {
const [isQueuing, setIsQueuing] = useState(false);
// Clear queuing state and close modal once button state changes from download
useEffect(() => {
if (isQueuing && buttonState.state !== 'download') {
setIsQueuing(false);
// Close modal after status has updated
const timer = setTimeout(onClose, 500);
return () => clearTimeout(timer);
}
}, [buttonState.state, isQueuing, onClose]);
// Handle ESC key to close modal
useEffect(() => {
const handleEscape = (e: KeyboardEvent) => {
if (e.key === 'Escape') {
onClose();
}
};
document.addEventListener('keydown', handleEscape);
return () => document.removeEventListener('keydown', handleEscape);
}, [onClose]);
useEffect(() => {
const previousOverflow = document.body.style.overflow;
document.body.style.overflow = 'hidden';
return () => {
document.body.style.overflow = previousOverflow;
};
}, []);
if (!book) return null;
const titleId = `book-details-title-${book.id}`;
const handleDownload = async () => {
setIsQueuing(true);
try {
await onDownload(book);
// Don't close here - wait for button state to change
} catch (error) {
setIsQueuing(false);
// Close on error
setTimeout(onClose, 300);
}
};
const publisherInfo = { label: 'Publisher', value: book.publisher || '-' };
const metadata = [
{ label: 'Year', value: book.year || '-' },
{ label: 'Language', value: book.language || '-' },
{ label: 'Format', value: book.format || '-' },
{ label: 'Size', value: book.size || '-' },
];
const artworkMaxHeight = 'calc(90vh - 220px)';
const artworkMaxWidth = 'min(45vw, 520px, calc((90vh - 220px) / 1.6))';
const additionalInfo =
book.info && Object.keys(book.info).length > 0
? Object.entries(book.info).filter(([key]) => {
const normalized = key.toLowerCase();
return normalized !== 'language' && normalized !== 'year';
})
: [];
const extendedInfoEntries = [[publisherInfo.label, publisherInfo.value], ...additionalInfo];
const infoCardClass = 'rounded-2xl border border-[var(--border-muted)] px-4 py-3 text-sm';
const infoCardStyle = { background: 'var(--bg)' };
const infoLabelClass = 'text-[11px] uppercase tracking-wide text-gray-500 dark:text-gray-400';
const infoValueClass = 'text-gray-900 dark:text-gray-100';
return (
<div
className="modal-overlay active px-4 py-6 sm:px-6"
onClick={e => {
if (e.target === e.currentTarget) onClose();
}}
>
<div
className="details-container w-full max-w-4xl animate-fade-in-up"
role="dialog"
aria-modal="true"
aria-labelledby={titleId}
>
<div className="flex max-h-[90vh] flex-col overflow-hidden rounded-2xl border border-[var(--border-muted)] bg-[var(--bg-soft)] text-[var(--text)] shadow-2xl">
<header className="flex items-start gap-4 border-b border-[var(--border-muted)] px-5 py-4">
<div className="flex-1 space-y-1">
<p className="text-xs uppercase tracking-wide text-gray-500 dark:text-gray-400">Book</p>
<h3 id={titleId} className="text-lg font-semibold leading-snug">
{book.title || 'Untitled'}
</h3>
<p className="text-sm text-gray-600 dark:text-gray-300">
{book.author || 'Unknown author'}
</p>
</div>
<button
type="button"
onClick={onClose}
className="rounded-full p-2 text-gray-500 transition-colors hover-action hover:text-gray-900 dark:hover:text-gray-100"
aria-label="Close details"
>
<svg
className="h-5 w-5"
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
strokeWidth={1.5}
>
<path strokeLinecap="round" strokeLinejoin="round" d="M6 18L18 6M6 6l12 12" />
</svg>
</button>
</header>
<div className="flex-1 min-h-0 overflow-y-auto px-5 py-6">
<div className="flex flex-col gap-6 lg:flex-row lg:items-stretch lg:gap-8 lg:min-h-0">
<div className="flex w-full justify-center lg:w-auto lg:flex-none lg:justify-start lg:self-stretch lg:pr-4">
{book.preview ? (
<div
className="flex w-full items-center justify-center lg:h-full lg:max-w-none"
style={{ maxHeight: artworkMaxHeight, maxWidth: artworkMaxWidth }}
>
<img
src={book.preview}
alt="Book cover"
className="h-auto max-h-full w-auto max-w-full rounded-xl object-contain shadow-lg"
style={{ maxHeight: '100%', maxWidth: '100%' }}
/>
</div>
) : (
<div
className="flex w-full items-center justify-center rounded-xl border border-dashed border-[var(--border-muted)] bg-[var(--bg)]/60 p-6 text-sm text-gray-500 lg:h-full lg:max-w-none"
style={{ maxHeight: artworkMaxHeight, maxWidth: artworkMaxWidth }}
>
No cover
</div>
)}
</div>
<div className="flex flex-1 flex-col gap-4 sm:gap-5 lg:min-h-0">
{book.description && (
<div className={`${infoCardClass} space-y-1`} style={infoCardStyle}>
<p className={infoLabelClass}>Description</p>
<p className={`${infoValueClass} whitespace-pre-line`}>{book.description}</p>
</div>
)}
<div className="grid grid-cols-2 gap-3 sm:grid-cols-2 lg:grid-cols-4 lg:gap-4">
{metadata.map(item => (
<div key={item.label} className={`${infoCardClass} space-y-1`} style={infoCardStyle}>
<p className={infoLabelClass}>{item.label}</p>
<p className={infoValueClass}>{item.value}</p>
</div>
))}
</div>
{extendedInfoEntries.length > 0 && (
<div className={`${infoCardClass} space-y-3`} style={infoCardStyle}>
<ul className="space-y-3 list-none">
{extendedInfoEntries.map(([key, value]) => (
<li key={key} className="space-y-1">
<p className={infoLabelClass}>{key}</p>
<p className={infoValueClass}>{Array.isArray(value) ? value.join(', ') : value}</p>
</li>
))}
</ul>
</div>
)}
</div>
</div>
</div>
<footer className="border-t border-[var(--border-muted)] bg-[var(--bg-soft)] px-5 py-4">
<div className="flex justify-end">
<BookDownloadButton
buttonState={buttonState}
onDownload={handleDownload}
size="md"
fullWidth
className="rounded-full px-4 py-3 text-sm font-medium"
ariaLabel={`Download ${book.title || 'book'}`}
/>
</div>
</footer>
</div>
</div>
</div>
);
};
@@ -0,0 +1,374 @@
import { useEffect } from 'react';
import { StatusData, Book } from '../types';
interface DownloadsSidebarProps {
isOpen: boolean;
onClose: () => void;
status: StatusData;
onRefresh: () => void;
onClearCompleted: () => void;
onCancel: (id: string) => void;
activeCount: number;
}
const STATUS_STYLES: Record<string, { bg: string; text: string; label: string; waveColor: string }> = {
queued: { bg: 'bg-amber-500/20', text: 'text-amber-700 dark:text-amber-300', label: 'Queued', waveColor: 'rgba(217, 119, 6, 0.3)' },
resolving: { bg: 'bg-indigo-500/20', text: 'text-indigo-700 dark:text-indigo-300', label: 'Resolving', waveColor: 'rgba(79, 70, 229, 0.3)' },
downloading: { bg: 'bg-sky-500/20', text: 'text-sky-700 dark:text-sky-300', label: 'Downloading', waveColor: 'rgba(2, 132, 199, 0.3)' },
complete: { bg: 'bg-green-500/20', text: 'text-green-700 dark:text-green-300', label: 'Complete', waveColor: '' },
error: { bg: 'bg-red-500/20', text: 'text-red-700 dark:text-red-300', label: 'Error', waveColor: '' },
cancelled: { bg: 'bg-gray-500/20', text: 'text-gray-700 dark:text-gray-300', label: 'Cancelled', waveColor: '' },
};
// Add keyframe animation for wave effect
const styleSheet = document.createElement('style');
styleSheet.textContent = `
@keyframes wave {
0% { background-position: 200% 0; }
100% { background-position: -200% 0; }
}
`;
if (!document.head.querySelector('style[data-wave-animation]')) {
styleSheet.setAttribute('data-wave-animation', 'true');
document.head.appendChild(styleSheet);
}
// Helper to get book preview image
const getBookPreview = (book: Book): string => {
return book.preview || '/placeholder-book.png';
};
// Helper to get progress percentage based on status
const getStatusProgress = (statusName: string, bookProgress?: number): number => {
switch (statusName) {
case 'queued':
return 5;
case 'resolving':
return 15;
case 'downloading':
// Map actual progress (0-100) to 20-100 range
if (typeof bookProgress === 'number') {
return 20 + (bookProgress * 0.8);
}
return 20;
case 'complete':
case 'error':
return 100;
default:
return 0;
}
};
// Helper to get progress bar color based on status
const getProgressBarColor = (statusName: string): string => {
if (statusName === 'complete') return 'bg-green-600';
if (statusName === 'error') return 'bg-red-600';
if (statusName === 'queued') return 'bg-amber-600';
if (statusName === 'resolving') return 'bg-indigo-600';
if (statusName === 'downloading') return 'bg-sky-600';
return 'bg-sky-600';
};
export const DownloadsSidebar = ({
isOpen,
onClose,
status,
onRefresh,
onClearCompleted,
onCancel,
activeCount,
}: DownloadsSidebarProps) => {
// Handle ESC key to close sidebar
useEffect(() => {
if (!isOpen) return; // Only listen when sidebar is open
const handleEscape = (e: KeyboardEvent) => {
if (e.key === 'Escape') {
onClose();
}
};
document.addEventListener('keydown', handleEscape);
return () => document.removeEventListener('keydown', handleEscape);
}, [isOpen, onClose]);
// Collect all download items from different status sections
const allDownloadItems: Array<{ book: Book; status: string }> = [];
const statusTypes = ['downloading', 'resolving', 'queued', 'error', 'complete', 'cancelled'];
statusTypes.forEach((statusName) => {
const items = (status as any)[statusName];
if (items && Object.keys(items).length > 0) {
Object.values(items).forEach((book: any) => {
allDownloadItems.push({ book, status: statusName });
});
}
});
// Sort by added_time descending (newest first)
allDownloadItems.sort((a, b) => (b.book.added_time || 0) - (a.book.added_time || 0));
const renderDownloadItem = (item: { book: Book; status: string }) => {
const { book, status: statusName } = item;
const statusStyle = STATUS_STYLES[statusName] || {
bg: 'bg-gray-500/10',
text: 'text-gray-600',
label: statusName.charAt(0).toUpperCase() + statusName.slice(1),
};
const isInProgress = ['queued', 'resolving', 'downloading'].includes(statusName);
const isCompleted = statusName === 'complete';
const hasError = statusName === 'error';
// Get progress information
const progress = getStatusProgress(statusName, book.progress);
const progressBarColor = getProgressBarColor(statusName);
// Format progress text - use status_message if available, otherwise fall back to label
let progressText = book.status_message || statusStyle.label;
if (statusName === 'downloading' && book.progress && book.size) {
const sizeValue = parseFloat(book.size.replace(/[^\d.]/g, ''));
const sizeUnit = book.size.replace(/[\d.\s]/g, ''); // Extract unit as-is from backend
const downloadedSize = (book.progress / 100) * sizeValue;
const sizeProgress = `${downloadedSize.toFixed(1)}${sizeUnit} / ${book.size}`;
// If there's attempt info in the status message, prepend it to the progress
if (book.status_message?.startsWith('Attempt')) {
progressText = `${book.status_message} - ${sizeProgress}`;
} else {
progressText = sizeProgress;
}
} else if (isCompleted) {
progressText = 'Complete';
} else if (hasError) {
progressText = book.status_message || 'Failed';
}
return (
<div
key={book.id}
className="relative rounded-lg border hover:shadow-md transition-shadow overflow-hidden"
style={{ borderColor: 'var(--border-muted)', background: 'var(--bg-soft)' }}
>
{/* Cancel/Clear Button - top right corner */}
<button
type="button"
onClick={(e) => {
e.stopPropagation();
onCancel(book.id);
}}
className="absolute top-1 right-1 z-10 flex items-center justify-center w-6 h-6 rounded-full hover:bg-red-100 dark:hover:bg-red-900/30 text-gray-500 hover:text-red-600 transition-colors"
title={isInProgress ? "Cancel download" : "Clear from list"}
aria-label={isInProgress ? "Cancel download" : "Clear from list"}
>
<svg className="w-3.5 h-3.5" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2.5">
<path strokeLinecap="round" strokeLinejoin="round" d="M6 18L18 6M6 6l12 12" />
</svg>
</button>
{/* Main content area */}
<div className="flex gap-2">
{/* Book Thumbnail - left side */}
<div className="flex-shrink-0">
<img
src={getBookPreview(book)}
alt={book.title || 'Book cover'}
className="w-16 h-24 object-cover rounded shadow-sm"
style={{ aspectRatio: '2/3' }}
onError={(e) => {
const target = e.target as HTMLImageElement;
target.src = '/placeholder-book.png';
}}
/>
</div>
{/* Book Info - right side */}
<div className="flex-1 min-w-0 flex flex-col justify-between px-3 pt-2 pb-3">
{/* Title & Author - with safe area for cancel/clear button */}
<div className="mb-1 pr-6">
<h3 className="font-semibold text-sm truncate" title={book.title}>
{isCompleted && book.download_path ? (
<a
href={`/api/localdownload?id=${encodeURIComponent(book.id)}`}
className="text-sky-600 hover:underline"
>
{book.title || 'Unknown Title'}
</a>
) : (
book.title || 'Unknown Title'
)}
</h3>
<p className="text-xs opacity-70 truncate" title={book.author}>
{book.author || 'Unknown Author'}
</p>
</div>
{/* Details Row */}
<div className="space-y-1 pb-8">
<div className="flex items-center gap-2">
{/* Format and Size */}
<div className="text-xs opacity-70">
{book.format && <span className="uppercase">{book.format}</span>}
{book.format && book.size && <span> • </span>}
{book.size && <span>{book.size}</span>}
</div>
</div>
</div>
</div>
</div>
{/* Progress Bar - absolute positioned at bottom - always visible */}
<div className="absolute bottom-0 left-0 right-0 pointer-events-none">
{/* ml-16 clears the 64px thumbnail, gap-2 adds spacing */}
<div className="flex justify-end p-2 ml-16 gap-2">
<span
className={`relative px-2 py-0.5 rounded-lg text-xs font-medium text-right ${statusStyle.bg} ${statusStyle.text}`}
>
{/* Wave animation overlay for in-progress states */}
{isInProgress && statusStyle.waveColor && (
<span
key={statusName}
className="absolute inset-0 rounded-lg"
style={{
background: `linear-gradient(90deg, transparent 0%, ${statusStyle.waveColor} 50%, transparent 100%)`,
backgroundSize: '200% 100%',
animation: 'wave 2s linear infinite',
}}
/>
)}
<span className="relative">{progressText}</span>
</span>
</div>
<div className="h-1.5 bg-gray-200 dark:bg-gray-700 overflow-hidden relative">
<div
className={`h-full ${progressBarColor} transition-all duration-300 relative overflow-hidden`}
style={{ width: `${Math.min(100, Math.max(0, progress))}%` }}
>
{/* Animated wave effect for in-progress states */}
{isInProgress && progress < 100 && (
<div
className="absolute inset-0 opacity-30"
style={{
background: 'linear-gradient(90deg, transparent 0%, rgba(255, 255, 255, 0.5) 50%, transparent 100%)',
backgroundSize: '200% 100%',
animation: 'wave 2s ease-in-out infinite',
}}
/>
)}
</div>
</div>
</div>
</div>
);
};
return (
<>
{/* Backdrop */}
<div
className={`fixed inset-0 bg-black/50 z-40 transition-opacity duration-300 ${
isOpen ? 'opacity-100' : 'opacity-0 pointer-events-none'
}`}
onClick={onClose}
/>
{/* Sidebar */}
<div
className={`fixed top-0 right-0 h-full w-full sm:w-96 z-50 flex flex-col shadow-2xl transition-transform duration-300 ${
isOpen ? 'translate-x-0' : 'translate-x-full'
}`}
style={{ background: 'var(--bg)' }}
>
{/* Header */}
<div
className="flex items-center justify-between p-4"
style={{ paddingTop: 'calc(1rem + env(safe-area-inset-top))' }}
>
<h2 className="text-lg font-semibold">Downloads</h2>
<button
type="button"
onClick={onClose}
className="flex h-10 w-10 items-center justify-center rounded-full hover-action transition-colors"
aria-label="Close sidebar"
>
<svg
className="w-5 h-5"
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth="2"
stroke="currentColor"
>
<path strokeLinecap="round" strokeLinejoin="round" d="M6 18L18 6M6 6l12 12" />
</svg>
</button>
</div>
{/* Controls */}
<div
className="flex items-center gap-2 p-4 border-b"
style={{ borderColor: 'var(--border-muted)' }}
>
<button
type="button"
onClick={onClearCompleted}
className="flex-1 flex items-center justify-center px-3 py-2 h-10 rounded border text-sm hover-action transition-colors"
style={{ borderColor: 'var(--border-muted)' }}
>
Clear Completed
</button>
<button
type="button"
onClick={onRefresh}
className="flex items-center justify-center h-10 w-10 rounded-full text-sm hover-action transition-colors ml-auto"
aria-label="Refresh"
title="Refresh"
>
<svg
className="w-4 h-4"
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth="2"
stroke="currentColor"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M16.023 9.348h4.992v-.001M2.985 19.644v-4.992m0 0h4.992m-4.993 0l3.181 3.183a8.25 8.25 0 0013.803-3.7M4.031 9.865a8.25 8.25 0 0113.803-3.7l3.181 3.182m0-4.991v4.99"
/>
</svg>
</button>
</div>
{/* Queue Items */}
<div
className="flex-1 overflow-y-auto p-4 space-y-3"
style={{ paddingBottom: 'calc(1rem + env(safe-area-inset-bottom))' }}
>
{allDownloadItems.length > 0 ? (
allDownloadItems.map((item) => renderDownloadItem(item))
) : (
<div className="text-center text-sm opacity-70 mt-8">
No downloads in queue
</div>
)}
</div>
{/* Footer with active count */}
{activeCount > 0 && (
<div
className="p-3 border-t text-xs text-center opacity-70"
style={{
borderColor: 'var(--border-muted)',
paddingBottom: 'calc(0.75rem + env(safe-area-inset-bottom))',
}}
>
{activeCount} active {activeCount === 1 ? 'download' : 'downloads'}
</div>
)}
</div>
</>
);
};
+144
View File
@@ -0,0 +1,144 @@
import { ReactNode, useEffect, useLayoutEffect, useRef, useState } from 'react';
interface DropdownProps {
label?: string;
summary?: ReactNode;
children: (helpers: { close: () => void }) => ReactNode;
align?: 'left' | 'right';
widthClassName?: string;
buttonClassName?: string;
panelClassName?: string;
disabled?: boolean;
renderTrigger?: (props: { isOpen: boolean; toggle: () => void }) => ReactNode;
}
export const Dropdown = ({
label,
summary,
children,
align = 'left',
widthClassName = 'w-full',
buttonClassName = '',
panelClassName = '',
disabled = false,
renderTrigger,
}: DropdownProps) => {
const [isOpen, setIsOpen] = useState(false);
const containerRef = useRef<HTMLDivElement>(null);
const panelRef = useRef<HTMLDivElement>(null);
const [panelDirection, setPanelDirection] = useState<'down' | 'up'>('down');
const toggleOpen = () => {
if (disabled) return;
setIsOpen(prev => !prev);
};
const close = () => setIsOpen(false);
useEffect(() => {
if (!isOpen) return;
const handleClick = (event: MouseEvent) => {
if (containerRef.current && !containerRef.current.contains(event.target as Node)) {
close();
}
};
const handleEscape = (event: KeyboardEvent) => {
if (event.key === 'Escape') {
close();
}
};
document.addEventListener('mousedown', handleClick);
document.addEventListener('keydown', handleEscape);
return () => {
document.removeEventListener('mousedown', handleClick);
document.removeEventListener('keydown', handleEscape);
};
}, [isOpen]);
useLayoutEffect(() => {
if (!isOpen) return;
const updatePanelDirection = () => {
if (!containerRef.current || !panelRef.current) {
return;
}
const rect = containerRef.current.getBoundingClientRect();
const panelHeight = panelRef.current.offsetHeight || panelRef.current.scrollHeight;
const spaceBelow = window.innerHeight - rect.bottom - 8;
const spaceAbove = rect.top - 8;
const shouldOpenUp = spaceBelow < panelHeight && spaceAbove >= panelHeight;
setPanelDirection(shouldOpenUp ? 'up' : 'down');
};
updatePanelDirection();
window.addEventListener('resize', updatePanelDirection);
window.addEventListener('scroll', updatePanelDirection, true);
return () => {
window.removeEventListener('resize', updatePanelDirection);
window.removeEventListener('scroll', updatePanelDirection, true);
};
}, [isOpen]);
return (
<div className={`relative ${widthClassName}`} ref={containerRef}>
{label && (
<label className="block text-sm mb-1 opacity-80" onClick={toggleOpen}>
{label}
</label>
)}
{renderTrigger ? (
renderTrigger({ isOpen, toggle: toggleOpen })
) : (
<button
type="button"
onClick={toggleOpen}
disabled={disabled}
className={`w-full px-3 py-2 rounded-md border flex items-center justify-between text-left text-base focus:outline-none focus-visible:outline-none focus-visible:ring-0 focus-visible:ring-offset-0 ${buttonClassName}`}
style={{
background: 'var(--bg-soft)',
color: 'var(--text)',
borderColor: 'var(--border-muted)',
}}
>
<span className="truncate text-base">
{summary ?? <span className="opacity-60">Select an option</span>}
</span>
<svg
className={`w-4 h-4 transition-transform ${isOpen ? 'rotate-180' : ''}`}
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
strokeWidth="1.5"
>
<path strokeLinecap="round" strokeLinejoin="round" d="m19.5 8.25-7.5 7.5-7.5-7.5" />
</svg>
</button>
)}
{isOpen && (
<div
ref={panelRef}
className={`absolute ${align === 'right' ? 'right-0' : 'left-0'} ${
panelDirection === 'down' ? 'mt-2' : 'bottom-full mb-2'
} rounded-md border shadow-lg z-20 ${panelClassName || widthClassName}`}
style={{
background: 'var(--bg)',
borderColor: 'var(--border-muted)',
}}
>
<div className="max-h-64 overflow-auto">
{children({ close })}
</div>
</div>
)}
</div>
);
};
@@ -0,0 +1,155 @@
import { ReactNode } from 'react';
import { Dropdown } from './Dropdown';
export interface DropdownListOption {
value: string;
label: string;
description?: string;
disabled?: boolean;
icon?: ReactNode;
}
interface DropdownListProps {
label?: string;
placeholder?: string;
options: DropdownListOption[];
multiple?: boolean;
showCheckboxes?: boolean;
value: string[] | string | null | undefined;
onChange: (value: string[] | string) => void;
align?: 'left' | 'right';
widthClassName?: string;
buttonClassName?: string;
summaryFormatter?: (selected: DropdownListOption[], placeholder: string) => ReactNode;
keepOpenOnSelect?: boolean;
}
export const DropdownList = ({
label,
placeholder = 'Select an option',
options,
multiple = false,
showCheckboxes,
value,
onChange,
align,
widthClassName,
buttonClassName,
summaryFormatter,
keepOpenOnSelect,
}: DropdownListProps) => {
const selectedValues = normalizeValue(value, multiple);
const selectedOptions = options.filter(opt => selectedValues.includes(opt.value));
const checkboxEnabled = showCheckboxes ?? multiple;
const stayOpenOnSelect = keepOpenOnSelect ?? multiple;
const renderSummary = () => {
if (summaryFormatter) {
return summaryFormatter(selectedOptions, placeholder);
}
if (selectedOptions.length === 0) {
return <span className="opacity-60 text-base">{placeholder}</span>;
}
if (!multiple) {
return selectedOptions[0]?.label ?? placeholder;
}
if (selectedOptions.length === 1) {
return selectedOptions[0].label;
}
const [first, second, ...rest] = selectedOptions.map(opt => opt.label);
const suffix = rest.length > 0 ? ` +${rest.length}` : '';
return `${first}, ${second ?? ''}${suffix}`.trim();
};
const handleOptionClick = (option: DropdownListOption, close: () => void) => {
if (option.disabled) return;
if (multiple) {
const next = selectedValues.includes(option.value)
? selectedValues.filter(v => v !== option.value)
: [...selectedValues, option.value];
onChange(next);
if (!stayOpenOnSelect) {
close();
}
return;
}
if (selectedValues[0] === option.value) {
close();
return;
}
onChange(option.value);
close();
};
return (
<Dropdown
label={label}
summary={renderSummary()}
align={align}
widthClassName={widthClassName}
buttonClassName={buttonClassName}
>
{({ close }) => (
<div role="listbox" aria-multiselectable={multiple}>
{options.map(option => (
<button
type="button"
key={option.value}
className={`w-full px-3 py-2 text-left text-base flex items-center gap-2 hover-surface ${
option.disabled ? 'opacity-50 cursor-not-allowed' : ''
}`}
onClick={() => handleOptionClick(option, close)}
disabled={option.disabled}
>
{checkboxEnabled && (
<input
type="checkbox"
checked={selectedValues.includes(option.value)}
readOnly
className="h-4 w-4 rounded border-gray-300 text-sky-600 focus:ring-sky-500 pointer-events-none"
/>
)}
{option.icon}
<div className="flex flex-col">
<span className="text-base">{option.label}</span>
{option.description && (
<span className="text-xs opacity-70">{option.description}</span>
)}
</div>
</button>
))}
</div>
)}
</Dropdown>
);
};
const normalizeValue = (value: string[] | string | null | undefined, multiple: boolean): string[] => {
if (multiple) {
if (Array.isArray(value)) {
return value;
}
if (typeof value === 'string') {
return [value];
}
return [];
}
if (Array.isArray(value)) {
return value.length ? [value[0]] : [];
}
if (typeof value === 'string' && value) {
return [value];
}
return [];
};
+48
View File
@@ -0,0 +1,48 @@
interface FooterProps {
buildVersion?: string;
releaseVersion?: string;
debug?: boolean;
}
export const Footer = ({ buildVersion, releaseVersion, debug }: FooterProps) => {
// Determine version display - show "dev" if no version is set
const versionDisplay = releaseVersion && releaseVersion !== 'N/A'
? releaseVersion
: 'dev';
return (
<footer
className="mt-8 border-t py-6"
style={{
borderColor: 'var(--border-muted)',
paddingBottom: 'calc(1.5rem + env(safe-area-inset-bottom))',
}}
>
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 flex flex-wrap items-center justify-between gap-3">
<div className="min-w-0 text-left">
<p className="text-sm opacity-80">Calibre Web Book Downloader</p>
<p className="text-xs opacity-60 mt-1">
Version: {versionDisplay}
{buildVersion && buildVersion !== 'N/A' && ` (${buildVersion})`}
{debug && ' • Debug Mode'}
</p>
</div>
<div className="flex items-center gap-3 shrink-0">
<a
href="https://github.com/calibrain/calibre-web-automated-book-downloader"
className="opacity-80 hover:opacity-100"
aria-label="GitHub"
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="currentColor"
className="w-6 h-6"
>
<path d="M12 1C5.923 1 1 5.923 1 12c0 4.867 3.149 8.979 7.521 10.436.55.096.756-.233.756-.522 0-.262-.013-1.128-.013-2.049-2.764.509-3.479-.674-3.699-1.292-.124-.317-.66-1.293-1.127-1.554-.385-.207-.936-.715-.014-.729.866-.014 1.485.797 1.691 1.128.99 1.663 2.571 1.196 3.204.907.096-.715.385-1.196.701-1.471-2.448-.275-5.005-1.224-5.005-5.432 0-1.196.426-2.186 1.128-2.956-.111-.275-.496-1.402.11-2.915 0 0 .921-.288 3.024 1.128a10.193 10.193 0 0 1 2.75-.371c.936 0 1.871.123 2.75.371 2.104-1.43 3.025-1.128 3.025-1.128.605 1.513.221 2.64.111 2.915.701.77 1.127 1.747 1.127 2.956 0 4.222-2.571 5.157-5.019 5.432.399.344.743 1.004.743 2.035 0 1.471-.014 2.654-.014 3.025 0 .289.206.632.756.522C19.851 20.979 23 16.854 23 12c0-6.077-4.922-11-11-11Z"></path>
</svg>
</a>
</div>
</div>
</footer>
);
};
+459
View File
@@ -0,0 +1,459 @@
import { useState, useEffect, useRef } from 'react';
import { SearchBar } from './SearchBar';
interface StatusCounts {
ongoing: number;
completed: number;
errored: number;
}
interface HeaderProps {
calibreWebUrl?: string;
debug?: boolean;
logoUrl?: string;
showSearch?: boolean;
searchInput?: string;
onSearchChange?: (value: string) => void;
onSearch?: () => void;
onAdvancedToggle?: () => void;
isLoading?: boolean;
onDownloadsClick?: () => void;
statusCounts?: StatusCounts;
onLogoClick?: () => void;
authRequired?: boolean;
isAuthenticated?: boolean;
onLogout?: () => void;
onShowToast?: (message: string, type: 'success' | 'error' | 'info', persistent?: boolean) => string;
onRemoveToast?: (id: string) => void;
}
export const Header = ({
calibreWebUrl,
debug,
logoUrl,
showSearch = false,
searchInput = '',
onSearchChange,
onSearch,
onAdvancedToggle,
isLoading = false,
onDownloadsClick,
statusCounts = { ongoing: 0, completed: 0, errored: 0 },
onLogoClick,
authRequired = false,
isAuthenticated = false,
onLogout,
onShowToast,
onRemoveToast,
}: HeaderProps) => {
const [theme, setTheme] = useState<string>('auto');
const [isDropdownOpen, setIsDropdownOpen] = useState(false);
const [isClosing, setIsClosing] = useState(false);
const [shouldAnimateIn, setShouldAnimateIn] = useState(false);
const dropdownRef = useRef<HTMLDivElement>(null);
useEffect(() => {
const saved = localStorage.getItem('preferred-theme') || 'auto';
setTheme(saved);
applyTheme(saved);
// Remove preload class after initial theme is applied to enable transitions
requestAnimationFrame(() => {
document.documentElement.classList.remove('preload');
});
}, []);
useEffect(() => {
const mq = window.matchMedia('(prefers-color-scheme: dark)');
const handler = (e: MediaQueryListEvent) => {
if (localStorage.getItem('preferred-theme') === 'auto') {
document.documentElement.setAttribute('data-theme', e.matches ? 'dark' : 'light');
}
};
mq.addEventListener('change', handler);
return () => mq.removeEventListener('change', handler);
}, []);
// Helper function to close dropdown with animation
const closeDropdown = () => {
setIsClosing(true);
setTimeout(() => {
setIsDropdownOpen(false);
setIsClosing(false);
}, 150); // Match the animation duration
};
// Close dropdown when clicking outside or pressing ESC
useEffect(() => {
const handleClickOutside = (event: MouseEvent) => {
if (dropdownRef.current && !dropdownRef.current.contains(event.target as Node)) {
closeDropdown();
}
};
const handleEscapeKey = (event: KeyboardEvent) => {
if (event.key === 'Escape') {
closeDropdown();
}
};
if (isDropdownOpen && !isClosing) {
document.addEventListener('mousedown', handleClickOutside);
document.addEventListener('keydown', handleEscapeKey);
}
return () => {
document.removeEventListener('mousedown', handleClickOutside);
document.removeEventListener('keydown', handleEscapeKey);
};
}, [isDropdownOpen, isClosing]);
const applyTheme = (pref: string) => {
if (pref === 'auto') {
const isDark = window.matchMedia('(prefers-color-scheme: dark)').matches;
document.documentElement.setAttribute('data-theme', isDark ? 'dark' : 'light');
} else {
document.documentElement.setAttribute('data-theme', pref);
}
};
const handleThemeChange = (newTheme: string) => {
localStorage.setItem('preferred-theme', newTheme);
setTheme(newTheme);
applyTheme(newTheme);
};
const cycleTheme = () => {
const themeOrder = ['light', 'dark', 'auto'];
const currentIndex = themeOrder.indexOf(theme);
const nextIndex = (currentIndex + 1) % themeOrder.length;
handleThemeChange(themeOrder[nextIndex]);
};
const handleLogout = () => {
closeDropdown();
onLogout?.();
};
const toggleDropdown = () => {
if (isDropdownOpen) {
closeDropdown();
} else {
setShouldAnimateIn(true);
setIsDropdownOpen(true);
// Reset animation flag after animation completes
setTimeout(() => setShouldAnimateIn(false), 200);
}
};
const handleHeaderSearch = () => {
onSearch?.();
};
const handleSearchChange = (value: string) => {
onSearchChange?.(value);
};
// Icon buttons component - reused for both states
const IconButtons = () => (
<div className="flex items-center gap-2">
{/* Calibre-Web Button */}
{calibreWebUrl && (
<a
href={calibreWebUrl}
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-2 px-3 py-2 rounded-full hover-action transition-all duration-200 text-gray-900 dark:text-gray-100"
aria-label="Open Calibre-Web"
title="Go To Library"
>
<svg className="w-5 h-5" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" strokeWidth="1.5" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" d="M12 6.042A8.967 8.967 0 006 3.75c-1.052 0-2.062.18-3 .512v14.25A8.987 8.987 0 016 18c2.305 0 4.408.867 6 2.292m0-14.25a8.966 8.966 0 016-2.292c1.052 0 2.062.18 3 .512v14.25A8.987 8.987 0 0018 18a8.967 8.967 0 00-6 2.292m0-14.25v14.25" />
</svg>
<span className="text-sm font-medium">Go To Library</span>
</a>
)}
{/* Downloads Button */}
{onDownloadsClick && (
<button
onClick={onDownloadsClick}
className="relative flex items-center gap-2 px-3 py-2 rounded-full hover-action transition-all duration-200 text-gray-900 dark:text-gray-100"
aria-label="View downloads"
title="Downloads"
>
<div className="relative">
<svg
className="w-5 h-5"
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth="1.5"
stroke="currentColor"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M3 16.5v2.25A2.25 2.25 0 005.25 21h13.5A2.25 2.25 0 0021 18.75V16.5M16.5 12L12 16.5m0 0L7.5 12m4.5 4.5V3"
/>
</svg>
{/* Show badge with appropriate color based on status */}
{(statusCounts.ongoing > 0 || statusCounts.completed > 0 || statusCounts.errored > 0) && (
<span
className={`absolute -top-1 -right-1 text-white text-[0.55rem] font-bold rounded-full w-3.5 h-3.5 flex items-center justify-center ${
statusCounts.errored > 0
? 'bg-red-500'
: statusCounts.ongoing > 0
? 'bg-blue-500'
: 'bg-green-500'
}`}
title={`${statusCounts.ongoing} ongoing, ${statusCounts.completed} completed, ${statusCounts.errored} failed`}
>
{statusCounts.ongoing + statusCounts.completed + statusCounts.errored}
</span>
)}
</div>
<span className="hidden sm:inline text-sm font-medium">Downloads</span>
</button>
)}
{/* User Menu Dropdown */}
<div className="relative" ref={dropdownRef}>
<button
onClick={toggleDropdown}
className={`relative p-2 rounded-full hover-action transition-colors ${
isDropdownOpen ? 'bg-gray-100 dark:bg-gray-700' : ''
}`}
aria-label="User menu"
aria-expanded={isDropdownOpen}
aria-haspopup="true"
>
<svg
className="w-5 h-5"
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth="1.5"
stroke="currentColor"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M3.75 6.75h16.5M3.75 12h16.5m-16.5 5.25h16.5"
/>
</svg>
</button>
{/* Dropdown Menu */}
{(isDropdownOpen || isClosing) && (
<div
className={`absolute right-0 mt-2 w-48 rounded-lg shadow-lg border z-50 ${
isClosing ? 'animate-fade-out-up' : shouldAnimateIn ? 'animate-fade-in-down' : ''
}`}
style={{
background: 'var(--bg)',
borderColor: 'var(--border-muted)',
}}
>
<div className="py-1">
{/* Theme Button */}
<button
type="button"
onClick={cycleTheme}
className="w-full text-left px-4 py-2 hover-surface transition-colors flex items-center gap-3"
>
{theme === 'light' && (
<svg className="w-5 h-5" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" strokeWidth="1.5" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" d="M12 3v2.25m6.364.386l-1.591 1.591M21 12h-2.25m-.386 6.364l-1.591-1.591M12 18.75V21m-4.773-4.227l-1.591 1.591M5.25 12H3m4.227-4.773L5.636 5.636M15.75 12a3.75 3.75 0 11-7.5 0 3.75 3.75 0 017.5 0z" />
</svg>
)}
{theme === 'dark' && (
<svg className="w-5 h-5" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" strokeWidth="1.5" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" d="M21.752 15.002A9.718 9.718 0 0118 15.75c-5.385 0-9.75-4.365-9.75-9.75 0-1.33.266-2.597.748-3.752A9.753 9.753 0 003 11.25C3 16.635 7.365 21 12.75 21a9.753 9.753 0 009.002-5.998z" />
</svg>
)}
{theme === 'auto' && (
<svg className="w-5 h-5" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" strokeWidth="1.5" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" d="M9 17.25v1.007a3 3 0 01-.879 2.122L7.5 21h9l-.621-.621A3 3 0 0115 18.257V17.25m6-12V15a2.25 2.25 0 01-2.25 2.25H5.25A2.25 2.25 0 013 15V5.25m18 0A2.25 2.25 0 0018.75 3H5.25A2.25 2.25 0 003 5.25m18 0V12a2.25 2.25 0 01-2.25 2.25H5.25A2.25 2.25 0 013 12V5.25" />
</svg>
)}
<span>Theme: {theme.charAt(0).toUpperCase() + theme.slice(1)}</span>
</button>
<a
href="https://github.com/calibrain/calibre-web-automated-book-downloader/issues"
target="_blank"
rel="noopener noreferrer"
className="w-full text-left px-4 py-2 hover-surface transition-colors flex items-center gap-3 text-slate-700 dark:text-slate-200"
title="Submit a bug report"
>
<svg
className="w-5 h-5"
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth="1.5"
stroke="currentColor"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M3 3v1.5M3 21v-6m0 0 2.77-.693a9 9 0 0 1 6.208.682l.108.054a9 9 0 0 0 6.086.71l3.114-.732a48.524 48.524 0 0 1-.005-10.499l-3.11.732a9 9 0 0 1-6.085-.711l-.108-.054a9 9 0 0 0-6.208-.682L3 4.5M3 15V4.5"
/>
</svg>
<span>Report a Bug</span>
</a>
{/* Debug Buttons */}
{debug && (
<>
<button
className="w-full text-left px-4 py-2 hover-surface transition-colors flex items-center gap-3 text-orange-600 dark:text-orange-400"
onClick={async () => {
closeDropdown();
// Show persistent toast while gathering logs
const loadingToastId = onShowToast?.('Gathering debug logs... This may take a minute.', 'info', true);
try {
const response = await fetch('/api/debug', {
method: 'GET',
credentials: 'include',
});
// Remove the loading toast
if (loadingToastId) onRemoveToast?.(loadingToastId);
if (!response.ok) {
const errorData = await response.json().catch(() => ({}));
onShowToast?.(`Debug download failed: ${errorData.error || response.statusText}`, 'error');
return;
}
// Get the filename from Content-Disposition header or use default
const contentDisposition = response.headers.get('Content-Disposition');
let filename = 'debug.zip';
if (contentDisposition) {
const filenameMatch = contentDisposition.match(/filename[^;=\n]*=((['"]).*?\2|[^;\n]*)/);
if (filenameMatch && filenameMatch[1]) {
filename = filenameMatch[1].replace(/['"]/g, '');
}
}
// Create blob and trigger download
const blob = await response.blob();
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
a.remove();
onShowToast?.('Debug logs downloaded successfully', 'success');
} catch (error) {
// Remove the loading toast on error too
if (loadingToastId) onRemoveToast?.(loadingToastId);
console.error('Debug download error:', error);
onShowToast?.('Debug download failed. Check console for details.', 'error');
}
}}
>
<svg className="w-5 h-5" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" strokeWidth="1.5" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" d="M12 12.75c1.148 0 2.278.08 3.383.237 1.037.146 1.866.966 1.866 2.013 0 3.728-2.35 6.75-5.25 6.75S6.75 18.728 6.75 15c0-1.046.83-1.867 1.866-2.013A24.204 24.204 0 0112 12.75zm0 0c2.883 0 5.647.508 8.207 1.44a23.91 23.91 0 01-1.152 6.06M12 12.75c-2.883 0-5.647.508-8.208 1.44.125 2.104.52 4.136 1.153 6.06M12 12.75a2.25 2.25 0 002.248-2.354M12 12.75a2.25 2.25 0 01-2.248-2.354M12 8.25c.995 0 1.971-.08 2.922-.236.403-.066.74-.358.795-.762a3.778 3.778 0 00-.399-2.25M12 8.25c-.995 0-1.97-.08-2.922-.236-.402-.066-.74-.358-.795-.762a3.734 3.734 0 01.4-2.253M12 8.25a2.25 2.25 0 00-2.248 2.146M12 8.25a2.25 2.25 0 012.248 2.146M8.683 5a6.032 6.032 0 01-1.155-1.002c.07-.63.27-1.222.574-1.747m.581 2.749A3.75 3.75 0 0115.318 5m0 0c.427-.283.815-.62 1.155-.999a4.471 4.471 0 00-.575-1.752M4.921 6a24.048 24.048 0 00-.392 3.314c1.668.546 3.416.914 5.223 1.082M19.08 6c.205 1.08.337 2.187.392 3.314a23.882 23.882 0 01-5.223 1.082" />
</svg>
<span>Debug</span>
</button>
<form action="/api/restart" method="get" className="w-full">
<button
className="w-full text-left px-4 py-2 hover-surface transition-colors flex items-center gap-3 text-orange-600 dark:text-orange-400"
type="submit"
>
<svg className="w-5 h-5" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" strokeWidth="1.5" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" d="M16.023 9.348h4.992v-.001M2.985 19.644v-4.992m0 0h4.992m-4.993 0l3.181 3.183a8.25 8.25 0 0013.803-3.7M4.031 9.865a8.25 8.25 0 0113.803-3.7l3.181 3.182m0-4.991v4.99" />
</svg>
<span>Restart</span>
</button>
</form>
</>
)}
{/* Logout Button */}
{authRequired && isAuthenticated && onLogout && (
<button
type="button"
onClick={handleLogout}
className="w-full text-left px-4 py-2 hover-surface transition-colors flex items-center gap-3 text-red-600 dark:text-red-400"
>
<svg className="w-5 h-5" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" strokeWidth="1.5" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" d="M15.75 9V5.25A2.25 2.25 0 0013.5 3h-6a2.25 2.25 0 00-2.25 2.25v13.5A2.25 2.25 0 007.5 21h6a2.25 2.25 0 002.25-2.25V15M12 9l-3 3m0 0l3 3m-3-3h12.75" />
</svg>
<span>Sign Out</span>
</button>
)}
</div>
</div>
)}
</div>
</div>
);
return (
<header
className="w-full sticky top-0 z-40 backdrop-blur-sm header-with-fade"
style={{ background: 'var(--bg)', paddingTop: 'env(safe-area-inset-top)' }}
>
<div className={`max-w-full mx-auto px-4 sm:px-6 lg:px-8 transition-all duration-500 ${
showSearch ? 'h-auto py-4' : 'h-24'
}`}>
{/* When search is active: stack on mobile, side-by-side on desktop */}
{showSearch && (
<div className="flex flex-col lg:flex-row lg:justify-between lg:items-center gap-3">
{/* Logo + Icon buttons - appear first on mobile (above search), last on desktop (right side) */}
<div className="flex items-center justify-between w-full lg:w-auto lg:justify-end lg:order-2">
{/* Logo - visible on mobile only, aligned left */}
{logoUrl && (
<img
src={logoUrl}
onClick={onLogoClick}
alt="Logo"
className="h-10 w-10 flex-shrink-0 cursor-pointer lg:hidden"
/>
)}
<IconButtons />
</div>
{/* Search bar - appear second on mobile (below logo+icons), first on desktop (left side) */}
<div className="flex items-center gap-4 lg:order-1 flex-1">
{/* Logo - visible on desktop only, aligned with search */}
{logoUrl && (
<img
src={logoUrl}
onClick={onLogoClick}
alt="Logo"
className="hidden lg:block h-12 w-12 flex-shrink-0 cursor-pointer"
/>
)}
<SearchBar
className="flex-1 lg:flex-initial"
inputClassName="lg:w-[50vw]"
value={searchInput}
onChange={handleSearchChange}
onSubmit={handleHeaderSearch}
onAdvancedToggle={onAdvancedToggle}
isLoading={isLoading}
/>
</div>
</div>
)}
{/* When search is NOT active: show icon buttons only on the right */}
{!showSearch && (
<div className="flex items-center justify-end h-full">
<IconButtons />
</div>
)}
</div>
</header>
);
};
@@ -0,0 +1,117 @@
import { Language } from '../types';
import {
formatDefaultLanguageLabel,
LANGUAGE_OPTION_ALL,
LANGUAGE_OPTION_DEFAULT,
normalizeLanguageSelection,
} from '../utils/languageFilters';
import { DropdownList, DropdownListOption } from './DropdownList';
interface LanguageMultiSelectProps {
options: Language[];
value: string[];
onChange: (value: string[]) => void;
defaultLanguageCodes: string[];
label?: string;
placeholder?: string;
}
export const LanguageMultiSelect = ({
options,
value,
onChange,
defaultLanguageCodes,
label,
placeholder,
}: LanguageMultiSelectProps) => {
const defaultLabel = formatDefaultLanguageLabel(defaultLanguageCodes, options);
const defaultCodeSet = new Set(defaultLanguageCodes);
const nonDefaultLanguages = options.filter(lang => !defaultCodeSet.has(lang.code));
const selectableValues = [LANGUAGE_OPTION_DEFAULT, ...nonDefaultLanguages.map(lang => lang.code)];
const optionList: DropdownListOption[] = [
{
value: LANGUAGE_OPTION_ALL,
label: 'All languages',
},
{
value: LANGUAGE_OPTION_DEFAULT,
label: defaultLabel,
},
...nonDefaultLanguages.map(lang => ({
value: lang.code,
label: lang.language,
})),
];
const includesAllSelection = value.includes(LANGUAGE_OPTION_ALL);
const effectiveValue = includesAllSelection ? selectableValues : value;
const selectedSet = new Set(effectiveValue);
const isAllSelected = selectableValues.every(code => selectedSet.has(code));
const displayedValue = isAllSelected ? [LANGUAGE_OPTION_ALL, ...effectiveValue] : effectiveValue;
const summaryFormatter = (_selected: DropdownListOption[], fallback: string) => {
if (isAllSelected) {
return 'All languages';
}
const labels: string[] = [];
if (selectedSet.has(LANGUAGE_OPTION_DEFAULT)) {
labels.push(defaultLabel);
}
nonDefaultLanguages.forEach(lang => {
if (selectedSet.has(lang.code)) {
labels.push(lang.language);
}
});
if (labels.length === 0) {
return placeholder || fallback;
}
if (labels.length === 1) {
return labels[0];
}
const [first, second, ...rest] = labels;
const suffix = rest.length > 0 ? ` +${rest.length}` : '';
return `${first}, ${second ?? ''}${suffix}`.trim();
};
const handleChange = (nextValue: string[] | string) => {
const nextArray = Array.isArray(nextValue) ? nextValue : [nextValue];
const includesAll = nextArray.includes(LANGUAGE_OPTION_ALL);
const toggledAllOn = includesAll && !isAllSelected;
const toggledAllOff =
isAllSelected && !includesAll && nextArray.length === effectiveValue.length;
let resolved = nextArray.filter(code => code !== LANGUAGE_OPTION_ALL);
if (toggledAllOn) {
resolved = [LANGUAGE_OPTION_ALL];
} else if (toggledAllOff) {
resolved = [];
}
const normalized = normalizeLanguageSelection(resolved);
onChange(normalized);
};
return (
<DropdownList
label={label}
options={optionList}
multiple
showCheckboxes
value={displayedValue}
onChange={handleChange}
placeholder={placeholder}
summaryFormatter={summaryFormatter}
keepOpenOnSelect
/>
);
};
+231
View File
@@ -0,0 +1,231 @@
import { FormEvent, KeyboardEvent, useEffect, useRef, useState } from 'react';
import { LoginCredentials } from '../types';
interface LoginFormProps {
onSubmit: (credentials: LoginCredentials) => void;
error?: string | null;
isLoading?: boolean;
autoFocus?: boolean;
}
const EyeIcon = () => (
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth={1.5}
stroke="currentColor"
className="w-5 h-5"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M2.036 12.322a1.012 1.012 0 010-.639C3.423 7.51 7.36 4.5 12 4.5c4.638 0 8.573 3.007 9.963 7.178.07.207.07.431 0 .639C20.577 16.49 16.64 19.5 12 19.5c-4.638 0-8.573-3.007-9.963-7.178z"
/>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M15 12a3 3 0 11-6 0 3 3 0 016 0z"
/>
</svg>
);
const EyeSlashIcon = () => (
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth={1.5}
stroke="currentColor"
className="w-5 h-5"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M3.98 8.223A10.477 10.477 0 001.934 12C3.226 16.338 7.244 19.5 12 19.5c.993 0 1.953-.138 2.863-.395M6.228 6.228A10.45 10.45 0 0112 4.5c4.756 0 8.773 3.162 10.065 7.498a10.523 10.523 0 01-4.293 5.774M6.228 6.228L3 3m3.228 3.228l3.65 3.65m7.894 7.894L21 21m-3.228-3.228l-3.65-3.65m0 0a3 3 0 10-4.243-4.243m4.242 4.242L9.88 9.88"
/>
</svg>
);
export const LoginForm = ({
onSubmit,
error = null,
isLoading = false,
autoFocus = true,
}: LoginFormProps) => {
const [username, setUsername] = useState('');
const [password, setPassword] = useState('');
const [rememberMe, setRememberMe] = useState(true);
const [showPassword, setShowPassword] = useState(false);
const usernameRef = useRef<HTMLInputElement>(null);
const passwordRef = useRef<HTMLInputElement>(null);
useEffect(() => {
if (autoFocus) {
usernameRef.current?.focus();
}
}, [autoFocus]);
const handleSubmit = (e: FormEvent<HTMLFormElement>) => {
e.preventDefault();
const formData = new FormData(e.currentTarget);
const usernameValue = (formData.get('username') as string)?.trim() || '';
const passwordValue = (formData.get('password') as string) || '';
if (usernameValue && passwordValue && !isLoading) {
onSubmit({
username: usernameValue,
password: passwordValue,
remember_me: rememberMe,
});
}
};
const handleUsernameKeyDown = (event: KeyboardEvent<HTMLInputElement>) => {
if (event.key === 'Enter') {
event.preventDefault();
passwordRef.current?.focus();
}
};
return (
<div>
{error && (
<div className="mb-4 p-3 rounded-lg text-sm bg-red-600 text-white">
{error}
</div>
)}
<form
method="post"
action="/api/login"
autoComplete="on"
id="login-form"
name="login"
data-form-type="login"
onSubmit={handleSubmit}
>
<div className="mb-4">
<label htmlFor="username" className="block text-sm font-medium mb-2">
Username
</label>
<input
ref={usernameRef}
type="text"
id="username"
name="username"
autoComplete="username"
autoCapitalize="none"
autoCorrect="off"
spellCheck={false}
inputMode="text"
enterKeyHint="next"
value={username}
onChange={(event) => setUsername(event.target.value)}
onKeyDown={handleUsernameKeyDown}
disabled={isLoading}
className="w-full px-4 py-2.5 rounded-lg border focus:outline-none focus:ring-2 focus:ring-sky-500 disabled:opacity-50 disabled:cursor-not-allowed transition-colors"
style={{
backgroundColor: 'var(--input-background)',
borderColor: 'var(--border-color)',
color: 'var(--text-color)',
}}
required
/>
</div>
<div className="mb-4">
<label htmlFor="password" className="block text-sm font-medium mb-2">
Password
</label>
<div className="relative">
<input
ref={passwordRef}
type={showPassword ? 'text' : 'password'}
id="password"
name="password"
autoComplete="current-password"
autoCapitalize="none"
autoCorrect="off"
spellCheck={false}
inputMode="text"
enterKeyHint="go"
value={password}
onChange={(event) => setPassword(event.target.value)}
disabled={isLoading}
className="w-full px-4 py-2.5 rounded-lg border focus:outline-none focus:ring-2 focus:ring-sky-500 disabled:opacity-50 disabled:cursor-not-allowed pr-10 transition-colors"
style={{
backgroundColor: 'var(--input-background)',
borderColor: 'var(--border-color)',
color: 'var(--text-color)',
}}
required
/>
<button
type="button"
onClick={() => setShowPassword((current) => !current)}
disabled={isLoading}
className="absolute right-2 top-1/2 transform -translate-y-1/2 p-1.5 rounded-full hover-action disabled:opacity-50 disabled:cursor-not-allowed transition-colors"
aria-label={showPassword ? 'Hide password' : 'Show password'}
>
{showPassword ? <EyeSlashIcon /> : <EyeIcon />}
</button>
</div>
</div>
<div className="mb-6 flex items-center">
<input
type="checkbox"
id="remember-me"
name="remember_me"
checked={rememberMe}
onChange={(event) => setRememberMe(event.target.checked)}
disabled={isLoading}
className="w-4 h-4 rounded focus:ring-2 focus:ring-sky-500 disabled:opacity-50 disabled:cursor-not-allowed accent-sky-900"
style={{ borderColor: 'var(--border-color)' }}
/>
<label htmlFor="remember-me" className="ml-2 text-sm">
Remember me for 7 days
</label>
</div>
<button
type="submit"
name="submit"
disabled={isLoading}
className="w-full py-2.5 px-4 rounded-lg font-medium text-white transition-colors disabled:opacity-50 disabled:cursor-not-allowed bg-sky-700 hover:bg-sky-800 disabled:hover:bg-sky-700"
aria-label="Sign in"
>
{isLoading ? (
<span className="flex items-center justify-center">
<svg
className="animate-spin -ml-1 mr-3 h-5 w-5 text-white"
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
>
<circle
className="opacity-25"
cx="12"
cy="12"
r="10"
stroke="currentColor"
strokeWidth="4"
></circle>
<path
className="opacity-75"
fill="currentColor"
d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"
></path>
</svg>
Signing in...
</span>
) : (
'Sign In'
)}
</button>
</form>
</div>
);
};
@@ -0,0 +1,263 @@
import { useState, useEffect } from 'react';
import { Book, ButtonStateInfo } from '../types';
import { CardView } from './resultsViews/CardView';
import { CompactView } from './resultsViews/CompactView';
import { ListView } from './resultsViews/ListView';
import { Dropdown } from './Dropdown';
import { SORT_OPTIONS } from '../data/filterOptions';
interface ResultsSectionProps {
books: Book[];
visible: boolean;
onDetails: (id: string) => Promise<void>;
onDownload: (book: Book) => Promise<void>;
getButtonState: (bookId: string) => ButtonStateInfo;
sortValue: string;
onSortChange: (value: string) => void;
}
export const ResultsSection = ({
books,
visible,
onDetails,
onDownload,
getButtonState,
sortValue,
onSortChange,
}: ResultsSectionProps) => {
const [viewMode, setViewMode] = useState<'card' | 'compact' | 'list'>(() => {
const saved = localStorage.getItem('bookViewMode');
return saved === 'card' || saved === 'compact' || saved === 'list' ? saved : 'compact';
});
const [isDesktop, setIsDesktop] = useState(false);
useEffect(() => {
localStorage.setItem('bookViewMode', viewMode);
}, [viewMode]);
// Track whether we're in desktop layout (sm breakpoint and above)
useEffect(() => {
const checkDesktop = () => {
setIsDesktop(window.innerWidth >= 640); // sm breakpoint
};
checkDesktop();
window.addEventListener('resize', checkDesktop);
return () => window.removeEventListener('resize', checkDesktop);
}, []);
if (!visible) return null;
return (
<section id="results-section" className="mb-4 sm:mb-8 w-full">
<div className="flex items-center justify-between mb-2 sm:mb-3 relative z-10">
<SortControl value={sortValue} onChange={onSortChange} />
{/* View toggle buttons - Desktop: show all 3, Mobile: show Compact and List only */}
<div className="flex items-center gap-2">
{isDesktop && (
<button
onClick={() => setViewMode('card')}
className={`p-2 rounded-full transition-all duration-200 ${
viewMode === 'card'
? 'text-white bg-sky-700 hover:bg-sky-800'
: 'hover-action text-gray-900 dark:text-gray-100'
}`}
title="Card view"
aria-label="Card view"
aria-pressed={viewMode === 'card'}
>
<svg
className="w-5 h-5"
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
strokeWidth="1.5"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M3.75 6A2.25 2.25 0 0 1 6 3.75h2.25A2.25 2.25 0 0 1 10.5 6v2.25a2.25 2.25 0 0 1-2.25 2.25H6a2.25 2.25 0 0 1-2.25-2.25V6ZM3.75 15.75A2.25 2.25 0 0 1 6 13.5h2.25a2.25 2.25 0 0 1 2.25 2.25V18a2.25 2.25 0 0 1-2.25 2.25H6A2.25 2.25 0 0 1 3.75 18v-2.25ZM13.5 6a2.25 2.25 0 0 1 2.25-2.25H18A2.25 2.25 0 0 1 20.25 6v2.25A2.25 2.25 0 0 1 18 10.5h-2.25a2.25 2.25 0 0 1-2.25-2.25V6ZM13.5 15.75a2.25 2.25 0 0 1 2.25-2.25H18a2.25 2.25 0 0 1 2.25 2.25V18A2.25 2.25 0 0 1 18 20.25h-2.25A2.25 2.25 0 0 1 13.5 18v-2.25Z"
/>
</svg>
</button>
)}
<button
onClick={() => setViewMode('compact')}
className={`p-2 rounded-full transition-all duration-200 ${
viewMode === 'compact'
? 'text-white bg-sky-700 hover:bg-sky-800'
: 'hover-action text-gray-900 dark:text-gray-100'
}`}
title="Compact view"
aria-label="Compact view"
aria-pressed={viewMode === 'compact'}
>
<svg
className="w-5 h-5"
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
strokeWidth="1.5"
>
<rect x="3.75" y="4.5" width="6" height="6" rx="1.125" />
<path strokeLinecap="round" strokeLinejoin="round" d="M12 6h8.25M12 8.25h6" />
<rect x="3.75" y="13.5" width="6" height="6" rx="1.125" />
<path strokeLinecap="round" strokeLinejoin="round" d="M12 15h8.25M12 17.25h6" />
</svg>
</button>
<button
onClick={() => setViewMode('list')}
className={`p-2 rounded-full transition-all duration-200 ${
viewMode === 'list'
? 'text-white bg-sky-700 hover:bg-sky-800'
: 'hover-action text-gray-900 dark:text-gray-100'
}`}
title="List view"
aria-label="List view"
aria-pressed={viewMode === 'list'}
>
<svg
className="w-5 h-5"
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
strokeWidth="1.5"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M8.25 6.75h12M8.25 12h12m-12 5.25h12M3.75 6.75h.007v.008H3.75V6.75Zm.375 0a.375.375 0 1 1-.75 0 .375.375 0 0 1 .75 0ZM3.75 12h.007v.008H3.75V12Zm.375 0a.375.375 0 1 1-.75 0 .375.375 0 0 1 .75 0Zm-.375 5.25h.007v.008H3.75v-.008Zm.375 0a.375.375 0 1 1-.75 0 .375.375 0 0 1 .75 0Z"
/>
</svg>
</button>
</div>
</div>
{viewMode === 'list' ? (
<ListView books={books} onDetails={onDetails} onDownload={onDownload} getButtonState={getButtonState} />
) : (
<div
id="results-grid"
className={`grid gap-8 ${
!isDesktop
? 'grid-cols-1 items-start'
: viewMode === 'card'
? 'grid-cols-1 sm:grid-cols-2 lg:grid-cols-3 xl:grid-cols-4 items-stretch'
: 'grid-cols-1 sm:grid-cols-2 lg:grid-cols-2 xl:grid-cols-3 items-start'
}`}
>
{books.map((book, index) => {
const shouldUseCardLayout = isDesktop && viewMode === 'card';
const animationDelay = index * 50;
return shouldUseCardLayout ? (
<CardView
key={book.id}
book={book}
onDetails={onDetails}
onDownload={onDownload}
buttonState={getButtonState(book.id)}
animationDelay={animationDelay}
/>
) : (
<CompactView
key={book.id}
book={book}
onDetails={onDetails}
onDownload={onDownload}
buttonState={getButtonState(book.id)}
showDetailsButton={!isDesktop}
animationDelay={animationDelay}
/>
);
})}
</div>
)}
{books.length === 0 && (
<div className="mt-4 text-sm opacity-80">No results found.</div>
)}
</section>
);
};
interface SortControlProps {
value: string;
onChange: (value: string) => void;
}
const SortControl = ({ value, onChange }: SortControlProps) => {
const selected = SORT_OPTIONS.find(option => option.value === value) ?? SORT_OPTIONS[0];
return (
<Dropdown
align="left"
widthClassName="w-60 sm:w-72"
renderTrigger={({ isOpen, toggle }) => (
<button
type="button"
onClick={toggle}
className={`relative flex items-center gap-2 px-3 py-2 rounded-full transition-all duration-200 text-gray-900 dark:text-gray-100 hover-action ${
isOpen ? 'bg-gray-100 dark:bg-gray-700' : ''
} animate-fade-in-up`}
aria-haspopup="listbox"
aria-expanded={isOpen}
aria-label="Change sort order"
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth={1.5}
stroke="currentColor"
className="w-5 h-5 sm:w-6 sm:h-6"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M3 7.5 7.5 3m0 0L12 7.5M7.5 3v13.5m13.5 0L16.5 21m0 0L12 16.5m4.5 4.5V7.5"
/>
</svg>
<span className="text-sm font-medium whitespace-nowrap">{selected.label}</span>
</button>
)}
>
{({ close }) => (
<div role="listbox" aria-label="Sort results">
{SORT_OPTIONS.map(option => {
const isSelected = option.value === selected.value;
return (
<button
type="button"
key={option.value || 'default'}
className={`w-full px-3 py-2 text-left text-base flex items-center justify-between gap-2 hover-surface ${
isSelected ? 'text-sky-600 dark:text-sky-300 font-medium' : ''
}`}
onClick={() => {
onChange(option.value);
close();
}}
role="option"
aria-selected={isSelected}
>
<span>{option.label}</span>
{isSelected && (
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth={1.5}
stroke="currentColor"
className="w-4 h-4"
>
<path strokeLinecap="round" strokeLinejoin="round" d="m4.5 12.75 6 6 9-13.5" />
</svg>
)}
</button>
);
})}
</div>
)}
</Dropdown>
);
};
+174
View File
@@ -0,0 +1,174 @@
import { KeyboardEvent, InputHTMLAttributes, useRef } from 'react';
interface SearchBarProps {
value: string;
onChange: (value: string) => void;
onSubmit: () => void;
isLoading?: boolean;
onAdvancedToggle?: () => void;
placeholder?: string;
inputAriaLabel?: string;
className?: string;
inputClassName?: string;
controlsClassName?: string;
clearButtonLabel?: string;
clearButtonTitle?: string;
advancedButtonLabel?: string;
advancedButtonTitle?: string;
searchButtonLabel?: string;
searchButtonTitle?: string;
autoComplete?: string;
enterKeyHint?: InputHTMLAttributes<HTMLInputElement>['enterKeyHint'];
}
export const SearchBar = ({
value,
onChange,
onSubmit,
isLoading = false,
onAdvancedToggle,
placeholder = 'Search by ISBN, title, author...',
inputAriaLabel = 'Search books',
className = '',
inputClassName = '',
controlsClassName = '',
clearButtonLabel = 'Clear search input',
clearButtonTitle = 'Clear search',
advancedButtonLabel = 'Advanced Search',
advancedButtonTitle = 'Advanced Search',
searchButtonLabel = 'Search books',
searchButtonTitle = 'Search',
autoComplete = 'off',
enterKeyHint = 'search',
}: SearchBarProps) => {
const inputRef = useRef<HTMLInputElement>(null);
const hasSearchQuery = value.trim().length > 0;
const handleKeyDown = (e: KeyboardEvent<HTMLInputElement>) => {
if (e.key === 'Enter') {
onSubmit();
(e.target as HTMLInputElement).blur();
}
};
const handleClearSearch = () => {
onChange('');
inputRef.current?.focus();
};
const wrapperClasses = ['relative', className].filter(Boolean).join(' ').trim();
const inputClasses = [
'w-full pl-4 pr-40 py-3 rounded-full border outline-none search-input',
inputClassName,
]
.filter(Boolean)
.join(' ')
.trim();
const controlsClasses = [
'absolute inset-y-0 right-0 flex items-center gap-1 pr-2',
controlsClassName,
]
.filter(Boolean)
.join(' ')
.trim();
return (
<div className={wrapperClasses}>
<input
type="search"
placeholder={placeholder}
aria-label={inputAriaLabel}
autoComplete={autoComplete}
enterKeyHint={enterKeyHint}
className={inputClasses}
style={{
background: 'var(--bg-soft)',
color: 'var(--text)',
borderColor: 'var(--border-muted)',
}}
value={value}
onChange={e => onChange(e.target.value)}
onKeyDown={handleKeyDown}
ref={inputRef}
/>
<div className={controlsClasses}>
{hasSearchQuery && (
<button
type="button"
onClick={handleClearSearch}
className="p-2 rounded-full hover-action flex items-center justify-center transition-colors"
aria-label={clearButtonLabel}
title={clearButtonTitle}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth="1.5"
stroke="currentColor"
className="w-5 h-5"
style={{ color: 'var(--text)' }}
>
<path strokeLinecap="round" strokeLinejoin="round" d="M6 18 18 6M6 6l12 12" />
</svg>
</button>
)}
{onAdvancedToggle && (
<button
type="button"
onClick={onAdvancedToggle}
className="p-2 rounded-full hover-action flex items-center justify-center transition-colors"
aria-label={advancedButtonLabel}
title={advancedButtonTitle}
>
<svg
className="w-5 h-5"
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth="1.5"
stroke="currentColor"
style={{ color: 'var(--text)' }}
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="M10.5 6h9.75M10.5 6a1.5 1.5 0 1 1-3 0m3 0a1.5 1.5 0 1 0-3 0M3.75 6H7.5m3 12h9.75m-9.75 0a1.5 1.5 0 0 1-3 0m3 0a1.5 1.5 0 0 0-3 0m-3.75 0H7.5m9-6h3.75m-3.75 0a1.5 1.5 0 0 1-3 0m3 0a1.5 1.5 0 0 0-3 0m-9.75 0h9.75"
/>
</svg>
</button>
)}
<button
type="button"
onClick={onSubmit}
className="p-2 rounded-full text-white bg-sky-700 hover:bg-sky-800 disabled:opacity-60 disabled:cursor-not-allowed flex items-center justify-center transition-colors search-bar-button"
aria-label={searchButtonLabel}
title={searchButtonTitle}
disabled={isLoading}
>
{!isLoading && (
<svg
className="w-5 h-5 search-bar-icon"
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
strokeWidth="2"
stroke="currentColor"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
d="m21 21-5.197-5.197m0 0A7.5 7.5 0 1 0 5.196 5.196a7.5 7.5 0 0 0 10.607 10.607Z"
/>
</svg>
)}
{isLoading && (
<div className="spinner w-3 h-3 border-2 border-white border-t-transparent search-bar-spinner" />
)}
</button>
</div>
</div>
);
};
@@ -0,0 +1,86 @@
import { AdvancedFilterState, Language } from '../types';
import { buildSearchQuery } from '../utils/buildSearchQuery';
import { AdvancedFilters } from './AdvancedFilters';
import { SearchBar } from './SearchBar';
interface SearchSectionProps {
onSearch: (query: string) => void;
isLoading: boolean;
isInitialState: boolean;
bookLanguages: Language[];
defaultLanguage: string[];
supportedFormats: string[];
logoUrl: string;
searchInput: string;
onSearchInputChange: (value: string) => void;
showAdvanced: boolean;
onAdvancedToggle: () => void;
advancedFilters: AdvancedFilterState;
onAdvancedFiltersChange: (updates: Partial<AdvancedFilterState>) => void;
}
export const SearchSection = ({
onSearch,
isLoading,
isInitialState,
bookLanguages,
defaultLanguage,
supportedFormats,
logoUrl,
searchInput,
onSearchInputChange,
showAdvanced,
onAdvancedToggle,
advancedFilters,
onAdvancedFiltersChange,
}: SearchSectionProps) => {
const handleSearch = () => {
const query = buildSearchQuery({
searchInput,
showAdvanced,
advancedFilters,
bookLanguages,
defaultLanguage,
});
onSearch(query);
};
return (
<section
id="search-section"
className={`transition-all duration-500 ease-in-out ${
isInitialState
? 'search-initial-state mb-6'
: 'mb-3 sm:mb-4'
} ${showAdvanced ? 'search-advanced-visible' : ''}`}
>
<div className={`flex items-center justify-center gap-3 transition-all duration-300 ${
isInitialState ? 'opacity-100 mb-6 sm:mb-8' : 'opacity-0 h-0 mb-0 overflow-hidden'
}`}>
<img src={logoUrl} alt="Logo" className="h-8 w-8" />
<h1 className="text-2xl font-semibold">Book Search & Download</h1>
</div>
<div className={`flex flex-col gap-3 search-wrapper transition-all duration-500 ${
isInitialState ? '' : 'hidden'
}`}>
<SearchBar
value={searchInput}
onChange={onSearchInputChange}
onSubmit={handleSearch}
isLoading={isLoading}
onAdvancedToggle={onAdvancedToggle}
/>
<AdvancedFilters
visible={showAdvanced}
bookLanguages={bookLanguages}
defaultLanguage={defaultLanguage}
supportedFormats={supportedFormats}
filters={advancedFilters}
onFiltersChange={onAdvancedFiltersChange}
formClassName="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4 px-2"
renderWrapper={form => form}
/>
</div>
</section>
);
};
@@ -0,0 +1,41 @@
import { useEffect, useState } from 'react';
import { Toast } from '../types';
interface ToastContainerProps {
toasts: Toast[];
}
export const ToastContainer = ({ toasts }: ToastContainerProps) => {
const [visibleToasts, setVisibleToasts] = useState<Set<string>>(new Set());
useEffect(() => {
toasts.forEach(toast => {
if (!visibleToasts.has(toast.id)) {
setTimeout(() => {
setVisibleToasts(prev => new Set([...prev, toast.id]));
}, 10);
}
});
}, [toasts]);
const toastTypeClasses: Record<Toast['type'], string> = {
success: 'bg-green-600 text-white',
error: 'bg-red-600 text-white',
info: 'bg-blue-600 text-white',
};
return (
<div id="toast-container" className="fixed bottom-4 right-4 z-50 space-y-2">
{toasts.map(toast => (
<div
key={toast.id}
className={`toast-notification px-4 py-3 rounded-md shadow-lg text-sm font-medium transition-all duration-300 ${
toastTypeClasses[toast.type]
} ${visibleToasts.has(toast.id) ? 'toast-visible' : ''}`}
>
{toast.message}
</div>
))}
</div>
);
};
+11
View File
@@ -0,0 +1,11 @@
export { Header } from './Header';
export { SearchSection } from './SearchSection';
export { SearchBar } from './SearchBar';
export { ResultsSection } from './ResultsSection';
export { DetailsModal } from './DetailsModal';
export { DownloadsSidebar } from './DownloadsSidebar';
export { ToastContainer } from './ToastContainer';
export { Footer } from './Footer';
export { CardView } from './resultsViews/CardView';
export { CompactView } from './resultsViews/CompactView';
export { ListView } from './resultsViews/ListView';
@@ -0,0 +1,151 @@
import { useState } from 'react';
import { Book, ButtonStateInfo } from '../../types';
import { BookDownloadButton } from '../BookDownloadButton';
const SkeletonLoader = () => (
<div className="w-full h-full bg-gradient-to-r from-gray-300 via-gray-200 to-gray-300 dark:from-gray-700 dark:via-gray-600 dark:to-gray-700 animate-pulse" />
);
interface CardViewProps {
book: Book;
onDetails: (id: string) => Promise<void>;
onDownload: (book: Book) => Promise<void>;
buttonState: ButtonStateInfo;
animationDelay?: number;
}
export const CardView = ({ book, onDetails, onDownload, buttonState, animationDelay = 0 }: CardViewProps) => {
const [isLoadingDetails, setIsLoadingDetails] = useState(false);
const [imageLoaded, setImageLoaded] = useState(false);
const [imageError, setImageError] = useState(false);
const [isHovered, setIsHovered] = useState(false);
const handleDetails = async (id: string) => {
setIsLoadingDetails(true);
try {
await onDetails(id);
} finally {
setIsLoadingDetails(false);
}
};
return (
<article
className="book-card overflow-hidden flex flex-col sm:flex-col max-sm:flex-row space-between w-full sm:max-w-[292px] max-sm:h-[180px] h-full transition-shadow duration-300 animate-slide-up will-change-transform"
style={{
background: 'var(--bg-soft)',
borderRadius: '.75rem',
boxShadow: isHovered ? '0 10px 30px rgba(0, 0, 0, 0.15)' : 'none',
animationDelay: `${animationDelay}ms`,
animationFillMode: 'both',
}}
onMouseEnter={() => setIsHovered(true)}
onMouseLeave={() => setIsHovered(false)}
>
<div className="relative w-full sm:w-full max-sm:w-[120px] max-sm:h-full max-sm:flex-shrink-0 group" style={{ aspectRatio: '2/3' }}>
{book.preview && !imageError ? (
<>
{!imageLoaded && (
<div className="absolute inset-0">
<SkeletonLoader />
</div>
)}
<img
src={book.preview}
alt={book.title || 'Book cover'}
className="w-full h-full"
style={{
opacity: imageLoaded ? 1 : 0,
transition: 'opacity 0.3s ease-in-out',
objectFit: 'cover',
objectPosition: 'top',
}}
onLoad={() => setImageLoaded(true)}
onError={() => setImageError(true)}
/>
</>
) : (
<div className="w-full h-full flex items-center justify-center text-sm opacity-50" style={{ background: 'var(--border-muted)' }}>
No Cover
</div>
)}
<div
className="absolute inset-0 bg-white transition-opacity duration-300 pointer-events-none"
style={{ opacity: isHovered ? 0.02 : 0 }}
/>
<button
className="absolute bottom-2 right-2 w-8 h-8 rounded-full bg-white/90 dark:bg-gray-800/90 backdrop-blur-sm flex items-center justify-center transition-all duration-300 shadow-lg hover:scale-110 max-sm:hidden"
style={{
opacity: isHovered || isLoadingDetails ? 1 : 0,
pointerEvents: isHovered || isLoadingDetails ? 'auto' : 'none',
}}
onClick={(e) => {
e.stopPropagation();
handleDetails(book.id);
}}
disabled={isLoadingDetails}
aria-label="Book details"
>
{isLoadingDetails ? (
<div className="w-4 h-4 border-2 border-current border-t-transparent rounded-full animate-spin" />
) : (
<svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
)}
</button>
</div>
<div className="p-4 max-sm:p-3 max-sm:py-2 flex flex-col gap-3 max-sm:gap-2 max-sm:flex-1 max-sm:justify-between max-sm:min-w-0 sm:flex-1 sm:flex sm:flex-col sm:justify-end">
<div className="space-y-1 max-sm:space-y-0.5 max-sm:min-w-0">
<h3 className="font-semibold leading-tight line-clamp-2 text-base max-sm:line-clamp-3 max-sm:min-w-0" title={book.title || 'Untitled'}>
{book.title || 'Untitled'}
</h3>
<p className="text-sm max-sm:text-xs opacity-80 truncate max-sm:min-w-0">{book.author || 'Unknown author'}</p>
<div className="text-xs max-sm:text-[10px] opacity-70 flex flex-wrap gap-2 max-sm:gap-1">
<span>{book.year || '-'}</span>
<span>•</span>
<span>{book.language || '-'}</span>
<span>•</span>
<span>{book.format || '-'}</span>
{book.size && (
<>
<span>•</span>
<span>{book.size}</span>
</>
)}
</div>
</div>
<div className="flex gap-1.5 sm:hidden">
<button
className="px-2 py-1.5 rounded border text-xs flex-1 flex items-center justify-center gap-1"
onClick={() => handleDetails(book.id)}
style={{ borderColor: 'var(--border-muted)' }}
disabled={isLoadingDetails}
>
<span className="details-button-text">{isLoadingDetails ? 'Loading' : 'Details'}</span>
<div
className={`details-spinner w-3 h-3 border-2 border-current border-t-transparent rounded-full ${isLoadingDetails ? '' : 'hidden'}`}
/>
</button>
<BookDownloadButton buttonState={buttonState} onDownload={() => onDownload(book)} size="sm" className="flex-1" />
</div>
</div>
<BookDownloadButton
buttonState={buttonState}
onDownload={() => onDownload(book)}
className="hidden sm:flex rounded-none"
fullWidth
style={{
borderBottomLeftRadius: '.75rem',
borderBottomRightRadius: '.75rem',
}}
/>
</article>
);
};
@@ -0,0 +1,146 @@
import { useState } from 'react';
import { Book, ButtonStateInfo } from '../../types';
import { BookDownloadButton } from '../BookDownloadButton';
const SkeletonLoader = () => (
<div className="w-full h-full bg-gradient-to-r from-gray-300 via-gray-200 to-gray-300 dark:from-gray-700 dark:via-gray-600 dark:to-gray-700 animate-pulse" />
);
interface CompactViewProps {
book: Book;
onDetails: (id: string) => Promise<void>;
onDownload: (book: Book) => Promise<void>;
buttonState: ButtonStateInfo;
showDetailsButton?: boolean;
animationDelay?: number;
}
export const CompactView = ({ book, onDetails, onDownload, buttonState, showDetailsButton = false, animationDelay = 0 }: CompactViewProps) => {
const [isLoadingDetails, setIsLoadingDetails] = useState(false);
const [imageLoaded, setImageLoaded] = useState(false);
const [imageError, setImageError] = useState(false);
const [isHovered, setIsHovered] = useState(false);
const handleDetails = async (id: string) => {
setIsLoadingDetails(true);
try {
await onDetails(id);
} finally {
setIsLoadingDetails(false);
}
};
return (
<article
className="book-card overflow-hidden !flex !flex-row w-full !h-[180px] transition-shadow duration-300 animate-slide-up will-change-transform"
style={{
background: 'var(--bg-soft)',
borderRadius: '.75rem',
boxShadow: isHovered ? '0 10px 30px rgba(0, 0, 0, 0.15)' : 'none',
animationDelay: `${animationDelay}ms`,
animationFillMode: 'both',
}}
onMouseEnter={() => setIsHovered(true)}
onMouseLeave={() => setIsHovered(false)}
>
<div className="relative w-[120px] h-full flex-shrink-0">
{book.preview && !imageError ? (
<>
{!imageLoaded && (
<div className="absolute inset-0">
<SkeletonLoader />
</div>
)}
<img
src={book.preview}
alt={book.title || 'Book cover'}
className="w-full h-full"
style={{
opacity: imageLoaded ? 1 : 0,
transition: 'opacity 0.3s ease-in-out',
objectFit: 'cover',
objectPosition: 'top',
}}
onLoad={() => setImageLoaded(true)}
onError={() => setImageError(true)}
/>
</>
) : (
<div className="w-full h-full flex items-center justify-center text-sm opacity-50" style={{ background: 'var(--border-muted)' }}>
No Cover
</div>
)}
<div className="absolute inset-0 bg-white transition-opacity duration-300 pointer-events-none" style={{ opacity: isHovered ? 0.02 : 0 }} />
{!showDetailsButton && (
<button
className="absolute bottom-2 right-2 w-8 h-8 rounded-full bg-white/90 dark:bg-gray-800/90 backdrop-blur-sm flex items-center justify-center transition-all duration-300 shadow-lg hover:scale-110"
style={{
opacity: isHovered || isLoadingDetails ? 1 : 0,
pointerEvents: isHovered || isLoadingDetails ? 'auto' : 'none',
}}
onClick={(e) => {
e.stopPropagation();
handleDetails(book.id);
}}
disabled={isLoadingDetails}
aria-label="Book details"
>
{isLoadingDetails ? (
<div className="w-4 h-4 border-2 border-current border-t-transparent rounded-full animate-spin" />
) : (
<svg className="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
)}
</button>
)}
</div>
<div className="p-3 py-2 flex flex-col flex-1 min-w-0">
<div className="space-y-0.5 min-w-0">
<h3 className="font-semibold leading-tight line-clamp-3 text-base min-w-0" title={book.title || 'Untitled'}>
{book.title || 'Untitled'}
</h3>
<p className="text-xs opacity-80 truncate min-w-0">{book.author || 'Unknown author'}</p>
<div className="text-[10px] opacity-70">
<span>{book.year || '-'}</span>
</div>
</div>
<div className="mt-auto flex flex-col gap-2">
<div className="text-[10px] opacity-70 flex flex-wrap gap-1">
<span>{book.language || '-'}</span>
<span>•</span>
<span>{book.format || '-'}</span>
{book.size && (
<>
<span>•</span>
<span>{book.size}</span>
</>
)}
</div>
{showDetailsButton ? (
<div className="flex gap-1.5">
<button
className="px-2 py-1.5 rounded border text-xs flex-shrink-0 flex items-center justify-center gap-1"
onClick={() => handleDetails(book.id)}
style={{ borderColor: 'var(--border-muted)' }}
disabled={isLoadingDetails}
>
<span className="details-button-text">{isLoadingDetails ? 'Loading' : 'Details'}</span>
{isLoadingDetails && <div className="w-3 h-3 border-2 border-current border-t-transparent rounded-full animate-spin" />}
</button>
<BookDownloadButton buttonState={buttonState} onDownload={() => onDownload(book)} size="sm" className="flex-1" />
</div>
) : (
<BookDownloadButton buttonState={buttonState} onDownload={() => onDownload(book)} size="sm" fullWidth />
)}
</div>
</div>
</article>
);
};
@@ -0,0 +1,215 @@
import { useState } from 'react';
import { Book, ButtonStateInfo } from '../../types';
import { BookDownloadButton } from '../BookDownloadButton';
interface ListViewProps {
books: Book[];
onDetails: (id: string) => Promise<void>;
onDownload: (book: Book) => Promise<void>;
getButtonState: (bookId: string) => ButtonStateInfo;
}
const ListViewThumbnail = ({ preview, title }: { preview?: string; title?: string }) => {
const [imageLoaded, setImageLoaded] = useState(false);
const [imageError, setImageError] = useState(false);
if (!preview || imageError) {
return (
<div
className="w-7 h-10 sm:w-10 sm:h-14 rounded bg-gray-200 dark:bg-gray-700 flex items-center justify-center text-[8px] sm:text-[9px] font-medium text-gray-500 dark:text-gray-300"
aria-label="No cover available"
>
No Cover
</div>
);
}
return (
<div className="relative w-7 h-10 sm:w-10 sm:h-14 rounded overflow-hidden bg-gray-100 dark:bg-gray-800 border border-white/40 dark:border-gray-700/70">
{!imageLoaded && (
<div className="absolute inset-0 bg-gradient-to-r from-gray-200 via-gray-100 to-gray-200 dark:from-gray-700 dark:via-gray-600 dark:to-gray-700 animate-pulse" />
)}
<img
src={preview}
alt={title || 'Book cover'}
className="w-full h-full object-cover object-top"
loading="lazy"
onLoad={() => setImageLoaded(true)}
onError={() => setImageError(true)}
style={{ opacity: imageLoaded ? 1 : 0, transition: 'opacity 0.2s ease-in-out' }}
/>
</div>
);
};
const getLanguageColor = (language?: string): string => {
if (!language || language === '-') return 'bg-gray-400 dark:bg-gray-600';
const lang = language.toLowerCase();
const colorMap: Record<string, string> = {
en: 'bg-blue-500 dark:bg-blue-600',
english: 'bg-blue-500 dark:bg-blue-600',
es: 'bg-orange-500 dark:bg-orange-600',
spanish: 'bg-orange-500 dark:bg-orange-600',
fr: 'bg-purple-500 dark:bg-purple-600',
french: 'bg-purple-500 dark:bg-purple-600',
de: 'bg-yellow-500 dark:bg-yellow-600',
german: 'bg-yellow-500 dark:bg-yellow-600',
it: 'bg-green-500 dark:bg-green-600',
italian: 'bg-green-500 dark:bg-green-600',
pt: 'bg-teal-500 dark:bg-teal-600',
portuguese: 'bg-teal-500 dark:bg-teal-600',
ru: 'bg-red-500 dark:bg-red-600',
russian: 'bg-red-500 dark:bg-red-600',
ja: 'bg-pink-500 dark:bg-pink-600',
japanese: 'bg-pink-500 dark:bg-pink-600',
zh: 'bg-rose-500 dark:bg-rose-600',
chinese: 'bg-rose-500 dark:bg-rose-600',
};
return colorMap[lang] || 'bg-indigo-500 dark:bg-indigo-600';
};
const getFormatColor = (format?: string): string => {
if (!format || format === '-') return 'bg-gray-400 dark:bg-gray-600';
const fmt = format.toLowerCase();
const colorMap: Record<string, string> = {
pdf: 'bg-red-500 dark:bg-red-600',
epub: 'bg-green-500 dark:bg-green-600',
mobi: 'bg-blue-500 dark:bg-blue-600',
azw3: 'bg-purple-500 dark:bg-purple-600',
txt: 'bg-gray-500 dark:bg-gray-600',
djvu: 'bg-orange-500 dark:bg-orange-600',
fb2: 'bg-teal-500 dark:bg-teal-600',
cbr: 'bg-yellow-500 dark:bg-yellow-600',
cbz: 'bg-amber-500 dark:bg-amber-600',
};
return colorMap[fmt] || 'bg-cyan-500 dark:bg-cyan-600';
};
export const ListView = ({ books, onDetails, onDownload, getButtonState }: ListViewProps) => {
const [detailsLoadingId, setDetailsLoadingId] = useState<string | null>(null);
if (books.length === 0) {
return null;
}
const handleDetails = async (bookId: string) => {
setDetailsLoadingId(bookId);
try {
await onDetails(bookId);
} finally {
setDetailsLoadingId((current) => (current === bookId ? null : current));
}
};
return (
<article
className="w-full overflow-hidden rounded-lg sm:rounded-2xl"
style={{
background: 'var(--bg-soft)',
boxShadow: '0 10px 30px rgba(15, 23, 42, 0.08)',
}}
role="region"
aria-label="List view of books"
>
<div className="divide-y divide-gray-200/60 dark:divide-gray-800/60 w-full">
{books.map((book, index) => {
const buttonState = getButtonState(book.id);
const isLoadingDetails = detailsLoadingId === book.id;
return (
<div
key={book.id}
className="px-1.5 sm:px-2 py-1.5 sm:py-2 transition-colors duration-200 hover-row w-full animate-slide-up will-change-transform"
style={{
animationDelay: `${index * 50}ms`,
animationFillMode: 'both',
}}
role="article"
>
{/* Mobile and Desktop: Single row layout */}
<div className="grid grid-cols-[auto_minmax(0,1fr)_auto_auto] sm:grid-cols-[auto_minmax(0,2fr)_minmax(50px,0.25fr)_minmax(60px,0.3fr)_minmax(60px,0.3fr)_minmax(60px,0.3fr)_auto] items-center gap-2 sm:gap-y-1 sm:gap-x-0.5 w-full">
{/* Thumbnail */}
<div className="flex items-center pl-1 sm:pl-3">
<ListViewThumbnail preview={book.preview} title={book.title} />
</div>
{/* Title and Author */}
<div className="min-w-0 flex flex-col justify-center sm:pl-3">
<h3 className="font-semibold text-xs min-[400px]:text-sm sm:text-base leading-tight line-clamp-1 sm:line-clamp-2" title={book.title || 'Untitled'}>
{book.title || 'Untitled'}
</h3>
<p className="text-[10px] min-[400px]:text-xs sm:text-sm text-gray-600 dark:text-gray-300 truncate">
{book.author || 'Unknown author'}
{book.year && <span className="sm:hidden"> • {book.year}</span>}
</p>
</div>
{/* Format and Size - Mobile only */}
<div className="flex sm:hidden flex-col items-end text-[10px] opacity-70 leading-tight">
<span>{book.format || '-'}</span>
{book.size && <span>{book.size}</span>}
</div>
{/* Year - Desktop only */}
<div className="hidden sm:flex text-xs text-gray-700 dark:text-gray-200 justify-center">
{book.year || '-'}
</div>
{/* Language Badge - Desktop only */}
<div className="hidden sm:flex justify-center">
<span
className={`${getLanguageColor(book.language)} text-white text-[11px] font-semibold px-2 py-0.5 rounded uppercase tracking-wide`}
title={book.language || 'Unknown'}
>
{book.language || '-'}
</span>
</div>
{/* Format Badge - Desktop only */}
<div className="hidden sm:flex justify-center">
<span
className={`${getFormatColor(book.format)} text-white text-[11px] font-semibold px-2 py-0.5 rounded uppercase tracking-wide`}
title={book.format || 'Unknown'}
>
{book.format || '-'}
</span>
</div>
{/* Size - Desktop only */}
<div className="hidden sm:flex text-xs text-gray-700 dark:text-gray-200 justify-center">
{book.size || '-'}
</div>
{/* Action Buttons */}
<div className="flex flex-row justify-end gap-0.5 sm:gap-1">
<button
className="flex items-center justify-center p-1.5 sm:p-2 rounded-full text-gray-600 dark:text-gray-200 hover-action transition-all duration-200"
onClick={() => handleDetails(book.id)}
disabled={isLoadingDetails}
aria-label={`View details for ${book.title || 'this book'}`}
>
{isLoadingDetails ? (
<div className="w-4 h-4 sm:w-5 sm:h-5 border-2 border-current border-t-transparent rounded-full animate-spin" />
) : (
<svg className="w-4 h-4 sm:w-5 sm:h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24" strokeWidth="1.5">
<path strokeLinecap="round" strokeLinejoin="round" d="M13 16h-1v-4h-1m1-4h.01M12 20a8 8 0 100-16 8 8 0 000 16z" />
</svg>
)}
</button>
<BookDownloadButton
buttonState={buttonState}
onDownload={() => onDownload(book)}
variant="icon"
size="md"
ariaLabel={buttonState.text}
/>
</div>
</div>
</div>
);
})}
</div>
</article>
);
};
+23
View File
@@ -0,0 +1,23 @@
export const SORT_OPTIONS = [
{ value: '', label: 'Most relevant' },
{ value: 'newest', label: 'Newest (publication year)' },
{ value: 'oldest', label: 'Oldest (publication year)' },
{ value: 'largest', label: 'Largest (filesize)' },
{ value: 'smallest', label: 'Smallest (filesize)' },
{ value: 'newest_added', label: 'Newest (open sourced)' },
{ value: 'oldest_added', label: 'Oldest (open sourced)' },
];
export const CONTENT_OPTIONS = [
{ value: '', label: 'All' },
{ value: 'book_nonfiction', label: 'Book (non-fiction)' },
{ value: 'book_fiction', label: 'Book (fiction)' },
{ value: 'book_unknown', label: 'Book (unknown)' },
{ value: 'magazine', label: 'Magazine' },
{ value: 'book_comic', label: 'Comic Book' },
{ value: 'standards_document', label: 'Standards document' },
{ value: 'other', label: 'Other' },
{ value: 'musical_score', label: 'Musical score' },
{ value: 'audiobook', label: 'Audiobook' },
];
+16
View File
@@ -0,0 +1,16 @@
// This data is loaded from the backend in production
// For now, provide a default set
export const DEFAULT_LANGUAGES = [
{ code: 'en', language: 'English' },
{ code: 'es', language: 'Spanish' },
{ code: 'fr', language: 'French' },
{ code: 'de', language: 'German' },
{ code: 'it', language: 'Italian' },
{ code: 'pt', language: 'Portuguese' },
{ code: 'ru', language: 'Russian' },
{ code: 'zh', language: 'Chinese' },
{ code: 'ja', language: 'Japanese' },
{ code: 'ko', language: 'Korean' },
];
export const DEFAULT_SUPPORTED_FORMATS = ['epub', 'mobi', 'azw3', 'fb2', 'djvu', 'cbz', 'cbr'];
+273
View File
@@ -0,0 +1,273 @@
import { useEffect, useRef, useState, useCallback } from 'react';
import { io, Socket } from 'socket.io-client';
import { StatusData } from '../types';
import { getStatus } from '../services/api';
interface UseRealtimeStatusOptions {
wsUrl: string;
pollInterval?: number;
reconnectAttempts?: number;
}
interface UseRealtimeStatusReturn {
status: StatusData;
connected: boolean;
isUsingWebSocket: boolean;
error: string | null;
forceRefresh: () => Promise<void>;
}
/**
* Hook for real-time status updates with WebSocket and polling fallback
*
* This hook attempts to connect via WebSocket first. If WebSocket connection
* fails or disconnects, it automatically falls back to polling. It will
* periodically retry WebSocket connections.
*/
export const useRealtimeStatus = ({
wsUrl,
pollInterval = 2000, // Reduced from 5s for better UX when WebSocket unavailable
reconnectAttempts = 3,
}: UseRealtimeStatusOptions): UseRealtimeStatusReturn => {
const [status, setStatus] = useState<StatusData>({});
const [connected, setConnected] = useState(false);
const [isUsingWebSocket, setIsUsingWebSocket] = useState(false);
const [error, setError] = useState<string | null>(null);
const socketRef = useRef<Socket | null>(null);
const pollIntervalRef = useRef<NodeJS.Timeout | null>(null);
const reconnectAttemptsRef = useRef(0);
const reconnectTimeoutRef = useRef<NodeJS.Timeout | null>(null);
const isConnectingRef = useRef(false);
// Polling function
const pollStatus = useCallback(async () => {
try {
const data = await getStatus();
setStatus(data);
setError(null);
} catch (err) {
console.error('Error polling status:', err);
setError('Failed to fetch status');
}
}, []);
// Start polling
const startPolling = useCallback(() => {
if (pollIntervalRef.current) return;
console.log('Starting polling fallback');
setIsUsingWebSocket(false);
// Poll immediately
pollStatus();
// Then poll at intervals
pollIntervalRef.current = setInterval(pollStatus, pollInterval);
}, [pollStatus, pollInterval]);
// Stop polling
const stopPolling = useCallback(() => {
if (pollIntervalRef.current) {
clearInterval(pollIntervalRef.current);
pollIntervalRef.current = null;
console.log('Stopped polling');
}
}, []);
// Attempt to reconnect WebSocket
const attemptReconnect = useCallback(() => {
if (reconnectAttemptsRef.current >= reconnectAttempts) {
console.log('Max reconnect attempts reached, using polling permanently');
return;
}
reconnectAttemptsRef.current += 1;
const delay = Math.min(1000 * Math.pow(2, reconnectAttemptsRef.current), 30000);
console.log(`Attempting WebSocket reconnect ${reconnectAttemptsRef.current}/${reconnectAttempts} in ${delay}ms`);
reconnectTimeoutRef.current = setTimeout(() => {
if (!isConnectingRef.current && !socketRef.current?.connected) {
initializeWebSocket();
}
}, delay);
}, [reconnectAttempts]);
// Initialize WebSocket connection
const initializeWebSocket = useCallback(() => {
if (isConnectingRef.current || socketRef.current?.connected) {
return;
}
isConnectingRef.current = true;
console.log('Initializing WebSocket connection to:', wsUrl);
try {
const socket = io(wsUrl, {
// Try websocket first, fall back to polling if needed
transports: ['websocket', 'polling'],
// Explicitly set the path to match backend
path: '/socket.io',
// Connection timeout
timeout: 10000,
// Reconnection settings
reconnection: true,
reconnectionAttempts: 5,
reconnectionDelay: 1000,
reconnectionDelayMax: 5000,
// Upgrade settings for reverse proxies
upgrade: true,
rememberUpgrade: true,
// Force new connection instead of reusing
forceNew: false,
// Enable multiplexing
multiplex: true,
// Auto-connect
autoConnect: true,
});
socketRef.current = socket;
socket.on('connect', () => {
console.log('✅ WebSocket connected successfully via', socket.io.engine.transport.name);
setConnected(true);
setIsUsingWebSocket(true);
setError(null);
reconnectAttemptsRef.current = 0;
isConnectingRef.current = false;
// Stop polling when WebSocket connects
stopPolling();
// Request initial status via WebSocket
socket.emit('request_status');
});
socket.on('disconnect', (reason: string) => {
console.log('WebSocket disconnected. Reason:', reason);
setConnected(false);
setIsUsingWebSocket(false);
isConnectingRef.current = false;
// Start polling as fallback
startPolling();
// Attempt to reconnect WebSocket for most disconnect reasons
// 'io server disconnect' = server initiated disconnect
// 'transport close' = network error or server unreachable
// 'transport error' = transport failed (like websocket failed to connect)
if (reason !== 'io client disconnect') {
console.log('Attempting to reconnect WebSocket after disconnect:', reason);
attemptReconnect();
}
});
socket.on('connect_error', (err: Error) => {
console.error('WebSocket connection error:', err.message);
setError(`WebSocket error: ${err.message}`);
setConnected(false);
setIsUsingWebSocket(false);
isConnectingRef.current = false;
// Start polling immediately on connection error
startPolling();
// Attempt to reconnect WebSocket
attemptReconnect();
});
// Listen for status updates (full status refresh)
socket.on('status_update', (data: StatusData) => {
console.debug('[WS] status_update received', Object.keys(data));
setStatus(data);
setError(null);
});
// Listen for real-time progress updates (incremental)
socket.on('download_progress', (data: { book_id: string; progress: number; status: string }) => {
console.debug('[WS] download_progress:', data.book_id, `${data.progress.toFixed(1)}%`);
setStatus(prev => {
const newStatus = { ...prev };
// Update progress in downloading state
if (newStatus.downloading?.[data.book_id]) {
newStatus.downloading = {
...newStatus.downloading,
[data.book_id]: {
...newStatus.downloading[data.book_id],
progress: data.progress,
},
};
}
// Also check resolving state in case status update hasn't arrived yet
else if (newStatus.resolving?.[data.book_id]) {
// Book is resolving - progress will apply when it moves to downloading
}
return newStatus;
});
});
socket.on('error', (err: Error) => {
console.error('WebSocket error:', err);
setError('WebSocket error occurred');
});
} catch (err) {
console.error('Failed to initialize WebSocket:', err);
setError('Failed to initialize WebSocket');
isConnectingRef.current = false;
startPolling();
}
}, [wsUrl, stopPolling, startPolling, attemptReconnect]);
// Force refresh function
const forceRefresh = useCallback(async () => {
if (socketRef.current?.connected) {
// Request update via WebSocket
socketRef.current.emit('request_status');
} else {
// Poll immediately
await pollStatus();
}
}, [pollStatus]);
// Initialize on mount
useEffect(() => {
// Try WebSocket first
initializeWebSocket();
// If WebSocket doesn't connect within 3 seconds, start polling
const fallbackTimeout = setTimeout(() => {
if (!socketRef.current?.connected) {
console.log('WebSocket connection timeout, starting polling');
startPolling();
}
}, 3000);
// Cleanup
return () => {
clearTimeout(fallbackTimeout);
if (reconnectTimeoutRef.current) {
clearTimeout(reconnectTimeoutRef.current);
}
stopPolling();
if (socketRef.current) {
socketRef.current.disconnect();
socketRef.current = null;
}
};
}, [initializeWebSocket, startPolling, stopPolling]);
return {
status,
connected,
isUsingWebSocket,
error,
forceRefresh,
};
};
+25
View File
@@ -0,0 +1,25 @@
import { useState, useCallback } from 'react';
import { Toast } from '../types';
export const useToast = () => {
const [toasts, setToasts] = useState<Toast[]>([]);
const showToast = useCallback((message: string, type: 'info' | 'success' | 'error' = 'info', persistent: boolean = false): string => {
const id = Date.now().toString();
setToasts(prev => [...prev, { id, message, type }]);
if (!persistent) {
setTimeout(() => {
setToasts(prev => prev.filter(t => t.id !== id));
}, 4000);
}
return id;
}, []);
const removeToast = useCallback((id: string) => {
setToasts(prev => prev.filter(t => t.id !== id));
}, []);
return { toasts, showToast, removeToast };
};
+15
View File
@@ -0,0 +1,15 @@
import { StrictMode } from 'react';
import { createRoot } from 'react-dom/client';
import { BrowserRouter } from 'react-router-dom';
import App from './App';
const root = document.getElementById('root');
if (!root) throw new Error('Root element not found');
createRoot(root).render(
<StrictMode>
<BrowserRouter>
<App />
</BrowserRouter>
</StrictMode>
);
+36
View File
@@ -0,0 +1,36 @@
import { LoginForm } from '../components/LoginForm';
import { LoginCredentials } from '../types';
interface LoginPageProps {
onLogin: (credentials: LoginCredentials) => void;
error: string | null;
isLoading: boolean;
}
export const LoginPage = ({ onLogin, error, isLoading }: LoginPageProps) => {
return (
<div
className="min-h-screen flex items-center justify-center px-4 py-8"
style={{ backgroundColor: 'var(--background-color)', color: 'var(--text-color)' }}
>
<div className="w-full max-w-md">
<div className="text-center mb-8">
<img src="/logo.png" alt="Logo" className="mx-auto mb-6 w-20 h-20" />
<h1 className="text-2xl font-semibold">Sign in to continue</h1>
</div>
<div
className="rounded-lg shadow-2xl p-8 border"
style={{
backgroundColor: 'var(--card-background)',
borderColor: 'var(--border-color)',
color: 'var(--text-color)',
}}
>
<LoginForm onSubmit={onLogin} error={error} isLoading={isLoading} />
</div>
</div>
</div>
);
};
+108
View File
@@ -0,0 +1,108 @@
import { Book, StatusData, AppConfig, LoginCredentials, AuthResponse } from '../types';
const API_BASE = '/api';
// API endpoints
const API = {
search: `${API_BASE}/search`,
info: `${API_BASE}/info`,
download: `${API_BASE}/download`,
status: `${API_BASE}/status`,
cancelDownload: `${API_BASE}/download`,
setPriority: `${API_BASE}/queue`,
clearCompleted: `${API_BASE}/queue/clear`,
config: `${API_BASE}/config`,
login: `${API_BASE}/auth/login`,
logout: `${API_BASE}/auth/logout`,
authCheck: `${API_BASE}/auth/check`
};
// Custom error class for authentication failures
export class AuthenticationError extends Error {
constructor(message: string) {
super(message);
this.name = 'AuthenticationError';
}
}
// Utility function for JSON fetch with credentials
async function fetchJSON<T>(url: string, opts: RequestInit = {}): Promise<T> {
const res = await fetch(url, {
...opts,
credentials: 'include', // Enable cookies for session
headers: {
'Content-Type': 'application/json',
...opts.headers,
},
});
if (!res.ok) {
// Try to parse error message from response body
let errorMessage = `${res.status} ${res.statusText}`;
try {
const errorData = await res.json();
if (errorData.error) {
errorMessage = errorData.error;
}
} catch (e) {
// If we can't parse JSON, use the default error message
}
// Throw appropriate error based on status code
if (res.status === 401) {
throw new AuthenticationError(errorMessage);
}
throw new Error(errorMessage);
}
return res.json();
}
// API functions
export const searchBooks = async (query: string): Promise<Book[]> => {
if (!query) return [];
return fetchJSON<Book[]>(`${API.search}?${query}`);
};
export const getBookInfo = async (id: string): Promise<Book> => {
return fetchJSON<Book>(`${API.info}?id=${encodeURIComponent(id)}`);
};
export const downloadBook = async (id: string): Promise<void> => {
await fetchJSON(`${API.download}?id=${encodeURIComponent(id)}`);
};
export const getStatus = async (): Promise<StatusData> => {
return fetchJSON<StatusData>(API.status);
};
export const cancelDownload = async (id: string): Promise<void> => {
await fetchJSON(`${API.cancelDownload}/${encodeURIComponent(id)}/cancel`, { method: 'DELETE' });
};
export const clearCompleted = async (): Promise<void> => {
await fetchJSON(`${API_BASE}/queue/clear`, { method: 'DELETE' });
};
export const getConfig = async (): Promise<AppConfig> => {
return fetchJSON<AppConfig>(API.config);
};
// Authentication functions
export const login = async (credentials: LoginCredentials): Promise<AuthResponse> => {
return fetchJSON<AuthResponse>(API.login, {
method: 'POST',
body: JSON.stringify(credentials),
});
};
export const logout = async (): Promise<AuthResponse> => {
return fetchJSON<AuthResponse>(API.logout, {
method: 'POST',
});
};
export const checkAuth = async (): Promise<AuthResponse> => {
return fetchJSON<AuthResponse>(API.authCheck);
};
+547
View File
@@ -0,0 +1,547 @@
/* Base styles and CSS reset */
@tailwind base;
@tailwind components;
@tailwind utilities;
/* Disable transitions on initial load to prevent flash */
html.preload *,
html.preload *::before,
html.preload *::after {
transition: none !important;
animation-duration: 0s !important;
}
/* Smooth theme transitions for all elements using CSS variables */
header, footer, section,
input, select, textarea, button,
.details-container, .modal-overlay {
transition: background-color 0.2s ease, color 0.2s ease, border-color 0.2s ease;
}
:root {
/* Light theme variables */
--primary-color: oklch(44.3% 0.11 240.79);
--primary-dark: oklch(39.1% 0.09 240.876);
--text-color: #333;
--background-color: #f8f8f8;
--border-color: #e5e5e5;
--loading-overlay: rgba(0, 0, 0, 0.5);
--card-background: #fff;
--input-background: #fff;
--heading-color: #333;
/* Modern UI alias tokens */
--bg: var(--background-color);
--text: var(--text-color);
--border-muted: var(--border-color);
--bg-soft: var(--card-background);
--hover-surface: rgba(15, 23, 42, 0.05);
--hover-action: rgba(15, 23, 42, 0.08);
--hover-row: rgba(15, 23, 42, 0.07);
}
[data-theme="dark"] {
/* Dark theme variables with improved contrast */
--background-color: #121212;
--text-color: #ffffff;
--heading-color: #ffffff;
--card-background: #1e1e1e;
--border-color: #404040;
--input-background: #2d2d2d;
--hover-surface: rgba(255, 255, 255, 0.05);
--hover-action: rgba(255, 255, 255, 0.12);
--hover-row: rgba(255, 255, 255, 0.08);
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
/* Typography */
html, body {
min-height: 100vh;
/* Support for iOS notch/dynamic island - extend to full viewport */
min-height: 100dvh;
margin: 0;
padding: 0;
}
html {
/* Transition on root element ensures smooth theme changes */
transition: background-color 0.2s ease, color 0.2s ease;
/* Extend background color to safe areas (status bar area) */
background-color: var(--background-color);
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
line-height: 1.6;
color: var(--text-color);
background: var(--background-color);
transition: background-color 0.2s ease, color 0.2s ease;
display: flex;
flex-direction: column;
/* Safe areas handled by individual components (Header, Footer, Sidebar) */
/* Left/right safe areas still needed for notched devices in landscape */
padding-left: env(safe-area-inset-left);
padding-right: env(safe-area-inset-right);
}
#root {
display: flex;
flex-direction: column;
flex: 1;
min-height: 100vh;
min-height: 100dvh;
transition: background-color 0.2s ease;
background-color: var(--background-color);
}
main {
flex: 1;
min-height: 100%;
background: var(--background-color);
display: flex;
flex-direction: column;
transition: background-color 0.2s ease;
}
/* Layout */
footer {
text-align: center;
padding: 1rem;
margin-top: auto;
}
.spinner {
display: inline-block;
width: 1.25rem;
height: 1.25rem;
border: 3px solid rgba(255, 255, 255, 0.3);
border-radius: 50%;
border-top-color: white;
animation: spin 1s linear infinite;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
@keyframes slide-up {
from {
opacity: 0;
transform: translateY(30px);
}
to {
opacity: 1;
transform: translateY(0);
}
}
@keyframes fade-in-up {
from {
opacity: 0;
transform: translateY(10px);
}
to {
opacity: 1;
transform: translateY(0);
}
}
@keyframes fade-in-down {
from {
opacity: 0;
transform: translateY(-10px);
}
to {
opacity: 1;
transform: translateY(0);
}
}
@keyframes fade-out-up {
from {
opacity: 1;
transform: translateY(0);
}
to {
opacity: 0;
transform: translateY(-10px);
}
}
.animate-slide-up {
animation: slide-up 0.5s ease-out;
}
.animate-fade-in-up {
animation: fade-in-up 0.4s ease-out;
}
.animate-fade-in-down {
animation: fade-in-down 0.2s ease-out;
}
.animate-fade-out-up {
animation: fade-out-up 0.15s ease-in;
}
/* Button spinner styles */
.search-bar-spinner {
border-radius: 50%;
animation: spin 0.8s linear infinite;
transition: opacity 0.2s ease;
}
.search-bar-icon {
transition: opacity 0.2s ease;
}
.search-bar-button:disabled {
cursor: not-allowed;
}
/* Download button spinner styles */
.download-spinner {
border-radius: 50%;
animation: spin 0.8s linear infinite;
transition: opacity 0.2s ease;
flex-shrink: 0;
}
.details-spinner {
border-radius: 50%;
animation: spin 0.8s linear infinite;
transition: opacity 0.2s ease;
flex-shrink: 0;
}
/* Modal Styles */
.modal-overlay {
display: none;
position: fixed;
inset: 0;
width: 100%;
height: 100%;
background: var(--loading-overlay);
z-index: 1000;
padding: 1.5rem;
justify-content: center;
align-items: center;
backdrop-filter: blur(6px);
-webkit-backdrop-filter: blur(6px);
}
.modal-overlay.active {
display: flex;
}
.details-container {
width: 100%;
max-width: 64rem;
max-height: 90vh;
pointer-events: auto;
}
/* Responsive Design */
@media (max-width: 768px) {
.modal-overlay {
padding: 1rem;
}
.details-container {
max-width: none;
}
}
/* Accessibility */
.visually-hidden {
position: absolute;
width: 1px;
height: 1px;
padding: 0;
margin: -1px;
overflow: hidden;
clip: rect(0, 0, 0, 0);
white-space: nowrap;
border: 0;
}
/* Toast Notification Styles */
#toast-container {
max-width: 400px;
pointer-events: none;
}
.toast-notification {
opacity: 0;
transform: translateX(100%);
max-width: 100%;
word-wrap: break-word;
pointer-events: auto;
}
.toast-notification.toast-visible {
opacity: 1;
transform: translateX(0);
}
/* Disabled button styles for queued/downloading states */
button:disabled {
opacity: 0.6;
cursor: not-allowed;
}
.hover-surface,
.hover-action,
.hover-row {
transition: background-color 0.2s ease, color 0.2s ease;
}
.hover-surface:hover {
background-color: var(--hover-surface);
}
.hover-action:hover {
background-color: var(--hover-action);
}
.hover-row:hover {
background-color: var(--hover-row);
}
/* Search wrapper and input - base styles */
.search-wrapper {
width: 100%;
}
.search-input {
min-width: 0; /* Allow input to shrink on mobile flex containers */
-webkit-appearance: none;
appearance: none;
}
.search-input::-webkit-search-cancel-button,
.search-input::-webkit-search-decoration,
.search-input::-webkit-search-results-button,
.search-input::-webkit-search-results-decoration {
display: none;
}
.search-input::-ms-clear {
display: none;
width: 0;
height: 0;
}
/* ============================================
MOBILE: Full viewport width, no fixed widths
============================================ */
@media (max-width: 639px) {
/* Break search section out to full viewport width, escaping the centered main container */
#search-section {
position: relative;
left: 50%;
right: 50%;
margin-left: -50vw;
margin-right: -50vw;
width: 100vw;
padding-left: 1rem;
padding-right: 1rem;
box-sizing: border-box;
}
/* Remove any width constraints on search wrapper */
.search-wrapper {
width: 100% !important;
min-width: 0 !important;
max-width: none !important;
}
/* Ensure flex containers are full width */
.search-wrapper > .flex,
#search-filters {
width: 100%;
max-width: 100%;
min-width: 0;
}
}
/* ============================================
DESKTOP: Fixed px widths for consistency
============================================ */
/* Medium screens (small tablets/large phones): 640px - 1023px */
@media (min-width: 640px) and (max-width: 1023px) {
.search-wrapper {
width: 600px; /* Fixed width for medium screens */
max-width: 600px; /* Prevent expansion */
}
}
/* Large screens (desktop): 1024px+ */
@media (min-width: 1024px) {
.search-wrapper {
width: 800px; /* Fixed width for large screens */
max-width: 800px; /* Prevent expansion */
}
}
/* Search section base styles - always center horizontally on desktop */
#search-section {
display: flex;
flex-direction: column;
align-items: center; /* Keep search box centered horizontally */
}
/* Center search section vertically when in initial state (no results, no queue) */
/* Responsive to different window sizes */
.search-initial-state {
flex: 1;
justify-content: center; /* Only add vertical centering in initial state */
}
/* Mobile-friendly book card layout */
/* On mobile (below 640px), use horizontal layout: image left, text right, buttons below */
/* Desktop remains unchanged - full width, full height artwork via Tailwind classes */
@media (max-width: 639px) {
.search-initial-state {
justify-content: flex-start;
padding-top: 18vh; /* push the search UI toward the upper third */
}
/* When advanced filters are visible, pull the search bar to the very top */
.search-initial-state.search-advanced-visible {
padding-top: calc(env(safe-area-inset-top) + 1rem);
}
.book-card-content {
flex-direction: row !important;
gap: 0.75rem !important;
align-items: flex-start;
}
.book-card-cover {
width: 50% !important;
height: auto !important;
flex-shrink: 0;
aspect-ratio: 2/3; /* Maintain book cover proportions */
}
.book-card-text {
width: 50% !important;
flex-shrink: 0;
padding-left: 0.5rem;
}
.book-card-buttons {
width: 100%;
margin-top: 0.5rem;
}
}
/* Desktop: align details and buttons to bottom, artwork stays at top */
@media (min-width: 640px) {
.book-card {
/* Ensure card can grow to accommodate content */
display: flex;
flex-direction: column;
height: 100%; /* Ensure card fills grid cell height */
}
.book-card-content {
/* Grow to fill available space, pushing buttons to bottom */
flex: 1;
display: flex;
flex-direction: column;
min-height: 0; /* Allow flex shrinking */
}
.book-card-cover {
/* Artwork stays at top, doesn't grow */
flex-shrink: 0;
}
.book-card-text {
/* Push text to bottom of content area, right above buttons */
margin-top: auto;
flex: 0 0 auto !important; /* Override flex-1 from HTML, don't grow or shrink */
}
}
/* Fix download button text clipping on mobile */
.download-button-text {
white-space: nowrap;
overflow: visible;
flex-shrink: 0;
display: inline-block;
}
/* Ensure download buttons can accommodate their text content */
[data-action="download"],
#download-button {
min-width: 0;
overflow: visible;
width: 100%;
}
/* On mobile, ensure button text is fully visible */
@media (max-width: 639px) {
/* Ensure button container allows overflow */
.book-card-buttons {
overflow: visible;
}
/* Make text span flexible to use available space */
.download-button-text {
white-space: nowrap;
overflow: visible;
}
/* Ensure spinner doesn't interfere with text layout */
.download-spinner {
flex-shrink: 0;
flex-grow: 0;
width: 1rem;
height: 1rem;
}
}
/* Sticky Header with Gradient Fade Effect */
.header-with-fade {
overflow: visible;
}
.header-with-fade::after {
content: '';
position: absolute;
bottom: -20px;
left: 0;
right: 0;
height: 20px;
/* Use background-color instead of gradient with CSS variables for smooth transitions */
background: var(--bg);
opacity: 1;
pointer-events: none;
/* Create fade effect using mask instead of gradient */
mask-image: linear-gradient(to bottom, black 0%, black 20%, transparent 100%);
-webkit-mask-image: linear-gradient(to bottom, black 0%, black 20%, transparent 100%);
transition: background-color 0.2s ease, opacity 0.2s ease;
}
/* Skeleton Loader Animation */
@keyframes shimmer {
0% {
background-position: -200% 0;
}
100% {
background-position: 200% 0;
}
}
.animate-pulse {
background-size: 200% 100%;
animation: shimmer 2s ease-in-out infinite;
}
+89
View File
@@ -0,0 +1,89 @@
// Book data types
export interface Book {
id: string;
title: string;
author: string;
year?: string;
language?: string;
format?: string;
size?: string;
preview?: string;
publisher?: string;
info?: Record<string, string | string[]>;
description?: string;
download_path?: string;
progress?: number;
status_message?: string; // Detailed status message (e.g., "Trying Libgen (2/5)")
added_time?: number; // Timestamp when added to queue
}
// Status response types
export interface StatusData {
queued?: Record<string, Book>;
resolving?: Record<string, Book>;
downloading?: Record<string, Book>;
complete?: Record<string, Book>;
error?: Record<string, Book>;
cancelled?: Record<string, Book>;
}
export interface ActiveDownloadsResponse {
active_downloads: Book[];
}
// Button states
export type ButtonState = 'download' | 'queued' | 'resolving' | 'downloading' | 'complete' | 'error';
export interface ButtonStateInfo {
text: string;
state: ButtonState;
progress?: number; // Download progress 0-100
}
// Language option
export interface Language {
code: string;
language: string;
}
export interface AdvancedFilterState {
isbn: string;
author: string;
title: string;
lang: string[];
sort: string;
content: string;
formats: string[];
}
// Toast notification
export interface Toast {
id: string;
message: string;
type: 'success' | 'error' | 'info';
}
// App configuration
export interface AppConfig {
calibre_web_url: string;
debug: boolean;
build_version: string;
release_version: string;
book_languages: Language[];
default_language: string[];
supported_formats: string[];
}
// Authentication types
export interface LoginCredentials {
username: string;
password: string;
remember_me: boolean;
}
export interface AuthResponse {
success?: boolean;
authenticated?: boolean;
auth_required?: boolean;
error?: string;
}
@@ -0,0 +1,46 @@
import { AdvancedFilterState, Language } from '../types';
import { getLanguageFilterValues } from './languageFilters';
interface BuildSearchQueryOptions {
searchInput: string;
showAdvanced: boolean;
advancedFilters: AdvancedFilterState;
bookLanguages: Language[];
defaultLanguage: string[];
}
export const buildSearchQuery = ({
searchInput,
showAdvanced,
advancedFilters,
bookLanguages,
defaultLanguage,
}: BuildSearchQueryOptions): string => {
const queryParts: string[] = [];
const basic = searchInput.trim();
if (basic) {
queryParts.push(`query=${encodeURIComponent(basic)}`);
}
if (showAdvanced) {
const { isbn, author, title, content, formats, lang } = advancedFilters;
if (isbn) queryParts.push(`isbn=${encodeURIComponent(isbn)}`);
if (author) queryParts.push(`author=${encodeURIComponent(author)}`);
if (title) queryParts.push(`title=${encodeURIComponent(title)}`);
const selectedLanguages = getLanguageFilterValues(lang, bookLanguages, defaultLanguage);
selectedLanguages?.forEach(code => queryParts.push(`lang=${encodeURIComponent(code)}`));
if (content) queryParts.push(`content=${encodeURIComponent(content)}`);
formats.forEach(format => queryParts.push(`format=${encodeURIComponent(format)}`));
}
if (advancedFilters.sort) {
queryParts.push(`sort=${encodeURIComponent(advancedFilters.sort)}`);
}
return queryParts.join('&');
};
+89
View File
@@ -0,0 +1,89 @@
import { Language } from '../types';
export const LANGUAGE_OPTION_DEFAULT = 'default';
export const LANGUAGE_OPTION_ALL = 'all';
export const normalizeLanguageSelection = (selected: string[]): string[] => {
const sanitized = (selected ?? []).filter(Boolean);
if (sanitized.length === 0) {
return [LANGUAGE_OPTION_DEFAULT];
}
const unique: string[] = [];
const seen = new Set<string>();
for (const value of sanitized) {
if (!seen.has(value)) {
unique.push(value);
seen.add(value);
}
}
if (unique.includes(LANGUAGE_OPTION_ALL)) {
return [LANGUAGE_OPTION_ALL];
}
return unique.length ? unique : [LANGUAGE_OPTION_DEFAULT];
};
export const getLanguageFilterValues = (
selection: string[],
supportedLanguages: Language[],
defaultLanguageCodes: string[] = [],
): string[] | null => {
if (!selection || selection.length === 0) {
return null;
}
const uniqueSelection = Array.from(new Set(selection.filter(Boolean)));
if (uniqueSelection.includes(LANGUAGE_OPTION_ALL)) {
return [LANGUAGE_OPTION_ALL];
}
const onlyDefaultSelected =
uniqueSelection.length === 1 && uniqueSelection[0] === LANGUAGE_OPTION_DEFAULT;
if (onlyDefaultSelected) {
return null;
}
const supportedCodes = new Set(supportedLanguages.map(lang => lang.code));
const defaultCodes = defaultLanguageCodes.filter(code => supportedCodes.has(code));
const resolved = new Set<string>();
uniqueSelection.forEach(code => {
if (code === LANGUAGE_OPTION_DEFAULT) {
defaultCodes.forEach(defaultCode => resolved.add(defaultCode));
return;
}
if (supportedCodes.has(code)) {
resolved.add(code);
}
});
return resolved.size ? Array.from(resolved) : null;
};
export const formatDefaultLanguageLabel = (
languageCodes: string[],
supportedLanguages: Language[],
): string => {
if (!languageCodes || languageCodes.length === 0) {
return 'Default (env config)';
}
const languageNames = supportedLanguages
.filter(lang => languageCodes.includes(lang.code))
.map(lang => lang.language);
if (languageNames.length === 0) {
return 'Default (env config)';
}
const joined = languageNames.slice(0, 3).join(', ');
const suffix = languageNames.length > 3 ? '…' : '';
return `Default (${joined}${suffix})`;
};
+1
View File
@@ -0,0 +1 @@
/// <reference types="vite/client" />
+12
View File
@@ -0,0 +1,12 @@
/** @type {import('tailwindcss').Config} */
export default {
content: [
"./index.html",
"./src/**/*.{js,ts,jsx,tsx}",
],
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {},
},
plugins: [],
}
+25
View File
@@ -0,0 +1,25 @@
{
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
"jsx": "react-jsx",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true,
"baseUrl": ".",
"paths": {
"@/*": ["./src/*"]
}
},
"include": ["src"],
"references": [{ "path": "./tsconfig.node.json" }]
}
+11
View File
@@ -0,0 +1,11 @@
{
"compilerOptions": {
"composite": true,
"skipLibCheck": true,
"module": "ESNext",
"moduleResolution": "bundler",
"allowSyntheticDefaultImports": true,
"strict": true
},
"include": ["vite.config.ts"]
}
+31
View File
@@ -0,0 +1,31 @@
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
import path from 'path';
export default defineConfig({
plugins: [react()],
resolve: {
alias: {
'@': path.resolve(__dirname, './src'),
},
},
server: {
port: 5173,
host: '0.0.0.0',
strictPort: true,
cors: true,
proxy: {
// Proxy API requests to the Docker backend
'/api': {
target: 'http://localhost:8084',
changeOrigin: true,
secure: false,
},
// Proxy debug endpoint (uses /api/debug so it's automatically proxied above)
},
},
build: {
outDir: 'dist',
sourcemap: true,
},
});
-544
View File
@@ -1,544 +0,0 @@
/* Base styles and CSS reset */
:root {
/* Light theme variables */
--primary-color: #0073e6;
--primary-dark: #005bb5;
--text-color: #333;
--background-color: #f8f8f8;
--border-color: #e5e5e5;
--header-bg: #333;
--header-text: #fff;
--loading-overlay: rgba(0, 0, 0, 0.5);
--card-background: #fff;
--table-background: #fff;
--table-hover-background: #f8f8f8;
--table-border-color: #e5e5e5;
--input-background: #fff;
--heading-color: #333;
}
[data-theme="dark"] {
/* Dark theme variables with improved contrast */
--background-color: #121212;
--text-color: #ffffff;
--heading-color: #ffffff;
--card-background: #1e1e1e;
--border-color: #404040;
--table-background: #1e1e1e;
--table-hover-background: #2d2d2d;
--table-border-color: #404040;
--input-background: #2d2d2d;
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
/* Typography */
html, body {
min-height: 100vh;
margin: 0;
padding: 0;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
line-height: 1.6;
color: var(--text-color);
background: var(--background-color);
transition: background-color 0.3s ease, color 0.3s ease;
display: flex;
flex-direction: column;
}
main {
flex: 1;
min-height: 100%;
background: var(--background-color);
}
/* Layout */
header {
background: var(--header-bg);
color: var(--header-text);
padding: 1rem;
text-align: center;
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1);
}
footer {
text-align: center;
padding: 1rem;
background: var(--header-bg);
color: var(--header-text);
margin-top: auto;
}
/* Search Section */
.search-section {
margin-bottom: 2rem;
}
.search-container {
max-width: 600px;
margin: 0 auto;
display: flex;
gap: 0.5rem;
}
.search-container input {
flex: 1;
padding: 0.75rem;
border: 2px solid var(--border-color);
border-radius: 4px;
font-size: 1rem;
transition: border-color 0.3s ease;
}
.search-container input:focus {
border-color: var(--primary-color);
outline: none;
}
.search-filter {
max-width: 210px;
}
/* Table Styles */
.table-responsive {
margin-bottom: 1rem;
border-radius: 4px;
box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
}
table {
width: 100%;
border-collapse: collapse;
background: white;
}
th, td {
padding: 0.75rem;
border: 1px solid var(--border-color);
text-align: left;
justify-content:center;
}
th {
background: #f5f5f5;
font-weight: 600;
}
tbody tr:hover {
background: #f8f9fa;
}
.details-button {
padding: 0.75rem 1.5rem;
background: #27ae60;
color: white;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 1rem;
transition: background-color 0.3s ease;
width: 100%;
margin-bottom: 0.25rem;
}
.details-button:hover {
background: #1f894b;
}
/* Results Section */
.results-section {
margin-bottom: 2rem;
background: white;
border-radius: 4px;
box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
}
.results-heading {
padding: 1rem;
display: flex;
justify-content: space-between;
align-items: center;
cursor: pointer;
}
.toggle-icon {
transition: transform 0.3s ease;
}
.collapsed .toggle-icon {
transform: rotate(-90deg);
}
.results-content {
transition: max-height 0.3s ease;
}
.collapsed .results-content {
max-height: 0;
overflow: hidden;
}
th[data-sort="index"] {
min-width: 60px;
}
/* Loading Indicator */
.loading-indicator {
display: none;
text-align: center;
padding: 1rem;
}
.spinner {
display: inline-block;
width: 2rem;
height: 2rem;
border: 3px solid rgba(0, 0, 0, 0.1);
border-radius: 50%;
border-top-color: var(--primary-color);
animation: spin 1s linear infinite;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
/* Modal Styles */
.modal-overlay {
display: none;
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: var(--loading-overlay);
z-index: 1000;
}
.modal-overlay.active {
display: flex;
justify-content: center;
align-items: center;
}
.details-container {
background: white;
padding: 0.25rem 0.25rem 2rem 3rem;
border-radius: 4px;
max-width: 800px;
width: 90%;
max-height: 90vh;
overflow-y: auto;
position: relative;
box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
}
.details-header {
display: grid;
grid-template-columns: auto 1fr;
gap: 2rem;
margin-bottom: 1.5rem;
}
.details-header img {
max-width: 200px;
height: auto;
border-radius: 4px;
}
.details-info h3 {
margin-bottom: 1rem;
color: var(--text-color);
}
.details-info p {
margin-bottom: 0.5rem;
}
.details-actions {
display: flex;
gap: 1rem;
margin-top: 1.5rem;
justify-content: flex-end;
}
.details-actions button {
padding: 0.75rem 1.5rem;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 1rem;
transition: background-color 0.3s ease;
}
.details-actions button:first-child {
background: var(--primary-color);
color: white;
}
.details-actions button:last-child {
background: #f5f5f5;
color: var(--text-color);
}
.details-actions button:hover {
opacity: 0.9;
}
/* Responsive Design */
@media (max-width: 768px) {
.search-container {
flex-direction: column;
}
.details-header {
grid-template-columns: 1fr;
text-align: center;
}
.details-header img {
margin: 0 auto;
}
.details-actions {
flex-direction: column;
}
th, td {
padding: 0.5rem;
font-size: 0.875rem;
}
}
/* Accessibility */
.visually-hidden {
position: absolute;
width: 1px;
height: 1px;
padding: 0;
margin: -1px;
overflow: hidden;
clip: rect(0, 0, 0, 0);
white-space: nowrap;
border: 0;
}
/* Status indicators */
.status-queued {
color: #f39c12;
font-weight: 600;
text-align: center;
}
.status-downloading {
color: #3498db;
font-weight: 600;
text-align: center;
}
.status-available {
color: #27ae60;
font-weight: 600;
text-align: center;
}
.status-error {
color: #e74c3c;
font-weight: 600;
text-align: center;
}
.status-done {
color: black;
font-weight: 600;
text-align: center;
}
/* Status table specific styles */
#status-table img {
border-radius: 4px;
box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
}
#status-table td {
vertical-align: middle;
}
.error-message {
color: #e74c3c;
text-align: center;
padding: 1rem !important;
}
/* Dark mode specific styles */
[data-theme="dark"] .uk-card,
[data-theme="dark"] .uk-modal-dialog {
background-color: var(--card-background);
color: var(--text-color);
}
[data-theme="dark"] .uk-table {
color: var(--text-color);
}
[data-theme="dark"] .uk-table-hover tbody tr:hover {
background-color: rgba(255, 255, 255, 0.1);
}
[data-theme="dark"] .uk-button-default {
background-color: var(--card-background);
color: var(--text-color);
border-color: var(--border-color);
}
[data-theme="dark"] .uk-search-input {
background-color: var(--card-background);
color: var(--text-color);
border-color: var(--border-color);
}
[data-theme="dark"] .uk-select {
background-color: var(--card-background);
color: var(--text-color);
border-color: var(--border-color);
}
/* Table styles */
.uk-table {
background-color: var(--table-background);
color: var(--text-color);
border-color: var(--table-border-color);
}
.uk-table th {
color: var(--text-color);
background-color: var(--table-background);
border-bottom-color: var(--table-border-color);
}
.uk-table td {
color: var(--text-color);
background-color: var(--table-background);
border-bottom-color: var(--table-border-color);
}
.uk-table-hover tbody tr:hover,
.uk-table-hover tbody tr:hover td {
background-color: var(--table-hover-background);
}
/* Input styles */
.uk-search-input,
.uk-input,
.uk-select,
.uk-textarea {
background-color: var(--input-background);
color: var(--text-color);
border-color: var(--border-color);
}
/* Button styles */
.uk-button-default {
background-color: var(--card-background);
color: var(--text-color);
border-color: var(--border-color);
}
.uk-button-default:hover {
background-color: var(--table-hover-background);
color: var(--text-color);
border-color: var(--border-color);
}
/* Dropdown styles */
.uk-dropdown {
background-color: var(--card-background);
color: var(--text-color);
border-color: var(--border-color);
}
.uk-dropdown-nav > li > a {
color: var(--text-color);
}
.uk-dropdown-nav > li > a:hover {
background-color: var(--table-hover-background);
}
/* Modal styles */
.modal-overlay {
background-color: rgba(0, 0, 0, 0.5);
}
.details-container {
background-color: var(--card-background);
color: var(--text-color);
border-color: var(--border-color);
}
/* Status colors - ensure they remain visible in dark mode */
.status-downloading {
color: #4CAF50 !important;
}
.status-available {
color: #2196F3 !important;
}
.status-error {
color: #f44336 !important;
}
/* Header specific button styles */
header .uk-button-default {
color: var(--header-text) !important;
background-color: transparent;
border-color: var(--header-text);
}
header .uk-button-default:hover {
background-color: rgba(255, 255, 255, 0.1);
border-color: var(--header-text);
color: var(--header-text) !important;
}
/* Ensure dropdown text is visible when opened */
header .uk-dropdown {
background-color: var(--card-background);
}
header .uk-dropdown-nav > li > a {
color: var(--text-color);
}
header .uk-dropdown-nav > li > a:hover {
color: var(--text-color);
background-color: var(--table-hover-background);
}
/* Update headings to use heading color */
h1, h2, h3, h4, h5, h6,
.uk-heading-small,
.uk-heading-medium,
.uk-heading-large,
.uk-heading-xlarge,
.uk-heading-2xlarge {
color: var(--heading-color) !important;
}
/* Ensure accordion titles are visible */
.uk-accordion-title {
color: var(--heading-color) !important;
}
/* Ensure search results heading is visible */
#results-section-accordion .uk-accordion-title h1 {
color: var(--heading-color) !important;
}
-887
View File
@@ -1,887 +0,0 @@
// Main application JavaScript
document.addEventListener('DOMContentLoaded', () => {
// DOM Elements
const elements = {
search: {
searchInput: document.getElementById('search-input'),
searchButton: document.getElementById('search-button'),
advanced: {
advSearchButton: document.getElementById('adv-search-button'),
searchFiltersForm: document.getElementById('search-filters')
}
},
selectAllCheckbox: document.getElementById('select-all-checkbox'),
downloadSelectedButton: document.getElementById('download-selected-button'),
resultsSectionAccordion: document.getElementById('results-section-accordion'),
searchAccordion: document.getElementById('search-accordion'),
resultsHeading: document.getElementById('results-heading'),
resultsTable: document.getElementById('results-table'),
resultsTableBody: document.querySelector('#results-table tbody'),
searchLoading: document.getElementById('search-loading'),
statusLoading: document.getElementById('status-loading'),
statusTable: document.getElementById('status-table'),
statusTableBody: document.querySelector('#status-table tbody'),
modalOverlay: document.getElementById('modal-overlay'),
detailsContainer: document.getElementById('details-container'),
theme: {
toggle: document.getElementById('theme-toggle'),
text: document.getElementById('theme-text'),
dropdown: document.querySelector('[uk-dropdown]'),
},
debug: {
form: document.getElementById('debug-form'),
button: document.getElementById('debug-button'),
spinner: document.getElementById('debug-spinner')
},
restart: {
form: document.getElementById('restart-form'),
button: document.getElementById('restart-button'),
spinner: document.getElementById('restart-spinner')
}
};
// State
let modalDetails = null;
const selectedBooks = new Set();
const STATE = {
isSearching: false,
isLoadingDetails: false,
};
// Constants
const REFRESH_INTERVAL = 60000; // 60 seconds
const API_ENDPOINTS = {
search: '/request/api/search',
info: '/request/api/info',
download: '/request/api/download',
status: '/request/api/status'
};
const FILTERS = ['isbn', 'author', 'title', 'lang' , 'sort', "content", "format"];
// Utility Functions
const utils = {
debounce(func, wait) {
let timeout;
return function executedFunction(...args) {
const later = () => {
clearTimeout(timeout);
func(...args);
};
clearTimeout(timeout);
timeout = setTimeout(later, wait);
};
},
showLoading(element) {
element.removeAttribute('hidden');
},
hideLoading(element) {
element.setAttribute('hidden', '');
},
showAccordion(element) {
UIkit.accordion(element).toggle(1, true);
},
async fetchJson(url, options = {}) {
try {
const response = await fetch(url, options);
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
return await response.json();
} catch (error) {
console.error('Fetch error:', error);
throw error;
}
},
createElement(tag, attributes = {}, children = []) {
const element = document.createElement(tag);
Object.entries(attributes).forEach(([key, value]) => {
element[key] = value;
});
children.forEach(child => {
if (typeof child === 'string') {
element.appendChild(document.createTextNode(child));
} else {
element.appendChild(child);
}
});
return element;
},
sortResultsTable(column, order = 'asc') {
const rows = Array.from(elements.resultsTableBody.querySelectorAll('tr'));
const headers = document.querySelectorAll('#results-table thead th');
const parseFileSize = (size) => {
const match = size.match(/([\d.]+)([KMGT]B)/i);
if (!match) return 0;
const [_, value, unit] = match;
const multiplier = { KB: 1, MB: 1024, GB: 1024 * 1024, TB: 1024 * 1024 * 1024 };
return parseFloat(value) * (multiplier[unit.toUpperCase()] || 1);
};
const parseTitle = (title) => {
return title.replace(/^(The)\s+/i, '').trim();
}
const columnName = headers[column].textContent.trim().toLowerCase();
const getCellValue = (row, column) => {
const cell = row.querySelector(`td:nth-child(${column + 1})`);
const text = cell ? cell.textContent.trim() : '';
if (columnName === 'size') return parseFileSize(text);
if (columnName === 'title') return parseTitle(text);
return text;
};
rows.sort((a, b) => {
const valA = getCellValue(a, column);
const valB = getCellValue(b, column);
if (!isNaN(valA) && !isNaN(valB)) {
return order === 'asc' ? valA - valB : valB - valA;
}
return order === 'asc'
? valA.localeCompare(valB)
: valB.localeCompare(valA);
});
elements.resultsTableBody.innerHTML = '';
rows.forEach(row => elements.resultsTableBody.appendChild(row));
headers.forEach(header => {
const icon = header.querySelector('.sort-icon');
if (icon) {
icon.removeAttribute('uk-icon');
}
});
const currentHeader = headers[column];
const icon = currentHeader.querySelector('.sort-icon');
if (icon) {
icon.setAttribute('uk-icon', `icon: ${order === 'asc' ? 'triangle-up' : 'triangle-down'}`);
}
},
updateDownloadSelectedButton() {
elements.downloadSelectedButton.disabled = selectedBooks.size === 0;
},
handleCheckboxChange(event) {
const checkbox = event.target;
if (checkbox.checked) {
selectedBooks.add(checkbox.value);
} else {
selectedBooks.delete(checkbox.value);
}
utils.updateDownloadSelectedButton();
},
async handleDownloadSelected() {
if (selectedBooks.size === 0) return;
const bookIds = Array.from(selectedBooks);
const books = bookIds.map((bookId) => {
const row = document.querySelector(`#book-${bookId}`).closest('tr');
return {
title: row.querySelector('td:nth-child(4)').textContent,
author: row.querySelector('td:nth-child(5)').textContent
};
});
// Confirmation modal
const confirmationContent = `
<h2>Confirm Download</h2>
<p>Are you sure you want to download ${books.length} book${books.length > 1 ? 's' : ''}?</p>
<div class="uk-overflow-auto">
<table class="uk-table uk-table-divider uk-table-small">
<thead>
<tr>
<th>Title</th>
<th>Author</th>
</tr>
</thead>
<tbody>
${books.map((book) => `
<tr>
<td>${book.title}</td>
<td>${book.author}</td>
</tr>
`).join('')}
</tbody>
</table>
</div>
<div class="uk-flex uk-flex-between uk-margin-top">
<button id="cancel-download" class="uk-button uk-button-default">Cancel</button>
<button id="confirm-download" class="uk-button uk-button-primary">Download</button>
</div>
`;
elements.detailsContainer.innerHTML = confirmationContent;
document.getElementById('cancel-download').addEventListener('click', modal.close);
document.getElementById('confirm-download').addEventListener('click', async () => {
await utils.confirmDownload(bookIds);
});
modal.open();
},
async confirmDownload(bookIds) {
bookIds.map((bookId) =>
utils.fetchJson(`${API_ENDPOINTS.download}?id=${encodeURIComponent(bookId)}`)
);
this.clearAllCheckboxes();
modal.close();
},
clearAllCheckboxes() {
selectedBooks.forEach((bookId) => {
const checkbox = document.getElementById(`book-${bookId}`);
if (checkbox) checkbox.checked = false;
});
const selectAllCheckbox = document.getElementById('select-all-checkbox');
if (selectAllCheckbox) selectAllCheckbox.checked = false;
selectedBooks.clear();
utils.updateDownloadSelectedButton();
}
};
// Search Functions
const search = {
async performSearch(query) {
utils.clearAllCheckboxes();
if (STATE.isSearching) return;
try {
STATE.isSearching = true;
utils.showLoading(elements.searchLoading);
if (!elements.searchAccordion.classList.contains('uk-open')) {
utils.showAccordion(elements.resultsSectionAccordion);
};
const data = await utils.fetchJson(
`${API_ENDPOINTS.search}?${query}`
);
this.displayResults(data);
} catch (error) {
this.handleSearchError(error);
} finally {
STATE.isSearching = false;
utils.hideLoading(elements.searchLoading);
}
},
buildQuery() {
let queryParams = [];
if(elements.search.searchInput.value.trim()) {
queryParams.push(`query=${encodeURIComponent(elements.search.searchInput.value.trim())}`);
}
if(elements.search.advanced.searchFiltersForm.hasAttribute('hidden')) {
//Not advanced search
return queryParams.join('&');
}
FILTERS.forEach(filterType => {
if (filterType === 'format') {
// Handle format checkboxes
const checkboxes = document.querySelectorAll(`[id^="${filterType}-"]:checked`);
checkboxes.forEach(checkbox => {
queryParams.push(`${filterType}=${encodeURIComponent(checkbox.value)}`);
});
} else {
const inputs = document.querySelectorAll(`[id^="${filterType}-input"]`);
inputs.forEach(input => {
const value = input.value.trim();
if (value) {
queryParams.push(`${filterType}=${encodeURIComponent(value.trim())}`);
}
});
}
});
return queryParams.join('&');
},
displayResults(books) {
elements.resultsTableBody.innerHTML = '';
if (!books.length) {
this.displayNoResults();
return;
}
books.forEach((book, index) => {
const row = this.createBookRow(book, index);
elements.resultsTableBody.appendChild(row);
});
},
displayNoResults() {
const row = utils.createElement('tr', {}, [
utils.createElement('td', {
colSpan: '10',
textContent: 'No results found.'
})
]);
elements.resultsTableBody.appendChild(row);
},
createBookRow(book, index) {
const checkboxCell = utils.createElement('td', {}, [
utils.createElement('input', {
type: 'checkbox',
className: 'uk-checkbox',
id: 'book-' + book.id,
name: 'book-' + book.id,
value: book.id,
onchange: utils.handleCheckboxChange
})
]);
return utils.createElement('tr', {}, [
checkboxCell,
utils.createElement('td', { textContent: index + 1 }),
this.createPreviewCell(book.preview),
utils.createElement('td', { textContent: book.title || 'N/A' }),
utils.createElement('td', { textContent: book.author || 'N/A' }),
utils.createElement('td', { textContent: book.publisher || 'N/A' }),
utils.createElement('td', { textContent: book.year || 'N/A' }),
utils.createElement('td', { textContent: book.language || 'N/A' }),
utils.createElement('td', { textContent: book.format || 'N/A' }),
utils.createElement('td', { textContent: book.size || 'N/A' }),
this.createActionCell(book)
]);
},
createPreviewCell(previewUrl) {
if (!previewUrl) {
return utils.createElement('td', { textContent: 'N/A' });
}
const img = utils.createElement('img', {
src: previewUrl,
alt: 'Book Preview',
style: 'max-width: 60px;'
});
return utils.createElement('td', {}, [img]);
},
createActionCell(book) {
const buttonDetails = utils.createElement('button', {
className: 'uk-button uk-button-default uk-align-center uk-margin-small uk-width-1-1',
onclick: () => bookDetails.show(book.id)
}, [utils.createElement('span', { textContent: 'Details' })]);
const downloadButton = utils.createElement('button', {
className: 'uk-button uk-button-primary uk-align-center uk-margin-small uk-width-1-1',
onclick: () => bookDetails.downloadBook(book)
}, [utils.createElement('span', { textContent: 'Download' })]);
return utils.createElement('td', {}, [buttonDetails, downloadButton]);
},
handleSearchError(error) {
console.error('Search error:', error);
elements.resultsTableBody.innerHTML = '';
const errorRow = utils.createElement('tr', {}, [
utils.createElement('td', {
colSpan: '10',
textContent: 'An error occurred while searching. Please try again.'
})
]);
elements.resultsTableBody.appendChild(errorRow);
}
};
// Book Details Functions
const bookDetails = {
async show(bookId) {
if (STATE.isLoadingDetails) return;
try {
STATE.isLoadingDetails = true;
modal.open();
elements.detailsContainer.innerHTML = '<p>Loading details...</p>';
const book = await utils.fetchJson(
`${API_ENDPOINTS.info}?id=${encodeURIComponent(bookId)}`
);
modalDetails = book;
this.displayDetails(book);
} catch (error) {
this.handleDetailsError(error);
} finally {
STATE.isLoadingDetails = false;
}
},
displayDetails(book) {
elements.detailsContainer.innerHTML = this.generateDetailsHTML(book);
// Add event listeners
document.getElementById('download-button')
.addEventListener('click', () => this.downloadBook(book));
document.getElementById('close-details')
.addEventListener('click', modal.close);
},
generateDetailsHTML(book) {
return `
<div class="uk-card uk-card-default uk-child-width-1-2" uk-grid>
<div class="uk-card-media-left uk-cover-container uk-padding">
<img class="uk-height-medium" src="${book.preview || ''}" alt="Book Preview" uk-cover>
<canvas width="299" height="461"></canvas>
</div>
<div class="uk-card-body">
<h3>${book.title || 'No title available'}</h3>
<p><strong>Author:</strong> ${book.author || 'N/A'}</p>
<p><strong>Publisher:</strong> ${book.publisher || 'N/A'}</p>
<p><strong>Year:</strong> ${book.year || 'N/A'}</p>
<p><strong>Language:</strong> ${book.language || 'N/A'}</p>
<p><strong>Format:</strong> ${book.format || 'N/A'}</p>
<p><strong>Size:</strong> ${book.size || 'N/A'}</p>
</div>
<button id="download-button" class="uk-button uk-button-primary" type="button">Download</button>
<button id="close-details" class="uk-button uk-button-default uk-modal-close" type="button">Close</button>
</div>
<ul uk-accordion>
<li>
<a class="uk-accordion-title" href>Further Information</a>
<div class="uk-accordion-content">
${this.generateInfoList(book.info)}
</div>
</li>
</ul>
`;
},
generateInfoList(info) {
if (!info) return '';
const listItems = Object.entries(info)
.map(([key, values]) => `
<li><strong>${key}:</strong> ${values.join(', ')}</li>
`)
.join('');
return `<ul class="uk-list uk-list-bullet">${listItems}</ul>`;
},
async downloadBook(book) {
if (!book) return;
try {
utils.showLoading(elements.searchLoading);
await utils.fetchJson(
`${API_ENDPOINTS.download}?id=${encodeURIComponent(book.id)}`
);
modal.close();
status.fetch();
} catch (error) {
console.error('Download error:', error);
} finally {
utils.hideLoading(elements.searchLoading);
}
},
handleDetailsError(error) {
console.error('Details error:', error);
elements.detailsContainer.innerHTML = `
<p>Error loading details. Please try again.</p>
<div class="details-actions">
<button id="close-details" onclick="modal.close()">Close</button>
</div>
`;
document.getElementById('close-details')
.addEventListener('click', modal.close);
}
};
// Status Functions
const status = {
async fetch() {
try {
utils.showLoading(elements.statusLoading);
const data = await utils.fetchJson(API_ENDPOINTS.status);
this.display(data);
} catch (error) {
this.handleError(error);
} finally {
utils.hideLoading(elements.statusLoading);
}
},
display(data) {
elements.statusTableBody.innerHTML = '';
// Handle each status type
Object.entries(data).forEach(([status, booksInStatus]) => {
// If the status section has books
if (Object.keys(booksInStatus).length > 0) {
// For each book in this status
Object.entries(booksInStatus).forEach(([bookId, bookData]) => {
this.addStatusRow(status, bookData);
});
}
});
},
addStatusRow(status, book) {
if (!book.id || !book.title) return;
const statusCell = utils.createElement('td', {
className: `status-${status.toLowerCase()}`,
textContent: status
});
let titleElement;
if (book.download_path != null) {
titleElement = utils.createElement('a', {
href: `/request/api/localdownload?id=${book.id}`,
target: '_blank',
textContent: book.title || 'N/A'
});
}
else {
titleElement = utils.createElement('td', { textContent: book.title || 'N/A' })
}
const row = utils.createElement('tr', {}, [
statusCell,
utils.createElement('td', { textContent: book.id }),
titleElement,
this.createPreviewCell(book.preview)
]);
elements.statusTableBody.appendChild(row);
},
createPreviewCell(previewUrl) {
const cell = utils.createElement('td');
if (previewUrl) {
const img = utils.createElement('img', {
src: previewUrl,
alt: 'Book Preview',
style: 'max-width: 60px; height: auto;'
});
cell.appendChild(img);
} else {
cell.textContent = 'N/A';
}
return cell;
},
handleError(error) {
console.error('Status error:', error);
elements.statusTableBody.innerHTML = '';
const errorRow = utils.createElement('tr', {}, [
utils.createElement('td', {
colSpan: '4',
className: 'error-message',
textContent: 'Error loading status. Will retry automatically.'
})
]);
elements.statusTableBody.appendChild(errorRow);
}
};
// Modal Functions
const modal = {
open() {
elements.modalOverlay.classList.add('active');
},
close() {
elements.modalOverlay.classList.remove('active');
modalDetails = null;
}
};
// Theme Management
const theme = {
STORAGE_KEY: 'preferred-theme',
init() {
this.loadTheme();
this.setupListeners();
},
loadTheme() {
const savedTheme = localStorage.getItem(this.STORAGE_KEY) || 'auto';
this.applyThemePreference(savedTheme);
this.updateButtonText(savedTheme);
},
applyThemePreference(preference) {
if (preference === 'auto') {
const isDarkMode = window.matchMedia('(prefers-color-scheme: dark)').matches;
document.documentElement.setAttribute('data-theme', isDarkMode ? 'dark' : 'light');
} else {
document.documentElement.setAttribute('data-theme', preference);
}
},
setupListeners() {
// Listen for system theme changes
const darkModeMediaQuery = window.matchMedia('(prefers-color-scheme: dark)');
darkModeMediaQuery.addEventListener('change', (e) => {
if (localStorage.getItem(this.STORAGE_KEY) === 'auto') {
document.documentElement.setAttribute('data-theme', e.matches ? 'dark' : 'light');
}
});
// Theme selection listeners
const themeLinks = document.querySelectorAll('.uk-dropdown-nav a[data-theme]');
themeLinks.forEach(link => {
link.addEventListener('click', (e) => {
e.preventDefault();
const selectedTheme = e.target.getAttribute('data-theme');
localStorage.setItem(this.STORAGE_KEY, selectedTheme);
this.applyThemePreference(selectedTheme);
this.updateButtonText(selectedTheme);
});
});
},
updateButtonText(currentTheme) {
const themeText = currentTheme.charAt(0).toUpperCase() + currentTheme.slice(1);
const themeTextElement = document.getElementById('theme-text');
if (themeTextElement) {
themeTextElement.textContent = themeText;
}
}
};
// Debug Functions
const debug = {
init() {
if (!elements.debug.form) return;
elements.debug.form.addEventListener('submit', (e) => {
e.preventDefault(); // Prevent default form submission
if (elements.debug.button.disabled) {
return; // Prevent multiple submissions
}
// Disable button and show spinner
elements.debug.spinner.classList.remove('uk-hidden');
elements.debug.button.disabled = true;
// Use fetch to make the request
fetch(elements.debug.form.action)
.then(response => {
if (!response.ok) {
throw new Error('Debug request failed');
}
// Store content disposition before returning blob
const contentDisposition = response.headers.get('Content-Disposition');
return response.blob().then(blob => {
return { blob, contentDisposition, url: response.url };
});
})
.then(data => {
// Create and click a download link
const url = URL.createObjectURL(data.blob);
const a = document.createElement('a');
a.style.display = 'none';
a.href = url;
// Extract filename from Content-Disposition header, URL, or use default
let filename;
if (data.contentDisposition) {
const filenameRegex = /filename[^;=\n]*=((['"]).*?\2|[^;\n]*)/;
const matches = filenameRegex.exec(data.contentDisposition);
if (matches && matches[1]) {
filename = matches[1].replace(/['"]/g, '');
}
}
if (!filename) {
// Extract from URL if header is not available
const urlParts = data.url.split('/');
filename = urlParts[urlParts.length - 1];
}
a.download = filename || "cwa-book-downloader-debug.zip";
document.body.appendChild(a);
a.click();
// Clean up
setTimeout(() => {
URL.revokeObjectURL(url);
document.body.removeChild(a);
// Only reset UI after download starts
elements.debug.spinner.classList.add('uk-hidden');
elements.debug.button.disabled = false;
}, 100);
})
.catch(error => {
console.error('Debug download error:', error);
// Reset UI on error
elements.debug.spinner.classList.add('uk-hidden');
elements.debug.button.disabled = false;
// Show error notification
UIkit.notification({
message: 'Error generating debug file. Please try again.',
status: 'danger',
pos: 'top-center',
timeout: 5000
});
});
});
}
};
// Restart Functions
const restart = {
init() {
if (!elements.restart.form) return;
elements.restart.form.addEventListener('submit', (e) => {
e.preventDefault(); // Prevent default form submission
if (elements.restart.button.disabled) {
return; // Prevent multiple submissions
}
// Show confirmation dialog
UIkit.modal.confirm('Are you sure you want to restart the application? This will interrupt any ongoing downloads.').then(() => {
// User confirmed, proceed with restart
elements.restart.spinner.classList.remove('uk-hidden');
elements.restart.button.disabled = true;
// Show restarting message
UIkit.notification({
message: 'Application restarting...',
status: 'warning',
pos: 'top-center',
timeout: 5000
});
// Make the restart request - this will kill the server
fetch(elements.restart.form.action)
.catch(() => {
// Expected to fail when server shuts down
});
// Wait a bit then refresh the page to reconnect to the restarted server
setTimeout(() => {
window.location.reload();
}, 3000);
}, () => {
// User cancelled, do nothing
});
});
}
};
// Event Listeners
function setupEventListeners() {
// Search events
elements.search.searchButton.addEventListener('click', () => {
const query = search.buildQuery();
if(query) search.performSearch(query);
});
elements.search.advanced.advSearchButton.addEventListener('click', () => {
const query = search.buildQuery();
if(query) search.performSearch(query);
});
elements.search.searchInput.addEventListener('keydown', (e) => {
if (e.key === 'Enter') {
e.preventDefault();
const query = search.buildQuery();
if(query) search.performSearch(query);
}
});
// Modal close on overlay click
elements.modalOverlay.addEventListener('click', (e) => {
if (e.target === elements.modalOverlay) {
modal.close();
}
});
// Keyboard accessibility
document.addEventListener('keydown', (e) => {
if (e.key === 'Escape' && elements.modalOverlay.classList.contains('active')) {
modal.close();
}
});
// Download selected books
elements.downloadSelectedButton.addEventListener('click', utils.handleDownloadSelected);
// Check/uncheck all book checkboxes
elements.selectAllCheckbox.addEventListener('change', (event) => {
const isChecked = event.target.checked;
document.querySelectorAll('.uk-checkbox').forEach((checkbox) => {
if (checkbox !== elements.selectAllCheckbox) {
checkbox.checked = isChecked;
if (isChecked) {
selectedBooks.add(checkbox.value);
} else {
selectedBooks.delete(checkbox.value);
}
}
});
utils.updateDownloadSelectedButton();
});
function setupSorting() {
const headers = document.querySelectorAll('#results-table thead th[data-sort]');
headers.forEach((header) => {
let sortOrder = 'asc';
header.addEventListener('click', () => {
const allHeaders = Array.from(document.querySelectorAll('#results-table thead th'));
const columnIndex = allHeaders.indexOf(header);
utils.sortResultsTable(columnIndex, sortOrder);
sortOrder = sortOrder === 'asc' ? 'desc' : 'asc';
});
});
}
setupSorting();
}
// Initialize
function init() {
setupEventListeners();
theme.init(); // Initialize theme management
debug.init(); // Initialize debug functionality
restart.init(); // Initialize restart functionality
status.fetch();
setInterval(() => status.fetch(), REFRESH_INTERVAL);
}
init();
});
Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

-233
View File
@@ -1,233 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Calibre Web Book Downloader Application">
<meta name="theme-color" content="#333333">
<title>Calibre Web Book Downloader</title>
<link rel="stylesheet" href="{{ url_for('static', filename='css/styles.css') }}">
<!-- Add favicon -->
<link rel="icon" type="image/x-icon" href="{{ url_for('static', filename='favicon.ico') }}">
<!-- UIkit CSS -->
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/uikit@3.21.16/dist/css/uikit.min.css" />
<!-- UIkit JS -->
<script src="https://cdn.jsdelivr.net/npm/uikit@3.21.16/dist/js/uikit.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/uikit@3.21.16/dist/js/uikit-icons.min.js"></script>
</head>
<body>
<header>
<div class="uk-flex uk-flex-between uk-flex-middle">
<h1 class="uk-heading-medium" style="color: white;">Book Search & Download</h1>
<div class="uk-margin-right">
{% if debug %}
<form action="/api/restart" method="get" style="display: inline;" id="restart-form">
<button class="uk-button uk-button-danger uk-margin-small-right" id="restart-button" type="submit">
RESTART <span uk-spinner="ratio: 0.8" class="uk-hidden" id="restart-spinner"></span>
</button>
</form>
<form action="/debug" method="get" style="display: inline;" id="debug-form">
<button class="uk-button uk-button-danger uk-margin-small-right" id="debug-button" type="submit">
DEBUG <span uk-spinner="ratio: 0.8" class="uk-hidden" id="debug-spinner"></span>
</button>
</form>
{% endif %}
<button class="uk-button uk-button-default" type="button" id="theme-toggle">
<span uk-icon="icon: paint-bucket"></span>
<span id="theme-text">Theme</span>
</button>
<div uk-dropdown="mode: click">
<ul class="uk-nav uk-dropdown-nav">
<li><a href="#" data-theme="light">Light</a></li>
<li><a href="#" data-theme="dark">Dark</a></li>
<li><a href="#" data-theme="auto">Auto (System)</a></li>
</ul>
</div>
</div>
</div>
</header>
<main>
<div class="uk-container">
<!-- Search Section -->
<section class="uk-section-xxsmall uk-padding">
<div class="uk-container uk-width-expand">
<form class="uk-search uk-search-default uk-width-expand">
<input class="uk-search-input" type="search" placeholder="Search by ISBN, title, author..." aria-label="Search books" id="search-input">
<button class="uk-search-icon-flip" uk-search-icon id="search-button" type="button"></button>
</form>
<button class="uk-button uk-button-default uk-float-right" uk-toggle="target: #search-filters" type="button">Advanced Search</button>
<form hidden id="search-filters" class="uk-search uk-search-default uk-inline uk-width-expand uk-flex uk-flex-wrap">
<div class="search-filter">
<label for="isbn-input">ISBN</label>
<div class="uk-flex uk-flex-wrap">
<input class="uk-search-input uk-width-4-5" type="search" placeholder="ISBN..." aria-label="Search by ISBN" id="isbn-input">
</div>
</div>
<div class="search-filter">
<label for="author-input">Author</label>
<div class="uk-flex uk-flex-wrap">
<input class="uk-search-input uk-width-4-5" type="search" placeholder="Author..." aria-label="Search by author" id="author-input">
</div>
</div>
<div class="search-filter">
<label for="title-input">Title</label>
<div class="uk-flex uk-flex-wrap">
<input class="uk-search-input uk-width-4-5" type="search" placeholder="Title" aria-label="Search by title" id="title-input">
</div>
</div>
<div class="search-filter">
<label for="lang-input">Language</label>
<div class="uk-flex uk-flex-wrap">
<select class="uk-select uk-width-4-5" id="lang-input">
<option value="all">All</option>
{% for lang in book_languages %}
<option value="{{ lang.code }}"
{% if lang.code == default_language[0] %}selected{% endif %}>
{{ lang.language }}
</option>
{% endfor %}
</select>
</div>
</div>
<div class="search-filter">
<label for="sort-input">Sort</label>
<div class="uk-flex uk-flex-wrap">
<select class="uk-select uk-width-4-5" id="sort-input">
<option value="">Most relevant</option>
<option value="newest">Newest (publication year)</option>
<option value="oldest">Oldest (publication year)</option>
<option value="largest">Largest (filesize)</option>
<option value="smallest">Smallest (filesize)</option>
<option value="newest_added">Newest (open sourced)</option>
<option value="oldest_added">Oldest (open sourced)</option>
</select>
</div>
</div>
<div class="search-filter">
<label for="content-input">Content</label>
<div class="uk-flex uk-flex-wrap">
<select class="uk-select uk-width-4-5" id="content-input">
<option value="">All</option>
<option value="book_nonfiction">Book (non-fiction)</option>
<option value="book_fiction">Book (fiction)</option>
<option value="book_unknown">Book (unkown)</option>
<option value="magazine">Magazine</option>
<option value="book_comic">Comic Book</option>
<option value="standards_document">Standards document</option>
<option value="other">Other</option>
<option value="musical_score">Musical score</option>
<option value="audiobook">Audiobook</option>
</select>
</div>
</div>
<div class="search-filter">
<label>Formats</label>
<div class="uk-margin uk-grid-small uk-child-width-auto uk-grid">
<label><input class="uk-checkbox" type="checkbox" id="format-pdf" value="pdf"> PDF</label>
<label><input class="uk-checkbox" type="checkbox" id="format-epub" value="epub" checked> EPUB</label>
<label><input class="uk-checkbox" type="checkbox" id="format-mobi" value="mobi" checked> MOBI</label>
<label><input class="uk-checkbox" type="checkbox" id="format-azw3" value="azw3" checked> AZW3</label>
<label><input class="uk-checkbox" type="checkbox" id="format-fb2" value="fb2" checked> FB2</label>
<label><input class="uk-checkbox" type="checkbox" id="format-djvu" value="djvu" checked> DJVU</label>
<label><input class="uk-checkbox" type="checkbox" id="format-cbz" value="cbz" checked> CBZ</label>
<label><input class="uk-checkbox" type="checkbox" id="format-cbr" value="cbr" checked> CBR</label>
</div>
</div>
<div class="uk-flex uk-margin-auto-top uk-margin-auto-left">
<button class="uk-button uk-button-default uk-margin-small-top" id="adv-search-button" type="button">Search</button>
</div>
</form>
</div>
</section>
<!-- Results Section -->
<ul uk-accordion="animation: false" id="results-section-accordion">
<li id="search-accordion">
<a class="uk-accordion-title" href><h1 class="uk-heading-xsmall">Search Results</h1></a>
<div class="uk-accordion-content">
<div id="search-loading" class="uk-flex uk-flex-center" role="status" hidden>
<div uk-spinner="ratio: 2"></div>
<span>Loading...</span>
</div>
<div class="results-content">
<button class="uk-button uk-button-primary uk-margin-small" id="download-selected-button" disabled>Download Selected</button>
<div class="uk-overflow-auto">
<table id="results-table" class="uk-table uk-table-hover uk-table-divider" role="grid">
<thead>
<tr>
<th>
<input type="checkbox" id="select-all-checkbox" class="uk-checkbox" />
</th>
<th scope="col" data-sort="index"># <span class="sort-icon" uk-icon></span></th>
<th scope="col">Preview</th>
<th scope="col" data-sort="title">Title <span class="sort-icon" uk-icon></span></th>
<th scope="col" data-sort="author">Author <span class="sort-icon" uk-icon></span></th>
<th scope="col" data-sort="publisher">Publisher <span class="sort-icon" uk-icon></span></th>
<th scope="col" data-sort="year">Year <span class="sort-icon" uk-icon></span></th>
<th scope="col" data-sort="language">Language <span class="sort-icon" uk-icon></span></th>
<th scope="col" data-sort="format">Format <span class="sort-icon" uk-icon></span></th>
<th scope="col" data-sort="size">Size <span class="sort-icon" uk-icon></span></th>
<th scope="col">Actions</th>
</tr>
</thead>
<tbody>
<!-- Search results will be injected here -->
</tbody>
</table>
</div>
</div>
</div>
</li>
</ul>
<!-- Modal -->
<div class="modal-overlay" id="modal-overlay" role="dialog" aria-modal="true">
<div class="details-container" id="details-container">
<!-- Details will be dynamically injected here -->
</div>
</div>
<!-- Status Section -->
<ul uk-accordion="animation: false" id="status-section">
<li id="search-accordion">
<a class="uk-accordion-title" href><h1 class="uk-heading-xsmall">Download Status</h1></a>
<div class="uk-accordion-content">
<div id="status-loading" class="uk-flex uk-flex-center" role="status" hidden>
<div uk-spinner="ratio: 2"></div>
<span>Loading...</span>
</div>
<div class="status-content">
<div class="uk-overflow-auto">
<table id="status-table" class="uk-table uk-table-hover uk-table-divider" role="grid">
<thead>
<tr>
<th scope="col">Status</th>
<th scope="col">Book ID</th>
<th scope="col">Title</th>
<th scope="col">Preview</th>
</tr>
</thead>
<tbody>
<!-- Search results will be injected here -->
</tbody>
</table>
</div>
</div>
</div>
</li>
</ul>
</div>
</main>
<footer>
<h1 class="uk-heading-xxsmall" style="color: white;">Calibre Web Book Downloader</h1>
<a href="https://github.com/calibrain/calibre-web-automated-book-downloader" class="uk-icon-link" uk-icon="icon:github; ratio: 2"></a>
</footer>
<!-- Scripts -->
<script src="{{ url_for('static', filename='js/main.js') }}" defer></script>
</body>
</html>
+20 -8
View File
@@ -6,13 +6,14 @@ import hashlib
# Thee server is already running, so let's grab some of the env vars:
# Use absolute import since the script is run from the root directory
import env as SERVER_ENV
from backend import _sanitize_filename # Moved import to top level
from models import BookInfo
# Now let's test the server:
port = SERVER_ENV.FLASK_PORT
server_url = f"http://localhost:{port}"
book_title = "077484a10743e5dd5d151013e8c732f4" # "Moby Dick"
# Directory where downloads should appear
download_paths = SERVER_ENV.DOWNLOAD_PATHS
download_dir = SERVER_ENV.INGEST_DIR
# Timeout for waiting for download
download_timeout_seconds = 60 * 5
@@ -38,7 +39,7 @@ def check_download_status(book_id):
continue
# Check success conditions based on download_path
for status_key in ["available", "done"]:
for status_key in ["available", "done", "complete"]:
if status_key in status_data and book_id in status_data[status_key]:
book_status_info = status_data[status_key].get(book_id)
# Check if the status info is a dictionary and has a non-empty download_path
@@ -108,15 +109,26 @@ print(f"Book {book_id} download confirmed as available.")
# Step 5 : Verify the file exists locally (optional but good)
print(f"Step 5: Verifying downloaded file exists...")
# Depend if env.USE_TITLE is true or false, the filename will be different
# Depend if env.USE_BOOK_TITLE is true or false, the filename will be different
if SERVER_ENV.USE_BOOK_TITLE:
# Ensure book_details is available; might need adjustment if Step 2 failed
# Assuming book_details was successfully fetched in Step 2
title_to_sanitize = book_details.get('title', book_title) # Use fetched title if available
expected_filename = _sanitize_filename(title_to_sanitize) + ".epub" # Add extension
# Build expected filename using BookInfo
book_info = BookInfo(
id=book_id,
title=book_details.get('title', ''),
author=book_details.get('author'),
year=book_details.get('year'),
format='epub'
)
expected_filename = book_info.get_filename()
else:
expected_filename = f"{book_id}.epub"
if book_details.get("content"):
content = book_details.get("content")
for key, path in SERVER_ENV.DOWNLOAD_PATHS.items():
if key in content:
download_dir = path
break
expected_filepath = os.path.join(download_dir, expected_filename)
assert os.path.exists(expected_filepath), f"Expected downloaded file not found at: {expected_filepath}"
@@ -124,7 +136,7 @@ print(f"Verified file exists: {expected_filepath}")
# Step 6 : Download the book
print(f"Step 6: Downloading book {book_id}...")
download_response = requests.get(f"{server_url}/request/api/localdownload?id={book_id}")
download_response = requests.get(f"{server_url}/api/localdownload?id={book_id}")
download_response.raise_for_status()
# Write book to temp file :
temp_file_path = os.path.join("/tmp", f"{book_id}.epub")
+94 -6
View File
@@ -11,9 +11,40 @@ echo "Log file: $LOG_FILE"
set +x
set -e
#!/bin/bash
# Check if EXT_BYPASSER_URL is defined
if [ -n "$EXT_BYPASSER_URL" ]; then
echo "Extracting hostname and ip from bypasser into /etc/hosts"
# Extract hostname
hostname=$(echo "$EXT_BYPASSER_URL" | cut -d'/' -f3 | cut -d':' -f1)
# Resolve to IP (using current DNS before switching to TOR)
ip=$(getent hosts "$hostname" 2>/dev/null | awk '{print $1}')
# If getent fails, try dig
if [ -z "$ip" ]; then
ip=$(dig +short "$hostname" 2>/dev/null | head -n1)
fi
# Only proceed if we got an IP and hostname is not already an IP
if [ -n "$ip" ] && [ "$ip" != "$hostname" ]; then
# Add to /etc/hosts (remove existing entry first to avoid duplicates)
sudo sed -i "/[[:space:]]$hostname$/d" /etc/hosts
echo "$ip $hostname" | sudo tee -a /etc/hosts > /dev/null
echo "Added to /etc/hosts: $ip $hostname"
else
echo "Skipping: $hostname is already an IP or could not be resolved"
fi
else
echo "EXT_BYPASSER_URL not defined, skipping /etc/hosts update"
fi
echo "[*] Running tor script..."
echo "Build version: $BUILD_VERSION"
echo "Release version: $RELEASE_VERSION"
echo "[*] Installing Tor and dependencies..."
echo "[*] Writing Tor transparent proxy config..."
@@ -24,6 +55,19 @@ AutomapHostsOnResolve 1
TransPort 9040
DNSPort 53
Log notice file /var/log/tor/notices.log
# Circuit management to prevent stale circuits after inactivity
MaxCircuitDirtiness 600
NewCircuitPeriod 30
CircuitBuildTimeout 60
LearnCircuitBuildTimeout 0
# Keep circuits alive
KeepalivePeriod 60
CircuitStreamTimeout 60
# Prevent connection timeouts
SocksTimeout 120
EOF
echo "[*] Setting up DNS..."
@@ -61,16 +105,20 @@ iptables -t nat -A OUTPUT -p tcp --syn -j REDIRECT --to-ports 9040
# For UDP DNS queries
iptables -t nat -A OUTPUT -p udp --dport 53 ! -d 127.0.0.1 -j DNAT --to-destination 127.0.0.1:53
# For TCP DNS queries (some DNS queries may use TCP)
iptables -t nat -A OUTPUT -p tcp --dport 53 ! -d 127.0.0.1 -j DNAT --to-destination 127.0.0.1:53
# Note: ICMP (ping) is NOT routed through Tor as Tor only supports TCP.
# ICMP will use default routing. If you need to test connectivity, use:
# curl -s https://check.torproject.org/api/ip
# or: curl -s https://icanhazip.com
echo "[✓] Transparent Tor routing enabled."
sleep 5
# Check if outgoing IP is using Tor
echo "[*] Verifying Tor connectivity..."
RESULT=$(pyrequests https://check.torproject.org/api/ip)
RESULT=$(curl -s https://check.torproject.org/api/ip)
echo "RESULT: $RESULT"
IS_TOR=$(echo "$RESULT" | grep -oP '"IsTor":\s*\K(true|false)')
IP=$(echo "$RESULT" | grep -oP '"IP":\s*"\K[^"]+')
@@ -87,10 +135,10 @@ fi
# Get timezone from IP
sleep 1
TIMEZONE=$(pyrequests https://ipapi.co/timezone) || \
TIMEZONE=$(pyrequests http://ip-api.com/line?fields=timezone) || \
TIMEZONE=$(pyrequests http://worldtimeapi.org/api/ip | grep -oP '"timezone":"\K[^"]+') || \
TIMEZONE=$(pyrequests https://ip2tz.isthe.link/v2 | grep -oP '"timezone": *"\K[^"]+') || \
TIMEZONE=$(curl -s https://ipapi.co/timezone) || \
TIMEZONE=$(curl -s http://ip-api.com/line?fields=timezone) || \
TIMEZONE=$(curl -s http://worldtimeapi.org/api/ip | grep -oP '"timezone":"\K[^"]+') || \
TIMEZONE=$(curl -s https://ip2tz.isthe.link/v2 | grep -oP '"timezone": *"\K[^"]+') || \
true
# If TIMEZONE is not set, use the default timezone
@@ -117,5 +165,45 @@ else
echo "[*] Falling back to container's default timezone: $TZ"
fi
# Start a background health check process to monitor Tor
echo "[*] Starting Tor health check monitor..."
(
check_count=0
while true; do
sleep 300 # Check every 5 minutes
check_count=$((check_count + 1))
echo "[*] Tor health check #$check_count at $(date)"
# Check if Tor service is running
if ! service tor status > /dev/null 2>&1; then
echo "[!] $(date): Tor service not running, restarting..."
service tor restart
sleep 10
fi
# Test DNS resolution through Tor
if ! timeout 10 nslookup google.com 127.0.0.1 > /dev/null 2>&1; then
echo "[!] $(date): DNS resolution failed, reloading Tor..."
service tor reload
sleep 5
# Verify DNS works after reload
if timeout 10 nslookup google.com 127.0.0.1 > /dev/null 2>&1; then
echo "[✓] $(date): DNS resolution restored"
else
echo "[✗] $(date): DNS still failing after reload, restarting Tor..."
service tor restart
sleep 10
fi
fi
# Send SIGHUP to Tor to rotate circuits (helps with stale circuits)
echo "[*] $(date): Rotating Tor circuits..."
pkill -HUP tor || true
done
) >> $LOG_FILE 2>&1 &
TOR_MONITOR_PID=$!
echo "[✓] Tor health check monitor started in background (PID: $TOR_MONITOR_PID)"
# Run the entrypoint script
echo "[*] End of tor script"
+160
View File
@@ -0,0 +1,160 @@
"""WebSocket manager for real-time status updates."""
import logging
import threading
from typing import Optional, Dict, Any, Callable, List
from flask_socketio import SocketIO, emit
logger = logging.getLogger(__name__)
class WebSocketManager:
"""Manages WebSocket connections and broadcasts."""
def __init__(self):
self.socketio: Optional[SocketIO] = None
self._enabled = False
self._connection_count = 0
self._connection_lock = threading.Lock()
self._on_first_connect_callbacks: List[Callable[[], None]] = []
self._on_all_disconnect_callbacks: List[Callable[[], None]] = []
self._needs_rewarm = False # Flag to trigger warmup callbacks on next connect
def init_app(self, app, socketio: SocketIO):
"""Initialize the WebSocket manager with Flask-SocketIO instance."""
self.socketio = socketio
self._enabled = True
logger.info("WebSocket manager initialized")
def register_on_first_connect(self, callback: Callable[[], None]):
"""Register a callback to be called when the first client connects.
This is useful for warming up resources (like the Cloudflare bypasser)
when a user starts using the web UI.
"""
self._on_first_connect_callbacks.append(callback)
logger.debug(f"Registered on_first_connect callback: {callback.__name__}")
def register_on_all_disconnect(self, callback: Callable[[], None]):
"""Register a callback to be called when all clients disconnect.
This can be used to trigger cleanup or resource release.
"""
self._on_all_disconnect_callbacks.append(callback)
logger.debug(f"Registered on_all_disconnect callback: {callback.__name__}")
def request_warmup_on_next_connect(self):
"""Request that warmup callbacks be triggered on the next client connect.
This is used when resources (like the Cloudflare bypasser) shut down due to
inactivity while clients are still connected. The next connect event should
trigger warmup even though it's not technically the "first" connection.
"""
with self._connection_lock:
self._needs_rewarm = True
logger.debug("Warmup requested for next client connect")
def client_connected(self):
"""Track a new client connection. Call this from the connect event handler."""
with self._connection_lock:
was_zero = self._connection_count == 0
needs_rewarm = self._needs_rewarm
self._connection_count += 1
current_count = self._connection_count
# Clear rewarm flag if we're going to trigger warmup
if was_zero or needs_rewarm:
self._needs_rewarm = False
logger.debug(f"Client connected. Active connections: {current_count}")
# Trigger warmup callbacks if this is the first connection OR if rewarm was requested
# (rewarm is requested when bypasser shuts down due to idle while clients are connected)
if was_zero or needs_rewarm:
reason = "First client connected" if was_zero else "Rewarm requested after idle shutdown"
logger.info(f"{reason}, triggering warmup callbacks...")
for callback in self._on_first_connect_callbacks:
try:
# Run callbacks in a separate thread to not block the connection
thread = threading.Thread(target=callback, daemon=True)
thread.start()
except Exception as e:
logger.error(f"Error in on_first_connect callback {callback.__name__}: {e}")
def client_disconnected(self):
"""Track a client disconnection. Call this from the disconnect event handler."""
with self._connection_lock:
self._connection_count = max(0, self._connection_count - 1)
current_count = self._connection_count
is_now_zero = current_count == 0
logger.debug(f"Client disconnected. Active connections: {current_count}")
# If all clients have disconnected, trigger cleanup callbacks
if is_now_zero:
logger.info("All clients disconnected, triggering disconnect callbacks...")
for callback in self._on_all_disconnect_callbacks:
try:
callback()
except Exception as e:
logger.error(f"Error in on_all_disconnect callback {callback.__name__}: {e}")
def get_connection_count(self) -> int:
"""Get the current number of active WebSocket connections."""
with self._connection_lock:
return self._connection_count
def has_active_connections(self) -> bool:
"""Check if there are any active WebSocket connections."""
return self.get_connection_count() > 0
def is_enabled(self) -> bool:
"""Check if WebSocket is enabled and ready."""
return self._enabled and self.socketio is not None
def broadcast_status_update(self, status_data: Dict[str, Any]):
"""Broadcast status update to all connected clients."""
if not self.is_enabled():
return
try:
# When calling socketio.emit() outside event handlers, it broadcasts by default
self.socketio.emit('status_update', status_data)
logger.debug(f"Broadcasted status update to all clients")
except Exception as e:
logger.error(f"Error broadcasting status update: {e}")
def broadcast_download_progress(self, book_id: str, progress: float, status: str):
"""Broadcast download progress update for a specific book."""
if not self.is_enabled():
return
try:
data = {
'book_id': book_id,
'progress': progress,
'status': status
}
# When calling socketio.emit() outside event handlers, it broadcasts by default
self.socketio.emit('download_progress', data)
logger.debug(f"Broadcasted progress for book {book_id}: {progress}%")
except Exception as e:
logger.error(f"Error broadcasting download progress: {e}")
def broadcast_notification(self, message: str, notification_type: str = 'info'):
"""Broadcast a notification message to all clients."""
if not self.is_enabled():
return
try:
data = {
'message': message,
'type': notification_type
}
# When calling socketio.emit() outside event handlers, it broadcasts by default
self.socketio.emit('notification', data)
logger.debug(f"Broadcasted notification: {message}")
except Exception as e:
logger.error(f"Error broadcasting notification: {e}")
# Global WebSocket manager instance
ws_manager = WebSocketManager()