I tried running the docker-compose.yml referenced in this comment
https://github.com/calibrain/calibre-web-automated-book-downloader/issues/120#issuecomment-2816539268
from issue #120 and was able to reproduce the same 'Server is not
responding' error. I found the issue would happen whenever the captcha
bypass took more than 30 seconds due to tor's slower connection. Adding
-t 300 would let gunicorn wait up to 5 minutes now.
Fix permission issues that occur if user forgets to create the
`cwa-book-downloader` folder before creating the docker container since
docker makes the directory with root ownership.
`sudo -E` was replacing `appuser`'s `$HOME` environment variable of
`/app` with `/root` causing permission issues when selenium tried to
write to `~/.cache/selenium`. This fixes [issue
#130](https://github.com/calibrain/calibre-web-automated-book-downloader/issues/130)
by allowing Driver from seleniumbase to successfully return instead of
timing out on line 172 in `cloudflare_bypasser.py`.
Refactor: Improve Docker build, add Tor support, use SeleniumBase
- Overhauled Dockerfile:
- Switched to python:3.10-slim base.
- Implemented multi-stage builds (base, standard, tor).
- Consolidated RUN layers for efficiency.
- Added locale/timezone setup.
- Added Tor setup and iptables configuration in dedicated stage/script.
- Replaced DrissionPage with SeleniumBase for Cloudflare bypassing.
- Added Tor support via `docker-compose.tor.yml` and `tor.sh` script.
- Updated `docker-compose.yml` to build locally and changed default port
to 8083.
- Added `docker-compose.dev.yml` and `docker-compose.tor.dev.yml`.
- Updated `entrypoint.sh` for timezone and sudo usage.
- Added/Updated environment variables (`TZ`, `USING_TOR`, etc.).
- Improved `.dockerignore`.
- Updated `readme.md` to reflect port changes, build process, document
`TZ`, and add details about the new Tor variant.
The Calibre dependency was due to the script testing for validity of the
downloaded file, as often they would be corrupted from aa. But CWA is
already doing that, so we are just having redundant code here.
For the cloudflarebypasser, I basically run my own version now, instead
of depending on an external library, this way we have better control for
debugging and on the docker image.
Fixes#18, #33, #27, #48, #65, #78, #86, #88, #89
---------
Co-authored-by: mik593 <91991279+mik593@users.noreply.github.com>
This PR addresses an issue where the "download selected" functionality
behaves unexpectedly. If a user makes a selection of books to download,
performs a new search without clearing the previous selection, and
selects new books, clicking "download selected" would still include the
books from the prior selection. This causes the "download selected"
button to fail, as the books from the previous selection are no longer
available.
A potential enhancement could allow users to select books across
multiple searches and download them collectively. This could be achieved
by creating a dedicated section in the UI to track and display the
currently selected books. This feature might go well with a broader UI
overhaul, as has been suggested in some issues - maybe after your CF
bypass rewrite?
Co-authored-by: CaliBrain <calibrain@l4n.xyz>
Related to #53
This PR adds sorting to the results table. This can be done by clicking
on the header of the table.
For narrow columns, the arrow indicating the ordering is underneath the
column title instead of next to the title - not sure how to solve that
elegantly without a minor redesign of the table :D
CaliBrain
Jacob Wolfe
DanielandDaniel Egberts
Cathelijne HornstraandCathelijne Hornstra
FunnyPocketBookandCaliBrain
Colin Yates
cthorne