ThePhaseless
7b904a5ffd
feat: continue after networkidle timeout once domcontentloaded completes
...
A page whose network never goes idle (background analytics, websockets)
used to fail the whole request with a 408 once the networkidle wait
expired. Since the DOM is fully usable after domcontentloaded, treat a
networkidle timeout as non-fatal and return the loaded page instead.
Fatal timeouts during initial load or challenge solving still return 408.
Adds unit coverage for both paths using a fake page that fails
configured load-state waits.
2026-08-08 00:27:54 +02:00
ThePhaseless
0b4d1a91ce
feat: accept FlareSolverr maxTimeout in milliseconds
...
Add a maxTimeout alias to LinkRequest.max_timeout for FlareSolverr
drop-in compatibility. Values of 1000 or more are treated as
milliseconds and normalized to seconds; smaller values keep the
native seconds semantics. Closes #382 .
2026-08-07 23:51:44 +02:00
ThePhaseless
8c3e7a9fe4
refactor: use pydantic-settings for environment configuration
...
- Add pydantic-settings as direct dependency
- Replace os.getenv calls with typed Settings class
- Add BLOCK_MEDIA and RETURN_ONLY_COOKIES env defaults
2026-07-04 17:42:53 +02:00
ThePhaseless
80a608a629
feat: add blockMedia, returnOnlyCookies, PDF handling and fix timeout/networkidle
...
- Catch both builtins.TimeoutError and playwright TimeoutError as 408
- Check challenge title before networkidle to avoid timeout on Cloudflare interstitial
- Add blockMedia and returnOnlyCookies request options
- Return raw PDF bytes as base64 with contentType application/pdf
- Skip tests on 408 timeouts; add PDF handling test
2026-07-04 17:34:54 +02:00
ThePhaseless
4800ef7f5f
feat: migrate from camoufox to invisible_playwright
...
- Replace camoufox[geoip] with invisible_playwright git dependency
- Switch playwright-captcha framework from CAMOUFOX to PLAYWRIGHT
- Remove camoufox addon path from consts
- Add git to Docker base image; fetch invisible_playwright binary
- Make /cache writable for runtime USER 1000
2026-07-04 17:34:48 +02:00
ThePhaseless
1ba0bb3c5e
add proxy per request
2026-02-08 12:17:06 +00:00
ThePhaseless
4be9674f82
ignore linter warning
2026-02-08 11:50:33 +00:00
ThePhaseless
39b804da55
add host and port as env
2025-10-01 12:09:53 +00:00
ThePhaseless
849e33dff3
use timer to timeout functions
2025-09-11 20:26:11 +00:00
ThePhaseless
d78b1df041
change healthcheck msg
2025-09-11 16:55:05 +00:00
ThePhaseless
c66d4e7084
return 200 if intersite challenge
2025-09-11 16:54:50 +00:00
ThePhaseless
9183a74fc3
use maxsize instead of power
2025-09-11 16:54:19 +00:00
ThePhaseless
9ffc2bb5b3
fix exception
2025-09-10 20:47:59 +00:00
ThePhaseless
5f053ecd29
add logger for lib if debug
2025-09-10 22:46:03 +02:00
ThePhaseless
258f0bdbad
add faster turnstile check
2025-09-10 22:45:40 +02:00
ThePhaseless
f98e823991
Migrate to camoufox ( #235 )
...
* use camoufox
* build custom branches
* organize compose
* ignore missing stubs
* add init method
* create ADDON PATH
* check if solving is required
* type checking and remove logger
* uv sync
* add timeout
* add descriptiuon
* cancel previous runs
* wildcard minor
* fix proxy
* syntax fix
* [skip ci] fix loop warning
* fix proxy format
* copy over docker ignore
* remove testing line
* comment out port expose
* remove idope
* use strict typechecking
* refactor
* adjust compose
* run init only before test
* try test without init
* add curl
* handle page response
* remove turnsile test
* fix dockerignore symlink
* use newer debian
* bump pytest
* add docker in docker in devcontaienr
* remove unused values
* handle timeouts
* fix edgecase
* use new envs for proxy
* remove init command
* remove testing line
* remove setuptools
* fix linters
* reimport
* readd setuptools
* better float handling
2025-09-05 23:08:35 +02:00
ThePhaseless
4ab174243c
fix exception
2025-08-17 00:38:37 +00:00
ThePhaseless
4ab01abbf0
ignore regex syntax warning
2025-08-16 22:49:24 +00:00
ThePhaseless
858bba598f
bump deps
2025-08-16 22:41:26 +00:00
Alvaro Santos Andres
6e1c073a6f
fix: set default CAPTCHA_RETRIES to 3
2025-08-12 07:16:33 +02:00
Alvaro Santos Andres
b153d4ae34
fix: simplify code to fix PR suggestions
2025-08-11 06:57:42 +00:00
Alvaro Santos Andres
707a9c7a83
fix: use of CDP Mode
2025-07-25 16:31:28 +00:00
Alvaro Santos Andres
23d606a701
fix: enable reload to debug errors
2025-07-25 14:59:57 +00:00
Alvaro Santos Andres
6b0da3cc5e
fix: add more logging to debug errors
2025-07-25 14:38:22 +00:00
Alvaro Santos Andres
39f758859d
fix: disabled HEADLESS mode by default and updated dependencies
2025-07-25 13:51:19 +02:00
ThePhaseless
cdb3223a87
fix typo
2025-05-17 22:45:55 +00:00
ThePhaseless
5031d1e554
set default values
2025-05-17 22:45:21 +00:00
ThePhaseless
98f9480157
fix invalid requests response
2025-04-29 10:57:21 +02:00
ThePhaseless
fe797ba2ac
add userAgent and version to healthcheck
2025-04-17 11:27:17 +00:00
ThePhaseless
81959df037
Added SOCKS5 with auth handling
2025-04-12 21:46:42 +00:00
ThePhaseless
05f9f44c79
add protocol to example
2025-04-12 20:47:52 +00:00
ThePhaseless
08cfd95d6a
make all pyfiles as not executable
2025-04-12 20:47:37 +00:00
ThePhaseless
c75c79a2ad
rename and add missing fields
2025-04-08 13:13:01 +00:00
ThePhaseless
f3fe419eae
Merge pull request #133 from MaxG88/fix/flaresolverr-compatibility
2025-04-08 15:12:32 +02:00
ThePhaseless
e1a2e77036
tweak camel aliases
2025-04-08 12:34:25 +00:00
Maxime Gagnon
dd74f80c18
restore flaresolverr compatibility
2025-03-30 16:31:05 -04:00
ThePhaseless
db0924d9ce
use default instead of factory
2025-03-01 20:09:33 +01:00
ThePhaseless
b8d2b80746
Fix timestamps
2025-03-01 19:55:35 +01:00
ThePhaseless
43fe522417
[skip ci] allow sha versions
2025-02-28 16:13:28 +01:00
ThePhaseless
d7138776a5
better logging
2025-02-26 11:03:46 +01:00
ThePhaseless
7e2c895a11
faster title check
2025-02-26 11:03:46 +01:00
ThePhaseless
43cd9297cb
fix #86
2025-02-20 20:36:48 +00:00
ThePhaseless
19670c8e06
fix deprecation warning
2025-02-20 20:36:33 +00:00
ThePhaseless
ecbb2ab182
separate files, add gzip, save screenshots on exception, add PROXY support
2025-02-17 23:08:14 +00:00
ThePhaseless
83169ce019
add headless env
2025-01-02 12:18:21 +00:00
ThePhaseless
096eda0fa3
fix utils
2024-12-18 13:32:37 +00:00
ThePhaseless
2ebb1bd820
optional xvfb support
2024-12-18 13:29:00 +00:00
ThePhaseless
3dbe3c424c
fix circular imports
2024-12-13 22:43:29 +00:00
ThePhaseless
bbb7c73065
kill old processes
2024-12-13 22:38:47 +00:00
ThePhaseless
0775d107ad
add version control and more verbose error return
2024-11-30 12:42:51 +00:00