mirror of
https://github.com/ThePhaseless/Byparr.git
synced 2026-11-15 03:58:20 +00:00
ThePhaselessandClaude Opus 5.5
804c776a80
The fallback zone used after a failed egress lookup came from hand-rolled TZ and /etc/localtime parsing. That parsing passed values such as TZ=UTC0, TZ=CET-1CEST,M3.5.0,M10.5.0/3 or TZ=:/usr/share/zoneinfo/... to the browser unchanged, and Playwright rejects them as an invalid timezone ID. tzlocal.get_localzone_name() handles those forms and returns only names it could load, or None. A None result still falls back to UTC. tzlocal already came in through trafilatura (htmldate -> dateparser). It is now a direct dependency because Byparr imports it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
142 lines
8.8 KiB
Markdown
142 lines
8.8 KiB
Markdown
# Byparr [](https://github.com/sponsors/ThePhaseless)
|
||
|
||
<p align="center">
|
||
<img src="icon/logo-byparr.svg" alt="Byparr logo" width="120" />
|
||
</p>
|
||
|
||
> [!IMPORTANT]
|
||
> This software does not **guarantee** (only greatly increases the chance) that any challenge will be bypassed. While this tool passes the initial browser check, Cloudflare and other captcha providers likely require valid network traffic originating from the user’s public IP address to mark a connection as legitimate. If any website does not pass the challenge, please run troubleshooting steps and check if other websites work before you create an GitHub issue.
|
||
|
||
## Options
|
||
|
||
| Environment Variable | Default | Description |
|
||
| -------------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||
| `HOST` | `0.0.0.0` | Host address to bind the server to. Use `0.0.0.0` to bind to all IPv4 interfaces, `::` for all IPv6 interfaces, or `127.0.0.1`/`localhost` for local access only. |
|
||
| `PORT` | `8191` | Port to bind the server to. |
|
||
| `PROXY_SERVER` | None | Proxy to use in format: `protocol://host:port`. |
|
||
| `PROXY_USERNAME` | None | Username for proxy authentication. |
|
||
| `PROXY_PASSWORD` | None | Password for proxy authentication. |
|
||
| `OWUI_API_KEY` | None | Bearer token for `/load` endpoint authentication. Must match `EXTERNAL_WEB_LOADER_API_KEY` in Open WebUI. |
|
||
| `BROWSER_LOCALE` | None | Override the browser's language with a [BCP-47](https://www.rfc-editor.org/rfc/bcp/bcp47.txt) tag, e.g. `en-US`, `de-DE`, `fr-FR`. When unset, the locale is derived from the egress country. |
|
||
| `BROWSER_TIMEZONE` | None | Override the browser's timezone with an [IANA](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) name, e.g. `Europe/Berlin`, `Asia/Shanghai`. When unset, the timezone is derived from the egress IP. |
|
||
|
||
#### Browser language
|
||
|
||
Set `BROWSER_LOCALE` to a [BCP-47](https://www.rfc-editor.org/rfc/bcp/bcp47.txt) language tag like `en-US`, `de-DE`, `fr-FR`, `pl-PL`, or `zh-CN` to fix the browser's language and `Accept-Language` header. When unset, Byparr derives the locale from the egress country (e.g. a French proxy → `fr-FR`), keeping the browser language consistent with the exit IP.
|
||
|
||
Valid tags are maintained in the [IANA Language Subtag Registry](https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry). For a friendlier list, see [List of ISO 639-1 codes](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) (language) combined with an [ISO 3166-1 alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) region code for the full tag, e.g. `pt-BR`.
|
||
|
||
#### Browser timezone
|
||
|
||
Set `BROWSER_TIMEZONE` to an [IANA timezone name](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) like `Europe/Berlin` or `Asia/Shanghai` to fix the browser's timezone. An explicit value is used as is, also behind a proxy.
|
||
|
||
When either `BROWSER_TIMEZONE` or `BROWSER_LOCALE` is unset, Byparr looks up its public IP through `api.ipify.org`, `icanhazip.com` or `checkip.amazonaws.com` and maps it to a timezone and language with a GeoIP database downloaded from GitHub. Without a proxy this happens once, on the first request, and the result is reused until Byparr restarts (restart it if its public IP moves to another country). If that lookup fails, the host timezone (`TZ`, otherwise the system's, otherwise UTC) and `en-US` are used, and the lookup is retried after 10 minutes. With a proxy (`PROXY_SERVER` or the `X-Proxy-Server` header) it runs for every request, through the proxy.
|
||
|
||
If those hosts are slow or blocked on your network, set both `BROWSER_TIMEZONE` and `BROWSER_LOCALE`: the lookup is then skipped entirely and no request is made before the browser starts.
|
||
|
||
## Proxy Recommendation
|
||
|
||
Recently I've partnered with a _new in town_ proxy service - ProxyBase - to offer affordable proxy services that seems to work seamlessly with Byparr! Using my affiliate code `byparr` (case sensitive!) when signing up will not only get you access to their cost-effective (**$0.69/GB with occasional promotions** _at the time of writing_) proxy network but will also help support the continued development of this project. ProxyBase's proxies can significantly improve your success rate when bypassing anti-bot challenges. [Check out ProxyBase](https://client.proxybase.org/signup?ref=byparr) and enhance your Byparr experience!
|
||
|
||
## Tags
|
||
|
||
- `v*.*.*`/`latest` - Releases considered stable
|
||
- `main` - Latest release from main branch (untested)
|
||
- `pr-{number}` - Pull request images for testing (automatically cleaned up when PR closes)
|
||
|
||
## Usage
|
||
|
||
> [!IMPORTANT]
|
||
> Support for NAS devices (like Synology) is minimal. Please report issues, but do not expect it to be fixed quickly. The only ARM device I have is a free Ampere Oracle VM, so I can only test ARM support on that. See [#22](https://github.com/ThePhaseless/Byparr/issues/22) and [#3](https://github.com/ThePhaseless/Byparr/issues/3)
|
||
|
||
### Docker Compose setup
|
||
|
||
1. Review settings in `compose.yaml`.
|
||
2. Start the service:
|
||
|
||
```bash
|
||
docker compose up -d
|
||
```
|
||
|
||
> [!WARNING]
|
||
> Byparr performs a *lot* of R/W operations on your disk. To reduce disk wear, it is recommended to allocate temp storage in RAM by adding the following snippet to your Docker Compose configuration:
|
||
>
|
||
> ```yaml
|
||
> shm_size: "1gb"
|
||
> tmpfs:
|
||
> - /tmp:exec,mode=1777,size=1G
|
||
> - /cache:mode=1777,size=512M
|
||
>```
|
||
|
||
### Docker install
|
||
|
||
1. Pull and run the image:
|
||
|
||
```bash
|
||
docker run -p 8191:8191 ghcr.io/thephaseless/byparr:latest
|
||
```
|
||
|
||
2. Optional: set env vars using `-e` or `--env-file`.
|
||
|
||
### Local install
|
||
|
||
1. Install ([or update when Python version changes](https://github.com/astral-sh/uv/issues/17887)) [uv](https://docs.astral.sh/uv/getting-started/installation/).
|
||
2. Clone this repo - `git clone https://github.com/ThePhaseless/Byparr`
|
||
3. Run `uv run main.py`
|
||
4. Enjoy!
|
||
|
||
### API Docs
|
||
|
||
Once running, open:
|
||
|
||
- `http://localhost:8191/docs`
|
||
- `http://localhost:8191/` (redirects to `/docs`)
|
||
|
||
### Open WebUI Integration
|
||
|
||
Byparr can serve as an external web loader for [Open WebUI](https://github.com/open-webui/open-webui), allowing it to fetch web content through Byparr's anti-bot bypassing capabilities.
|
||
|
||
Configure Open WebUI with these environment variables:
|
||
|
||
```bash
|
||
WEB_LOADER_ENGINE=external
|
||
EXTERNAL_WEB_LOADER_URL=http://byparr:8191/load
|
||
EXTERNAL_WEB_LOADER_API_KEY=your-secret-key # Optional, must match OWUI_API_KEY
|
||
```
|
||
|
||
The `/load` endpoint accepts `POST` requests with `{"urls": ["https://..."]}` and returns extracted text content for RAG pipelines.
|
||
|
||
## Troubleshooting
|
||
|
||
### Docker troubleshooting
|
||
|
||
1. Clone repo to the host that has issues with Byparr.
|
||
2. Run `docker build --target test .`
|
||
3. Depending of the build success:
|
||
1. If run successfully, try updating container or if already on newest stable release create an issue for creating new release with new dependencies
|
||
2. If build fails, try troubleshooting on another host/using other method
|
||
|
||
#### Proxmox OCI / LXC browser launch errors
|
||
|
||
If you are running Byparr as an OCI container in Proxmox (or another LXC-based setup) and see a `FileNotFoundError` from `multiprocessing.synchronize`/`camoufox` when processing requests, increase the service's shared memory in `compose.yaml`:
|
||
|
||
```yaml
|
||
services:
|
||
byparr:
|
||
shm_size: 512mb
|
||
stdin_open: true
|
||
tty: true
|
||
```
|
||
|
||
`shm_size: 512mb` is usually enough; `stdin_open` and `tty` are only needed if your orchestrator runs the container without a TTY.
|
||
|
||
### Local troubleshooting
|
||
|
||
1. Download [uv](https://docs.astral.sh/uv/getting-started/installation/)
|
||
2. Download dependencies using `uv sync --group test`
|
||
3. Run tests with `uv run pytest --retries 3` (You can add `-n auto` for parallelization)
|
||
4. If you see any `F` character in terminal, that means test failed even after retries.
|
||
5. Depending of the test success:
|
||
1. If run successfully, try updating container or if already on newest stable release create an issue for creating new release with new dependencies
|
||
2. If test fails, try troubleshooting on another host/using other method
|