v1.20.19

mkdir: translit illegals instead of failing
ftp: logging + fix xbu log
2026-11-14 09:58:22 +00:00 · 2026-07-27 22:02:50 +00:00 · 2026-07-27 21:25:21 +00:00 · 2026-07-27 19:06:25 +00:00 · 2026-07-27 18:50:44 +00:00 · 2026-07-27 18:45:35 +00:00
229 changed files with 29055 additions and 15954 deletions
@@ -4,3 +4,4 @@

 *.png binary
 *.gif binary
+*.gz binary
@@ -7,10 +7,11 @@ assignees: '9001'

 ---

-NOTE:
+<!-- NOTE:
 **please use english, or include an english translation.** aside from that,  
-all of the below are optional, consider them as inspiration, delete and rewrite at will, thx md
-
+include all of the information that might possibly be necessary to help diagnose your issue,
+especially things like the server config and server logs; that said,
+any parts below which are obviously and definitely irrelevant can be skipped -->

 ### Describe the bug
 a description of what the bug is
@@ -45,5 +46,11 @@ if the issue is possibly on the client-side, then mention some of the following:
 * OS version: 
 * browser version: 

+### The rest of the stack
+if you are connecting directly to copyparty then that's cool, otherwise please mention everything else between copyparty and the browser (reverseproxy, tunnels, etc.)
+
+### Server log
+if the issue might be server-related, include everything that appears in the copyparty log during startup, and also anything else you think might be relevant
+
 ### Additional context
 any other context about the problem here
@@ -7,9 +7,9 @@ assignees: '9001'

 ---

-NOTE:
+<!-- NOTE:
 **please use english, or include an english translation.** aside from that,  
-all of the below are optional, consider them as inspiration, delete and rewrite at will
+all of the below are optional, consider them as inspiration, delete and rewrite at will -->

 **is your feature request related to a problem? Please describe.**
 a description of what the problem is, for example, `I'm always frustrated when [...]` or `Why is it not possible to [...]`
@@ -29,8 +29,13 @@ copyparty.egg-info/
 copyparty/res/COPYING.txt
 copyparty/web/deps/
 srv/
+scripts/docker/base/b/
+scripts/docker/base/cver*
+scripts/docker/base/test-aac/
+scripts/docker/base/whl/
 scripts/docker/i/
 scripts/deps-docker/uncomment.py
+scripts/deps-docker/unhint.py
 contrib/package/arch/pkg/
 contrib/package/arch/src/

@@ -43,6 +48,7 @@ scripts/docker/*.err

 # nix build output link
 result
+result-*

 # IDEA config
 .idea/
@@ -39,7 +39,7 @@ if you wanna have a go at coding it up yourself then maybe mention the idea on d

 aside from documentation and ideas, some other things that would be cool to have some help with is:

-* **translations** -- the copyparty web-UI has translations for english and norwegian at the top of [browser.js](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/web/browser.js); if you'd like to add a translation for another language then that'd be welcome! and if that language has a grammar that doesn't fit into the way the strings are assembled, then we'll fix that as we go :>
+* **translations** -- the copyparty web-UI has translations in [copyparty/web/tl](https://github.com/9001/copyparty/tree/hovudstraum/copyparty/web/tl); if you'd like to [add a translation](https://github.com/9001/copyparty/tree/hovudstraum/docs/rice#translations) for another language then that'd be welcome! and if that language has a grammar that doesn't fit into the way the strings are assembled, then we'll fix that as we go :>

  * but please note that support for [RTL (Right-to-Left) languages](https://en.wikipedia.org/wiki/Right-to-left_script) is currently not planned, since the javascript is a bit too jank for that

@@ -5,7 +5,7 @@
 turn almost any device into a file server with resumable uploads/downloads using [*any*](#browser-support) web browser

 * server only needs Python (2 or 3), all dependencies optional
-* 🔌 protocols: [http](#the-browser) // [webdav](#webdav-server) // [ftp](#ftp-server) // [tftp](#tftp-server) // [smb/cifs](#smb-server)
+* 🔌 protocols: [http(s)](#the-browser) // [webdav](#webdav-server) // [sftp](#sftp-server) // [ftp(s)](#ftp-server) // [tftp](#tftp-server) // [smb/cifs](#smb-server)
 * 📱 [android app](#android-app) // [iPhone shortcuts](#ios-shortcuts)

 👉 **[Get started](#quickstart)!** or visit the **[read-only demo server](https://a.ocv.me/pub/demo/)** 👀 running on a nuc in my basement
@@ -14,13 +14,14 @@ turn almost any device into a file server with resumable uploads/downloads using

 🎬 **videos:** [upload](https://a.ocv.me/pub/demo/pics-vids/up2k.webm) // [cli-upload](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm) // [race-the-beam](https://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm) // 👉 **[feature-showcase](https://a.ocv.me/pub/demo/showcase-hq.webm)** ([youtube](https://www.youtube.com/watch?v=15_-hgsX2V0))

-made in Norway 🇳🇴
+built in Norway 🇳🇴 with contributions from [not-norway](https://github.com/9001/copyparty/graphs/contributors)


 ## readme toc

 * top
    * [quickstart](#quickstart) - just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** -- that's it! 🎉
+        * [mirrors](#mirrors) - other places to download copyparty from
        * [at home](#at-home) - make it accessible over the internet
        * [on servers](#on-servers) - you may also want these, especially on servers
    * [features](#features) - also see [comparison to similar software](./docs/versus.md)
@@ -63,15 +64,19 @@ made in Norway 🇳🇴
    * [other tricks](#other-tricks)
    * [searching](#searching) - search by size, date, path/name, mp3-tags, ...
 * [server config](#server-config) - using arguments or config files, or a mix of both
+    * [version-checker](#version-checker) - sleep better at night
+    * [logging](#logging) - serverlog is sent to stdout by default
    * [zeroconf](#zeroconf) - announce enabled services on the LAN ([pic](https://user-images.githubusercontent.com/241032/215344737-0eae8d98-9496-4256-9aa8-cd2f6971810d.png))
        * [mdns](#mdns) - LAN domain-name and feature announcer
        * [ssdp](#ssdp) - windows-explorer announcer
    * [qr-code](#qr-code) - print a qr-code [(screenshot)](https://user-images.githubusercontent.com/241032/194728533-6f00849b-c6ac-43c6-9359-83e454d11e00.png) for quick access
    * [ftp server](#ftp-server) - an FTP server can be started using `--ftp 3921`
+    * [sftp server](#sftp-server) - goes roughly 700 MiB/s (slower than webdav and ftp)
    * [webdav server](#webdav-server) - with read-write support
        * [connecting to webdav from windows](#connecting-to-webdav-from-windows) - using the GUI
    * [tftp server](#tftp-server) - a TFTP server (read/write) can be started using `--tftp 3969`
    * [smb server](#smb-server) - unsafe, slow, not recommended for wan
+    * [wopi server](#wopi-server) - edit office documents in the web-ui
    * [browser ux](#browser-ux) - tweaking the ui
    * [opengraph](#opengraph) - discord and social-media embeds
    * [file deduplication](#file-deduplication) - enable symlink-based upload deduplication
@@ -83,8 +88,12 @@ made in Norway 🇳🇴
    * [compress uploads](#compress-uploads) - files can be autocompressed on upload
    * [chmod and chown](#chmod-and-chown) - per-volume filesystem-permissions and ownership
    * [other flags](#other-flags)
+    * [descript.ion](#description) - add a description to each file in a folder
+    * [dothidden](#dothidden) - cosmetically hide specific files in a folder
+    * [thumbnail pregen](#thumbnail-pregen) - if you want to pre-generate everything on startup
    * [database location](#database-location) - in-volume (`.hist/up2k.db`, default) or somewhere else
    * [metadata from audio files](#metadata-from-audio-files) - set `-e2t` to index tags on upload
+        * [metadata from xattrs](#metadata-from-xattrs) - unix extended file attributes
    * [file parser plugins](#file-parser-plugins) - provide custom parsers to index additional tags
    * [event hooks](#event-hooks) - trigger a program on uploads, renames etc ([examples](./bin/hooks/))
        * [zeromq](#zeromq) - event-hooks can send zeromq messages
@@ -113,10 +122,12 @@ made in Norway 🇳🇴
 * [packages](#packages) - the party might be closer than you think
    * [arch package](#arch-package) - `pacman -S copyparty` (in [arch linux extra](https://archlinux.org/packages/extra/any/copyparty/))
    * [fedora package](#fedora-package) - does not exist yet
+    * [gentoo ::guru package](#gentoo-guru-package) - `emerge www-servers/copyparty::guru` (in [::guru](https://wiki.gentoo.org/wiki/Project:GURU))
    * [homebrew formulae](#homebrew-formulae) - `brew install copyparty ffmpeg`
    * [nix package](#nix-package) - `nix profile install github:9001/copyparty`
    * [nixos module](#nixos-module)
 * [browser support](#browser-support) - TLDR: yes
+* [server hall of fame](#server-hall-of-fame) - unexpected things that run copyparty
 * [client examples](#client-examples) - interact with copyparty using non-browser clients
    * [folder sync](#folder-sync) - sync folders to/from copyparty
    * [mount as drive](#mount-as-drive) - a remote copyparty server as a local filesystem
@@ -124,7 +135,7 @@ made in Norway 🇳🇴
 * [iOS shortcuts](#iOS-shortcuts) - there is no iPhone app, but
 * [performance](#performance) - defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
    * [client-side](#client-side) - when uploading files
-* [security](#security) - there is a [discord server](https://discord.gg/25J8CdTT6G)
+* [security](#security) - there is a [discord server](https://discord.gg/25J8CdTT6G) with announcements
    * [gotchas](#gotchas) - behavior that might be unexpected
    * [cors](#cors) - cross-site request config
    * [filekeys](#filekeys) - prevent filename bruteforcing
@@ -136,7 +147,7 @@ made in Norway 🇳🇴
        * [firefox wsod](#firefox-wsod) - firefox 87 can crash during uploads
 * [HTTP API](#HTTP-API) - see [devnotes](./docs/devnotes.md#http-api)
 * [dependencies](#dependencies) - mandatory deps
-    * [optional dependencies](#optional-dependencies) - install these to enable bonus features
+    * [optional dependencies](#optional-dependencies) - enable bonus features
        * [dependency chickenbits](#dependency-chickenbits) - prevent loading an optional dependency
        * [dependency unvendoring](#dependency-unvendoring) - force use of system modules
    * [optional gpl stuff](#optional-gpl-stuff)
@@ -157,7 +168,7 @@ just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/

 * or install through [pypi](https://pypi.org/project/copyparty/): `python3 -m pip install --user -U copyparty`
 * or if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
-* or install [on arch](#arch-package) ╱ [on NixOS](#nixos-module) ╱ [through nix](#nix-package)
+* or install [on arch](#arch-package) / [homebrew](#homebrew-formulae) ╱ [on NixOS](#nixos-module) ╱ [through nix](#nix-package)
 * or if you are on android, [install copyparty in termux](#install-on-android)
 * or maybe an iPhone or iPad? [install in a-Shell on iOS](#install-on-iOS)
 * or maybe you have a [synology nas / dsm](./docs/synology-dsm.md)
@@ -175,7 +186,7 @@ enable thumbnails (images/audio/video), media indexing, and audio transcoding by
 * **Alpine:** `apk add py3-pillow ffmpeg`
 * **Debian:** `apt install --no-install-recommends python3-pil ffmpeg`
 * **Fedora:** rpmfusion + `dnf install python3-pillow ffmpeg --allowerasing`
-* **FreeBSD:** `pkg install py39-sqlite3 py39-pillow ffmpeg`
+* **FreeBSD:** `pkg install py311-sqlite3 py311-pillow ffmpeg`
 * **MacOS:** `port install py-Pillow ffmpeg`
 * **MacOS** (alternative): `brew install pillow ffmpeg`
 * **Windows:** `python -m pip install --user -U Pillow`
@@ -192,7 +203,19 @@ some recommended options:
 * `-e2ts` enables audio metadata indexing (needs either FFprobe or Mutagen)
 * `-v /mnt/music:/music:r:rw,foo -a foo:bar` shares `/mnt/music` as `/music`, `r`eadable by anyone, and read-write for user `foo`, password `bar`
  * replace `:r:rw,foo` with `:r,foo` to only make the folder readable by `foo` and nobody else
-  * see [accounts and volumes](#accounts-and-volumes) (or `--help-accounts`) for the syntax and other permissions
+  * see [accounts and volumes](#accounts-and-volumes) (or [`--help-accounts`](https://copyparty.eu/cli/#accounts-help-page)) for the syntax and other permissions
+
+
+### mirrors
+
+other places to download copyparty from  (non-github links):
+
+* https://copyparty.eu/ (hetzner, finland, official mirror):
+  * https://copyparty.eu/py = https://copyparty.eu/copyparty-sfx.py = the sfx
+  * https://copyparty.eu/en = https://copyparty.eu/copyparty-en.py = the english-only sfx
+  * https://copyparty.eu/pyz = https://copyparty.eu/copyparty.pyz = the zipapp
+  * https://copyparty.eu/enz = https://copyparty.eu/copyparty-en.pyz = the enterprise pyz
+  * https://copyparty.eu/cli = online cli helptext


 ### at home
@@ -220,14 +243,19 @@ you may also want these, especially on servers:
 * [nixos module](#nixos-module) to run copyparty on NixOS hosts
 * [contrib/nginx/copyparty.conf](contrib/nginx/copyparty.conf) to [reverse-proxy](#reverse-proxy) behind nginx (for better https)

+because the following environment variables are commonly used in service-scripts, they are understood by copyparty:
+
+* `NOTIFY_SOCKET` as provided by systemd with service type=notify (see systemd/copyparty.service above)
+* `S6_NOTIFY_FD` for dinit [`ready-notification = pipevar:S6_NOTIFY_FD`](https://davmac.org/projects/dinit/man-pages-html/dinit-service.5.html#ready) and [s6 supervision suite notification](https://skarnet.org/software/s6/notifywhenup.html) support
+
 and remember to open the ports you want; here's a complete example including every feature copyparty has to offer:
 ```
-firewall-cmd --permanent --add-port={80,443,3921,3923,3945,3990}/tcp  # --zone=libvirt
+firewall-cmd --permanent --add-port={80,443,3921,3922,3923,3945,3990}/tcp  # --zone=libvirt
 firewall-cmd --permanent --add-port=12000-12099/tcp  # --zone=libvirt
 firewall-cmd --permanent --add-port={69,1900,3969,5353}/udp  # --zone=libvirt
 firewall-cmd --reload
 ```
-(69:tftp, 1900:ssdp, 3921:ftp, 3923:http/https, 3945:smb, 3969:tftp, 3990:ftps, 5353:mdns, 12000:passive-ftp)
+(69:tftp, 1900:ssdp, 3921:ftp, 3922:sftp, 3923:http/https, 3945:smb, 3969:tftp, 3990:ftps, 5353:mdns, 12000:passive-ftp)


 ## features
@@ -247,7 +275,7 @@ also see [comparison to similar software](./docs/versus.md)
  * ☑ [upnp / zeroconf / mdns / ssdp](#zeroconf)
  * ☑ [event hooks](#event-hooks) / script runner
  * ☑ [reverse-proxy support](https://github.com/9001/copyparty#reverse-proxy)
-  * ☑ cross-platform (Windows, Linux, Macos, Android, iOS, FreeBSD, arm32/arm64, ppc64le, s390x, risc-v/riscv64)
+  * ☑ cross-platform (Windows, Linux, Macos, Android, iOS, FreeBSD, arm32/arm64, ppc64le, s390x, risc-v/riscv64, SGI IRIX)
 * upload
  * ☑ basic: plain multipart, ie6 support
  * ☑ [up2k](#uploading): js, resumable, multithreaded
@@ -270,11 +298,12 @@ also see [comparison to similar software](./docs/versus.md)
    * ☑ play video files as audio (converted on server)
    * ☑ create and play [m3u8 playlists](#playlists)
  * ☑ image gallery with webm player
+    * ☑ and cbz manga/comics reader
  * ☑ [textfile browser](#textfile-viewer) with syntax highlighting
    * ☑ realtime streaming of growing files (logfiles and such)
  * ☑ [thumbnails](#thumbnails)
    * ☑ ...of images using Pillow, pyvips, or FFmpeg
-    * ☑ ...of RAW images using rawpy
+    * ☑ ...of RAW images using libraw-dcraw_emu or rawpy
    * ☑ ...of videos using FFmpeg
    * ☑ ...of audio (spectrograms) using FFmpeg
    * ☑ cache eviction (max-age; maybe max-size eventually)
@@ -309,7 +338,7 @@ small collection of user feedback

 project goals / philosophy

-* inverse linux philosophy -- do all the things, and do an *okay* job
+* inverse unix philosophy -- do all the things, and do an *okay* job
  * quick drop-in service to get a lot of features in a pinch
  * some of [the alternatives](./docs/versus.md) might be a better fit for you
 * run anywhere, support everything
@@ -383,6 +412,8 @@ same order here too

 * [Firefox issue 1790500](https://bugzilla.mozilla.org/show_bug.cgi?id=1790500) -- entire browser can crash after uploading ~4000 small files

+* Windows: Uploading from a webbrowser may fail with "directory iterator got stuck" due to the [max path length](https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry); try moving the files somewhere shorter before uploading
+
 * Android: music playback randomly stops due to [battery usage settings](#fix-unreliable-playback-on-android)

 * iPhones: the volume control doesn't work because [apple doesn't want it to](https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Device-SpecificConsiderations/Device-SpecificConsiderations.html#//apple_ref/doc/uid/TP40009523-CH5-SW11)
@@ -437,6 +468,12 @@ upgrade notes
 * CopyParty?
  * nope! the name is either copyparty (all-lowercase) or Copyparty -- it's [one word](https://en.wiktionary.org/wiki/copyparty) after all :>

+* what is a volflag?
+  * per-volume configuration; many (not all) global-options can be set as volflags, and most (not all) volflags can be set as global-options; [complete list of volflags](https://copyparty.eu/cli/#flags-help-page)
+
+* what is a volume?
+  * a mapping from a URL (`/music/`) to a folder on your server's local filesystem (`C:\Users\ed\Music`) which can then be accessed through copyparty, depending on the permissions and options you set on it -- see [accounts and volumes](#accounts-and-volumes)
+
 * can I change the 🌲 spinning pine-tree loading animation?
  * [yeah...](https://github.com/9001/copyparty/tree/hovudstraum/docs/rice#boring-loader-spinner) :-(

@@ -447,6 +484,7 @@ upgrade notes
 * can I link someone to a password-protected volume/file by including the password in the URL?
  * yes, by adding `?pw=hunter2` to the end; replace `?` with `&` if there are parameters in the URL already, meaning it contains a `?` near the end
    * if you have enabled `--usernames` then do `?pw=username:password` instead
+    * `?pw` can be disabled with `--pw-urlp=A` but this breaks support for many clients

 * how do I stop `.hist` folders from appearing everywhere on my HDD?
  * by default, a `.hist` folder is created inside each volume for the filesystem index, thumbnails, audio transcodes, and markdown document history. Use the `--hist` global-option or the `hist` volflag to move it somewhere else; see [database location](#database-location)
@@ -466,6 +504,9 @@ upgrade notes
 * thumbnails are broken (you get a colorful square which says the filetype instead)
  * you need to install `FFmpeg` or `Pillow`; see [thumbnails](#thumbnails)

+* thumbnails are broken, specifically for photos and videos taken by iphones
+  * the [docker image](https://github.com/9001/copyparty/blob/hovudstraum/scripts/docker) and [bootable flashdrive](https://a.ocv.me/pub/stuff/edcd001/enterprise-edition/) are not able to read heif/heic images and h265/HEVC video due to [legal reasons](docs/bad-codecs.md)
+
 * thumbnails are broken (some images appear, but other files just get a blank box, and/or the broken-image placeholder)
  * probably due to a reverse-proxy messing with the request URLs and stripping the query parameters (`?th=w`), so check your URL rewrite rules
  * could also be due to incorrect caching settings in reverse-proxies and/or CDNs, so make sure that nothing is set to ignore the query string
@@ -484,7 +525,7 @@ per-folder, per-user permissions  - if your setup is getting complex, consider m
 * much easier to manage, and you can modify the config at runtime with `systemctl reload copyparty` or more conveniently using the `[reload cfg]` button in the control-panel (if the user has `a`/admin in any volume)
  * changes to the `[global]` config section requires a restart to take effect

-a quick summary can be seen using `--help-accounts`
+a quick summary can be seen using [`--help-accounts`](https://copyparty.eu/cli/#accounts-help-page)

 configuring accounts/volumes with arguments:
 * `-a usr:pwd` adds account `usr` with password `pwd`
@@ -523,6 +564,9 @@ examples:

 if you want to grant access to all users who are logged in, the group `acct` will always contain all known users, so for example `-v /mnt/music:music:r,@acct`

+* to do the opposite, granting access to everyone who is NOT logged in. `*,-@acct` does the trick, for example `-v /srv/welcome:welcome:r,*,-@acct`
+* single users can also be subtracted from a group: `@admins,-james`
+
 anyone trying to bruteforce a password gets banned according to `--ban-pw`; default is 24h ban for 9 failed attempts in 1 hour

 and if you want to use config files instead of commandline args (good!) then here's the same examples as a configfile; save it as `foobar.conf` and use it like this: `python copyparty-sfx.py -c foobar.conf`
@@ -571,12 +615,28 @@ and if you want to use config files instead of commandline args (good!) then her

 hiding specific subfolders  by mounting another volume on top of them

-for example `-v /mnt::r -v /var/empty:web/certs:r` mounts the server folder `/mnt` as the webroot, but another volume is mounted at `/web/certs` -- so visitors can only see the contents of `/mnt` and `/mnt/web` (at URLs `/` and `/web`), but not `/mnt/web/certs` because URL `/web/certs` is mapped to `/var/empty`
+for example `-v /mnt::r -v /var/empty:web/certs:` (note: no permissions) mounts the server folder `/mnt` as the webroot, but another volume is mounted at `/web/certs` -- so visitors can only see the contents of `/mnt` and `/mnt/web` (at URLs `/` and `/web`), but not `/mnt/web/certs` because URL `/web/certs` is mapped to `/var/empty`
+
+to fully unmap it from the filesystem, specify `//NULL` instead of a real path such as `/var/empty`, so for example `-v /mnt::r -v //NULL:web/certs:` ensures `/web/certs` will never be accessible by anyone

 the example config file right above this section may explain this better; the first volume `/` is mapped to `/srv` which means http://127.0.0.1:3923/music would try to read `/srv/music` on the server filesystem, but since there's another volume at `/music` mapped to `/mnt/music` then it'll go to `/mnt/music` instead

+so, to shadow a file/folder, define a volume but leave out the `accs:` section
+
 > ℹ️ this also works for single files, because files can also be volumes

+config file example for unmapping folders by shadowing:
+
+```yaml
+[/drives]
+  /mnt   # url "/drives" goes to "/mnt"
+  accs:
+    r: *  # everyone can read
+
+[/drives/foo/bar]
+  //NULL  # blocks access to "/mnt/foo/bar"
+```
+

 ## dotfiles

@@ -586,6 +646,8 @@ anyone can access these if they know the name, but they normally don't appear in

 a client can request to see dotfiles in directory listings if global option `-ed` is specified, or the volume has volflag `dots`, or the user has permission `.`

+> for [shares](#shares), the `dots` volflag is ignored
+
 dotfiles do not appear in search results unless one of the above is true, **and** the global option / volflag `dotsrch` is set

 > even if user has permission to see dotfiles, they are default-hidden unless `--see-dots` is set, and/or user has enabled the `dotfiles` option in the settings tab
@@ -618,7 +680,7 @@ the main tabs in the ui
 * `[🧯]` [unpost](#unpost): undo/delete accidental uploads
 * `[🚀]` and `[🎈]` are the [uploaders](#uploading)
 * `[📂]` mkdir: create directories
-* `[📝]` new-md: create a new markdown document
+* `[📝]` new-file: create a new textfile
 * `[📟]` send-msg: either to server-log or into textfiles if `--urlform save`
 * `[🎺]` audio-player config options
 * `[⚙️]` general client config options
@@ -712,7 +774,7 @@ press `g` or `田` to toggle grid-view instead of the file listing  and `t` togg
 ![copyparty-thumbs-fs8](https://user-images.githubusercontent.com/241032/129636211-abd20fa2-a953-4366-9423-1c88ebb96ba9.png)

 it does static images with Pillow / pyvips / FFmpeg, and uses FFmpeg for video files, so you may want to `--no-thumb` or maybe just `--no-vthumb` depending on how dangerous your users are
-* pyvips is 3x faster than Pillow, Pillow is 3x faster than FFmpeg
+* Pillow is 3x faster (and safer) than FFmpeg
 * disable thumbnails for specific volumes with volflag `dthumb` for all, or `dvthumb` / `dathumb` / `dithumb` for video/audio/images only
 * for installing FFmpeg on windows, see [optional dependencies](#optional-dependencies)

@@ -721,6 +783,7 @@ audio files are converted into spectrograms using FFmpeg unless you `--no-athumb
 images with the following names (see `--th-covers`) become the thumbnail of the folder they're in: `folder.png`, `folder.jpg`, `cover.png`, `cover.jpg`
 * the order is significant, so if both `cover.png` and `folder.jpg` exist in a folder, it will pick the first matching `--th-covers` entry (`folder.jpg`)
 * and, if you enable [file indexing](#file-indexing), it will also try those names as dotfiles (`.folder.jpg` and so), and then fallback on the first picture in the folder (if it has any pictures at all)
+* disable folderthumbs with `--th-covers no`

 enabling `multiselect` lets you click files to select them, and then shift-click another file for range-select
 * `multiselect` is mostly intended for phones/tablets, but the `sel` option in the `[⚙️] settings` tab is better suited for desktop use, allowing selection by CTRL-clicking and range-selection with SHIFT-click, all without affecting regular clicking
@@ -732,8 +795,8 @@ to show `/icons/exe.png` and `/icons/elf.gif` as the thumbnail for all `.exe` an
  * be careful with svg; chrome will crash if you have too many unique svg files showing on the same page (the limit is 250 or so) -- showing the same handful of svg files thousands of times is ok however

 note:
-* heif/heifs/heic/heics images usually require the `libvips` [optional dependency](#optional-dependencies) (available in the `iv` docker image, `withFastThumbnails` in nixos)
-  * technical trivia: FFmpeg has basic support for tiled heic as of v7.0; need `-show_stream_groups` for correct resolution
+* heif/heifs/heic/heics images usually require the `libvips` [optional dependency](#optional-dependencies) but this is not possible with the docker-images due to [legal reasons](docs/bad-codecs.md)
+* if you do not want thumbnails to be generated on-the-fly, and instead wish to generate all of them on server startup, then see [thumbnail pregen](#thumbnail-pregen)

 config file example:

@@ -789,8 +852,11 @@ you can also zip a selection of files or folders by clicking them in the browser

 cool trick: download a folder by appending url-params `?tar&opus` or `?tar&mp3` to transcode all audio files (except aac|m4a|mp3|ogg|opus|wma) to opus/mp3 before they're added to the archive
 * super useful if you're 5 minutes away from takeoff and realize you don't have any music on your phone but your server only has flac files and downloading those will burn through all your data + there wouldn't be enough time anyways
+* and url-param `&name=foo` changes the name of the toplevel folder in the archive to `foo`, and just `&name` removes the folder entirely
+* and url-param `&nodot` skips dotfiles/dotfolders; they are included by default if your account has permission to see them
 * and url-params `&j` / `&w` produce jpeg/webm thumbnails/spectrograms instead of the original audio/video/images (`&p` for audio waveforms)
  * can also be used to pregenerate thumbnails; combine with `--th-maxage=9999999` or `--th-clean=0`
+    * but now there is also a real [thumbnail pregen](#thumbnail-pregen) so just use that


 ## uploading
@@ -972,6 +1038,8 @@ specify `--shr /foobar` to enable this feature; a toplevel virtual folder named

 users can delete their own shares in the controlpanel, and a list of privileged users (`--shr-adm`) are allowed to see and/or delet any share on the server

+the volflag `--shr-who` lets you control who can create a share from that volume, either `no` (nobody), `a` (people with admin permission), or `auth` (people who are logged in) 
+
 after a share has expired, it remains visible in the controlpanel for `--shr-rt` minutes (default is 1 day), and the owner can revive it by extending the expiration time there

 **security note:** using this feature does not mean that you can skip the [accounts and volumes](#accounts-and-volumes) section -- you still need to restrict access to volumes that you do not intend to share with unauthenticated users! it is not sufficient to use rules in the reverseproxy to restrict access to just the `/share` folder.
@@ -1135,7 +1203,7 @@ open the `[🎺]` media-player-settings tab to configure it,
  * `[flac]` converts `flac` and `wav` files into opus (if supported by browser) or mp3
  * `[aac]` converts `aac` and `m4a` files into opus (if supported by browser) or mp3
  * `[oth]` converts all other known formats into opus (if supported by browser) or mp3
-    * `aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk`
+    * `aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|m4b|m4r|mka|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk`
 * "transcode to":
  * `[opus]` produces an `opus` whenever transcoding is necessary (the best choice on Android and PCs)
  * `[awo]` is `opus` in a `weba` file, good for iPhones (iOS 17.5 and newer) but Apple is still fixing some state-confusion bugs as of iOS 18.2.1
@@ -1246,7 +1314,7 @@ see [./srv/expand/](./srv/expand/) for usage and examples

  * and `PREADME.md` / `preadme.md` is shown above directory listings unless `--no-readme` or `.prologue.html`

-* `README.md` and `*logue.html` can contain placeholder values which are replaced server-side before embedding into directory listings; see `--help-exp`
+* `README.md` and `*logue.html` can contain placeholder values which are replaced server-side before embedding into directory listings; see [`--help-exp`](https://copyparty.eu/cli/#exp-help-page)


 ## searching
@@ -1271,15 +1339,75 @@ for the above example to work, add the commandline argument `-e2ts` to also scan
 # server config

 using arguments or config files, or a mix of both:
-* config files (`-c some.conf`) can set additional commandline arguments; see [./docs/example.conf](docs/example.conf) and [./docs/example2.conf](docs/example2.conf)
+* using a config-file (`-c some.conf`) is best, easier to read/maintain; see [./docs/example.conf](docs/example.conf)
+  * for complex and intricate setups, see how [./docs/example2.conf](docs/example2.conf) includes additional config-files/folders with `% copyparty.d` which loads [./docs/copyparty.d/some.conf](docs/copyparty.d/some.conf)
 * `kill -s USR1` (same as `systemctl reload copyparty`) to reload accounts and volumes from config files without restarting
  * or click the `[reload cfg]` button in the control-panel if the user has `a`/admin in any volume
  * changes to the `[global]` config section requires a restart to take effect

-**NB:** as humongous as this readme is, there is also a lot of undocumented features. Run copyparty with `--help` to see all available global options; all of those can be used in the `[global]` section of config files, and everything listed in `--help-flags` can be used in volumes as volflags.
+**NB:** as humongous as this readme is, there is also a lot of undocumented features. Run copyparty with [`--help`](https://copyparty.eu/cli/) (or click that link) to see all available global options; all of those can be used in the `[global]` section of config files, and everything listed in [`--help-flags`](https://copyparty.eu/cli/#flags-help-page) can be used in volumes as volflags (per-volume configuration).
 * if running in docker/podman, try this: `docker run --rm -it copyparty/ac --help`
-* or see this: https://ocv.me/copyparty/helptext.html
-* or if you prefer plaintext, https://ocv.me/copyparty/helptext.txt
+* or if you prefer plaintext, https://copyparty.eu/helptext.txt
+
+
+## version-checker
+
+sleep better at night  by telling copyparty to periodically check whether your version has a [known vulnerability](https://github.com/9001/copyparty/security/advisories)
+
+this feature can be enabled by setting the global-option `--vc-url` to one of the following URLs; choose what severity level you want to be notified for:
+* `https://api.copyparty.eu/advisories-panic` -- only really bad stuff, the "UPGRADE NOW" kind
+* `https://api.copyparty.eu/advisories` -- everything important / noteworthy, "upgrade when you can"
+* `https://api.copyparty.eu/advisories-all` -- *everything*, including stuff that's unlikely to affect anyone
+* `https://api.github.com/repos/9001/copyparty/security-advisories?per_page=9` -- same as `advisories-all`
+
+note that `https://api.copyparty.eu/advisories` may (for example) skip some advisories rated `High` but include some `Low`; that's because an easily-reachable `Low` in a default-enabled feature is more severe than a `High` which is a theoretical bug in a contrived use of a fringe feature, but the CVE calculator would still classify that as `High`
+
+if you want to use the github advisory feed but only care about advisories rated `medium`/`moderate` or higher, then global-option `--vc-sev medium` does that, but see previous paragraph
+
+> to see what happens when a bad version is detected, try `--vc-url https://api.copyparty.eu/advisories-test`
+
+also consider the following options:
+* global-option `--vc-age` is how often (in hours) to check that URL; default is 3
+* global-option `--vc-exit` can be enabled to panic and immediately exit if a vulnerability is indicated
+  * if `--vc-exit` is not enabled, it just shows a warning on the controlpanel for all users with permission `a` or `A`
+
+config file example:
+
+```yaml
+[global]
+  vc-url: https://api.copyparty.eu/advisories
+  vc-age: 3  # how many hours to wait between each check
+  vc-exit    # emergency-exit if current version is vulnerable
+  vc-sev: medium  # only care about severity 'Medium'/'Moderate' or higher (github-only; don't use this with api.copyparty.eu)
+```
+
+
+## logging
+
+serverlog is sent to stdout by default  (but logging to a file is also possible)
+
+"stdout" usually means either the terminal, or journalctl, or whatever is collecting logs from your docker containers, so that depends on your setup
+
+* [-q](https://copyparty.eu/cli/#g-q) disables logging to stdout, and may improve performance a little bit
+  * combine it with `-lo logfolder/cpp-%Y-%m-%d.txt` to log to a file instead
+  * the `%Y-%m-%d` makes it create a new logfile every day, with the date as filename
+  * global-option [--rlo](https://copyparty.eu/cli/#rlo-help-page) decides what happens if the filename is taken
+* `-lo whatever.txt` can be used without `-q` to log to both at the same time
+  * by default, the logfile will have colors if the terminal does (usually the case)
+  * use the [textfile-viewer](https://github.com/user-attachments/assets/8a828947-2fae-4df9-bd2a-3de46f42d478) or `less -R` in a terminal to see colors correctly
+* if you want [no colors](https://youtu.be/biW5UVGkPMA?t=148):
+  * `--flo 2` disables colors for just the logfile
+  * `--no-ansi` disables colors for both the terminal and logfile
+
+config file example:
+
+```yaml
+[global]
+  log-date: %Y-%m-%d  # show dates on stdout too
+  lo: /var/log/cpp/%Y-%m-%d.txt  # logfile path
+  flo: 2  # just text (no colors) in logfile
+  q       # disable stdout; use logfile only
+```


 ## zeroconf
@@ -1376,6 +1504,26 @@ config file example, which restricts FTP to only use ports 3921 and 12000-12099
 ```


+## sftp server
+
+goes roughly 700 MiB/s (slower than webdav and ftp)
+
+> this is **not** [ftps](#ftp-server) (which copyparty also supports); [ftps](#ftp-server) is ftp-tls (think http/https), while **sftp** is ssh-based and (preferably) uses ssh-keys for authentication
+
+the sftp-server requires the optional dependency [paramiko](https://pypi.org/project/paramiko/);
+* if you are **not** using docker, then install paramiko somehow
+* if you **are** using docker, then use one of the following image variants: `ac` / `im` / `iv` / `dj`
+
+enable sftpd with `--sftp 3922` to listen on port 3922;
+* use global-option `sftp-key` to associate an ssh-key with a user;
+  * commandline: `--sftp-key 'david ssh-ed25519 AAAAC3NzaC...'`
+  * config-file: `sftp-key: david ssh-ed25519 AAAAC3NzaC...`
+* `--sftp-pw` enables login with passwords (default is ssh-keys only)
+* `--sftp-anon foo` enables login with username `foo` and no password; gives the same access/permissions as the website does when not logged in
+
+see the [sftp section in --help](https://copyparty.eu/cli/#g-sftp) for the other options
+
+
 ## webdav server

 with read-write support,  supports winXP and later, macos, nautilus/gvfs  ... a great way to [access copyparty straight from the file explorer in your OS](#mount-as-drive)
@@ -1389,7 +1537,7 @@ general usage:
 on macos, connect from finder:
 * [Go] -> [Connect to Server...] -> http://192.168.123.1:3923/

-to upload or edit files with WebDAV clients, enable the `daw` volflag (because most WebDAV clients expect this) and give your account the delete-permission. This avoids getting several copies of the same file on the server. HOWEVER: This will also make all PUT-uploads overwrite existing files if the user has delete-access, so use with caution.
+to be able to edit existing files, the client must have the Delete-permission, and some webdav clients will also require the [daw](https://copyparty.eu/cli/#g-daw) volflag or global-option (not necessary if the client sends the `x-oc-mtime` header). Without `daw`, those clients will fail to modify existing files and instead create new copies with names like `notes.txt-1771978661.726032-3i9GPghL.txt`. **NOTE:** Enabling `daw` will also make all PUT-uploads overwrite existing files if the user has delete-access, so use with caution. Another alternative is the [dav-port](https://copyparty.eu/cli/#g-dav-port) option

 > note: if you have enabled [IdP authentication](#identity-providers) then that may cause issues for some/most webdav clients; see [the webdav section in the IdP docs](https://github.com/9001/copyparty/blob/hovudstraum/docs/idp.md#connecting-webdav-clients)

@@ -1450,7 +1598,7 @@ unsafe, slow, not recommended for wan,  enable with `--smb` for read-only or `--

 click the [connect](http://127.0.0.1:3923/?hc) button in the control-panel to see connection instructions for windows, linux, macos

-dependencies: `python3 -m pip install --user -U impacket==0.11.0`
+dependencies: `python3 -m pip install --user -U impacket==0.13.0`
 * newer versions of impacket will hopefully work just fine but there is monkeypatching so maybe not

 some **BIG WARNINGS** specific to SMB/CIFS, in decreasing importance:
@@ -1484,6 +1632,15 @@ authenticate with one of the following:
 * username `$password`, password `k`


+## wopi server
+
+edit office documents in the web-ui
+
+needs a wopi client to work, and the only supported wopi client is a selfhosted collabora online server; eventually onlyoffice will probably also be supported: [issue #1574](https://github.com/9001/copyparty/issues/1574)
+
+no further documentation or examples yet, [issue #1575](https://github.com/9001/copyparty/issues/1575)
+
+
 ## browser ux

 tweaking the ui
@@ -1493,7 +1650,12 @@ tweaking the ui
  * to sort in music order (album, track, artist, title) with filename as fallback, you could `--sort tags/Circle,tags/.tn,tags/Artist,tags/Title,href`
  * to sort by upload date, first enable showing the upload date in the listing with `-e2d -mte +.up_at` and then `--sort tags/.up_at`

-see [./docs/rice](./docs/rice) for more, including how to add stuff (css/`<meta>`/...) to the html `<head>` tag, or to add your own translation
+see [./docs/rice](./docs/rice) for more, including:
+* how to [hide ui-elements](./docs/rice/README.md#hide-ui-elements)
+* [custom fonts](./docs/rice/README.md#custom-fonts)
+* [custom loading-spinner](./docs/rice/README.md#boring-loader-spinner)
+* adding stuff (css/`<meta>`/...) [to the html `<head>` tag](./docs/rice/README.md#head)
+* [adding your own translation](./docs/rice/README.md#translations)


 ## opengraph
@@ -1634,7 +1796,7 @@ avoid traversing into other filesystems  using `--xdev` / volflag `:c,xdev`, ski

 and/or you can `--xvol` / `:c,xvol` to ignore all symlinks leaving the volume's top directory, but still allow bind-mounts pointing elsewhere

-* symlinks are permitted with `xvol` if they point into another volume where the user has the same level of access
+* symlinks are permitted with `xvol` if they point into another volume where the user also has some sort of access, keeping permissions from outer/initial volume

 these options will reduce performance; unlikely worst-case estimates are 14% reduction for directory listings, 35% for download-as-tar

@@ -1680,7 +1842,7 @@ set upload rules using volflags,  some examples:
    * just to avoid additional complexity in up2k which is enough of a mess already
 * `:c,lifetime=300` delete uploaded files when they become 5 minutes old

-you can also set transaction limits which apply per-IP and per-volume, but these assume `-j 1` (default) otherwise the limits will be off, for example `-j 4` would allow anywhere between 1x and 4x the limits you set depending on which processing node the client gets routed to
+you can also set transaction limits which apply per-IP and per-volume, but these assume `-j 1` (default) otherwise the limits will be messed up, for example `-j 4` would allow anywhere between 1x and 4x the limits you set depending on which processing node the client gets routed to

 * `:c,maxn=250,3600` allows 250 files over 1 hour from each IP (tracked per-volume)
 * `:c,maxb=1g,300` allows 1 GiB total over 5 minutes from each IP (tracked per-volume)
@@ -1761,6 +1923,57 @@ notes:
 * `:c,magic` enables filetype detection for nameless uploads, same as `--magic`
  * needs https://pypi.org/project/python-magic/ `python3 -m pip install --user -U python-magic`
  * on windows grab this instead `python3 -m pip install --user -U python-magic-bin`
+* `cachectl` changes how webbrowser will cache responses (the `Cache-Control` response-header); default is `no-cache` which will prevent repeated downloading of the same file unless necessary (browser will ask copyparty if the file has changed)
+  * adding `?cache` to a link will override this with "fully cache this for 69 seconds"; `?cache=321` is 321 seconds, and `?cache=i` is 7 days
+
+
+## descript.ion
+
+add a description to each file in a folder  by adding them to a textfile named `descript.ion`
+
+see https://copyparty.eu/beta/ for an example -- here's a basic `descript.ion` file:
+
+```
+bookmark.mp3 Taishi feat. Rita - Bookmark Memories
+slowstep.mp3 Taishi feat. 向日葵 - Slow Step -F.L.C.A-
+prsnlzr.mp3 Taishi feat. みとせのりこ - Personalizer
+cosmos.mp3 Taishi feat. Rita - Into the cosmos
+```
+
+
+## dothidden
+
+cosmetically hide specific files in a folder  by adding them to a textfile named `.hidden`
+
+this option is default-disabled; enable the volflag and/or global-option `dothidden`
+
+this is **cosmetic only!** the files are still easily accessible in many ways, for example with download-as-zip/tar, so **do not** rely on this for security.
+
+> also see the [--unlist](https://copyparty.eu/cli/#g-unlist) option which is somewhat similar -- `unlist` applies to the whole volume instead of just one folder; however, while dothidden also affects sftp and ftp, the `unlist` option is http/https-only
+
+
+## thumbnail pregen
+
+if you want to pre-generate everything on startup  (usually a bad idea);
+
+by default, thumbnails are created on-the-fly when a client needs it, and then cached on the server for [--th-maxage](https://copyparty.eu/cli/#g-th-maxage) seconds (default is one week), so most thumbnails only need to be created once, and are then eventually deleted from the cache to preserver disk space
+
+but if you need every thumbnail instantly available when a folder is viewed, then first increase the thumbnail expiration time to something really big, and then set global-option `th-pregen` and volflag `th_pregen` to a comma-separated list of thumbnail formats to automatically generate on server startup;
+
+the full list of all possible formats is: `j,jf,jf3,j3,w,wf,wf3,w3,x,xf,xf3,x3,opus,mp3,flac,wav` and I'll explain what those mean soon
+
+* `j` = jpeg cropped, `jf` = jpeg uncropped, `jf3` = jpeg uncropped triplesize, `j3` jpeg cropped triplesize
+* `w` = webm cropped, `wf` = webm uncropped, ..., `x` = jxl cropped, `xf` = jxl uncropped, ...
+* and yes, audio-transcodes are technically thumbnails according to copyparty -- don't think too much about it ( ﾟ ヮﾟ)
+  * unlike thumbnails, the expiry time for audio-transcodes is configured with [--ac-maxage](https://copyparty.eu/cli/#g-ac-maxage)
+
+anyways, obviously you **do not** want to pregenerate flac/wav because they're HUGE, and everything else also gets pretty big because it all adds up;
+
+* each regular thumbnail ( j, jf, w, wf, x, xf ) takes about 16 KiB of disk space
+* each triplesize thumb ( j3, jf3, w3, wf3, x3, xf3 ) takes about 96 KiB
+* each opus / mp3 audiotranscode takes... idk, 6 MiB? depends on song length
+
+so a thousand pictures converted to every possible regular-size image format (`j,jf,w,wf,x,xf`) takes **96 MiB,** and every possible 3x-size (`jf3,j3,wf3,w3,xf3,x3`) takes **562 MiB,** alternatively **658 MiB** in total for all, so that's why the default is to *not* pregenerate on startup, but instead do on-demand with a cache


 ## database location
@@ -1825,6 +2038,23 @@ see the beautiful mess of a dictionary in [mtag.py](https://github.com/9001/copy
 `--mtag-to` sets the tag-scan timeout; very high default (60 sec) to cater for zfs and other randomly-freezing filesystems. Lower values like 10 are usually safe, allowing for faster processing of tricky files


+### metadata from xattrs
+
+unix extended file attributes  (Linux-only) can be indexed into the db and made searchable;
+
+* `--db-xattr user.foo,user.bar` will index the xattrs `user.foo` and `user.bar`,
+* `--db-xattr user.foo=foo,user.bar=bar` will index them with the names `foo` and `bar`,
+* `--db-xattr ~~user.foo,user.bar` will index everything *except* `user.foo` and `user.bar`,
+* `--db-xattr ~~` will index everything
+
+however note that the tags must also be enabled with `-mte` so here are some complete examples:
+* `-e2ts --db-xattr user.foo,user.bar -mte +user.foo,user.bar`
+* `-e2ts --db-xattr user.foo=foo,user.bar=bar -mte +foo,bar`
+
+as for actually adding the xattr `user.foo` to a file in the first place,
+* `setfattr -n user.foo -v 'utsikt fra fløytoppen' photo.jpg`
+
+
 ## file parser plugins

 provide custom parsers to index additional tags,  also see [./bin/mtag/README.md](./bin/mtag/README.md)
@@ -1867,7 +2097,7 @@ trigger a program on uploads, renames etc ([examples](./bin/hooks/))

 you can set hooks before and/or after an event happens, and currently you can hook uploads, moves/renames, and deletes

-there's a bunch of flags and stuff, see `--help-hooks`
+there's a bunch of flags and stuff, see [`--help-hooks`](https://copyparty.eu/cli/#hooks-help-page)

 if you want to write your own hooks, see [devnotes](./docs/devnotes.md#event-hooks)

@@ -2130,9 +2360,7 @@ if you want to change the fonts, see [./docs/rice/](./docs/rice/)

 become a *real* webserver  which people can access by just going to your IP or domain without specifying a port

-**if you're on windows,** then you just need to add the commandline argument `-p 80,443` and you're done! nice
-
-**if you're on macos,** sorry, I don't know
+**if you're on windows or macos,** then you just need to add the commandline argument `-p 80,443` and you're done! nice

 **if you're on Linux,** you have the following 4 options:

@@ -2183,7 +2411,7 @@ example webserver / reverse-proxy configs:
 * [lighttpd subdomain](contrib/lighttpd/subdomain.conf) -- entire domain/subdomain
 * [lighttpd subpath](contrib/lighttpd/subpath.conf) -- location-based (not optimal, but in case you need it)
 * [nginx config](contrib/nginx/copyparty.conf) -- recommended
-* [traefik config](contrib/traefik/copyparty.yaml)
+* [traefik config](contrib/traefik/copyparty.yaml) -- only use v3.6.7 or newer [due to CVE-2025-66490](https://github.com/9001/copyparty/issues/1205)


 ### real-ip
@@ -2351,12 +2579,15 @@ buggy feature? rip it out  by setting any of the following environment variables

 | env-var              | what it does |
 | -------------------- | ------------ |
+| `PRTY_NO_CTYPES`     | do not use features from external libraries such as kernel32 |
 | `PRTY_NO_DB_LOCK`    | do not lock session/shares-databases for exclusive access |
+| `PRTY_NO_ENVEXPAND`  | do not expand environment-variables in configs and args |
 | `PRTY_NO_IFADDR`     | disable ip/nic discovery by poking into your OS with ctypes |
 | `PRTY_NO_IMPRESO`    | do not try to load js/css files using `importlib.resources` |
 | `PRTY_NO_IPV6`       | disable some ipv6 support (should not be necessary since windows 2000) |
 | `PRTY_NO_LZMA`       | disable streaming xz compression of incoming uploads |
 | `PRTY_NO_MP`         | disable all use of the python `multiprocessing` module (actual multithreading, cpu-count for parsers/thumbnailers) |
+| `PRTY_NO_UNIDATA`    | disable loading the unicodedata c-extension; saves 400k ram, breaks `--srch-nfkc` |
 | `PRTY_NO_SQLITE`     | disable all database-related functionality (file indexing, metadata indexing, most file deduplication logic) |
 | `PRTY_NO_TLS`        | disable native HTTPS support; if you still want to accept HTTPS connections then TLS must now be terminated by a reverse-proxy |
 | `PRTY_NO_TPOKE`      | disable systemd-tmpfilesd avoider |
@@ -2390,12 +2621,31 @@ it comes with a [systemd service](./contrib/systemd/copyparty@.service) as well

 after installing, start either the system service or the user service and navigate to http://127.0.0.1:3923 for further instructions (unless you already edited the config files, in which case you are good to go, probably)

+> to start the systemd service, either do `systemctl start --user copyparty` to start it as your own user, or `systemctl start copyparty@bob` to use unix-user `bob`
+

 ## fedora package

 does not exist yet;  there are rumours that it is being packaged! keep an eye on this space...


+## gentoo ::guru package
+
+`emerge www-servers/copyparty::guru` (in [::guru](https://wiki.gentoo.org/wiki/Project:GURU))
+
+but first enable the `::guru` repo;
+
+```bash
+emerge -an app-eselect/eselect-repository
+eselect repository enable guru
+emerge --sync guru
+```
+
+to start the service as a user:
+* OpenRC: `rc-service -U copyparty start && rc-update -U add copyparty default`
+* systemd: [todo]
+
+
 ## homebrew formulae

 `brew install copyparty ffmpeg`  -- https://formulae.brew.sh/formula/copyparty
@@ -2413,8 +2663,6 @@ requires a [flake-enabled](https://nixos.wiki/wiki/Flakes) installation of nix

 some recommended dependencies are enabled by default; [override the package](https://github.com/9001/copyparty/blob/hovudstraum/contrib/package/nix/copyparty/default.nix#L3-L22) if you want to add/remove some features/deps

-`ffmpeg-full` was chosen over `ffmpeg-headless` mainly because we need `withWebp` (and `withOpenmpt` is also nice) and being able to use a cached build felt more important than optimizing for size at the time -- PRs welcome if you disagree 👍
-

 ## nixos module

@@ -2540,6 +2788,12 @@ services.copyparty = {
  };
  # you may increase the open file limit for the process
  openFilesLimit = 8192;
+  
+  # override the package used by the module to add dependencies, e.g. for hooks
+  package = pkgs.copyparty.override {
+    # provides exiftool for bin/hooks/image-noexif.py
+    extraPackages = [ pkgs.exiftool ];
+  };
 };
 ```

@@ -2602,6 +2856,17 @@ quick summary of more eccentric web-browsers trying to view a directory index:
 <p align="center"><img src="https://github.com/user-attachments/assets/88deab3d-6cad-4017-8841-2f041472b853" /></p>


+# server hall of fame
+
+unexpected things that run copyparty:
+
+* an old [allwinner](https://a.ocv.me/pub/g/nerd-stuff/cpp/servers/aallwinner.jpg) android tv-box (ziptie-strapped to an HDD) running a firmware which flips the CPU into Big-Endian mode early during boot
+  * copyparty is [certified BE ready](https://a.ocv.me/pub/g/nerd-stuff/cpp/servers/be-ready.png) -- thanks, [Øl Telecom](http://ol-tele.com/)!
+* an [SGI O2 (photo)](https://a.ocv.me/pub/g/nerd-stuff/cpp/servers/sgi-o2.jpg?cache) with a grand total of 64 MiB RAM running SGI IRIX; [screenshot](https://a.ocv.me/pub/g/nerd-stuff/cpp/servers/sgi-o2.png?cache)
+  * thanks again to the wonderful people at [Øl Telecom](http://ol-tele.com/)
+* a [wristwatch](https://a.ocv.me/pub/g/nerd-stuff/cpp/servers/clockyparty.jpg)
+
+
 # client examples

 interact with copyparty using non-browser clients
@@ -2622,6 +2887,10 @@ interact with copyparty using non-browser clients
  * `chunk(){ curl -H pw:wark -T- http://127.0.0.1:3923/;}`  
    `chunk <movie.mkv`

+* curl: append to existing file with `?apnd`
+  * `log(){ curl -H pw:wark -T- http://127.0.0.1:3923/logfile.txt?apnd;}`  
+    `echo hey | log`
+
 * bash: when curl and wget is not available or too boring
  * `(printf 'PUT /junk?pw=wark HTTP/1.1\r\n\r\n'; cat movie.mkv) | nc 127.0.0.1 3923`
  * `(printf 'PUT / HTTP/1.1\r\n\r\n'; cat movie.mkv) >/dev/tcp/127.0.0.1/3923`
@@ -2744,12 +3013,12 @@ below are some tweaks roughly ordered by usefulness:
 * `--no-htp --hash-mt=0 --mtag-mt=1 --th-mt=1` minimizes the number of threads; can help in some eccentric environments (like the vscode debugger)
 * when running on AlpineLinux or other musl-based distro, try mimalloc for higher performance (and twice as much RAM usage); `apk add mimalloc2` and run copyparty with env-var `LD_PRELOAD=/usr/lib/libmimalloc-secure.so.2`
  * note that mimalloc requires special care when combined with prisonparty and/or bubbleparty/bubblewrap; you must give it access to `/proc` and `/sys` otherwise you'll encounter issues with FFmpeg (audio transcoding, thumbnails)
-* `-j0` enables multiprocessing (actual multithreading), can reduce latency to `20+80/numCores` percent and generally improve performance in cpu-intensive workloads, for example:
+* `-j0` (usually *not* recommended) enables multiprocessing (actual multithreading), can reduce latency to `20+80/numCores` percent and generally improve performance in cpu-intensive workloads, for example:
  * lots of connections (many users or heavy clients)
  * simultaneous downloads and uploads saturating a 20gbps connection
  * if `-e2d` is enabled, `-j2` gives 4x performance for directory listings; `-j4` gives 16x
  
-  ...however it also increases the server/filesystem/HDD load during uploads, and adds an overhead to internal communication, so it is usually a better idea to don't
+  ...however it will probably *reduce* performance in most cases, since it also increases the server/filesystem/HDD load during uploads, and adds an overhead to internal communication, so keeping the default is generally best
 * using [pypy](https://www.pypy.org/) instead of [cpython](https://www.python.org/) *can* be 70% faster for some workloads, but slower for many others
  * and pypy can sometimes crash on startup with `-j0` (TODO make issue)

@@ -2781,15 +3050,20 @@ when uploading files,

 # security

-there is a [discord server](https://discord.gg/25J8CdTT6G)  with an `@everyone` for all important updates (at the lack of better ideas)
+there is a [discord server](https://discord.gg/25J8CdTT6G) with announcements  ; an `@everyone` for all important updates (at the lack of better ideas)

 some notes on hardening

 * set `--rproxy 0` *if and only if* your copyparty is directly facing the internet (not through a reverse-proxy)
  * cors doesn't work right otherwise
 * if you allow anonymous uploads or otherwise don't trust the contents of a volume, you can prevent XSS with volflag `nohtml`
-  * this returns html documents as plaintext, and also disables markdown rendering
-* when running behind a reverse-proxy, listen on a unix-socket for tighter access control (and more performance); see [reverse-proxy](#reverse-proxy) or `--help-bind`
+  * this returns html documents and svg images as plaintext, and also disables markdown rendering
+  * the `nohtml` volflag also enables `noscript` which, on its own, prevents *most* javascript from running; enabling just `noscript` without `nohtml` makes it probably-safe (see below) to view html and svg files, but `nohtml` is necessary to block javascript in markdown documents
+    * "probably-safe" because it relies on `Content-Security-Policy` so it depends on the reverseproxy to forward it, and the browser to understand it, but `nohtml` (the nuclear option) always works
+* when running behind a reverse-proxy, listen on a unix-socket for tighter access control (and more performance); see [reverse-proxy](#reverse-proxy) or [`--help-bind`](https://copyparty.eu/cli/#bind-help-page)
+* put ffmpeg into a sandbox (helps against future ffmpeg vulns) by configuring [--th-bwrap](https://copyparty.eu/cli/#g-th-bwrap) and enable it with `--use-bwrap y`
+  * usually impossible when running in docker or podman
+  * the default value of `--th-bwrap` (see `--help` on your own server) is autogenerated best-effort guess based on your setup; installing `bwrap` and configuring this is up to you (good luck)

 safety profiles:

@@ -2859,6 +3133,8 @@ permissions `wG` (write + upget) lets users upload files and receive their own f

 share specific folders in a volume  without giving away full read-access to the rest -- the visitor only needs the `g` (get) permission to view the link

+> ℹ️ a better alternative to this is [shares](#shares)
+
 volflag `dk` generates dirkeys (per-directory accesskeys) for all folders, granting read-access to that folder; by default only that folder itself, no subfolders

 volflag `dky` disables the actual key-check, meaning anyone can see the contents of a folder where they have `g` access, but not its subdirectories
@@ -2877,7 +3153,7 @@ dirkeys are generated based on another salt (`--dk-salt`) + filesystem-path and

 ## password hashing

-you can hash passwords  before putting them into config files / providing them as arguments; see `--help-pwhash` for all the details
+you can hash passwords  before putting them into config files / providing them as arguments; see [`--help-pwhash`](https://copyparty.eu/cli/#pwhash-help-page) for all the details

 `--ah-alg argon2` enables it, and if you have any plaintext passwords then it'll print the hashed versions on startup so you can replace them

@@ -2894,15 +3170,26 @@ when generating hashes using `--ah-cli` for docker or systemd services, make sur

 ## https

-both HTTP and HTTPS are accepted  by default, but letting a [reverse proxy](#reverse-proxy) handle the https/tls/ssl would be better (probably more secure by default)
+both HTTP and HTTPS are accepted  by default, but please ignore copyparty's built-in https/tls support and instead use a [reverse proxy](#reverse-proxy) to handle https/tls/ssl

-copyparty doesn't speak HTTP/2 or QUIC, so using a reverse proxy would solve that as well -- but note that HTTP/1 is usually faster than both HTTP/2 and HTTP/3
+* reverseproxies do a better job following [best practices](https://cipherlist.eu/) meaning they are more secure, and probably also have higher performance
+* also, copyparty doesn't speak HTTP/2 or QUIC, so using a reverse proxy would solve that as well -- but note that HTTP/1 is usually faster than both HTTP/2 and HTTP/3

 if [cfssl](https://github.com/cloudflare/cfssl/releases/latest) is installed, copyparty will automatically create a CA and server-cert on startup
 * the certs are written to `--crt-dir` for distribution, see `--help` for the other `--crt` options
 * this will be a self-signed certificate so you must install your `ca.pem` into all your browsers/devices
 * if you want to avoid the hassle of distributing certs manually, please consider using a reverse proxy

+to install cfssl on windows:
+* [download](https://github.com/cloudflare/cfssl/releases/latest) `cfssl_windows_amd64.exe`, `cfssljson_windows_amd64.exe`, `cfssl-certinfo_windows_amd64.exe`
+* rename them to `cfssl.exe`, `cfssljson.exe`, `cfssl-certinfo.exe`
+* put them in PATH, for example inside `c:\windows\system32`
+
+if you really wanna give copyparty an existing TLS certificate then do one of the following:
+* `--no-crt --cert server.pem` where `server.pem` is a concatenation of key + cert + chain (in that order), or...
+* `--no-crt --cert server.crt --certkey server.key` where `server.key` is the key, and `server.crt` is a concatenation of cert + chain (in that order)
+* file-extensions don't matter, but all files are expected to be [PEM-style](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/res/insecure.pem)
+

 # recovering from crashes

@@ -2937,7 +3224,7 @@ mandatory deps:

 ## optional dependencies

-install these to enable bonus features
+enable bonus features  by installing these python-packages from pypi or so:

 enable [hashed passwords](#password-hashing) in config: `argon2-cffi`

@@ -2945,6 +3232,8 @@ enable [ftp-server](#ftp-server):
 * for just plaintext FTP, `pyftpdlib` (is built into the SFX)
 * with TLS encryption, `pyftpdlib pyopenssl`

+enable [sftp-server](#sftp-server): `paramiko`
+
 enable [music tags](#metadata-from-audio-files):
 * either `mutagen` (fast, pure-python, skips a few tags, makes copyparty GPL? idk)
 * or `ffprobe` (20x slower, more accurate, possibly dangerous depending on your distro and users)
@@ -2955,16 +3244,20 @@ enable [thumbnails](#thumbnails) of...
 * **HEIF pictures:** `pyvips` or `ffmpeg` or `pillow-heif`
 * **AVIF pictures:** `pyvips` or `ffmpeg` or `pillow-avif-plugin` or pillow v11.3+
 * **JPEG XL pictures:** `pyvips` or `ffmpeg`
-* **RAW images:** `rawpy`, plus one of `pyvips` or `Pillow` (for some formats)
+* **RAW photos:** either `libraw dcraw_emu` or `rawpy`, plus either `pyvips` or `Pillow`

 enable sending [zeromq messages](#zeromq) from event-hooks: `pyzmq`

-enable [smb](#smb-server) support (**not** recommended): `impacket==0.12.0`
+enable [smb](#smb-server) support (**not** recommended): `impacket==0.13.0`

-`pyvips` gives higher quality thumbnails than `Pillow` and is 320% faster, using 270% more ram: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`
+`pyvips` adds support for some additional image formats, but can use extreme amounts of RAM
+* to install `pyvips` on Linux: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`
+* to install `pyvips` on windows: `pip install --user -U "pyvips[binary]"`

 to install FFmpeg on Windows, grab [a recent build](https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z) -- you need `ffmpeg.exe` and `ffprobe.exe` from inside the `bin` folder; copy them into `C:\Windows\System32` or any other folder that's in your `%PATH%`

+if your ffmpeg/ffprobe binaries have nonstandard names -- such as `ffmpeg8` (macports) -- set environment variables `PRTY_FFMPEG_BIN` and `PRTY_FFPROBE_BIN` to the corret name (or full path)
+

 ### dependency chickenbits

@@ -2979,17 +3272,23 @@ set any of the following environment variables to disable its associated optiona
 | -------------------- | ------------ |
 | `PRTY_NO_ARGON2`     | disable argon2-cffi password hashing |
 | `PRTY_NO_CFSSL`      | never attempt to generate self-signed certificates using [cfssl](https://github.com/cloudflare/cfssl) |
+| `PRTY_NO_DCRAW`      | disable all [libraw](https://www.libraw.org/homepage)-based thumbnail support for RAW images |
 | `PRTY_NO_FFMPEG`     | **audio transcoding** goes byebye, **thumbnailing** must be handled by Pillow/libvips |
 | `PRTY_NO_FFPROBE`    | **audio transcoding** goes byebye, **thumbnailing** must be handled by Pillow/libvips, **metadata-scanning** must be handled by mutagen |
 | `PRTY_NO_MAGIC`      | do not use [magic](https://pypi.org/project/python-magic/) for filetype detection |
 | `PRTY_NO_MUTAGEN`    | do not use [mutagen](https://pypi.org/project/mutagen/) for reading metadata from media files; will fallback to ffprobe |
+| `PRTY_NO_PARAMIKO`   | disable sftp server ([paramiko](https://www.paramiko.org/)-based) |
+| `PRTY_NO_PARTFTPY`   | disable tftp server ([partftpy](https://github.com/9001/partftpy)-based) |
 | `PRTY_NO_PIL`        | disable all [Pillow](https://pypi.org/project/pillow/)-based thumbnail support; will fallback to libvips or ffmpeg |
 | `PRTY_NO_PILF`       | disable Pillow `ImageFont` text rendering, used for folder thumbnails |
 | `PRTY_NO_PIL_AVIF`   | disable Pillow avif support (internal and/or [plugin](https://pypi.org/project/pillow-avif-plugin/)) |
 | `PRTY_NO_PIL_HEIF`   | disable 3rd-party Pillow plugin for [HEIF support](https://pypi.org/project/pillow-heif/) |
+| `PRTY_NO_PIL_JXL`    | disable 3rd-party Pillow plugin for [JXL support](https://pypi.org/project/pillow-jxl-plugin/) |
 | `PRTY_NO_PIL_WEBP`   | disable use of native webp support in Pillow |
 | `PRTY_NO_PSUTIL`     | do not use [psutil](https://pypi.org/project/psutil/) for reaping stuck hooks and plugins on Windows |
-| `PRTY_NO_RAW`        | disable all [rawpy](https://pypi.org/project/rawpy/)-based thumbnail support for RAW images |
+| `PRTY_NO_PYFTPD`     | disable ftp(s) server ([pyftpdlib](https://pypi.org/project/pyftpdlib/)-based) |
+| `PRTY_NO_RAW`        | same as `PRTY_NO_DCRAW` plus `PRTY_NO_RAWPY` |
+| `PRTY_NO_RAWPY`      | disable all [rawpy](https://pypi.org/project/rawpy/)-based thumbnail support for RAW images |
 | `PRTY_NO_VIPS`       | disable all [libvips](https://pypi.org/project/pyvips/)-based thumbnail support; will fallback to Pillow or ffmpeg |

 example: `PRTY_NO_PIL=1 python3 copyparty-sfx.py`
@@ -3058,7 +3357,8 @@ another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty
 run it by doubleclicking it, or try typing `python copyparty.pyz` in your terminal/console/commandline/telex if that fails

 it is a python [zipapp](https://docs.python.org/3/library/zipapp.html) meaning it doesn't have to unpack its own python code anywhere to run, so if the filesystem is busted it has a better chance of getting somewhere
-* but note that it currently still needs to extract the web-resources somewhere (they'll land in the default TEMP-folder of your OS)
+
+> there is also [copyparty-en.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty-en.pyz), english-only and without smb support (enterprise-friendly)


 # install on android
@@ -3090,6 +3390,9 @@ and then copypaste the following command into `a-Shell`:
 curl -L https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh | sh
 ```

+> if you want the latest copyparty beta, then do this instead:  
+> `curl -L https://copyparty.eu/beta/setup-ashell.sh | sh`
+
 what this does:
 * creates a basic [config file](#accounts-and-volumes) named `cpc` which you can edit with `vim cpc`
 * adds the command `cpp` to launch copyparty with that config file
@@ -1,9 +1,7 @@
 # Security Policy

-if you hit something extra juicy pls let me know on either of the following
+if you hit something extra juicy pls let me know on one of the following:
 * email -- `copyparty@ocv.ze` except `ze` should be `me`
-* [mastodon dm](https://layer8.space/@tripflag) -- `@tripflag@layer8.space`
 * [github private vulnerability report](https://github.com/9001/copyparty/security/advisories/new), wow that form is complicated
-* [twitter dm](https://twitter.com/tripflag) (if im somehow not banned yet)

 no bug bounties sorry! all i can offer is greetz in the release notes
@@ -26,12 +26,12 @@ and consider using [../docs/rclone.md](../docs/rclone.md) instead; usually a bit
 ## to run this on windows:
 * install [winfsp](https://github.com/billziss-gh/winfsp/releases/latest) and [python 3](https://www.python.org/downloads/)
  * [x] add python 3.x to PATH (it asks during install)
-* `python -m pip install --user fusepy` (or grab a copy of `fuse.py` from the `connect` page on your copyparty, and keep it in the same folder)
+* `python -m pip install --user mfusepy` (or grab a copy of `mfusepy.py` from the `connect` page on your copyparty, and keep it in the same folder)
 * `python ./partyfuse.py n: http://192.168.1.69:3923/`

 10% faster in [msys2](https://www.msys2.org/), 700% faster if debug prints are enabled:
 * `pacman -S mingw64/mingw-w64-x86_64-python{,-pip}`
-* `/mingw64/bin/python3 -m pip install --user fusepy`
+* `/mingw64/bin/python3 -m pip install --user mfusepy`
 * `/mingw64/bin/python3 ./partyfuse.py [...]`

 you could replace winfsp with [dokan](https://github.com/dokan-dev/dokany/releases/latest), let me know if you [figure out how](https://github.com/dokan-dev/dokany/wiki/FUSE)  
@@ -78,6 +78,7 @@ cd /mnt/nas/music/.hist
 # [`prisonparty.sh`](prisonparty.sh)
 * run copyparty in a chroot, preventing any accidental file access
 * creates bindmounts for /bin, /lib, and so on, see `sysdirs=`
+* **not compatible with:** `--th-bwrap`

 # [`bubbleparty.sh`](bubbleparty.sh)
 * run copyparty in an isolated process, preventing any accidental file access and more
@@ -0,0 +1,36 @@
+# /!\ Warning: be careful, as webdav clients often generate a large number of 404 requets.
+
+# In your `jail.local`, add:
+# [copyparty]
+# enabled = true
+# logtimezone = UTC
+# logpath = /path/to/log/file # or keep the default value if you're using systemd
+
+# Create the `copyparty.conf` file in `filter.d` with the following:
+# [Definition]
+# failregex = ^ <ADDR>$
+# ignoreregex =
+# datepattern = ^fail2ban: %%Y-%%m-%%dT%%H:%%M:%%S
+
+# First check `--dont-ban`, and if it doesn't match, log the line to be intercepted by fail2ban.
+from datetime import datetime, UTC
+def main(cli, vn="", rem=""):
+    now = datetime.now(UTC).isoformat()[:19]
+    msg = "\nfail2ban: %s %s"
+    if not vn and not rem:
+        # got called by --xban
+        cli["log"](msg % (now, cli["ip"]), 3)
+        return {"rc": 0}
+
+    cond = cli.args.dont_ban
+    if (
+        cond == "any"
+        or (cond == "auth" and cli.uname != "*")
+        or (cond == "aa" and cli.avol)
+        or (cond == "av" and cli.can_admin)
+        or (cond == "rw" and cli.can_read and cli.can_write)
+    ):
+        return ""
+
+    cli.log(msg % (now, cli.ip), 3)
+    return ""
@@ -22,10 +22,11 @@ each plugin must define a `main()` which takes 3 arguments;

 * [redirect.py](redirect.py) sends an HTTP 301 or 302, redirecting the client to another page/file
 * [randpic.py](randpic.py) redirects `/foo/bar/randpic.jpg` to a random pic in `/foo/bar/`
-* [sorry.py](answer.py) replies with a custom message instead of the usual 404
+* [sorry.py](sorry.py) replies with a custom message instead of the usual 404
 * [nooo.py](nooo.py) replies with an endless noooooooooooooo
 * [never404.py](never404.py) 100% guarantee that 404 will never be a thing again as it automatically creates dummy files whenever necessary
 * [caching-proxy.py](caching-proxy.py) transforms copyparty into a squid/varnish knockoff
+* [404-to-fail2ban.py](404-to-fail2ban.py) creates 404 logs for fail2ban

 ## on403

@@ -8,7 +8,7 @@ def say_no():


 def main(cli, vn, rem):
-    cli.send_headers(None, 404, "text/plain")
+    cli.send_headers("oh_f", None, 404, "text/plain")

    for chunk in say_no():
        cli.s.sendall(chunk)
@@ -4,6 +4,11 @@ these programs either take zero arguments, or a filepath (the affected file), or

 run copyparty with `--help-hooks` for usage details / hook type explanations (xm/xbu/xau/xiu/xbc/xac/xbr/xar/xbd/xad/xban)

+in particular, if a hook is loaded into copyparty with the hook-flag `c` ("check") then its exit-code controls the action that launched the hook:
+* exit-code `0` = allow the action, and/or continue running the next hook
+* exit-code `100` = allow the action, and stop running any remaining consecutive hooks
+* anything else = reject/prevent the original action, and don't run the remaining hooks
+
 > **note:** in addition to event hooks (the stuff described here), copyparty has another api to run your programs/scripts while providing way more information such as audio tags / video codecs / etc and optionally daisychaining data between scripts in a processing pipeline; if that's what you want then see [mtp plugins](../mtag/) instead


@@ -33,8 +38,13 @@ these are `--xiu` hooks; unlike `xbu` and `xau` (which get executed on every sin
  * this hook uses the `I` flag which makes it 140x faster, but if the plugin has a bug it may crash copyparty


+# more upload stuff
+* combine [reloc-by-wark-xbu.py](reloc-by-wark-xbu.py) and [reloc-by-wark-xau.py](reloc-by-wark-xau.py) to rename uploads to the checksum of the file contents
+
+
 # on message
 * [wget.py](wget.py) lets you download files by POSTing URLs to copyparty
+  * [wget-i.py](wget-i.py) is an import-safe modification of this hook (starts 140x faster, but higher chance of bugs)
 * [qbittorrent-magnet.py](qbittorrent-magnet.py) starts downloading a torrent if you post a magnet url
 * [usb-eject.py](usb-eject.py) adds web-UI buttons to safe-remove usb flashdrives shared through copyparty
 * [msg-log.py](msg-log.py) is a guestbook; logs messages to a doc in the same folder
@@ -42,3 +52,4 @@ these are `--xiu` hooks; unlike `xbu` and `xau` (which get executed on every sin

 # general concept demos
 * [import-me.py](import-me.py) shows how the `I` flag makes the hook 140x faster (but you need to be Very Careful when writing the plugin)
+  * [wget-i.py](wget-i.py) is an import-safe modification of [wget.py](wget.py)
@@ -0,0 +1,92 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+
+
+_ = r"""
+rename incoming uploads according to the "wark" (the file identifier)
+which is basically but not exactly a sha512 hash of the file contents
+
+NOTE: this does NOT work with up2k uploads (dragdrop into browser);
+  combine this hook with reloc-by-wark-xbu.py to fix that
+
+example usage as global config:
+    -e2d --xau I,c,bin/hooks/reloc-by-wark-xau.py
+
+parameters explained,
+    e2d = enable up2k database (mandatory for xau hooks)
+    xau = execute before upload
+    I   = import this hook for performance; do not fork / subprocess
+    c   = "check"; reject upload if this hook crashes due to a bug
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,e2d,xau=I,c,bin/hooks/reloc-by-wark-xau.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all uploads with the params explained above)
+
+example usage as a volflag in a copyparty config file:
+    [/inc]
+      srv/inc
+      accs:
+        r: *
+        rw: ed
+      flags:
+        e2d, xau: I,c,bin/hooks/reloc-by-wark-xau.py
+"""
+
+
+def main(inf):
+    if inf.get("wark"):
+        # this is an up2k upload; nothing to be done, just bail
+        return {}
+
+    abspath = inf["ap"]  # filesystem path to the uploaded file
+
+    # we don't have the wark yet so need to calculate it;
+    # generating a regular sha512 would of course be much easier,
+    # but then filenames would be different depending on how the
+    # file was uploaded (laaame) so let's do it the hard way
+
+    # use the standard up2k-salt which nobody ever changes:
+    salt = "hunter2"
+
+    # to generate the wark we'll need some functions from copyparty;
+    # follow the trail to the copyparty module and grab them from there:
+
+    import inspect
+
+    libpath = inspect.getfile(inf["log"])
+    libpath = os.path.dirname(os.path.dirname(libpath))
+    sys.path.insert(0, libpath)
+
+    from copyparty.up2k import up2k_hashlist_from_file, up2k_wark_from_hashlist
+
+    chunklist, st = up2k_hashlist_from_file(abspath)
+    wark = up2k_wark_from_hashlist(salt, st.st_size, chunklist)
+
+    # okay nice
+    # the rest of the code below is just copied from reloc-by-wark-xbu.py
+    # -------------------------------------------------------------------------
+
+    # grab the original filename from the vpath...
+    vdir, fn = os.path.split(inf["vp"])
+
+    # ...to retain the original file extension, if any
+    try:
+        fn, ext = fn.rsplit(".", 1)
+    except:
+        ext = ""
+
+    # use the first 16 characters; 12 bytes of entropy,
+    # roughly one collision for every 26 million files
+    fn = wark[:16]
+
+    if ext:
+        ext = ext.lower()
+        fn += "." + ext
+
+    return {"reloc": {"fn": fn}}
@@ -0,0 +1,69 @@
+#!/usr/bin/env python3
+
+import os
+
+
+_ = r"""
+rename incoming uploads according to the "wark" (the file identifier)
+which is basically but not exactly a sha512 hash of the file contents
+
+NOTE: this only works for up2k uploads (dragdrop into browser);
+  combine this with reloc-by-wark-xau.py to cover the other protocols
+
+example usage as global config:
+    -e2d --xbu I,c,bin/hooks/reloc-by-wark-xbu.py
+
+parameters explained,
+    e2d = enable up2k database (mandatory for xbu hooks)
+    xbu = execute before upload
+    I   = import this hook for performance; do not fork / subprocess
+    c   = "check"; reject upload if this hook crashes due to a bug
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,e2d,xbu=I,c,bin/hooks/reloc-by-wark-xbu.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all uploads with the params explained above)
+
+example usage as a volflag in a copyparty config file:
+    [/inc]
+      srv/inc
+      accs:
+        r: *
+        rw: ed
+      flags:
+        e2d, xbu: I,c,bin/hooks/reloc-by-wark-xbu.py
+"""
+
+
+def main(inf):
+    wark = inf.get("wark")
+    if not wark:
+        # not an up2k upload, so we don't have the hash;
+
+        # option 1: let upload proceed with original filename
+        return {}
+
+        # option 2: reject the upload
+        return {"rejectmsg": "only up2k uploads are allowed in this volume"}
+
+    # grab the original filename from the vpath...
+    vdir, fn = os.path.split(inf["vp"])
+
+    # ...to retain the original file extension, if any
+    try:
+        fn, ext = fn.rsplit(".", 1)
+    except:
+        ext = ""
+
+    # use the first 16 characters; 12 bytes of entropy,
+    # roughly one collision for every 26 million files
+    fn = wark[:16]
+
+    if ext:
+        ext = ext.lower()
+        fn += "." + ext
+
+    return {"reloc": {"fn": fn}}
@@ -34,7 +34,10 @@ MOUNT_BASE = b"/run/media/egon/"

 def main():
    try:
-        label = sys.argv[1].split(":usb-eject:")[1].split(":")[0]
+        msg = sys.argv[1]
+        if msg.startswith("upload-queue-empty;"):
+            return
+        label = msg.split(":usb-eject:")[1].split(":")[0]
        mp = MOUNT_BASE + unquote(label)
        # print("ejecting [%s]... " % (mp,), end="")
        mp = os.path.abspath(os.path.realpath(mp))
@@ -0,0 +1,100 @@
+#!/usr/bin/env python3
+
+import os
+import threading
+import subprocess as sp
+
+
+_ = r"""
+use copyparty as a file downloader by POSTing URLs as
+application/x-www-form-urlencoded (for example using the
+📟 message-to-server-log in the web-ui)
+
+this hook is a modified copy of wget.py, modified to
+make it import-safe so it can be run with the 'I' flag,
+which speeds up the startup time of the hook by 140x
+
+example usage as global config:
+    --xm aw,I,bin/hooks/wget-i.py
+
+parameters explained,
+    xm = execute on message-to-server-log
+    aw = only users with write-access can use this
+    I = import; do not fork / subprocess
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,xm=aw,I,bin/hooks/wget.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all messages with the params explained above)
+
+example usage as a volflag in a copyparty config file:
+    [/inc]
+      srv/inc
+      accs:
+        r: *
+        rw: ed
+      flags:
+        xm: aw,I,bin/hooks/wget.py
+
+the volflag examples only kicks in if you send the message
+while you're in the /inc folder (or any folder below there)
+
+IMPORTANT NOTE:
+    because this hook uses the 'I' flag to run inside copyparty,
+    many other flags will not work (f,j,c3,t3600 as seen in the
+    original wget.py), and furthermore + more importantly we
+    need to be EXCEPTIONALLY CAREFUL to avoid side-effects, so
+    the os.chdir has been replaced with cwd=dirpath for example
+"""
+
+
+def do_stuff(inf):
+    """
+    worker function which is executed in another thread to
+    avoid blocking copyparty while the download is running,
+    since we cannot use the 'f,t3600' hook-flags with 'I'
+    """
+
+    # first things first; grab the logger-function which copyparty is letting us borrow
+    log = inf["log"]
+
+    url = inf["txt"]
+    if url.startswith("upload-queue-empty;"):
+        return
+
+    if "://" not in url:
+        url = "https://" + url
+
+    proto = url.split("://")[0].lower()
+    if proto not in ("http", "https", "ftp", "ftps"):
+        raise Exception("bad proto {}".format(proto))
+
+    dirpath = inf["ap"]
+
+    name = url.split("?")[0].split("/")[-1]
+    msg = "-- DOWNLOADING " + name
+    log(msg)
+    tfn = os.path.join(dirpath, msg)
+    open(tfn, "wb").close()
+
+    cmd = ["wget", "--trust-server-names", "-nv", "--", url]
+
+    try:
+        # two things to note here:
+        # - cannot use the `c3` hook-flag with `I` so mute output with stdout=sp.DEVNULL instead;
+        # - MUST NOT use os.chdir with 'I' so use cwd=dirpath instead 
+        sp.check_call(cmd, cwd=dirpath, stdout=sp.DEVNULL)
+    except:
+        t = "-- FAILED TO DOWNLOAD " + name
+        log(t, 3)  # 3=yellow=warning
+        open(os.path.join(dirpath, t), "wb").close()
+        raise  # have copyparty scream about the details in the log
+
+    os.unlink(tfn)
+
+
+def main(inf):
+    threading.Thread(target=do_stuff, args=(inf,), daemon=True).start()
@@ -47,6 +47,9 @@ while you're in the /inc folder (or any folder below there)
 def main():
    inf = json.loads(sys.argv[1])
    url = inf["txt"]
+    if url.startswith("upload-queue-empty;"):
+        return
+
    if "://" not in url:
        url = "https://" + url

@@ -68,3 +68,8 @@ instead of affecting all volumes, you can set the options for just one volume li
 * `:c,mtp=key=f,audio-key.py`
 * `:c,mtp=.bpm=f,audio-bpm.py`
 * `:c,mtp=ahash,vhash=f,media-hash.py`
+
+
+# tips & tricks
+
+* to delete tags for all files below `blog*` and rescan that, `sqlite3 .hist/up2k.db "delete from mt where w in (select substr(w,1,16) from up where rd like 'blog%')";`
@@ -4,13 +4,19 @@ import os
 import sys
 import tempfile
 import subprocess as sp
-import keyfinder
+
+try:
+    import keyfinder
+
+    PKF = True
+except:
+    PKF = False

 from copyparty.util import fsenc

 """
 dep: github/mixxxdj/libkeyfinder
-dep: pypi/keyfinder
+dep: pypi/keyfinder  -OR-  EvanPurkhiser/keyfinder-cli
 dep: ffmpeg
 """

@@ -35,7 +41,17 @@ def det(tf):
    ])
    # fmt: on

-    print(keyfinder.key(tf).camelot())
+    if PKF:
+        print(keyfinder.key(tf).camelot())
+    else:
+        # fmt: off
+        sp.check_call([
+            b"keyfinder-cli",
+            b"-n",
+            b"camelot",
+            fsenc(tf)
+        ])
+        # fmt: on


 def main():
@@ -0,0 +1,53 @@
+import json
+import re
+import sys
+
+from copyparty.util import fsenc, runcmd
+
+
+"""
+uses exiftool to geotag images based on embedded gps coordinates in exif data
+
+adds four new metadata keys:
+    .gps_lat = latitute
+    .gps_lon = longitude
+    .masl = meters above sea level
+    city = "city, subregion, region"
+
+usage: -mtp .masl,.gps_lat,.gps_lon,city=ad,t10,bin/mtag/geotag.py
+
+example: https://a.ocv.me/pub/blog/j7/8/?grid=0
+"""
+
+
+def main():
+    cmd = b"exiftool -api geolocation -n".split(b" ")
+    rc, so, se = runcmd(cmd + [fsenc(sys.argv[1])])
+    ptn = re.compile("([^:]*[^ :]) *: (.*)")
+    city = ["", "", ""]
+    ret = {}
+    for ln in so.split("\n"):
+        m = ptn.match(ln)
+        if not m:
+            continue
+        k, v = m.groups()
+        if k == "Geolocation City":
+            city[2] = v
+        elif k == "Geolocation Subregion":
+            city[1] = v
+        elif k == "Geolocation Region":
+            city[0] = v
+        elif k == "GPS Latitude":
+            ret[".gps_lat"] = "%.04f" % (float(v),)
+        elif k == "GPS Longitude":
+            ret[".gps_lon"] = "%.04f" % (float(v),)
+        elif k == "GPS Altitude":
+            ret[".masl"] = str(int(float(v)))
+    v = ", ".join(city).strip(", ")
+    if v:
+        ret["city"] = v
+    print(json.dumps(ret))
+
+
+if __name__ == "__main__":
+    main()
@@ -155,6 +155,11 @@ install_keyfinder() {
 		return
 	}

+	(cat /etc/alpine-release || echo a) 2>&1 | grep -E '3\.2[3-9]' && {
+		echo "alpine too new; ffmpeg8 is keyfinder-py incompat; giving up"
+		return
+	}
+
 	cd "$td"
 	github_tarball https://api.github.com/repos/mixxxdj/libkeyfinder/releases/latest
 	ls -al
@@ -189,7 +194,7 @@ install_keyfinder() {
 		exit 1
 	}

-	x=${-//[^x]/}; set -x; cat /etc/alpine-release
+	x=${-//[^x]/}; set -x; cat /etc/alpine-release || true
 	# rm -rf /Users/ed/Library/Python/3.9/lib/python/site-packages/*keyfinder*
 	CFLAGS="-I$h/pe/keyfinder/include -I/opt/local/include -I/usr/include/ffmpeg" \
 	CXXFLAGS="-I$h/pe/keyfinder/include -I/opt/local/include -I/usr/include/ffmpeg" \
@@ -6,8 +6,8 @@ __copyright__ = 2019
 __license__ = "MIT"
 __url__ = "https://github.com/9001/copyparty/"

-S_VERSION = "2.1"
-S_BUILD_DT = "2025-09-06"
+S_VERSION = "2.2"
+S_BUILD_DT = "2025-12-16"

 """
 mount a copyparty server (local or remote) as a filesystem
@@ -21,7 +21,7 @@ usage:
  python partyfuse.py http://192.168.1.69:3923/  ./music

 dependencies:
-  python3 -m pip install --user fusepy  # or grab it from the connect page
+  python3 -m pip install --user mfusepy  # or grab it from the connect page
  + on Linux: sudo apk add fuse
  + on Macos: https://osxfuse.github.io/
  + on Windows: https://github.com/billziss-gh/winfsp/releases/latest
@@ -92,8 +92,13 @@ is_dbg = False


 try:
-    from fuse import FUSE, FuseOSError, Operations
+    from mfusepy import FUSE, FuseOSError, Operations
 except:
+    try:
+        from fuse import FUSE, FuseOSError, Operations
+    except:
+        FUSE = None
+
    if WINDOWS:
        libfuse = "install https://github.com/billziss-gh/winfsp/releases/latest"
    elif MACOS:
@@ -102,12 +107,13 @@ except:
        libfuse = "apt install libfuse2\n    modprobe fuse"

    m = """\033[33m
-  could not import fuse; these may help:
-    {} -m pip install --user fusepy
+  could not import mfusepy; these may help:
+    {} -m pip install --user mfusepy
    {}
 \033[0m"""
-    print(m.format(sys.executable, libfuse))
-    raise
+    if not FUSE:
+        print(m.format(sys.executable, libfuse))
+        raise


 def termsafe(txt):
@@ -143,10 +149,10 @@ def fancy_log(fmt, *a):


 def register_wtf8():
-    def wtf8_enc(text):
+    def wtf8_enc(text, errors=""):
        return str(text).encode("utf-8", "surrogateescape"), len(text)

-    def wtf8_dec(binary):
+    def wtf8_dec(binary, errors=""):
        return bytes(binary).decode("utf-8", "surrogateescape"), len(binary)

    def wtf8_search(encoding_name):
@@ -284,8 +290,8 @@ class Gateway(object):
            if ar.td:
                self.ssl_context = ssl._create_unverified_context()
            elif ar.te:
-                self.ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS)
-                self.ssl_context.load_verify_locations(ar.te)
+                self.ssl_context = ssl.create_default_context(cafile=ar.te)
+                self.ssl_context.check_hostname = ar.teh

        self.conns = {}

@@ -620,7 +626,7 @@ class Gateway(object):
            "st_atime": ts,
            "st_mtime": ts,
            "st_ctime": ts,
-            "st_blocks": int((sz + 511) / 512),
+            "st_blocks": -int(-sz // 512),
        }

    def stat_file(self, ts, sz):
@@ -632,7 +638,7 @@ class Gateway(object):
            "st_atime": ts,
            "st_mtime": ts,
            "st_ctime": ts,
-            "st_blocks": int((sz + 511) / 512),
+            "st_blocks": -int(-sz // 512),
        }


@@ -1165,6 +1171,7 @@ NOTE: if server has --usernames enabled, then password is "username:password"

    ap2 = ap.add_argument_group("https/TLS")
    ap2.add_argument("-te", metavar="PEMFILE", help="certificate to expect/verify")
+    ap2.add_argument("-teh", action="store_true", help="require correct hostname in -te cert")
    ap2.add_argument("-td", action="store_true", help="disable certificate check")

    ap2 = ap.add_argument_group("cache/perf")
@@ -20,6 +20,8 @@ example:
 example for running straight from source (instead of using an sfx):
  PYTHONPATH=$PWD ./prisonparty.sh /var/lib/copyparty-jail cpp cpp /mnt/nas/music -- python3 -um copyparty -v /mnt/nas/music::rwmd

+NOT COMPATIBLE WITH: --th-bwrap
+
 note that if you have python modules installed as --user (such as bpm/key detectors),
  you should add /home/foo/.local as a VOLDIR

@@ -97,6 +99,7 @@ jail="${jail%/}"


 # bind-mount system directories and volumes
+mkdir -p "$jail"
 for a in {1..30}; do mkdir "$jail/.prisonlock" && break; sleep 0.1; done
 printf '%s\n' "${sysdirs[@]}" "${vols[@]}" | sed -r 's`/$``' | LC_ALL=C sort | uniq |
 while IFS= read -r v; do
@@ -1,8 +1,8 @@
 #!/usr/bin/env python3
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

-S_VERSION = "2.13"
-S_BUILD_DT = "2025-09-05"
+S_VERSION = "2.22"
+S_BUILD_DT = "2026-06-13"

 """
 u2c.py: upload to copyparty
@@ -100,7 +100,7 @@ except:
    ub64enc = base64.urlsafe_b64encode


-class BadAuth(Exception):
+class Fatal(Exception):
    pass


@@ -165,8 +165,8 @@ class HCli(object):
            elif self.verify is True:
                self.ctx = None
            else:
-                self.ctx = ssl.SSLContext(ssl.PROTOCOL_TLS)
-                self.ctx.load_verify_locations(self.verify)
+                self.ctx = ssl.create_default_context(cafile=self.verify)
+                self.ctx.check_hostname = ar.teh

        self.base_hdrs = {
            "Accept": "*/*",
@@ -196,6 +196,8 @@ class HCli(object):
        hdrs.update(self.base_hdrs)
        if self.ar.a:
            hdrs["PW"] = self.ar.a
+        if self.ar.ba:
+            hdrs["Authorization"] = self.ar.ba
        if ctype:
            hdrs["Content-Type"] = ctype
        if meth == "POST" and CLEN not in hdrs:
@@ -232,6 +234,7 @@ class HCli(object):

 MJ = "application/json"
 MO = "application/octet-stream"
+MM = "application/x-www-form-urlencoded"
 CLEN = "Content-Length"

 web = None  # type: HCli
@@ -395,7 +398,7 @@ class MTHash(object):
            self.csz = chunksz

            chunks = {}
-            nchunks = int(math.ceil(fsz / chunksz))
+            nchunks = -int(-fsz // chunksz)
            for nch in range(nchunks):
                self.work_q.put(nch)

@@ -491,6 +494,12 @@ print = safe_print if VT100 else flushing_print


 def termsize():
+    try:
+        w, h = os.get_terminal_size()
+        return w, h
+    except:
+        pass
+
    env = os.environ

    def ioctl_GWINSZ(fd):
@@ -716,7 +725,7 @@ def up2k_chunksize(filesize):
    stepsize = 512 * 1024
    while True:
        for mul in [1, 2]:
-            nchunks = math.ceil(filesize * 1.0 / chunksize)
+            nchunks = -int(-filesize // chunksize)
            if nchunks <= 256 or (chunksize >= 32 * 1024 * 1024 and nchunks <= 4096):
                return chunksize

@@ -724,16 +733,22 @@ def up2k_chunksize(filesize):
            stepsize *= mul


-# mostly from copyparty/up2k.py
 def get_hashlist(file, pcb, mth):
    # type: (File, Any, Any) -> None
+    with open(file.abs, "rb", 512 * 1024) as f:
+        get_hashlist_f(file, pcb, mth, f)
+
+
+# mostly from copyparty/up2k.py
+def get_hashlist_f(file, pcb, mth, f):
+    # type: (File, Any, Any, Any) -> None
    """generates the up2k hashlist from file contents, inserts it into `file`"""

    chunk_sz = up2k_chunksize(file.size)
    file_rem = file.size
    file_ofs = 0
    ret = []
-    with open(file.abs, "rb", 512 * 1024) as f:
+    if True:
        t0 = time.time()

        if mth and file.size >= 1024 * 512:
@@ -769,6 +784,18 @@ def get_hashlist(file, pcb, mth):
            file.kchunks[k] = [v1, v2]


+def wark_stdin(ar):
+    file = File(b"", b"", int(ar.files[0]), 0)
+    get_hashlist_f(file, None, False, sys.stdin if PY2 else sys.stdin.buffer)
+    if ar.chs:
+        zsl = ["%s %d %d" % (zsii[0], n, zsii[1]) for n, zsii in enumerate(file.cids)]
+        print("chs:\n" + "\n".join(zsl))
+    zsl = [ar.wsalt, str(file.size)] + [x[0] for x in file.cids]
+    zb = hashlib.sha512("\n".join(zsl).encode("utf-8")).digest()[:33]
+    wark = ub64enc(zb).decode("utf-8")
+    print(wark)
+
+
 def printlink(ar, purl, name, fk):
    if not name:
        url = purl  # srch
@@ -826,10 +853,15 @@ def handshake(ar, file, search):
            url = ""
        url = ar.vtop + url

+    t0 = time.time()
+    tmax = t0 + ar.t_hs
    while True:
        sc = 600
        txt = ""
-        t0 = time.time()
+        t1 = time.time()
+        if t1 >= tmax:
+            print("\nERROR: server offline for longer than --t-hs; giving up")
+            raise Fatal()
        try:
            zs = json.dumps(req, separators=(",\n", ": "))
            sc, txt = web.req("POST", url, {}, zs.encode("utf-8"), MJ)
@@ -850,13 +882,13 @@ def handshake(ar, file, search):
                return [], False
            elif sc == 409 or "<pre>upload rejected, file already exists" in txt:
                return [], False
-            elif sc == 403:
+            elif sc == 403 or sc == 401:
                print("\nERROR: login required, or wrong password:\n%s" % (txt,))
-                raise BadAuth()
+                raise Fatal()

-            t = "handshake failed, retrying: %s\n  t0=%.3f t1=%.3f td=%.3f\n  %s\n\n"
+            t = "handshake failed, retrying: %s\n  t0=%.3f t1=%.3f t2=%.3f td1=%.3f td2=%.3f\n  %s\n\n"
            now = time.time()
-            eprint(t % (file.name, t0, now, now - t0, em))
+            eprint(t % (file.name, t0, t1, now, now - t0, now - t1, em))
            time.sleep(ar.cd)

    try:
@@ -979,6 +1011,7 @@ class Ctl(object):
        self.nfiles, self.nbytes = self.stats
        self.filegen = walkdirs([], ar.files, ar.x)
        self.recheck = []  # type: list[File]
+        self.last_file = None

        if ar.safe:
            self._safe()
@@ -1015,6 +1048,11 @@ class Ctl(object):

            self._fancy()

+        file = self.last_file
+        if self.up_br and file:
+            zs = quotep(file.name.encode("utf-8", WTF8))
+            web.req("POST", file.url, {}, b"msg=upload-queue-empty;" + zs, MM)
+
        self.ok = not self.errs

    def _safe(self):
@@ -1036,7 +1074,7 @@ class Ctl(object):
                print("  hs...")
                try:
                    hs, _ = handshake(self.ar, file, search)
-                except BadAuth:
+                except Fatal:
                    sys.exit(1)

                if search:
@@ -1164,7 +1202,7 @@ class Ctl(object):
            handshake(self.ar, file, False)

    def cleanup_vt100(self):
-        if VT100:
+        if VT100 and not self.ar.ns:
            ss.scroll_region(None)
        else:
            eprint("\033]9;4;0\033\\")
@@ -1225,9 +1263,7 @@ class Ctl(object):
                            while req:
                                print("DELETING ~%s#%s" % (srd, len(req)))
                                body = json.dumps(req).encode("utf-8")
-                                sc, txt = web.req(
-                                    "POST", self.ar.url + "?delete", {}, body, MJ
-                                )
+                                sc, txt = web.req("POST", "/?delete", {}, body, MJ)
                                if sc == 413 and "json 2big" in txt:
                                    print(" (delete request too big; slicing...)")
                                    req = req[: len(req) // 2]
@@ -1293,7 +1329,8 @@ class Ctl(object):
                if self.ar.jw:
                    print("%s  %s" % (wark, vp))
                else:
-                    zd = datetime.datetime.fromtimestamp(max(0, file.lmod), UTC)
+                    tsdt = datetime.datetime.fromtimestamp
+                    zd = tsdt(max(0, min(2 << 36, file.lmod)), UTC)
                    dt = "%04d-%02d-%02d %02d:%02d:%02d" % (
                        zd.year,
                        zd.month,
@@ -1343,7 +1380,7 @@ class Ctl(object):

            try:
                hs, sprs = handshake(self.ar, file, search)
-            except BadAuth:
+            except Fatal:
                self.panik = 1
                break

@@ -1455,6 +1492,7 @@ class Ctl(object):

            file = fsl.file
            cids = fsl.cids
+            self.last_file = file

            with self.mutex:
                if not self.uploader_busy:
@@ -1538,7 +1576,8 @@ NOTE: if server has --usernames enabled, then password is "username:password"
    ap.add_argument("url", type=unicode, help="server url, including destination folder")
    ap.add_argument("files", type=files_decoder, nargs="+", help="files and/or folders to process")
    ap.add_argument("-v", action="store_true", help="verbose")
-    ap.add_argument("-a", metavar="PASSWD", help="password or $filepath")
+    ap.add_argument("-a", metavar="PASSWD", default="", help="password (or $filepath) for copyparty (is sent in header 'PW')")
+    ap.add_argument("--ba", metavar="PASSWD", default="", help="password (or $filepath) for basic-auth (usually not necessary)")
    ap.add_argument("-s", action="store_true", help="file-search (disables upload)")
    ap.add_argument("-x", type=unicode, metavar="REGEX", action="append", help="skip file if filesystem-abspath matches REGEX (option can be repeated), example: '.*/\\.hist/.*'")
    ap.add_argument("--ok", action="store_true", help="continue even if some local files are inaccessible")
@@ -1566,6 +1605,7 @@ NOTE: if server has --usernames enabled, then password is "username:password"
    ap.add_argument("--wsalt", type=unicode, metavar="S", default="hunter2", help="salt to use when creating warks; must match server config")
    ap.add_argument("--chs", action="store_true", help="verbose (print the hash/offset of each chunk in each file)")
    ap.add_argument("--jw", action="store_true", help="just identifier+filepath, not mtime/size too")
+    ap.add_argument("--stdin", action="store_true", help="calculate file-ID of stdin; u2c.py --stdin - $LEN < a.mkv")

    ap = app.add_argument_group("performance tweaks")
    ap.add_argument("-j", type=int, metavar="CONNS", default=2, help="parallel connections")
@@ -1576,11 +1616,13 @@ NOTE: if server has --usernames enabled, then password is "username:password"
    ap.add_argument("-ns", action="store_true", help="no status panel (for slow consoles and macos)")
    ap.add_argument("--cxp", type=float, metavar="SEC", default=57, help="assume http connections expired after SEConds")
    ap.add_argument("--cd", type=float, metavar="SEC", default=5, help="delay before reattempting a failed handshake/upload")
+    ap.add_argument("--t-hs", type=float, metavar="SEC", default=186, help="crash if handshakes fail due to server-offline for this long")
    ap.add_argument("--safe", action="store_true", help="use simple fallback approach")
    ap.add_argument("-z", action="store_true", help="ZOOMIN' (skip uploading files if they exist at the destination with the ~same last-modified timestamp, so same as yolo / turbo with date-chk but even faster)")

    ap = app.add_argument_group("tls")
    ap.add_argument("-te", metavar="PATH", help="path to ca.pem or cert.pem to expect/verify")
+    ap.add_argument("-teh", action="store_true", help="require correct hostname in -te cert")
    ap.add_argument("-td", action="store_true", help="disable certificate check")
    # fmt: on

@@ -1594,6 +1636,10 @@ NOTE: if server has --usernames enabled, then password is "username:password"
            except:
                pass

+    if ar.stdin:
+        wark_stdin(ar)
+        return
+
    # msys2 doesn't uncygpath absolute paths with whitespace
    if not VT100:
        zsl = []
@@ -1676,11 +1722,15 @@ NOTE: if server has --usernames enabled, then password is "username:password"
            print("\n\n   %s\n\n" % (t,))
            raise

-    if ar.a and ar.a.startswith("$"):
-        fn = ar.a[1:]
-        print("reading password from file [{0}]".format(fn))
-        with open(fn, "rb") as f:
-            ar.a = f.read().decode("utf-8").strip()
+    for k in ("a", "ba"):
+        zs = getattr(ar, k)
+        if zs.startswith("$"):
+            print("reading password from file [%s]" % (zs[1:],))
+            with open(zs[1:], "rb") as f:
+                setattr(ar, k, f.read().decode("utf-8").strip())
+
+    if ar.ba:
+        ar.ba = "Basic " + base64.b64encode(ar.ba.encode("utf-8")).decode("utf-8")

    for n in range(ar.rh):
        try:
@@ -61,6 +61,8 @@ def rep_server():
        print("copyparty says %r" % (sck.recv_string(),))
        reply = b"thx"
        # reply = b"return 1"  # non-zero to block an upload
+        # reply = b'{"rc":1}'  # or as json, that's fine too
+        # reply = b'{"rejectmsg":"naw dude"}'  # or custom message
        sck.send(reply)


@@ -6,7 +6,7 @@
 #
 # to reverse-proxy a specific path/subpath/location below a domain
 # (rather than a complete subdomain), for example "/qw/er", you must
-# run copyparty with --rp-loc /qw/as and also change the following:
+# run copyparty with --rp-loc /qw/er and also change the following:
 # 	location / {
 # 		proxy_pass http://cpp_tcp;
 # to this:
@@ -75,9 +75,8 @@ server {
 		# recommendation: replace cpp_tcp with cpp_uds below
 		proxy_pass http://cpp_tcp;
 		proxy_redirect off;
-		# disable buffering (next 4 lines)
+		# disable buffering (next 3 lines)
 		proxy_http_version 1.1;
-		client_max_body_size 0;
 		proxy_buffering off;
 		proxy_request_buffering off;
 		# improve download speed from 600 to 1500 MiB/s
@@ -48,6 +48,8 @@ let

  accountsWithPlaceholders = mapAttrs (name: attrs: passwordPlaceholder name);

+  volumesWithoutVariables = filterAttrs (k: v: !(hasInfix "\${" v.path)) cfg.volumes;
+
  configStr = ''
    ${mkSection "global" cfg.settings}
    ${cfg.globalExtraConfig}
@@ -67,11 +69,8 @@ in
  options.services.copyparty = {
    enable = mkEnableOption "web-based file manager";

-    package = mkOption {
-      type = types.package;
-      default = pkgs.copyparty;
-      defaultText = "pkgs.copyparty";
-      description = ''
+    package = mkPackageOption pkgs "copyparty" {
+      extraDescription = ''
        Package of the application to run, exposed for overriding purposes.
      '';
    };
@@ -80,7 +79,7 @@ in
      type = types.bool;
      default = true;
      description = ''
-        Make a shell script wrapper called 'copyparty-hash' with all options set here,
+        Make a shell script wrapper called {command}`copyparty-hash` with all options set here,
        that launches the hashing cli.
      '';
    };
@@ -115,9 +114,9 @@ in
      type = types.attrs;
      description = ''
        Global settings to apply.
-        Directly maps to values in the [global] section of the copyparty config.
+        Directly maps to values in the `[global]` section of the copyparty config.
        Cannot set "c" or "hist", those are set by this module.
-        See `${getExe cfg.package} --help` for more details.
+        See {command}`copyparty --help` for more details.
      '';
      default = {
        i = "127.0.0.1";
@@ -136,7 +135,7 @@ in
    globalExtraConfig = mkOption {
      type = types.str;
      default = "";
-      description = "Appended to the end of the [global] section verbatim. This is useful for flags which are used in a repeating manner (e.g. ipu: 255.255.255.1=user) which can't be repeated in the settings = {} attribute set.";
+      description = "Appended to the end of the `[global]` section verbatim. This is useful for flags which are used in a repeating manner (e.g. `ipu: 255.255.255.1=user`) which can't be repeated in the settings = {} attribute set.";
    };

    accounts = mkOption {
@@ -199,21 +198,21 @@ in
                  Attribute list of permissions and the users to apply them to.

                  The key must be a string containing any combination of allowed permission:
-                    "r" (read):   list folder contents, download files
-                    "w" (write):  upload files; need "r" to see the uploads
-                    "m" (move):   move files and folders; need "w" at destination
-                    "d" (delete): permanently delete files and folders
-                    "g" (get):    download files, but cannot see folder contents
-                    "G" (upget):  "get", but can see filekeys of their own uploads
-                    "h" (html):   "get", but folders return their index.html
-                    "a" (admin):  can see uploader IPs, config-reload
+                  * "r" (read):   list folder contents, download files
+                  * "w" (write):  upload files; need "r" to see the uploads
+                  * "m" (move):   move files and folders; need "w" at destination
+                  * "d" (delete): permanently delete files and folders
+                  * "g" (get):    download files, but cannot see folder contents
+                  * "G" (upget):  "get", but can see filekeys of their own uploads
+                  * "h" (html):   "get", but folders return their index.html
+                  * "a" (admin):  can see uploader IPs, config-reload

                  For example: "rwmd"

                  The value must be one of:
-                    an account name, defined in `accounts`
-                    a list of account names
-                    "*", which means "any account"
+                  * an account name, defined in `accounts`
+                  * a list of account names
+                  * "*", which means "any account"
                '';
                example = literalExpression ''
                  {
@@ -228,7 +227,7 @@ in
                type = types.attrs;
                description = ''
                  Attribute list of volume flags to apply.
-                  See `${getExe cfg.package} --help-flags` for more details.
+                  See {command}`copyparty --help-flags` for more details.
                '';
                example = literalExpression ''
                  {
@@ -325,7 +324,7 @@ in
          BindPaths =
            (if cfg.settings ? hist then [ cfg.settings.hist ] else [ ])
            ++ [ externalStateDir ]
-            ++ (mapAttrsToList (k: v: v.path) cfg.volumes);
+            ++ (mapAttrsToList (k: v: v.path) volumesWithoutVariables);
          # ProtectSystem = "strict";
          # Note that unlike what 'ro' implies,
          # this actually makes it impossible to read anything in the root FS,
@@ -364,10 +363,20 @@ in
              #: in front of things means it wont change it if the directory already exists.
              group = ":${cfg.group}";
              user = ":${cfg.user}";
-              mode = ":755";
+              mode = ":${
+                # Use volume permissions if set
+                if (value.flags ? chmod_d) then
+                  value.flags.chmod_d
+                # Else, use global permission if set
+                else if (cfg.settings ? chmod-d) then
+                  cfg.settings.chmod-d
+                # Else, use the default permission
+                else
+                  "755"
+              }";
            };
          }
-        ) cfg.volumes
+        ) volumesWithoutVariables
      );

      users.groups = lib.mkIf (cfg.group == "copyparty") {
@@ -9,10 +9,19 @@
 # you may want to:
 #   change '/usr/bin/python' to another interpreter
 #   change '/mnt::rw' to another location or permission-set
+#   use a config file instead of command arguments, e.g.:
+#      command_args="-c /etc/copyparty.conf"

 name="$SVCNAME"
 command_background=true
+extra_commands="checkconfig"
 pidfile="/var/run/$SVCNAME.pid"

 command="/usr/bin/python3 /usr/local/bin/copyparty-sfx.py"
 command_args="-q -v /mnt::rw"
+
+checkconfig() {
+        ebegin "Checking $RC_SVCNAME configuration"
+        $command $command_args --exit cfg
+        eend $?
+}
@@ -3,9 +3,9 @@
 # NOTE: You generally shouldn't use this PKGBUILD on Arch, as it is mainly for testing purposes. Install copyparty using pacman instead.

 pkgname=copyparty
-pkgver="1.19.15"
+pkgver="1.20.18"
 pkgrel=1
-pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
+pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++"
 arch=("any")
 url="https://github.com/9001/${pkgname}"
 license=('MIT')
@@ -14,8 +14,9 @@ makedepends=("python-wheel" "python-setuptools" "python-build" "python-installer
 optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tags"
            "cfssl: generate TLS certificates on startup"
            "python-mutagen: music tags (alternative)"
+            "python-paramiko: sftp server",
            "python-pillow: thumbnails for images"
-            "python-pyvips: thumbnails for images (higher quality, faster, uses more ram)"
+            "python-pyvips: thumbnails for images (eats ram, prefer pillow)"
            "libkeyfinder: detection of musical keys"
            "python-pyopenssl: ftps functionality"
            "python-pyzmq: send zeromq messages from event-hooks"
@@ -23,7 +24,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
 )
 source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
 backup=("etc/${pkgname}/copyparty.conf" )
-sha256sums=("95f4c1e517acaadd5fdc38b47047f16a7cd51495bd94ae9182274f256db59834")
+sha256sums=("ea1347a2e30b4261282ac4609c9b844768a925eed717ac8af1ff87a8a326508e")

 build() {
    cd "${srcdir}/${pkgname}-${pkgver}/copyparty/web"
@@ -2,9 +2,9 @@


 pkgname=copyparty
-pkgver=1.19.15
+pkgver=1.20.18
 pkgrel=1
-pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
+pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++"
 arch=("any")
 url="https://github.com/9001/${pkgname}"
 license=('MIT')
@@ -13,6 +13,7 @@ makedepends=("python3-wheel" "python3-setuptools" "python3-build" "python3-insta
 optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tags"
            "golang-cfssl: generate TLS certificates on startup"
            "python3-mutagen: music tags (alternative)"
+            "python3-paramiko: sftp server"
            "python3-pil: thumbnails for images"
            "python3-openssl: ftps functionality"
            "python3-zmq: send zeromq messages from event-hooks"
@@ -20,7 +21,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
 )
 source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
 backup=("/etc/${pkgname}.d/init" )
-sha256sums=("95f4c1e517acaadd5fdc38b47047f16a7cd51495bd94ae9182274f256db59834")
+sha256sums=("ea1347a2e30b4261282ac4609c9b844768a925eed717ac8af1ff87a8a326508e")

 build() {
    cd "${srcdir}/${pkgname}-${pkgver}/copyparty/web"
@@ -15,6 +15,7 @@
  pyzmq,
  ffmpeg,
  mutagen,
+  paramiko,
  pyftpdlib,
  magic,
  partftpy,
@@ -29,7 +30,7 @@
  # create thumbnails with Pillow; faster than FFmpeg / MediaProcessing
  withThumbnails ? true,

-  # create thumbnails with PyVIPS; even faster, uses more memory
+  # create thumbnails with PyVIPS; uses more memory, pillow is preferred
  # -- can be combined with Pillow to support more filetypes
  withFastThumbnails ? false,

@@ -44,6 +45,9 @@
  # send ZeroMQ messages from event-hooks
  withZeroMQ ? true,

+  # enable SFTP server
+  withSFTP ? false,
+
  # enable FTP server
  withFTP ? true,

@@ -67,24 +71,71 @@
  # additional dependencies
  extraPythonPackages ? (_p: [ ]),

+  # to build stable + unstable with the same file
+  stable ? true,
+
+  # for commit date, only used when stable = false
+  copypartyFlake ? null,
+
+  nix-gitignore,
 }:

 let
  pinData = lib.importJSON ./pin.json;
  runtimeDeps = ([ util-linux ] ++ extraPackages ++ lib.optional withMediaProcessing ffmpeg);
+  inherit (copypartyFlake) lastModifiedDate;
+  # ex: "1970" "01" "01"
+  dateStringsZeroPrefixed = {
+    year = builtins.substring 0 4 lastModifiedDate;
+    month = builtins.substring 4 2 lastModifiedDate;
+    day = builtins.substring 6 2 lastModifiedDate;
+  };
+  # ex: "1970" "1" "1"
+  dateStringsShort = builtins.mapAttrs (_: val: toString (lib.toIntBase10 val)) dateStringsZeroPrefixed;
+  unstableVersion =
+    if copypartyFlake == null then
+      "${pinData.version}-unstable"
+    else
+      with dateStringsZeroPrefixed; "${pinData.version}-unstable-${year}-${month}-${day}"
+  ;
+  version = if stable then pinData.version else unstableVersion;
+  stableSrc = fetchurl {
+    inherit (pinData) url hash;
+  };
+  root = ../../../..;
+  unstableSrc = nix-gitignore.gitignoreSource [] root;
+  src = if stable then stableSrc else unstableSrc;
+  rev = copypartyFlake.shortRev or copypartyFlake.dirtyShortRev or "unknown";
+  unstableCodename = "unstable" + (lib.optionalString (copypartyFlake != null) "-${rev}");
 in
 buildPythonApplication {
  pname = "copyparty";
-  inherit (pinData) version;
-  src = fetchurl {
-    inherit (pinData) url hash;
-  };
+  inherit version src;
+  postPatch = lib.optionalString (!stable) ''
+    old_src="$(mktemp -d)"
+    tar -C "$old_src" -xf ${stableSrc}
+    declare -a folders
+    folders=("$old_src"/*)
+    count_folders="''${#folders[@]}"
+    if [[ $count_folders != 1 ]]; then
+      declare -p folders
+      echo "Expected 1 folder, found $count_folders" >&2
+      exit 1
+    fi
+    old_src_folder="''${folders[0]}"
+    cp -r "$old_src_folder"/copyparty/web/deps copyparty/web/deps
+    sed -i 's/^CODENAME =.*$/CODENAME = "${unstableCodename}"/' copyparty/__version__.py
+    ${lib.optionalString (copypartyFlake != null) (with dateStringsShort; ''
+      sed -i 's/^BUILD_DT =.*$/BUILD_DT = (${year}, ${month}, ${day})/' copyparty/__version__.py
+    '')}
+  '';
  dependencies =
    [
      jinja2
      fusepy
    ]
    ++ lib.optional withSMB impacket
+    ++ lib.optional withSFTP paramiko
    ++ lib.optional withFTP pyftpdlib
    ++ lib.optional withFTPS pyopenssl
    ++ lib.optional withTFTP partftpy
@@ -106,7 +157,7 @@ buildPythonApplication {
  meta = {
    description = "Turn almost any device into a file server";
    longDescription = ''
-      Portable file server with accelerated resumable uploads, dedup, WebDAV,
+      Portable file server with accelerated resumable uploads, dedup, WebDAV, SFTP,
      FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps
    '';
    homepage = "https://github.com/9001/copyparty";
@@ -1,5 +1,5 @@
 {
-    "url": "https://github.com/9001/copyparty/releases/download/v1.19.15/copyparty-1.19.15.tar.gz",
-    "version": "1.19.15",
-    "hash": "sha256-lfTB5Resqt1f3Di0cEfxanzVFJW9lK6RgidPJW21mDQ="
+    "url": "https://github.com/9001/copyparty/releases/download/v1.20.18/copyparty-1.20.18.tar.gz",
+    "version": "1.20.18",
+    "hash": "sha256-6hNHouMLQmEoKsRgnJuER2ipJe7XF6yK8f+HqKMmUI4="
 }
@@ -1,8 +1,29 @@
-final: prev: {
-  copyparty = final.python3.pkgs.callPackage ./copyparty {
-    ffmpeg = final.ffmpeg-full;
+final: prev:
+let
+  fullAttrs = {
+    withHashedPasswords = true;
+    withCertgen = true;
+    withThumbnails = true;
+    withFastThumbnails = true;
+    withMediaProcessing = true;
+    withBasicAudioMetadata = true;
+    withZeroMQ = true;
+    withSFTP = true;
+    withFTP = true;
+    withFTPS = true;
+    withTFTP = true;
+    withSMB = true;
+    withMagic = true;
  };

+  call = attrs: final.python3.pkgs.callPackage ./copyparty ({ ffmpeg = final.ffmpeg-headless; } // attrs);
+in
+{
+  copyparty = call { stable = true; };
+  copyparty-unstable = call { stable = false; };
+  copyparty-full = call (fullAttrs // { stable = true; });
+  copyparty-unstable-full = call (fullAttrs // { stable = false; });
+
  python3 = prev.python3.override {
    packageOverrides = pyFinal: pyPrev: {
      partftpy = pyFinal.callPackage ./partftpy { };
@@ -5,15 +5,15 @@ License:        MIT
 Group:          Utilities
 URL:            https://github.com/9001/copyparty
 Source0:        copyparty-$pkgver.tar.gz
-Summary:        File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++
+Summary:        File server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++
 BuildArch:      noarch
 BuildRequires:  python3, python3-devel, pyproject-rpm-macros, python-setuptools, python-wheel, make
 Requires:       python3, (python3-jinja2 or python-jinja2), lsof
 Recommends:     ffmpeg, (golang-github-cloudflare-cfssl or cfssl), python-mutagen, python-pillow, python-pyvips
-Recommends:     qm-vamp-plugins, python-argon2-cffi, (python-pyopenssl or pyopenssl), python-impacket
+Recommends:     qm-vamp-plugins, python-argon2-cffi, (python-pyopenssl or pyopenssl), python-paramiko, python-impacket

 %description
-Portable file server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps
+Portable file server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps

 See release at https://github.com/9001/copyparty/releases

@@ -39,3 +39,9 @@ point `--css-browser` to one of these by URL:

 * turns copyparty into chromecast just more flexible (and probably way more buggy)
 * usage: put the js somewhere in the webroot and `--js-browser /memes/meadup.js`
+
+
+
+# junk
+
+* [**rave.js**](./rave.js): april-fools joke, [demo (epilepsy warning)](https://cd.ocv.me/b/d2/d21/#af-9b927c42,sorthref), not maintained, very buggy
@@ -36,7 +36,7 @@

 		for (var a = 0; a < files.length; a++) {
 			var file = files[a],
-				is_pic = /\.(jpe?g|png|gif|webp)$/i.exec(file.vp),
+				is_pic = /\.(jpe?g|png|gif|webp|jxl)$/i.exec(file.vp),
 				is_audio = re_au_all.exec(file.vp),
 				basename = file.vp.replace(/\.[^\.]+$/, ""),
 				entry = pairs[basename];
@@ -66,11 +66,14 @@ var u2min = `

 </style>

-<a href="#" onclick="this.parentNode.innerHTML='';">show advanced options</a>
+<a href="#" id="u2min_off">show advanced options</a>
 `;

 if (!has(perms, 'read')) {
  var e2 = mknod('div');
  e2.innerHTML = u2min;
  ebi('wrap').insertBefore(e2, QS('#wfp'));
+  ebi('u2min_off').onclick = function () {
+    this.parentNode.innerHTML='';
+  };
 }
@@ -24,8 +24,8 @@ Note that you can select the owner and group of this volume by changing the `uid
 To install and start copyparty with Podman and systemd as the root user, run the following:

 ```shell
-sudo mkdir -pv /etc/systemd/container/ /etc/copyparty/
-sudo cp -v copyparty.container /etc/systemd/containers/
+sudo mkdir -pv /etc/containers/systemd/ /etc/copyparty/
+sudo cp -v copyparty.container /etc/containers/systemd/
 sudo cp -v copyparty.conf /etc/copyparty/
 sudo systemctl daemon-reload
 sudo systemctl start copyparty
@@ -8,7 +8,7 @@
  # and copyparty replaces %Y-%m%d with Year-MonthDay, so the
  # full path will be something like /var/log/copyparty/2023-1130.txt
  # (note: enable compression by adding .xz at the end)
-  # q, lo: $LOGS_DIRECTORY/%Y-%m%d.log
+  # q, lo: ${LOGS_DIRECTORY}/%Y-%m%d.log

  # p: 80,443,3923   # listen on 80/443 as well (requires CAP_NET_BIND_SERVICE)
  # i: 127.0.0.1     # only allow connections from localhost (reverse-proxies)
@@ -22,6 +22,12 @@
  # stats, nos-dup   # enable the prometheus endpoint, but disable the dupes counter (too slow)
  # no-robots, force-js  # make it harder for search engines to read your server

+  # enable version-checking by uncommenting one of the vc-url lines below;
+  # shows a warning-banner in the controlpanel if your version has a known vulnerability
+  #vc-url: https://api.github.com/repos/9001/copyparty/security-advisories?per_page=9
+  #vc-url: https://api.copyparty.eu/advisories
+  vc-exit  # panic and shutdown instead of just showing the warning
+

 [accounts]
  ed: wark  # username: password
@@ -32,5 +38,7 @@
  accs:
    rw: *      # everyone gets read-write access, but
    rwmda: ed  # the user "ed" gets read-write-move-delete-admin
+  flags:
+    e2ds       # enable filesystem-scanning for this volume only
    # uid: 1000  # If you're running as root, you can change the owner of this volume here
    # gid: 1000  # If you're running as root, you can change the group of this volume here
@@ -1,7 +1,7 @@
 #!/bin/sh
 #
 # PROVIDE: copyparty
-# REQUIRE: networking
+# REQUIRE: networking DAEMON FILESYSTEMS mountd
 # KEYWORD:

 . /etc/rc.subr
@@ -10,7 +10,7 @@ name="copyparty"
 rcvar="copyparty_enable"
 copyparty_user="copyparty"
 copyparty_args="-e2dsa -v /storage:/storage:r" # change as you see fit
-copyparty_command="/usr/local/bin/python3.9 /usr/local/copyparty/copyparty-sfx.py ${copyparty_args}"
+copyparty_command="/usr/local/bin/python3.11 /usr/local/copyparty/copyparty-sfx.py ${copyparty_args}"
 pidfile="/var/run/copyparty/${name}.pid"
 command="/usr/sbin/daemon"
 command_args="-P ${pidfile} -r -f ${copyparty_command}"
@@ -6,7 +6,7 @@
 #   https://apps.apple.com/us/app/a-shell/id1473805438
 #
 # step 2: copypaste the following command into a-Shell:
-#   curl -L https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh
+#   curl -L https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh | sh
 #
 # step 3: launch copyparty with this command: cpp
 #
@@ -16,7 +16,16 @@
  # and copyparty replaces %Y-%m%d with Year-MonthDay, so the
  # full path will be something like /var/log/copyparty/2023-1130.txt
  # (note: enable compression by adding .xz at the end)
-  q, lo: $LOGS_DIRECTORY/%Y-%m%d.log
+  q, lo: ${LOGS_DIRECTORY}/%Y-%m%d.log
+
+  # enable version-checker by uncommenting one of the 'vc-url' lines below; this will
+  # periodically check if your copyparty version has a known security vulnerability,
+  # showing a warning on /?h (control-panel) for all users with permission 'a' or 'A'
+  #vc-url: https://api.github.com/repos/9001/copyparty/security-advisories?per_page=9
+  #vc-url: https://api.copyparty.eu/advisories
+
+  vc-age: 3  # how many hours to wait between each version-check
+  vc-exit  # emergency-exit if a version-check indicates that the current version is vulnerable

  # p: 80,443,3923   # listen on 80/443 as well (requires CAP_NET_BIND_SERVICE)
  # i: 127.0.0.1     # only allow connections from localhost (reverse-proxies)
@@ -40,3 +49,5 @@
  accs:
    rw: *      # everyone gets read-write access, but
    rwmda: ed  # the user "ed" gets read-write-move-delete-admin
+  flags:
+    e2ds       # enable filesystem-scanning for this volume only
@@ -0,0 +1,36 @@
+# docker-compose.yml for copyparty
+services:
+  copyparty:
+    image: copyparty/ac:latest
+    container_name: copyparty
+    user: "1000:1000"
+    volumes:
+      - ./:/cfg:z
+      - /path/to/your/fileshare/top/folder:/w:z
+  
+    environment:
+      LD_PRELOAD: /usr/lib/libmimalloc-secure.so.NOPE
+      # enable mimalloc by replacing "NOPE" with "2" for a nice speed-boost (will use twice as much ram)
+
+      PYTHONUNBUFFERED: 1
+      # ensures log-messages are not delayed (but can reduce speed a tiny bit)
+
+    stop_grace_period: 15s  # thumbnailer is allowed to continue finishing up for 10s after the shutdown signal
+    healthcheck:
+      # hide it from logs with "/._" so it matches the default --lf-url filter 
+      test: ["CMD-SHELL", "wget --spider -q 127.0.0.1:3923/?reset=/._"]
+      interval: 1m
+      timeout: 2s
+      retries: 5
+      start_period: 15s
+    networks:
+      - traefik_default
+    labels:
+      - traefik.http.routers.copyparty.rule=Host(`<DOMAIN>`)
+      - traefik.http.services.copyparty.loadbalancer.server.port=3923
+      - traefik.http.routers.copyparty.tls.certresolver=le
+      - traefik.http.routers.copyparty.entrypoints=websecure
+  
+networks:
+  traefik_default:
+    external: true
@@ -0,0 +1,43 @@
+# note: copied from copyparty/docs/examples/docker/basic-docker-compose/copyparty.conf
+# not actually YAML but lets pretend:
+# -*- mode: yaml -*-
+# vim: ft=yaml:
+
+
+[global]
+  e2dsa  # enable file indexing and filesystem scanning
+  e2ts   # enable multimedia indexing
+  ansi   # enable colors in log messages (both in logfiles and stdout)
+
+  # q, lo: /cfg/log/%Y-%m%d.log   # log to file instead of docker
+
+  # p: 3939          # listen on another port
+  # ipa: 10.89.      # only allow connections from 10.89.*
+  # ipa: 172.16.4.0/23  # ...or only 172.16.4.* and 172.16.5.*
+  # ipa: lan         # ...or allow LAN only; reject internet IPs
+  # df: 16           # stop accepting uploads if less than 16 GB free disk space
+  # ver              # show copyparty version in the controlpanel
+  # grid             # show thumbnails/grid-view by default
+  # theme: 2         # monokai
+  # name: datasaver  # change the server-name that's displayed in the browser
+  # stats, nos-dup   # enable the prometheus endpoint, but disable the dupes counter (too slow)
+  # no-robots, force-js  # make it harder for search engines to read your server
+
+  # enable version-checking by uncommenting one of the vc-url lines below;
+  # shows a warning-banner in the controlpanel if your version has a known vulnerability
+  #vc-url: https://api.github.com/repos/9001/copyparty/security-advisories?per_page=9
+  #vc-url: https://api.copyparty.eu/advisories
+  vc-exit  # panic and shutdown instead of just showing the warning
+
+
+[accounts]
+  ed: wark  # username: password
+
+
+[/]            # create a volume at "/" (the webroot), which will
+  /w           # share /w (the docker data volume)
+  accs:
+    rw: *      # everyone gets read-write access, but
+    rwmda: ed  # the user "ed" gets read-write-move-delete-admin
+  flags:
+    e2ds       # enable filesystem-scanning for this volume only
@@ -0,0 +1,34 @@
+# docker-compose.yml for traefik
+services:
+  traefik:
+    image: traefik:latest
+    command:
+      - --api.insecure=false
+      - --api.dashboard=true
+      - --providers.docker=true
+      - --entrypoints.web.address=:80
+      - --entrypoints.websecure.address=:443
+      - --entryPoints.websecure.http.tls=true
+      - --entryPoints.web.http.redirections.entryPoint.to=websecure
+      - --entryPoints.web.http.redirections.entryPoint.scheme=https
+      - --certificatesresolvers.le.acme.email=<your email here>
+      - --certificatesresolvers.le.acme.storage=/letsencrypt/acme.json
+      - --certificatesresolvers.le.acme.httpchallenge.entrypoint=web
+    ports:
+      - 80:80
+      - 443:443
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock
+      - ./letsencrypt:/letsencrypt
+    labels:
+      # OPTIONAL: host your traefik dashboard on <DOMAIN>
+      # - traefik.http.routers.traefik-dash.rule=Host(`<DOMAIN>`)
+      # - traefik.http.routers.traefik-dash.entrypoints=websecure
+      # - traefik.http.routers.traefik-dash.service=api@internal
+      # - traefik.http.routers.traefik-dash.tls.certresolver=le
+      # - traefik.http.routers.traefik-dash.middlewares=traefik-dash-auth,traefik-dash-redirect
+      # - traefik.http.middlewares.traefik-dash-auth.basicauth.users=<USERNAME>:<HTPASSWD_ENCODED_PASSWD>
+      # - traefik.http.middlewares.traefik-dash-redirect.redirectregex.regex=^https://<SUBDOMAIN>\.<DOMAIN>\.TLD/?$
+      # - traefik.http.middlewares.traefik-dash-redirect.redirectregex.replacement=https://<DOMAIN>/dashboard/
+
+networks: {}
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os
 import platform
@@ -44,6 +44,16 @@ ANYWIN = WINDOWS or sys.platform in ["msys", "cygwin"]

 MACOS = platform.system() == "Darwin"

+FREEBSD = platform.system() == "FreeBSD"
+
+OPENBSD = platform.system() == "OpenBSD"
+
+ANYBSD = FREEBSD or OPENBSD
+
+UNIX = MACOS or ANYBSD
+
+GRAAL = platform.python_implementation() == "GraalVM"
+
 EXE = bool(getattr(sys, "frozen", False))

 try:
@@ -55,7 +65,7 @@ except:
 zs = """
 web/a/partyfuse.py
 web/a/u2c.py
-web/a/webdav-cfg.bat
+web/a/webdav-cfg.txt
 web/baguettebox.js
 web/browser.css
 web/browser.html
@@ -67,9 +77,10 @@ web/deps/busy.mp3
 web/deps/easymde.css
 web/deps/easymde.js
 web/deps/marked.js
-web/deps/fuse.py
+web/deps/mfusepy.py
 web/deps/mini-fa.css
 web/deps/mini-fa.woff
+web/deps/orbitron.woff2
 web/deps/prism.css
 web/deps/prism.js
 web/deps/prismd.css
@@ -86,7 +97,6 @@ web/md2.js
 web/mde.css
 web/mde.html
 web/mde.js
-web/msg.css
 web/msg.html
 web/opds.xml
 web/rups.css
@@ -100,12 +110,39 @@ web/splash.html
 web/splash.js
 web/svcs.html
 web/svcs.js
+web/tl/chi.js
+web/tl/cze.js
+web/tl/deu.js
+web/tl/epo.js
+web/tl/fin.js
+web/tl/fra.js
+web/tl/grc.js
+web/tl/hun.js
+web/tl/ita.js
+web/tl/jpn.js
+web/tl/kor.js
+web/tl/nld.js
+web/tl/nno.js
+web/tl/nor.js
+web/tl/pol.js
+web/tl/por.js
+web/tl/rus.js
+web/tl/spa.js
+web/tl/swe.js
+web/tl/tur.js
+web/tl/ukr.js
+web/tl/vie.js
 web/ui.css
 web/up2k.js
 web/util.js
-web/w.hash.js
+web/wopi.html
 """
 RES = set(zs.strip().split("\n"))
+RESM = {
+    "web/a/partyfuse.txt": "web/a/partyfuse.py",
+    "web/a/u2c.txt": "web/a/u2c.py",
+    "web/a/webdav-cfg.bat": "web/a/webdav-cfg.txt",
+}


 class EnvParams(object):
@@ -1,6 +1,6 @@
 #!/usr/bin/env python3
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 """copyparty: http file sharing hub (py2/py3)"""
 __author__ = "ed <copyparty@ocv.me>"
@@ -28,6 +28,7 @@ from .__init__ import (
    MACOS,
    PY2,
    PY36,
+    UNIX,
    VT100,
    WINDOWS,
    E,
@@ -45,9 +46,11 @@ from .util import (
    DEF_EXP,
    DEF_MTE,
    DEF_MTH,
+    HAVE_BWRAP,
    HAVE_IPV6,
    IMPLICATIONS,
    JINJA_VER,
+    MIKO_VER,
    MIMES,
    PARTFTPY_VER,
    PY_DESC,
@@ -62,13 +65,22 @@ from .util import (
    Daemon,
    align_tab,
    b64enc,
+    ctypes,
    dedent,
+    expand_osenv_c,
+    expand_osenv_cs,
+    expand_osenv_noop,
+    expand_osenv_s,
    has_resource,
+    list_ips,
+    list_nics,
    load_resource,
+    lprint,
    min_ex,
    pybin,
    read_utf8,
    termsize,
+    wk32,
    wrap,
 )

@@ -91,13 +103,14 @@ except:
    HAVE_SSL = False

 u = unicode
-printed: list[str] = []
 zsid = uuid.uuid4().urn[4:]

 CFG_DEF = [os.environ.get("PRTY_CONFIG", "")]
 if not CFG_DEF[0]:
    CFG_DEF.pop()

+FULL_HELP = os.environ.get("PRTY_FULL_HELP", "")
+

 class RiceFormatter(argparse.HelpFormatter):
    def __init__(self, *args: Any, **kwargs: Any) -> None:
@@ -167,16 +180,6 @@ class BasicDodge11874(
        super(BasicDodge11874, self).__init__(*args, **kwargs)


-def lprint(*a: Any, **ka: Any) -> None:
-    eol = ka.pop("end", "\n")
-    txt: str = " ".join(unicode(x) for x in a) + eol
-    printed.append(txt)
-    if not VT100:
-        txt = RE_ANSI.sub("", txt)
-
-    print(txt, end="", **ka)
-
-
 def warn(msg: str) -> None:
    lprint("\033[1mwarning:\033[0;33m {}\033[0m\n".format(msg))

@@ -424,9 +427,22 @@ def configure_ssl_ciphers(al: argparse.Namespace) -> None:
        sys.exit(0)


+def expand_cvars(argv) -> list[str]:
+    n = 0
+    for v in argv:
+        if "=" in v:
+            a, b = v.split("=", 1)
+            v = "%s=%s" % (a, os.path.expanduser(expand_osenv_c(b)))
+        else:
+            v = os.path.expanduser(expand_osenv_c(v))
+        argv[n] = v
+        n += 1
+    return argv
+
+
 def args_from_cfg(cfg_path: str) -> list[str]:
    lines: list[str] = []
-    expand_config_file(None, lines, cfg_path, "")
+    expand_config_file(None, expand_osenv_c, lines, cfg_path, "")
    lines = upgrade_cfg_fmt(None, argparse.Namespace(vc=False), lines, "")

    ret: list[str] = []
@@ -450,10 +466,12 @@ def args_from_cfg(cfg_path: str) -> list[str]:
            else:
                ret.append(prefix + k + "=" + v)

-    return ret
+    return expand_cvars(ret)


 def expand_cfg(argv) -> list[str]:
+    argv = expand_cvars(argv)
+
    if CFG_DEF:
        supp = args_from_cfg(CFG_DEF[0])
        argv = argv[:1] + supp + argv[1:]
@@ -476,6 +494,21 @@ def expand_cfg(argv) -> list[str]:
    return argv


+def quotecheck(al):
+    for zs1, zco in vars(al).items():
+        zsl = [u(x) for x in zco] if isinstance(zco, list) else [u(zco)]
+        for zs2 in zsl:
+            zs2 = zs2.strip()
+            zs3 = zs2.strip("\"'")
+            if zs2 == zs3 or len(zs2) - len(zs3) < 2:
+                continue
+            if al.c:
+                t = "found the following global-config:   %s: %s\n values should not be quoted; did you mean:   %s: %s"
+            else:
+                t = 'found the following config-option:  "--%s=%s"\n values should not be quoted; did you mean:  "--%s=%s"'
+            warn(t % (zs1, zs2, zs1, zs3))
+
+
 def sighandler(sig: Optional[int] = None, frame: Optional[FrameType] = None) -> None:
    msg = [""] * 5
    for th in threading.enumerate():
@@ -488,8 +521,10 @@ def sighandler(sig: Optional[int] = None, frame: Optional[FrameType] = None) ->


 def disable_quickedit() -> None:
+    if not ctypes:
+        raise Exception("no ctypes")
+
    import atexit
-    import ctypes
    from ctypes import wintypes

    def ecb(ok: bool, fun: Any, args: list[Any]) -> list[Any]:
@@ -499,10 +534,10 @@ def disable_quickedit() -> None:
                raise ctypes.WinError(err)  # type: ignore
        return args

-    k32 = ctypes.WinDLL(str("kernel32"), use_last_error=True)  # type: ignore
    if PY2:
        wintypes.LPDWORD = ctypes.POINTER(wintypes.DWORD)

+    k32 = wk32
    k32.GetStdHandle.errcheck = ecb  # type: ignore
    k32.GetConsoleMode.errcheck = ecb  # type: ignore
    k32.SetConsoleMode.errcheck = ecb  # type: ignore
@@ -605,6 +640,9 @@ def get_sects():
            \033[32m-i fd:\033[33m3\033[0m uses the socket passed to copyparty on file descriptor 3

            \033[33m-p\033[0m (tcp ports) is ignored for unix-sockets and FDs
+
+            \033[33m--list-nics\033[0m shows all network adapters (also offline ones);
+            \033[33m--list-ips\033[0m shows all LAN IPs
            """
            ),
        ],
@@ -649,8 +687,11 @@ def get_sects():
            if no accounts or volumes are configured,
            current folder will be read/write for everyone

-            the group @acct will always have every user with an account
-            (the name of that group can be changed with --grp-all)
+            the group \033[33m@acct\033[0m will always have every user with an account
+            (the name of that group can be changed with \033[32m--grp-all\033[0m)
+
+            to hide a volume from authenticated users, specify \033[33m*,-@acct\033[0m
+            to subtract \033[33m@acct\033[0m from \033[33m*\033[0m (can subtract users from groups too)

            consider the config file for more flexible account/volume management,
            including dynamic reload at runtime (and being more readable w)
@@ -672,12 +713,12 @@ def get_sects():

            send the password in the '\033[36mPW\033[0m' http-header:
              \033[36mPW: \033[35mhunter2\033[0m
-            or if you have \033[33m--accounts\033[0m enabled,
+            or if you have \033[33m--usernames\033[0m enabled,
              \033[36mPW: \033[35med:hunter2\033[0m

            send the password in the URL itself:
              \033[36mhttp://127.0.0.1:3923/\033[35m?pw=hunter2\033[0m
-            or if you have \033[33m--accounts\033[0m enabled,
+            or if you have \033[33m--usernames\033[0m enabled,
              \033[36mhttp://127.0.0.1:3923/\033[35m?pw=ed:hunter2\033[0m

            use basic-authentication:
@@ -806,7 +847,8 @@ def get_sects():
            \033[0m
            hooks specified as commandline --args are appended to volflags;
            each commandline --arg and volflag can be specified multiple times,
-            each hook will execute in order unless one returns non-zero
+            each hook will execute in order unless one returns non-zero, or
+            "100" which means "stop daisychaining and return 0 (success/OK)"

            optionally prefix the command with comma-sep. flags similar to -mtp:

@@ -926,6 +968,7 @@ def get_sects():
            values for --urlform:
              \033[36mstash\033[35m dumps the data to file and returns length + checksum
              \033[36msave,get\033[35m dumps to file and returns the page like a GET
+              \033[36mget\033[35m ignores the message and returns the page like a GET
              \033[36mprint    \033[35m prints the data to log and returns an error
              \033[36mprint,xm \033[35m prints the data to log and returns --xm output
              \033[36mprint,get\033[35m prints the data to log and returns GET\033[0m
@@ -986,6 +1029,28 @@ def get_sects():
            """
            ),
        ],
+        [
+            "rlo",
+            "logrotate format",
+            dedent(
+                """
+            a logrotate-counter is added if the logfile filename is taken;
+            by default at the end, unless \033[32m%R\033[0m is somewhere in the \033[36m-lo\033[0m pattern,
+            for example:  -lo /var/log/cpp/%Y-%m-%d%R.txt
+
+            \033[36m--rlo\033[0m configures the logrotate format; examples:
+            .1   = when necessary, append a dot followed by a single digit
+            .1!  = counter is always added, even when not necessary
+            -3   = a hyphen followed by three-digit counter
+            no   = disable counter; append to existing file
+            del  = delete existing logfile and create new
+            over = overwrite existing logfile
+
+            append is not possible for .xz-compressed logfiles;
+            if logfile is *.xz and rlo is no then rlo will be .1
+            """
+            ),
+        ],
        [
            "ls",
            "volume inspection",
@@ -1035,13 +1100,17 @@ def get_sects():

            similarly, \033[33m--chmod-d\033[0m and \033[33mchmod_d\033[0m sets the directory/folder perm

-            the value is a three-digit octal number such as \033[32m755\033[0m, \033[32m750\033[0m, \033[32m644\033[0m, etc.
+            the value is a three- or four-digit octal number such as \033[32m755\033[0m, \033[32m0644\033[0m, \033[32m2750\033[0m, etc.

-            first digit = "User"; permission for the unix-user
-            second digit = "Group"; permission for the unix-group
-            third digit = "Other"; permission for all other users/groups
+            if 3 digits: User, Group, Other
+            if 4 digits: Special, User, Group, Other

-            for files:
+            "Special" digit (sum of the following):
+            \033[32m1\033[0m = sticky (files: n/a; dirs: files in directory can be deleted only by their owners)
+            \033[32m2\033[0m = setgid (files: run executable as file group; dirs: files inherit group from directory)
+            \033[32m4\033[0m = setuid (files: run executable as file owner; dirs: n/a)
+
+            "User", "Group", "Other" digits for files:
            \033[32m0\033[0m = \033[35m---\033[0m = no access
            \033[32m1\033[0m = \033[35m--x\033[0m = can execute the file as a program
            \033[32m2\033[0m = \033[35m-w-\033[0m = can write
@@ -1051,7 +1120,7 @@ def get_sects():
            \033[32m6\033[0m = \033[35mrw-\033[0m = can read and write
            \033[32m7\033[0m = \033[35mrwx\033[0m = can read, write, execute

-            for directories/folders:
+            "User", "Group", "Other" digits for directories/folders:
            \033[32m0\033[0m = \033[35m---\033[0m = no access
            \033[32m1\033[0m = \033[35m--x\033[0m = can read files in folder but not list contents
            \033[32m2\033[0m = \033[35m-w-\033[0m = n/a
@@ -1157,7 +1226,6 @@ def add_general(ap, nc, srvname):
    ap2 = ap.add_argument_group("general options")
    ap2.add_argument("-c", metavar="PATH", type=u, default=CFG_DEF, action="append", help="\033[34mREPEATABLE:\033[0m add config file")
    ap2.add_argument("-nc", metavar="NUM", type=int, default=nc, help="max num clients")
-    ap2.add_argument("-j", metavar="CORES", type=int, default=1, help="max num cpu cores, 0=all")
    ap2.add_argument("-a", metavar="ACCT", type=u, action="append", help="\033[34mREPEATABLE:\033[0m add account, \033[33mUSER\033[0m:\033[33mPASS\033[0m; example [\033[32med:wark\033[0m]")
    ap2.add_argument("-v", metavar="VOL", type=u, action="append", help="\033[34mREPEATABLE:\033[0m add volume, \033[33mSRC\033[0m:\033[33mDST\033[0m:\033[33mFLAG\033[0m; examples [\033[32m.::r\033[0m], [\033[32m/mnt/nas/music:/music:r:aed\033[0m], see --help-accounts")
    ap2.add_argument("--grp", metavar="G:N,N", type=u, action="append", help="\033[34mREPEATABLE:\033[0m add group, \033[33mNAME\033[0m:\033[33mUSER1\033[0m,\033[33mUSER2\033[0m,\033[33m...\033[0m; example [\033[32madmins:ed,foo,bar\033[0m]")
@@ -1169,11 +1237,20 @@ def add_general(ap, nc, srvname):
    ap2.add_argument("--name", metavar="TXT", type=u, default=srvname, help="server name (displayed topleft in browser and in mDNS)")
    ap2.add_argument("--name-url", metavar="TXT", type=u, help="URL for server name hyperlink (displayed topleft in browser)")
    ap2.add_argument("--name-html", type=u, help=argparse.SUPPRESS)
+    ap2.add_argument("--site", metavar="URL", type=u, default="", help="public URL to assume when creating links; example: [\033[32mhttps://example.com/\033[0m]")
+    ap2.add_argument("--env-expand", metavar="N", type=int, default=-1, help="expand environment-variables in config-files? [\033[32m0\033[0m]=no, [\033[32m1\033[0m]=$VAR (old scary syntax), [\033[32m2\033[0m]=${VAR} (new recommended syntax); default is new-syntax with panic if old-syntax is seen")
    ap2.add_argument("--mime", metavar="EXT=MIME", type=u, action="append", help="\033[34mREPEATABLE:\033[0m map file \033[33mEXT\033[0mension to \033[33mMIME\033[0mtype, for example [\033[32mjpg=image/jpeg\033[0m]")
    ap2.add_argument("--mimes", action="store_true", help="list default mimetype mapping and exit")
    ap2.add_argument("--rmagic", action="store_true", help="do expensive analysis to improve accuracy of returned mimetypes; will make file-downloads, rss, and webdav slower (volflag=rmagic)")
+    ap2.add_argument("-j", metavar="CORES", type=int, default=1, help="num cpu-cores for uploads/downloads (0=all); keeping the default is almost always best")
+    ap2.add_argument("--reload-sig", metavar="S", type=u, default=("" if ANYWIN else "USR1"), help="reload server config when unix-signal \033[33mS\033[0m is received; examples: [\033[32mSIGUSR1\033[0m], [\033[32mUSR1\033[0m], [\033[32m10\033[0m]")
+    ap2.add_argument("--vc-url", metavar="URL", type=u, default="", help="URL to check for vulnerable versions (default-disabled)")
+    ap2.add_argument("--vc-age", metavar="HOURS", type=int, default=3, help="how many hours to wait between vulnerability checks")
+    ap2.add_argument("--vc-sev", metavar="LEVEL", type=u, default="low", help="minimum severity to care about; one of these: \033[32mlow medium high critical\033[0m")
+    ap2.add_argument("--vc-exit", action="store_true", help="panic and exit if current version is vulnerable")
    ap2.add_argument("--license", action="store_true", help="show licenses and exit")
    ap2.add_argument("--version", action="store_true", help="show versions and exit")
+    ap2.add_argument("--versionb", action="store_true", help="show version and exit")


 def add_qr(ap, tty):
@@ -1199,6 +1276,7 @@ def add_fs(ap):
    ap2 = ap.add_argument_group("filesystem options")
    rm_re_def = "15/0.1" if ANYWIN else "0/0"
    ap2.add_argument("--casechk", metavar="N", type=u, default="auto", help="detect and prevent CI (case-insensitive) behavior if the underlying filesystem is CI? [\033[32my\033[0m] = detect and prevent, [\033[32mn\033[0m] = ignore and allow, [\033[32mauto\033[0m] = \033[32my\033[0m if CI fs detected. NOTE: \033[32my\033[0m is very slow but necessary for correct WebDAV behavior on Windows/Macos (volflag=casechk)")
+    ap2.add_argument("--fsnt", metavar="OS", type=u, default="auto", help="which characters to allow in file/folder names; [\033[32mwin\033[0m] = windows (not <>:|?*\"\\/), [\033[32mmac\033[0m] = macos (not :), [\033[32mlin\033[0m] = linux (anything goes) (volflag=fsnt)")
    ap2.add_argument("--rm-retry", metavar="T/R", type=u, default=rm_re_def, help="if a file cannot be deleted because it is busy, continue trying for \033[33mT\033[0m seconds, retry every \033[33mR\033[0m seconds; disable with 0/0 (volflag=rm_retry)")
    ap2.add_argument("--mv-retry", metavar="T/R", type=u, default=rm_re_def, help="if a file cannot be renamed because it is busy, continue trying for \033[33mT\033[0m seconds, retry every \033[33mR\033[0m seconds; disable with 0/0 (volflag=mv_retry)")
    ap2.add_argument("--iobuf", metavar="BYTES", type=int, default=256*1024, help="file I/O buffer-size; if your volumes are on a network drive, try increasing to \033[32m524288\033[0m or even \033[32m4194304\033[0m (and let me know if that improves your performance)")
@@ -1213,6 +1291,7 @@ def add_share(ap):
    ap2.add_argument("--shr-who", metavar="TXT", type=u, default="auth", help="who can create a share? [\033[32mno\033[0m]=nobody, [\033[32ma\033[0m]=admin-permission, [\033[32mauth\033[0m]=authenticated (volflag=shr_who)")
    ap2.add_argument("--shr-adm", metavar="U,U", type=u, default="", help="comma-separated list of users allowed to view/delete any share")
    ap2.add_argument("--shr-rt", metavar="MIN", type=int, default=1440, help="shares can be revived by their owner if they expired less than MIN minutes ago; [\033[32m60\033[0m]=hour, [\033[32m1440\033[0m]=day, [\033[32m10080\033[0m]=week")
+    ap2.add_argument("--shr-site", metavar="URL", type=u, default="--site", help="public URL to assume when creating share-links; example: [\033[32mhttps://example.com/\033[0m]")
    ap2.add_argument("--shr-v", action="store_true", help="debug")


@@ -1220,18 +1299,20 @@ def add_upload(ap):
    ap2 = ap.add_argument_group("upload options")
    ap2.add_argument("--dotpart", action="store_true", help="dotfile incomplete uploads, hiding them from clients unless \033[33m-ed\033[0m")
    ap2.add_argument("--plain-ip", action="store_true", help="when avoiding filename collisions by appending the uploader's ip to the filename: append the plaintext ip instead of salting and hashing the ip")
+    ap2.add_argument("--up-site", metavar="URL", type=u, default="--site", help="public URL to assume when creating links to uploaded files; example: [\033[32mhttps://example.com/\033[0m]")
    ap2.add_argument("--put-name", metavar="TXT", type=u, default="put-{now.6f}-{cip}.bin", help="filename for nameless uploads (when uploader doesn't provide a name); default is [\033[32mput-UNIXTIME-IP.bin\033[0m] (the \033[32m.6f\033[0m means six decimal places) (volflag=put_name)")
    ap2.add_argument("--put-ck", metavar="ALG", type=u, default="sha512", help="default checksum-hasher for PUT/WebDAV uploads: no / md5 / sha1 / sha256 / sha512 / b2 / blake2 / b2s / blake2s (volflag=put_ck)")
    ap2.add_argument("--bup-ck", metavar="ALG", type=u, default="sha512", help="default checksum-hasher for bup/basic-uploader: no / md5 / sha1 / sha256 / sha512 / b2 / blake2 / b2s / blake2s (volflag=bup_ck)")
    ap2.add_argument("--unpost", metavar="SEC", type=int, default=3600*12, help="grace period where uploads can be deleted by the uploader, even without delete permissions; 0=disabled, default=12h")
    ap2.add_argument("--unp-who", metavar="NUM", type=int, default=1, help="clients can undo recent uploads by using the unpost tab (requires \033[33m-e2d\033[0m). [\033[32m0\033[0m] = never allowed (disable feature), [\033[32m1\033[0m] = allow if client has the same IP as the upload AND is using the same account, [\033[32m2\033[0m] = just check the IP, [\033[32m3\033[0m] = just check account-name (volflag=unp_who)")
+    ap2.add_argument("--apnd-who", metavar="NUM", type=u, default="dw", help="who can append to existing files? [\033[32mno\033[0m]=nobody, [\033[32maw\033[0m]=admin+write, [\033[32mdw\033[0m]=delete+write, [\033[32mw\033[0m]=write (volflag=apnd_who)")
    ap2.add_argument("--u2abort", metavar="NUM", type=int, default=1, help="clients can abort incomplete uploads by using the unpost tab (requires \033[33m-e2d\033[0m). [\033[32m0\033[0m] = never allowed (disable feature), [\033[32m1\033[0m] = allow if client has the same IP as the upload AND is using the same account, [\033[32m2\033[0m] = just check the IP, [\033[32m3\033[0m] = just check account-name (volflag=u2abort)")
    ap2.add_argument("--blank-wt", metavar="SEC", type=int, default=300, help="file write grace period (any client can write to a blank file last-modified more recently than \033[33mSEC\033[0m seconds ago)")
    ap2.add_argument("--reg-cap", metavar="N", type=int, default=38400, help="max number of uploads to keep in memory when running without \033[33m-e2d\033[0m; roughly 1 MiB RAM per 600")
    ap2.add_argument("--no-fpool", action="store_true", help="disable file-handle pooling -- instead, repeatedly close and reopen files during upload (bad idea to enable this on windows and/or cow filesystems)")
    ap2.add_argument("--use-fpool", action="store_true", help="force file-handle pooling, even when it might be dangerous (multiprocessing, filesystems lacking sparse-files support, ...)")
    ap2.add_argument("--chmod-f", metavar="UGO", type=u, default="", help="unix file permissions to use when creating files; default is probably 644 (OS-decided), see --help-chmod. Examples: [\033[32m644\033[0m] = owner-RW + all-R, [\033[32m755\033[0m] = owner-RWX + all-RX, [\033[32m777\033[0m] = full-yolo (volflag=chmod_f)")
-    ap2.add_argument("--chmod-d", metavar="UGO", type=u, default="755", help="unix file permissions to use when creating directories; see --help-chmod. Examples: [\033[32m755\033[0m] = owner-RW + all-R, [\033[32m777\033[0m] = full-yolo (volflag=chmod_d)")
+    ap2.add_argument("--chmod-d", metavar="UGO", type=u, default="755", help="unix file permissions to use when creating directories; see --help-chmod. Examples: [\033[32m755\033[0m] = owner-RW + all-R, [\033[32m2750\033[0m] = setgid + owner-RW + group-R, [\033[32m777\033[0m] = full-yolo (volflag=chmod_d)")
    ap2.add_argument("--uid", metavar="N", type=int, default=-1, help="unix user-id to chown new files/folders to; default = -1 = do-not-change (volflag=uid)")
    ap2.add_argument("--gid", metavar="N", type=int, default=-1, help="unix group-id to chown new files/folders to; default = -1 = do-not-change (volflag=gid)")
    ap2.add_argument("--wram", action="store_true", help="allow uploading even if a volume is inside a ramdisk, meaning that all data will be lost on the next server reboot (volflag=wram)")
@@ -1241,6 +1322,7 @@ def add_upload(ap):
    ap2.add_argument("--hardlink-only", action="store_true", help="do not fallback to symlinks when a hardlink cannot be made (volflag=hardlinkonly)")
    ap2.add_argument("--reflink", action="store_true", help="enable reflink-based dedup; will fallback on full copies when that is impossible (non-CoW filesystem) (volflag=reflink)")
    ap2.add_argument("--no-dupe", action="store_true", help="reject duplicate files during upload; only matches within the same volume (volflag=nodupe)")
+    ap2.add_argument("--no-dupe-m", action="store_true", help="also reject dupes when moving a file into another volume (volflag=nodupem)")
    ap2.add_argument("--no-clone", action="store_true", help="do not use existing data on disk to satisfy dupe uploads; reduces server HDD reads in exchange for much more network load (volflag=noclone)")
    ap2.add_argument("--no-snap", action="store_true", help="disable snapshots -- forget unfinished uploads on shutdown; don't create .hist/up2k.snap files -- abandoned/interrupted uploads must be cleaned up manually")
    ap2.add_argument("--snap-wri", metavar="SEC", type=int, default=300, help="write upload state to ./hist/up2k.snap every \033[33mSEC\033[0m seconds; allows resuming incomplete uploads after a server crash")
@@ -1252,7 +1334,7 @@ def add_upload(ap):
    ap2.add_argument("--nrand", metavar="NUM", type=int, default=9, help="randomized filenames length (volflag=nrand)")
    ap2.add_argument("--magic", action="store_true", help="enable filetype detection on nameless uploads (volflag=magic)")
    ap2.add_argument("--df", metavar="GiB", type=u, default="0", help="ensure \033[33mGiB\033[0m free disk space by rejecting upload requests; assumes gigabytes unless a unit suffix is given: [\033[32m256m\033[0m], [\033[32m4\033[0m], [\033[32m2T\033[0m] (volflag=df)")
-    ap2.add_argument("--sparse", metavar="MiB", type=int, default=4, help="windows-only: minimum size of incoming uploads through up2k before they are made into sparse files")
+    ap2.add_argument("--sparse", metavar="MiB", type=int, default=1, help="windows-only: minimum size of incoming uploads through up2k before they are made into sparse files")
    ap2.add_argument("--turbo", metavar="LVL", type=int, default=0, help="configure turbo-mode in up2k client; [\033[32m-1\033[0m] = forbidden/always-off, [\033[32m0\033[0m] = default-off and warn if enabled, [\033[32m1\033[0m] = default-off, [\033[32m2\033[0m] = on, [\033[32m3\033[0m] = on and disable datecheck")
    ap2.add_argument("--nosubtle", metavar="N", type=int, default=0, help="when to use a wasm-hasher instead of the browser's builtin; faster on chrome, but buggy in older chrome versions. [\033[32m0\033[0m] = only when necessary (non-https), [\033[32m1\033[0m] = always (all browsers), [\033[32m2\033[0m] = always on chrome/firefox, [\033[32m3\033[0m] = always on chrome, [\033[32mN\033[0m] = chrome-version N and newer (recommendation: 137)")
    ap2.add_argument("--u2j", metavar="JOBS", type=int, default=2, help="web-client: number of file chunks to upload in parallel; 1 or 2 is good when latency is low (same-country), 2~4 for android-clients, 2~6 for cross-atlantic. Max is 6 in most browsers. Big values increase network-speed but may reduce HDD-speed")
@@ -1269,13 +1351,19 @@ def add_network(ap):
    ap2.add_argument("--ll", action="store_true", help="include link-local IPv4/IPv6 in mDNS replies, even if the NIC has routable IPs (breaks some mDNS clients)")
    ap2.add_argument("--rproxy", metavar="DEPTH", type=int, default=9999999, help="which ip to associate clients with; [\033[32m0\033[0m]=tcp, [\033[32m1\033[0m]=origin (first x-fwd, unsafe), [\033[32m-1\033[0m]=closest-proxy, [\033[32m-2\033[0m]=second-hop, [\033[32m-3\033[0m]=third-hop")
    ap2.add_argument("--xff-hdr", metavar="NAME", type=u, default="x-forwarded-for", help="if reverse-proxied, which http header to read the client's real ip from")
+    ap2.add_argument("--xf-host", metavar="NAME", type=u, default="x-forwarded-host", help="if reverse-proxied, which http header to read the correct Host value from; this header must contain the server's external domain name")
+    ap2.add_argument("--xf-proto", metavar="NAME", type=u, default="x-forwarded-proto", help="if reverse-proxied, which http header to read the correct protocol value from; this header must contain either 'http' or 'https'")
+    ap2.add_argument("--xf-proto-fb", metavar="T", type=u, default="", help="protocol to assume if the X-Forwarded-Proto header (\033[33m--xf-proto\033[0m) is not provided by the reverseproxy; either 'http' or 'https'")
    ap2.add_argument("--xff-src", metavar="CIDR", type=u, default="127.0.0.0/8, ::1/128", help="list of trusted reverse-proxy CIDRs (comma-separated); only accept the real-ip header (\033[33m--xff-hdr\033[0m) and IdP headers if the incoming connection is from an IP within either of these subnets. Specify [\033[32mlan\033[0m] to allow all LAN / private / non-internet IPs. Can be disabled with [\033[32many\033[0m] if you are behind cloudflare (or similar) and are using \033[32m--xff-hdr=cf-connecting-ip\033[0m (or similar)")
-    ap2.add_argument("--ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m (comma-separated); examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
+    ap2.add_argument("--ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m (comma-separated); examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]\n └─for performance and security, this only looks at the TCP/Network-level IP, and will NOT work behind a reverseproxy")
+    ap2.add_argument("--ipar", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m (comma-separated).\n └─this is reverseproxy-compatible; reads client-IP from 'X-Forwarded-For' if possible, with TCP/Network IP as fallback")
    ap2.add_argument("--rp-loc", metavar="PATH", type=u, default="", help="if reverse-proxying on a location instead of a dedicated domain/subdomain, provide the base location here; example: [\033[32m/foo/bar\033[0m]")
+    ap2.add_argument("--cachectl", metavar="TXT", default="no-cache", help="default-value of the 'Cache-Control' response-header (controls caching in webbrowsers). Default prevents repeated downloading of the same file unless necessary (browser will ask copyparty if the file has changed). Examples: [\033[32mmax-age=604869\033[0m] will cache for 7 days, [\033[32mno-store, max-age=0\033[0m] will always redownload. (volflag=cachectl)")
+    ap2.add_argument("--http-vary", metavar="TXT", type=u, default="Origin, PW, Cookie", help="value of the 'Vary' response-header; a hint for caching proxies")
    ap2.add_argument("--http-no-tcp", action="store_true", help="do not listen on TCP/IP for http/https; only listen on unix-domain-sockets")
-    if ANYWIN:
-        ap2.add_argument("--reuseaddr", action="store_true", help="set reuseaddr on listening sockets on windows; allows rapid restart of copyparty at the expense of being able to accidentally start multiple instances")
-    elif not MACOS:
+    if FULL_HELP or ANYWIN:
+        ap2.add_argument("--reuseaddr", action="store_true", help="set reuseaddr on listening sockets (windows-only); allows rapid restart of copyparty at the expense of being able to accidentally start multiple instances")
+    if FULL_HELP or (not ANYWIN and not MACOS):
        ap2.add_argument("--freebind", action="store_true", help="allow listening on IPs which do not yet exist, for example if the network interfaces haven't finished going up. Only makes sense for IPs other than '0.0.0.0', '127.0.0.1', '::', and '::1'. May require running as root (unless net.ipv6.ip_nonlocal_bind)")
    ap2.add_argument("--wr-h-eps", metavar="PATH", type=u, default="", help="write list of listening-on ip:port to textfile at \033[33mPATH\033[0m when http-servers have started")
    ap2.add_argument("--wr-h-aon", metavar="PATH", type=u, default="", help="write list of accessible-on ip:port to textfile at \033[33mPATH\033[0m when http-servers have started")
@@ -1286,13 +1374,16 @@ def add_network(ap):
    ap2.add_argument("--s-wr-slp", metavar="SEC", type=float, default=0.0, help="debug: socket write delay in seconds")
    ap2.add_argument("--rsp-slp", metavar="SEC", type=float, default=0.0, help="debug: response delay in seconds")
    ap2.add_argument("--rsp-jtr", metavar="SEC", type=float, default=0.0, help="debug: response delay, random duration 0..\033[33mSEC\033[0m")
+    ap2.add_argument("--list-nics", action="store_true", help="debug: list detected network adapters")
+    ap2.add_argument("--list-ips", action="store_true", help="debug: list detected LAN IPs")


 def add_tls(ap, cert_path):
    ap2 = ap.add_argument_group("SSL/TLS options")
    ap2.add_argument("--http-only", action="store_true", help="disable ssl/tls -- force plaintext")
    ap2.add_argument("--https-only", action="store_true", help="disable plaintext -- force tls")
-    ap2.add_argument("--cert", metavar="PATH", type=u, default=cert_path, help="path to file containing a concatenation of TLS key and certificate chain")
+    ap2.add_argument("--cert", metavar="PATH", type=u, default=cert_path, help="path to file containing a concatenation of TLS key and certificate chain (if \033[33m--certkey\033[0m is not set), or just the certificate chain (if \033[33m--certkey\033[0m is set)")
+    ap2.add_argument("--certkey", metavar="PATH", type=u, default="", help="path to file containing just the certificate key; if this is set, then \033[33m--cert\033[0m should only contain the certificate chain")
    ap2.add_argument("--ssl-ver", metavar="LIST", type=u, default="", help="set allowed ssl/tls versions; [\033[32mhelp\033[0m] shows available versions; default is what your python version considers safe")
    ap2.add_argument("--ciphers", metavar="LIST", type=u, default="", help="set allowed ssl/tls ciphers; [\033[32mhelp\033[0m] shows available ciphers")
    ap2.add_argument("--ssl-dbg", action="store_true", help="dump some tls info")
@@ -1327,6 +1418,7 @@ def add_auth(ap):
    ap2.add_argument("--idp-h-grp", metavar="HN", type=u, default="", help="assume the request-header \033[33mHN\033[0m contains the groupname of the requesting user; can be referenced in config files for group-based access control")
    ap2.add_argument("--idp-h-key", metavar="HN", type=u, default="", help="optional but recommended safeguard; your reverse-proxy will insert a secret header named \033[33mHN\033[0m into all requests, and the other IdP headers will be ignored if this header is not present")
    ap2.add_argument("--idp-gsep", metavar="RE", type=u, default="|:;+,", help="if there are multiple groups in \033[33m--idp-h-grp\033[0m, they are separated by one of the characters in \033[33mRE\033[0m")
+    ap2.add_argument("--idp-chsub", metavar="TXT", type=u, default="", help="characters to replace in usernames/groupnames; a list of pairs of characters separated by | so for example | _| will replace spaces with _ to make configuration easier, or |%%_|^_|@_| will replace %%/^/@ with _")
    ap2.add_argument("--idp-db", metavar="PATH", type=u, default=idp_db, help="where to store the known IdP users/groups (if you run multiple copyparty instances, make sure they use different DBs)")
    ap2.add_argument("--idp-store", metavar="N", type=int, default=1, help="how to use \033[33m--idp-db\033[0m; [\033[32m0\033[0m] = entirely disable, [\033[32m1\033[0m] = write-only (effectively disabled), [\033[32m2\033[0m] = remember users, [\033[32m3\033[0m] = remember users and groups.\nNOTE: Will remember and restore the IdP-volumes of all users for all eternity if set to 2 or 3, even when user is deleted from your IdP")
    ap2.add_argument("--idp-adm", metavar="U,U", type=u, default="", help="comma-separated list of users allowed to use /?idp (the cache management UI)")
@@ -1335,10 +1427,12 @@ def add_auth(ap):
    ap2.add_argument("--idp-login-t", metavar="T", type=u, default="Login with SSO", help="the label/text for the idp-login button")
    ap2.add_argument("--idp-logout", metavar="L", type=u, default="", help="replace all logout-buttons with a link to URL \033[33mL\033[0m")
    ap2.add_argument("--auth-ord", metavar="TXT", type=u, default="idp,ipu", help="controls auth precedence; examples: [\033[32mpw,idp,ipu\033[0m], [\033[32mipu,pw,idp\033[0m], see --help-auth-ord")
+    ap2.add_argument("--pw-hdr", metavar="NAME", type=u, default="pw", help="lowercase name of password-header (NAME: foo); \033[1;31mWARNING:\033[0m Changing this will break support for many clients")
+    ap2.add_argument("--pw-urlp", metavar="NAME", type=u, default="pw", help="lowercase name of password url-param (?NAME=foo); \033[1;31mWARNING:\033[0m Changing this will break support for many clients")
    ap2.add_argument("--no-bauth", action="store_true", help="disable basic-authentication support; do not accept passwords from the 'Authenticate' header at all. NOTE: This breaks support for the android app")
    ap2.add_argument("--bauth-last", action="store_true", help="keeps basic-authentication enabled, but only as a last-resort; if a cookie is also provided then the cookie wins")
    ap2.add_argument("--ses-db", metavar="PATH", type=u, default=ses_db, help="where to store the sessions database (if you run multiple copyparty instances, make sure they use different DBs)")
-    ap2.add_argument("--ses-len", metavar="CHARS", type=int, default=20, help="session key length; default is 120 bits ((20//4)*4*6)")
+    ap2.add_argument("--ses-len", metavar="CHARS", type=int, default=24, help="session key length; default is 144 bits ((24//4)*4*6)")
    ap2.add_argument("--no-ses", action="store_true", help="disable sessions; use plaintext passwords in cookies")
    ap2.add_argument("--grp-all", metavar="NAME", type=u, default="acct", help="the name of the auto-generated group which contains every username which is known")
    ap2.add_argument("--ipu", metavar="CIDR=USR", type=u, action="append", help="\033[34mREPEATABLE:\033[0m users with IP matching \033[33mCIDR\033[0m are auto-authenticated as username \033[33mUSR\033[0m; example: [\033[32m172.16.24.0/24=dave]")
@@ -1406,60 +1500,99 @@ def add_zc_ssdp(ap):
    ap2.add_argument("--zsid", metavar="UUID", type=u, default=zsid, help="USN (device identifier) to announce")


+def add_sftp(ap):
+    ap2 = ap.add_argument_group("SFTP options")
+    ap2.add_argument("--sftp", metavar="PORT", type=int, default=0, help="enable SFTP server on \033[33mPORT\033[0m, for example \033[32m3922")
+    ap2.add_argument("--sftpv", action="store_true", help="verbose")
+    ap2.add_argument("--sftpvv", action="store_true", help="verboser")
+    ap2.add_argument("--sftp-i", metavar="IP", type=u, default="-i", help="IPs to listen on (comma-separated list). Set this to override \033[33m-i\033[0m for this protocol")
+    ap2.add_argument("--sftp4", action="store_true", help="only listen on IPv4")
+    ap2.add_argument("--sftp-key", metavar="U K", type=u, action="append", help="\033[34mREPEATABLE:\033[0m add ssh-key \033[33mK\033[0m for user \033[33mU\033[0m (username, space, key-type, space, base64); if user has multiple keys, then repeat this option for each key\n └─commandline example: --sftp-key 'david ssh-ed25519 AAAAC3NzaC...'\n └─config-file example: sftp-key: david ssh-ed25519 AAAAC3NzaC...")
+    ap2.add_argument("--sftp-key2u", action="append", help=argparse.SUPPRESS)
+    ap2.add_argument("--sftp-pw", action="store_true", help="allow password-authentication with sftp (not just ssh-keys)")
+    ap2.add_argument("--sftp-anon", metavar="TXT", type=u, default="", help="allow anonymous/unauthenticated connections with \033[33mTXT\033[0m as username")
+    ap2.add_argument("--sftp-hostk", metavar="FP", type=u, default=E.cfg, help="path to folder with hostkeys, for example 'ssh_host_rsa_key'; missing keys will be generated")
+    ap2.add_argument("--sftp-banner", metavar="T", type=u, default="", help="bannertext to send when someone connects; can be @filepath")
+    ap2.add_argument("--sftp-ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m (comma-separated); specify [\033[32many\033[0m] to disable inheriting \033[33m--ipa\033[0m / \033[33m--ipar\033[0m. Examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
+    ap2.add_argument("--sftp-hs-t", metavar="SEC", type=int, default=15, help="connection handshake timeout in seconds")
+    ap2.add_argument("--sftp-hs-n", metavar="NUM", type=int, default=4, help="max num ongoing/incomplete handshakes")
+
+
 def add_ftp(ap):
    ap2 = ap.add_argument_group("FTP options (TCP only)")
    ap2.add_argument("--ftp", metavar="PORT", type=int, default=0, help="enable FTP server on \033[33mPORT\033[0m, for example \033[32m3921")
    ap2.add_argument("--ftps", metavar="PORT", type=int, default=0, help="enable FTPS server on \033[33mPORT\033[0m, for example \033[32m3990")
    ap2.add_argument("--ftpv", action="store_true", help="verbose")
+    ap2.add_argument("--ftp-i", metavar="IP", type=u, default="-i", help="IPs to listen on (comma-separated list). Set this to override \033[33m-i\033[0m for this protocol")
    ap2.add_argument("--ftp4", action="store_true", help="only listen on IPv4")
-    ap2.add_argument("--ftp-ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m (comma-separated); specify [\033[32many\033[0m] to disable inheriting \033[33m--ipa\033[0m. Examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
+    ap2.add_argument("--ftp-ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m (comma-separated); specify [\033[32many\033[0m] to disable inheriting \033[33m--ipa\033[0m / \033[33m--ipar\033[0m. Examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
    ap2.add_argument("--ftp-no-ow", action="store_true", help="if target file exists, reject upload instead of overwrite")
    ap2.add_argument("--ftp-wt", metavar="SEC", type=int, default=7, help="grace period for resuming interrupted uploads (any client can write to any file last-modified more recently than \033[33mSEC\033[0m seconds ago)")
    ap2.add_argument("--ftp-nat", metavar="ADDR", type=u, default="", help="the NAT address to use for passive connections")
    ap2.add_argument("--ftp-pr", metavar="P-P", type=u, default="", help="the range of TCP ports to use for passive connections, for example \033[32m12000-13000")
+    ap2.add_argument("--ftp-banner", metavar="T", type=u, default="welcome to the party", help="bannertext to send when someone connects; \\n is newline, can be @filepath (loaded into memory on startup); if it has a newline then banner must be longer than 75 chars")


 def add_webdav(ap):
    ap2 = ap.add_argument_group("WebDAV options")
-    ap2.add_argument("--daw", action="store_true", help="enable full write support, even if client may not be webdav. \033[1;31mWARNING:\033[0m This has side-effects -- PUT-operations will now \033[1;31mOVERWRITE\033[0m existing files, rather than inventing new filenames to avoid loss of data. You might want to instead set this as a volflag where needed. By not setting this flag, uploaded files can get written to a filename which the client does not expect (which might be okay, depending on client)")
+    ap2.add_argument("--daw", action="store_true", help="enable full write support, even if client may not be webdav. Some webdav clients need this option for editing existing files; not necessary for clients that send the 'x-oc-mtime' header. Regardless, the delete-permission must always be given. \033[1;31mWARNING:\033[0m This has side-effects -- PUT-operations will now \033[1;31mOVERWRITE\033[0m existing files, rather than inventing new filenames to avoid loss of data. You might want to instead set this as a volflag where needed. By not setting this flag, uploaded files can get written to a filename which the client does not expect (which might be okay, depending on client)")
    ap2.add_argument("--dav-inf", action="store_true", help="allow depth:infinite requests (recursive file listing); extremely server-heavy but required for spec compliance -- luckily few clients rely on this")
    ap2.add_argument("--dav-mac", action="store_true", help="disable apple-garbage filter -- allow macos to create junk files (._* and .DS_Store, .Spotlight-*, .fseventsd, .Trashes, .AppleDouble, __MACOS)")
    ap2.add_argument("--dav-rt", action="store_true", help="show symlink-destination's lastmodified instead of the link itself; always enabled for recursive listings (volflag=davrt)")
    ap2.add_argument("--dav-auth", action="store_true", help="force auth for all folders (required by davfs2 when only some folders are world-readable) (volflag=davauth)")
-    ap2.add_argument("--dav-ua1", metavar="PTN", type=u, default=r" kioworker/", help="regex of tricky user-agents which expect 401 from GET requests; disable with [\033[32mno\033[0m] or blank")
+    ap2.add_argument("--dav-ua1", metavar="PTN", type=u, default=r" kioworker/", help="regex of user-agents which ARE webdav-clients, and expect 401 from GET requests; disable with [\033[32mno\033[0m] or blank")
+    ap2.add_argument("--dav-port", metavar="P", type=int, default=0, help="additional port to listen on for misbehaving webdav clients which pretend they are graphical browsers; an alternative/supplement to dav-ua1")
+    ap2.add_argument("--ua-nodav", metavar="PTN", type=u, default=r"^(Mozilla/|NetworkingExtension/|com\.apple\.WebKit)", help="regex of user-agents which are NOT webdav-clients")
+    ap2.add_argument("--p-nodav", metavar="P,P", type=u, default="", help="server-ports (comma-sep.) which are NOT webdav-clients; an alternative/supplement to ua-nodav")


 def add_tftp(ap):
    ap2 = ap.add_argument_group("TFTP options (UDP only)")
    ap2.add_argument("--tftp", metavar="PORT", type=int, default=0, help="enable TFTP server on \033[33mPORT\033[0m, for example \033[32m69 \033[0mor \033[32m3969")
+    ap2.add_argument("--tftp-i", metavar="IP", type=u, default="-i", help="IPs to listen on (comma-separated list). Set this to override \033[33m-i\033[0m for this protocol")
    ap2.add_argument("--tftp4", action="store_true", help="only listen on IPv4")
    ap2.add_argument("--tftpv", action="store_true", help="verbose")
    ap2.add_argument("--tftpvv", action="store_true", help="verboser")
    ap2.add_argument("--tftp-no-fast", action="store_true", help="debug: disable optimizations")
    ap2.add_argument("--tftp-lsf", metavar="PTN", type=u, default="\\.?(dir|ls)(\\.txt)?", help="return a directory listing if a file with this name is requested and it does not exist; defaults matches .ls, dir, .dir.txt, ls.txt, ...")
    ap2.add_argument("--tftp-nols", action="store_true", help="if someone tries to download a directory, return an error instead of showing its directory listing")
-    ap2.add_argument("--tftp-ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m (comma-separated); specify [\033[32many\033[0m] to disable inheriting \033[33m--ipa\033[0m. Examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
+    ap2.add_argument("--tftp-ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m (comma-separated); specify [\033[32many\033[0m] to disable inheriting \033[33m--ipa\033[0m / \033[33m--ipar\033[0m. Examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
    ap2.add_argument("--tftp-pr", metavar="P-P", type=u, default="", help="the range of UDP ports to use for data transfer, for example \033[32m12000-13000")


 def add_smb(ap):
    ap2 = ap.add_argument_group("SMB/CIFS options")
    ap2.add_argument("--smb", action="store_true", help="enable smb (read-only) -- this requires running copyparty as root on linux and macos unless \033[33m--smb-port\033[0m is set above 1024 and your OS does port-forwarding from 445 to that.\n\033[1;31mWARNING:\033[0m this protocol is DANGEROUS and buggy! Never expose to the internet!")
+    ap2.add_argument("--smb-i", metavar="IP", type=u, default="-i", help="IPs to listen on (comma-separated list). Set this to override \033[33m-i\033[0m for this protocol")
    ap2.add_argument("--smbw", action="store_true", help="enable write support (please dont)")
    ap2.add_argument("--smb1", action="store_true", help="disable SMBv2, only enable SMBv1 (CIFS)")
    ap2.add_argument("--smb-port", metavar="PORT", type=int, default=445, help="port to listen on -- if you change this value, you must NAT from TCP:445 to this port using iptables or similar")
    ap2.add_argument("--smb-nwa-1", action="store_true", help="truncate directory listings to 64kB (~400 files); avoids impacket-0.11 bug, fixes impacket-0.12 performance")
    ap2.add_argument("--smb-nwa-2", action="store_true", help="disable impacket workaround for filecopy globs")
    ap2.add_argument("--smba", action="store_true", help="small performance boost: disable per-account permissions, enables account coalescing instead (if one user has write/delete-access, then everyone does)")
+    ap2.add_argument("--smb6", action="store_true", help="enable IPv6")
    ap2.add_argument("--smbv", action="store_true", help="verbose")
    ap2.add_argument("--smbvv", action="store_true", help="verboser")
    ap2.add_argument("--smbvvv", action="store_true", help="verbosest")

+
 def add_opds(ap):
    ap2 = ap.add_argument_group("OPDS options")
    ap2.add_argument("--opds", action="store_true", help="enable opds -- allows e-book readers to browse and download files (volflag=opds)")
    ap2.add_argument("--opds-exts", metavar="T,T", type=u, default="epub,cbz,pdf", help="file formats to list in OPDS feeds; leave empty to show everything (volflag=opds_exts)")

+
+def add_wopi(ap):
+    ap2 = ap.add_argument_group("WOPI options")
+    ap2.add_argument("--wopi", action="store_true", help="enable integration with office suites using WOPI")
+    ap2.add_argument("--wopi-api", metavar="URL", type=u, default="", help="URL that the WOPI-client should use to communicate with copyparty; default is same as user's webbrowser. Example: [\033[32mhttps://party.example.com/\033[0m]")
+    ap2.add_argument("--wopi-url", metavar="URL", type=u, default="", help="URL to your WOPI client; the host of e.g. Collabora Online. Example: [\033[32mhttps://code.example.com/\033[0m]")
+    ap2.add_argument("--wopi-crt", metavar="TXT", type=u, default="", help="if \033[33m--wopi-url\033[0m is selfsigned: path to ca.pem or cert.pem to expect/verify (can be [\033[32mno\033[0m] for full-yolo)")
+    ap2.add_argument("--wopi-crt-icn", action="store_true", help="if \033[33m--wopi-url\033[0m is selfsigned: ignore the CN (server ip/name) in cert")
+    ap2.add_argument("--wopi-ttl", metavar="SEC", type=int, default=1800, help="session lifetime; allow editing for this many seconds (default is 30 min)")
+    ap2.add_argument("--wopi-wdel", action="store_true", help="require permissions read+write+delete for writing to file; default is just read+write")
+
+
 def add_handlers(ap):
    ap2 = ap.add_argument_group("handlers (see --help-handlers)")
    ap2.add_argument("--on404", metavar="PY", type=u, action="append", help="\033[34mREPEATABLE:\033[0m handle 404s by executing \033[33mPY\033[0m file")
@@ -1486,6 +1619,7 @@ def add_hooks(ap):
 def add_stats(ap):
    ap2 = ap.add_argument_group("grafana/prometheus metrics endpoint")
    ap2.add_argument("--stats", action="store_true", help="enable openmetrics at /.cpr/metrics for admin accounts")
+    ap2.add_argument("--stats-u", metavar="U,U", type=u, default="", help="comma-separated list of users allowed to access /.cpr/metrics even if they aren't admin")
    ap2.add_argument("--nos-hdd", action="store_true", help="disable disk-space metrics (used/free space)")
    ap2.add_argument("--nos-vol", action="store_true", help="disable volume size metrics (num files, total bytes, vmaxb/vmaxn)")
    ap2.add_argument("--nos-vst", action="store_true", help="disable volume state metrics (indexing, analyzing, activity)")
@@ -1495,10 +1629,10 @@ def add_stats(ap):

 def add_yolo(ap):
    ap2 = ap.add_argument_group("yolo options")
-    ap2.add_argument("--allow-csrf", action="store_true", help="disable csrf protections; let other domains/sites impersonate you through cross-site requests")
+    ap2.add_argument("--allow-csrf", action="store_true", help="disable csrf protections; let other domains/sites impersonate you through cross-site requests; \033[1;31mDANGEROUS\033[0m / LAN-only")
    ap2.add_argument("--cookie-lax", action="store_true", help="allow cookies from other domains (if you follow a link from another website into your server, you will arrive logged-in); this reduces protection against CSRF")
    ap2.add_argument("--no-fnugg", action="store_true", help="disable the smoketest for caching-related issues in the web-UI")
-    ap2.add_argument("--getmod", action="store_true", help="permit ?move=[...] and ?delete as GET")
+    ap2.add_argument("--getmod", action="store_true", help="permit ?move=[...] and ?delete as GET -- \033[1;31mDANGEROUS\033[0m, removes csrf protection")
    ap2.add_argument("--wo-up-readme", action="store_true", help="allow users with write-only access to upload logues and readmes without adding the _wo_ filename prefix (volflag=wo_up_readme)")
    ap2.add_argument("--unsafe-state", action="store_true", help="when one of the emergency fallback locations are used for runtime state ($TMPDIR, /tmp), certain features will be force-disabled for security reasons by default. This option overrides that safeguard and allows unsafe storage of secrets")

@@ -1512,10 +1646,10 @@ def add_optouts(ap):
    ap2.add_argument("--no-mv", action="store_true", help="disable move/rename operations")
    ap2.add_argument("--no-cp", action="store_true", help="disable copy operations")
    ap2.add_argument("--no-fs-abrt", action="store_true", help="disable ability to abort ongoing copy/move")
-    ap2.add_argument("-nth", action="store_true", help="no title hostname; don't show \033[33m--name\033[0m in <title>")
-    ap2.add_argument("-nih", action="store_true", help="no info hostname -- don't show in UI")
-    ap2.add_argument("-nid", action="store_true", help="no info disk-usage -- don't show in UI. This is the same as --du-who no")
+    ap2.add_argument("-nih", action="store_true", help="no info hostname -- removes it from the UI corner, but the value of \033[33m--bname\033[0m still shows in the browsertab title")
+    ap2.add_argument("-nid", action="store_true", help="no info disk-usage -- don't show in UI. This is the same as \033[33m--du-who no\033[0m")
    ap2.add_argument("-nb", action="store_true", help="no powered-by-copyparty branding in UI")
+    ap2.add_argument("--smsg", metavar="T,T", type=u, default="POST", help="HTTP-methods to allow ?smsg for; will execute xm hooks like urlform / message-to-serverlog; dangerous example: [\033[32mGET,POST\033[0m]. \033[1;31mWARNING:\033[0m The default (POST) is safe, but GET is dangerous; security/CSRF hazard")
    ap2.add_argument("--zipmaxn", metavar="N", type=u, default="0", help="reject download-as-zip if more than \033[33mN\033[0m files in total; optionally takes a unit suffix: [\033[32m256\033[0m], [\033[32m9K\033[0m], [\033[32m4G\033[0m] (volflag=zipmaxn)")
    ap2.add_argument("--zipmaxs", metavar="SZ", type=u, default="0", help="reject download-as-zip if total download size exceeds \033[33mSZ\033[0m bytes; optionally takes a unit suffix: [\033[32m256M\033[0m], [\033[32m4G\033[0m], [\033[32m2T\033[0m] (volflag=zipmaxs)")
    ap2.add_argument("--zipmaxt", metavar="TXT", type=u, default="", help="custom errormessage when download size exceeds max (volflag=zipmaxt)")
@@ -1528,24 +1662,56 @@ def add_optouts(ap):
    ap2.add_argument("--no-pipe", action="store_true", help="disable race-the-beam (lockstep download of files which are currently being uploaded) (volflag=nopipe)")
    ap2.add_argument("--no-tail", action="store_true", help="disable streaming a growing files with ?tail (volflag=notail)")
    ap2.add_argument("--no-db-ip", action="store_true", help="do not write uploader-IP into the database; will also disable unpost, you may want \033[32m--forget-ip\033[0m instead (volflag=no_db_ip)")
+    ap2.add_argument("--no-zls", action="store_true", help="disable browsing the contents of zip/cbz files, does not affect thumbnails")


 def add_safety(ap):
+    th_bwrap = ""
+    if HAVE_BWRAP:
+        zsl = [
+            HAVE_BWRAP,
+            "--proc /proc",
+            "--tmpfs /tmp",
+            "--tmpfs /var",
+            "--tmpfs /run",
+            "--dev-bind /dev/null /dev/null",
+            "--dev-bind /dev/random /dev/random",
+            "--dev-bind /dev/urandom /dev/urandom",
+            "--chdir /tmp",
+            "--clearenv",
+            "--unshare-all",
+            "--cap-drop ALL",
+            "--die-with-parent",
+            "--new-session",
+        ]
+        for d in ("/lib", "/lib64", "/usr/lib", "/usr/lib64"):
+            if os.path.isdir(d):
+                zsl.append(" --ro-bind %s %s" % (d, d))
+        th_bwrap = " ".join(zsl)
+
    ap2 = ap.add_argument_group("safety options")
    ap2.add_argument("-s", action="count", default=0, help="increase safety: Disable thumbnails / potentially dangerous software (ffmpeg/pillow/vips), hide partial uploads, avoid crawlers.\n └─Alias of\033[32m --dotpart --no-thumb --no-mtag-ff --no-robots --force-js")
-    ap2.add_argument("-ss", action="store_true", help="further increase safety: Prevent js-injection, accidental move/delete, broken symlinks, webdav requires login, 404 on 403, ban on excessive 404s.\n └─Alias of\033[32m -s --unpost=0 --no-del --no-mv --reflink --dav-auth --vague-403 -nih")
+    ap2.add_argument("-ss", action="store_true", help="further increase safety: Prevent js-injection, accidental move/delete, broken symlinks, webdav requires login, 404 on 403, ban on excessive 404s.\n └─Alias of\033[32m -s --no-html --no-readme --no-logues --unpost=0 --no-del --no-mv --reflink --dav-auth --vague-403 -nih")
    ap2.add_argument("-sss", action="store_true", help="further increase safety: Enable logging to disk, scan for dangerous symlinks.\n └─Alias of\033[32m -ss --no-dav --no-logues --no-readme -lo=cpp-%%Y-%%m%%d-%%H%%M%%S.txt.xz --ls=**,*,ln,p,r")
    ap2.add_argument("--ls", metavar="U[,V[,F]]", type=u, default="", help="do a sanity/safety check of all volumes on startup; arguments \033[33mUSER\033[0m,\033[33mVOL\033[0m,\033[33mFLAGS\033[0m (see \033[33m--help-ls\033[0m); example [\033[32m**,*,ln,p,r\033[0m]")
-    ap2.add_argument("--xvol", action="store_true", help="never follow symlinks leaving the volume root, unless the link is into another volume where the user has similar access (volflag=xvol)")
+    ap2.add_argument("--xvol", action="store_true", help="never follow symlinks leaving the volume root, unless the link is into another volume where the user also has access (keeps permissions from the outer/initial volume) (volflag=xvol)")
    ap2.add_argument("--xdev", action="store_true", help="stay within the filesystem of the volume root; do not descend into other devices (symlink or bind-mount to another HDD, ...) (volflag=xdev)")
+    ap2.add_argument("--vol-nospawn", action="store_true", help="if a volume's folder does not exist on the HDD, then do not create it (continue with warning) (volflag=nospawn)")
+    ap2.add_argument("--vol-or-crash", action="store_true", help="if a volume's folder does not exist on the HDD, then burst into flames (volflag=assert_root)")
    ap2.add_argument("--no-dot-mv", action="store_true", help="disallow moving dotfiles; makes it impossible to move folders containing dotfiles")
    ap2.add_argument("--no-dot-ren", action="store_true", help="disallow renaming dotfiles; makes it impossible to turn something into a dotfile")
-    ap2.add_argument("--no-logues", action="store_true", help="disable rendering .prologue/.epilogue.html into directory listings")
-    ap2.add_argument("--no-readme", action="store_true", help="disable rendering readme/preadme.md into directory listings")
-    ap2.add_argument("--vague-403", action="store_true", help="send 404 instead of 403 (security through ambiguity, very enterprise)")
+    ap2.add_argument("--no-logues", action="store_true", help="disable rendering .prologue/.epilogue.html into directory listings (volflag=no_logues)")
+    ap2.add_argument("--no-readme", action="store_true", help="disable rendering readme/preadme.md into directory listings (volflag=no_readme)")
+    ap2.add_argument("--csp-ui", metavar="TXT", default="script-src 'unsafe-eval' 'nonce-{{ js_nonce }}'; worker-src 'self'", help="content-security-policy to apply for the web-UI; default helps prevent XSS by blocking <script> / onclick / ... (volflag=csp_ui)")
+    ap2.add_argument("--csp-dl", metavar="TXT", default="", help="content-security-policy to apply for static files (volflag=csp_dl)")
+    ap2.add_argument("--no-script", action="store_true", help="disables javascript in html files; helps prevent XSS but kills interactive websites; this will override \033[33m--csp-dl\033[0m with [\033[32mscript-src 'none'\033[0m] (volflag=noscript)")
+    ap2.add_argument("--no-html", action="store_true", help="show html-files as plain text; helps prevent XSS but kills websites/blogs, also enables --no-script (volflag=nohtml)")
+    ap2.add_argument("--vague-403", action="store_true", help="send 404 instead of 403 (security through ambiguity, very enterprise). \033[1;31mWARNING:\033[0m Not compatible with WebDAV")
    ap2.add_argument("--force-js", action="store_true", help="don't send folder listings as HTML, force clients to use the embedded json instead -- slight protection against misbehaving search engines which ignore \033[33m--no-robots\033[0m")
    ap2.add_argument("--no-robots", action="store_true", help="adds http and html headers asking search engines to not index anything (volflag=norobots)")
    ap2.add_argument("--logout", metavar="H", type=float, default=8086.0, help="logout clients after \033[33mH\033[0m hours of inactivity; [\033[32m0.0028\033[0m]=10sec, [\033[32m0.1\033[0m]=6min, [\033[32m24\033[0m]=day, [\033[32m168\033[0m]=week, [\033[32m720\033[0m]=month, [\033[32m8760\033[0m]=year)")
+    ap2.add_argument("--dont-ban", metavar="TXT", type=u, default="no", help="anyone at this accesslevel or above will not get banned: [\033[32mav\033[0m]=admin-in-volume, [\033[32maa\033[0m]=has-admin-anywhere, [\033[32mrw\033[0m]=read-write, [\033[32mauth\033[0m]=authenticated, [\033[32many\033[0m]=disable-all-bans, [\033[32mno\033[0m]=anyone-can-get-banned")
+    ap2.add_argument("--banmsg", metavar="TXT", type=u, default="thank you for playing \u00a0 (see fileserver log and readme)", help="the response to send to banned users; can be @ban.html to send the contents of ban.html")
    ap2.add_argument("--ban-pw", metavar="N,W,B", type=u, default="9,60,1440", help="more than \033[33mN\033[0m wrong passwords in \033[33mW\033[0m minutes = ban for \033[33mB\033[0m minutes; disable with [\033[32mno\033[0m]")
    ap2.add_argument("--ban-pwc", metavar="N,W,B", type=u, default="5,60,1440", help="more than \033[33mN\033[0m password-changes in \033[33mW\033[0m minutes = ban for \033[33mB\033[0m minutes; disable with [\033[32mno\033[0m]")
    ap2.add_argument("--ban-404", metavar="N,W,B", type=u, default="50,60,1440", help="hitting more than \033[33mN\033[0m 404's in \033[33mW\033[0m minutes = ban for \033[33mB\033[0m minutes; only affects users who cannot see directory listings because their access is either g/G/h")
@@ -1561,6 +1727,13 @@ def add_safety(ap):
    ap2.add_argument("--loris", metavar="B", type=int, default=60, help="if a client maxes out the server connection limit without sending headers, ban it for \033[33mB\033[0m minutes; disable with [\033[32m0\033[0m]")
    ap2.add_argument("--acao", metavar="V[,V]", type=u, default="*", help="Access-Control-Allow-Origin; list of origins (domains/IPs without port) to accept requests from; [\033[32mhttps://1.2.3.4\033[0m]. Default [\033[32m*\033[0m] allows requests from all sites but removes cookies and http-auth; only ?pw=hunter2 survives")
    ap2.add_argument("--acam", metavar="V[,V]", type=u, default="GET,HEAD", help="Access-Control-Allow-Methods; list of methods to accept from offsite ('*' behaves like \033[33m--acao\033[0m's description)")
+    if FULL_HELP or ANYWIN:
+        ap2.add_argument("--unsafe-tools", action="store_true", help="windows-only: allow running programs (ffmpeg, dcraw_emu, ...) when their location is inside a folder that can be uploaded to (dangerous due to DLL-hijacking)")
+    if FULL_HELP or (not ANYWIN and not UNIX):
+        ap2.add_argument("--th-bwrap", metavar="CMD", type=u, default=th_bwrap, help="optional bwrap sandbox command for FFmpeg and dcraw (Linux-only)")
+        ap2.add_argument("--use-bwrap", metavar="TXT", type=u, default="n", help="a/n/f; [\033[32ma\033[0m]=auto (yes if the program 'bwrap' exists (assumes it works)), [\033[32mn\033[0m]=no (assumes bwrap is broken), [\033[32mf\033[0m]=force (disables FFmpeg if bwrap unavailable)")
+    else:
+        ap2.add_argument("--use-bwrap", metavar="TXT", type=u, default="n", help=argparse.SUPPRESS)


 def add_salt(ap, fk_salt, dk_salt, ah_salt):
@@ -1581,26 +1754,30 @@ def add_shutdown(ap):
    ap2 = ap.add_argument_group("shutdown options")
    ap2.add_argument("--ign-ebind", action="store_true", help="continue running even if it's impossible to listen on some of the requested endpoints")
    ap2.add_argument("--ign-ebind-all", action="store_true", help="continue running even if it's impossible to receive connections at all")
-    ap2.add_argument("--exit", metavar="WHEN", type=u, default="", help="shutdown after \033[33mWHEN\033[0m has finished; [\033[32mcfg\033[0m] config parsing, [\033[32midx\033[0m] volscan + multimedia indexing")
+    ap2.add_argument("--exit", metavar="WHEN", type=u, default="", help="shutdown after \033[33mWHEN\033[0m has finished; [\033[32mcfg\033[0m] config parsing, [\033[32midx\033[0m] volscan + multimedia indexing, [\033[32mthgen\033[0m] thumbnail-pregen")


 def add_logging(ap):
    ap2 = ap.add_argument_group("logging options")
    ap2.add_argument("-q", action="store_true", help="quiet; disable most STDOUT messages")
    ap2.add_argument("-lo", metavar="PATH", type=u, default="", help="logfile; use .txt for plaintext or .xz for compressed. Example: \033[32mcpp-%%Y-%%m%%d-%%H%%M%%S.txt.xz\033[0m (NB: some errors may appear on STDOUT only)")
+    ap2.add_argument("--flo", metavar="N", type=int, default=1, help="log format for \033[33m-lo\033[0m; [\033[32m1\033[0m]=classic/colors, [\033[32m2\033[0m]=no-color")
+    ap2.add_argument("--rlo", metavar="TXT", type=u, default="no", help="logrotate counter format; see \033[33m--help-rlo\033[0m")
+    ap2.add_argument("--logrot-sig", metavar="S", type=u, default="", help="immediately logrotate when unix-signal \033[33mS\033[0m is received; examples: [\033[32mSIGHUP\033[0m], [\033[32mHUP\033[0m], [\033[32m1\033[0m]")
    ap2.add_argument("--no-ansi", action="store_true", default=not VT100, help="disable colors; same as environment-variable NO_COLOR")
    ap2.add_argument("--ansi", action="store_true", help="force colors; overrides environment-variable NO_COLOR")
    ap2.add_argument("--no-logflush", action="store_true", help="don't flush the logfile after each write; tiny bit faster")
    ap2.add_argument("--no-voldump", action="store_true", help="do not list volumes and permissions on startup")
    ap2.add_argument("--log-utc", action="store_true", help="do not use local timezone; assume the TZ env-var is UTC (tiny bit faster)")
    ap2.add_argument("--log-tdec", metavar="N", type=int, default=3, help="timestamp resolution / number of timestamp decimals")
+    ap2.add_argument("--log-date", metavar="TXT", type=u, default="", help="date-format, for example [\033[32m%%Y-%%m-%%d\033[0m] (default is disabled; no date, just HH:MM:SS)")
    ap2.add_argument("--log-badpwd", metavar="N", type=int, default=2, help="log failed login attempt passwords: 0=terse, 1=plaintext, 2=hashed")
    ap2.add_argument("--log-badxml", action="store_true", help="log any invalid XML received from a client")
    ap2.add_argument("--log-conn", action="store_true", help="debug: print tcp-server msgs")
    ap2.add_argument("--log-htp", action="store_true", help="debug: print http-server threadpool scaling")
    ap2.add_argument("--ihead", metavar="HEADER", type=u, action='append', help="print request \033[33mHEADER\033[0m; [\033[32m*\033[0m]=all")
    ap2.add_argument("--ohead", metavar="HEADER", type=u, action='append', help="print response \033[33mHEADER\033[0m; [\033[32m*\033[0m]=all")
-    ap2.add_argument("--lf-url", metavar="RE", type=u, default=r"^/\.cpr/|[?&]th=[wjp]|/\.(_|ql_|DS_Store$|localized$)", help="dont log URLs matching regex \033[33mRE\033[0m")
+    ap2.add_argument("--lf-url", metavar="RE", type=u, default=r"^/\.cpr/|[?&]th=[xwjp]|/\.(_|ql_|DS_Store$|localized$)", help="dont log URLs matching regex \033[33mRE\033[0m")
    ap2.add_argument("--scan-st-r", metavar="SEC", type=float, default=0.1, help="fs-indexing: wait \033[33mSEC\033[0m between each status-message")
    ap2.add_argument("--scan-pr-r", metavar="SEC", type=float, default=10, help="fs-indexing: wait \033[33mSEC\033[0m between each 'progress:' message")
    ap2.add_argument("--scan-pr-s", metavar="MiB", type=float, default=1, help="fs-indexing: say 'file: <name>' when a file larger than \033[33mMiB\033[0m is about to be hashed")
@@ -1610,12 +1787,14 @@ def add_admin(ap):
    ap2 = ap.add_argument_group("admin panel options")
    ap2.add_argument("--no-reload", action="store_true", help="disable ?reload=cfg (reload users/volumes/volflags from config file)")
    ap2.add_argument("--no-rescan", action="store_true", help="disable ?scan (volume reindexing)")
-    ap2.add_argument("--no-stack", action="store_true", help="disable ?stack (list all stacks)")
+    ap2.add_argument("--no-stack", action="store_true", help="disable ?stack (list all stacks); same as --stack-who=no")
    ap2.add_argument("--no-ups-page", action="store_true", help="disable ?ru (list of recent uploads)")
    ap2.add_argument("--no-up-list", action="store_true", help="don't show list of incoming files in controlpanel")
    ap2.add_argument("--dl-list", metavar="LVL", type=int, default=2, help="who can see active downloads in the controlpanel? [\033[32m0\033[0m]=nobody, [\033[32m1\033[0m]=admins, [\033[32m2\033[0m]=everyone")
    ap2.add_argument("--ups-who", metavar="LVL", type=int, default=2, help="who can see recent uploads on the ?ru page? [\033[32m0\033[0m]=nobody, [\033[32m1\033[0m]=admins, [\033[32m2\033[0m]=everyone (volflag=ups_who)")
    ap2.add_argument("--ups-when", action="store_true", help="let everyone see upload timestamps on the ?ru page, not just admins")
+    ap2.add_argument("--stack-who", metavar="LVL", type=u, default="a", help="who can see the ?stack page (list of threads)? [\033[32mno\033[0m]=nobody, [\033[32ma\033[0m]=admins, [\033[32mrw\033[0m]=read+write, [\033[32mall\033[0m]=everyone")
+    ap2.add_argument("--stack-v", action="store_true", help="verbose ?stack")


 def add_thumbnail(ap):
@@ -1632,26 +1811,33 @@ def add_thumbnail(ap):
    ap2.add_argument("--th-ram-max", metavar="GB", type=float, default=th_ram, help="max memory usage (GiB) permitted by thumbnailer; not very accurate")
    ap2.add_argument("--th-crop", metavar="TXT", type=u, default="y", help="crop thumbnails to 4:3 or keep dynamic height; client can override in UI unless force. [\033[32my\033[0m]=crop, [\033[32mn\033[0m]=nocrop, [\033[32mfy\033[0m]=force-y, [\033[32mfn\033[0m]=force-n (volflag=crop)")
    ap2.add_argument("--th-x3", metavar="TXT", type=u, default="n", help="show thumbs at 3x resolution; client can override in UI unless force. [\033[32my\033[0m]=yes, [\033[32mn\033[0m]=no, [\033[32mfy\033[0m]=force-yes, [\033[32mfn\033[0m]=force-no (volflag=th3x)")
-    ap2.add_argument("--th-dec", metavar="LIBS", default="vips,pil,raw,ff", help="image decoders, in order of preference")
+    ap2.add_argument("--th-qv", metavar="N", type=int, default=40, help="webp/jpg thumbnail quality (10~90); higher is larger filesize and better quality (volflag=th_qv)")
+    ap2.add_argument("--th-qvx", metavar="N", type=int, default=64, help="jxl thumbnail quality (10~90); higher is larger filesize and better quality (volflag=th_qvx)")
+    ap2.add_argument("--th-dec", metavar="LIBS", default="raw,pil,ff,vips", help="image decoders, in order of preference")
    ap2.add_argument("--th-no-jpg", action="store_true", help="disable jpg output")
    ap2.add_argument("--th-no-webp", action="store_true", help="disable webp output")
+    ap2.add_argument("--th-no-jxl", action="store_true", help="disable jpeg-xl output")
    ap2.add_argument("--th-ff-jpg", action="store_true", help="force jpg output for video thumbs (avoids issues on some FFmpeg builds)")
    ap2.add_argument("--th-ff-swr", action="store_true", help="use swresample instead of soxr for audio thumbs (faster, lower accuracy, avoids issues on some FFmpeg builds)")
-    ap2.add_argument("--th-poke", metavar="SEC", type=int, default=300, help="activity labeling cooldown -- avoids doing keepalive pokes (updating the mtime) on thumbnail folders more often than \033[33mSEC\033[0m seconds")
+    ap2.add_argument("--th-vips-jxl", metavar="N", type=int, default=1, help="when to allow generating jxl thumbnails with libvips; 0=never, 1=musl-mallocng, 2=always")
+    ap2.add_argument("--th-poke", metavar="SEC", type=int, default=300, help="activity labeling cooldown -- avoids doing keepalive pokes (updating the mtime) on thumbnail folders more often than \033[33mSEC\033[0m seconds; 0=no-keepalive")
    ap2.add_argument("--th-clean", metavar="SEC", type=int, default=43200, help="cleanup interval; 0=disabled")
    ap2.add_argument("--th-maxage", metavar="SEC", type=int, default=604800, help="max folder age -- folders which haven't been poked for longer than \033[33m--th-poke\033[0m seconds will get deleted every \033[33m--th-clean\033[0m seconds")
-    ap2.add_argument("--th-covers", metavar="N,N", type=u, default="folder.png,folder.jpg,cover.png,cover.jpg", help="folder thumbnails to stat/look for; enabling \033[33m-e2d\033[0m will make these case-insensitive, and try them as dotfiles (.folder.jpg), and also automatically select thumbnails for all folders that contain pics, even if none match this pattern")
+    ap2.add_argument("--th-pregen", metavar="F,F", type=u, default="", help="pregenerate thumbnails on startup; \033[33mF,F\033[0m is comma-separated list of formats; example: [\033[32mj,jf,w,w3,wf,wf3,x,xf\033[0m] NOTE: remember to set \033[33m--th-maxage 123456789\033[0m (volflag=th_pregen)")
+    ap2.add_argument("--th-pre-rl", metavar="SEC", type=int, default=30, help="while pregen is running, ratelimit the thumbnailer logger to one message every \033[33mSEC\033[0m seconds (only works with \033[33m-j1\033[0m); set 0 to disable ratelimit")
+    ap2.add_argument("--th-covers", metavar="N,N", type=u, default="folder.png,folder.jpg,cover.png,cover.jpg", help="folder thumbnails to stat/look for; enabling \033[33m-e2d\033[0m will make these case-insensitive, and try them as dotfiles (.folder.jpg), and also automatically select thumbnails for all folders that contain pics, even if none match this pattern (volflag=th_covers)")
    ap2.add_argument("--th-spec-p", metavar="N", type=u, default=1, help="for music, do spectrograms or embedded coverart? [\033[32m0\033[0m]=only-art, [\033[32m1\033[0m]=prefer-art, [\033[32m2\033[0m]=only-spec")
+    ap2.add_argument("--th-spec-fl", action="store_true", help="generate spectrograms with logarithmic frequency scale instead of linear")
    # https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html
    # https://github.com/libvips/libvips
    # https://stackoverflow.com/a/47612661
    # ffmpeg -hide_banner -demuxers | awk '/^ D  /{print$2}' | while IFS= read -r x; do ffmpeg -hide_banner -h demuxer=$x; done | grep -E '^Demuxer |extensions:'
-    ap2.add_argument("--th-r-pil", metavar="T,T", type=u, default="avif,avifs,blp,bmp,cbz,dcx,dds,dib,emf,eps,epub,fits,flc,fli,fpx,gif,heic,heics,heif,heifs,icns,ico,im,j2p,j2k,jp2,jpeg,jpg,jpx,pbm,pcx,pgm,png,pnm,ppm,psd,qoi,sgi,spi,tga,tif,tiff,webp,wmf,xbm,xpm", help="image formats to decode using pillow")
-    ap2.add_argument("--th-r-vips", metavar="T,T", type=u, default="avif,exr,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,jp2,jpeg,jpg,jpx,jxl,nii,pfm,pgm,png,ppm,svg,tif,tiff,webp", help="image formats to decode using pyvips")
-    ap2.add_argument("--th-r-raw", metavar="T,T", type=u, default="arw,cr2,cr3,crw,dcr,dng,erf,k25,kdc,mrw,nef,orf,pef,raf,raw,sr2,srf,x3f", help="image formats to decode using rawpy")
+    ap2.add_argument("--th-r-pil", metavar="T,T", type=u, default="avif,avifs,blp,bmp,cbz,dcx,dds,dib,emf,eps,epub,fits,flc,fli,fpx,gif,heic,heics,heif,heifs,icns,ico,im,j2p,j2k,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pgm,png,pnm,ppm,psd,qoi,sgi,spi,tga,tif,tiff,webp,wmf,xbm,xpm", help="image formats to decode using pillow")
+    ap2.add_argument("--th-r-vips", metavar="T,T", type=u, default="3fr,arw,avif,cr2,cr3,crw,dcr,dng,erf,exr,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,jp2,jpeg,jpg,jpx,jxl,k25,kdc,mdc,mef,mrw,nef,nii,nrw,orf,pfm,pgm,png,ppm,raf,raw,rw2,sr2,srf,srw,svg,tif,tiff,webp,x3f", help="image formats to decode using pyvips")
+    ap2.add_argument("--th-r-raw", metavar="T,T", type=u, default="3fr,arw,cr2,cr3,crw,dcr,dng,erf,k25,kdc,mdc,mef,mos,mrw,nef,nrw,orf,pef,raf,raw,rw2,sr2,srf,srw,x3f", help="image formats to decode using rawpy (if available) or libraw's dcraw_emu")
    ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,cbz,dds,dib,epub,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,qoi,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
    ap2.add_argument("--th-r-ffv", metavar="T,T", type=u, default="3gp,asf,av1,avc,avi,flv,h264,h265,hevc,m4v,mjpeg,mjpg,mkv,mov,mp4,mpeg,mpeg2,mpegts,mpg,mpg2,mts,nut,ogm,ogv,rm,ts,vob,webm,wmv", help="video formats to decode using ffmpeg")
-    ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,ac3,aif,aiff,alac,alaw,amr,apac,ape,au,bonk,dfpwm,dts,flac,gsm,ilbc,it,itgz,itxz,itz,m4a,mdgz,mdxz,mdz,mo3,mod,mp2,mp3,mpc,mptm,mt2,mulaw,oga,ogg,okt,opus,ra,s3m,s3gz,s3xz,s3z,tak,tta,ulaw,wav,wma,wv,xm,xmgz,xmxz,xmz,xpk", help="audio formats to decode using ffmpeg")
+    ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,ac3,aif,aiff,alac,alaw,amr,apac,ape,au,bcstm,bfstm,brstm,bonk,dfpwm,dts,flac,gsm,ilbc,it,itgz,itxz,itz,m4a,m4b,m4r,mdgz,mdxz,mdz,mka,mo3,mod,mp2,mp3,mpc,mptm,mt2,mulaw,oga,ogg,okt,opus,ra,s3m,s3gz,s3xz,s3z,tak,tta,ulaw,wav,wma,wv,xm,xmgz,xmxz,xmz,xpk", help="audio formats to decode using ffmpeg")
    ap2.add_argument("--th-spec-cnv", metavar="T", type=u, default="it,itgz,itxz,itz,mdgz,mdxz,mdz,mo3,mod,s3m,s3gz,s3xz,s3z,xm,xmgz,xmxz,xmz,xpk", help="audio formats which provoke https://trac.ffmpeg.org/ticket/10797 (huge ram usage for s3xmodit spectrograms)")
    ap2.add_argument("--au-unpk", metavar="E=F.C", type=u, default="mdz=mod.zip, mdgz=mod.gz, mdxz=mod.xz, s3z=s3m.zip, s3gz=s3m.gz, s3xz=s3m.xz, xmz=xm.zip, xmgz=xm.gz, xmxz=xm.xz, itz=it.zip, itgz=it.gz, itxz=it.xz, cbz=jpg.cbz, epub=jpg.epub", help="audio/image formats to decompress before passing to ffmpeg")

@@ -1684,7 +1870,9 @@ def add_rss(ap):
    ap2.add_argument("--rss", action="store_true", help="enable RSS output (experimental) (volflag=rss)")
    ap2.add_argument("--rss-nf", metavar="HITS", type=int, default=250, help="default number of files to return (url-param 'nf')")
    ap2.add_argument("--rss-fext", metavar="E,E", type=u, default="", help="default list of file extensions to include (url-param 'fext'); blank=all")
-    ap2.add_argument("--rss-sort", metavar="ORD", type=u, default="m", help="default sort order (url-param 'sort'); [\033[32mm\033[0m]=last-modified [\033[32mu\033[0m]=upload-time [\033[32mn\033[0m]=filename [\033[32ms\033[0m]=filesize; Uppercase=oldest-first. Note that upload-time is 0 for non-uploaded files")
+    ap2.add_argument("--rss-sort", metavar="ORD", type=u, default="m", help="default sort order (url-param 'sort'); [\033[32mm\033[0m]=last-modified [\033[32mu\033[0m]=upload-time [\033[32mn\033[0m]=filename [\033[32ms\033[0m]=filesize; Uppercase=oldest-first. Note that upload-time is 0 for non-uploaded files (volflag=rss_sort)")
+    ap2.add_argument("--rss-fmt-t", metavar="TXT", type=u, default="{fname}", help="title format (url-param 'rss_fmt_t') (volflag=rss_fmt_t)")
+    ap2.add_argument("--rss-fmt-d", metavar="TXT", type=u, default="{artist} - {title}", help="description format (url-param 'rss_fmt_d') (volflag=rss_fmt_d)")


 def add_db_general(ap, hcores):
@@ -1698,6 +1886,7 @@ def add_db_general(ap, hcores):
    ap2.add_argument("-e2vp", action="store_true", help="on hash mismatch: panic and quit copyparty")
    ap2.add_argument("--hist", metavar="PATH", type=u, default="", help="where to store volume data (db, thumbs); default is a folder named \".hist\" inside each volume (volflag=hist)")
    ap2.add_argument("--dbpath", metavar="PATH", type=u, default="", help="override where the volume databases are to be placed; default is the same as \033[33m--hist\033[0m (volflag=dbpath)")
+    ap2.add_argument("--fika", metavar="TXT", type=u, default="ucd", help="list of user-actions to allow while filesystem-indexer is still busy; set blank to never interrupt indexing (old default). [\033[32mu\033[0m]=uploads, [\033[32mc\033[0m]=filecopy, [\033[32mm\033[0m]=move/rename, [\033[32md\033[0m]=delete. NOTE: [\033[32mm\033[0m] is untested/scary, and blank is recommended if dedup enabled")
    ap2.add_argument("--no-hash", metavar="PTN", type=u, default="", help="regex: disable hashing of matching absolute-filesystem-paths during e2ds folder scans (must be specified as one big regex, not multiple times) (volflag=nohash)")
    ap2.add_argument("--no-idx", metavar="PTN", type=u, default=noidx, help="regex: disable indexing of matching absolute-filesystem-paths during e2ds folder scan (must be specified as one big regex, not multiple times) (volflag=noidx)")
    ap2.add_argument("--no-dirsz", action="store_true", help="do not show total recursive size of folders in listings, show inode size instead; slightly faster (volflag=nodirsz)")
@@ -1711,7 +1900,8 @@ def add_db_general(ap, hcores):
    ap2.add_argument("--hash-mt", metavar="CORES", type=int, default=hcores, help="num cpu cores to use for file hashing; set 0 or 1 for single-core hashing")
    ap2.add_argument("--re-maxage", metavar="SEC", type=int, default=0, help="rescan filesystem for changes every \033[33mSEC\033[0m seconds; 0=off (volflag=scan)")
    ap2.add_argument("--db-act", metavar="SEC", type=float, default=10.0, help="defer any scheduled volume reindexing until \033[33mSEC\033[0m seconds after last db write (uploads, renames, ...)")
-    ap2.add_argument("--srch-icase", action="store_true", help="case-insensitive search for all unicode characters (the default is icase for just ascii). NOTE: will make searches much slower (around 4x), and NOTE: only applies to filenames/paths, not tags")
+    ap2.add_argument("--srch-icase", action="store_true", help="case-insensitive search for all unicode characters (the default is icase for just ascii). NOTE: will make searches much slower (around 3x), and NOTE: only applies to filenames/paths, not tags")
+    ap2.add_argument("--srch-nfkc", action="store_true", help="case-insensitive and unicode-normalization-insensitive search (NFC/NFD) for filenames/paths; slightly slower than \033[33m--srch-icase\033[0m (about 15%% slower / 87%% as fast)")
    ap2.add_argument("--srch-time", metavar="SEC", type=int, default=45, help="search deadline -- terminate searches running for more than \033[33mSEC\033[0m seconds")
    ap2.add_argument("--srch-hits", metavar="N", type=int, default=7999, help="max search results to allow clients to fetch; 125 results will be shown initially")
    ap2.add_argument("--srch-excl", metavar="PTN", type=u, default="", help="regex: exclude files from search results if the file-URL matches \033[33mPTN\033[0m (case-sensitive). Example: [\033[32mpassword|logs/[0-9]\033[0m] any URL containing 'password' or 'logs/DIGIT' (volflag=srch_excl)")
@@ -1729,16 +1919,21 @@ def add_db_metadata(ap):
    ap2.add_argument("--mtag-mt", metavar="CORES", type=int, default=CORES, help="num cpu cores to use for tag scanning")
    ap2.add_argument("--mtag-v", action="store_true", help="verbose tag scanning; print errors from mtp subprocesses and such")
    ap2.add_argument("--mtag-vv", action="store_true", help="debug mtp settings and mutagen/FFprobe parsers")
+    ap2.add_argument("--db-xattr", metavar="t,t", type=u, default="", help="read file xattrs as metadata tags; [\033[32ma,b\033[0m] reads keys \033[33ma\033[0m and \033[33mb\033[0m as tags \033[33ma\033[0m and \033[33mb\033[0m, [\033[32ma=foo,b=bar\033[0m] reads keys \033[33ma\033[0m and \033[33mb\033[0m as tags \033[33mfoo\033[0m and \033[33mbar\033[0m, [\033[32m~~a,b\033[0m] does everything except \033[33ma\033[0m and \033[33mb\033[0m, [\033[32m~~\033[0m] does everything. NOTE: Each tag must also be enabled with \033[33m-mte\033[0m (volflag=db_xattr)")
    ap2.add_argument("-mtm", metavar="M=t,t,t", type=u, action="append", help="\033[34mREPEATABLE:\033[0m add/replace metadata mapping")
    ap2.add_argument("-mte", metavar="M,M,M", type=u, help="tags to index/display (comma-sep.); either an entire replacement list, or add/remove stuff on the default-list with +foo or /bar", default=DEF_MTE)
    ap2.add_argument("-mth", metavar="M,M,M", type=u, help="tags to hide by default (comma-sep.); assign/add/remove same as \033[33m-mte\033[0m", default=DEF_MTH)
    ap2.add_argument("-mtp", metavar="M=[f,]BIN", type=u, action="append", help="\033[34mREPEATABLE:\033[0m read tag \033[33mM\033[0m using program \033[33mBIN\033[0m to parse the file")
+    ap2.add_argument("--have-db-xattr", action="store_true", help=argparse.SUPPRESS)


 def add_txt(ap):
    ap2 = ap.add_argument_group("textfile options")
+    ap2.add_argument("--rw-edit", metavar="T,T", type=u, default="md", help="comma-sep. list of file-extensions to allow editing with permissions read+write; all others require read+write+delete (volflag=rw_edit)")
    ap2.add_argument("--md-no-br", action="store_true", help="markdown: disable newline-is-newline; will only render a newline into the html given two trailing spaces or a double-newline (volflag=md_no_br)")
-    ap2.add_argument("--md-hist", metavar="TXT", type=u, default="s", help="where to store old version of markdown files; [\033[32ms\033[0m]=subfolder, [\033[32mv\033[0m]=volume-histpath, [\033[32mn\033[0m]=nope/disabled (volflag=md_hist)")
+    ap2.add_argument("--md-hist", metavar="TXT", type=u, default="s", help="where to store old version of textfiles; [\033[32ms\033[0m]=subfolder, [\033[32mv\033[0m]=volume-histpath, [\033[32mn\033[0m]=nope/disabled (volflag=md_hist)")
+    ap2.add_argument("--md-nhist", metavar="TXT", type=int, default=99, help="how many old versions of textfiles to keep; 0=infinite (volflag=md_nhist)")
+    ap2.add_argument("--show-hist", action="store_true", help="allow browsing the .hist folder; enable to view old markdown files (volflag=show_hist)")
    ap2.add_argument("--txt-eol", metavar="TYPE", type=u, default="", help="enable EOL conversion when writing documents; supported: CRLF, LF (volflag=txt_eol)")
    ap2.add_argument("-mcr", metavar="SEC", type=int, default=60, help="the textfile editor will check for serverside changes every \033[33mSEC\033[0m seconds")
    ap2.add_argument("-emp", action="store_true", help="enable markdown plugins -- neat but dangerous, big XSS risk")
@@ -1772,8 +1967,11 @@ def add_ui(ap, retry: int):
    ap2.add_argument("--grid", action="store_true", help="show grid/thumbnails by default (volflag=grid)")
    ap2.add_argument("--gsel", action="store_true", help="select files in grid by ctrl-click (volflag=gsel)")
    ap2.add_argument("--localtime", action="store_true", help="default to local timezone instead of UTC")
-    ap2.add_argument("--ui-filesz", metavar="FMT", type=u, default="1", help="default filesize format; one of these: 0, 1, 2, 2c, 3, 3c, 4, 4c, 5, 5c, fuzzy (see UI)")
+    ap2.add_argument("--ui-filesz", metavar="FMT", type=u, default="1", help="default filesize format; one of these: 0, 1, 2, 2c, 3, 3c, 4, 4c, 5, 5c, 6, 6c, 7, 7c, fuzzy (see UI)")
+    ap2.add_argument("--gauto", metavar="PERCENT", type=int, default=0, help="switch to gridview if more than \033[33mPERCENT\033[0m of files are pics/vids; 0=disabled")
+    ap2.add_argument("--rcm", metavar="TXT", default="yy", help="rightclick-menu; two yes/no options: 1st y/n is enable-custom-menu, 2nd y/n is enable-double")
    ap2.add_argument("--lang", metavar="LANG", type=u, default="eng", help="language, for example \033[32meng\033[0m / \033[32mnor\033[0m / ...")
+    ap2.add_argument("--glang", action="store_true", help="guess the browser's default language, otherwise fall back to \033[33m--lang\033[0m")
    ap2.add_argument("--theme", metavar="NUM", type=int, default=0, help="default theme to use (0..%d)" % (THEMES - 1,))
    ap2.add_argument("--themes", metavar="NUM", type=int, default=THEMES, help="number of themes installed")
    ap2.add_argument("--au-vol", metavar="0-100", type=int, default=50, choices=range(0, 101), help="default audio/video volume percent")
@@ -1782,20 +1980,27 @@ def add_ui(ap, retry: int):
    ap2.add_argument("--hsortn", metavar="N", type=int, default=2, help="number of sorting rules to include in media URLs by default (volflag=hsortn)")
    ap2.add_argument("--see-dots", action="store_true", help="default-enable seeing dotfiles; only takes effect if user has the necessary permissions")
    ap2.add_argument("--qdel", metavar="LVL", type=int, default=2, help="number of confirmations to show when deleting files (2/1/0)")
+    ap2.add_argument("--dlni", action="store_true", help="force download (don't show inline) when files are clicked (volflag:dlni)")
    ap2.add_argument("--unlist", metavar="REGEX", type=u, default="", help="don't show files/folders matching \033[33mREGEX\033[0m in file list. WARNING: Purely cosmetic! Does not affect API calls, just the browser. Example: [\033[32m\\.(js|css)$\033[0m] (volflag=unlist)")
+    ap2.add_argument("--dothidden", action="store_true", help="hide specific files in a folder by listing them in a file named .hidden -- WARNING: Mostly cosmetic! Download-as-zip/tar will still download them. Do not rely on this for security (volflag=dothidden)")
    ap2.add_argument("--favico", metavar="TXT", type=u, default="c 000 none" if retry else "🎉 000 none", help="\033[33mfavicon-text\033[0m [ \033[33mforeground\033[0m [ \033[33mbackground\033[0m ] ], set blank to disable")
    ap2.add_argument("--ufavico", metavar="TXT", type=u, default="", help="URL to .ico/png/gif/svg file; \033[33m--favico\033[0m takes precedence unless disabled (volflag=ufavico)")
    ap2.add_argument("--ext-th", metavar="E=VP", type=u, action="append", help="\033[34mREPEATABLE:\033[0m use thumbnail-image \033[33mVP\033[0m for file-extension \033[33mE\033[0m, example: [\033[32mexe=/.res/exe.png\033[0m] (volflag=ext_th)")
    ap2.add_argument("--mpmc", type=u, default="", help=argparse.SUPPRESS)
+    ap2.add_argument("--notooltips", action="store_true", help="tooltips disabled as default")
    ap2.add_argument("--spinner", metavar="TXT", type=u, default="🌲", help="\033[33memoji\033[0m or \033[33memoji,css\033[0m Example: [\033[32m🥖,padding:0\033[0m]")
    ap2.add_argument("--css-browser", metavar="L", type=u, default="", help="URL to additional CSS to include in the filebrowser html")
    ap2.add_argument("--js-browser", metavar="L", type=u, default="", help="URL to additional JS to include in the filebrowser html")
    ap2.add_argument("--js-other", metavar="L", type=u, default="", help="URL to additional JS to include in all other pages")
-    ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages (except for basic-browser); can be @PATH to send the contents of a file at PATH, and/or begin with %% to render as jinja2 template (volflag=html_head)")
+    ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages (except for basic-browser); can be @PATH to send the contents of a file at PATH, and/or begin with %% to render as jinja2 template; \033[32m<script>\033[0m will not work, use \033[32m<script nonce=\"{{ js_nonce }}\">\033[0m (volflag=html_head)")
    ap2.add_argument("--html-head-s", metavar="T", type=u, default="", help="text to append to the <head> of all HTML pages (except for basic-browser); similar to (and can be combined with) --html-head but only accepts static text (volflag=html_head_s)")
    ap2.add_argument("--ih", action="store_true", help="if a folder contains index.html, show that instead of the directory listing by default (can be changed in the client settings UI, or add ?v to URL for override)")
    ap2.add_argument("--textfiles", metavar="CSV", type=u, default="txt,nfo,diz,cue,readme", help="file extensions to present as plaintext")
    ap2.add_argument("--txt-max", metavar="KiB", type=int, default=64, help="max size of embedded textfiles on ?doc= (anything bigger will be lazy-loaded by JS)")
+    ap2.add_argument("--prologues", metavar="T,T", type=u, default=".prologue.html", help="comma-sep. list of filenames to scan for and use as prologues (embed above/before directory listing) (volflag=prologues)")
+    ap2.add_argument("--epilogues", metavar="T,T", type=u, default=".epilogue.html", help="comma-sep. list of filenames to scan for and use as epilogues (embed below/after directory listing) (volflag=epilogues)")
+    ap2.add_argument("--preadmes", metavar="T,T", type=u, default="preadme.md,PREADME.md", help="comma-sep. list of filenames to scan for and use as preadmes (embed above/before directory listing) (volflag=preadmes)")
+    ap2.add_argument("--readmes", metavar="T,T", type=u, default="readme.md,README.md", help="comma-sep. list of filenames to scan for and use as readmes (embed below/after directory listing) (volflag=readmes)")
    ap2.add_argument("--doctitle", metavar="TXT", type=u, default="copyparty @ --name", help="title / service-name to show in html documents")
    ap2.add_argument("--bname", metavar="TXT", type=u, default="--name", help="server name (displayed in filebrowser document title)")
    ap2.add_argument("--pb-url", metavar="URL", type=u, default=URL_PRJ, help="powered-by link; disable with \033[33m-nb\033[0m")
@@ -1813,6 +2018,15 @@ def add_ui(ap, retry: int):
    ap2.add_argument("--lg-sba", metavar="TXT", type=u, default="", help="the value of the iframe 'allow' attribute for prologue/epilogue docs (volflag=lg_sba); see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Permissions-Policy#iframes")
    ap2.add_argument("--no-sb-md", action="store_true", help="don't sandbox README/PREADME.md documents (volflags: no_sb_md | sb_md)")
    ap2.add_argument("--no-sb-lg", action="store_true", help="don't sandbox prologue/epilogue docs (volflags: no_sb_lg | sb_lg); enables non-js support")
+    ap2.add_argument("--ui-nombar", action="store_true", help="hide top-menu in the UI (volflag=ui_nombar)")
+    ap2.add_argument("--ui-noacci", action="store_true", help="hide account-info in the UI (volflag=ui_noacci)")
+    ap2.add_argument("--ui-nosrvi", action="store_true", help="hide server-info in the UI (volflag=ui_nosrvi)")
+    ap2.add_argument("--ui-nonav", action="store_true", help="hide navpane+breadcrumbs (volflag=ui_nonav)")
+    ap2.add_argument("--ui-notree", action="store_true", help="hide navpane in the UI (volflag=ui_notree)")
+    ap2.add_argument("--ui-nocpla", action="store_true", help="hide cpanel-link in the UI (volflag=ui_nocpla)")
+    ap2.add_argument("--ui-nolbar", action="store_true", help="hide link-bar in the UI (volflag=ui_nolbar)")
+    ap2.add_argument("--ui-noctxb", action="store_true", help="hide context-buttons in the UI (volflag=ui_noctxb)")
+    ap2.add_argument("--ui-norepl", action="store_true", help="hide repl-button in the UI (volflag=ui_norepl)")
    ap2.add_argument("--have-unlistc", action="store_true", help=argparse.SUPPRESS)


@@ -1821,16 +2035,21 @@ def add_debug(ap):
    ap2.add_argument("--vc", action="store_true", help="verbose config file parser (explain config)")
    ap2.add_argument("--cgen", action="store_true", help="generate config file from current config (best-effort; probably buggy)")
    ap2.add_argument("--deps", action="store_true", help="list information about detected optional dependencies")
-    if hasattr(select, "poll"):
+    if FULL_HELP or hasattr(select, "poll"):
        ap2.add_argument("--no-poll", action="store_true", help="kernel-bug workaround: disable poll; use select instead (limits max num clients to ~700)")
    ap2.add_argument("--no-sendfile", action="store_true", help="kernel-bug workaround: disable sendfile; do a safe and slow read-send-loop instead")
    ap2.add_argument("--no-scandir", action="store_true", help="kernel-bug workaround: disable scandir; do a listdir + stat on each file instead")
    ap2.add_argument("--no-fastboot", action="store_true", help="wait for initial filesystem indexing before accepting client requests")
    ap2.add_argument("--no-htp", action="store_true", help="disable httpserver threadpool, create threads as-needed instead")
+    if sys.version_info > (3, 7) and not ANYWIN:
+        ap2.add_argument("--sig-thr", action="store_true", help="start separate thread for OS-signals (try this if CTRL-C is busted)")
+    else:
+        ap2.add_argument("--sig-thr", action="store_true", default=True, help=argparse.SUPPRESS)
    ap2.add_argument("--rm-sck", action="store_true", help="when listening on unix-sockets, do a basic delete+bind instead of the default atomic bind")
    ap2.add_argument("--srch-dbg", action="store_true", help="explain search processing, and do some extra expensive sanity checks")
    ap2.add_argument("--rclone-mdns", action="store_true", help="use mdns-domain instead of server-ip on /?hc")
    ap2.add_argument("--stackmon", metavar="P,S", type=u, default="", help="write stacktrace to \033[33mP\033[0math every \033[33mS\033[0m second, for example --stackmon=\033[32m./st/%%Y-%%m/%%d/%%H%%M.xz,60")
+    ap2.add_argument("--stack-sig", metavar="S", type=u, default="", help="show stacktrace when unix-signal \033[33mS\033[0m is received; examples: [\033[32mSIGUSR2\033[0m], [\033[32mUSR2\033[0m], [\033[32m12\033[0m]")
    ap2.add_argument("--log-thrs", metavar="SEC", type=float, default=0.0, help="list active threads every \033[33mSEC\033[0m")
    ap2.add_argument("--log-fk", metavar="REGEX", type=u, default="", help="log filekey params for files where path matches \033[33mREGEX\033[0m; [\033[32m.\033[0m] (a single dot) = all files")
    ap2.add_argument("--bak-flips", action="store_true", help="[up2k] if a client uploads a bitflipped/corrupted chunk, store a copy according to \033[33m--bf-nc\033[0m and \033[33m--bf-dir\033[0m")
@@ -1886,11 +2105,13 @@ def run_argparse(
    add_thumbnail(ap)
    add_transcoding(ap)
    add_rss(ap)
+    add_sftp(ap)
    add_ftp(ap)
    add_webdav(ap)
    add_tftp(ap)
    add_smb(ap)
    add_opds(ap)
+    add_wopi(ap)
    add_safety(ap)
    add_salt(ap, fk_salt, dk_salt, ah_salt)
    add_optouts(ap)
@@ -1940,16 +2161,20 @@ def run_argparse(


 def main(argv: Optional[list[str]] = None) -> None:
+    if argv is None:
+        argv = sys.argv
+
+    if "--versionb" in argv:
+        print(S_VERSION)
+        sys.exit(0)
+
    time.strptime("19970815", "%Y%m%d")  # python#7980
    if WINDOWS:
        os.system("rem")  # enables colors

    init_E(E)

-    if argv is None:
-        argv = sys.argv
-
-    f = '\033[36mcopyparty v{} "\033[35m{}\033[36m" ({})\n{}\033[0;36m\n   sqlite {} | jinja {} | pyftpd {} | tftp {}\n\033[0m'
+    f = '\033[36mcopyparty v{} "\033[35m{}\033[36m" ({})\n{}\033[0;36m\n   sqlite {} | jinja {} | pyftpd {} | tftp {} | miko {}\n\033[0m'
    f = f.format(
        S_VERSION,
        CODENAME,
@@ -1959,6 +2184,7 @@ def main(argv: Optional[list[str]] = None) -> None:
        JINJA_VER,
        PYFTPD_VER,
        PARTFTPY_VER,
+        MIKO_VER,
    )
    lprint(f)

@@ -1973,6 +2199,14 @@ def main(argv: Optional[list[str]] = None) -> None:
        print("\n".join("%8s %s" % (k, v) for k, v in sorted(MIMES.items())))
        sys.exit(0)

+    if "--list-ips" in argv:
+        print("\n".join(str(x) for x in sorted(list_ips())))
+        sys.exit(0)
+
+    if "--list-nics" in argv:
+        print("\n".join(str(x) for x in sorted(list_nics(True).items())))
+        sys.exit(0)
+
    if EXE:
        print("pybin: {}\n".format(pybin), end="")

@@ -2056,6 +2290,17 @@ def main(argv: Optional[list[str]] = None) -> None:
    except:
        sys.exit(1)

+    quotecheck(al)
+
+    if al.env_expand == 2:
+        al.shenvexp = expand_osenv_c
+    elif al.env_expand == 1:
+        al.shenvexp = expand_osenv_s
+    elif al.env_expand == 0:
+        al.shenvexp = expand_osenv_noop
+    else:
+        al.shenvexp = expand_osenv_cs
+
    if al.chdir:
        os.chdir(al.chdir)

@@ -2075,7 +2320,7 @@ def main(argv: Optional[list[str]] = None) -> None:

    # propagate implications
    for k1, k2 in IMPLICATIONS:
-        if getattr(al, k1):
+        if getattr(al, k1, None):
            setattr(al, k2, True)

    # propagate unplications
@@ -2083,10 +2328,17 @@ def main(argv: Optional[list[str]] = None) -> None:
        if getattr(al, k1):
            setattr(al, k2, False)

-    if not HAVE_IPV6 and al.i == "::":
-        al.i = "0.0.0.0"
+    protos = "ftp tftp sftp smb".split()
+    opts = ["%s_i" % (zs,) for zs in protos]
+    for opt in opts:
+        if getattr(al, opt) == "-i":
+            setattr(al, opt, al.i)
+    for opt in ["i"] + opts:
+        zs = getattr(al, opt)
+        if not HAVE_IPV6 and zs == "::":
+            zs = "0.0.0.0"
+        setattr(al, opt, [x.strip() for x in zs.split(",")])

-    al.i = [x.strip() for x in al.i.split(",")]
    try:
        if "-" in al.p:
            lo, hi = [int(x) for x in al.p.split("-")]
@@ -2142,7 +2394,7 @@ def main(argv: Optional[list[str]] = None) -> None:

    # signal.signal(signal.SIGINT, sighandler)

-    SvcHub(al, dal, argv, "".join(printed)).run()
+    SvcHub(al, dal, argv).run()


 if __name__ == "__main__":
@@ -1,8 +1,8 @@
 # coding: utf-8

-VERSION = (1, 19, 16)
-CODENAME = "usernames"
-BUILD_DT = (2025, 10, 5)
+VERSION = (1, 20, 19)
+CODENAME = "sftp is fine too"
+BUILD_DT = (2026, 7, 27)

 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os
 import time
@@ -40,7 +40,7 @@ def makedirs(name: str, vf: dict[str, Any] = MKD_755, exist_ok: bool = True) ->
    todo = []
    bname = fsenc(name)
    while bname:
-        if os.path.isdir(bname):
+        if os.path.isdir(bname) or bname in todo:
            break
        todo.append(bname)
        bname = os.path.dirname(bname)
@@ -104,12 +104,16 @@ def utime(


 def utime_c(
-    log: Union["NamedLogger", Any], p: str, ts: int, follow_symlinks: bool = True, throw: bool = False
-) -> Optional[int]:
+    log: Union["NamedLogger", Any],
+    p: str,
+    ts: float,
+    follow_symlinks: bool = True,
+    throw: bool = False,
+) -> Optional[float]:
    clamp = 0
    ov = ts
    bp = fsenc(p)
-    now = int(time.time())
+    now = time.time()
    while True:
        try:
            if SYMTIME:
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os

@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import threading
 import time
@@ -146,13 +146,13 @@ class BrokerMp(object):
        returns a Queue object which eventually contains the response if want_retval
        (not-impl here since nothing uses it yet)
        """
-        if dest == "httpsrv.listen":
-            for p in self.procs:
-                p.q_pend.put((0, dest, [args[0], len(self.procs)]))
-
-        elif dest == "httpsrv.set_netdevs":
-            for p in self.procs:
-                p.q_pend.put((0, dest, list(args)))
+        if dest.startswith("httpsrv."):
+            if dest == "httpsrv.listen":
+                for p in self.procs:
+                    p.q_pend.put((0, dest, [args[0], len(self.procs)]))
+            else:
+                for p in self.procs:
+                    p.q_pend.put((0, dest, list(args)))

        elif dest == "cb_httpsrv_up":
            self.hub.cb_httpsrv_up()
@@ -160,6 +160,13 @@ class BrokerMp(object):
        else:
            raise Exception("what is " + str(dest))

+    def say1(self, dest: str, *args: Any) -> None:
+        """
+        send message to one lucky recipient
+        """
+        p = self.procs[0]
+        p.q_pend.put((0, dest, list(args)))
+
    def periodic(self) -> None:
        while True:
            time.sleep(1)
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse
 import os
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os
 import threading
@@ -53,12 +53,12 @@ class BrokerThr(BrokerCli):
        return NotExQueue(obj(*args))  # type: ignore

    def say(self, dest: str, *args: Any) -> None:
-        if dest == "httpsrv.listen":
-            self.httpsrv.listen(args[0], 1)
-            return
+        if dest.startswith("httpsrv."):
+            if dest == "httpsrv.listen":
+                self.httpsrv.listen(args[0], 1)
+                return

-        if dest == "httpsrv.set_netdevs":
-            self.httpsrv.set_netdevs(args[0])
+            getattr(self.httpsrv, dest[8:])(*args)
            return

        # new ipc invoking managed service in hub
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse

@@ -16,6 +16,7 @@ if True:  # pylint: disable=using-constant-test

 if TYPE_CHECKING:
    from .httpsrv import HttpSrv
+    from .svchub import SvcHub


 class ExceptionalQueue(Queue, object):
@@ -50,6 +51,7 @@ class BrokerCli(object):
    for example resolving httpconn.* in httpcli -- see lines tagged #mypy404
    """

+    hub: "SvcHub"
    log: "RootLogger"
    args: argparse.Namespace
    asrv: AuthSrv
@@ -2,6 +2,7 @@ import calendar
 import errno
 import json
 import os
+import shutil
 import time

 from .__init__ import ANYWIN
@@ -19,6 +20,19 @@ else:
    VF = {"mv_re_t": 0, "rm_re_t": 0}


+def _sp_err(exe, what, rc, so, se, sin):
+    try:
+        zs = shutil.which(exe)
+    except:
+        zs = "<?>"
+    try:
+        zi = os.path.getsize(zs)
+    except:
+        zi = 0
+    t = "failed to %s; error %s using %s (%s):\n  STDOUT: %s\n  STDERR: %s\n  STDIN: %s\n"
+    raise Exception(t % (what, rc, zs, zi, so, se, sin.decode("utf-8")))
+
+
 def ensure_cert(log: "RootLogger", args) -> None:
    """
    the default cert (and the entire TLS support) is only here to enable the
@@ -37,9 +51,12 @@ def ensure_cert(log: "RootLogger", args) -> None:
        with open(args.cert, "wb") as f:
            f.write(cert_insec)

+    if args.certkey and not os.path.isfile(args.certkey):
+        raise Exception("certificate-key file does not exist: " + args.certkey)
+
    with open(args.cert, "rb") as f:
        buf = f.read()
-        o1 = buf.find(b" PRIVATE KEY-")
+        o1 = buf.find(b" PRIVATE KEY-") if not args.certkey else 0
        o2 = buf.find(b" CERTIFICATE-")
        m = "unsupported certificate format: "
        if o1 < 0:
@@ -107,13 +124,13 @@ def _gen_ca(log: "RootLogger", args):
    cmd = "cfssl gencert -initca -"
    rc, so, se = runcmd(cmd.split(), 30, sin=sin)
    if rc:
-        raise Exception("failed to create ca-cert: {}, {}".format(rc, se), 3)
+        _sp_err("cfssl", "create ca-cert", rc, so, se, sin)

    cmd = "cfssljson -bare ca"
    sin = so.encode("utf-8")
    rc, so, se = runcmd(cmd.split(), 10, sin=sin, cwd=args.crt_dir)
    if rc:
-        raise Exception("failed to translate ca-cert: {}, {}".format(rc, se), 3)
+        _sp_err("cfssljson", "translate ca-cert", rc, so, se, sin)

    bname = os.path.join(args.crt_dir, "ca")
    try:
@@ -201,13 +218,13 @@ def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
    acmd = cmd.split() + ["-hostname=" + ",".join(names), "-"]
    rc, so, se = runcmd(acmd, 30, sin=sin, cwd=args.crt_dir)
    if rc:
-        raise Exception("failed to create cert: {}, {}".format(rc, se))
+        _sp_err("cfssl", "create cert", rc, so, se, sin)

    cmd = "cfssljson -bare srv"
    sin = so.encode("utf-8")
    rc, so, se = runcmd(cmd.split(), 10, sin=sin, cwd=args.crt_dir)
    if rc:
-        raise Exception("failed to translate cert: {}, {}".format(rc, se))
+        _sp_err("cfssljson", "translate cert", rc, so, se, sin)

    bname = os.path.join(args.crt_dir, "srv")
    try:
@@ -238,7 +255,7 @@ def gencert(log: "RootLogger", args, netdevs: dict[str, Netdev]):
    if args.http_only:
        return

-    if args.no_crt or not HAVE_CFSSL:
+    if args.no_crt or args.certkey or not HAVE_CFSSL:
        ensure_cert(log, args)
        return

@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 # awk -F\" '/add_argument\("-[^-]/{print(substr($2,2))}' copyparty/__main__.py | sort | tr '\n' ' '
 zs = "a c e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vp e2vu ed emp i j lo mcr mte mth mtm mtp nb nc nid nih nth nw p q s ss sss v z zv"
@@ -19,16 +19,23 @@ def vf_bmap() -> dict[str, str]:
        "no_clone": "noclone",
        "no_dirsz": "nodirsz",
        "no_dupe": "nodupe",
+        "no_dupe_m": "nodupem",
        "no_forget": "noforget",
+        "no_html": "nohtml",
        "no_pipe": "nopipe",
        "no_robots": "norobots",
+        "no_script": "noscript",
        "no_tail": "notail",
        "no_thumb": "dthumb",
        "no_vthumb": "dvthumb",
        "no_athumb": "dathumb",
+        "vol_nospawn": "nospawn",
+        "vol_or_crash": "assert_root",
    }
    for k in (
        "dedup",
+        "dlni",
+        "dothidden",
        "dotsrch",
        "e2d",
        "e2ds",
@@ -46,6 +53,8 @@ def vf_bmap() -> dict[str, str]:
        "magic",
        "md_no_br",
        "no_db_ip",
+        "no_logues",
+        "no_readme",
        "no_sb_md",
        "no_sb_lg",
        "nsort",
@@ -58,6 +67,16 @@ def vf_bmap() -> dict[str, str]:
        "rm_partial",
        "rmagic",
        "rss",
+        "show_hist",
+        "ui_noacci",
+        "ui_nocpla",
+        "ui_nolbar",
+        "ui_nombar",
+        "ui_nonav",
+        "ui_notree",
+        "ui_norepl",
+        "ui_nosrvi",
+        "ui_noctxb",
        "wo_up_readme",
        "wram",
        "xdev",
@@ -83,14 +102,21 @@ def vf_vmap() -> dict[str, str]:
        "th_x3": "th3x",
    }
    for k in (
+        "apnd_who",
        "bup_ck",
+        "cachectl",
        "casechk",
        "chmod_d",
        "chmod_f",
+        "csp_ui",
+        "csp_dl",
        "dbd",
+        "db_xattr",
        "du_who",
+        "epilogues",
        "ufavico",
        "forget_ip",
+        "fsnt",
        "hsortn",
        "html_head",
        "html_head_s",
@@ -99,6 +125,7 @@ def vf_vmap() -> dict[str, str]:
        "lg_sba",
        "md_sba",
        "md_hist",
+        "md_nhist",
        "nrand",
        "u2ow",
        "og_desc",
@@ -111,10 +138,18 @@ def vf_vmap() -> dict[str, str]:
        "og_tpl",
        "og_ua",
        "opds_exts",
+        "prologues",
+        "preadmes",
        "put_ck",
        "put_name",
+        "readmes",
+        "rotf_tz",
        "mv_retry",
        "rm_retry",
+        "rss_sort",
+        "rss_fmt_t",
+        "rss_fmt_d",
+        "rw_edit",
        "shr_who",
        "sort",
        "tail_fd",
@@ -122,6 +157,10 @@ def vf_vmap() -> dict[str, str]:
        "tail_tmax",
        "tail_who",
        "tcolor",
+        "th_covers",
+        "th_pregen",
+        "th_qv",
+        "th_qvx",
        "th_spec_p",
        "txt_eol",
        "unlist",
@@ -189,10 +228,13 @@ flagcats = {
        "safededup": "verify on-disk data before using it for dedup",
        "noclone": "take dupe data from clients, even if available on HDD",
        "nodupe": "rejects existing files (instead of linking/cloning them)",
+        "nodupem": "rejects existing files during moves as well",
+        "casechk=auto": "actively prevent case-insensitive filesystem? y/n",
        "chmod_d=755": "unix-permission for new dirs/folders",
        "chmod_f=644": "unix-permission for new files",
        "uid=573": "change owner of new files/folders to unix-user 573",
        "gid=999": "change owner of new files/folders to unix-group 999",
+        "fsnt=auto": "filesystem filename traits (lin/win/mac/auto)",
        "wram": "allow uploading into ramdisks",
        "sparse": "force use of sparse files, mainly for s3-backed storage",
        "nosparse": "deny use of sparse files, mainly for slow storage",
@@ -208,6 +250,7 @@ flagcats = {
        "pk": "forces server-side compression, optional arg: xz,9",
    },
    "upload rules": {
+        "apnd_who=dw": "who can append? (aw/dw/w/no)",
        "maxn=250,600": "max 250 uploads over 15min",
        "maxb=1g,300": "max 1 GiB over 5min (suffixes: b, k, m, g, t)",
        "vmaxb=1g": "total volume size max 1 GiB (suffixes: b, k, m, g, t)",
@@ -254,13 +297,13 @@ flagcats = {
        "no_db_ip": "never store uploader-IP in the db; disables unpost",
        "fat32": "avoid excessive reindexing on android sdcardfs",
        "dbd=[acid|swal|wal|yolo]": "database speed-durability tradeoff",
-        "casechk=auto": "actively prevent case-insensitive filesystem? y/n",
        "xlink": "cross-volume dupe detection / linking (dangerous)",
        "xdev": "do not descend into other filesystems",
        "xvol": "do not follow symlinks leaving the volume root",
        "dotsrch": "show dotfiles in search results",
        "nodotsrch": "hide dotfiles in search results (default)",
        "srch_excl": "exclude search results with URL matching this regex",
+        "db_xattr=user.foo,user.bar": "index file xattrs as media-tags",
    },
    'database, audio tags\n"mte", "mth", "mtp", "mtm" all work the same as -mte, -mth, ...': {
        "mte=artist,title": "media-tags to index/display",
@@ -277,10 +320,14 @@ flagcats = {
        "thsize": "thumbnail res; WxH",
        "crop": "center-cropping (y/n/fy/fn)",
        "th3x": "3x resolution (y/n/fy/fn)",
+        "th_qv=40": "webp/jpg thumbnail quality (10~90)",
+        "th_qvx=40": "jxl thumbnail quality (10~90)",
+        "th_covers=a.jpg,a.png": "filenames to prefer as folder-thumbs",
        "convt": "convert-to-image timeout in seconds",
        "aconvt": "convert-to-audio timeout in seconds",
        "th_spec_p=1": "make spectrograms? 0=never 1=fallback 2=always",
        "ext_th=s=/b.png": "use /b.png as thumbnail for file-extension s",
+        "th_pregen=w,wf": "pregenerate thumbs for these formats",
    },
    "handlers\n(better explained in --help-handlers)": {
        "on404=PY": "handle 404s by executing PY file",
@@ -307,6 +354,8 @@ flagcats = {
        "hsortn": "number of sort-rules to add to media URLs",
        "ufavico=URL": "per-volume favicon (.ico/png/gif/svg)",
        "unlist": "dont list files matching REGEX",
+        "dothidden": "enable support for .hidden files",
+        "dlni": "force-download (no-inline) files on click",
        "html_head=TXT": "includes TXT in the <head>, or @PATH for file at PATH",
        "html_head_s=TXT": "additional static text in the html <head>",
        "tcolor=#fc0": "theme color (a hint for webbrowsers, discord, etc.)",
@@ -316,6 +365,14 @@ flagcats = {
        "norobots": "kindly asks search engines to leave",
        "unlistcr": "don't list read-access in controlpanel",
        "unlistcw": "don't list write-access in controlpanel",
+        "prologues=.prologue.html": "files to embed above/before files",
+        "epilogues=.epilogue.html": "files to embed below/after files",
+        "readmes=readme.md,README.md": "files to embed as readmes",
+        "preadmes=preadme.md,PREADME.md": "files to embed as preadmes",
+        "plainreadme": "embed readme/preadme as monospace plaintext",
+        "plainlogues": "embed prologue/epilogue as sans-serif plaintext",
+        "no_readme": "disable embedding readme/preadme",
+        "no_logues": "disable embedding prologue/epilogue",
        "no_sb_md": "disable js sandbox for markdown files",
        "no_sb_lg": "disable js sandbox for prologue/epilogue",
        "sb_md": "enable js sandbox for markdown files (default)",
@@ -324,7 +381,19 @@ flagcats = {
        "lg_sbf": "list of *logue-sandbox safeguards to disable",
        "md_sba": "value of iframe allow-prop for markdown-sandbox",
        "lg_sba": "value of iframe allow-prop for *logue-sandbox",
+        "csp_ui": "content-security-policy for the web-UI",
+        "csp_dl": "content-security-policy for static files",
        "nohtml": "return html and markdown as text/html",
+        "noscript": "disable most javascript by use of CSP",
+        "ui_noacci": "hide account-info in the UI",
+        "ui_nocpla": "hide cpanel-link in the UI",
+        "ui_nolbar": "hide link-bar in the UI",
+        "ui_nombar": "hide top-menu in the UI",
+        "ui_nonav": "hide navpane+breadcrumbs in the UI",
+        "ui_notree": "hide navpane in the UI",
+        "ui_norepl": "hide repl-button in the UI",
+        "ui_nosrvi": "hide server-info in the UI",
+        "ui_noctxb": "hide context-buttons in the UI",
    },
    "opengraph (discord embeds)": {
        "og": "enable OG (disables hotlinking)",
@@ -345,8 +414,11 @@ flagcats = {
        "opds_exts": "file formats to list in OPDS feeds; leave empty to show everything",
    },
    "textfiles": {
+        "rw_edit=md,txt": "only require read+write to edit .md and .txt",
        "md_no_br": "newline only on double-newline or two tailing spaces",
-        "md_hist": "where to put markdown backups; s=subfolder, v=volHist, n=nope",
+        "md_hist": "where to put text-editor backups; s=subfolder, v=volHist, n=nope",
+        "md_nhist=99": "how many text-editor backups to keep",
+        "show_hist": "allow browsing the .hist folder (old textfiles)",
        "exp": "enable textfile expansion; see --help-exp",
        "exp_md": "placeholders to expand in markdown files; see --help",
        "exp_lg": "placeholders to expand in prologue/epilogue; see --help",
@@ -359,6 +431,12 @@ flagcats = {
        "tail_tmax=30": "kill connection after 30 sec",
        "tail_who=2": "restrict ?tail access (1=admins,2=authed,3=everyone)",
    },
+    "rss": {
+        "rss": "allow '?rss' URL suffix (experimental)",
+        "rss_sort=m": "default sort-order (m/u/n/s)",
+        "rss_fmt_t={fname}": "default title-format",
+        "rss_fmt_d={album},{.tn}": "default description-format",
+    },
    "others": {
        "dots": "allow all users with read-access to\nenable the option to show dotfiles in listings",
        "fk=8": 'generates per-file accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
@@ -366,7 +444,6 @@ flagcats = {
        "dk=8": 'generates per-directory accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
        "dks": "per-directory accesskeys allow browsing into subdirs",
        "dky": 'allow seeing files (not folders) inside a specific folder\nwith "g" perm, and does not require a valid dirkey to do so',
-        "rss": "allow '?rss' URL suffix (experimental)",
        "rmagic": "expensive analysis for mimetype accuracy",
        "shr_who=auth": "who can create shares? no/auth/a",
        "unp_who=2": "unpost only if same... 1=ip+name, 2=ip, 3=name",
@@ -377,8 +454,11 @@ flagcats = {
        "zipmaxt=no": "reply with 'no' if download-as-zip exceeds max",
        "zipmaxu": "zip-size-limit does not apply to authenticated users",
        "nopipe": "disable race-the-beam (download unfinished uploads)",
+        "cachectl=no-cache": "controls caching in webbrowsers",
        "mv_retry": "ms-windows: timeout for renaming busy files",
        "rm_retry": "ms-windows: timeout for deleting busy files",
+        "nospawn": "don't create volume's folder if not exist",
+        "assert_root": "crash on startup if volume's folder not exist",
        "davauth": "ask webdav clients to login for all folders",
        "davrt": "show lastmod time of symlink destination, not the link itself\n(note: this option is always enabled for recursive listings)",
    },
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import importlib
 import sys
@@ -1,15 +1,16 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse
+import json
 import os
 import re
 import time

-from .__init__ import ANYWIN, MACOS
+from .__init__ import ANYWIN, FREEBSD, MACOS, UNIX
 from .authsrv import AXS, VFS, AuthSrv
 from .bos import bos
-from .util import chkcmd, min_ex, undot
+from .util import chkcmd, json_hesc, min_ex, undot

 if True:  # pylint: disable=using-constant-test
    from typing import Optional, Union
@@ -87,7 +88,7 @@ class Fstab(object):

    def _from_sp_mount(self) -> dict[str, str]:
        sptn = r"^.*? on (.*) type ([^ ]+) \(.*"
-        if MACOS:
+        if MACOS or FREEBSD:
            sptn = r"^.*? on (.*) \(([^ ]+), .*"

        ptn = re.compile(sptn)
@@ -117,7 +118,7 @@ class Fstab(object):

    def build_tab(self) -> None:
        self.log("inspecting mtab for changes")
-        dtab = self._from_sp_mount() if MACOS else self._from_proc()
+        dtab = self._from_sp_mount() if UNIX else self._from_proc()

        # keep empirically-correct values if mounttab unchanged
        srctab = str(sorted(dtab.items()))
@@ -127,6 +128,24 @@ class Fstab(object):

        self.log("mtab has changed; reevaluating support for sparse files")

+        try:
+            fuses = [mp for mp, fs in dtab.items() if fs == "fuseblk"]
+            if not fuses or UNIX:
+                raise Exception()
+            try:
+                so, _ = chkcmd(["lsblk", "-nrfo", "FSTYPE,MOUNTPOINT"])  # centos6
+            except:
+                so, _ = chkcmd(["lsblk", "-nrfo", "FSTYPE,MOUNTPOINTS"])  # future
+            for ln in so.split("\n"):
+                zsl = ln.split(" ", 1)
+                if len(zsl) != 2:
+                    continue
+                fs, mp = zsl
+                if mp in fuses:
+                    dtab[mp] = fs
+        except:
+            pass
+
        tab1 = list(dtab.items())
        tab1.sort(key=lambda x: (len(x[0]), x[0]))
        path1, fs1 = tab1[0]
@@ -194,24 +213,54 @@ class Fstab(object):
        return ret.realpath, ""


+_fstab: Optional[Fstab] = None
+winfs = set(("msdos", "vfat", "ntfs", "exfat"))
+# "msdos" = vfat on macos
+
+
 def ramdisk_chk(asrv: AuthSrv) -> None:
    # should have been in authsrv but that's a circular import
+    global _fstab
    mods = []
    ramfs = ("tmpfs", "overlay")
    log = asrv.log_func or print
-    fstab = Fstab(log, asrv.args, False)
+    if not _fstab:
+        _fstab = Fstab(log, asrv.args, False)
    for vn in asrv.vfs.all_nodes.values():
        if not vn.axs.uwrite or "wram" in vn.flags:
            continue
        ap = vn.realpath
        if not ap or os.path.isfile(ap):
            continue
-        fs, mp = fstab.get(ap)
+        fs, mp = _fstab.get(ap)
        mp = "/" + mp.strip("/")
        if fs == "tmpfs" or (mp == "/" and fs in ramfs):
            mods.append((vn.vpath, ap, fs, mp))
            vn.axs.uwrite.clear()
+            vn.axs.umove.clear()
+            for un, ztsp in list(vn.uaxs.items()):
+                zsl = list(ztsp)
+                zsl[1] = False
+                zsl[2] = False
+                vn.uaxs[un] = tuple(zsl)  # type: ignore
    if mods:
        t = "WARNING: write-access was removed from the following volumes because they are not mapped to an actual HDD for storage! All uploaded data would live in RAM only, and all uploaded files would be LOST on next reboot. To allow uploading and ignore this hazard, enable the 'wram' option (global/volflag). List of affected volumes:"
        t2 = ["\n  volume=[/%s], abspath=%r, type=%s, root=%r" % x for x in mods]
        log("vfs", t + "".join(t2) + "\n", 1)
+
+    assume = "mac" if MACOS else "win" if ANYWIN else "lin"
+    for vol in asrv.vfs.all_nodes.values():
+        if not vol.realpath or vol.flags.get("is_file"):
+            continue
+        zs = vol.flags["fsnt"].strip()[:3].lower()
+        if ANYWIN and not zs:
+            zs = "win"
+        if zs in ("lin", "win", "mac"):
+            vol.flags["fsnt"] = zs
+            continue
+        fs = _fstab.get(vol.realpath)[0]
+        fs = "win" if fs in winfs else assume
+        htm = json.loads(vol.js_htm)
+        vol.flags["fsnt"] = vol.js_ls["fsnt"] = htm["fsnt"] = fs
+        vol.js_htm = json_hesc(json.dumps(htm))
+        # md_htm is dontcare; only relevant for up2k request body
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse
 import errno
@@ -19,15 +19,16 @@ from .__init__ import PY2, TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
 from .util import (
-    FN_EMB,
    VF_CAREFUL,
    Daemon,
    ODict,
    Pebkac,
    exclude_dotfiles,
+    exclude_dothidden,
    fsenc,
    ipnorm,
    pybin,
+    read_utf8,
    relchk,
    runhook,
    sanitize_fn,
@@ -86,7 +87,7 @@ class FtpAuth(DummyAuthorizer):
            if args.usernames:
                alts = ["%s:%s" % (username, password)]
            else:
-                alts = password, username
+                alts = [password, username]

            for zs in alts:
                zs = asrv.iacct.get(asrv.ah.hash(zs), "")
@@ -148,6 +149,7 @@ class FtpFs(AbstractedFS):
        self.hub: "SvcHub" = cmd_channel.hub
        self.args = cmd_channel.args
        self.uname = cmd_channel.uname
+        self.log_src = "%s:FTP" % (cmd_channel.cli_ip,)

        self.cwd = "/"  # pyftpdlib convention of leading slash
        self.root = "/var/lib/empty"
@@ -155,8 +157,8 @@ class FtpFs(AbstractedFS):
        self.listdirinfo = self.listdir
        self.chdir(".")

-    def log(self, msg: str, c: Union[int, str] = 0) -> None:
-        self.hub.log("ftpd", msg, c)
+    def clog(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log(self.log_src, msg, c)

    def v2a(
        self,
@@ -174,12 +176,12 @@ class FtpFs(AbstractedFS):
                t = "Unsupported characters in [{}]"
                raise FSE(t.format(vpath), 1)

-            fn = sanitize_fn(fn or "", "")
+            fn = sanitize_fn(fn or "")
            vpath = vjoin(rd, fn)
            vfs, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
            if (
                w
-                and fn.lower() in FN_EMB
+                and fn.lower() in vfs.flags["emb_all"]
                and self.h.uname not in vfs.axs.uread
                and "wo_up_readme" not in vfs.flags
            ):
@@ -198,14 +200,16 @@ class FtpFs(AbstractedFS):
                if not avfs:
                    raise FSE(t.format(vpath), 1)

-                cr, cw, cm, cd, _, _, _, _ = avfs.can_access("", self.h.uname)
+                cr, cw, cm, cd, _, _, _, _, _ = avfs.uaxs[self.h.uname]
                if r and not cr or w and not cw or m and not cm or d and not cd:
                    raise FSE(t.format(vpath), 1)
+            else:
+                ap = vfs.canonical(rem, False)

            if "bcasechk" in vfs.flags and not vfs.casechk(rem, True):
                raise FSE("No such file or directory", 1)

-            return os.path.join(vfs.realpath, rem), vfs, rem
+            return ap, vfs, rem
        except Pebkac as ex:
            raise FSE(str(ex))

@@ -235,6 +239,8 @@ class FtpFs(AbstractedFS):
        return True

    def open(self, filename: str, mode: str) -> typing.IO[Any]:
+        self.clog("open(%r,%r) @%s" % (filename, mode, self.uname))
+
        r = "r" in mode
        w = "w" in mode or "a" in mode or "+" in mode

@@ -249,9 +255,36 @@ class FtpFs(AbstractedFS):
                need_unlink = False
                td = 0

-        if w and need_unlink:
+            xbu = vfs.flags.get("xbu")
+            if xbu:
+                hr = runhook(
+                    self.clog,
+                    None,
+                    self.hub.up2k,
+                    "xbu.ftp",
+                    xbu,
+                    ap,
+                    filename,
+                    "",
+                    "",
+                    "",
+                    0,
+                    0,
+                    "1.3.8.7",
+                    time.time(),
+                    None,
+                )
+                t = hr.get("rejectmsg") or ""
+                if t or hr.get("rc") != 0:
+                    if not t:
+                        t = "upload blocked by xbu server config: %r" % (filename,)
+                    self.clog(t, 3)
+                    raise FSE(t)
+
+        if w and need_unlink:  # type: ignore  # !rm
+            assert td  # type: ignore  # !rm
            if td >= -1 and td <= self.args.ftp_wt:
-                # within permitted timeframe; unlink and accept
+                # within permitted timeframe; allow overwrite or resume
                do_it = True
            elif self.args.no_del or self.args.ftp_no_ow:
                # file too old, or overwrite not allowed; reject
@@ -268,12 +301,15 @@ class FtpFs(AbstractedFS):
            if not do_it:
                raise FSE("File already exists")

-            wunlink(self.log, ap, VF_CAREFUL)
+            # Don't unlink file for append mode
+            elif "a" not in mode:
+                wunlink(self.clog, ap, VF_CAREFUL)

        ret = open(fsenc(ap), mode, self.args.iobuf)
        if w and "fperms" in vfs.flags:
            set_fperms(ret, vfs.flags)

+        self.clog(" `-- %r" % (ap,))
        return ret

    def chdir(self, path: str) -> None:
@@ -299,10 +335,12 @@ class FtpFs(AbstractedFS):
        self.cwd = nwd

    def mkdir(self, path: str) -> None:
+        self.clog("mkdir(%r) @%s" % (path, self.uname))
        ap, vfs, _ = self.rv2a(path, w=True)
        bos.makedirs(ap, vf=vfs.flags)  # filezilla expects this

    def listdir(self, path: str) -> list[str]:
+        self.clog("ls(%r) @%s" % (path, self.uname))
        vpath = join(self.cwd, path)
        try:
            ap, vfs, rem = self.v2a(vpath, True, False)
@@ -320,7 +358,10 @@ class FtpFs(AbstractedFS):
            vfs_ls.extend(vfs_virt.keys())

            if self.uname not in vfs.axs.udot:
-                vfs_ls = exclude_dotfiles(vfs_ls)
+                if "dothidden" in vfs.flags and ".hidden" in [x[0] for x in vfs_ls]:
+                    vfs_ls = exclude_dothidden(vfs_ls, fsroot)
+                else:
+                    vfs_ls = exclude_dotfiles(vfs_ls)

            vfs_ls.sort()
            return vfs_ls
@@ -349,6 +390,7 @@ class FtpFs(AbstractedFS):
            return ret

    def rmdir(self, path: str) -> None:
+        self.clog("rmdir(%r) @%s" % (path, self.uname))
        ap = self.rv2a(path, d=True)[0]
        try:
            bos.rmdir(ap)
@@ -357,6 +399,7 @@ class FtpFs(AbstractedFS):
                raise

    def remove(self, path: str) -> None:
+        self.clog("rm(%r) @%s" % (path, self.uname))
        if self.args.no_del:
            raise FSE("The delete feature is disabled in server config")

@@ -367,6 +410,7 @@ class FtpFs(AbstractedFS):
            raise FSE(str(ex))

    def rename(self, src: str, dst: str) -> None:
+        self.clog("mv(%r,%r) @%s" % (src, dst, self.uname))
        if self.args.no_mv:
            raise FSE("The rename/move feature is disabled in server config")

@@ -396,6 +440,7 @@ class FtpFs(AbstractedFS):
            return st

    def utime(self, path: str, timeval: float) -> None:
+        self.clog("utime(%r,%r) @%s" % (path, timeval, self.uname))
        ap = self.rv2a(path, w=True)[0]
        bos.utime_c(logging.warning, ap, int(timeval), False)

@@ -403,6 +448,9 @@ class FtpFs(AbstractedFS):
        ap = self.rv2a(path)[0]
        return bos.stat(ap)

+    def readlink(self, *a) -> None:
+        raise OSError(errno.EINVAL, "nope")
+
    def isfile(self, path: str) -> bool:
        try:
            st = self.stat(path)
@@ -445,9 +493,13 @@ class FtpFs(AbstractedFS):
    def get_user_by_uid(self, uid: int) -> str:
        return "root"

-    def get_group_by_uid(self, gid: int) -> str:
+    def get_group_by_gid(self, gid: int) -> str:
        return "root"

+    def mkstemp(self, *a, **ka):
+        self.clog("mkstemp%r @%s" % (a, self.uname))
+        raise FSE("nope")
+

 class FtpHandler(FTPHandler):
    abstracted_fs = FtpFs
@@ -476,6 +528,7 @@ class FtpHandler(FTPHandler):
            return

        self.cli_ip = cip
+        self.log_src = "%s:FTP" % (cip,)

        # abspath->vpath mapping to resolve log_transfer paths
        self.vfs_map: dict[str, str] = {}
@@ -483,6 +536,15 @@ class FtpHandler(FTPHandler):
        # reduce non-debug logging
        self.log_cmds_list = [x for x in self.log_cmds_list if x not in ("CWD", "XCWD")]

+    def clog(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log(self.log_src, msg, c)
+
+    def ftp_NOOP(self, *a) -> None:
+        self.respond("200 pong", logging.debug)
+
+    def ftp_STOU(self, *a) -> None:
+        self.respond("550 nope", logging.warning)
+
    def ftp_STOR(self, file: str, mode: str = "w") -> Any:
        # Optional[str]
        vp = join(self.fs.cwd, file).lstrip("/")
@@ -495,7 +557,7 @@ class FtpHandler(FTPHandler):
        xbu = vfs.flags.get("xbu")
        if xbu:
            hr = runhook(
-                None,
+                self.clog,
                None,
                self.hub.up2k,
                "xbu.ftpd",
@@ -509,10 +571,10 @@ class FtpHandler(FTPHandler):
                0,
                self.cli_ip,
                time.time(),
-                "",
+                None,
            )
            t = hr.get("rejectmsg") or ""
-            if t or not hr:
+            if t or hr.get("rc") != 0:
                if not t:
                    t = "Upload blocked by xbu server config: %r" % (vp,)
                self.respond("550 %s" % (t,), logging.info)
@@ -582,17 +644,25 @@ class Ftpd(object):
                print(t.format(pybin))
                sys.exit(1)

+            if self.args.certkey:
+                h1.keyfile = self.args.certkey
            h1.certfile = self.args.cert
            h1.tls_control_required = True
            h1.tls_data_required = True

            hs.append([h1, self.args.ftps])

+        zs = self.args.ftp_banner.replace("\\n", "\n")
+        if zs.startswith("@"):
+            zs = read_utf8(None, zs[1:], False)
+        banner = zs.replace("\r", "").replace("\n", "\r\n").strip()
+
        for h_lp in hs:
            h2, lp = h_lp
            FtpHandler.hub = h2.hub = hub
            FtpHandler.args = h2.args = hub.args
            FtpHandler.authorizer = h2.authorizer = FtpAuth(hub)
+            FtpHandler.banner = banner

            if self.args.ftp_pr:
                p1, p2 = [int(x) for x in self.args.ftp_pr.split("-")]
@@ -612,7 +682,7 @@ class Ftpd(object):
        lgr = logging.getLogger("pyftpdlib")
        lgr.setLevel(logging.DEBUG if self.args.ftpv else logging.INFO)

-        ips = self.args.i
+        ips = self.args.ftp_i
        if "::" in ips:
            ips.append("0.0.0.0")

@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse  # typechk
 import os
@@ -21,12 +21,8 @@ from . import util as Util
 from .__init__ import TYPE_CHECKING, EnvParams
 from .authsrv import AuthSrv  # typechk
 from .httpcli import HttpCli
-from .ico import Ico
-from .mtag import HAVE_FFMPEG
-from .th_cli import ThumbCli
-from .th_srv import HAVE_PIL, HAVE_VIPS
 from .u2idx import U2idx
-from .util import HMaccas, NetMap, shut_socket
+from .util import HMaccas, NetMap, min_ex, shut_socket

 if True:  # pylint: disable=using-constant-test
    from typing import Optional, Pattern, Union
@@ -62,27 +58,22 @@ class HttpConn(object):
        self.ipu_iu: Optional[dict[str, str]] = hsrv.ipu_iu
        self.ipu_nm: Optional[NetMap] = hsrv.ipu_nm
        self.ipa_nm: Optional[NetMap] = hsrv.ipa_nm
+        self.ipar_nm: Optional[NetMap] = hsrv.ipar_nm
        self.xff_nm: Optional[NetMap] = hsrv.xff_nm
        self.xff_lan: NetMap = hsrv.xff_lan  # type: ignore
        self.iphash: HMaccas = hsrv.broker.iphash
        self.bans: dict[str, int] = hsrv.bans
        self.aclose: dict[str, int] = hsrv.aclose

-        enth = (HAVE_PIL or HAVE_VIPS or HAVE_FFMPEG) and not self.args.no_thumb
-        self.thumbcli: Optional[ThumbCli] = ThumbCli(hsrv) if enth else None  # mypy404
-        self.ico: Ico = Ico(self.args)  # mypy404
-
        self.t0: float = time.time()  # mypy404
        self.freshen_pwd: float = 0.0
        self.stopping = False
        self.nreq: int = -1  # mypy404
        self.nbyte: int = 0  # mypy404
        self.u2idx: Optional[U2idx] = None
+        self.lf_url: Optional[Pattern[str]] = self.args.lf_url
        self.log_func: "Util.RootLogger" = hsrv.log  # mypy404
        self.log_src: str = "httpconn"  # mypy404
-        self.lf_url: Optional[Pattern[str]] = (
-            re.compile(self.args.lf_url) if self.args.lf_url else None
-        )  # mypy404
        self.set_rproxy()

    def shutdown(self) -> None:
@@ -166,7 +157,7 @@ class HttpConn(object):
            try:
                assert ssl  # type: ignore  # !rm
                ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
-                ctx.load_cert_chain(self.args.cert)
+                ctx.load_cert_chain(self.args.cert, self.args.certkey)
                if self.args.ssl_ver:
                    ctx.options &= ~self.args.ssl_flags_en
                    ctx.options |= self.args.ssl_flags_de
@@ -203,12 +194,12 @@ class HttpConn(object):
            except Exception as ex:
                em = str(ex)

-                if "ALERT_CERTIFICATE_UNKNOWN" in em:
-                    # android-chrome keeps doing this
-                    pass
+                if "ALERT_" in em:
+                    self.log("client refused our TLS cert or config: " + em, c=6)

                else:
-                    self.log("handshake\033[0m " + em, c=5)
+                    t = "https-handshake failed, probably due to client:\n"
+                    self.log(t + min_ex(), c=5)

                return

@@ -221,6 +212,21 @@ class HttpConn(object):
            if not self.cli.run():
                return

+            if self.sr.te == 1:
+                self.log("closing socket (leftover TE)", "90")
+                return
+
+            if (
+                "content-length" in self.cli.headers
+                and int(self.cli.headers["content-length"]) != self.sr.nb
+            ):
+                self.log("closing socket (CL mismatch)", "90")
+                return
+
+            # note: proxies reject PUT sans Content-Length; illegal for HTTP/1.1
+
+            self.sr.nb = self.sr.te = 0
+
            if self.u2idx:
                self.hsrv.put_u2idx(str(self.addr), self.u2idx)
                self.u2idx = None
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import hashlib
 import math
@@ -55,8 +55,14 @@ except SyntaxError:
    )
    sys.exit(1)

+from .authsrv import LEELOO_DALLAS
 from .httpconn import HttpConn
+from .ico import Ico
 from .metrics import Metrics
+from .mtag import HAVE_FFMPEG
+from .sutil import gfilter2
+from .th_cli import ThumbCli
+from .th_srv import HAVE_PIL, HAVE_VIPS
 from .u2idx import U2idx
 from .util import (
    E_SCK,
@@ -133,16 +139,20 @@ class HttpSrv(object):
        self.bans: dict[str, int] = {}
        self.aclose: dict[str, int] = {}

+        self.thumbcli: Optional[ThumbCli] = None
+        self.ico: Ico = Ico(self.args)
+
        dli: dict[str, tuple[float, int, "VFS", str, str]] = {}  # info
        dls: dict[str, tuple[float, int]] = {}  # state
        self.dli = self.tdli = dli
        self.dls = self.tdls = dls
-        self.iiam = '<img src="%s.cpr/iiam.gif?cache=i" />' % (self.args.SRS,)
+        self.iiam = '<img src="%s.cpr/w/iiam.gif?cache=i" />' % (self.args.SRS,)

        self.bound: set[tuple[str, int]] = set()
        self.name = "hsrv" + nsuf
        self.mutex = threading.Lock()
        self.u2mutex = threading.Lock()
+        self.bad_ver = False
        self.stopping = False

        self.tp_nthr = 0  # actual
@@ -170,6 +180,8 @@ class HttpSrv(object):
        self.u2idx_free: dict[str, U2idx] = {}
        self.u2idx_n = 0

+        self.wopi_files: dict[str, dict[str, str]] = {}
+
        assert jinja2  # type: ignore  # !rm
        env = jinja2.Environment()
        env.loader = jinja2.FunctionLoader(lambda f: load_jinja2_resource(self.E, f))
@@ -185,9 +197,11 @@ class HttpSrv(object):
            "shares",
            "splash",
            "svcs",
+            "wopi",
        ]
        self.j2 = {x: env.get_template(x + ".html") for x in jn}
        self.j2["opds"] = env.get_template("opds.xml")
+        self.j2["opds_osd"] = env.get_template("opds_osd.xml")
        self.prism = has_resource(self.E, "web/deps/prism.js.gz")

        if self.args.ipu:
@@ -201,6 +215,7 @@ class HttpSrv(object):
            self.ipr = None

        self.ipa_nm = build_netmap(self.args.ipa)
+        self.ipar_nm = build_netmap(self.args.ipar)
        self.xff_nm = build_netmap(self.args.xff_src)
        self.xff_lan = build_netmap("lan")

@@ -227,16 +242,23 @@ class HttpSrv(object):
            if self.args.log_thrs:
                start_log_thrs(self.log, self.args.log_thrs, nid)

-        self.th_cfg: dict[str, set[str]] = {}
-        Daemon(self.post_init, "hsrv-init2")
+        if (HAVE_PIL or HAVE_VIPS or HAVE_FFMPEG) and not self.args.no_thumb:
+            Daemon(self.post_init, "hsrv-init2")

    def post_init(self) -> None:
        try:
            x = self.broker.ask("thumbsrv.getcfg")
-            self.th_cfg = x.get()
+            self.thumbcli = ThumbCli(self, x.get())
        except:
            pass

+    def set_th_cfg(self, c: dict[str, set[str]], opts: tuple[bool]) -> None:
+        self.args.th_no_jxl = opts[0]
+        self.thumbcli = ThumbCli(self, c)
+
+    def set_bad_ver(self) -> None:
+        self.bad_ver = True
+
    def set_netdevs(self, netdevs: dict[str, Netdev]) -> None:
        ips = set()
        for ip, _ in self.bound:
@@ -296,7 +318,7 @@ class HttpSrv(object):

        self.srvs.append(sck)
        self.bound.add((ip, port))
-        self.nclimax = math.ceil(self.args.nc * 1.0 / nlisteners)
+        self.nclimax = -int(-self.args.nc // nlisteners)
        Daemon(
            self.thr_listen,
            "httpsrv-n{}-listen-{}-{}".format(self.nid or "0", ip, port),
@@ -383,8 +405,8 @@ class HttpSrv(object):
                if nloris < nconn / 2:
                    continue

-                t = "slowloris (idle-conn): {} banned for {} min"
-                self.log(self.name, t.format(ip, self.args.loris, nclose), 1)
+                t = "slow%s (idle-conn): %s banned for %d min"  # slowloris
+                self.log(self.name, t % ("loris", ip, self.args.loris), 1)
                self.bans[ip] = int(time.time() + self.args.loris * 60)

            if self.args.log_conn:
@@ -640,3 +662,73 @@ class HttpSrv(object):

        self.tdli = dli
        self.tdls = dls
+
+    def pregen_thumbs(self) -> None:
+        Daemon(self._pregen_thumbs, "th_pregen")
+
+    def _pregen_thumbs(self) -> None:
+        def log(msg, n):
+            self.log("thumb-pregen", msg, n)
+
+        if getattr(self, "_pregen", False):
+            log("already running", 1)
+            return
+
+        self._pregen = True
+
+        for n in range(9999999):
+            x = self.broker.ask("up2k.is_busy")
+            zb, zi = x.get()
+            if zi:
+                break
+            if not n:
+                log("waiting for up2k to finish initializing", 6)
+            time.sleep(1 if n < 10 else 5 if n < 300 else 15)
+
+        if not self.thumbcli:
+            log("no thumbcli", 1)
+            return
+
+        if self.args.th_pre_rl:
+            try:
+                self.broker.hub.thumbsrv.log = self.broker.hub.thumbsrv._slog
+            except:
+                pass
+
+        nfiles = 0
+        t0 = time.time()
+        scandir = not self.args.no_scandir
+        for vn in self.asrv.vfs.all_nodes.values():
+            fmts = vn.flags.get("th_pregen", "")
+            if not fmts:
+                continue
+            log("starting for volume /%s" % (vn.vpath,), 6)
+            g = vn.walk("", "/", [], LEELOO_DALLAS, [[True]], 2, scandir, False, False)
+            g = gfilter2(g, self, vn.vpath, fmts.split(","))
+            for f in g:
+                nfiles += 1
+                if not nfiles % 256:
+                    now = time.time()
+                    for n in range(9999999):
+                        x = self.broker.ask("up2k.is_busy")
+                        zb, zi = x.get()
+                        if not zb:
+                            if n:
+                                t0 += time.time() - now
+                            break
+                        if not n:
+                            log("waiting for up2k to finish indexing", 6)
+                        time.sleep(5)
+
+        if self.args.th_pre_rl:
+            try:
+                self.broker.hub.thumbsrv.log = self.broker.hub.thumbsrv._log
+            except:
+                pass
+
+        t = "finished; %d files in %d seconds"
+        log(t % (nfiles, time.time() - t0), 6)
+        self._pregen = False
+
+        if self.args.exit == "thgen":
+            self.broker.say("sigterm")
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse  # typechk
 import colorsys
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import errno
 import os
@@ -16,7 +16,7 @@ from .multicast import MC_Sck, MCast
 from .util import IP6_LL, CachedSet, Daemon, Netdev, list_ips, min_ex

 try:
-    if os.getenv("PRTY_SYS_ALL") or os.getenv("PRTY_SYS_DNSLIB"):
+    if os.environ.get("PRTY_SYS_ALL") or os.environ.get("PRTY_SYS_DNSLIB"):
        raise ImportError()
    from .stolen.dnslib import (
        AAAA,
@@ -66,7 +66,7 @@ if TYPE_CHECKING:
 if True:  # pylint: disable=using-constant-test
    from typing import Any, Optional, Union

-if os.getenv("PRTY_MODSPEC"):
+if os.environ.get("PRTY_MODSPEC"):
    from inspect import getsourcefile

    print("PRTY_MODSPEC: dnslib:", getsourcefile(A))
@@ -110,7 +110,7 @@ class MDNS(MCast):
        self.ttl = 300

        if not self.args.zm_nwa_1 and DNS_VND:
-            set_avahi_379()
+            set_avahi_379()  # type: ignore

        zs = self.args.zm_fqdn or (self.args.name + ".local")
        zs = zs.replace("--name", self.args.name).rstrip(".") + "."
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import json
 import time
@@ -17,10 +17,10 @@ class Metrics(object):
        self.hsrv = hsrv

    def tx(self, cli: "HttpCli") -> bool:
-        if not cli.avol:
+        args = cli.args
+        if not cli.avol and cli.uname.lower() not in args.stats_u_set:
            raise Pebkac(403, "'stats' not allowed for user " + cli.uname)

-        args = cli.args
        if not args.stats:
            raise Pebkac(403, "the stats feature is not enabled in server config")

@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse
 import json
@@ -15,8 +15,12 @@ from .authsrv import VFS
 from .bos import bos
 from .util import (
    FFMPEG_URL,
+    LOG,
    REKOBO_LKEY,
+    SCWD,
    VF_CAREFUL,
+    absreal,
+    expand_osenv_c,
    fsenc,
    gzip,
    min_ex,
@@ -45,29 +49,42 @@ except:
    HAVE_MUTAGEN = False


-def have_ff(scmd: str) -> bool:
-    if ANYWIN:
+def have_ff(name: str) -> list[bytes]:
+    uname = name.upper()
+    if os.environ.get("PRTY_NO_" + uname):
+        return []
+
+    ebin = os.environ.get("PRTY_%s_BIN" % (uname,))
+    try:
+        scmd = (ebin or name).decode("utf-8")
+    except:
+        scmd: str = ebin or name
+
+    if ANYWIN and not ebin:
        scmd += ".exe"

    if PY2:
-        print("# checking {}".format(scmd))
-        acmd = (scmd + " -version").encode("ascii").split(b" ")
+        print("# checking %s" % (scmd,))
+        bcmd = scmd.encode("utf-8")
        try:
-            sp.Popen(acmd, stdout=sp.PIPE, stderr=sp.PIPE).communicate()
-            return True
+            sp.Popen([bcmd, b"-version"], stdout=sp.PIPE, stderr=sp.PIPE).communicate()
+            return [bcmd]
        except:
-            return False
+            return []
    else:
-        return bool(shutil.which(scmd))
+        ret = shutil.which(scmd)
+        return [absreal(ret).encode("utf-8")] if ret else []


-HAVE_FFMPEG = not os.environ.get("PRTY_NO_FFMPEG") and have_ff("ffmpeg")
-HAVE_FFPROBE = not os.environ.get("PRTY_NO_FFPROBE") and have_ff("ffprobe")
+HAVE_FFMPEG = have_ff("ffmpeg")
+HAVE_FFPROBE = have_ff("ffprobe")
+TH_BWRAP = []

-CBZ_PICS = set("png jpg jpeg gif bmp tga tif tiff webp avif".split())
+CBZ_PICS = set("png jpg jpeg gif bmp tga tif tiff webp avif jxl".split())
 CBZ_01 = re.compile(r"(^|[^0-9v])0+[01]\b")

 FMT_AU = set("mp3 ogg flac wav".split())
+M4A = set("aac m4a m4b m4r".split())


 class MParser(object):
@@ -85,7 +102,7 @@ class MParser(object):

        while True:
            try:
-                bp = os.path.expanduser(args)
+                bp = os.path.expanduser(expand_osenv_c(args))
                if WINDOWS:
                    bp = uncyg(bp)

@@ -132,6 +149,7 @@ class MParser(object):
 def au_unpk(
    log: "NamedLogger", fmt_map: dict[str, str], abspath: str, vn: Optional[VFS] = None
 ) -> str:
+    fd = 0
    ret = ""
    maxsz = 1024 * 1024 * 64
    try:
@@ -168,12 +186,12 @@ def au_unpk(
            znil = [x for x in znil if "cover" in x[0]] or znil
            znil = [x for x in znil if CBZ_01.search(x[0])] or znil
            t = "cbz: %d files, %d hits" % (nf, len(znil))
+            if not znil:
+                raise Exception("no images inside cbz")
            using = sorted(znil)[0][1].filename
            if znil:
                t += ", using " + using
            log(t)
-            if not znil:
-                raise Exception("no images inside cbz")
            fi = zf.open(using)

        elif pk == "epub":
@@ -185,6 +203,7 @@ def au_unpk(

        fsz = 0
        with os.fdopen(fd, "wb") as fo:
+            fd = 0
            while True:
                buf = fi.read(32768)
                if not buf:
@@ -199,6 +218,8 @@ def au_unpk(
        return ret

    except Exception as ex:
+        if fd:
+            os.close(fd)
        if ret:
            t = "failed to decompress file %r: %r"
            log(t % (abspath, ex))
@@ -208,18 +229,37 @@ def au_unpk(
        return abspath


+def bwrap(prog: bytes, ap_in: bytes, ap_out: bytes) -> list[bytes]:
+    if not TH_BWRAP:
+        return [prog]
+    ret = TH_BWRAP + [b"--ro-bind", prog, prog, b"--ro-bind", ap_in, ap_in]
+    if ap_out:
+        zs = ap_out.rsplit(b"/", 1)[0]
+        ret += [b"--bind", zs, zs]
+    ret.append(prog)
+    return ret
+
+
+def bwrap_fail(serr: str) -> None:
+    if "bwrap:" in serr:
+        LOG[0]("root", "ffmpeg failed due to --th-bwrap;\n  " + serr, 3)
+
+
 def ffprobe(
    abspath: str, timeout: int = 60
 ) -> tuple[dict[str, tuple[int, Any]], dict[str, list[Any]], list[Any], dict[str, Any]]:
-    cmd = [
-        b"ffprobe",
+    # ffprobe -hide_banner -show_streams -show_format --
+    bap = fsenc(abspath)
+    cmd = bwrap(HAVE_FFPROBE[0], bap, b"") + [
        b"-hide_banner",
        b"-show_streams",
        b"-show_format",
        b"--",
-        fsenc(abspath),
+        bap,
    ]
-    rc, so, se = runcmd(cmd, timeout=timeout, nice=True, oom=200)
+    rc, so, se = runcmd(cmd, cwd=SCWD, timeout=timeout, nice=True, oom=200)
+    if rc and TH_BWRAP:
+        bwrap_fail(se)
    retchk(rc, cmd, se)
    return parse_ffprobe(so)

@@ -259,7 +299,7 @@ def parse_ffprobe(
    md: dict[str, list[Any]] = {}  # raw tags

    is_audio = fmt.get("format_name") in FMT_AU
-    if fmt.get("filename", "").split(".")[-1].lower() in ["m4a", "aac"]:
+    if fmt.get("filename", "").split(".")[-1].lower() in M4A:
        is_audio = True

    # if audio file, ensure audio stream appears first
@@ -385,7 +425,7 @@ def get_cover_from_epub(log: "NamedLogger", abspath: str) -> Optional[IO[bytes]]
    from .dxml import parse_xml

    try:
-        from urlparse import urljoin  # Python2
+        from urlparse import urljoin  # type: ignore  # Python2
    except ImportError:
        from urllib.parse import urljoin  # Python3

@@ -465,6 +505,8 @@ class MTag(object):
            "ffprobe" if args.no_mutagen or (HAVE_FFPROBE and EXE) else "mutagen"
        )
        self.can_ffprobe = HAVE_FFPROBE and not args.no_mtag_ff
+        self.read_xattrs = args.have_db_xattr
+        self.get = self._get_xattr if self.read_xattrs else self._get_main
        mappings = args.mtm
        or_ffprobe = " or FFprobe"

@@ -486,7 +528,13 @@ class MTag(object):
                msg = "found FFprobe but it was disabled by --no-mtag-ff"
                self.log(msg, c=3)

+        if self.read_xattrs and not self.usable:
+            t = "don't have the necessary dependencies to read conventional media tags, but will read xattrs"
+            self.log(t)
+            self.usable = True
+
        if not self.usable:
+            self._get = None
            if EXE:
                t = "copyparty.exe cannot use mutagen; need ffprobe.exe to read media tags: "
                self.log(t + FFMPEG_URL)
@@ -523,7 +571,7 @@ class MTag(object):
            ],
            ".tn": ["tracknumber", "trck", "trkn", "track"],
            "genre": ["genre", "tcon", "\u00a9gen"],
-            "date": [
+            "tdate": [
                "original-release-date",
                "release-date",
                "date",
@@ -645,7 +693,36 @@ class MTag(object):

        return r1

-    def get(self, abspath: str) -> dict[str, Union[str, float]]:
+    def _get_xattr(
+        self, abspath: str, vf: dict[str, Any]
+    ) -> dict[str, Union[str, float]]:
+        ret = self._get_main(abspath, vf) if self._get else {}
+        if "db_xattr_no" in vf:
+            try:
+                neg = vf["db_xattr_no"]
+                zsl = os.listxattr(abspath)
+                zsl = [x for x in zsl if x not in neg]
+                for xattr in zsl:
+                    zb = os.getxattr(abspath, xattr)
+                    ret[xattr] = zb.decode("utf-8", "replace")
+            except:
+                self.log("failed to read xattrs from [%s]\n%s" % (abspath, min_ex()), 3)
+        elif "db_xattr_yes" in vf:
+            for xattr in vf["db_xattr_yes"]:
+                if "=" in xattr:
+                    xattr, name = xattr.split("=", 1)
+                else:
+                    name = xattr
+                try:
+                    zs = os.getxattr(abspath, xattr)
+                    ret[name] = zs.decode("utf-8", "replace")
+                except:
+                    pass
+        return ret
+
+    def _get_main(
+        self, abspath: str, vf: dict[str, Any]
+    ) -> dict[str, Union[str, float]]:
        ext = abspath.split(".")[-1].lower()
        if ext not in self.args.au_unpk:
            return self._get(abspath)
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import socket
 import time
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse
 import base64
@@ -1,10 +1,10 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os

 try:
-    if os.getenv("PRTY_SYS_ALL") or os.getenv("PRTY_SYS_QRCG"):
+    if os.environ.get("PRTY_SYS_ALL") or os.environ.get("PRTY_SYS_QRCG"):
        raise ImportError()
    from .stolen.qrcodegen import QrCode

@@ -14,7 +14,7 @@ except ImportError:
    VENDORED = False
    from qrcodegen import QrCode

-if os.getenv("PRTY_MODSPEC"):
+if os.environ.get("PRTY_MODSPEC"):
    from inspect import getsourcefile

    print("PRTY_MODSPEC: qrcode:", getsourcefile(QrCode))
@@ -0,0 +1,913 @@
+# coding: utf-8
+from __future__ import division, print_function, unicode_literals
+
+import errno
+import hashlib
+import logging
+import os
+import select
+import socket
+import time
+from threading import ExceptHookArgs
+
+import paramiko
+import paramiko.common
+import paramiko.sftp_attr
+from paramiko.common import AUTH_FAILED, AUTH_SUCCESSFUL
+from paramiko.sftp import (
+    SFTP_FAILURE,
+    SFTP_NO_SUCH_FILE,
+    SFTP_OK,
+    SFTP_OP_UNSUPPORTED,
+    SFTP_PERMISSION_DENIED,
+)
+from queue import Queue
+
+from .__init__ import ANYWIN, TYPE_CHECKING
+from .authsrv import LEELOO_DALLAS, VFS, AuthSrv
+from .bos import bos
+from .util import (
+    VF_CAREFUL,
+    Daemon,
+    ODict,
+    Pebkac,
+    exclude_dotfiles,
+    exclude_dotfiles_ls,
+    exclude_dothidden,
+    exclude_dothidden_ls,
+    ipnorm,
+    min_ex,
+    read_utf8,
+    relchk,
+    runhook,
+    sanitize_fn,
+    ub64enc,
+    undot,
+    vjoin,
+    wunlink,
+)
+
+if TYPE_CHECKING:
+    from .svchub import SvcHub
+
+if True:  # pylint: disable=using-constant-test
+    from typing import Any, BinaryIO, Optional, Union
+
+SATTR = paramiko.sftp_attr.SFTPAttributes
+
+
+class SSH_Srv(paramiko.ServerInterface):
+    def __init__(self, hub: "SvcHub", addr: Any):
+        self.hub = hub
+        self.args = args = hub.args
+        self.log_func = hub.log
+        self.uname = "*"
+
+        self.addr = addr
+        self.ip = addr[0]
+        if self.ip.startswith("::ffff:"):
+            self.ip = self.ip[7:]
+
+        zsl = []
+        if args.sftp_anon:
+            zsl.append("none")
+        if args.sftp_key2u:
+            zsl.append("publickey")
+        if args.sftp_pw or args.sftp_anon:
+            zsl.append("password")
+        self._auths = ",".join(zsl)
+
+    def log(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log("sftp:%s" % (self.ip,), msg, c)
+
+    def get_allowed_auths(self, username: str) -> str:
+        return self._auths
+
+    def get_banner(self) -> tuple[Optional[str], Optional[str]]:
+        if self.args.sftpv:
+            self.log("get_banner")
+        t = self.args.sftp_banner
+        if not t:
+            return (None, None)
+        if t.startswith("@"):
+            t = read_utf8(self.log, t[1:], False)
+        if t and not t.endswith("\n"):
+            t += "\n"
+        return (t, "en-US")
+
+    def check_channel_request(self, kind: str, chanid: int) -> int:
+        if self.args.sftpv:
+            self.log("channel-request: %r, %r" % (kind, chanid))
+        if kind == "session":
+            return paramiko.common.OPEN_SUCCEEDED
+        return paramiko.common.OPEN_FAILED_ADMINISTRATIVELY_PROHIBITED
+
+    def check_auth_none(self, username: str) -> int:
+        try:
+            return self._check_auth_none(username)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return AUTH_FAILED
+
+    def _check_auth_none(self, uname: str) -> int:
+        args = self.args
+        if uname != args.sftp_anon or not uname:
+            return AUTH_FAILED
+
+        ipn = ipnorm(self.ip)
+        bans = self.hub.bans
+        if ipn in bans:
+            rt = bans[ipn] - time.time()
+            if rt < 0:
+                self.log("client unbanned")
+                del bans[ipn]
+            else:
+                self.log("client is banned")
+                return AUTH_FAILED
+
+        self.uname = "*"
+        self.log("auth-none OK: *")
+        return AUTH_SUCCESSFUL
+
+    def check_auth_password(self, username: str, password: str) -> int:
+        try:
+            return self._check_auth_password(username, password)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return AUTH_FAILED
+
+    def _check_auth_password(self, uname: str, pw: str) -> int:
+        args = self.args
+        if args.sftpv:
+            logpw = pw
+            if args.log_badpwd == 0:
+                logpw = ""
+            elif args.log_badpwd == 2:
+                zb = hashlib.sha512(pw.encode("utf-8", "replace")).digest()
+                logpw = "%" + ub64enc(zb[:12]).decode("ascii")
+            self.log("auth-pw: %r, %r" % (uname, logpw))
+
+        ipn = ipnorm(self.ip)
+        bans = self.hub.bans
+        if ipn in bans:
+            rt = bans[ipn] - time.time()
+            if rt < 0:
+                self.log("client unbanned")
+                del bans[ipn]
+            else:
+                self.log("client is banned")
+                return AUTH_FAILED
+
+        anon = args.sftp_anon
+        if anon and uname == anon:
+            self.uname = "*"
+            self.log("auth-pw OK: *")
+            return AUTH_SUCCESSFUL
+
+        if not args.sftp_pw:
+            return AUTH_FAILED
+
+        if args.usernames:
+            alts = ["%s:%s" % (uname, pw)]
+        else:
+            alts = [pw, uname]
+
+        attempt = "%s:%s" % (uname, pw)
+        uname = ""
+        asrv = self.hub.asrv
+        for zs in alts:
+            zs = asrv.iacct.get(asrv.ah.hash(zs), "")
+            if zs:
+                uname = zs
+                break
+
+        if args.ipu and uname == "*":
+            uname = args.ipu_iu[args.ipu_nm.map(self.ip)]
+        if args.ipr and uname in args.ipr_u:
+            if not args.ipr_u[uname].map(self.ip):
+                logging.warning("username [%s] rejected by --ipr", uname)
+                return AUTH_FAILED
+
+        if not uname or not (asrv.vfs.aread.get(uname) or asrv.vfs.awrite.get(uname)):
+            g = self.hub.gpwd
+            if g.lim:
+                bonk, ip = g.bonk(self.ip, attempt)
+                if bonk:
+                    logging.warning("client banned: invalid passwords")
+                    bans[self.ip] = bonk
+                    try:
+                        # only possible if multiprocessing disabled
+                        self.hub.broker.httpsrv.bans[ip] = bonk  # type: ignore
+                        self.hub.broker.httpsrv.nban += 1  # type: ignore
+                    except:
+                        pass
+            return AUTH_FAILED
+
+        self.uname = uname
+        self.log("auth-pw OK: %s" % (uname,))
+        return AUTH_SUCCESSFUL
+
+    def check_auth_publickey(self, username: str, key: paramiko.PKey) -> int:
+        try:
+            return self._check_auth_publickey(username, key)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return AUTH_FAILED
+
+    def _check_auth_publickey(self, uname: str, key: paramiko.PKey) -> int:
+        args = self.args
+        if args.sftpv:
+            zs = key.get_name() + "," + key.get_base64()[:32]
+            self.log("auth-key: %r, %r" % (uname, zs))
+
+        ipn = ipnorm(self.ip)
+        bans = self.hub.bans
+        if ipn in bans:
+            rt = bans[ipn] - time.time()
+            if rt < 0:
+                self.log("client unbanned")
+                del bans[ipn]
+            else:
+                self.log("client is banned")
+                return AUTH_FAILED
+
+        anon = args.sftp_anon
+        if anon and uname == anon:
+            self.uname = "*"
+            self.log("auth-key OK: *")
+            return AUTH_SUCCESSFUL
+
+        attempt = "%s %s" % (key.get_name(), key.get_base64())
+        ok = args.sftp_key2u.get(attempt) == uname
+
+        if ok and args.ipr and uname in args.ipr_u:
+            if not args.ipr_u[uname].map(self.ip):
+                logging.warning("username [%s] rejected by --ipr", uname)
+                return AUTH_FAILED
+
+        asrv = self.hub.asrv
+        if not ok or not (asrv.vfs.aread.get(uname) or asrv.vfs.awrite.get(uname)):
+            self.log("auth-key REJECTED: %s" % (uname,))
+            return AUTH_FAILED
+
+        self.uname = uname
+        self.log("auth-key OK: %s" % (uname,))
+        return AUTH_SUCCESSFUL
+
+
+class SFTP_FH(paramiko.SFTPHandle):
+    def __init__(self, flags: int = 0) -> None:
+        self.filename = ""
+        self.readfile: Optional[BinaryIO] = None
+        self.writefile: Optional[BinaryIO] = None
+        super(SFTP_FH, self).__init__(flags)
+
+    def stat(self):
+        try:
+            f = self.readfile or self.writefile
+            return SATTR.from_stat(os.fstat(f.fileno()))
+        except OSError as ex:
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def chattr(self, attr):
+        # python doesn't have equivalents to fchown or fchmod, so we have to
+        # use the stored filename
+        if not self.writefile:
+            return SFTP_PERMISSION_DENIED
+        try:
+            paramiko.SFTPServer.set_file_attr(self.filename, attr)
+            return SFTP_OK
+        except OSError as ex:
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+
+class SFTP_Srv(paramiko.SFTPServerInterface):
+    def __init__(self, ssh: paramiko.ServerInterface, *a, **ka):
+        super(SFTP_Srv, self).__init__(ssh, *a, **ka)
+        self.ssh = ssh
+        self.ip: str = ssh.ip  # type: ignore
+        self.hub: "SvcHub" = ssh.hub  # type: ignore
+        self.uname: str = ssh.uname  # type: ignore
+        self.args = self.hub.args
+        self.asrv: "AuthSrv" = self.hub.asrv
+        self.v = self.args.sftpv
+        self.vv = self.args.sftpvv
+
+        if self.uname == LEELOO_DALLAS:
+            raise Exception("send her back")
+
+        self.vols = [
+            vp
+            for vp, vn in self.asrv.vfs.all_vols.items()
+            if self.uname in vn.axs.uread
+            or self.uname in vn.axs.uwrite
+            or self.uname in vn.axs.uget
+        ]
+        self.vis = set()
+        for zs in self.vols:
+            self.vis.add(zs)
+            while zs:
+                zs = zs.rsplit("/", 1)[0] if "/" in zs else ""
+                self.vis.add(zs)
+
+    def log(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log("sftp:%s" % (self.ip,), msg, c)
+
+    def v2a(
+        self,
+        vpath: str,
+        r: bool = False,
+        w: bool = False,
+        m: bool = False,
+        d: bool = False,
+    ) -> tuple[str, VFS, str]:
+        vpath = vpath.replace(os.sep, "/").strip("/")
+        rd, fn = os.path.split(vpath)
+        if relchk(rd):
+            self.log("malicious vpath: %s", vpath)
+            raise Exception("Unsupported characters in [%s]" % (vpath,))
+
+        fn = sanitize_fn(fn or "")
+        vpath = vjoin(rd, fn)
+        vn, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
+        if (
+            w
+            and fn.lower() in vn.flags["emb_all"]
+            and self.uname not in vn.axs.uread
+            and "wo_up_readme" not in vn.flags
+        ):
+            fn = "_wo_" + fn
+            vpath = vjoin(rd, fn)
+            vn, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
+
+        if not vn.realpath:
+            # return "", vn, rem
+            raise OSError(errno.ENOENT, "no filesystem mounted at [/%s]" % (vpath,))
+
+        if "xdev" in vn.flags or "xvol" in vn.flags:
+            ap = vn.canonical(rem)
+            avn = vn.chk_ap(ap)
+            t = "Permission denied in [{}]"
+            if not avn:
+                raise OSError(errno.EPERM, "permission denied in [/%s]" % (vpath,))
+
+            cr, cw, cm, cd, _, _, _, _, _ = avn.uaxs[self.uname]
+            if r and not cr or w and not cw or m and not cm or d and not cd:
+                raise OSError(errno.EPERM, "permission denied in [/%s]" % (vpath,))
+        else:
+            ap = vn.canonical(rem, False)
+
+        if "bcasechk" in vn.flags and not vn.casechk(rem, True):
+            raise OSError(errno.ENOENT, "file does not exist case-sensitively")
+
+        return ap, vn, rem
+
+    def list_folder(self, path: str) -> list[SATTR] | int:
+        try:
+            return self._list_folder(path)
+        except Pebkac as ex:
+            if ex.code == 404:
+                self.log("folder 404: %s" % (path,))
+                return SFTP_NO_SUCH_FILE
+            return SFTP_PERMISSION_DENIED
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _list_folder(self, path: str) -> list[SATTR] | int:
+        if self.v:
+            self.log("ls(%s):" % (path,))
+        path = path.strip("/")
+        try:
+            ap, vn, rem = self.v2a(path, r=True)
+        except Pebkac:
+            try:
+                self.v2a(path, w=True)
+                self.log("ls(%s): [] (write-only)" % (path,))
+                return []  # display write-only folders as empty
+            except:
+                pass
+            if path not in self.vis:
+                self.log("ls(%s): EPERM" % (path,))
+                return SFTP_PERMISSION_DENIED
+            # list of accessible volumes
+            ret = []
+            zi = int(time.time())
+            vst = os.stat_result((16877, -1, -1, 1, 1000, 1000, 8, zi, zi, zi))
+            prefix = path + "/"
+            for vn in self.asrv.vfs.all_nodes.values():
+                if path and not vn.vpath.startswith(prefix):
+                    continue  # vn is parent
+                vname = vn.vpath[len(prefix) :]
+                if "/" in vname or not vname:
+                    continue  # only include vols at current level
+                ret.append(SATTR.from_stat(vst, filename=vn.vpath))
+            ret.sort(key=lambda x: x.filename)
+            self.log("ls(%s): vfs-vols; |%d|" % (path, len(ret)))
+            return ret
+
+        fsroot, vfs_ls, vfs_virt = vn.ls(
+            rem,
+            self.uname,
+            not self.args.no_scandir,
+            [[True, False], [False, True]],
+            throw=True,
+        )
+        vnames = list(vfs_virt)
+        if self.uname not in vn.axs.udot:
+            if "dothidden" in vn.flags and ".hidden" in [x[0] for x in vfs_ls]:
+                vfs_ls = exclude_dothidden_ls(vfs_ls, fsroot)
+                vnames = exclude_dothidden(vnames, fsroot)
+            else:
+                vfs_ls = exclude_dotfiles_ls(vfs_ls)
+                vnames = exclude_dotfiles(vnames)
+        ret = [SATTR.from_stat(x[1], filename=x[0]) for x in vfs_ls]
+        for zs, vn2 in vfs_virt.items():
+            if not vn2.realpath or zs not in vnames:
+                continue
+            st = bos.stat(vn2.realpath)
+            ret.append(SATTR.from_stat(st, filename=zs))
+        ret.sort(key=lambda x: x.filename)
+        self.log("ls(%s): |%d|" % (path, len(ret)))
+        return ret
+
+    def stat(self, path: str) -> SATTR | int:
+        try:
+            return self._stat(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def lstat(self, path: str) -> SATTR | int:
+        try:
+            return self._stat(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _stat(self, vp: str) -> SATTR | int:
+        vp = vp.strip("/")
+        try:
+            ap, vn, _ = self.v2a(vp)
+            if (
+                self.uname not in vn.axs.uread
+                and self.uname not in vn.axs.uwrite
+                and self.uname not in vn.axs.uget
+            ):
+                self.log("stat(%s): EPERM" % (vp,))
+                return SFTP_PERMISSION_DENIED
+            st = bos.stat(ap)
+            self.log("stat(%s): %s" % (vp, st))
+        except:
+            if vp not in self.vis:
+                self.log("stat(%s): ENOENT" % (vp,))
+                return SFTP_NO_SUCH_FILE
+            zi = int(time.time())
+            st = os.stat_result((16877, -1, -1, 1, 1000, 1000, 8, zi, zi, zi))
+            self.log("stat(%s): vfs-vols")
+        return SATTR.from_stat(st)
+
+    def open(self, path: str, flags: int, attr: SATTR) -> paramiko.SFTPHandle | int:
+        try:
+            return self._open(path, flags, attr)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _open(self, vp: str, iflag: int, attr: SATTR) -> paramiko.SFTPHandle | int:
+        if ANYWIN:
+            iflag |= os.O_BINARY
+        if iflag & os.O_WRONLY:
+            rd = False
+            wr = True
+            if iflag & os.O_APPEND:
+                smode = "ab"
+            else:
+                smode = "wb"
+        elif iflag & os.O_RDWR:
+            rd = wr = True
+            if iflag & os.O_APPEND:
+                smode = "a+b"
+            else:
+                smode = "r+b"
+        else:
+            rd = True
+            wr = False
+            smode = "rb"
+
+        try:
+            vn, rem = self.asrv.vfs.get(vp, self.uname, rd, wr)
+            ap = vn.canonical(rem, False)
+            vf = vn.flags
+        except Pebkac as ex:
+            t = "denied open file [%s], iflag=%s, read=%s, write=%s: %s"
+            self.log(t % (vp, iflag, rd, wr, ex))
+            return SFTP_PERMISSION_DENIED
+
+        self.log("open(%s, %x, %s)" % (vp, iflag, smode))
+
+        if wr:
+            try:
+                st = bos.stat(ap)
+                td = time.time() - st.st_mtime
+                need_unlink = True
+            except:
+                need_unlink = False
+                td = 0
+
+            xbu = vn.flags.get("xbu")
+            if xbu:
+                hr = runhook(
+                    self.log,
+                    None,
+                    self.hub.up2k,
+                    "xbu.sftp",
+                    xbu,
+                    ap,
+                    vp,
+                    "",
+                    "",
+                    "",
+                    0,
+                    0,
+                    "7.3.8.7",
+                    time.time(),
+                    None,
+                )
+                t = hr.get("rejectmsg") or ""
+                if t or hr.get("rc") != 0:
+                    if not t:
+                        t = "upload blocked by xbu server config: %r" % (vp,)
+                    self.log(t, 3)
+                    return SFTP_PERMISSION_DENIED
+
+            self.log("writing to [%s] => [%s]" % (vp, ap))
+
+        if wr and need_unlink:  # type: ignore  # !rm
+            assert td  # type: ignore  # !rm
+            if td >= -1 and td <= self.args.ftp_wt:
+                # within permitted timeframe; allow overwrite or resume
+                do_it = True
+            elif self.args.no_del or self.args.ftp_no_ow:
+                # file too old, or overwrite not allowed; reject
+                do_it = False
+            else:
+                # allow overwrite if user has delete permission
+                do_it = self.uname in vn.axs.udel
+
+            if not do_it:
+                t = "file already exists and no permission to overwrite: %s"
+                self.log(t % (vp,))
+                return SFTP_PERMISSION_DENIED
+
+            # Don't unlink file for append mode
+            elif "a" not in smode:
+                wunlink(self.log, ap, VF_CAREFUL)
+
+        chmod = getattr(attr, "st_mode", None)
+        if chmod is None:
+            chmod = vf.get("chmod_f", 0o644)
+            self.log("open(%s, %x): client did not chmod" % (vp, iflag))
+        else:
+            self.log("open(%s, %x): client set chmod 0%o" % (vp, iflag, chmod))
+
+        try:
+            fd = os.open(ap, iflag, chmod)
+        except OSError as ex:
+            t = "failed to os.open [%s] -> [%s] with iflag [%s] and chmod [%s]: %r"
+            self.log(t % (vp, ap, iflag, chmod, ex), 3)
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+        if iflag & os.O_CREAT:
+            paramiko.SFTPServer.set_file_attr(ap, attr)
+
+        try:
+            f = os.fdopen(fd, smode)
+        except OSError as ex:
+            t = "failed to os.fdpen [%s] -> [%s] with smode [%s]: %r"
+            self.log(t % (vp, ap, smode, ex), 3)
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+        ret = SFTP_FH(iflag)
+        ret.filename = ap
+        ret.readfile = f if rd else None
+        ret.writefile = f if wr else None
+        return ret
+
+    def remove(self, path: str) -> int:
+        try:
+            return self._remove(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _remove(self, vp: str) -> int:
+        self.log("rm(%s)" % (vp,))
+        if self.args.no_del:
+            self.log("The delete feature is disabled in server config")
+            return SFTP_PERMISSION_DENIED
+        try:
+            self.hub.up2k.handle_rm(self.uname, self.ip, [vp], [], False, False)
+            self.log("rm(%s): ok" % (vp,))
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied delete [%s]: %s"
+            self.log(t % (vp, ex))
+            if str(ex).startswith("file not found"):
+                return SFTP_NO_SUCH_FILE
+            try:
+                # write-only client trying to rm before upload?
+                ap, vn, _ = self.v2a(vp)
+                if (
+                    self.uname not in vn.axs.uread
+                    and self.uname not in vn.axs.uwrite
+                    and self.uname not in vn.axs.uget
+                ):
+                    self.log("rm(%s): EPERM" % (vp,))
+                    return SFTP_PERMISSION_DENIED
+                if not bos.path.exists(ap):
+                    self.log(" `- file didn't exist; returning ENOENT")
+                    return SFTP_NO_SUCH_FILE
+            except:
+                pass
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            self.log("failed: rm(%s): %r" % (vp, ex))
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def rename(self, oldpath: str, newpath: str) -> int:
+        try:
+            return self._rename(oldpath, newpath)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _rename(self, svp: str, dvp: str) -> int:
+        self.log("mv(%s, %s)" % (svp, dvp))
+        if self.args.no_mv:
+            self.log("The rename/move feature is disabled in server config")
+        svp = svp.strip("/")
+        dvp = dvp.strip("/")
+        try:
+            self.hub.up2k.handle_mv("", self.uname, self.ip, svp, dvp)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied rename [%s] to [%s]: %s"
+            self.log(t % (svp, dvp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            self.log("mv(%s, %s): %r" % (svp, dvp, ex))
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def mkdir(self, path: str, attr: SATTR) -> int:
+        try:
+            return self._mkdir(path, attr)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _mkdir(self, vp: str, attr: SATTR) -> int:
+        self.log("mkdir(%s)" % (vp,))
+        try:
+            vn, rem = self.asrv.vfs.get(vp, self.uname, False, True)
+            ap = vn.canonical(rem, False)
+            bos.makedirs(ap, vf=vn.flags)  # filezilla expects this
+            if attr is not None:
+                paramiko.SFTPServer.set_file_attr(ap, attr)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied mkdir [%s]: %s"
+            self.log(t % (vp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            self.log("mkdir(%s): %r" % (vp, ex))
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def rmdir(self, path: str) -> int:
+        try:
+            return self._rmdir(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _rmdir(self, vp: str) -> int:
+        self.log("rmdir(%s)" % (vp,))
+        try:
+            vn, rem = self.asrv.vfs.get(vp, self.uname, False, False, will_del=True)
+            ap = os.path.join(vn.realpath, rem)
+            bos.rmdir(ap)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied rmdir [%s]: %s"
+            self.log(t % (vp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            self.log("rmdir(%s): %r" % (vp, ex))
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def chattr(self, path: str, attr: SATTR) -> int:
+        try:
+            return self._chattr(path, attr)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _chattr(self, vp: str, attr: SATTR) -> int:
+        self.log("chattr(%s, %s)" % (vp, attr))
+        try:
+            vn, rem = self.asrv.vfs.get(vp, self.uname, False, True, will_del=True)
+            ap = os.path.join(vn.realpath, rem)
+            paramiko.SFTPServer.set_file_attr(ap, attr)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied chattr [%s]: %s"
+            self.log(t % (vp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            self.log("chattr(%s): %r" % (vp, ex))
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def symlink(self, target_path: str, path: str) -> int:
+        return SFTP_OP_UNSUPPORTED
+
+    def readlink(self, path: str) -> str | int:
+        return path
+
+    def canonicalize(self, path: str) -> str:
+        return "/%s" % (undot(path),)
+
+
+class Transport2(paramiko.Transport):
+    def __init__(self, cli, addr):
+        self.addr = addr
+        super(Transport2, self).__init__(cli)
+
+    def _check_banner(self):
+        try:
+            return super()._check_banner()
+        except Exception as ex:
+            if "Error reading SSH protocol bann" in str(ex):
+                self.logger.warning("%s rejected: %s" % (self.addr, ex))
+                self.addr = None
+            else:
+                raise
+
+
+class Sftpd(object):
+    def __init__(self, hub: "SvcHub") -> None:
+        self.hub = hub
+        self.args = args = hub.args
+        self.log_func = hub.log
+        self.srv: list[socket.socket] = []
+        self.bound: list[str] = []
+        self.hs_q = Queue()
+        self.sessions = {}
+
+        ips = args.sftp_i
+        if "::" in ips:
+            ips.append("0.0.0.0")
+
+        ips = [x for x in ips if not x.startswith(("unix:", "fd:"))]
+
+        if args.sftp4:
+            ips = [x for x in ips if ":" not in x]
+
+        if not ips:
+            self.log("cannot start sftp-server; no compatible IPs in -i", 1)
+            return
+
+        self.hostkeys = []
+        hostkeytypes = (
+            ("ed25519", "Ed25519Key", {}),  # best
+            ("ecdsa", "ECDSAKey", {"bits": 384}),
+            ("rsa", "RSAKey", {"bits": 4096}),
+            ("dsa", "DSSKey", {}),  # worst
+        )
+        for fname, aname, opts in hostkeytypes:
+            fpath = "%s/ssh_host_%s_key" % (args.sftp_hostk, fname.lower())
+            try:
+                pkey = getattr(paramiko, aname).from_private_key_file(fpath)
+            except Exception as ex:
+                try:
+                    genfun = getattr(paramiko, aname).generate
+                except Exception as ex2:
+                    if args.sftpv or fname not in ("dsa", "ed25519"):
+                        # dsa dropped in 4.0
+                        # ed25519 not supported yet
+                        self.log("cannot generate %s hostkey: %r" % (aname, ex2), 3)
+                    continue
+                self.log("generating hostkey [%s] due to %r" % (fpath, ex))
+                pkey = genfun(**opts)
+                pkey.write_private_key_file(fpath)
+                pkey = getattr(paramiko, aname).from_private_key_file(fpath)
+            self.hostkeys.append(pkey)
+            if args.sftpv:
+                self.log("loaded hostkey %r" % (pkey,))
+
+        ips = list(ODict.fromkeys(ips))  # dedup
+
+        for ip in ips:
+            self._bind(ip)
+
+        self.log("listening @ %s port %s" % (self.bound, args.sftp))
+
+        for n in range(self.args.sftp_hs_n):
+            Daemon(self.acceptor, "sftp-hs-%d" % (n,))
+
+    def log(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log("sftp", msg, c)
+
+    def _bind(self, ip: str) -> None:
+        port = self.args.sftp
+        try:
+            ipv = socket.AF_INET6 if ":" in ip else socket.AF_INET
+            srv = socket.socket(ipv, socket.SOCK_STREAM)
+            if not ANYWIN or self.args.reuseaddr:
+                srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            srv.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+            srv.settimeout(0)  # == srv.setblocking(False)
+            try:
+                srv.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, False)
+            except:
+                pass  # will create another ipv4 socket instead
+            if getattr(self.args, "freebind", False):
+                srv.setsockopt(socket.SOL_IP, socket.IP_FREEBIND, 1)
+            srv.bind((ip, port))
+            srv.listen(10)
+            self.srv.append(srv)
+            self.bound.append(ip)
+        except Exception as ex:
+            if ip == "0.0.0.0" and "::" in self.bound:
+                try:
+                    srv.close()  # type: ignore
+                except:
+                    pass
+                return  # dualstack
+            self.log("could not listen on (%s,%s): %r" % (ip, port, ex), 3)
+
+    def _accept(self, srv: socket.socket) -> None:
+        cli, addr = srv.accept()
+        # cli.settimeout(0)  # == srv.setblocking(False)
+        self.log("%r is connecting" % (addr,))
+        self.hs_q.put((cli, addr))
+
+    def acceptor(self) -> None:
+        while True:
+            try:
+                cli, addr = self.hs_q.get()
+                self._accept2(cli, addr)
+            except:
+                time.sleep(1)
+
+    def _accept2(self, cli, addr) -> None:
+        tra = Transport2(cli, addr)
+        for hkey in self.hostkeys:
+            tra.add_server_key(hkey)
+        tra.set_subsystem_handler("sftp", paramiko.SFTPServer, SFTP_Srv)
+        psrv = SSH_Srv(self.hub, addr)
+        try:
+            tra.start_server(server=psrv)
+        except Exception as ex:
+            if tra.addr:
+                self.log("%r could not establish connection: %r" % (addr, ex), 3)
+            cli.close()
+            return
+
+        chan = tra.accept(timeout=self.args.sftp_hs_t)
+        if chan is None:
+            self.log("%r did not open an sftp channel" % (addr,), 3)
+            cli.close()
+            return
+
+        self.sessions[addr] = (chan, tra, psrv)
+        # tra.join()
+        # self.log("%r disconnected" % (addr,))
+
+    def run(self):
+        lgr = logging.getLogger("paramiko.transport")
+        lgr.setLevel(logging.DEBUG if self.args.sftpvv else logging.INFO)
+
+        if self.args.no_poll:
+            fun = self._run_select
+        else:
+            fun = self._run_poll
+        Daemon(fun, "sftpd")
+
+    def _run_select(self):
+        while not self.hub.stopping:
+            rx, _, _ = select.select(self.srv, [], [], 180)
+            for sck in rx:
+                self._accept(sck)
+
+    def _run_poll(self):
+        fd2sck = {}
+        poll = select.poll()
+        for sck in self.srv:
+            fd = sck.fileno()
+            fd2sck[fd] = sck
+            poll.register(fd, select.POLLIN)
+        while not self.hub.stopping:
+            pr = poll.poll(180 * 1000)
+            rx = [fd2sck[x[0]] for x in pr if x[1] & select.POLLIN]
+            for sck in rx:
+                self._accept(sck)
@@ -89,13 +89,15 @@ class SMB(object):
        smbserver.isInFileJail = self._is_in_file_jail
        self._disarm()

-        ip = next((x for x in self.args.i if ":" not in x), None)
+        zs = " " if self.args.smb6 else ":"
+        ip = next((x for x in self.args.smb_i if zs not in x), None)
        if not ip:
-            self.log("smb", "IPv6 not supported for SMB; listening on 0.0.0.0", 3)
+            self.log("smb", "IPv6 not enabled with --smb6; listening on 0.0.0.0", 3)
            ip = "0.0.0.0"

        port = int(self.args.smb_port)
-        srv = smbserver.SimpleSMBServer(listenAddress=ip, listenPort=port)
+        kw = {"ipv6": True} if ":" in ip else {}
+        srv = smbserver.SimpleSMBServer(listenAddress=ip, listenPort=port, **kw)
        try:
            if self.accs:
                srv.setAuthCallback(self._auth_cb)
@@ -121,6 +123,7 @@ class SMB(object):

        self.srv = srv
        self.stop = srv.stop
+        ip = "[%s]" % (ip,) if kw else ip
        self.log("smb", "listening @ {}:{}".format(ip, port))

    def nlog(self, msg: str, c: Union[int, str] = 0) -> None:
@@ -191,7 +194,7 @@ class SMB(object):
        vfs, rem = self.asrv.vfs.get(vpath, uname, *perms)
        if not vfs.realpath:
            raise Exception("unmapped vfs")
-        return vfs, vjoin(vfs.realpath, rem)
+        return vfs, vfs.canonical(rem, False)

    def _listdir(self, vpath: str, *a: Any, **ka: Any) -> list[str]:
        vpath = vpath.replace("\\", "/").lstrip("/")
@@ -262,10 +265,10 @@ class SMB(object):
                    0,
                    "1.7.6.2",
                    time.time(),
-                    "",
+                    None,
                )
                t = hr.get("rejectmsg") or ""
-                if t or not hr:
+                if t or hr.get("rc") != 0:
                    if not t:
                        t = "blocked by xbu server config: %r" % (vpath,)
                    yeet(t)
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import errno
 import re
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import re
 import stat
@@ -8,6 +8,7 @@ https://github.com/pydron/ifaddr/tree/0.2.0
 """

 import os
+import platform

 from ._shared import IP, Adapter

@@ -16,20 +17,28 @@ def nope(include_unconfigured=False):
    return []


-try:
-    S390X = os.uname().machine == "s390x"
-except:
-    S390X = False
+host_os = platform.system()
+machine = platform.machine()
+py_impl = platform.python_implementation()


-if os.environ.get("PRTY_NO_IFADDR") or S390X:
+if os.environ.get("PRTY_NO_IFADDR"):
+    get_adapters = nope
+elif machine in ("s390x",) or host_os in ("IRIX32",):
    # s390x deadlocks at libc.getifaddrs
+    # irix libc does not have getifaddrs at all
+    print("ifaddr unavailable; can't determine LAN IP: unsupported OS")
+    get_adapters = nope
+elif py_impl in ("GraalVM",):
+    print("ifaddr unavailable; can't determine LAN IP: unsupported interpreter")
    get_adapters = nope
 elif os.name == "nt":
    from ._win32 import get_adapters
 elif os.name == "posix":
    from ._posix import get_adapters
 else:
-    raise RuntimeError("Unsupported Operating System: %s" % os.name)
+    print("ifaddr unavailable; can't determine LAN IP: unsupported OS")
+    get_adapters = nope
+

 __all__ = ["Adapter", "IP", "get_adapters"]
@@ -1,23 +1,27 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os
 import tempfile
 from datetime import datetime

-from .__init__ import CORES
-from .authsrv import VFS, AuthSrv
+from .__init__ import CORES, TYPE_CHECKING
+from .authsrv import LEELOO_DALLAS, VFS, AuthSrv
 from .bos import bos
 from .th_cli import ThumbCli
-from .util import UTC, vjoin, vol_san
+from .th_srv import TH_CH
+from .util import UTC, sigblock, vjoin, vol_san

 if True:  # pylint: disable=using-constant-test
    from typing import Any, Generator, Optional

    from .util import NamedLogger

+if TYPE_CHECKING:
+    from httpsrv import HttpSrv

-TAR_NO_OPUS = set("aac|m4a|mp3|oga|ogg|opus|wma".split("|"))
+
+TAR_NO_OPUS = set("aac|m4a|m4b|m4r|mp3|oga|ogg|opus|wma".split("|"))


 class StreamArc(object):
@@ -41,20 +45,33 @@ class StreamArc(object):
        self.stopped = True


+_pools = {}
+
+
+def close_pools() -> None:
+    for p in list(_pools):
+        try:
+            p.shutdown(wait=False, cancel_futures=True)
+        except:
+            pass
+
+
 def gfilter(
    fgen: Generator[dict[str, Any], None, None],
    thumbcli: ThumbCli,
    uname: str,
    vtop: str,
+    vname: str,
    fmt: str,
 ) -> Generator[dict[str, Any], None, None]:
    from concurrent.futures import ThreadPoolExecutor

    pend = []
-    with ThreadPoolExecutor(max_workers=CORES) as tp:
+    with ThreadPoolExecutor(max_workers=CORES, initializer=sigblock) as tp:
+        _pools[tp] = 1
        try:
            for f in fgen:
-                task = tp.submit(enthumb, thumbcli, uname, vtop, f, fmt)
+                task = tp.submit(enthumb, thumbcli, uname, vtop, vname, f, fmt)
                pend.append((task, f))
                if pend[0][0].done() or len(pend) > CORES * 4:
                    task, f = pend.pop(0)
@@ -78,24 +95,82 @@ def gfilter(
                except:
                    pass
            thumbcli.log("gfilter flushed")
+        _pools.pop(tp, None)
+
+
+def gfilter2(
+    fgen: Generator[
+        tuple[
+            "VFS",
+            str,
+            str,
+            str,
+            list[tuple[str, os.stat_result]],
+            list[tuple[str, os.stat_result]],
+            dict[str, "VFS"],
+        ],
+        None,
+        None,
+    ],
+    hsrv: "HttpSrv",
+    vtop: str,
+    fmts: list[str],
+) -> Generator[dict[str, Any], None, None]:
+    from concurrent.futures import ThreadPoolExecutor
+
+    pend = []
+    with ThreadPoolExecutor(max_workers=CORES, initializer=sigblock) as tp:
+        _pools[tp] = 1
+        for _, _, vpath, apath, files, rd, vd in fgen:
+            if "/.hist/" in vpath:
+                continue
+            fnames = [n[0] for n in files]
+            vpaths = [vpath + "/" + n for n in fnames] if vpath else fnames
+            for vp, fi in zip(vpaths, files):
+                for fmt in fmts:
+                    try:
+                        f = {"vp": vp, "st": fi[1]}
+                        task = tp.submit(
+                            enthumb, hsrv.thumbcli, LEELOO_DALLAS, vtop, "", f, fmt
+                        )
+                        pend.append((task, f))
+                        if pend[0][0].done() or len(pend) > CORES * 4:
+                            task, f = pend.pop(0)
+                            try:
+                                f = task.result(600)
+                            except:
+                                pass
+                            yield f
+                    except:
+                        pass
+        for task, f in pend:
+            try:
+                f = task.result(600)
+            except:
+                pass
+            yield f
+        _pools.pop(tp, None)


 def enthumb(
-    thumbcli: ThumbCli, uname: str, vtop: str, f: dict[str, Any], fmt: str
+    thumbcli: ThumbCli, uname: str, vtop: str, vname: str, f: dict[str, Any], fmt: str
 ) -> dict[str, Any]:
    rem = f["vp"]
    ext = rem.rsplit(".", 1)[-1].lower()
    if (fmt == "mp3" and ext == "mp3") or (fmt == "opus" and ext in TAR_NO_OPUS):
        raise Exception()

-    vp = vjoin(vtop, rem.split("/", 1)[1])
+    if vname:
+        vp = vjoin(vtop, rem.split("/", 1)[1])
+    else:
+        vp = vjoin(vtop, rem)
    vn, rem = thumbcli.asrv.vfs.get(vp, uname, True, False)
    dbv, vrem = vn.get_dbv(rem)
    thp = thumbcli.get(dbv, vrem, f["st"].st_mtime, fmt)
    if not thp:
        raise Exception()

-    ext = "jpg" if fmt == "j" else "webp" if fmt == "w" else fmt
+    ext = fmt if fmt == "wav" else TH_CH.get(fmt[:1], fmt)
    sz = bos.path.getsize(thp)
    st: os.stat_result = f["st"]
    ts = st.st_mtime
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import calendar
 import stat
@@ -8,7 +8,7 @@ import time
 from .authsrv import AuthSrv
 from .bos import bos
 from .sutil import StreamArc, errdesc
-from .util import min_ex, sanitize_fn, spack, sunpack, yieldfile, zlib
+from .util import VPTL_WIN, min_ex, sanitize_to, spack, sunpack, yieldfile, zlib

 if True:  # pylint: disable=using-constant-test
    from typing import Any, Generator, Optional
@@ -104,7 +104,7 @@ def gen_hdr(
    ret += spack(b"<LL", vsz, vsz)

    # windows support (the "?" replace below too)
-    fn = sanitize_fn(fn, "/")
+    fn = sanitize_to(fn, VPTL_WIN)
    bfn = fn.encode("utf-8" if utf8 else "cp437", "replace").replace(b"?", b"_")

    # add ntfs (0x24) and/or unix (0x10) extrafields for utc, add z64 if requested
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os
 import re
@@ -7,7 +7,7 @@ import socket
 import sys
 import time

-from .__init__ import ANYWIN, PY2, TYPE_CHECKING, unicode
+from .__init__ import ANYWIN, OPENBSD, PY2, TYPE_CHECKING, UNIX, unicode
 from .cert import gencert
 from .qrkode import QrCode, qr2png, qr2svg, qr2txt, qrgen
 from .util import (
@@ -21,7 +21,9 @@ from .util import (
    VF_CAREFUL,
    Netdev,
    atomic_move,
+    chkcmd,
    get_adapters,
+    list_nics,
    min_ex,
    sunpack,
    termsize,
@@ -422,6 +424,7 @@ class TcpSrv(object):
        self.hub.broker.say("httpsrv.set_netdevs", self.netdevs)
        self.hub.start_zeroconf()
        gencert(self.log, self.args, self.netdevs)
+        self.hub.restart_sftpd()
        self.hub.restart_ftpd()
        self.hub.restart_tftpd()

@@ -459,23 +462,7 @@ class TcpSrv(object):
    def detect_interfaces(self, listen_ips: list[str]) -> dict[str, Netdev]:
        listen_ips = [x for x in listen_ips if not x.startswith(("unix:", "fd:"))]

-        nics = get_adapters(True)
-        eps: dict[str, Netdev] = {}
-        for nic in nics:
-            for nip in nic.ips:
-                ipa = nip.ip[0] if ":" in str(nip.ip) else nip.ip
-                sip = "{}/{}".format(ipa, nip.network_prefix)
-                nd = Netdev(sip, nic.index or 0, nic.nice_name, "")
-                eps[sip] = nd
-                try:
-                    idx = socket.if_nametoindex(nd.name)
-                    if idx and idx != nd.idx:
-                        t = "netdev idx mismatch; ifaddr={} cpython={}"
-                        self.log("tcpsrv", t.format(nd.idx, idx), 3)
-                        nd.idx = idx
-                except:
-                    pass
-
+        eps = list_nics()
        netlist = str(sorted(eps.items()))
        if netlist == self.netlist and self.netdevs:
            return {}
@@ -509,6 +496,13 @@ class TcpSrv(object):
        return eps

    def _extdevs_nix(self) -> Generator[str, None, None]:
+        if UNIX:
+            so, _ = chkcmd(["netstat", "-nrf", "inet"])
+            for ln in so.split("\n"):
+                if not ln.startswith("default"):
+                    continue
+                yield ln.split()[7] if OPENBSD else ln.split()[3]
+            return
        with open("/proc/net/route", "rb") as f:
            next(f)
            for ln in f:
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 try:
    from types import SimpleNamespace
@@ -37,12 +37,12 @@ from .__init__ import EXE, PY2, TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
 from .util import (
-    FN_EMB,
    UTC,
    BytesIO,
    Daemon,
    ODict,
    exclude_dotfiles,
+    exclude_dothidden,
    min_ex,
    runhook,
    set_fperms,
@@ -175,7 +175,7 @@ class Tftpd(object):
            p1, p2 = [int(x) for x in self.args.tftp_pr.split("-")]
            ports = list(range(p1, p2 + 1))

-        ips = self.args.i
+        ips = self.args.tftp_i
        if "::" in ips:
            ips.append("0.0.0.0")

@@ -268,7 +268,7 @@ class Tftpd(object):
        vfs, rem = self.asrv.vfs.get(vpath, "*", *perms)
        if perms[1] and "*" not in vfs.axs.uread and "wo_up_readme" not in vfs.flags:
            zs, fn = vsplit(vpath)
-            if fn.lower() in FN_EMB:
+            if fn.lower() in vfs.flags["emb_all"]:
                vpath = vjoin(zs, "_wo_" + fn)
                vfs, rem = self.asrv.vfs.get(vpath, "*", *perms)

@@ -319,7 +319,11 @@ class Tftpd(object):
        ls = virs + reals

        if "*" not in vn.axs.udot:
-            names = set(exclude_dotfiles([x[2] for x in ls]))
+            zsl = [x[2] for x in ls]
+            if "dothidden" in vn.flags and ".hidden" in zsl:
+                names = set(exclude_dothidden(zsl, fsroot))
+            else:
+                names = set(exclude_dotfiles(zsl))
            ls = [x for x in ls if x[2] in names]

        try:
@@ -379,10 +383,10 @@ class Tftpd(object):
                    0,
                    "8.3.8.7",
                    time.time(),
-                    "",
+                    None,
                )
                t = hr.get("rejectmsg") or ""
-                if t or not hr:
+                if t or hr.get("rc") != 0:
                    if not t:
                        t = "upload blocked by xbu server config: %r" % (vpath,)
                    yeet(t)
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import errno
 import os
@@ -8,7 +8,7 @@ import stat
 from .__init__ import TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
-from .th_srv import EXTS_AC, HAVE_WEBP, thumb_path
+from .th_srv import EXTS_AC, H_PIL_JXL, H_PIL_WEBP, thumb_path
 from .util import Cooldown, Pebkac

 if True:  # pylint: disable=using-constant-test
@@ -20,26 +20,18 @@ if TYPE_CHECKING:

 IOERROR = "reading the file was denied by the server os; either due to filesystem permissions, selinux, apparmor, or similar:\n%r"

+IMG_EXTS = set(["webp", "jpg", "png", "jxl"])
+

 class ThumbCli(object):
-    def __init__(self, hsrv: "HttpSrv") -> None:
+    def __init__(self, hsrv: "HttpSrv", c: dict[str, set[str]]) -> None:
        self.broker = hsrv.broker
        self.log_func = hsrv.log
        self.args = hsrv.args
        self.asrv = hsrv.asrv

        # cache on both sides for less broker spam
-        self.cooldown = Cooldown(self.args.th_poke)
-
-        try:
-            c = hsrv.th_cfg
-            if not c:
-                raise Exception()
-        except:
-            c = {
-                k: set()
-                for k in ["thumbable", "pil", "vips", "raw", "ffi", "ffv", "ffa"]
-            }
+        self.cooldown = Cooldown(self.args.th_poke) if self.args.th_poke else None

        self.thumbable = c["thumbable"]
        self.fmt_pil = c["pil"]
@@ -50,8 +42,9 @@ class ThumbCli(object):
        self.fmt_ffa = c["ffa"]

        # defer args.th_ff_jpg, can change at runtime
-        d = next((x for x in self.args.th_dec if x in ("vips", "pil")), None)
-        self.can_webp = HAVE_WEBP or d == "vips"
+        nonpil = next((x for x in self.args.th_dec if x in ("vips", "ff")), None)
+        self.can_webp = (H_PIL_WEBP or nonpil) and not self.args.th_no_webp
+        self.can_jxl = (H_PIL_JXL or nonpil) and not self.args.th_no_jxl

    def log(self, msg: str, c: Union[int, str] = 0) -> None:
        self.log_func("thumbcli", msg, c)
@@ -89,23 +82,24 @@ class ThumbCli(object):

        preferred = self.args.th_dec[0] if self.args.th_dec else ""

-        if rem.startswith(".hist/th/") and rem.split(".")[-1] in ["webp", "jpg", "png"]:
+        if rem.startswith(".hist/th/") and rem.split(".")[-1] in IMG_EXTS:
            return os.path.join(ptop, rem)

-        if fmt[:1] in "jw" and fmt != "wav":
-            sfmt = fmt[:1]
+        sfmt = fmt[:1]
+        if sfmt in "jwx" and fmt != "wav":

            if sfmt == "j" and self.args.th_no_jpg:
                sfmt = "w"

            if sfmt == "w":
-                if (
-                    self.args.th_no_webp
-                    or (is_img and not self.can_webp)
-                    or (self.args.th_ff_jpg and (not is_img or preferred == "ff"))
+                if not self.can_webp or (
+                    self.args.th_ff_jpg and (not is_img or preferred == "ff")
                ):
                    sfmt = "j"

+            if sfmt == "x" and not self.can_jxl:
+                sfmt = "w"
+
            vf_crop = dbv.flags["crop"]
            vf_th3x = dbv.flags["th3x"]

@@ -121,7 +115,7 @@ class ThumbCli(object):

            fmt = sfmt

-        elif fmt[:1] == "p" and not is_au and not is_vid:
+        elif sfmt == "p" and not is_au and not is_vid:
            t = "cannot thumbnail %r: png only allowed for waveforms"
            self.log(t % (rem,), 6)
            return None
@@ -133,9 +127,12 @@ class ThumbCli(object):

        tpath = thumb_path(histpath, rem, mtime, fmt, self.fmt_ffa)
        tpaths = [tpath]
-        if fmt[:1] == "w" and fmt != "wav":
+        fmtc = fmt[:1]
+        if fmtc == "w" and fmt != "wav":
            # also check for jpg (maybe webp is unavailable)
            tpaths.append(tpath.rsplit(".", 1)[0] + ".jpg")
+        elif fmtc == "x":
+            tpaths.append(tpath.rsplit(".", 1)[0] + ".webp")

        ret = None
        abort = False
@@ -152,14 +149,15 @@ class ThumbCli(object):
                pass

        if ret:
-            tdir = os.path.dirname(tpath)
-            if self.cooldown.poke(tdir):
-                self.broker.say("thumbsrv.poke", tdir)
+            if self.cooldown:
+                tdir = os.path.dirname(tpath)
+                if self.cooldown.poke(tdir):
+                    self.broker.say("thumbsrv.poke", tdir)

-            if want_opus:
-                # audio files expire individually
-                if self.cooldown.poke(tpath):
-                    self.broker.say("thumbsrv.poke", tpath)
+                if want_opus:
+                    # audio files expire individually
+                    if self.cooldown.poke(tpath):
+                        self.broker.say("thumbsrv.poke", tpath)

            return ret

@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import calendar
 import os
@@ -22,11 +22,20 @@ from .util import (
    quotep,
    s3dec,
    vjoin,
+    vjoins,
 )

 if HAVE_SQLITE3:
    import sqlite3

+try:
+    if os.environ.get("PRTY_NO_UNIDATA"):
+        raise Exception()
+
+    import unicodedata
+except:
+    pass
+
 try:
    from pathlib import Path
 except:
@@ -42,6 +51,14 @@ if PY2:
    range = xrange  # type: ignore


+def norm_icase(txt: str) -> str:
+    return txt.casefold() if txt else txt
+
+
+def norm_nfkc(txt: str) -> str:
+    return unicodedata.normalize("NFKC", txt).casefold() if txt else txt
+
+
 class U2idx(object):
    def __init__(self, hsrv: "HttpSrv") -> None:
        self.log_func = hsrv.log
@@ -54,6 +71,10 @@ class U2idx(object):
            return

        if self.args.srch_icase:
+            if self.args.srch_nfkc:
+                self.normfun = norm_nfkc
+            else:
+                self.normfun = norm_icase
            self._open_db = self._open_db_icase
        else:
            self._open_db = self._open_db_std
@@ -81,7 +102,7 @@ class U2idx(object):

    def _open_db_icase(self, *args, **kwargs):
        db = self._open_db_std(*args, **kwargs)
-        db.create_function("casefold", 1, lambda x: x.casefold() if x else x)
+        db.create_function("casefold", 1, self.normfun)
        return db

    def shutdown(self) -> None:
@@ -322,7 +343,7 @@ class U2idx(object):

            if icase and "casefold(" in q:
                try:
-                    v = unicode(v).casefold()
+                    v = self.normfun(unicode(v))
                except:
                    v = unicode(v).lower()

@@ -437,7 +458,7 @@ class U2idx(object):
                if rd.startswith("//") or fn.startswith("//"):
                    rd, fn = s3dec(rd, fn)

-                vp = vjoin(vjoin(vtop, rd), fn)
+                vp = vjoins(vtop, rd, fn)

                if vp in seen_rps:
                    continue
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import errno
 import hashlib
@@ -10,6 +10,7 @@ import re
 import shutil
 import stat
 import subprocess as sp
+import sys
 import tempfile
 import threading
 import time
@@ -18,7 +19,7 @@ from copy import deepcopy

 from queue import Queue

-from .__init__ import ANYWIN, MACOS, PY2, TYPE_CHECKING, WINDOWS, E
+from .__init__ import ANYWIN, PY2, TYPE_CHECKING, UNIX, WINDOWS, E
 from .authsrv import LEELOO_DALLAS, SEESLOG, VFS, AuthSrv
 from .bos import bos
 from .cfg import vf_bmap, vf_cmap, vf_vmap
@@ -27,6 +28,7 @@ from .mtag import MParser, MTag
 from .util import (
    E_FS_CRIT,
    E_FS_MEH,
+    HAVE_FICLONE,
    HAVE_SQLITE3,
    SYMTIME,
    VF_CAREFUL,
@@ -59,6 +61,8 @@ from .util import (
    s3dec,
    s3enc,
    sanitize_fn,
+    set_ap_perms,
+    set_fperms,
    sfsenc,
    spack,
    statdir,
@@ -66,9 +70,11 @@ from .util import (
    ub64enc,
    unhumanize,
    vjoin,
+    vjoins,
    vsplit,
    w8b64dec,
    w8b64enc,
+    winsparse,
    wunlink,
 )

@@ -88,13 +94,17 @@ if True:  # pylint: disable=using-constant-test
 if TYPE_CHECKING:
    from .svchub import SvcHub

+USE_FICLONE = HAVE_FICLONE and sys.version_info < (3, 14)
+if USE_FICLONE:
+    import fcntl
+
 zsg = "avif,avifs,bmp,gif,heic,heics,heif,heifs,ico,j2p,j2k,jp2,jpeg,jpg,jpx,png,tga,tif,tiff,webp"
 ICV_EXTS = set(zsg.split(","))

 zsg = "3gp,asf,av1,avc,avi,flv,m4v,mjpeg,mjpg,mkv,mov,mp4,mpeg,mpeg2,mpegts,mpg,mpg2,mts,nut,ogm,ogv,rm,vob,webm,wmv"
 VCV_EXTS = set(zsg.split(","))

-zsg = "aif,aiff,alac,ape,flac,m4a,mp3,oga,ogg,opus,tak,tta,wav,wma,wv,cbz,epub"
+zsg = "aif,aiff,alac,ape,flac,m4a,m4b,m4r,mka,mp3,oga,ogg,opus,tak,tta,wav,wma,wv,cbz,epub"
 ACV_EXTS = set(zsg.split(","))

 zsg = "nohash noidx xdev xvol"
@@ -124,6 +134,7 @@ class Mpqe(object):
        self,
        mtp: dict[str, MParser],
        entags: set[str],
+        vf: dict[str, Any],
        w: str,
        abspath: str,
        oth_tags: dict[str, Any],
@@ -131,6 +142,7 @@ class Mpqe(object):
        # mtp empty = mtag
        self.mtp = mtp
        self.entags = entags
+        self.vf = vf
        self.w = w
        self.abspath = abspath
        self.oth_tags = oth_tags
@@ -156,6 +168,7 @@ class Up2k(object):
        self.gt0 = 0
        self.gt1 = 0
        self.stop = False
+        self.fika = ""
        self.mutex = threading.Lock()
        self.reload_mutex = threading.Lock()
        self.reload_flag = 0
@@ -326,6 +339,10 @@ class Up2k(object):
            if not self.stop:
                self.log("uploads are now possible", 2)

+    def is_busy(self) -> tuple[bool, float]:
+        # returns ( currently-busy , have-finished-at-least-once )
+        return bool(self.pp), self.gt1
+
    def get_state(self, get_q: bool, uname: str) -> str:
        mtpq: Union[int, str] = 0
        ups = []
@@ -650,19 +667,20 @@ class Up2k(object):

            for _ in range(2):
                q = "select ip, at from up where ip > '' order by +at limit 1"
-                hits = cur.execute(q).fetchall()
-                if not hits:
-                    break
+                with self.mutex:
+                    hits = cur.execute(q).fetchall()
+                    if not hits:
+                        break

-                remains = hits[0][1] - cutoff
-                if remains > 0:
-                    timeout = min(timeout, now + remains)
-                    break
+                    remains = hits[0][1] - cutoff
+                    if remains > 0:
+                        timeout = min(timeout, now + remains)
+                        break

-                q = "update up set ip = '' where ip > '' and at <= %d"
-                cur.execute(q % (cutoff,))
-                zi = cur.rowcount
-                cur.connection.commit()
+                    q = "update up set ip = '' where ip > '' and at <= %d"
+                    cur.execute(q % (cutoff,))
+                    zi = cur.rowcount
+                    cur.connection.commit()

                t = "forget-ip(%d) removed %d IPs from db [/%s]"
                self.log(t % (maxage, zi, vol.vpath))
@@ -1142,7 +1160,7 @@ class Up2k(object):
        ft = "\033[0;32m{}{:.0}"
        ff = "\033[0;35m{}{:.0}"
        fv = "\033[0;36m{}:\033[90m{}"
-        zs = "bcasechk du_iwho ext_th_d html_head html_head_d html_head_s put_name2 mv_re_r mv_re_t rm_re_r rm_re_t srch_re_dots srch_re_nodot zipmax zipmaxn_v zipmaxs_v"
+        zs = "bcasechk du_iwho emb_all emb_lgs emb_mds ext_th_d html_head html_head_d html_head_s ls_q_m put_name2 mv_re_r mv_re_t rm_re_r rm_re_t oh_f oh_g rw_edit_set srch_re_dots srch_re_nodot th_coversd th_coversl th_covers_set th_coversd_set zipmax zipmaxn_v zipmaxs_v"
        fx = set(zs.split())
        fd = vf_bmap()
        fd.update(vf_cmap())
@@ -1414,6 +1432,8 @@ class Up2k(object):
                    rtop,
                    rei,
                    reh,
+                    vol.flags["th_coversd"],
+                    vol.flags["th_coversd_set"],
                    n4g,
                    ffat,
                    [],
@@ -1465,6 +1485,23 @@ class Up2k(object):

            return True, bool(n_add or n_rm or do_vac)

+    def _fika(self, db: Dbw) -> None:
+        zs = self.fika
+        self.fika = ""
+        if zs not in self.args.fika:
+            return
+
+        t = "fika(%s); commit %d new files; %d updates"
+        self.log(t % (zs, db.nf, db.n))
+        db.c.connection.commit()
+        db.n = db.nf = 0
+        db.t = time.time()
+
+        self.mutex.release()
+        time.sleep(0.5)
+        self.mutex.acquire()
+        db.t = time.time()
+
    def _build_dir(
        self,
        db: Dbw,
@@ -1474,6 +1511,8 @@ class Up2k(object):
        rcdir: str,
        rei: Optional[Pattern[str]],
        reh: Optional[Pattern[str]],
+        th_cvd: list[str],
+        th_cvds: set[str],
        n4g: bool,
        ffat: bool,
        seen: list[str],
@@ -1499,8 +1538,6 @@ class Up2k(object):
        cv = vcv = acv = ""
        e_d = {}

-        th_cvd = self.args.th_coversd
-        th_cvds = self.args.th_coversd_set
        scan_pr_s = self.args.scan_pr_s

        assert self.pp and self.mem_cur  # !rm
@@ -1517,8 +1554,12 @@ class Up2k(object):
        gl = sorted(g)
        partials = set([x[0] for x in gl if "PARTIAL" in x[0]])
        for iname, inf in gl:
-            if self.stop:
-                return -1, 0, 0
+            if self.fika:
+                if not self.stop:
+                    self._fika(db)
+                if self.stop:
+                    self.fika = "f"
+                    return -1, 0, 0

            rp = rds + iname
            abspath = cdirs + iname
@@ -1563,6 +1604,8 @@ class Up2k(object):
                        rap,
                        rei,
                        reh,
+                        th_cvd,
+                        th_cvds,
                        n4g,
                        fat32,
                        seen,
@@ -1593,7 +1636,7 @@ class Up2k(object):

                rsz += sz
                files.append((sz, lmod, iname))
-                if sz:
+                if sz and th_cvd:
                    liname = iname.lower()
                    ext = liname.rsplit(".", 1)[-1]
                    if (
@@ -1665,8 +1708,12 @@ class Up2k(object):

        seen_files = set([x[2] for x in files])  # for dropcheck
        for sz, lmod, fn in files:
-            if self.stop:
-                return -1, 0, 0
+            if self.fika:
+                if not self.stop:
+                    self._fika(db)
+                if self.stop:
+                    self.fika = "f"
+                    return -1, 0, 0

            rp = rds + fn
            abspath = cdirs + fn
@@ -2193,7 +2240,7 @@ class Up2k(object):
            abspath = djoin(ptop, rd, fn)
            self.pp.msg = "c%d %s" % (nq, abspath)
            if not mpool:
-                n_tags = self._tagscan_file(cur, entags, w, abspath, ip, at, un)
+                n_tags = self._tagscan_file(cur, entags, flags, w, abspath, ip, at, un)
            else:
                oth_tags = {}
                if ip:
@@ -2203,7 +2250,7 @@ class Up2k(object):
                if un:
                    oth_tags["up_by"] = un

-                mpool.put(Mpqe({}, entags, w, abspath, oth_tags))
+                mpool.put(Mpqe({}, entags, flags, w, abspath, oth_tags))
                with self.mutex:
                    n_tags = len(self._flush_mpool(cur))

@@ -2310,9 +2357,10 @@ class Up2k(object):
            return

        entags = self.entags[ptop]
+        vf = self.flags[ptop]

        parsers = {}
-        for parser in self.flags[ptop]["mtp"]:
+        for parser in vf["mtp"]:
            try:
                parser = MParser(parser)
            except:
@@ -2387,7 +2435,7 @@ class Up2k(object):
                    if un:
                        oth_tags["up_by"] = un

-                    jobs.append(Mpqe(parsers, set(), w, abspath, oth_tags))
+                    jobs.append(Mpqe(parsers, set(), vf, w, abspath, oth_tags))
                    in_progress[w] = True

            with self.mutex:
@@ -2518,7 +2566,7 @@ class Up2k(object):
            return

        for _ in range(mpool.maxsize):
-            mpool.put(Mpqe({}, set(), "", "", {}))
+            mpool.put(Mpqe({}, set(), {}, "", "", {}))

        mpool.join()

@@ -2537,7 +2585,7 @@ class Up2k(object):
                        t = "tag-thr: {}({})"
                        self.log(t.format(self.mtag.backend, qe.abspath), "90")

-                    tags = self.mtag.get(qe.abspath) if st.st_size else {}
+                    tags = self.mtag.get(qe.abspath, qe.vf) if st.st_size else {}
                else:
                    if self.args.mtag_vv:
                        t = "tag-thr: {}({})"
@@ -2570,6 +2618,7 @@ class Up2k(object):
        self,
        write_cur: "sqlite3.Cursor",
        entags: set[str],
+        vf: dict[str, Any],
        wark: str,
        abspath: str,
        ip: str,
@@ -2588,7 +2637,7 @@ class Up2k(object):
            return 0

        try:
-            tags = self.mtag.get(abspath) if st.st_size else {}
+            tags = self.mtag.get(abspath, vf) if st.st_size else {}
        except Exception as ex:
            self._log_tag_err("", abspath, ex)
            return 0
@@ -2687,6 +2736,17 @@ class Up2k(object):
            t = "-" * 72
            raise Exception("%s\n%s\n%s" % (t, txt, t))

+    def _sql_ok(self, cur: "sqlite3.Cursor", q: str) -> bool:
+        try:
+            cur.execute(q).fetchone()
+            return True
+        except Exception as ex:
+            if "no such table" not in str(ex):
+                t = "WARNING: database issues, maybe due to server filesystem or hardware (check %sOS-logs):\n  %r -- \033[31m%r\n"
+                t2 = "" if ANYWIN or UNIX else "dmesg / "
+                self.log(t % (t2, q, ex), 3)
+        return False
+
    def _orz(self, db_path: str) -> "sqlite3.Cursor":
        assert sqlite3  # type: ignore  # !rm
        c = sqlite3.connect(
@@ -2860,11 +2920,8 @@ class Up2k(object):

    def _add_dhash_tab(self, cur: "sqlite3.Cursor") -> None:
        # v5 -> v5a
-        try:
-            cur.execute("select d, h from dh limit 1").fetchone()
+        if self._sql_ok(cur, "select d, h from dh limit 1"):
            return
-        except:
-            pass

        for cmd in [
            r"create table dh (d text, h text)",
@@ -2878,11 +2935,8 @@ class Up2k(object):
    def _add_xiu_tab(self, cur: "sqlite3.Cursor") -> None:
        # v5a -> v5b
        # store rd+fn rather than warks to support nohash vols
-        try:
-            cur.execute("select c, w, rd, fn from iu limit 1").fetchone()
+        if self._sql_ok(cur, "select c, w, rd, fn from iu limit 1"):
            return
-        except:
-            pass

        try:
            cur.execute("drop table iu")
@@ -2900,11 +2954,8 @@ class Up2k(object):

    def _add_cv_tab(self, cur: "sqlite3.Cursor") -> None:
        # v5b -> v5c
-        try:
-            cur.execute("select rd, dn, fn from cv limit 1").fetchone()
+        if self._sql_ok(cur, "select rd, dn, fn from cv limit 1"):
            return
-        except:
-            pass

        for cmd in [
            r"create table cv (rd text, dn text, fn text)",
@@ -2939,11 +2990,8 @@ class Up2k(object):

    def _add_ds_tab(self, cur: "sqlite3.Cursor") -> None:
        # v5d -> v5e
-        try:
-            cur.execute("select rd, sz from ds limit 1").fetchone()
+        if self._sql_ok(cur, "select rd, sz from ds limit 1"):
            return
-        except:
-            pass

        for cmd in [
            r"create table ds (rd text, sz int, nf int)",
@@ -2966,6 +3014,7 @@ class Up2k(object):
            raise Pebkac(503, SBUSY % ("fs-reload",))

        got_lock = False
+        self.fika = "u"
        try:
            # bit expensive; 3.9=10x 3.11=2x
            if self.mutex.acquire(timeout=10):
@@ -2993,7 +3042,7 @@ class Up2k(object):
            raise Pebkac(500, "too many xbu relocs, giving up")

        ptop = cj["ptop"]
-        if not self.register_vpath(ptop, cj["vcfg"]):
+        if not self.register_vpath(ptop, cj.pop("vcfg")):
            if ptop not in self.registry:
                raise Pebkac(410, "location unavailable")

@@ -3033,7 +3082,7 @@ class Up2k(object):
                )

            zi = cj["lmod"]
-            bad_mt = zi <= 0 or zi > 0xAAAAAAAA
+            bad_mt = zi <= 0 or zi > (2 << 36)
            if bad_mt or vfs.flags.get("up_ts", "") == "fu":
                # force upload time rather than last-modified
                cj["lmod"] = int(time.time())
@@ -3146,7 +3195,7 @@ class Up2k(object):
                c2 = None
                for cur, dp_dir, dp_fn in lost:
                    t = "forgetting desynced db entry: %r"
-                    self.log(t % ("/" + vjoin(vjoin(vfs.vpath, dp_dir), dp_fn)))
+                    self.log(t % ("/" + vjoins(vfs.vpath, dp_dir, dp_fn)))
                    self.db_rm(cur, vfs.flags, dp_dir, dp_fn, cj["size"])
                    if c2 and c2 != cur:
                        c2.connection.commit()
@@ -3221,7 +3270,25 @@ class Up2k(object):
                    dst = djoin(cj["ptop"], cj["prel"], cj["name"])
                    vsrc = djoin(job["vtop"], job["prel"], job["name"])
                    vsrc = vsrc.replace("\\", "/")  # just for prints anyways
-                    if "done" not in job:
+                    if vfs.lim:
+                        dst, cj["prel"] = vfs.lim.all(
+                            cj["addr"],
+                            cj["prel"],
+                            cj["size"],
+                            cj["ptop"],
+                            djoin(cj["ptop"], cj["prel"]),
+                            self.hub.broker,
+                            reg,
+                            "up2k._get_volsize",
+                        )
+                        bos.makedirs(dst, vf=vfs.flags)
+                        vfs.lim.nup(cj["addr"])
+                        vfs.lim.bup(cj["addr"], cj["size"])
+
+                    if "rvp0" in job and wark in reg:
+                        # xbu reloc; accept wrong path
+                        job["addr"] = cj["addr"]
+                    elif "done" not in job:
                        self.log("unfinished:\n  %r\n  %r" % (src, dst))
                        err = "partial upload exists at a different location; please resume uploading here instead:\n"
                        err += "/" + quotep(vsrc) + " "
@@ -3298,10 +3365,10 @@ class Up2k(object):
                                job["size"],
                                job["addr"],
                                job["at"],
-                                "",
+                                [dwark],
                            )
                            t = hr.get("rejectmsg") or ""
-                            if t or not hr:
+                            if t or hr.get("rc") != 0:
                                if not t:
                                    t = "upload blocked by xbu server config: %r"
                                    t = t % (vp,)
@@ -3311,18 +3378,29 @@ class Up2k(object):
                                x = pathmod(self.vfs, dst, vp, hr["reloc"])
                                if x:
                                    ud1 = (vfs.vpath, job["prel"], job["name"])
+                                    job["rvp0"] = vjoins(*ud1)
                                    pdir, _, job["name"], (vfs, rem) = x
                                    dst = os.path.join(pdir, job["name"])
-                                    job["vcfg"] = vfs.flags
                                    job["ptop"] = vfs.realpath
                                    job["vtop"] = vfs.vpath
                                    job["prel"] = rem
-                                    job["name"] = sanitize_fn(job["name"], "")
+                                    job["name"] = sanitize_fn(job["name"])
                                    ud2 = (vfs.vpath, job["prel"], job["name"])
                                    if ud1 != ud2:
                                        # print(json.dumps(job, sort_keys=True, indent=4))
+                                        job["vcfg"] = vfs.flags
                                        job["hash"] = cj["hash"]
-                                        self.log("xbu reloc1:%d..." % (depth,), 6)
+                                        t = "xbu reloc1=%d ptop=%r vtop=%r prel=%r name=%r"
+                                        t = t % (
+                                            depth,
+                                            job["ptop"],
+                                            job["vtop"],
+                                            job["prel"],
+                                            job["name"],
+                                        )
+                                        self.log(t, 6)
+                                        zs = djoin(job["ptop"], job["prel"])
+                                        bos.makedirs(zs, vf=vfs.flags)
                                        return self._handle_json(job, depth + 1)

                        job["name"] = self._untaken(pdir, job, now)
@@ -3462,7 +3540,15 @@ class Up2k(object):
        fp = djoin(fdir, fname)

        ow = job.get("replace") and bos.path.exists(fp)
-        if ow and "mt" in str(job["replace"]).lower():
+        if ow:
+            replace_arg = str(job["replace"]).lower()
+
+        if ow and "skip" in replace_arg:  # type: ignore
+            self.log("skipping upload, filename already exists: %r" % fp)
+            err = "upload rejected, a file with that name already exists"
+            raise Pebkac(409, err)
+
+        if ow and "mt" in replace_arg:  # type: ignore
            mts = bos.stat(fp).st_mtime
            mtc = job["lmod"]
            if mtc < mts:
@@ -3528,13 +3614,30 @@ class Up2k(object):
        if self.args.nw:
            return

-        linked = False
+        linked = 0
        try:
-            if "reflink" in flags:
-                raise Exception("reflink")
+            if rm and bos.path.exists(dst):
+                wunlink(self.log, dst, flags)
+
            if not is_mv and not flags.get("dedup"):
                raise Exception("dedup is disabled in config")

+            if "reflink" in flags:
+                if not USE_FICLONE:
+                    raise Exception("reflink")  # python 3.14 or newer; no need
+                try:
+                    with open(fsenc(src), "rb") as fi, open(fsenc(dst), "wb") as fo:
+                        fcntl.ioctl(fo.fileno(), fcntl.FICLONE, fi.fileno())
+                        if "fperms" in flags:
+                            set_fperms(fo, flags)
+                except:
+                    if bos.path.exists(dst):
+                        wunlink(self.log, dst, flags)
+                    raise
+                if lmod:
+                    bos.utime_c(self.log, dst, int(lmod), False)
+                return
+
            lsrc = src
            ldst = dst
            fs1 = bos.stat(os.path.dirname(src)).st_dev
@@ -3561,13 +3664,10 @@ class Up2k(object):
                lsrc = lsrc.replace("/", "\\")
                ldst = ldst.replace("/", "\\")

-            if rm and bos.path.exists(dst):
-                wunlink(self.log, dst, flags)
-
            try:
                if "hardlink" in flags:
                    os.link(fsenc(absreal(src)), fsenc(dst))
-                    linked = True
+                    linked = 2
            except Exception as ex:
                self.log("cannot hardlink: " + repr(ex))
                if "hardlinkonly" in flags:
@@ -3586,7 +3686,7 @@ class Up2k(object):
                else:
                    os.symlink(fsenc(lsrc), fsenc(ldst))

-                linked = True
+                linked = 1
        except Exception as ex:
            if str(ex) != "reflink":
                self.log("cannot link; creating copy: " + repr(ex))
@@ -3600,12 +3700,16 @@ class Up2k(object):
                raise Exception(t % (src, fsrc, dst))
            trystat_shutil_copy2(self.log, fsenc(csrc), fsenc(dst))

-        if lmod and (not linked or SYMTIME):
-            bos.utime_c(self.log, dst, int(lmod), False)
+        if linked < 2 and (linked < 1 or SYMTIME):
+            if lmod:
+                bos.utime_c(self.log, dst, int(lmod), False)
+            if "fperms" in flags:
+                set_ap_perms(dst, flags)

    def handle_chunks(
        self, ptop: str, wark: str, chashes: list[str]
    ) -> tuple[list[str], int, list[list[int]], str, float, int, bool]:
+        self.fika = "u"
        with self.mutex, self.reg_mutex:
            self.db_act = self.vol_act[ptop] = time.time()
            job = self.registry[ptop].get(wark)
@@ -3708,6 +3812,7 @@ class Up2k(object):
    def confirm_chunks(
        self, ptop: str, wark: str, written: list[str], locked: list[str]
    ) -> tuple[int, str]:
+        self.fika = "u"
        with self.mutex, self.reg_mutex:
            return self._confirm_chunks(ptop, wark, written, locked, True)

@@ -3747,6 +3852,7 @@ class Up2k(object):

    def finish_upload(self, ptop: str, wark: str, busy_aps: dict[str, int]) -> None:
        self.busy_aps = busy_aps
+        self.fika = "u"
        with self.mutex, self.reg_mutex:
            self._finish_upload(ptop, wark)

@@ -3982,10 +4088,10 @@ class Up2k(object):
                sz,
                ip,
                at or time.time(),
-                "",
+                [dwark],
            )
            t = hr.get("rejectmsg") or ""
-            if t or not hr:
+            if t or hr.get("rc") != 0:
                if not t:
                    t = "upload blocked by xau server config: %r"
                    t = t % (djoin(vtop, rd, fn),)
@@ -4016,31 +4122,7 @@ class Up2k(object):
                with self.rescan_cond:
                    self.rescan_cond.notify_all()

-        if rd and sz and fn.lower() in self.args.th_coversd_set:
-            # wasteful; db_add will re-index actual covers
-            # but that won't catch existing files
-            crd, cdn = rd.rsplit("/", 1) if "/" in rd else ("", rd)
-            try:
-                q = "select fn from cv where rd=? and dn=?"
-                db_cv = db.execute(q, (crd, cdn)).fetchone()[0]
-                db_lcv = db_cv.lower()
-                if db_lcv in self.args.th_coversd_set:
-                    idx_db = self.args.th_coversd.index(db_lcv)
-                    idx_fn = self.args.th_coversd.index(fn.lower())
-                    add_cv = idx_fn < idx_db
-                else:
-                    add_cv = True
-            except:
-                add_cv = True
-
-            if add_cv:
-                try:
-                    db.execute("delete from cv where rd=? and dn=?", (crd, cdn))
-                    db.execute("insert into cv values (?,?,?)", (crd, cdn, fn))
-                except:
-                    pass
-
-        if "nodirsz" not in vflags:
+        if vflags and "nodirsz" not in vflags:
            try:
                q = "update ds set nf=nf+1, sz=sz+? where rd=?"
                q2 = "insert into ds values(?,?,1)"
@@ -4101,6 +4183,7 @@ class Up2k(object):
            vn0, rem0 = self.vfs.get(vpath, uname, *permsets[0])
            vn, rem = vn0.get_dbv(rem0)
            ptop = vn.realpath
+            self.fika = "d"
            with self.mutex, self.reg_mutex:
                abrt_cfg = vn.flags.get("u2abort", 1)
                addr = (ip or "\n") if abrt_cfg in (1, 2) else ""
@@ -4156,6 +4239,7 @@ class Up2k(object):
            st = bos.lstat(atop)
            is_dir = stat.S_ISDIR(st.st_mode)
        except:
+            # NOTE: "file not found" *sftpd
            raise Pebkac(400, "file not found on disk (already deleted?)")

        if "bcasechk" in vn.flags and not vn.casechk(rem, False):
@@ -4203,7 +4287,7 @@ class Up2k(object):
                    _ = dbv.get(volpath, uname, *permsets[0])

                if xbd:
-                    if not runhook(
+                    hr = runhook(
                        self.log,
                        None,
                        self,
@@ -4218,13 +4302,17 @@ class Up2k(object):
                        st.st_size,
                        ip,
                        time.time(),
-                        "",
-                    ):
-                        t = "delete blocked by xbd server config: %r"
-                        self.log(t % (abspath,), 1)
+                        None,
+                    )
+                    t = hr.get("rejectmsg") or ""
+                    if t or hr.get("rc") != 0:
+                        if not t:
+                            t = "delete blocked by xbd server config: %r" % (abspath,)
+                        self.log(t, 1)
                        continue

                n_files += 1
+                self.fika = "d"
                with self.mutex, self.reg_mutex:
                    cur = None
                    try:
@@ -4258,7 +4346,7 @@ class Up2k(object):
                        st.st_size,
                        ip,
                        time.time(),
-                        "",
+                        None,
                    )

        if is_dir:
@@ -4289,6 +4377,7 @@ class Up2k(object):
            raise Pebkac(400, "file does not exist case-sensitively")

        if stat.S_ISREG(st.st_mode) or stat.S_ISLNK(st.st_mode):
+            self.fika = "c"
            with self.mutex:
                try:
                    ret = self._cp_file(uname, ip, svp, dvp, curs)
@@ -4310,6 +4399,7 @@ class Up2k(object):

        # don't use svn_dbv; would skip subvols due to _ls `if not rem:`
        g = svn.walk("", srem, [], uname, permsets, dots, scandir, True)
+        self.fika = "c"
        with self.mutex:
            try:
                for dbv, vrem, _, atop, files, rd, vd in g:
@@ -4371,7 +4461,7 @@ class Up2k(object):
        xbc = svn.flags.get("xbc")
        xac = dvn.flags.get("xac")
        if xbc:
-            if not runhook(
+            hr = runhook(
                self.log,
                None,
                self,
@@ -4386,9 +4476,12 @@ class Up2k(object):
                fsize,
                ip,
                time.time(),
-                "",
-            ):
-                t = "copy blocked by xbr server config: %r" % (svp,)
+                None,
+            )
+            t = hr.get("rejectmsg") or ""
+            if t or hr.get("rc") != 0:
+                if not t:
+                    t = "copy blocked by xbr server config: %r" % (svp,)
                self.log(t, 1)
                raise Pebkac(405, t)

@@ -4470,6 +4563,8 @@ class Up2k(object):
                    bos.utime(dabs, times, False)
                except:
                    pass
+            if "fperms" in dvn.flags:
+                set_ap_perms(dabs, dvn.flags)

        if xac:
            runhook(
@@ -4487,7 +4582,7 @@ class Up2k(object):
                fsize,
                ip,
                time.time(),
-                "",
+                None,
            )

        return "k"
@@ -4510,6 +4605,7 @@ class Up2k(object):
            raise Pebkac(400, "file does not exist case-sensitively")

        if stat.S_ISREG(st.st_mode) or stat.S_ISLNK(st.st_mode):
+            self.fika = "m"
            with self.mutex:
                try:
                    ret = self._mv_file(uname, ip, svp, dvp, curs)
@@ -4533,6 +4629,7 @@ class Up2k(object):
                raise Pebkac(400, "mv: source folder contains other volumes")

        g = svn.walk("", srem, [], uname, permsets, 2, scandir, True)
+        self.fika = "m"
        with self.mutex:
            try:
                for dbv, vrem, _, atop, files, rd, vd in g:
@@ -4623,7 +4720,7 @@ class Up2k(object):
        xbr = svn.flags.get("xbr")
        xar = dvn.flags.get("xar")
        if xbr:
-            if not runhook(
+            hr = runhook(
                self.log,
                None,
                self,
@@ -4638,9 +4735,12 @@ class Up2k(object):
                fsize,
                ip,
                time.time(),
-                "",
-            ):
-                t = "move blocked by xbr server config: %r" % (svp,)
+                [dabs],
+            )
+            t = hr.get("rejectmsg") or ""
+            if t or hr.get("rc") != 0:
+                if not t:
+                    t = "move blocked by xbr server config: %r" % (svp,)
                self.log(t, 1)
                raise Pebkac(405, t)

@@ -4678,7 +4778,7 @@ class Up2k(object):
                    fsize,
                    ip,
                    time.time(),
-                    "",
+                    [sabs],
                )

            return "k"
@@ -4690,6 +4790,12 @@ class Up2k(object):
        if w:
            assert c1  # !rm
            if c2 and c2 != c1:
+                if "nodupem" in dvn.flags:
+                    q = "select w from up where substr(w,1,16) = ?"
+                    for (w2,) in c2.execute(q, (w[:16],)):
+                        if w == w2:
+                            t = "file exists in target volume, and dupes are forbidden in config"
+                            raise Pebkac(400, t)
                self._copy_tags(c1, c2, w)

            xlink = bool(svn.flags.get("xlink"))
@@ -4746,6 +4852,8 @@ class Up2k(object):
                wunlink(self.log, sabs, svn.flags)
            else:
                atomic_move(self.log, sabs, dabs, svn.flags)
+                if svn != dvn and "fperms" in dvn.flags:
+                    set_ap_perms(dabs, dvn.flags)

        except OSError as ex:
            if ex.errno != errno.EXDEV:
@@ -4779,6 +4887,8 @@ class Up2k(object):
                    bos.utime(dabs, times, False)
                except:
                    pass
+            if "fperms" in dvn.flags:
+                set_ap_perms(dabs, dvn.flags)

            wunlink(self.log, sabs, svn.flags)

@@ -4798,7 +4908,7 @@ class Up2k(object):
                fsize,
                ip,
                time.time(),
-                "",
+                [sabs],
            )

        return "k"
@@ -5102,7 +5212,7 @@ class Up2k(object):
        if eno not in E_FS_CRIT:
            return self.log("hashing failed; %r @ %r\n%s" % (ex, ap, min_ex()), 3)
        t = "hashing failed; %r @ %r\n%s\nWARNING: This MAY indicate a serious issue with your harddisk or filesystem! Please investigate %sOS-logs\n"
-        t2 = "" if ANYWIN or MACOS else "dmesg and "
+        t2 = "" if ANYWIN or UNIX else "dmesg and "
        return self.log(t % (ex, ap, min_ex(), t2), 1)

    def _new_upload(self, job: dict[str, Any], vfs: VFS, depth: int) -> dict[str, str]:
@@ -5136,10 +5246,10 @@ class Up2k(object):
                job["size"],
                job["addr"],
                job["t0"],
-                "",
+                [job["dwrk"]],
            )
            t = hr.get("rejectmsg") or ""
-            if t or not hr:
+            if t or hr.get("rc") != 0:
                if not t:
                    t = "upload blocked by xbu server config: %r" % (vp_chk,)
                self.log(t, 1)
@@ -5148,15 +5258,25 @@ class Up2k(object):
                x = pathmod(self.vfs, ap_chk, vp_chk, hr["reloc"])
                if x:
                    ud1 = (vfs.vpath, job["prel"], job["name"])
+                    job["rvp0"] = vjoins(*ud1)
                    pdir, _, job["name"], (vfs, rem) = x
-                    job["vcfg"] = vf = vfs.flags
+                    vf = vfs.flags
                    job["ptop"] = vfs.realpath
                    job["vtop"] = vfs.vpath
                    job["prel"] = rem
-                    job["name"] = sanitize_fn(job["name"], "")
+                    job["name"] = sanitize_fn(job["name"])
                    ud2 = (vfs.vpath, job["prel"], job["name"])
                    if ud1 != ud2:
-                        self.log("xbu reloc2:%d..." % (depth,), 6)
+                        job["vcfg"] = vf
+                        t = "xbu reloc2=%d ptop=%r vtop=%r prel=%r name=%r" % (
+                            depth,
+                            job["ptop"],
+                            job["vtop"],
+                            job["prel"],
+                            job["name"],
+                        )
+                        self.log(t, 6)
+                        bos.makedirs(djoin(job["ptop"], job["prel"]), vf=vf)
                        return self._handle_json(job, depth + 1)

        job["name"] = self._untaken(pdir, job, job["t0"])
@@ -5196,9 +5316,12 @@ class Up2k(object):
                and self.args.sparse * 1024 * 1024 <= sz
            ):
                try:
-                    sp.check_call(["fsutil", "sparse", "setflag", abspath])
-                except:
-                    self.log("could not sparse %r" % (abspath,), 3)
+                    winsparse(f)
+                    # some kernels/etc are buggy; force sync:
+                    f.close()
+                    f = open(abspath, "rb+")
+                except Exception as ex:
+                    self.log("could not sparse %r: %s" % (abspath, ex), 3)
                    relabel = True
                    sprs = False

@@ -5258,6 +5381,7 @@ class Up2k(object):
                self.do_snapshot()

    def do_snapshot(self) -> None:
+        self.fika = "u"
        with self.mutex, self.reg_mutex:
            for k, reg in self.registry.items():
                self._snap_reg(k, reg)
@@ -5362,7 +5486,7 @@ class Up2k(object):
            # self.log("\n  " + repr([ptop, rd, fn]))
            abspath = djoin(ptop, rd, fn)
            try:
-                tags = self.mtag.get(abspath) if sz else {}
+                tags = self.mtag.get(abspath, self.flags[ptop]) if sz else {}
                ntags1 = len(tags)
                parsers = self._get_parsers(ptop, tags, abspath)
                if self.args.mtag_vv:
@@ -5538,6 +5662,7 @@ class Up2k(object):

    def shutdown(self) -> None:
        self.stop = True
+        self.fika = "f"

        if self.mth:
            self.mth.stop = True
@@ -5583,7 +5708,7 @@ def up2k_chunksize(filesize: int) -> int:
    stepsize = 512 * 1024
    while True:
        for mul in [1, 2]:
-            nchunks = math.ceil(filesize * 1.0 / chunksize)
+            nchunks = -int(-filesize // chunksize)
            if nchunks <= 256 or (chunksize >= 32 * 1024 * 1024 and nchunks <= 4096):
                return chunksize

@@ -5591,6 +5716,31 @@ def up2k_chunksize(filesize: int) -> int:
            stepsize *= mul


+def up2k_hashlist_from_file(path: str) -> tuple[list[str], os.stat_result]:
+    """not used by copyparty itself, only by some hooks"""
+    st = bos.stat(path)
+    fsz = st.st_size
+    csz = up2k_chunksize(fsz)
+    ret = []
+    with open(fsenc(path), "rb", 256 * 1024) as f:
+        while fsz > 0:
+            hashobj = hashlib.sha512()
+            rem = min(csz, fsz)
+            fsz -= rem
+            while rem > 0:
+                buf = f.read(min(rem, 64 * 1024))
+                if not buf:
+                    raise Exception("EOF at " + str(f.tell()))
+
+                hashobj.update(buf)
+                rem -= len(buf)
+
+            digest = hashobj.digest()[:33]
+            ret.append(ub64enc(digest).decode("ascii"))
+
+    return ret, st
+
+
 def up2k_wark_from_hashlist(salt: str, filesize: int, hashes: list[str]) -> str:
    """server-reproducible file identifier, independent of name or location"""
    values = [salt, str(filesize)] + hashes
@@ -2,11 +2,19 @@
 # which should help on really slow connections
 # but then why are you using copyparty in the first place

-pk: $(addsuffix .gz, $(wildcard *.js *.css))
-un: $(addsuffix .un, $(wildcard *.gz))
+pk: $(addsuffix .gz, $(wildcard tl/*.js *.js *.css) \
+	a/webdav-cfg.txt )
+un: $(addsuffix .un, $(wildcard tl/*.gz *.gz a/*.gz))
+
+tl/%.js.gz: tl/%.js
+	./Makefile.s1 <$< | pigz -c11 -J 34 -I 573 >$@
+	touch -r $< $@
+	rm $<

 %.gz: %
-	pigz -11 -J 34 -I 573 $<
+	pigz -c11 -J 34 -I 573 <$< >$@
+	touch -r $< $@
+	rm $<

 %.un: %
 	pigz -d $<
@@ -0,0 +1 @@
+sed -r 's/^\t+//;s/^"([a-zA-Z][a-zA-Z0-9_]*)": /\1:/; /^\/\//d; s/([^\]["'\''],) *\/\/.*/\1/; /^$/d'
@@ -1,3 +1,5 @@
+"use strict";
+
 /*!
 * baguetteBox.js
 * @author  feimosi
@@ -5,6 +7,8 @@
 * @url https://github.com/feimosi/baguetteBox.js
 */

+var J_BBX = 1;
+
 window.baguetteBox = (function () {
    'use strict';

@@ -24,7 +28,7 @@ window.baguetteBox = (function () {
            onChange: null,
            readDirRtl: false,
        },
-        overlay, slider, btnPrev, btnNext, btnHelp, btnAnim, btnRotL, btnRotR, btnSel, btnFull, btnVmode, btnReadDir, btnClose,
+        overlay, slider, btnPrev, btnNext, btnHelp, btnAnim, btnRotL, btnRotR, btnSel, btnFull, btnZoom, btnVmode, btnReadDir, btnGoPage, btnClose,
        currentGallery = [],
        currentIndex = 0,
        isOverlayVisible = false,
@@ -32,13 +36,17 @@ window.baguetteBox = (function () {
        touchFlag = false,  // busy
        scrollCSS = ['', ''],
        scrollTimer = 0,
-        re_i = /^[^?]+\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp)(\?|$)/i,
+        re_i = APPLE ?
+            /^[^?]+\.(a?png|avif|bmp|gif|hei[cf]s?|jfif|jpe?g|jxl|svg|tiff?|webp)(\?|$)/i :
+            /^[^?]+\.(a?png|avif|bmp|gif|jfif|jpe?g|jxl|svg|tiff?|webp)(\?|$)/i,
        re_v = /^[^?]+\.(webm|mkv|mp4|m4v|mov)(\?|$)/i,
+        re_cbz = /^[^?]+\.(cbz)(\?|$)/i,
        anims = ['slideIn', 'fadeIn', 'none'],
        data = {},  // all galleries
        imagesElements = [],
        documentLastFocus = null,
        isFullscreen = false,
+        isCbz = false,
        vmute = false,
        vloop = sread('vmode') == 'L',
        vnext = sread('vmode') == 'C',
@@ -58,6 +66,15 @@ window.baguetteBox = (function () {
            hideOverlay();
    };

+    var vtouch = function (e) {
+        var v = vid(),
+            bv = v.getBoundingClientRect(),
+            tp = e.changedTouches[0];
+
+        if (bv.bottom - tp.clientY < 90)
+            touchFlag = true;
+    };
+
    var touchstartHandler = function (e) {
        touch.count = e.touches.length;
        if (touch.count > 1)
@@ -121,6 +138,9 @@ window.baguetteBox = (function () {
    };

    var trapFocusInsideOverlay = function (e) {
+        if (modal.busy)
+            return;
+
        if (overlay.style.display === 'block' && (overlay.contains && !overlay.contains(e.target))) {
            e.stopPropagation();
            btnClose.focus();
@@ -147,6 +167,8 @@ window.baguetteBox = (function () {
                tagsNodeList = [galleryElement];
            else
                tagsNodeList = galleryElement.getElementsByTagName('a');
+            if (have_zls)
+                bindCbzClickListeners(tagsNodeList, userOptions);

            tagsNodeList = [].filter.call(tagsNodeList, function (element) {
                if (element.className.indexOf(userOptions && userOptions.ignoreClass) === -1)
@@ -167,7 +189,7 @@ window.baguetteBox = (function () {
                };
                var imageItem = {
                    eventHandler: imageElementClickHandler,
-                    imageElement: imageElement
+                    imageElement: imageElement,
                };
                bind(imageElement, 'click', imageElementClickHandler);
                gallery.push(imageItem);
@@ -178,6 +200,95 @@ window.baguetteBox = (function () {
        return [selectorData.galleries, options];
    }

+    function getCbzPage() {
+        try {
+            var ret = parseInt(/,p([0-9]+)(,|$)/.exec(location.hash)[1]);
+            if (isNum(ret))
+                return Math.max(Math.min(ret, currentGallery.length), 1) - 1;
+        }
+        catch (ex) { }
+        return 0;
+    }
+
+    function bindCbzClickListeners(tagsNodeList, userOptions) {
+        var cbzNodes = [].filter.call(tagsNodeList, function (element) {
+            return re_cbz.test(element.href);
+        });
+        if (!tagsNodeList.length) {
+            return;
+        }
+
+        [].forEach.call(cbzNodes, function (cbzElement, index) {
+            var gallery = [];
+            var eventHandler = function (e) {
+                if (ctrl(e) || e && e.shiftKey)
+                    return true;
+
+                e.preventDefault ? e.preventDefault() : e.returnValue = false;
+                fillCbzGallery(gallery, cbzElement, eventHandler).then(function () {
+                    prepareOverlay(gallery, userOptions, true);
+                    showOverlay(getCbzPage());
+                }).catch(function (reason) {
+                    console.error("cbz-ded", reason);
+                    var t;
+                    try {
+                        t = uricom_dec(cbzElement.href.split('/').pop());
+                    } catch (ex) { }
+
+                    var msg = "Could not browse " + (t ? t : 'archive');
+                    try {
+                        msg += "\n\n" + reason.message;
+                    } catch (ex) { }
+                    toast.err(20, msg, 'cbz-ded');
+                });
+            }
+
+            bind(cbzElement, "click", eventHandler);
+        })
+    }
+
+    function fillCbzGallery(gallery, cbzElement, eventHandler) {
+        if (gallery.length !== 0) {
+            return Promise.resolve();
+        }
+        var href = cbzElement.href;
+        var zlsHref = href + (href.indexOf("?") === -1 ? "?" : "&") + "zls";
+        return fetch(zlsHref)
+            .then(function (response) {
+                if (response.ok) {
+                    return response.json();
+                } else {
+                    throw new Error("Archive is invalid");
+                }
+            })
+            .then(function (fileList) {
+                var imagesList = fileList.map(function (file) {
+                    return file["fn"];
+                }).filter(function (file) {
+                    return re_i.test(file);
+                }).sort();
+
+                if (imagesList.length === 0) {
+                    throw new Error("Archive does not contain any images");
+                }
+
+                imagesList.forEach(function (imageName, index) {
+                    var imageHref = href
+                        + (href.indexOf("?") === -1 ? "?" : "&")
+                        + "zget="
+                        + encodeURIComponent(imageName);
+
+                    var galleryItem = {
+                        href: imageHref,
+                        imageElement: cbzElement,
+                        eventHandler: eventHandler,
+                        page: index + 1,
+                    };
+                    gallery.push(galleryItem);
+                });
+            });
+    }
+
    function clearCachedData() {
        for (var selector in data)
            if (data.hasOwnProperty(selector))
@@ -217,8 +328,10 @@ window.baguetteBox = (function () {
                '<button id="bbox-rotl" type="button">↶</button>' +
                '<button id="bbox-rotr" type="button">↷</button>' +
                '<button id="bbox-tsel" type="button">sel</button>' +
-                '<button id="bbox-full" type="button">⛶</button>' +
+                '<button id="bbox-full" type="button" tt="full-screen">⛶</button>' +
+                '<button id="bbzoom" type="button" tt="zoom/stretch">z</button>' +
                '<button id="bbox-vmode" type="button" tt="a"></button>' +
+                '<button id="bbox-gopage" type="button" tt="go to page">pg</button>' +
                '<button id="bbox-close" type="button" aria-label="Close">X</button>' +
                '</div></div>'
            );
@@ -236,8 +349,13 @@ window.baguetteBox = (function () {
        btnRotR = ebi('bbox-rotr');
        btnSel = ebi('bbox-tsel');
        btnFull = ebi('bbox-full');
+        btnZoom = ebi('bbzoom');
        btnVmode = ebi('bbox-vmode');
+        btnGoPage = ebi('bbox-gopage');
        btnClose = ebi('bbox-close');
+
+        bcfg_bind(options, 'bbzoom', 'bbzoom', false, setzoom);
+        setzoom();
    }

    function halp() {
@@ -253,7 +371,9 @@ window.baguetteBox = (function () {
            ['end', 'last file'],
            ['R', 'rotate (shift=ccw)'],
            ['F', 'toggle fullscreen'],
+            ['Z', 'toggle zoom/stretch'],
            ['S', 'toggle file selection'],
+            ['G', 'cbz: go to page'],
            ['space, P, K', 'video: play / pause'],
            ['U', 'video: seek 10sec back'],
            ['O', 'video: seek 10sec ahead'],
@@ -317,7 +437,7 @@ window.baguetteBox = (function () {
            playpause();
        else if (kl == "u" || kl == "o")
            relseek(kl == "u" ? -10 : 10);
-        else if (v && /^[0-9]$/.test(k))
+        else if (v && /^[0-9]$/.test(k) && isNum(v.duration))
            v.currentTime = v.duration * parseInt(k) * 0.1;
        else if (kl == "m" && v) {
            v.muted = vmute = !vmute;
@@ -335,8 +455,12 @@ window.baguetteBox = (function () {
        }
        else if (kl == "f")
            tglfull();
+        else if (kl == "z")
+            btnZoom.click();
        else if (kl == "s")
            tglsel();
+        else if (kl == "g" && isCbz)
+            gotoCbzPage(e);
        else if (kl == "r")
            rotn(e.shiftKey ? -1 : 1);
        else if (kl == "y")
@@ -402,6 +526,24 @@ window.baguetteBox = (function () {
            v.play();
    }

+    function gotoCbzPage(e) {
+        ev(e);
+
+        var cur = currentIndex + 1,
+            max = currentGallery.length;
+
+        modal.prompt('Go to page (1-' + max + ')', '' + cur, function (v) {
+            if (!v)
+                return;
+
+            v = parseInt(v);
+            if (!isNum(v))
+                return toast.warn(4, 'invalid input');
+
+            show(Math.max(Math.min(v, max), 1) - 1);
+        });
+    }
+
    function tglVmode() {
        if (vloop) {
            vnext = true;
@@ -442,6 +584,12 @@ window.baguetteBox = (function () {
        }
    }

+    function setzoom() {
+        var sel = clgot(btnZoom, 'on')
+        clmod(ebi('bbox-overlay'), 'fill', sel);
+        btnState(btnZoom, sel);
+    }
+
    function tglsel() {
        var o = findfile()[3];
        clmod(o.closest('tr'), 'sel', 't');
@@ -469,10 +617,14 @@ window.baguetteBox = (function () {
            'rgba(153,34,85,0.7)' : '';

        img.style.borderRadius = sel ? '1em' : '';
-        btnSel.style.color = sel ? '#fff' : '';
-        btnSel.style.background = sel ? '#d48' : '';
-        btnSel.style.textShadow = sel ? '1px 1px 0 #b38' : '';
-        btnSel.style.boxShadow = sel ? '.15em .15em 0 #502' : '';
+        btnState(btnSel, sel);
+    }
+
+    function btnState(btn, sel) {
+        btn.style.color = sel ? '#fff' : '';
+        btn.style.background = sel ? '#d48' : '';
+        btn.style.textShadow = sel ? '1px 1px 0 #b38' : '';
+        btn.style.boxShadow = sel ? '.15em .15em 0 #502' : '';
    }

    function keyUpHandler(e) {
@@ -521,6 +673,7 @@ window.baguetteBox = (function () {
        bind(btnRotR, 'click', rotr);
        bind(btnSel, 'click', tglsel);
        bind(btnFull, 'click', tglfull);
+        bind(btnGoPage, 'click', gotoCbzPage);
        bind(slider, 'contextmenu', contextmenuHandler);
        bind(overlay, 'touchstart', touchstartHandler, nonPassiveEvent);
        bind(overlay, 'touchmove', touchmoveHandler, passiveEvent);
@@ -545,6 +698,7 @@ window.baguetteBox = (function () {
        unbind(btnRotR, 'click', rotr);
        unbind(btnSel, 'click', tglsel);
        unbind(btnFull, 'click', tglfull);
+        unbind(btnGoPage, 'click', gotoCbzPage);
        unbind(slider, 'contextmenu', contextmenuHandler);
        unbind(overlay, 'touchstart', touchstartHandler, nonPassiveEvent);
        unbind(overlay, 'touchmove', touchmoveHandler, passiveEvent);
@@ -553,10 +707,13 @@ window.baguetteBox = (function () {
        timer.rm(rotn);
    }

-    function prepareOverlay(gallery, userOptions) {
+    function prepareOverlay(gallery, userOptions, cbz) {
        if (currentGallery === gallery)
            return;

+        isCbz = cbz;
+        btnGoPage.style.display = isCbz ? '' : 'none';
+
        currentGallery = gallery;
        setOptions(userOptions);
        slider.innerHTML = '';
@@ -649,7 +806,7 @@ window.baguetteBox = (function () {
        overlay.style.display = 'block';
        // Fade in overlay
        setTimeout(function () {
-            overlay.className = 'visible';
+            clmod(overlay, 'visible', 1);
            if (options.bodyClass && document.body.classList)
                document.body.classList.add(options.bodyClass);

@@ -658,7 +815,7 @@ window.baguetteBox = (function () {
        }, 50);

        if (options.onChange && !url_ts)
-            options.onChange(currentIndex, imagesElements.length);
+            options.onChange.call(currentGallery, currentIndex, imagesElements.length);

        url_ts = null;
        documentLastFocus = document.activeElement;
@@ -695,7 +852,7 @@ window.baguetteBox = (function () {
        unbindEvents();

        // Fade out and hide the overlay
-        overlay.className = '';
+        clmod(overlay, 'visible');
        setTimeout(function () {
            overlay.style.display = 'none';
            if (options.bodyClass && document.body.classList)
@@ -720,6 +877,7 @@ window.baguetteBox = (function () {
            if (v == keep)
                continue;

+            unbind(v, 'touchstart', vtouch, nonPassiveEvent);
            unbind(v, 'error', lerr);
            v.src = '';
            v.load();
@@ -786,11 +944,11 @@ window.baguetteBox = (function () {
            imageContainer.removeChild(imageContainer.firstChild);

        var imageElement = galleryItem.imageElement,
-            imageSrc = imageElement.href,
+            imageSrc = galleryItem.href || imageElement.href,
            is_vid = re_v.test(imageSrc),
            thumbnailElement = imageElement.querySelector('img, video'),
            imageCaption = typeof options.captions === 'function' ?
-                options.captions.call(currentGallery, imageElement) :
+                options.captions.call(currentGallery, imageElement, index) :
                imageElement.getAttribute('data-caption') || imageElement.title;

        imageSrc = addq(imageSrc, 'cache');
@@ -849,7 +1007,7 @@ window.baguetteBox = (function () {
            un_pp = 0;
            return playpause();  // browser undid space hotkey
        }
-        show_buttons(this.paused ? 1 : 0);
+        show_buttons(this.paused ? 0 : 1);
    }

    function showRightImage(e) {
@@ -902,7 +1060,9 @@ window.baguetteBox = (function () {
        if (index >= imagesElements.length)
            return bounceAnimation(options.readDirRtl ? 'left' : 'right');

+        var orot;
        try {
+            orot = vidimg().getAttribute('rot');
            vid().pause();
        }
        catch (ex) { }
@@ -920,6 +1080,9 @@ window.baguetteBox = (function () {
        else if (toast.tag == 'bb-ded')
            toast.hide();

+        if (orot && im.getAttribute('rot') === null)
+            rotn(orot / 90, 1);
+
        if (options.animation == 'none')
            unvid(vid());
        else
@@ -930,13 +1093,13 @@ window.baguetteBox = (function () {
        unfig(index);

        if (options.onChange)
-            options.onChange(currentIndex, imagesElements.length);
+            options.onChange.call(currentGallery, currentIndex, imagesElements.length);

        return true;
    }

    var prev_cw = 0, prev_ch = 0, unrot_timer = null;
-    function rotn(n) {
+    function rotn(n, asap) {
        var el = vidimg(),
            orot = parseInt(el.getAttribute('rot') || 0),
            frot = orot + (n || 0) * 90;
@@ -951,6 +1114,8 @@ window.baguetteBox = (function () {
        if (!n && prev_cw === cw && prev_ch === ch)
            return;  // reflow noop

+        clmod(el, 'asap', asap);
+
        prev_cw = cw;
        prev_ch = ch;
        var rot = frot,
@@ -960,8 +1125,13 @@ window.baguetteBox = (function () {
            dl = el.closest('div').querySelector('figcaption a'),
            vw = cw,
            vh = ch - dl.offsetHeight + magic,
-            pmag = Math.min(1, Math.min(vw / ih, vh / iw)),
-            wmag = Math.min(1, Math.min(vw / iw, vh / ih));
+            pmag = Math.min(vw / ih, vh / iw),
+            wmag = Math.min(vw / iw, vh / ih);
+
+        if (!options.bbzoom) {
+            pmag = Math.min(1, pmag);
+            wmag = Math.min(1, wmag);
+        }

        while (rot < 0) rot += 360;
        while (rot >= 360) rot -= 360;
@@ -1005,7 +1175,7 @@ window.baguetteBox = (function () {
            return;

        clmod(el, 'nt', 1);
-        el.removeAttribute('rot');
+        el.setAttribute('rot', 0);
        el.removeAttribute("style");
        rot = el.offsetHeight;
        clmod(el, 'nt');
@@ -1048,8 +1218,11 @@ window.baguetteBox = (function () {
    }

    function relseek(sec) {
-        if (vid())
-            vid().currentTime += sec;
+        var v = vid(),
+            t = v && v.currentTime;
+
+        if (isNum(t))
+            v.currentTime = t + sec;
    }

    function vidEnd() {
@@ -1148,6 +1321,7 @@ window.baguetteBox = (function () {
                    setloop();
                }
            }
+            bind(v, 'touchstart', vtouch, nonPassiveEvent);
        }
        selbg();
        mp_ctl();
@@ -1234,3 +1408,5 @@ window.baguetteBox = (function () {
        destroy: destroyPlugin
    };
 })();
+
+J_BBX = 2;
@@ -1036,6 +1036,7 @@ html.dz #flogout {
 	box-shadow: 0 .1em 1.2em var(--g-play-sh);
 }
 #files tbody tr.sel td,
+#files tbody tr.sel span,
 #ggrid>a.sel,
 #ggrid>a[tt].sel {
 	color: var(--g-sel-fg);
@@ -1455,7 +1456,8 @@ html.dz input {
 	width: calc(100% - 16.2em);
 }
 input.drc_v,
-input.eq_gain {
+input.eq_gain,
+input.ssconf_v {
 	width: 3em;
 	text-align: center;
 	margin: 0 .6em;
@@ -1465,7 +1467,8 @@ input.eq_gain {
 	border-collapse: collapse;
 }
 #audio_drc td,
-#audio_eq td {
+#audio_eq td,
+#audio_ss td {
 	text-align: center;
 }
 #audio_eq a.eq_step {
@@ -1473,15 +1476,22 @@ input.eq_gain {
 	display: block;
 	padding: 0;
 }
+#au_ss,
 #au_drc,
 #au_eq {
 	display: block;
 	margin-top: .5em;
 	padding: 1.3em .3em;
 }
+#au_ss,
 #au_drc {
 	padding: .4em .3em;
 }
+#au_ss.on {
+    width: 3em;
+	height: 1em;
+	overflow: hidden;
+}
 #ico1 {
 	cursor: pointer;
 }
@@ -2165,6 +2175,16 @@ html.noscroll .sbar::-webkit-scrollbar {
 	vertical-align: middle;
 	transition: transform .23s, left .23s, top .23s, width .23s, height .23s;
 }
+.full-image img.asap,
+.full-image video.asap {
+	transition: none;
+}
+#bbox-overlay.fill .full-image img,
+#bbox-overlay.fill .full-image video {
+    width: 100%;
+    height: 100%;
+    object-fit: contain;
+}
 html.bb_fsc .full-image img,
 html.bb_fsc .full-image video {
 	max-height: 100%;
@@ -3140,9 +3160,6 @@ html.bz .ghead {
 html.b #files td {
 	padding: .5em .7em;
 }
-html.b #ggrid>a {
-	margin: .8em;
-}
 html.b .btn {
 	top: -.1em;
 }
@@ -3287,8 +3304,8 @@ html.d #treepar {
 	}

 	html.b #ggrid {
-		padding: 0 2em 2em 0;
-		gap: .5em 3em;
+		padding: 0 0 2em 0;
+		gap: 1em 1.5em;
 	}

 	#ggrid > a {
@@ -4117,3 +4134,57 @@ html.e #doc {
 html.e #detree {
  padding: 0px;
 }
+
+#rcm {
+	position: absolute;
+	display: none;
+	background: #fff;
+	background: var(--bg);
+	border: 1px solid var(--bg-u3);
+	outline: none;
+	border-radius: .3rem;
+	box-shadow: 0 0 .3rem var(--bg-d3);
+	z-index: 3;
+}
+
+#rcm > * {
+	display: block;
+}
+
+#rcm > a {
+	padding: .2rem .3rem;
+}
+
+#rcm > .hide {
+	display: none;
+}
+
+#rcm > a:hover {
+	background: var(--bg-d2);
+}
+
+#rcm > .sep {
+	margin: 0 .2rem;
+	border-bottom: 1px solid var(--a-gray);
+}
+
+#tempname {
+	color: var(--fg);
+	background: var(--txt-bg);
+	border: none;
+	box-shadow: 0 0 2px var(--txt-sh);
+	border-bottom: 1px solid #999;
+	border-color: var(--a);
+	border-radius: .2em;
+	padding: .2em .3em;
+}
+
+.selbox {
+    position: fixed;
+    border: .5em solid #f0f;
+    border: .2em solid var(--btn-1h-bg);
+    background-color: rgba(128, 128, 128, 0.6);
+    background-color: rgb(from var(--btn-1h-bg) r g b / 0.5);
+    pointer-events: none;
+	z-index: 99;
+}
@@ -1,5 +1,5 @@
 <!DOCTYPE html>
-<html lang="en">
+<html lang="en" id="ht_brw">

 <head>
 	<meta charset="utf-8">
@@ -7,8 +7,8 @@
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.8, minimum-scale=0.6">
 	<meta name="theme-color" content="#{{ tcolor }}">
-	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
-	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/browser.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/w/ui.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/w/browser.css?_={{ ts }}">
 {{ html_head }}
 	{%- if css %}
 	<link rel="stylesheet" media="screen" href="{{ css }}_={{ ts }}">
@@ -51,8 +51,9 @@
 		<form method="post" enctype="multipart/form-data" accept-charset="utf-8" action="{{ url_suf }}">
 			<input type="hidden" name="act" value="new_md" />
 			📝<input type="text" name="name" class="i" placeholder="weekend-plans">
-			<input type="submit" value="new markdown doc">
+			<input type="submit" value="new file">
 		</form>
+		<span id="new_mdi"></span>
 	</div>

 	<div id="op_msg" class="opview opbox {% if not ls0 %}act{% endif %}">
@@ -128,8 +129,11 @@

 	<div id="widget"></div>

-	<script>
+	<div id="rcm" tabindex="0"></div>
+
+	<script nonce="{{ js_nonce }}">
 		var SR = "{{ r }}",
+			JS_NONCE = "{{ js_nonce }}",
 			CGV1 = {{ cgv1 }},
 			CGV = {{ cgv|tojson }},
 			TS = "{{ ts }}",
@@ -137,21 +141,29 @@
 			lang = "{{ lang }}",
 			dfavico = "{{ favico }}",
 			have_tags_idx = {{ have_tags_idx }},
-			sb_lg = "{{ sb_lg }}",
 			logues = {{ logues|tojson if sb_lg else "[]" }},
 			ls0 = {{ ls0|tojson }};

 		var STG = window.localStorage;
 		document.documentElement.className = (STG && STG.cpp_thm) || dtheme;
 	</script>
-	<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
-	<script src="{{ r }}/.cpr/baguettebox.js?_={{ ts }}"></script>
-	<script src="{{ r }}/.cpr/browser.js?_={{ ts }}"></script>
-	<script src="{{ r }}/.cpr/up2k.js?_={{ ts }}"></script>
-	{%- if js %}
-	<script src="{{ js }}_={{ ts }}"></script>
+	<script nonce="{{ js_nonce }}" src="{{ r }}/.cpr/w/util.js?_={{ ts }}"></script>
+	{%- if lang != "eng" %}
+	<script nonce="{{ js_nonce }}" src="{{ r }}/.cpr/w/tl/{{ lang }}.js?_={{ ts }}"></script>
 	{%- endif %}
+	<script nonce="{{ js_nonce }}" src="{{ r }}/.cpr/w/baguettebox.js?_={{ ts }}"></script>
+	<script nonce="{{ js_nonce }}" src="{{ r }}/.cpr/w/browser.js?_={{ ts }}"></script>
+	<script nonce="{{ js_nonce }}" src="{{ r }}/.cpr/w/up2k.js?_={{ ts }}"></script>
+	{%- if js %}
+	<script nonce="{{ js_nonce }}" src="{{ js }}_={{ ts }}"></script>
+	{%- endif %}
+	<script nonce="{{ js_nonce }}">
+		Date.now();function jsldp(a,b){2!=window[a]&&alert("FATAL ERROR: cannot load "+b+".js due to unreliable network or broken reverse-proxy; try CTRL-SHIFT-R")}
+		jsldp("J_UTL","util");
+		jsldp("J_BBX","baguettebox");
+		jsldp("J_BRW","browser");
+		jsldp("J_U2K","up2k");
+	</script>
 </body>
-
 </html>

@@ -28,7 +28,11 @@
 	{%- endif %}

 	{%- if logues[0] %}
-	<div>{{ logues[0] }}</div><br />
+		{%- if sb_lg %}
+	<div>{{ logues[0][:2000]|e }}</div><br />
+		{%- else %}
+	<div>{{ logues[0][:2000] }}</div><br />
+		{%- endif %}
 	{%- endif %}

 	<table id="files">
@@ -53,7 +57,11 @@
 	</table>

 	{%- if logues[1] %}
-	<div>{{ logues[1] }}</div><br />
+		{%- if sb_lg %}
+	<div>{{ logues[1][:2000]|e }}</div><br />
+		{%- else %}
+	<div>{{ logues[1][:2000] }}</div><br />
+		{%- endif %}
 	{%- endif %}

 	<h2><a href="{{ r }}/{{ url_suf }}{{ url_suf and '&amp;' or '?' }}h">control-panel</a></h2>
@@ -14,7 +14,7 @@
 		<p>sorry for the inconvenience</p>
 	</div>

-	<script>
+	<script nonce="{{ js_nonce }}">
 		setTimeout(function() {
 			document.getElementById('box').style.opacity = 1;
 		}, 500);
@@ -0,0 +1,18 @@
+this folder *mostly* contains third-party dependencies; run `make -C scripts/deps-docker` to build the following files and have them appear here:
+
+* `easymde.css.gz` and `easymde.js.gz` is the fancy markdown editor, [EasyMDE](https://github.com/Ionaru/easy-markdown-editor)
+* `marked.js.gz` is the markdown rendering library [Marked](https://github.com/markedjs/marked)
+* `mini-fa.css.gz` and `mini-fa.woff` is a small subset of [fontawesome](https://github.com/FortAwesome/Font-Awesome)
+* `prism.css.gz` and `prism.js.gz` is the syntax highlighter [PrismJS](https://prismjs.com/)
+* `scp.woff2` is a subset of the monospace font [Source Code Pro](https://github.com/adobe-fonts/source-code-pro)
+* `sha512.hw.js.gz` is the Wasm sha512 library [hash-wasm](https://github.com/Daninet/hash-wasm)
+
+additionally, the following files are vendored into the copyparty git repository, but do NOT originate from the copyparty project (as mentioned in `--license`):
+
+* `sha512.ac.js.gz` is a compiled and slightly golfed/modified [asmcrypto.js](https://github.com/asmcrypto/asmcrypto.js), © 2013 Artem S Vybornov (MIT-Licensed)
+  * vendored because it no longer builds with modern versions of NodeJS/npm
+  * is only loaded by *really old* webbrowsers (ie11, firefox 51, chrome 56)
+
+finally, there is also the following files which *does* originate from the copyparty project, yet appear here for technical reasons:
+
+* `busy.mp3.gz` is a short mp3-file to make iphones stop glitching out
@@ -1,5 +1,5 @@
 <!DOCTYPE html>
-<html lang="en">
+<html lang="en" id="ht_idp">

 <head>
 	<meta charset="utf-8">
@@ -8,8 +8,8 @@
 	<meta name="viewport" content="width=device-width, initial-scale=0.8">
    <meta name="robots" content="noindex, nofollow">
 	<meta name="theme-color" content="#{{ tcolor }}">
-	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/shares.css?_={{ ts }}">
-	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/w/shares.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/w/ui.css?_={{ ts }}">
 {{ html_head }}
 </head>

@@ -36,9 +36,10 @@
        {%- endif %}
    </div>
 	<a href="#" id="repl">π</a>
-	<script>
+	<script nonce="{{ js_nonce }}">

 var SR="{{ r }}",
+	JS_NONCE = "{{ js_nonce }}",
 	lang="{{ lang }}",
 	dfavico="{{ favico }}";

@@ -46,10 +47,14 @@ var STG = window.localStorage;
 document.documentElement.className = (STG && STG.cpp_thm) || "{{ this.args.theme }}";

 </script>
-<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
+<script nonce="{{ js_nonce }}" src="{{ r }}/.cpr/w/util.js?_={{ ts }}"></script>
 {%- if js %}
-<script src="{{ js }}_={{ ts }}"></script>
+<script nonce="{{ js_nonce }}" src="{{ js }}_={{ ts }}"></script>
 {%- endif %}
+<script nonce="{{ js_nonce }}">
+    Date.now();function jsldp(a,b){2!=window[a]&&alert("FATAL ERROR: cannot load "+b+".js due to unreliable network or broken reverse-proxy; try CTRL-SHIFT-R")}
+    jsldp("J_UTL","util");
+</script>
 </body>
 </html>

@@ -1,16 +1,20 @@
-<!DOCTYPE html><html><head>
+<!DOCTYPE html>
+<html lang="en" id="ht_md">
+
+<head>
 	<meta charset="utf-8">
 	<title>📝 {{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.7">
 	<meta name="theme-color" content="#{{ tcolor }}">
-	<link rel="stylesheet" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
-	<link rel="stylesheet" href="{{ r }}/.cpr/md.css?_={{ ts }}">
+	<link rel="stylesheet" href="{{ r }}/.cpr/w/ui.css?_={{ ts }}">
+	<link rel="stylesheet" href="{{ r }}/.cpr/w/md.css?_={{ ts }}">
 	{%- if edit %}
-	<link rel="stylesheet" href="{{ r }}/.cpr/md2.css?_={{ ts }}">
+	<link rel="stylesheet" href="{{ r }}/.cpr/w/md2.css?_={{ ts }}">
 	{%- endif %}
 {{ html_head }}
 </head>
+
 <body>
 	<div id="mn"></div>
 	<div id="mh">
@@ -22,7 +26,8 @@
 			<a id="nsbs" href="#" tt="switch between editor and preview$NHotkey: ctrl-e">editor</a>
 			<div id="toolsbox">
 				<a id="tools" href="#">tools</a>
-				<a id="fmt_table" href="#">prettify table (ctrl-k)</a>
+				<a id="fmt_table" href="#">beautify table (ctrl-k)</a>
+				<a id="fmt_json" href="#">beautify json (ctrl-j)</a>
 				<a id="iter_uni" href="#">non-ascii: iterate (ctrl-u)</a>
 				<a id="mark_uni" href="#">non-ascii: markup</a>
 				<a id="cfg_uni" href="#">non-ascii: whitelist</a>
@@ -126,19 +131,14 @@ write markdown (most html is 🙆 too)
 	</div>
 	{%- endif %}

-	<script>
+	<script nonce="{{ js_nonce }}">

 var SR = "{{ r }}",
+	JS_NONCE = "{{ js_nonce }}",
+	CGV1 = {{ cgv1 }},
 	last_modified = {{ lastmod }},
-	have_emp = {{ have_emp }},
-	md_no_br = {{ md_no_br }},
 	dfavico = "{{ favico }}";

-var md_opt = {
-	link_md_as_html: false,
-	modpoll_freq: {{ md_chk_rate }}
-};
-
 (function () {
 	var l = window.localStorage,
 		drk = (l && l.light) != 1,
@@ -154,14 +154,23 @@ try { l.light = drk? 0:1; } catch (ex) { }
 })();

 	</script>
-	<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
-	<script src="{{ r }}/.cpr/deps/marked.js?_={{ ts }}"></script>
-	<script src="{{ r }}/.cpr/md.js?_={{ ts }}"></script>
+	<script nonce="{{ js_nonce }}" src="{{ r }}/.cpr/w/util.js?_={{ ts }}"></script>
+	<script nonce="{{ js_nonce }}" src="{{ r }}/.cpr/w/deps/marked.js?_={{ ts }}"></script>
+	<script nonce="{{ js_nonce }}" src="{{ r }}/.cpr/w/md.js?_={{ ts }}"></script>
 	{%- if edit %}
-	<script src="{{ r }}/.cpr/md2.js?_={{ ts }}"></script>
+	<script nonce="{{ js_nonce }}" src="{{ r }}/.cpr/w/md2.js?_={{ ts }}"></script>
 	{%- endif %}
+	<script nonce="{{ js_nonce }}">
+		Date.now();function jsldp(a,b){2!=window[a]&&alert("FATAL ERROR: cannot load "+b+".js due to unreliable network or broken reverse-proxy; try CTRL-SHIFT-R")}
+		jsldp("J_UTL","util");
+		jsldp("J_MD","md");
+		{%- if edit %}
+		jsldp("J_MD2","md2");
+		{%- endif %}
+	</script>
 	{%- if js %}
-	<script src="{{ js }}_={{ ts }}"></script>
+	<script nonce="{{ js_nonce }}" src="{{ js }}_={{ ts }}"></script>
 	{%- endif %}
-</body></html>
+</body>
+</html>

@@ -1,5 +1,6 @@
 "use strict";

+var J_MD = 1;
 var dom_toc = ebi('toc'),
    dom_wrap = ebi('mw'),
    dom_hbar = ebi('mh'),
@@ -239,6 +240,12 @@ function convert_markdown(md_text, dest_dom) {
        var href = nodes[a].getAttribute('href');
        var txt = nodes[a].innerHTML;

+        if (/\.[Mm][Dd]$/.test(href)) {
+            var o = new URL(href, location.href).origin;
+            if (!o || o == location.origin)
+                nodes[a].href = href + '?v';
+        }
+
        if (!txt)
            nodes[a].textContent = href;
        else if (href !== txt && !nodes[a].className)
@@ -518,3 +525,5 @@ if (sread('hidenav') == 1)

 if (window.tt && tt.init)
    tt.init();
+
+J_MD = 2;
				`@@ -0,0 +1 @@`
				`sed -r 's/^\t+//;s/^"([a-zA-Z][a-zA-Z0-9_])": /\1:/; /^\/\//d; s/([^\]["'\''],) \/\/.*/\1/; /^$/d'`