v1.20.23

archlinux: add post_upgrade
update pkgs to 1.20.22
2026-11-14 09:48:22 +00:00 · 2026-09-06 22:49:08 +00:00 · 2026-09-06 22:08:42 +00:00 · 2026-09-06 21:18:21 +00:00 · 2026-09-06 21:12:32 +00:00 · 2026-09-06 21:07:39 +00:00
248 changed files with 30590 additions and 16421 deletions
@@ -4,3 +4,5 @@

 *.png binary
 *.gif binary
+*.ico binary
+*.gz binary
@@ -7,10 +7,11 @@ assignees: '9001'

 ---

-NOTE:
+<!-- NOTE:
 **please use english, or include an english translation.** aside from that,  
-all of the below are optional, consider them as inspiration, delete and rewrite at will, thx md
-
+include all of the information that might possibly be necessary to help diagnose your issue,
+especially things like the server config and server logs; that said,
+any parts below which are obviously and definitely irrelevant can be skipped -->

 ### Describe the bug
 a description of what the bug is
@@ -45,5 +46,11 @@ if the issue is possibly on the client-side, then mention some of the following:
 * OS version: 
 * browser version: 

+### The rest of the stack
+if you are connecting directly to copyparty then that's cool, otherwise please mention everything else between copyparty and the browser (reverseproxy, tunnels, etc.)
+
+### Server log
+if the issue might be server-related, include everything that appears in the copyparty log during startup, and also anything else you think might be relevant
+
 ### Additional context
 any other context about the problem here
@@ -7,9 +7,9 @@ assignees: '9001'

 ---

-NOTE:
+<!-- NOTE:
 **please use english, or include an english translation.** aside from that,  
-all of the below are optional, consider them as inspiration, delete and rewrite at will
+all of the below are optional, consider them as inspiration, delete and rewrite at will -->

 **is your feature request related to a problem? Please describe.**
 a description of what the problem is, for example, `I'm always frustrated when [...]` or `Why is it not possible to [...]`
@@ -29,8 +29,13 @@ copyparty.egg-info/
 copyparty/res/COPYING.txt
 copyparty/web/deps/
 srv/
+scripts/docker/base/b/
+scripts/docker/base/cver*
+scripts/docker/base/test-aac/
+scripts/docker/base/whl/
 scripts/docker/i/
 scripts/deps-docker/uncomment.py
+scripts/deps-docker/unhint.py
 contrib/package/arch/pkg/
 contrib/package/arch/src/

@@ -43,6 +48,7 @@ scripts/docker/*.err

 # nix build output link
 result
+result-*

 # IDEA config
 .idea/
@@ -39,7 +39,7 @@ if you wanna have a go at coding it up yourself then maybe mention the idea on d

 aside from documentation and ideas, some other things that would be cool to have some help with is:

-* **translations** -- the copyparty web-UI has translations for english and norwegian at the top of [browser.js](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/web/browser.js); if you'd like to add a translation for another language then that'd be welcome! and if that language has a grammar that doesn't fit into the way the strings are assembled, then we'll fix that as we go :>
+* **translations** -- the copyparty web-UI has translations in [copyparty/web/tl](https://github.com/9001/copyparty/tree/hovudstraum/copyparty/web/tl); if you'd like to [add a translation](https://github.com/9001/copyparty/tree/hovudstraum/docs/rice#translations) for another language then that'd be welcome! and if that language has a grammar that doesn't fit into the way the strings are assembled, then we'll fix that as we go :>

  * but please note that support for [RTL (Right-to-Left) languages](https://en.wikipedia.org/wiki/Right-to-left_script) is currently not planned, since the javascript is a bit too jank for that

@@ -5,7 +5,7 @@
 turn almost any device into a file server with resumable uploads/downloads using [*any*](#browser-support) web browser

 * server only needs Python (2 or 3), all dependencies optional
-* 🔌 protocols: [http](#the-browser) // [webdav](#webdav-server) // [ftp](#ftp-server) // [tftp](#tftp-server) // [smb/cifs](#smb-server)
+* 🔌 protocols: [http(s)](#the-browser) // [webdav](#webdav-server) // [sftp](#sftp-server) // [ftp(s)](#ftp-server) // [tftp](#tftp-server) // [smb/cifs](#smb-server)
 * 📱 [android app](#android-app) // [iPhone shortcuts](#ios-shortcuts)

 👉 **[Get started](#quickstart)!** or visit the **[read-only demo server](https://a.ocv.me/pub/demo/)** 👀 running on a nuc in my basement
@@ -14,13 +14,14 @@ turn almost any device into a file server with resumable uploads/downloads using

 🎬 **videos:** [upload](https://a.ocv.me/pub/demo/pics-vids/up2k.webm) // [cli-upload](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm) // [race-the-beam](https://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm) // 👉 **[feature-showcase](https://a.ocv.me/pub/demo/showcase-hq.webm)** ([youtube](https://www.youtube.com/watch?v=15_-hgsX2V0))

-made in Norway 🇳🇴
+built in Norway 🇳🇴 with contributions from [not-norway](https://github.com/9001/copyparty/graphs/contributors)


 ## readme toc

 * top
    * [quickstart](#quickstart) - just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** -- that's it! 🎉
+        * [mirrors](#mirrors) - other places to download copyparty from
        * [at home](#at-home) - make it accessible over the internet
        * [on servers](#on-servers) - you may also want these, especially on servers
    * [features](#features) - also see [comparison to similar software](./docs/versus.md)
@@ -63,15 +64,19 @@ made in Norway 🇳🇴
    * [other tricks](#other-tricks)
    * [searching](#searching) - search by size, date, path/name, mp3-tags, ...
 * [server config](#server-config) - using arguments or config files, or a mix of both
+    * [version-checker](#version-checker) - sleep better at night
+    * [logging](#logging) - serverlog is sent to stdout by default
    * [zeroconf](#zeroconf) - announce enabled services on the LAN ([pic](https://user-images.githubusercontent.com/241032/215344737-0eae8d98-9496-4256-9aa8-cd2f6971810d.png))
        * [mdns](#mdns) - LAN domain-name and feature announcer
        * [ssdp](#ssdp) - windows-explorer announcer
    * [qr-code](#qr-code) - print a qr-code [(screenshot)](https://user-images.githubusercontent.com/241032/194728533-6f00849b-c6ac-43c6-9359-83e454d11e00.png) for quick access
    * [ftp server](#ftp-server) - an FTP server can be started using `--ftp 3921`
+    * [sftp server](#sftp-server) - goes roughly 700 MiB/s (slower than webdav and ftp)
    * [webdav server](#webdav-server) - with read-write support
        * [connecting to webdav from windows](#connecting-to-webdav-from-windows) - using the GUI
    * [tftp server](#tftp-server) - a TFTP server (read/write) can be started using `--tftp 3969`
    * [smb server](#smb-server) - unsafe, slow, not recommended for wan
+    * [wopi server](#wopi-server) - edit office documents in the web-ui
    * [browser ux](#browser-ux) - tweaking the ui
    * [opengraph](#opengraph) - discord and social-media embeds
    * [file deduplication](#file-deduplication) - enable symlink-based upload deduplication
@@ -79,12 +84,17 @@ made in Norway 🇳🇴
        * [exclude-patterns](#exclude-patterns) - to save some time
        * [filesystem guards](#filesystem-guards) - avoid traversing into other filesystems
        * [periodic rescan](#periodic-rescan) - filesystem monitoring
+    * [u2sz](#u2sz) - configure upload-chunking
    * [upload rules](#upload-rules) - set upload rules using volflags
    * [compress uploads](#compress-uploads) - files can be autocompressed on upload
    * [chmod and chown](#chmod-and-chown) - per-volume filesystem-permissions and ownership
    * [other flags](#other-flags)
+    * [descript.ion](#description) - add a description to each file in a folder
+    * [dothidden](#dothidden) - cosmetically hide specific files in a folder
+    * [thumbnail pregen](#thumbnail-pregen) - if you want to pre-generate everything on startup
    * [database location](#database-location) - in-volume (`.hist/up2k.db`, default) or somewhere else
    * [metadata from audio files](#metadata-from-audio-files) - set `-e2t` to index tags on upload
+        * [metadata from xattrs](#metadata-from-xattrs) - unix extended file attributes
    * [file parser plugins](#file-parser-plugins) - provide custom parsers to index additional tags
    * [event hooks](#event-hooks) - trigger a program on uploads, renames etc ([examples](./bin/hooks/))
        * [zeromq](#zeromq) - event-hooks can send zeromq messages
@@ -113,10 +123,12 @@ made in Norway 🇳🇴
 * [packages](#packages) - the party might be closer than you think
    * [arch package](#arch-package) - `pacman -S copyparty` (in [arch linux extra](https://archlinux.org/packages/extra/any/copyparty/))
    * [fedora package](#fedora-package) - does not exist yet
+    * [gentoo ::guru package](#gentoo-guru-package) - `emerge www-servers/copyparty::guru` (in [::guru](https://wiki.gentoo.org/wiki/Project:GURU))
    * [homebrew formulae](#homebrew-formulae) - `brew install copyparty ffmpeg`
    * [nix package](#nix-package) - `nix profile install github:9001/copyparty`
    * [nixos module](#nixos-module)
 * [browser support](#browser-support) - TLDR: yes
+* [server hall of fame](#server-hall-of-fame) - unexpected things that run copyparty
 * [client examples](#client-examples) - interact with copyparty using non-browser clients
    * [folder sync](#folder-sync) - sync folders to/from copyparty
    * [mount as drive](#mount-as-drive) - a remote copyparty server as a local filesystem
@@ -124,8 +136,9 @@ made in Norway 🇳🇴
 * [iOS shortcuts](#iOS-shortcuts) - there is no iPhone app, but
 * [performance](#performance) - defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
    * [client-side](#client-side) - when uploading files
-* [security](#security) - there is a [discord server](https://discord.gg/25J8CdTT6G)
+* [security](#security) - there is a [discord server](https://discord.gg/25J8CdTT6G) with announcements
    * [gotchas](#gotchas) - behavior that might be unexpected
+    * [csp nonce](#csp-nonce) - unauthorized javascript is blocked
    * [cors](#cors) - cross-site request config
    * [filekeys](#filekeys) - prevent filename bruteforcing
        * [dirkeys](#dirkeys) - share specific folders in a volume
@@ -136,7 +149,7 @@ made in Norway 🇳🇴
        * [firefox wsod](#firefox-wsod) - firefox 87 can crash during uploads
 * [HTTP API](#HTTP-API) - see [devnotes](./docs/devnotes.md#http-api)
 * [dependencies](#dependencies) - mandatory deps
-    * [optional dependencies](#optional-dependencies) - install these to enable bonus features
+    * [optional dependencies](#optional-dependencies) - enable bonus features
        * [dependency chickenbits](#dependency-chickenbits) - prevent loading an optional dependency
        * [dependency unvendoring](#dependency-unvendoring) - force use of system modules
    * [optional gpl stuff](#optional-gpl-stuff)
@@ -157,7 +170,7 @@ just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/

 * or install through [pypi](https://pypi.org/project/copyparty/): `python3 -m pip install --user -U copyparty`
 * or if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
-* or install [on arch](#arch-package) ╱ [on NixOS](#nixos-module) ╱ [through nix](#nix-package)
+* or install [on arch](#arch-package) / [homebrew](#homebrew-formulae) ╱ [on NixOS](#nixos-module) ╱ [through nix](#nix-package)
 * or if you are on android, [install copyparty in termux](#install-on-android)
 * or maybe an iPhone or iPad? [install in a-Shell on iOS](#install-on-iOS)
 * or maybe you have a [synology nas / dsm](./docs/synology-dsm.md)
@@ -175,7 +188,7 @@ enable thumbnails (images/audio/video), media indexing, and audio transcoding by
 * **Alpine:** `apk add py3-pillow ffmpeg`
 * **Debian:** `apt install --no-install-recommends python3-pil ffmpeg`
 * **Fedora:** rpmfusion + `dnf install python3-pillow ffmpeg --allowerasing`
-* **FreeBSD:** `pkg install py39-sqlite3 py39-pillow ffmpeg`
+* **FreeBSD:** `pkg install py311-sqlite3 py311-pillow ffmpeg`
 * **MacOS:** `port install py-Pillow ffmpeg`
 * **MacOS** (alternative): `brew install pillow ffmpeg`
 * **Windows:** `python -m pip install --user -U Pillow`
@@ -192,7 +205,19 @@ some recommended options:
 * `-e2ts` enables audio metadata indexing (needs either FFprobe or Mutagen)
 * `-v /mnt/music:/music:r:rw,foo -a foo:bar` shares `/mnt/music` as `/music`, `r`eadable by anyone, and read-write for user `foo`, password `bar`
  * replace `:r:rw,foo` with `:r,foo` to only make the folder readable by `foo` and nobody else
-  * see [accounts and volumes](#accounts-and-volumes) (or `--help-accounts`) for the syntax and other permissions
+  * see [accounts and volumes](#accounts-and-volumes) (or [`--help-accounts`](https://copyparty.eu/cli/#accounts-help-page)) for the syntax and other permissions
+
+
+### mirrors
+
+other places to download copyparty from  (non-github links):
+
+* https://copyparty.eu/ (hetzner, finland, official mirror):
+  * https://copyparty.eu/py = https://copyparty.eu/copyparty-sfx.py = the sfx
+  * https://copyparty.eu/en = https://copyparty.eu/copyparty-en.py = the english-only sfx
+  * https://copyparty.eu/pyz = https://copyparty.eu/copyparty.pyz = the zipapp
+  * https://copyparty.eu/enz = https://copyparty.eu/copyparty-en.pyz = the enterprise pyz
+  * https://copyparty.eu/cli = online cli helptext


 ### at home
@@ -220,14 +245,19 @@ you may also want these, especially on servers:
 * [nixos module](#nixos-module) to run copyparty on NixOS hosts
 * [contrib/nginx/copyparty.conf](contrib/nginx/copyparty.conf) to [reverse-proxy](#reverse-proxy) behind nginx (for better https)

+because the following environment variables are commonly used in service-scripts, they are understood by copyparty:
+
+* `NOTIFY_SOCKET` as provided by systemd with service type=notify (see systemd/copyparty.service above)
+* `S6_NOTIFY_FD` for dinit [`ready-notification = pipevar:S6_NOTIFY_FD`](https://davmac.org/projects/dinit/man-pages-html/dinit-service.5.html#ready) and [s6 supervision suite notification](https://skarnet.org/software/s6/notifywhenup.html) support
+
 and remember to open the ports you want; here's a complete example including every feature copyparty has to offer:
 ```
-firewall-cmd --permanent --add-port={80,443,3921,3923,3945,3990}/tcp  # --zone=libvirt
+firewall-cmd --permanent --add-port={80,443,3921,3922,3923,3945,3990}/tcp  # --zone=libvirt
 firewall-cmd --permanent --add-port=12000-12099/tcp  # --zone=libvirt
 firewall-cmd --permanent --add-port={69,1900,3969,5353}/udp  # --zone=libvirt
 firewall-cmd --reload
 ```
-(69:tftp, 1900:ssdp, 3921:ftp, 3923:http/https, 3945:smb, 3969:tftp, 3990:ftps, 5353:mdns, 12000:passive-ftp)
+(69:tftp, 1900:ssdp, 3921:ftp, 3922:sftp, 3923:http/https, 3945:smb, 3969:tftp, 3990:ftps, 5353:mdns, 12000:passive-ftp)


 ## features
@@ -247,7 +277,7 @@ also see [comparison to similar software](./docs/versus.md)
  * ☑ [upnp / zeroconf / mdns / ssdp](#zeroconf)
  * ☑ [event hooks](#event-hooks) / script runner
  * ☑ [reverse-proxy support](https://github.com/9001/copyparty#reverse-proxy)
-  * ☑ cross-platform (Windows, Linux, Macos, Android, iOS, FreeBSD, arm32/arm64, ppc64le, s390x, risc-v/riscv64)
+  * ☑ cross-platform (Windows, Linux, Macos, Android, iOS, FreeBSD, arm32/arm64, ppc64le, s390x, risc-v/riscv64, SGI IRIX)
 * upload
  * ☑ basic: plain multipart, ie6 support
  * ☑ [up2k](#uploading): js, resumable, multithreaded
@@ -270,11 +300,12 @@ also see [comparison to similar software](./docs/versus.md)
    * ☑ play video files as audio (converted on server)
    * ☑ create and play [m3u8 playlists](#playlists)
  * ☑ image gallery with webm player
+    * ☑ and cbz manga/comics reader
  * ☑ [textfile browser](#textfile-viewer) with syntax highlighting
    * ☑ realtime streaming of growing files (logfiles and such)
  * ☑ [thumbnails](#thumbnails)
    * ☑ ...of images using Pillow, pyvips, or FFmpeg
-    * ☑ ...of RAW images using rawpy
+    * ☑ ...of RAW images using libraw-dcraw_emu or rawpy
    * ☑ ...of videos using FFmpeg
    * ☑ ...of audio (spectrograms) using FFmpeg
    * ☑ cache eviction (max-age; maybe max-size eventually)
@@ -309,7 +340,7 @@ small collection of user feedback

 project goals / philosophy

-* inverse linux philosophy -- do all the things, and do an *okay* job
+* inverse unix philosophy -- do all the things, and do an *okay* job
  * quick drop-in service to get a lot of features in a pinch
  * some of [the alternatives](./docs/versus.md) might be a better fit for you
 * run anywhere, support everything
@@ -383,6 +414,8 @@ same order here too

 * [Firefox issue 1790500](https://bugzilla.mozilla.org/show_bug.cgi?id=1790500) -- entire browser can crash after uploading ~4000 small files

+* Windows: Uploading from a webbrowser may fail with "directory iterator got stuck" due to the [max path length](https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry); try moving the files somewhere shorter before uploading
+
 * Android: music playback randomly stops due to [battery usage settings](#fix-unreliable-playback-on-android)

 * iPhones: the volume control doesn't work because [apple doesn't want it to](https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Device-SpecificConsiderations/Device-SpecificConsiderations.html#//apple_ref/doc/uid/TP40009523-CH5-SW11)
@@ -437,6 +470,12 @@ upgrade notes
 * CopyParty?
  * nope! the name is either copyparty (all-lowercase) or Copyparty -- it's [one word](https://en.wiktionary.org/wiki/copyparty) after all :>

+* what is a volflag?
+  * per-volume configuration; many (not all) global-options can be set as volflags, and most (not all) volflags can be set as global-options; [complete list of volflags](https://copyparty.eu/cli/#flags-help-page)
+
+* what is a volume?
+  * a mapping from a URL (`/music/`) to a folder on your server's local filesystem (`C:\Users\ed\Music`) which can then be accessed through copyparty, depending on the permissions and options you set on it -- see [accounts and volumes](#accounts-and-volumes)
+
 * can I change the 🌲 spinning pine-tree loading animation?
  * [yeah...](https://github.com/9001/copyparty/tree/hovudstraum/docs/rice#boring-loader-spinner) :-(

@@ -447,6 +486,7 @@ upgrade notes
 * can I link someone to a password-protected volume/file by including the password in the URL?
  * yes, by adding `?pw=hunter2` to the end; replace `?` with `&` if there are parameters in the URL already, meaning it contains a `?` near the end
    * if you have enabled `--usernames` then do `?pw=username:password` instead
+    * `?pw` can be disabled with `--pw-urlp=A` but this breaks support for many clients

 * how do I stop `.hist` folders from appearing everywhere on my HDD?
  * by default, a `.hist` folder is created inside each volume for the filesystem index, thumbnails, audio transcodes, and markdown document history. Use the `--hist` global-option or the `hist` volflag to move it somewhere else; see [database location](#database-location)
@@ -466,6 +506,9 @@ upgrade notes
 * thumbnails are broken (you get a colorful square which says the filetype instead)
  * you need to install `FFmpeg` or `Pillow`; see [thumbnails](#thumbnails)

+* thumbnails are broken, specifically for photos and videos taken by iphones
+  * the [docker image](https://github.com/9001/copyparty/blob/hovudstraum/scripts/docker) and [bootable flashdrive](https://a.ocv.me/pub/stuff/edcd001/enterprise-edition/) are not able to read heif/heic images and h265/HEVC video due to [legal reasons](docs/bad-codecs.md)
+
 * thumbnails are broken (some images appear, but other files just get a blank box, and/or the broken-image placeholder)
  * probably due to a reverse-proxy messing with the request URLs and stripping the query parameters (`?th=w`), so check your URL rewrite rules
  * could also be due to incorrect caching settings in reverse-proxies and/or CDNs, so make sure that nothing is set to ignore the query string
@@ -483,8 +526,9 @@ upgrade notes
 per-folder, per-user permissions  - if your setup is getting complex, consider making a [config file](./docs/example.conf) instead of using arguments
 * much easier to manage, and you can modify the config at runtime with `systemctl reload copyparty` or more conveniently using the `[reload cfg]` button in the control-panel (if the user has `a`/admin in any volume)
  * changes to the `[global]` config section requires a restart to take effect
+* when using config-files, ignore all the `-a` and `-v` stuff below, and CTRL-F `config file example` on this page instead

-a quick summary can be seen using `--help-accounts`
+a quick summary can be seen using [`--help-accounts`](https://copyparty.eu/cli/#accounts-help-page)

 configuring accounts/volumes with arguments:
 * `-a usr:pwd` adds account `usr` with password `pwd`
@@ -523,12 +567,17 @@ examples:

 if you want to grant access to all users who are logged in, the group `acct` will always contain all known users, so for example `-v /mnt/music:music:r,@acct`

+* to do the opposite, granting access to everyone who is NOT logged in. `*,-@acct` does the trick, for example `-v /srv/welcome:welcome:r,*,-@acct`
+* single users can also be subtracted from a group: `@admins,-james`
+
 anyone trying to bruteforce a password gets banned according to `--ban-pw`; default is 24h ban for 9 failed attempts in 1 hour

 and if you want to use config files instead of commandline args (good!) then here's the same examples as a configfile; save it as `foobar.conf` and use it like this: `python copyparty-sfx.py -c foobar.conf`

 * you can also `PRTY_CONFIG=foobar.conf python copyparty-sfx.py` (convenient in docker etc)

+config file example:
+
 ```yaml
 [accounts]
  u1: p1  # create account "u1" with password "p1"
@@ -571,12 +620,28 @@ and if you want to use config files instead of commandline args (good!) then her

 hiding specific subfolders  by mounting another volume on top of them

-for example `-v /mnt::r -v /var/empty:web/certs:r` mounts the server folder `/mnt` as the webroot, but another volume is mounted at `/web/certs` -- so visitors can only see the contents of `/mnt` and `/mnt/web` (at URLs `/` and `/web`), but not `/mnt/web/certs` because URL `/web/certs` is mapped to `/var/empty`
+for example `-v /mnt::r -v /var/empty:web/certs:` (note: no permissions) mounts the server folder `/mnt` as the webroot, but another volume is mounted at `/web/certs` -- so visitors can only see the contents of `/mnt` and `/mnt/web` (at URLs `/` and `/web`), but not `/mnt/web/certs` because URL `/web/certs` is mapped to `/var/empty`
+
+to fully unmap it from the filesystem, specify `//NULL` instead of a real path such as `/var/empty`, so for example `-v /mnt::r -v //NULL:web/certs:` ensures `/web/certs` will never be accessible by anyone

 the example config file right above this section may explain this better; the first volume `/` is mapped to `/srv` which means http://127.0.0.1:3923/music would try to read `/srv/music` on the server filesystem, but since there's another volume at `/music` mapped to `/mnt/music` then it'll go to `/mnt/music` instead

+so, to shadow a file/folder, define a volume but leave out the `accs:` section
+
 > ℹ️ this also works for single files, because files can also be volumes

+config file example for unmapping folders by shadowing:
+
+```yaml
+[/drives]
+  /mnt   # url "/drives" goes to "/mnt"
+  accs:
+    r: *  # everyone can read
+
+[/drives/foo/bar]
+  //NULL  # blocks access to "/mnt/foo/bar"
+```
+

 ## dotfiles

@@ -586,6 +651,8 @@ anyone can access these if they know the name, but they normally don't appear in

 a client can request to see dotfiles in directory listings if global option `-ed` is specified, or the volume has volflag `dots`, or the user has permission `.`

+> for [shares](#shares), the `dots` volflag is ignored
+
 dotfiles do not appear in search results unless one of the above is true, **and** the global option / volflag `dotsrch` is set

 > even if user has permission to see dotfiles, they are default-hidden unless `--see-dots` is set, and/or user has enabled the `dotfiles` option in the settings tab
@@ -618,7 +685,7 @@ the main tabs in the ui
 * `[🧯]` [unpost](#unpost): undo/delete accidental uploads
 * `[🚀]` and `[🎈]` are the [uploaders](#uploading)
 * `[📂]` mkdir: create directories
-* `[📝]` new-md: create a new markdown document
+* `[📝]` new-file: create a new textfile
 * `[📟]` send-msg: either to server-log or into textfiles if `--urlform save`
 * `[🎺]` audio-player config options
 * `[⚙️]` general client config options
@@ -712,7 +779,7 @@ press `g` or `田` to toggle grid-view instead of the file listing  and `t` togg
 ![copyparty-thumbs-fs8](https://user-images.githubusercontent.com/241032/129636211-abd20fa2-a953-4366-9423-1c88ebb96ba9.png)

 it does static images with Pillow / pyvips / FFmpeg, and uses FFmpeg for video files, so you may want to `--no-thumb` or maybe just `--no-vthumb` depending on how dangerous your users are
-* pyvips is 3x faster than Pillow, Pillow is 3x faster than FFmpeg
+* Pillow is 3x faster (and safer) than FFmpeg
 * disable thumbnails for specific volumes with volflag `dthumb` for all, or `dvthumb` / `dathumb` / `dithumb` for video/audio/images only
 * for installing FFmpeg on windows, see [optional dependencies](#optional-dependencies)

@@ -721,6 +788,7 @@ audio files are converted into spectrograms using FFmpeg unless you `--no-athumb
 images with the following names (see `--th-covers`) become the thumbnail of the folder they're in: `folder.png`, `folder.jpg`, `cover.png`, `cover.jpg`
 * the order is significant, so if both `cover.png` and `folder.jpg` exist in a folder, it will pick the first matching `--th-covers` entry (`folder.jpg`)
 * and, if you enable [file indexing](#file-indexing), it will also try those names as dotfiles (`.folder.jpg` and so), and then fallback on the first picture in the folder (if it has any pictures at all)
+* disable folderthumbs with `--th-covers no`

 enabling `multiselect` lets you click files to select them, and then shift-click another file for range-select
 * `multiselect` is mostly intended for phones/tablets, but the `sel` option in the `[⚙️] settings` tab is better suited for desktop use, allowing selection by CTRL-clicking and range-selection with SHIFT-click, all without affecting regular clicking
@@ -732,8 +800,8 @@ to show `/icons/exe.png` and `/icons/elf.gif` as the thumbnail for all `.exe` an
  * be careful with svg; chrome will crash if you have too many unique svg files showing on the same page (the limit is 250 or so) -- showing the same handful of svg files thousands of times is ok however

 note:
-* heif/heifs/heic/heics images usually require the `libvips` [optional dependency](#optional-dependencies) (available in the `iv` docker image, `withFastThumbnails` in nixos)
-  * technical trivia: FFmpeg has basic support for tiled heic as of v7.0; need `-show_stream_groups` for correct resolution
+* heif/heifs/heic/heics images usually require the `libvips` [optional dependency](#optional-dependencies) but this is not possible with the docker-images due to [legal reasons](docs/bad-codecs.md)
+* if you do not want thumbnails to be generated on-the-fly, and instead wish to generate all of them on server startup, then see [thumbnail pregen](#thumbnail-pregen)

 config file example:

@@ -789,8 +857,11 @@ you can also zip a selection of files or folders by clicking them in the browser

 cool trick: download a folder by appending url-params `?tar&opus` or `?tar&mp3` to transcode all audio files (except aac|m4a|mp3|ogg|opus|wma) to opus/mp3 before they're added to the archive
 * super useful if you're 5 minutes away from takeoff and realize you don't have any music on your phone but your server only has flac files and downloading those will burn through all your data + there wouldn't be enough time anyways
+* and url-param `&name=foo` changes the name of the toplevel folder in the archive to `foo`, and just `&name` removes the folder entirely
+* and url-param `&nodot` skips dotfiles/dotfolders; they are included by default if your account has permission to see them
 * and url-params `&j` / `&w` produce jpeg/webm thumbnails/spectrograms instead of the original audio/video/images (`&p` for audio waveforms)
  * can also be used to pregenerate thumbnails; combine with `--th-maxage=9999999` or `--th-clean=0`
+    * but now there is also a real [thumbnail pregen](#thumbnail-pregen) so just use that


 ## uploading
@@ -855,7 +926,7 @@ note that since up2k has to read each file twice, `[🎈] bup` can *theoreticall

 if you are resuming a massive upload and want to skip hashing the files which already finished, you can enable `turbo` in the `[⚙️] config` tab, but please read the tooltip on that button

-if the server is behind a proxy which imposes a request-size limit, you can configure up2k to sneak below the limit with server-option `--u2sz` (the default is 96 MiB to support Cloudflare)
+if the server is behind a proxy which imposes a request-size limit, you can configure up2k to sneak below the limit with server-option [--u2sz](#u2sz)

 if you want to replace existing files on the server with new uploads by default, run with `--u2ow 2` (only works if users have the delete-permission, and can still be disabled with `🛡️` in the UI)

@@ -972,6 +1043,8 @@ specify `--shr /foobar` to enable this feature; a toplevel virtual folder named

 users can delete their own shares in the controlpanel, and a list of privileged users (`--shr-adm`) are allowed to see and/or delet any share on the server

+the volflag `--shr-who` lets you control who can create a share from that volume, either `no` (nobody), `a` (people with admin permission), or `auth` (people who are logged in) 
+
 after a share has expired, it remains visible in the controlpanel for `--shr-rt` minutes (default is 1 day), and the owner can revive it by extending the expiration time there

 **security note:** using this feature does not mean that you can skip the [accounts and volumes](#accounts-and-volumes) section -- you still need to restrict access to volumes that you do not intend to share with unauthenticated users! it is not sufficient to use rules in the reverseproxy to restrict access to just the `/share` folder.
@@ -1135,7 +1208,7 @@ open the `[🎺]` media-player-settings tab to configure it,
  * `[flac]` converts `flac` and `wav` files into opus (if supported by browser) or mp3
  * `[aac]` converts `aac` and `m4a` files into opus (if supported by browser) or mp3
  * `[oth]` converts all other known formats into opus (if supported by browser) or mp3
-    * `aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk`
+    * `aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|m4b|m4r|mka|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk`
 * "transcode to":
  * `[opus]` produces an `opus` whenever transcoding is necessary (the best choice on Android and PCs)
  * `[awo]` is `opus` in a `weba` file, good for iPhones (iOS 17.5 and newer) but Apple is still fixing some state-confusion bugs as of iOS 18.2.1
@@ -1246,7 +1319,7 @@ see [./srv/expand/](./srv/expand/) for usage and examples

  * and `PREADME.md` / `preadme.md` is shown above directory listings unless `--no-readme` or `.prologue.html`

-* `README.md` and `*logue.html` can contain placeholder values which are replaced server-side before embedding into directory listings; see `--help-exp`
+* `README.md` and `*logue.html` can contain placeholder values which are replaced server-side before embedding into directory listings; see [`--help-exp`](https://copyparty.eu/cli/#exp-help-page)


 ## searching
@@ -1271,15 +1344,75 @@ for the above example to work, add the commandline argument `-e2ts` to also scan
 # server config

 using arguments or config files, or a mix of both:
-* config files (`-c some.conf`) can set additional commandline arguments; see [./docs/example.conf](docs/example.conf) and [./docs/example2.conf](docs/example2.conf)
+* using a config-file (`-c some.conf`) is best, easier to read/maintain; see [./docs/example.conf](docs/example.conf)
+  * for complex and intricate setups, see how [./docs/example2.conf](docs/example2.conf) includes additional config-files/folders with `% copyparty.d` which loads [./docs/copyparty.d/some.conf](docs/copyparty.d/some.conf)
 * `kill -s USR1` (same as `systemctl reload copyparty`) to reload accounts and volumes from config files without restarting
  * or click the `[reload cfg]` button in the control-panel if the user has `a`/admin in any volume
  * changes to the `[global]` config section requires a restart to take effect

-**NB:** as humongous as this readme is, there is also a lot of undocumented features. Run copyparty with `--help` to see all available global options; all of those can be used in the `[global]` section of config files, and everything listed in `--help-flags` can be used in volumes as volflags.
+**NB:** as humongous as this readme is, there is also a lot of undocumented features. Run copyparty with [`--help`](https://copyparty.eu/cli/) (or click that link) to see all available global options; all of those can be used in the `[global]` section of config files, and everything listed in [`--help-flags`](https://copyparty.eu/cli/#flags-help-page) can be used in volumes as volflags (per-volume configuration).
 * if running in docker/podman, try this: `docker run --rm -it copyparty/ac --help`
-* or see this: https://ocv.me/copyparty/helptext.html
-* or if you prefer plaintext, https://ocv.me/copyparty/helptext.txt
+* or if you prefer plaintext, https://copyparty.eu/helptext.txt
+
+
+## version-checker
+
+sleep better at night  by telling copyparty to periodically check whether your version has a [known vulnerability](https://github.com/9001/copyparty/security/advisories)
+
+this feature can be enabled by setting the global-option `--vc-url` to one of the following URLs; choose what severity level you want to be notified for:
+* `https://api.copyparty.eu/advisories-panic` -- only really bad stuff, the "UPGRADE NOW" kind
+* `https://api.copyparty.eu/advisories` -- everything important / noteworthy, "upgrade when you can"
+* `https://api.copyparty.eu/advisories-all` -- *everything*, including stuff that's unlikely to affect anyone
+* `https://api.github.com/repos/9001/copyparty/security-advisories?per_page=9` -- same as `advisories-all`
+
+note that `https://api.copyparty.eu/advisories` may (for example) skip some advisories rated `High` but include some `Low`; that's because an easily-reachable `Low` in a default-enabled feature is more severe than a `High` which is a theoretical bug in a contrived use of a fringe feature, but the CVE calculator would still classify that as `High`
+
+if you want to use the github advisory feed but only care about advisories rated `medium`/`moderate` or higher, then global-option `--vc-sev medium` does that, but see previous paragraph
+
+> to see what happens when a bad version is detected, try `--vc-url https://api.copyparty.eu/advisories-test`
+
+also consider the following options:
+* global-option `--vc-age` is how often (in hours) to check that URL; default is 3
+* global-option `--vc-exit` can be enabled to panic and immediately exit if a vulnerability is indicated
+  * if `--vc-exit` is not enabled, it just shows a warning on the controlpanel for all users with permission `a` or `A`
+
+config file example:
+
+```yaml
+[global]
+  vc-url: https://api.copyparty.eu/advisories
+  vc-age: 3  # how many hours to wait between each check
+  vc-exit    # emergency-exit if current version is vulnerable
+  vc-sev: medium  # only care about severity 'Medium'/'Moderate' or higher (github-only; don't use this with api.copyparty.eu)
+```
+
+
+## logging
+
+serverlog is sent to stdout by default  (but logging to a file is also possible)
+
+"stdout" usually means either the terminal, or journalctl, or whatever is collecting logs from your docker containers, so that depends on your setup
+
+* [-q](https://copyparty.eu/cli/#g-q) disables logging to stdout, and may improve performance a little bit
+  * combine it with `-lo logfolder/cpp-%Y-%m-%d.txt` to log to a file instead
+  * the `%Y-%m-%d` makes it create a new logfile every day, with the date as filename
+  * global-option [--rlo](https://copyparty.eu/cli/#rlo-help-page) decides what happens if the filename is taken
+* `-lo whatever.txt` can be used without `-q` to log to both at the same time
+  * by default, the logfile will have colors if the terminal does (usually the case)
+  * use the [textfile-viewer](https://github.com/user-attachments/assets/8a828947-2fae-4df9-bd2a-3de46f42d478) or `less -R` in a terminal to see colors correctly
+* if you want [no colors](https://youtu.be/biW5UVGkPMA?t=148):
+  * `--flo 2` disables colors for just the logfile
+  * `--no-ansi` disables colors for both the terminal and logfile
+
+config file example:
+
+```yaml
+[global]
+  log-date: %Y-%m-%d  # show dates on stdout too
+  lo: /var/log/cpp/%Y-%m-%d.txt  # logfile path
+  flo: 2  # just text (no colors) in logfile
+  q       # disable stdout; use logfile only
+```


 ## zeroconf
@@ -1376,6 +1509,26 @@ config file example, which restricts FTP to only use ports 3921 and 12000-12099
 ```


+## sftp server
+
+goes roughly 700 MiB/s (slower than webdav and ftp)
+
+> this is **not** [ftps](#ftp-server) (which copyparty also supports); [ftps](#ftp-server) is ftp-tls (think http/https), while **sftp** is ssh-based and (preferably) uses ssh-keys for authentication
+
+the sftp-server requires the optional dependency [paramiko](https://pypi.org/project/paramiko/);
+* if you are **not** using docker, then install paramiko somehow
+* if you **are** using docker, then use one of the following image variants: `ac` / `im` / `iv` / `dj`
+
+enable sftpd with `--sftp 3922` to listen on port 3922;
+* use global-option `sftp-key` to associate an ssh-key with a user;
+  * commandline: `--sftp-key 'david ssh-ed25519 AAAAC3NzaC...'`
+  * config-file: `sftp-key: david ssh-ed25519 AAAAC3NzaC...`
+* `--sftp-pw` enables login with passwords (default is ssh-keys only)
+* `--sftp-anon foo` enables login with username `foo` and no password; gives the same access/permissions as the website does when not logged in
+
+see the [sftp section in --help](https://copyparty.eu/cli/#g-sftp) for the other options
+
+
 ## webdav server

 with read-write support,  supports winXP and later, macos, nautilus/gvfs  ... a great way to [access copyparty straight from the file explorer in your OS](#mount-as-drive)
@@ -1389,7 +1542,7 @@ general usage:
 on macos, connect from finder:
 * [Go] -> [Connect to Server...] -> http://192.168.123.1:3923/

-to upload or edit files with WebDAV clients, enable the `daw` volflag (because most WebDAV clients expect this) and give your account the delete-permission. This avoids getting several copies of the same file on the server. HOWEVER: This will also make all PUT-uploads overwrite existing files if the user has delete-access, so use with caution.
+to be able to edit existing files, the client must have the Delete-permission, and some webdav clients will also require the [daw](https://copyparty.eu/cli/#g-daw) volflag or global-option (not necessary if the client sends the `x-oc-mtime` header). Without `daw`, those clients will fail to modify existing files and instead create new copies with names like `notes.txt-1771978661.726032-3i9GPghL.txt`. **NOTE:** Enabling `daw` will also make all PUT-uploads overwrite existing files if the user has delete-access, so use with caution. Another alternative is the [dav-port](https://copyparty.eu/cli/#g-dav-port) option

 > note: if you have enabled [IdP authentication](#identity-providers) then that may cause issues for some/most webdav clients; see [the webdav section in the IdP docs](https://github.com/9001/copyparty/blob/hovudstraum/docs/idp.md#connecting-webdav-clients)

@@ -1450,7 +1603,7 @@ unsafe, slow, not recommended for wan,  enable with `--smb` for read-only or `--

 click the [connect](http://127.0.0.1:3923/?hc) button in the control-panel to see connection instructions for windows, linux, macos

-dependencies: `python3 -m pip install --user -U impacket==0.11.0`
+dependencies: `python3 -m pip install --user -U impacket==0.13.0`
 * newer versions of impacket will hopefully work just fine but there is monkeypatching so maybe not

 some **BIG WARNINGS** specific to SMB/CIFS, in decreasing importance:
@@ -1484,6 +1637,15 @@ authenticate with one of the following:
 * username `$password`, password `k`


+## wopi server
+
+edit office documents in the web-ui
+
+needs a wopi client to work, and the only supported wopi client is a selfhosted collabora online server; eventually onlyoffice will probably also be supported: [issue #1574](https://github.com/9001/copyparty/issues/1574)
+
+no further documentation or examples yet, [issue #1575](https://github.com/9001/copyparty/issues/1575)
+
+
 ## browser ux

 tweaking the ui
@@ -1493,7 +1655,12 @@ tweaking the ui
  * to sort in music order (album, track, artist, title) with filename as fallback, you could `--sort tags/Circle,tags/.tn,tags/Artist,tags/Title,href`
  * to sort by upload date, first enable showing the upload date in the listing with `-e2d -mte +.up_at` and then `--sort tags/.up_at`

-see [./docs/rice](./docs/rice) for more, including how to add stuff (css/`<meta>`/...) to the html `<head>` tag, or to add your own translation
+see [./docs/rice](./docs/rice) for more, including:
+* how to [hide ui-elements](./docs/rice/README.md#hide-ui-elements)
+* [custom fonts](./docs/rice/README.md#custom-fonts)
+* [custom loading-spinner](./docs/rice/README.md#boring-loader-spinner)
+* adding stuff (css/`<meta>`/...) [to the html `<head>` tag](./docs/rice/README.md#head)
+* [adding your own translation](./docs/rice/README.md#translations)


 ## opengraph
@@ -1634,7 +1801,7 @@ avoid traversing into other filesystems  using `--xdev` / volflag `:c,xdev`, ski

 and/or you can `--xvol` / `:c,xvol` to ignore all symlinks leaving the volume's top directory, but still allow bind-mounts pointing elsewhere

-* symlinks are permitted with `xvol` if they point into another volume where the user has the same level of access
+* symlinks are permitted with `xvol` if they point into another volume where the user also has some sort of access, keeping permissions from outer/initial volume

 these options will reduce performance; unlikely worst-case estimates are 14% reduction for directory listings, 35% for download-as-tar

@@ -1663,6 +1830,23 @@ config file example:
 ```


+## u2sz
+
+configure upload-chunking;  the default `64,96` is optimal for nearly every scenario, however:
+
+* if you are seeing the message `bad server config; see "u2sz" in readme` then you have a proxy/CDN/whatever with a `max-request-body-size` which is too small, and you must either increase the limit in the reverseproxy config to at least 96 MiB (recommended) or reduce the target chunksize in copyparty (continue reading)
+
+the global-option `--u2sz` is two numbers, `target,max` MiB of each chunk to send, aiming for `target` MiB and allowing up-to-and-including `max` MiB [depending on filesize](./docs/devnotes.md#list-of-chunk-sizes), and if you only specify one number then it sets that for both
+
+cloudflare max is 96 MiB, but if you have a reverseproxy (or CDN/WAF/whatever) which is more restrictive, then try some of the following:
+
+* `u2sz: 32` is still acceptable performance in most cases
+* `u2sz: 8` is the lowest recommended value; anything lower will reduce performance over long physical distances and/or shitty routes
+* `u2sz: 1` is the absolute minimum, and will be slow in many cases
+
+the setting can also be changed in the web-UI; the textbox next to `date-chk` in the settings tab
+
+
 ## upload rules

 set upload rules using volflags,  some examples:
@@ -1680,7 +1864,7 @@ set upload rules using volflags,  some examples:
    * just to avoid additional complexity in up2k which is enough of a mess already
 * `:c,lifetime=300` delete uploaded files when they become 5 minutes old

-you can also set transaction limits which apply per-IP and per-volume, but these assume `-j 1` (default) otherwise the limits will be off, for example `-j 4` would allow anywhere between 1x and 4x the limits you set depending on which processing node the client gets routed to
+you can also set transaction limits which apply per-IP and per-volume, but these assume `-j 1` (default) otherwise the limits will be messed up, for example `-j 4` would allow anywhere between 1x and 4x the limits you set depending on which processing node the client gets routed to

 * `:c,maxn=250,3600` allows 250 files over 1 hour from each IP (tracked per-volume)
 * `:c,maxb=1g,300` allows 1 GiB total over 5 minutes from each IP (tracked per-volume)
@@ -1761,6 +1945,57 @@ notes:
 * `:c,magic` enables filetype detection for nameless uploads, same as `--magic`
  * needs https://pypi.org/project/python-magic/ `python3 -m pip install --user -U python-magic`
  * on windows grab this instead `python3 -m pip install --user -U python-magic-bin`
+* `cachectl` changes how webbrowser will cache responses (the `Cache-Control` response-header); default is `no-cache` which will prevent repeated downloading of the same file unless necessary (browser will ask copyparty if the file has changed)
+  * adding `?cache` to a link will override this with "fully cache this for 69 seconds"; `?cache=321` is 321 seconds, and `?cache=i` is 7 days
+
+
+## descript.ion
+
+add a description to each file in a folder  by adding them to a textfile named `descript.ion`
+
+see https://copyparty.eu/beta/ for an example -- here's a basic `descript.ion` file:
+
+```
+bookmark.mp3 Taishi feat. Rita - Bookmark Memories
+slowstep.mp3 Taishi feat. 向日葵 - Slow Step -F.L.C.A-
+prsnlzr.mp3 Taishi feat. みとせのりこ - Personalizer
+cosmos.mp3 Taishi feat. Rita - Into the cosmos
+```
+
+
+## dothidden
+
+cosmetically hide specific files in a folder  by adding them to a textfile named `.hidden`
+
+this option is default-disabled; enable the volflag and/or global-option `dothidden`
+
+this is **cosmetic only!** the files are still easily accessible in many ways, for example with download-as-zip/tar, so **do not** rely on this for security.
+
+> also see the [--unlist](https://copyparty.eu/cli/#g-unlist) option which is somewhat similar -- `unlist` applies to the whole volume instead of just one folder; however, while dothidden also affects sftp and ftp, the `unlist` option is http/https-only
+
+
+## thumbnail pregen
+
+if you want to pre-generate everything on startup  (usually a bad idea);
+
+by default, thumbnails are created on-the-fly when a client needs it, and then cached on the server for [--th-maxage](https://copyparty.eu/cli/#g-th-maxage) seconds (default is one week), so most thumbnails only need to be created once, and are then eventually deleted from the cache to preserver disk space
+
+but if you need every thumbnail instantly available when a folder is viewed, then first increase the thumbnail expiration time to something really big, and then set global-option `th-pregen` and volflag `th_pregen` to a comma-separated list of thumbnail formats to automatically generate on server startup;
+
+the full list of all possible formats is: `j,jf,jf3,j3,w,wf,wf3,w3,x,xf,xf3,x3,opus,mp3,flac,wav` and I'll explain what those mean soon
+
+* `j` = jpeg cropped, `jf` = jpeg uncropped, `jf3` = jpeg uncropped triplesize, `j3` jpeg cropped triplesize
+* `w` = webm cropped, `wf` = webm uncropped, ..., `x` = jxl cropped, `xf` = jxl uncropped, ...
+* and yes, audio-transcodes are technically thumbnails according to copyparty -- don't think too much about it ( ﾟ ヮﾟ)
+  * unlike thumbnails, the expiry time for audio-transcodes is configured with [--ac-maxage](https://copyparty.eu/cli/#g-ac-maxage)
+
+anyways, obviously you **do not** want to pregenerate flac/wav because they're HUGE, and everything else also gets pretty big because it all adds up;
+
+* each regular thumbnail ( j, jf, w, wf, x, xf ) takes about 16 KiB of disk space
+* each triplesize thumb ( j3, jf3, w3, wf3, x3, xf3 ) takes about 96 KiB
+* each opus / mp3 audiotranscode takes... idk, 6 MiB? depends on song length
+
+so a thousand pictures converted to every possible regular-size image format (`j,jf,w,wf,x,xf`) takes **96 MiB,** and every possible 3x-size (`jf3,j3,wf3,w3,xf3,x3`) takes **562 MiB,** alternatively **658 MiB** in total for all, so that's why the default is to *not* pregenerate on startup, but instead do on-demand with a cache


 ## database location
@@ -1825,6 +2060,23 @@ see the beautiful mess of a dictionary in [mtag.py](https://github.com/9001/copy
 `--mtag-to` sets the tag-scan timeout; very high default (60 sec) to cater for zfs and other randomly-freezing filesystems. Lower values like 10 are usually safe, allowing for faster processing of tricky files


+### metadata from xattrs
+
+unix extended file attributes  (Linux-only) can be indexed into the db and made searchable;
+
+* `--db-xattr user.foo,user.bar` will index the xattrs `user.foo` and `user.bar`,
+* `--db-xattr user.foo=foo,user.bar=bar` will index them with the names `foo` and `bar`,
+* `--db-xattr ~~user.foo,user.bar` will index everything *except* `user.foo` and `user.bar`,
+* `--db-xattr ~~` will index everything
+
+however note that the tags must also be enabled with `-mte` so here are some complete examples:
+* `-e2ts --db-xattr user.foo,user.bar -mte +user.foo,user.bar`
+* `-e2ts --db-xattr user.foo=foo,user.bar=bar -mte +foo,bar`
+
+as for actually adding the xattr `user.foo` to a file in the first place,
+* `setfattr -n user.foo -v 'utsikt fra fløytoppen' photo.jpg`
+
+
 ## file parser plugins

 provide custom parsers to index additional tags,  also see [./bin/mtag/README.md](./bin/mtag/README.md)
@@ -1867,7 +2119,7 @@ trigger a program on uploads, renames etc ([examples](./bin/hooks/))

 you can set hooks before and/or after an event happens, and currently you can hook uploads, moves/renames, and deletes

-there's a bunch of flags and stuff, see `--help-hooks`
+there's a bunch of flags and stuff, see [`--help-hooks`](https://copyparty.eu/cli/#hooks-help-page)

 if you want to write your own hooks, see [devnotes](./docs/devnotes.md#event-hooks)

@@ -1946,7 +2198,7 @@ as for client-side stuff, there is [plugins for modifying UI/UX](./contrib/plugi

 autologin based on IP range (CIDR)  , using the global-option `--ipu`

-for example, if everyone with an IP that starts with `192.168.123` should automatically log in as the user `spartacus`, then you can either specify `--ipu=192.168.123.0/24=spartacus` as a commandline option, or put this in a config file:
+for example, if everyone with an IP that starts with `192.168.123` should automatically log in as the user `spartacus`, then you can either specify `--ipu=192.168.123.0/24=spartacus` as a commandline option, same as this config file example:

 ```yaml
 [global]
@@ -1962,7 +2214,7 @@ repeat the option to map additional subnets

 limit a user to certain IP ranges (CIDR)  , using the global-option `--ipr`

-for example, if the user `spartacus` should get rejected if they're not connecting from an IP that starts with `192.168.123` or `172.16`, then you can either specify `--ipr=192.168.123.0/24,172.16.0.0/16=spartacus` as a commandline option, or put this in a config file:
+for example, if the user `spartacus` should get rejected if they're not connecting from an IP that starts with `192.168.123` or `172.16`, then you can either specify `--ipr=192.168.123.0/24,172.16.0.0/16=spartacus` as a commandline option, same as this config file example:

 ```yaml
 [global]
@@ -1999,7 +2251,7 @@ but if you just want to let users change their own passwords, then you probably

 other ways to auth by header

-if you have a middleware which adds a header with a user identifier, for example tailscale's `Tailscale-User-Login: alice.m@forest.net` then you can automatically auth as `alice` by defining that mapping with `--idp-hm-usr '^Tailscale-User-Login^alice.m@forest.net^alice'` or the following config file:
+if you have a middleware which adds a header with a user identifier, for example tailscale's `Tailscale-User-Login: alice.m@forest.net` then you can automatically auth as `alice` by defining that mapping with `--idp-hm-usr '^Tailscale-User-Login^alice.m@forest.net^alice'` or the following config file example:

 ```yaml
 [global]
@@ -2130,9 +2382,7 @@ if you want to change the fonts, see [./docs/rice/](./docs/rice/)

 become a *real* webserver  which people can access by just going to your IP or domain without specifying a port

-**if you're on windows,** then you just need to add the commandline argument `-p 80,443` and you're done! nice
-
-**if you're on macos,** sorry, I don't know
+**if you're on windows or macos,** then you just need to add the commandline argument `-p 80,443` and you're done! nice

 **if you're on Linux,** you have the following 4 options:

@@ -2183,7 +2433,7 @@ example webserver / reverse-proxy configs:
 * [lighttpd subdomain](contrib/lighttpd/subdomain.conf) -- entire domain/subdomain
 * [lighttpd subpath](contrib/lighttpd/subpath.conf) -- location-based (not optimal, but in case you need it)
 * [nginx config](contrib/nginx/copyparty.conf) -- recommended
-* [traefik config](contrib/traefik/copyparty.yaml)
+* [traefik config](contrib/traefik/copyparty.yaml) -- only use v3.6.7 or newer [due to CVE-2025-66490](https://github.com/9001/copyparty/issues/1205)


 ### real-ip
@@ -2322,7 +2572,7 @@ change the association of a file extension

 using commandline args, you can do something like `--mime gif=image/jif` and `--mime ts=text/x.typescript` (can be specified multiple times)

-in a config file, this is the same as:
+that's the same as this config file example:

 ```yaml
 [global]
@@ -2351,12 +2601,15 @@ buggy feature? rip it out  by setting any of the following environment variables

 | env-var              | what it does |
 | -------------------- | ------------ |
+| `PRTY_NO_CTYPES`     | do not use features from external libraries such as kernel32 |
 | `PRTY_NO_DB_LOCK`    | do not lock session/shares-databases for exclusive access |
+| `PRTY_NO_ENVEXPAND`  | do not expand environment-variables in configs and args |
 | `PRTY_NO_IFADDR`     | disable ip/nic discovery by poking into your OS with ctypes |
 | `PRTY_NO_IMPRESO`    | do not try to load js/css files using `importlib.resources` |
 | `PRTY_NO_IPV6`       | disable some ipv6 support (should not be necessary since windows 2000) |
 | `PRTY_NO_LZMA`       | disable streaming xz compression of incoming uploads |
 | `PRTY_NO_MP`         | disable all use of the python `multiprocessing` module (actual multithreading, cpu-count for parsers/thumbnailers) |
+| `PRTY_NO_UNIDATA`    | disable loading the unicodedata c-extension; saves 400k ram, breaks `--srch-nfkc` |
 | `PRTY_NO_SQLITE`     | disable all database-related functionality (file indexing, metadata indexing, most file deduplication logic) |
 | `PRTY_NO_TLS`        | disable native HTTPS support; if you still want to accept HTTPS connections then TLS must now be terminated by a reverse-proxy |
 | `PRTY_NO_TPOKE`      | disable systemd-tmpfilesd avoider |
@@ -2386,16 +2639,35 @@ if your distro/OS is not mentioned below, there might be some hints in the [«on

 `pacman -S copyparty` (in [arch linux extra](https://archlinux.org/packages/extra/any/copyparty/))

-it comes with a [systemd service](./contrib/systemd/copyparty@.service) as well as a [user service](./contrib/systemd/copyparty-user.service), and expects to find a [config file](./contrib/systemd/copyparty.example.conf) in `/etc/copyparty/copyparty.conf` or `~/.config/copyparty/copyparty.conf`
+it comes with a [systemd service](./contrib/systemd/copyparty@.service) as well as a [user service](./contrib/systemd/copyparty-user.service), and expects to find a [config file](./contrib/systemd/copyparty.example.conf) in `/etc/copyparty.conf` or `~/.config/copyparty/copyparty.conf`

 after installing, start either the system service or the user service and navigate to http://127.0.0.1:3923 for further instructions (unless you already edited the config files, in which case you are good to go, probably)

+> to start the systemd service, either do `systemctl start --user copyparty` to start it as your own user, or `systemctl start copyparty@bob` to use unix-user `bob`
+

 ## fedora package

 does not exist yet;  there are rumours that it is being packaged! keep an eye on this space...


+## gentoo ::guru package
+
+`emerge www-servers/copyparty::guru` (in [::guru](https://wiki.gentoo.org/wiki/Project:GURU))
+
+but first enable the `::guru` repo;
+
+```bash
+emerge -an app-eselect/eselect-repository
+eselect repository enable guru
+emerge --sync guru
+```
+
+to start the service as a user:
+* OpenRC: `rc-service -U copyparty start && rc-update -U add copyparty default`
+* systemd: [todo]
+
+
 ## homebrew formulae

 `brew install copyparty ffmpeg`  -- https://formulae.brew.sh/formula/copyparty
@@ -2413,8 +2685,6 @@ requires a [flake-enabled](https://nixos.wiki/wiki/Flakes) installation of nix

 some recommended dependencies are enabled by default; [override the package](https://github.com/9001/copyparty/blob/hovudstraum/contrib/package/nix/copyparty/default.nix#L3-L22) if you want to add/remove some features/deps

-`ffmpeg-full` was chosen over `ffmpeg-headless` mainly because we need `withWebp` (and `withOpenmpt` is also nice) and being able to use a cached build felt more important than optimizing for size at the time -- PRs welcome if you disagree 👍
-

 ## nixos module

@@ -2540,6 +2810,12 @@ services.copyparty = {
  };
  # you may increase the open file limit for the process
  openFilesLimit = 8192;
+  
+  # override the package used by the module to add dependencies, e.g. for hooks
+  package = pkgs.copyparty.override {
+    # provides exiftool for bin/hooks/image-noexif.py
+    extraPackages = [ pkgs.exiftool ];
+  };
 };
 ```

@@ -2602,6 +2878,17 @@ quick summary of more eccentric web-browsers trying to view a directory index:
 <p align="center"><img src="https://github.com/user-attachments/assets/88deab3d-6cad-4017-8841-2f041472b853" /></p>


+# server hall of fame
+
+unexpected things that run copyparty:
+
+* an old [allwinner](https://a.ocv.me/pub/g/nerd-stuff/cpp/servers/aallwinner.jpg) android tv-box (ziptie-strapped to an HDD) running a firmware which flips the CPU into Big-Endian mode early during boot
+  * copyparty is [certified BE ready](https://a.ocv.me/pub/g/nerd-stuff/cpp/servers/be-ready.png) -- thanks, [Øl Telecom](http://ol-tele.com/)!
+* an [SGI O2 (photo)](https://a.ocv.me/pub/g/nerd-stuff/cpp/servers/sgi-o2.jpg?cache) with a grand total of 64 MiB RAM running SGI IRIX; [screenshot](https://a.ocv.me/pub/g/nerd-stuff/cpp/servers/sgi-o2.png?cache)
+  * thanks again to the wonderful people at [Øl Telecom](http://ol-tele.com/)
+* a [wristwatch](https://a.ocv.me/pub/g/nerd-stuff/cpp/servers/clockyparty.jpg)
+
+
 # client examples

 interact with copyparty using non-browser clients
@@ -2622,6 +2909,10 @@ interact with copyparty using non-browser clients
  * `chunk(){ curl -H pw:wark -T- http://127.0.0.1:3923/;}`  
    `chunk <movie.mkv`

+* curl: append to existing file with `?apnd`
+  * `log(){ curl -H pw:wark -T- http://127.0.0.1:3923/logfile.txt?apnd;}`  
+    `echo hey | log`
+
 * bash: when curl and wget is not available or too boring
  * `(printf 'PUT /junk?pw=wark HTTP/1.1\r\n\r\n'; cat movie.mkv) | nc 127.0.0.1 3923`
  * `(printf 'PUT / HTTP/1.1\r\n\r\n'; cat movie.mkv) >/dev/tcp/127.0.0.1/3923`
@@ -2744,12 +3035,12 @@ below are some tweaks roughly ordered by usefulness:
 * `--no-htp --hash-mt=0 --mtag-mt=1 --th-mt=1` minimizes the number of threads; can help in some eccentric environments (like the vscode debugger)
 * when running on AlpineLinux or other musl-based distro, try mimalloc for higher performance (and twice as much RAM usage); `apk add mimalloc2` and run copyparty with env-var `LD_PRELOAD=/usr/lib/libmimalloc-secure.so.2`
  * note that mimalloc requires special care when combined with prisonparty and/or bubbleparty/bubblewrap; you must give it access to `/proc` and `/sys` otherwise you'll encounter issues with FFmpeg (audio transcoding, thumbnails)
-* `-j0` enables multiprocessing (actual multithreading), can reduce latency to `20+80/numCores` percent and generally improve performance in cpu-intensive workloads, for example:
+* `-j0` (usually *not* recommended) enables multiprocessing (actual multithreading), can reduce latency to `20+80/numCores` percent and generally improve performance in cpu-intensive workloads, for example:
  * lots of connections (many users or heavy clients)
  * simultaneous downloads and uploads saturating a 20gbps connection
  * if `-e2d` is enabled, `-j2` gives 4x performance for directory listings; `-j4` gives 16x
  
-  ...however it also increases the server/filesystem/HDD load during uploads, and adds an overhead to internal communication, so it is usually a better idea to don't
+  ...however it will probably *reduce* performance in most cases, since it also increases the server/filesystem/HDD load during uploads, and adds an overhead to internal communication, so keeping the default is generally best
 * using [pypy](https://www.pypy.org/) instead of [cpython](https://www.python.org/) *can* be 70% faster for some workloads, but slower for many others
  * and pypy can sometimes crash on startup with `-j0` (TODO make issue)

@@ -2781,15 +3072,20 @@ when uploading files,

 # security

-there is a [discord server](https://discord.gg/25J8CdTT6G)  with an `@everyone` for all important updates (at the lack of better ideas)
+there is a [discord server](https://discord.gg/25J8CdTT6G) with announcements  ; an `@everyone` for all important updates (at the lack of better ideas)

 some notes on hardening

 * set `--rproxy 0` *if and only if* your copyparty is directly facing the internet (not through a reverse-proxy)
  * cors doesn't work right otherwise
 * if you allow anonymous uploads or otherwise don't trust the contents of a volume, you can prevent XSS with volflag `nohtml`
-  * this returns html documents as plaintext, and also disables markdown rendering
-* when running behind a reverse-proxy, listen on a unix-socket for tighter access control (and more performance); see [reverse-proxy](#reverse-proxy) or `--help-bind`
+  * this returns html documents and svg images as plaintext, and also disables markdown rendering
+  * the `nohtml` volflag also enables `noscript` which, on its own, prevents *most* javascript from running; enabling just `noscript` without `nohtml` makes it probably-safe (see below) to view html and svg files, but `nohtml` is necessary to block javascript in markdown documents
+    * "probably-safe" because it relies on `Content-Security-Policy` so it depends on the reverseproxy to forward it, and the browser to understand it, but `nohtml` (the nuclear option) always works
+* when running behind a reverse-proxy, listen on a unix-socket for tighter access control (and more performance); see [reverse-proxy](#reverse-proxy) or [`--help-bind`](https://copyparty.eu/cli/#bind-help-page)
+* put ffmpeg into a sandbox (helps against future ffmpeg vulns) by configuring [--th-bwrap](https://copyparty.eu/cli/#g-th-bwrap) and enable it with `--use-bwrap y`
+  * usually impossible when running in docker or podman
+  * the default value of `--th-bwrap` (see `--help` on your own server) is autogenerated best-effort guess based on your setup; installing `bwrap` and configuring this is up to you (good luck)

 safety profiles:

@@ -2833,6 +3129,23 @@ behavior that might be unexpected
  * or eliminate the problem entirely by only giving write-access to trustworthy people :^)


+## csp nonce
+
+unauthorized javascript is blocked  by means of the default [content-security-policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CSP) which enforces "script nonces", to help prevent XSS and similar
+
+usually this Just Works and is correctly configured by default; you can stop reading this section and move on, nothing to see here
+
+but if you are the unlucky 0.1% with a hosting-setup funky enough for this to not work as intended, then you will also be seeing this message: `Looks like javascript is broken; see "csp nonce" in the readme`
+
+there is two possible reasons why this might happen:
+
+1. your browser is simply too old to understand the javascript, which is not likely, because just about every browser is supported, even internet explorer 11
+
+2. something in your hosting-setup is tampering with the responses from copyparty and destroying this security feature; the culprit is probably a misconfigured reverseproxy, a CDN, a WAF, or some other mitm, but a popular offender is **"cloudflare rocket loader"**
+
+you can try to **temporarily** set global-options `--csp-ui no` and `--csp-dl no` and restart copyparty to check whether it is due to reason #2. Remember to refresh the webbrowser with CTRL-SHIFT-R to check properly. If that makes the problem go away, then turn them back off (**never** keep them enabled), restart copyparty, open the `Network` tab in the webbrowser, and refresh the website; you should see the `content-security-policy` header including a nonce, for example `nonce-BxcJbvdu1z3S/ycPBjVWuA==` which should also appear inside the HTML of that response like this: `<script nonce="BxcJbvdu1z3S/ycPBjVWuA==">` -- if it doesn't then that's the problem; something in your hosting stack is dangerously misconfigured
+
+
 ## cors

 cross-site request config
@@ -2859,6 +3172,8 @@ permissions `wG` (write + upget) lets users upload files and receive their own f

 share specific folders in a volume  without giving away full read-access to the rest -- the visitor only needs the `g` (get) permission to view the link

+> ℹ️ a better alternative to this is [shares](#shares)
+
 volflag `dk` generates dirkeys (per-directory accesskeys) for all folders, granting read-access to that folder; by default only that folder itself, no subfolders

 volflag `dky` disables the actual key-check, meaning anyone can see the contents of a folder where they have `g` access, but not its subdirectories
@@ -2877,7 +3192,7 @@ dirkeys are generated based on another salt (`--dk-salt`) + filesystem-path and

 ## password hashing

-you can hash passwords  before putting them into config files / providing them as arguments; see `--help-pwhash` for all the details
+you can hash passwords  before putting them into config files / providing them as arguments; see [`--help-pwhash`](https://copyparty.eu/cli/#pwhash-help-page) for all the details

 `--ah-alg argon2` enables it, and if you have any plaintext passwords then it'll print the hashed versions on startup so you can replace them

@@ -2894,15 +3209,26 @@ when generating hashes using `--ah-cli` for docker or systemd services, make sur

 ## https

-both HTTP and HTTPS are accepted  by default, but letting a [reverse proxy](#reverse-proxy) handle the https/tls/ssl would be better (probably more secure by default)
+both HTTP and HTTPS are accepted  by default, but please ignore copyparty's built-in https/tls support and instead use a [reverse proxy](#reverse-proxy) to handle https/tls/ssl

-copyparty doesn't speak HTTP/2 or QUIC, so using a reverse proxy would solve that as well -- but note that HTTP/1 is usually faster than both HTTP/2 and HTTP/3
+* reverseproxies do a better job following [best practices](https://cipherlist.eu/) meaning they are more secure, and probably also have higher performance
+* also, copyparty doesn't speak HTTP/2 or QUIC, so using a reverse proxy would solve that as well -- but note that HTTP/1 is usually faster than both HTTP/2 and HTTP/3

 if [cfssl](https://github.com/cloudflare/cfssl/releases/latest) is installed, copyparty will automatically create a CA and server-cert on startup
 * the certs are written to `--crt-dir` for distribution, see `--help` for the other `--crt` options
 * this will be a self-signed certificate so you must install your `ca.pem` into all your browsers/devices
 * if you want to avoid the hassle of distributing certs manually, please consider using a reverse proxy

+to install cfssl on windows:
+* [download](https://github.com/cloudflare/cfssl/releases/latest) `cfssl_windows_amd64.exe`, `cfssljson_windows_amd64.exe`, `cfssl-certinfo_windows_amd64.exe`
+* rename them to `cfssl.exe`, `cfssljson.exe`, `cfssl-certinfo.exe`
+* put them in PATH, for example inside `c:\windows\system32`
+
+if you really wanna give copyparty an existing TLS certificate then do one of the following:
+* `--no-crt --cert server.pem` where `server.pem` is a concatenation of key + cert + chain (in that order), or...
+* `--no-crt --cert server.crt --certkey server.key` where `server.key` is the key, and `server.crt` is a concatenation of cert + chain (in that order)
+* file-extensions don't matter, but all files are expected to be [PEM-style](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/res/insecure.pem)
+

 # recovering from crashes

@@ -2937,7 +3263,7 @@ mandatory deps:

 ## optional dependencies

-install these to enable bonus features
+enable bonus features  by installing these python-packages from pypi or so:

 enable [hashed passwords](#password-hashing) in config: `argon2-cffi`

@@ -2945,6 +3271,8 @@ enable [ftp-server](#ftp-server):
 * for just plaintext FTP, `pyftpdlib` (is built into the SFX)
 * with TLS encryption, `pyftpdlib pyopenssl`

+enable [sftp-server](#sftp-server): `paramiko`
+
 enable [music tags](#metadata-from-audio-files):
 * either `mutagen` (fast, pure-python, skips a few tags, makes copyparty GPL? idk)
 * or `ffprobe` (20x slower, more accurate, possibly dangerous depending on your distro and users)
@@ -2955,16 +3283,20 @@ enable [thumbnails](#thumbnails) of...
 * **HEIF pictures:** `pyvips` or `ffmpeg` or `pillow-heif`
 * **AVIF pictures:** `pyvips` or `ffmpeg` or `pillow-avif-plugin` or pillow v11.3+
 * **JPEG XL pictures:** `pyvips` or `ffmpeg`
-* **RAW images:** `rawpy`, plus one of `pyvips` or `Pillow` (for some formats)
+* **RAW photos:** either `libraw dcraw_emu` or `rawpy`, plus either `pyvips` or `Pillow`

 enable sending [zeromq messages](#zeromq) from event-hooks: `pyzmq`

-enable [smb](#smb-server) support (**not** recommended): `impacket==0.12.0`
+enable [smb](#smb-server) support (**not** recommended): `impacket==0.13.0`

-`pyvips` gives higher quality thumbnails than `Pillow` and is 320% faster, using 270% more ram: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`
+`pyvips` adds support for some additional image formats, but can use extreme amounts of RAM
+* to install `pyvips` on Linux: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`
+* to install `pyvips` on windows: `pip install --user -U "pyvips[binary]"`

 to install FFmpeg on Windows, grab [a recent build](https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z) -- you need `ffmpeg.exe` and `ffprobe.exe` from inside the `bin` folder; copy them into `C:\Windows\System32` or any other folder that's in your `%PATH%`

+if your ffmpeg/ffprobe binaries have nonstandard names -- such as `ffmpeg8` (macports) -- set environment variables `PRTY_FFMPEG_BIN` and `PRTY_FFPROBE_BIN` to the corret name (or full path)
+

 ### dependency chickenbits

@@ -2979,17 +3311,23 @@ set any of the following environment variables to disable its associated optiona
 | -------------------- | ------------ |
 | `PRTY_NO_ARGON2`     | disable argon2-cffi password hashing |
 | `PRTY_NO_CFSSL`      | never attempt to generate self-signed certificates using [cfssl](https://github.com/cloudflare/cfssl) |
+| `PRTY_NO_DCRAW`      | disable all [libraw](https://www.libraw.org/homepage)-based thumbnail support for RAW images |
 | `PRTY_NO_FFMPEG`     | **audio transcoding** goes byebye, **thumbnailing** must be handled by Pillow/libvips |
 | `PRTY_NO_FFPROBE`    | **audio transcoding** goes byebye, **thumbnailing** must be handled by Pillow/libvips, **metadata-scanning** must be handled by mutagen |
 | `PRTY_NO_MAGIC`      | do not use [magic](https://pypi.org/project/python-magic/) for filetype detection |
 | `PRTY_NO_MUTAGEN`    | do not use [mutagen](https://pypi.org/project/mutagen/) for reading metadata from media files; will fallback to ffprobe |
+| `PRTY_NO_PARAMIKO`   | disable sftp server ([paramiko](https://www.paramiko.org/)-based) |
+| `PRTY_NO_PARTFTPY`   | disable tftp server ([partftpy](https://github.com/9001/partftpy)-based) |
 | `PRTY_NO_PIL`        | disable all [Pillow](https://pypi.org/project/pillow/)-based thumbnail support; will fallback to libvips or ffmpeg |
 | `PRTY_NO_PILF`       | disable Pillow `ImageFont` text rendering, used for folder thumbnails |
 | `PRTY_NO_PIL_AVIF`   | disable Pillow avif support (internal and/or [plugin](https://pypi.org/project/pillow-avif-plugin/)) |
 | `PRTY_NO_PIL_HEIF`   | disable 3rd-party Pillow plugin for [HEIF support](https://pypi.org/project/pillow-heif/) |
+| `PRTY_NO_PIL_JXL`    | disable 3rd-party Pillow plugin for [JXL support](https://pypi.org/project/pillow-jxl-plugin/) |
 | `PRTY_NO_PIL_WEBP`   | disable use of native webp support in Pillow |
 | `PRTY_NO_PSUTIL`     | do not use [psutil](https://pypi.org/project/psutil/) for reaping stuck hooks and plugins on Windows |
-| `PRTY_NO_RAW`        | disable all [rawpy](https://pypi.org/project/rawpy/)-based thumbnail support for RAW images |
+| `PRTY_NO_PYFTPD`     | disable ftp(s) server ([pyftpdlib](https://pypi.org/project/pyftpdlib/)-based) |
+| `PRTY_NO_RAW`        | same as `PRTY_NO_DCRAW` plus `PRTY_NO_RAWPY` |
+| `PRTY_NO_RAWPY`      | disable all [rawpy](https://pypi.org/project/rawpy/)-based thumbnail support for RAW images |
 | `PRTY_NO_VIPS`       | disable all [libvips](https://pypi.org/project/pyvips/)-based thumbnail support; will fallback to Pillow or ffmpeg |

 example: `PRTY_NO_PIL=1 python3 copyparty-sfx.py`
@@ -3058,7 +3396,8 @@ another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty
 run it by doubleclicking it, or try typing `python copyparty.pyz` in your terminal/console/commandline/telex if that fails

 it is a python [zipapp](https://docs.python.org/3/library/zipapp.html) meaning it doesn't have to unpack its own python code anywhere to run, so if the filesystem is busted it has a better chance of getting somewhere
-* but note that it currently still needs to extract the web-resources somewhere (they'll land in the default TEMP-folder of your OS)
+
+> there is also [copyparty-en.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty-en.pyz), english-only and without smb support (enterprise-friendly)


 # install on android
@@ -3090,6 +3429,9 @@ and then copypaste the following command into `a-Shell`:
 curl -L https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh | sh
 ```

+> if you want the latest copyparty beta, then do this instead:  
+> `curl -L https://copyparty.eu/beta/setup-ashell.sh | sh`
+
 what this does:
 * creates a basic [config file](#accounts-and-volumes) named `cpc` which you can edit with `vim cpc`
 * adds the command `cpp` to launch copyparty with that config file
@@ -1,9 +1,7 @@
 # Security Policy

-if you hit something extra juicy pls let me know on either of the following
+if you hit something extra juicy pls let me know on one of the following:
 * email -- `copyparty@ocv.ze` except `ze` should be `me`
-* [mastodon dm](https://layer8.space/@tripflag) -- `@tripflag@layer8.space`
 * [github private vulnerability report](https://github.com/9001/copyparty/security/advisories/new), wow that form is complicated
-* [twitter dm](https://twitter.com/tripflag) (if im somehow not banned yet)

 no bug bounties sorry! all i can offer is greetz in the release notes
@@ -26,12 +26,12 @@ and consider using [../docs/rclone.md](../docs/rclone.md) instead; usually a bit
 ## to run this on windows:
 * install [winfsp](https://github.com/billziss-gh/winfsp/releases/latest) and [python 3](https://www.python.org/downloads/)
  * [x] add python 3.x to PATH (it asks during install)
-* `python -m pip install --user fusepy` (or grab a copy of `fuse.py` from the `connect` page on your copyparty, and keep it in the same folder)
+* `python -m pip install --user mfusepy` (or grab a copy of `mfusepy.py` from the `connect` page on your copyparty, and keep it in the same folder)
 * `python ./partyfuse.py n: http://192.168.1.69:3923/`

 10% faster in [msys2](https://www.msys2.org/), 700% faster if debug prints are enabled:
 * `pacman -S mingw64/mingw-w64-x86_64-python{,-pip}`
-* `/mingw64/bin/python3 -m pip install --user fusepy`
+* `/mingw64/bin/python3 -m pip install --user mfusepy`
 * `/mingw64/bin/python3 ./partyfuse.py [...]`

 you could replace winfsp with [dokan](https://github.com/dokan-dev/dokany/releases/latest), let me know if you [figure out how](https://github.com/dokan-dev/dokany/wiki/FUSE)  
@@ -78,6 +78,7 @@ cd /mnt/nas/music/.hist
 # [`prisonparty.sh`](prisonparty.sh)
 * run copyparty in a chroot, preventing any accidental file access
 * creates bindmounts for /bin, /lib, and so on, see `sysdirs=`
+* **not compatible with:** `--th-bwrap`

 # [`bubbleparty.sh`](bubbleparty.sh)
 * run copyparty in an isolated process, preventing any accidental file access and more
@@ -0,0 +1,36 @@
+# /!\ Warning: be careful, as webdav clients often generate a large number of 404 requets.
+
+# In your `jail.local`, add:
+# [copyparty]
+# enabled = true
+# logtimezone = UTC
+# logpath = /path/to/log/file # or keep the default value if you're using systemd
+
+# Create the `copyparty.conf` file in `filter.d` with the following:
+# [Definition]
+# failregex = ^ <ADDR>$
+# ignoreregex =
+# datepattern = ^fail2ban: %%Y-%%m-%%dT%%H:%%M:%%S
+
+# First check `--dont-ban`, and if it doesn't match, log the line to be intercepted by fail2ban.
+from datetime import datetime, UTC
+def main(cli, vn="", rem=""):
+    now = datetime.now(UTC).isoformat()[:19]
+    msg = "\nfail2ban: %s %s"
+    if not vn and not rem:
+        # got called by --xban
+        cli["log"](msg % (now, cli["ip"]), 3)
+        return {"rc": 0}
+
+    cond = cli.args.dont_ban
+    if (
+        cond == "any"
+        or (cond == "auth" and cli.uname != "*")
+        or (cond == "aa" and cli.avol)
+        or (cond == "av" and cli.can_admin)
+        or (cond == "rw" and cli.can_read and cli.can_write)
+    ):
+        return ""
+
+    cli.log(msg % (now, cli.ip), 3)
+    return ""
@@ -22,10 +22,11 @@ each plugin must define a `main()` which takes 3 arguments;

 * [redirect.py](redirect.py) sends an HTTP 301 or 302, redirecting the client to another page/file
 * [randpic.py](randpic.py) redirects `/foo/bar/randpic.jpg` to a random pic in `/foo/bar/`
-* [sorry.py](answer.py) replies with a custom message instead of the usual 404
+* [sorry.py](sorry.py) replies with a custom message instead of the usual 404
 * [nooo.py](nooo.py) replies with an endless noooooooooooooo
 * [never404.py](never404.py) 100% guarantee that 404 will never be a thing again as it automatically creates dummy files whenever necessary
 * [caching-proxy.py](caching-proxy.py) transforms copyparty into a squid/varnish knockoff
+* [404-to-fail2ban.py](404-to-fail2ban.py) creates 404 logs for fail2ban

 ## on403

@@ -8,7 +8,7 @@ def say_no():


 def main(cli, vn, rem):
-    cli.send_headers(None, 404, "text/plain")
+    cli.send_headers("oh_f", None, 404, "text/plain")

    for chunk in say_no():
        cli.s.sendall(chunk)
@@ -4,6 +4,11 @@ these programs either take zero arguments, or a filepath (the affected file), or

 run copyparty with `--help-hooks` for usage details / hook type explanations (xm/xbu/xau/xiu/xbc/xac/xbr/xar/xbd/xad/xban)

+in particular, if a hook is loaded into copyparty with the hook-flag `c` ("check") then its exit-code controls the action that launched the hook:
+* exit-code `0` = allow the action, and/or continue running the next hook
+* exit-code `100` = allow the action, and stop running any remaining consecutive hooks
+* anything else = reject/prevent the original action, and don't run the remaining hooks
+
 > **note:** in addition to event hooks (the stuff described here), copyparty has another api to run your programs/scripts while providing way more information such as audio tags / video codecs / etc and optionally daisychaining data between scripts in a processing pipeline; if that's what you want then see [mtp plugins](../mtag/) instead


@@ -31,10 +36,16 @@ these are `--xiu` hooks; unlike `xbu` and `xau` (which get executed on every sin
 * [reject-and-explain.py](reject-and-explain.py) shows a custom error-message when it rejects an upload
 * [reject-ramdisk.py](reject-ramdisk.py) rejects the upload if the destination is a ramdisk
  * this hook uses the `I` flag which makes it 140x faster, but if the plugin has a bug it may crash copyparty
+* [phonecam-sorter.py](phonecam-sorter.py) is for syncing backups of phone gallery into year/month folders
+
+
+# more upload stuff
+* combine [reloc-by-wark-xbu.py](reloc-by-wark-xbu.py) and [reloc-by-wark-xau.py](reloc-by-wark-xau.py) to rename uploads to the checksum of the file contents


 # on message
 * [wget.py](wget.py) lets you download files by POSTing URLs to copyparty
+  * [wget-i.py](wget-i.py) is an import-safe modification of this hook (starts 140x faster, but higher chance of bugs)
 * [qbittorrent-magnet.py](qbittorrent-magnet.py) starts downloading a torrent if you post a magnet url
 * [usb-eject.py](usb-eject.py) adds web-UI buttons to safe-remove usb flashdrives shared through copyparty
 * [msg-log.py](msg-log.py) is a guestbook; logs messages to a doc in the same folder
@@ -42,3 +53,4 @@ these are `--xiu` hooks; unlike `xbu` and `xau` (which get executed on every sin

 # general concept demos
 * [import-me.py](import-me.py) shows how the `I` flag makes the hook 140x faster (but you need to be Very Careful when writing the plugin)
+  * [wget-i.py](wget-i.py) is an import-safe modification of [wget.py](wget.py)
@@ -0,0 +1,85 @@
+#!/usr/bin/env python3
+
+import os
+import re
+import sys
+
+
+_ = r"""
+purpose:
+  sync/backup phone gallery (pics/vids) to a nas which runs copyparty
+
+what this hook does:
+  redirect photos/videos into date-based folders, named "%Y%m" according
+  to filename; that is, "/cam/sort/DCIM/D027/PXL_20250828_202820075.jpg"
+  will be redirected to "/cam/pics/ts/202508/PXL_20250828_202820075.jpg"
+
+requires this setup on the server:
+  * /cam/ is a volume with all your pics/vids
+  * the phone must upload to somewhere below /cam/sort/
+  * photo/video filenames match the following regex:
+     ^[a-z]{3,4}_([0-9]{6})[0-9]{2}_[0-9]{6}.*\.(jpg|mp4)$
+  * so for example these filenames are OK:
+    * PXL_20250828_202820075.jpg
+    * IMG_20251123_202405.jpg
+    * PANO_20180421_121245_03.t8o5.jpg
+    * VID_20250307_220250-crf34.mp4
+
+example server config as a volflag:
+  -v /mnt/thedisk/gallery:cam:rwa,ed:c,e2ds,xbu=I,~/hooks/phonecam-sorter.py
+
+example server config in a copyparty config file:
+  [/cam]
+    /mnt/thedisk/gallery
+    accs:
+      rwa: ed
+    flags:
+      e2ds
+      xbu: I,~/hooks/phonecam-sorter.py
+
+command to run on phone to sync to server, sending new files only:
+  ~/bin/u2c.py -te homenas.ca -a '$homenas.pw' https://10.0.0.2:3923/cam/sort/ ~/storage/dcim/Camera
+
+the phone-command above assumes this setup on the phone:
+* termux or another linux-like thing with access to gallery files
+* homenas.ca is a textfile with the server's CA certificate
+* homenas.pw is a textfile with the copyparty password inside
+* ~/storage/dcim/ exists; if not, run termux-setup-storage
+* ~/bin/u2c.py exists; if not, curl -LO https://10.0.0.2:3923/.cpr/a/u2c.py && chmod 755 u2c.py
+
+unfortunate consequences:
+* due to the redirection, u2c -z doesn't do anything, so every local
+   file is hashed every time, wasting some time and cpu on the phone
+   but otoh certainly avoids corruption on either side (phone/server)
+
+"""
+
+
+def main(inf):
+    log = inf["log"]
+    vdir, fn = os.path.split(inf["vp"])
+    if not vdir.startswith("cam/sort"):
+        return {"rc": 0}  # 2nd call after 1st reloc; already done
+
+    m = re.match(r"^[a-z]{3,4}_([0-9]{6})[0-9]{2}_[0-9]{6}.*\.(jpg|mp4)$", fn.lower())
+    if not m:
+        return {"rc": 0}  # not recognized; write to original path in /cam/sort/
+
+    ym, ext = m.groups()
+    if ext == "jpg":
+        dst = f"/cam/pics/ts/{ym}/"
+    elif ext == "mp4":
+        dst = f"/cam/vids/ts/{ym}/"
+    else:
+        return {"rc": 0}
+
+    ap = "%s%s" % (inf["ap"].split("/cam/sort/")[0], dst)
+    if not os.path.isdir(ap):
+        os.makedirs(ap)
+
+    ret = {"rc": 0, "reloc": {"vp": dst}}
+    return ret
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,92 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+
+
+_ = r"""
+rename incoming uploads according to the "wark" (the file identifier)
+which is basically but not exactly a sha512 hash of the file contents
+
+NOTE: this does NOT work with up2k uploads (dragdrop into browser);
+  combine this hook with reloc-by-wark-xbu.py to fix that
+
+example usage as global config:
+    -e2d --xau I,c,bin/hooks/reloc-by-wark-xau.py
+
+parameters explained,
+    e2d = enable up2k database (mandatory for xau hooks)
+    xau = execute before upload
+    I   = import this hook for performance; do not fork / subprocess
+    c   = "check"; reject upload if this hook crashes due to a bug
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,e2d,xau=I,c,bin/hooks/reloc-by-wark-xau.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all uploads with the params explained above)
+
+example usage as a volflag in a copyparty config file:
+    [/inc]
+      srv/inc
+      accs:
+        r: *
+        rw: ed
+      flags:
+        e2d, xau: I,c,bin/hooks/reloc-by-wark-xau.py
+"""
+
+
+def main(inf):
+    if inf.get("wark"):
+        # this is an up2k upload; nothing to be done, just bail
+        return {}
+
+    abspath = inf["ap"]  # filesystem path to the uploaded file
+
+    # we don't have the wark yet so need to calculate it;
+    # generating a regular sha512 would of course be much easier,
+    # but then filenames would be different depending on how the
+    # file was uploaded (laaame) so let's do it the hard way
+
+    # use the standard up2k-salt which nobody ever changes:
+    salt = "hunter2"
+
+    # to generate the wark we'll need some functions from copyparty;
+    # follow the trail to the copyparty module and grab them from there:
+
+    import inspect
+
+    libpath = inspect.getfile(inf["log"])
+    libpath = os.path.dirname(os.path.dirname(libpath))
+    sys.path.insert(0, libpath)
+
+    from copyparty.up2k import up2k_hashlist_from_file, up2k_wark_from_hashlist
+
+    chunklist, st = up2k_hashlist_from_file(abspath)
+    wark = up2k_wark_from_hashlist(salt, st.st_size, chunklist)
+
+    # okay nice
+    # the rest of the code below is just copied from reloc-by-wark-xbu.py
+    # -------------------------------------------------------------------------
+
+    # grab the original filename from the vpath...
+    vdir, fn = os.path.split(inf["vp"])
+
+    # ...to retain the original file extension, if any
+    try:
+        fn, ext = fn.rsplit(".", 1)
+    except:
+        ext = ""
+
+    # use the first 16 characters; 12 bytes of entropy,
+    # roughly one collision for every 26 million files
+    fn = wark[:16]
+
+    if ext:
+        ext = ext.lower()
+        fn += "." + ext
+
+    return {"reloc": {"fn": fn}}
@@ -0,0 +1,69 @@
+#!/usr/bin/env python3
+
+import os
+
+
+_ = r"""
+rename incoming uploads according to the "wark" (the file identifier)
+which is basically but not exactly a sha512 hash of the file contents
+
+NOTE: this only works for up2k uploads (dragdrop into browser);
+  combine this with reloc-by-wark-xau.py to cover the other protocols
+
+example usage as global config:
+    -e2d --xbu I,c,bin/hooks/reloc-by-wark-xbu.py
+
+parameters explained,
+    e2d = enable up2k database (mandatory for xbu hooks)
+    xbu = execute before upload
+    I   = import this hook for performance; do not fork / subprocess
+    c   = "check"; reject upload if this hook crashes due to a bug
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,e2d,xbu=I,c,bin/hooks/reloc-by-wark-xbu.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all uploads with the params explained above)
+
+example usage as a volflag in a copyparty config file:
+    [/inc]
+      srv/inc
+      accs:
+        r: *
+        rw: ed
+      flags:
+        e2d, xbu: I,c,bin/hooks/reloc-by-wark-xbu.py
+"""
+
+
+def main(inf):
+    wark = inf.get("wark")
+    if not wark:
+        # not an up2k upload, so we don't have the hash;
+
+        # option 1: let upload proceed with original filename
+        return {}
+
+        # option 2: reject the upload
+        return {"rejectmsg": "only up2k uploads are allowed in this volume"}
+
+    # grab the original filename from the vpath...
+    vdir, fn = os.path.split(inf["vp"])
+
+    # ...to retain the original file extension, if any
+    try:
+        fn, ext = fn.rsplit(".", 1)
+    except:
+        ext = ""
+
+    # use the first 16 characters; 12 bytes of entropy,
+    # roughly one collision for every 26 million files
+    fn = wark[:16]
+
+    if ext:
+        ext = ext.lower()
+        fn += "." + ext
+
+    return {"reloc": {"fn": fn}}
@@ -34,7 +34,10 @@ MOUNT_BASE = b"/run/media/egon/"

 def main():
    try:
-        label = sys.argv[1].split(":usb-eject:")[1].split(":")[0]
+        msg = sys.argv[1]
+        if msg.startswith("upload-queue-empty;"):
+            return
+        label = msg.split(":usb-eject:")[1].split(":")[0]
        mp = MOUNT_BASE + unquote(label)
        # print("ejecting [%s]... " % (mp,), end="")
        mp = os.path.abspath(os.path.realpath(mp))
@@ -0,0 +1,100 @@
+#!/usr/bin/env python3
+
+import os
+import threading
+import subprocess as sp
+
+
+_ = r"""
+use copyparty as a file downloader by POSTing URLs as
+application/x-www-form-urlencoded (for example using the
+📟 message-to-server-log in the web-ui)
+
+this hook is a modified copy of wget.py, modified to
+make it import-safe so it can be run with the 'I' flag,
+which speeds up the startup time of the hook by 140x
+
+example usage as global config:
+    --xm aw,I,bin/hooks/wget-i.py
+
+parameters explained,
+    xm = execute on message-to-server-log
+    aw = only users with write-access can use this
+    I = import; do not fork / subprocess
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,xm=aw,I,bin/hooks/wget.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all messages with the params explained above)
+
+example usage as a volflag in a copyparty config file:
+    [/inc]
+      srv/inc
+      accs:
+        r: *
+        rw: ed
+      flags:
+        xm: aw,I,bin/hooks/wget.py
+
+the volflag examples only kicks in if you send the message
+while you're in the /inc folder (or any folder below there)
+
+IMPORTANT NOTE:
+    because this hook uses the 'I' flag to run inside copyparty,
+    many other flags will not work (f,j,c3,t3600 as seen in the
+    original wget.py), and furthermore + more importantly we
+    need to be EXCEPTIONALLY CAREFUL to avoid side-effects, so
+    the os.chdir has been replaced with cwd=dirpath for example
+"""
+
+
+def do_stuff(inf):
+    """
+    worker function which is executed in another thread to
+    avoid blocking copyparty while the download is running,
+    since we cannot use the 'f,t3600' hook-flags with 'I'
+    """
+
+    # first things first; grab the logger-function which copyparty is letting us borrow
+    log = inf["log"]
+
+    url = inf["txt"]
+    if url.startswith("upload-queue-empty;"):
+        return
+
+    if "://" not in url:
+        url = "https://" + url
+
+    proto = url.split("://")[0].lower()
+    if proto not in ("http", "https", "ftp", "ftps"):
+        raise Exception("bad proto {}".format(proto))
+
+    dirpath = inf["ap"]
+
+    name = url.split("?")[0].split("/")[-1]
+    msg = "-- DOWNLOADING " + name
+    log(msg)
+    tfn = os.path.join(dirpath, msg)
+    open(tfn, "wb").close()
+
+    cmd = ["wget", "--trust-server-names", "-nv", "--", url]
+
+    try:
+        # two things to note here:
+        # - cannot use the `c3` hook-flag with `I` so mute output with stdout=sp.DEVNULL instead;
+        # - MUST NOT use os.chdir with 'I' so use cwd=dirpath instead 
+        sp.check_call(cmd, cwd=dirpath, stdout=sp.DEVNULL)
+    except:
+        t = "-- FAILED TO DOWNLOAD " + name
+        log(t, 3)  # 3=yellow=warning
+        open(os.path.join(dirpath, t), "wb").close()
+        raise  # have copyparty scream about the details in the log
+
+    os.unlink(tfn)
+
+
+def main(inf):
+    threading.Thread(target=do_stuff, args=(inf,), daemon=True).start()
@@ -47,6 +47,9 @@ while you're in the /inc folder (or any folder below there)
 def main():
    inf = json.loads(sys.argv[1])
    url = inf["txt"]
+    if url.startswith("upload-queue-empty;"):
+        return
+
    if "://" not in url:
        url = "https://" + url

@@ -68,3 +68,8 @@ instead of affecting all volumes, you can set the options for just one volume li
 * `:c,mtp=key=f,audio-key.py`
 * `:c,mtp=.bpm=f,audio-bpm.py`
 * `:c,mtp=ahash,vhash=f,media-hash.py`
+
+
+# tips & tricks
+
+* to delete tags for all files below `blog*` and rescan that, `sqlite3 .hist/up2k.db "delete from mt where w in (select substr(w,1,16) from up where rd like 'blog%')";`
@@ -4,13 +4,19 @@ import os
 import sys
 import tempfile
 import subprocess as sp
-import keyfinder
+
+try:
+    import keyfinder
+
+    PKF = True
+except:
+    PKF = False

 from copyparty.util import fsenc

 """
 dep: github/mixxxdj/libkeyfinder
-dep: pypi/keyfinder
+dep: pypi/keyfinder  -OR-  EvanPurkhiser/keyfinder-cli
 dep: ffmpeg
 """

@@ -35,7 +41,17 @@ def det(tf):
    ])
    # fmt: on

-    print(keyfinder.key(tf).camelot())
+    if PKF:
+        print(keyfinder.key(tf).camelot())
+    else:
+        # fmt: off
+        sp.check_call([
+            b"keyfinder-cli",
+            b"-n",
+            b"camelot",
+            fsenc(tf)
+        ])
+        # fmt: on


 def main():
@@ -0,0 +1,53 @@
+import json
+import re
+import sys
+
+from copyparty.util import fsenc, runcmd
+
+
+"""
+uses exiftool to geotag images based on embedded gps coordinates in exif data
+
+adds four new metadata keys:
+    .gps_lat = latitute
+    .gps_lon = longitude
+    .masl = meters above sea level
+    city = "city, subregion, region"
+
+usage: -mtp .masl,.gps_lat,.gps_lon,city=ad,t10,bin/mtag/geotag.py
+
+example: https://a.ocv.me/pub/blog/j7/8/?grid=0
+"""
+
+
+def main():
+    cmd = b"exiftool -api geolocation -n".split(b" ")
+    rc, so, se = runcmd(cmd + [fsenc(sys.argv[1])])
+    ptn = re.compile("([^:]*[^ :]) *: (.*)")
+    city = ["", "", ""]
+    ret = {}
+    for ln in so.split("\n"):
+        m = ptn.match(ln)
+        if not m:
+            continue
+        k, v = m.groups()
+        if k == "Geolocation City":
+            city[2] = v
+        elif k == "Geolocation Subregion":
+            city[1] = v
+        elif k == "Geolocation Region":
+            city[0] = v
+        elif k == "GPS Latitude":
+            ret[".gps_lat"] = "%.04f" % (float(v),)
+        elif k == "GPS Longitude":
+            ret[".gps_lon"] = "%.04f" % (float(v),)
+        elif k == "GPS Altitude":
+            ret[".masl"] = str(int(float(v)))
+    v = ", ".join(city).strip(", ")
+    if v:
+        ret["city"] = v
+    print(json.dumps(ret))
+
+
+if __name__ == "__main__":
+    main()
@@ -155,6 +155,11 @@ install_keyfinder() {
 		return
 	}

+	(cat /etc/alpine-release || echo a) 2>&1 | grep -E '3\.2[3-9]' && {
+		echo "alpine too new; ffmpeg8 is keyfinder-py incompat; giving up"
+		return
+	}
+
 	cd "$td"
 	github_tarball https://api.github.com/repos/mixxxdj/libkeyfinder/releases/latest
 	ls -al
@@ -189,7 +194,7 @@ install_keyfinder() {
 		exit 1
 	}

-	x=${-//[^x]/}; set -x; cat /etc/alpine-release
+	x=${-//[^x]/}; set -x; cat /etc/alpine-release || true
 	# rm -rf /Users/ed/Library/Python/3.9/lib/python/site-packages/*keyfinder*
 	CFLAGS="-I$h/pe/keyfinder/include -I/opt/local/include -I/usr/include/ffmpeg" \
 	CXXFLAGS="-I$h/pe/keyfinder/include -I/opt/local/include -I/usr/include/ffmpeg" \
@@ -6,8 +6,8 @@ __copyright__ = 2019
 __license__ = "MIT"
 __url__ = "https://github.com/9001/copyparty/"

-S_VERSION = "2.1"
-S_BUILD_DT = "2025-09-06"
+S_VERSION = "2.2"
+S_BUILD_DT = "2025-12-16"

 """
 mount a copyparty server (local or remote) as a filesystem
@@ -21,7 +21,7 @@ usage:
  python partyfuse.py http://192.168.1.69:3923/  ./music

 dependencies:
-  python3 -m pip install --user fusepy  # or grab it from the connect page
+  python3 -m pip install --user mfusepy  # or grab it from the connect page
  + on Linux: sudo apk add fuse
  + on Macos: https://osxfuse.github.io/
  + on Windows: https://github.com/billziss-gh/winfsp/releases/latest
@@ -92,8 +92,13 @@ is_dbg = False


 try:
-    from fuse import FUSE, FuseOSError, Operations
+    from mfusepy import FUSE, FuseOSError, Operations
 except:
+    try:
+        from fuse import FUSE, FuseOSError, Operations
+    except:
+        FUSE = None
+
    if WINDOWS:
        libfuse = "install https://github.com/billziss-gh/winfsp/releases/latest"
    elif MACOS:
@@ -102,12 +107,13 @@ except:
        libfuse = "apt install libfuse2\n    modprobe fuse"

    m = """\033[33m
-  could not import fuse; these may help:
-    {} -m pip install --user fusepy
+  could not import mfusepy; these may help:
+    {} -m pip install --user mfusepy
    {}
 \033[0m"""
-    print(m.format(sys.executable, libfuse))
-    raise
+    if not FUSE:
+        print(m.format(sys.executable, libfuse))
+        raise


 def termsafe(txt):
@@ -143,10 +149,10 @@ def fancy_log(fmt, *a):


 def register_wtf8():
-    def wtf8_enc(text):
+    def wtf8_enc(text, errors=""):
        return str(text).encode("utf-8", "surrogateescape"), len(text)

-    def wtf8_dec(binary):
+    def wtf8_dec(binary, errors=""):
        return bytes(binary).decode("utf-8", "surrogateescape"), len(binary)

    def wtf8_search(encoding_name):
@@ -284,8 +290,8 @@ class Gateway(object):
            if ar.td:
                self.ssl_context = ssl._create_unverified_context()
            elif ar.te:
-                self.ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS)
-                self.ssl_context.load_verify_locations(ar.te)
+                self.ssl_context = ssl.create_default_context(cafile=ar.te)
+                self.ssl_context.check_hostname = ar.teh

        self.conns = {}

@@ -620,7 +626,7 @@ class Gateway(object):
            "st_atime": ts,
            "st_mtime": ts,
            "st_ctime": ts,
-            "st_blocks": int((sz + 511) / 512),
+            "st_blocks": -int(-sz // 512),
        }

    def stat_file(self, ts, sz):
@@ -632,7 +638,7 @@ class Gateway(object):
            "st_atime": ts,
            "st_mtime": ts,
            "st_ctime": ts,
-            "st_blocks": int((sz + 511) / 512),
+            "st_blocks": -int(-sz // 512),
        }


@@ -1165,6 +1171,7 @@ NOTE: if server has --usernames enabled, then password is "username:password"

    ap2 = ap.add_argument_group("https/TLS")
    ap2.add_argument("-te", metavar="PEMFILE", help="certificate to expect/verify")
+    ap2.add_argument("-teh", action="store_true", help="require correct hostname in -te cert")
    ap2.add_argument("-td", action="store_true", help="disable certificate check")

    ap2 = ap.add_argument_group("cache/perf")
@@ -20,6 +20,8 @@ example:
 example for running straight from source (instead of using an sfx):
  PYTHONPATH=$PWD ./prisonparty.sh /var/lib/copyparty-jail cpp cpp /mnt/nas/music -- python3 -um copyparty -v /mnt/nas/music::rwmd

+NOT COMPATIBLE WITH: --th-bwrap
+
 note that if you have python modules installed as --user (such as bpm/key detectors),
  you should add /home/foo/.local as a VOLDIR

@@ -97,6 +99,7 @@ jail="${jail%/}"


 # bind-mount system directories and volumes
+mkdir -p "$jail"
 for a in {1..30}; do mkdir "$jail/.prisonlock" && break; sleep 0.1; done
 printf '%s\n' "${sysdirs[@]}" "${vols[@]}" | sed -r 's`/$``' | LC_ALL=C sort | uniq |
 while IFS= read -r v; do
@@ -0,0 +1,55 @@
+# thumbnail extractors
+
+extract (or generate) thumbnails for custom file formats or override default thumbnail extraction for standard media files
+
+
+## usage
+
+* load plugins with `--th-extract baz=~/dev/copyparty/bin/thumbs/randomcolor.py`
+  * `baz` is the file extension (may be comma-separated list of extensions)
+  * add multiple different extractor plugins by repeating the `--th-extract` argument
+
+
+## api
+
+### in
+
+each plugin must define a function `main(abspath, **kwargs)`:
+* `abspath` is the path to a file to extract thumbnail for
+* `kwargs` currently receives these keyword arguments:
+  * `vn` is the VFS which contains the requested file
+  * `th_srv` is an instance of [copyparty/th_srv](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/th_srv.py)
+
+### out
+
+the `main()` function must
+* return `None` or raise an `Exception` in case it is unable to extract thumbnail
+* return tuple `fmt, stream, offset, whence, size` otherwise:
+  * `fmt: str` – format (filename extension) of the thumbnail; should be one of the image formats supported by copyparty (see `--th-r-*` options)
+  * `stream: IO[bytes]` – binary file-like object supporting `seek()`, `read()`, `close()` methods; should contain extracted thumbnail image
+  * `offset: int, whence: int` – arguments to `stream.seek()`; copyparty will call `stream.seek(offset, whence)` once before starting reading the stream contents
+  * `size: int | None` – number of bytes to read from the `stream`; if `size` is negative or `None`, copyparty will read until the end of the stream
+
+
+### notes
+
+it is possible to extract or generate custom thumbnails for standard media files too,
+e.g. run copyparty with `--th-extract mp3,m4a,aac,flac,opus=~/dev/copyparty/bin/thumbs/my_custom_mp3_th_extractor.py` and do your magic with audio file thumbnails
+
+in case copyparty doesn't like the `fmt` option, it will `close()` the `stream` without ever calling `seek()` or `read()`
+
+if your scenario involves heavy computation or i/o, it is better to defer the heavy parts until the first `seek()` or `read()` to avoid redundant work in case the result gets rejected based on `fmt` value
+
+if you provide custom `stream` object:
+copyparty performs buffered reading, so expect multiple `read()` calls, respect `size` argument in `read(size)` calls, and maintain correct seek position
+
+
+## examples
+
+* [randomcolor](https://github.com/9001/copyparty/blob/hovudstraum/bin/thumbs/randomcolor.py) generates a random .gif image
+* [collabora](https://github.com/9001/copyparty/blob/hovudstraum/bin/thumbs/collabora.py) uses external self-hosted service (collabora online) to generate previews for various office documents, pdf and fb2 books, html, markdown, text files and some image formats; collabora is also supported as office editor in copyparty, see [wopi](https://github.com/9001/copyparty#wopi-server)
+
+
+## some other known plugins seen on the internets
+
+* [fpkg_thumb](https://github.com/kamaeff/fpkg-vault/blob/1c248de62a9ff9c4088d4b3e7962023cde9d6426/old/fpkg_thumb.py) extracts cover images from playstation4 software installation packages ("pkg" and "fpkg" files)
@@ -0,0 +1,125 @@
+"""
+CAUTION!
+looks like collabora spawns separate jailed libreoffice instance to do the conversion
+this may eat up too much ram in folders with many docs or with --th-pregen enabled
+
+consider limiting resources available for collabora
+e.g. for me these lines in docker-compose.yml helped alot:
+    deploy:
+      resources:
+        limits:
+          cpus: '2'
+          memory: '2G'
+/CAUTION!
+
+if you use collabora office editor with copyparty, you already have a thumbnailer for office formats:
+https://sdk.collaboraonline.com/docs/conversion_api.html
+the 'convert-to' api accepts files in multipart/form-data format
+
+just add this extractor to copyparty with an argument like this:
+  --th-extract=docx,xlsx,pptx,odt,ods,pdf,fb2=~/dev/copyparty/bin/thumbs/collabora.py
+and set COLLABORA_INSTANCE_URL below to your collabora instace url
+
+full list of supported file types:
+https://sdk.collaboraonline.com/docs/conversion_api.html#supported-input-formats
+
+this is also an example of deferred IO:
+no files opened and no network requests made if copyparty doesn't like the file type
+"""
+
+import os
+import string
+import random
+import mimetypes
+from urllib.error import HTTPError
+from urllib.request import Request, urlopen
+
+COLLABORA_INSTANCE_URL = "http://collabora:9980"
+BOUNDARY_CHARS = string.ascii_letters + string.digits
+
+
+def main(abspath, **kwargs):
+    boundary = "CopypartyFormData" + "".join(random.choices(BOUNDARY_CHARS, k=53))
+    start, end = make_headers(abspath, boundary)
+    size = len(start) + os.path.getsize(abspath) + len(end)
+    request = Request(
+        COLLABORA_INSTANCE_URL + "/cool/convert-to/png",
+        method="POST",
+        data=iterate(start, abspath, end),
+        headers={
+            "Content-Type": f"multipart/form-data; boundary={boundary}",
+            "Content-Length": str(size),
+        },
+    )
+
+    file_like = Wrapper(request)
+    offset, whence, length = 0, 0, None
+
+    return "png", file_like, offset, whence, length
+
+
+def make_headers(abspath, boundary):
+    """stuff to wrap file contents by for multipart"""
+
+    fname = "file." + abspath.rsplit(".", 1)[-1]
+    ftype = mimetypes.guess_file_type(fname)[0] or "application/octet-stream"
+    start = (
+        f"--{boundary}\r\n"
+        f'Content-Disposition: form-data; name="data"; filename="{fname}"\r\n'
+        f"Content-Type: {ftype}\r\n\r\n"
+    )
+    end = f"\r\n--{boundary}--"
+    return bytes(start, "ascii"), bytes(end, "ascii")
+
+
+def iterate(start, fpath, end):
+    """docs may be big so read and send them in chunks"""
+
+    yield start
+    with open(fpath, "rb") as f:
+        while True:
+            b = f.read(32768)
+            if not b:
+                break
+            yield b
+    yield end
+
+
+class Wrapper:
+    """
+    This file-like wrapper defers expensive work until first .seek() call
+    Useful  to not waste resources in case copyparty decides
+    to throw our thumbnail away based on returned file extension
+    """
+
+    def __init__(self, request):
+        self.request = request
+        self.response = None
+
+    def seek(self, offset, whence=0):
+        if self.response:
+            return 0
+
+        try:
+            # the heavy part:
+            # - reading probably big file
+            # - sending it over the network
+            # - making collabora do some work
+            self.response = urlopen(self.request, timeout=10)
+        except HTTPError as e:
+            e.close()
+            raise e
+        except Exception as e:
+            self.close()
+            raise e
+
+        # actual response is not seekable
+        # just return 0
+        return 0
+
+    def read(self, amount):
+        return self.response.read(amount)
+
+    def close(self):
+        if self.response:
+            self.response.close()
@@ -0,0 +1,29 @@
+import io
+import os
+import struct
+from base64 import b64decode
+
+
+def create_gif() -> bytes:
+    head = r"R0lGODdhIAAgAIAB"
+    tail = r"AAD/LAAAAAAgACAAAAIehI+py+0Po5y02ouz3rz7D4biSJbmiabqyrbuC5sFADs="
+    #color = struct.pack(">L", random.randrange(0xffffff))  # wow, way too stable
+    color = b"\x00" + os.urandom(3)
+    return b64decode(head) + color + b64decode(tail)
+
+
+def main(abspath, **ka) -> Optional[tuple[str, IO[bytes], int, int, Optional[int]]]:
+    print("hello from randomcolor.py pretending to extract a thumb from " + abspath)
+
+    # need to return a file-like object
+    gif_bytes = create_gif()
+    file_obj = io.BytesIO(gif_bytes)
+
+    # where copyparty should seek to in the "file" to start reading from
+    seek_offset = 0
+    seek_whence = 0
+
+    filesize = None  # auto; read until end
+
+    # "gif" is the file-extension to decode the file as
+    return "gif", file_obj, seek_offset, seek_whence, filesize
@@ -1,8 +1,8 @@
 #!/usr/bin/env python3
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

-S_VERSION = "2.13"
-S_BUILD_DT = "2025-09-05"
+S_VERSION = "2.23"
+S_BUILD_DT = "2026-09-06"

 """
 u2c.py: upload to copyparty
@@ -100,7 +100,7 @@ except:
    ub64enc = base64.urlsafe_b64encode


-class BadAuth(Exception):
+class Fatal(Exception):
    pass


@@ -165,8 +165,8 @@ class HCli(object):
            elif self.verify is True:
                self.ctx = None
            else:
-                self.ctx = ssl.SSLContext(ssl.PROTOCOL_TLS)
-                self.ctx.load_verify_locations(self.verify)
+                self.ctx = ssl.create_default_context(cafile=self.verify)
+                self.ctx.check_hostname = ar.teh

        self.base_hdrs = {
            "Accept": "*/*",
@@ -196,6 +196,8 @@ class HCli(object):
        hdrs.update(self.base_hdrs)
        if self.ar.a:
            hdrs["PW"] = self.ar.a
+        if self.ar.ba:
+            hdrs["Authorization"] = self.ar.ba
        if ctype:
            hdrs["Content-Type"] = ctype
        if meth == "POST" and CLEN not in hdrs:
@@ -232,6 +234,7 @@ class HCli(object):

 MJ = "application/json"
 MO = "application/octet-stream"
+MM = "application/x-www-form-urlencoded"
 CLEN = "Content-Length"

 web = None  # type: HCli
@@ -395,7 +398,7 @@ class MTHash(object):
            self.csz = chunksz

            chunks = {}
-            nchunks = int(math.ceil(fsz / chunksz))
+            nchunks = -int(-fsz // chunksz)
            for nch in range(nchunks):
                self.work_q.put(nch)

@@ -491,6 +494,12 @@ print = safe_print if VT100 else flushing_print


 def termsize():
+    try:
+        w, h = os.get_terminal_size()
+        return w, h
+    except:
+        pass
+
    env = os.environ

    def ioctl_GWINSZ(fd):
@@ -716,7 +725,7 @@ def up2k_chunksize(filesize):
    stepsize = 512 * 1024
    while True:
        for mul in [1, 2]:
-            nchunks = math.ceil(filesize * 1.0 / chunksize)
+            nchunks = -int(-filesize // chunksize)
            if nchunks <= 256 or (chunksize >= 32 * 1024 * 1024 and nchunks <= 4096):
                return chunksize

@@ -724,16 +733,22 @@ def up2k_chunksize(filesize):
            stepsize *= mul


-# mostly from copyparty/up2k.py
 def get_hashlist(file, pcb, mth):
    # type: (File, Any, Any) -> None
+    with open(file.abs, "rb", 512 * 1024) as f:
+        get_hashlist_f(file, pcb, mth, f)
+
+
+# mostly from copyparty/up2k.py
+def get_hashlist_f(file, pcb, mth, f):
+    # type: (File, Any, Any, Any) -> None
    """generates the up2k hashlist from file contents, inserts it into `file`"""

    chunk_sz = up2k_chunksize(file.size)
    file_rem = file.size
    file_ofs = 0
    ret = []
-    with open(file.abs, "rb", 512 * 1024) as f:
+    if True:
        t0 = time.time()

        if mth and file.size >= 1024 * 512:
@@ -769,6 +784,18 @@ def get_hashlist(file, pcb, mth):
            file.kchunks[k] = [v1, v2]


+def wark_stdin(ar):
+    file = File(b"", b"", int(ar.files[0]), 0)
+    get_hashlist_f(file, None, False, sys.stdin if PY2 else sys.stdin.buffer)
+    if ar.chs:
+        zsl = ["%s %d %d" % (zsii[0], n, zsii[1]) for n, zsii in enumerate(file.cids)]
+        print("chs:\n" + "\n".join(zsl))
+    zsl = [ar.wsalt, str(file.size)] + [x[0] for x in file.cids]
+    zb = hashlib.sha512("\n".join(zsl).encode("utf-8")).digest()[:33]
+    wark = ub64enc(zb).decode("utf-8")
+    print(wark)
+
+
 def printlink(ar, purl, name, fk):
    if not name:
        url = purl  # srch
@@ -826,10 +853,15 @@ def handshake(ar, file, search):
            url = ""
        url = ar.vtop + url

+    t0 = time.time()
+    tmax = t0 + ar.t_hs
    while True:
        sc = 600
        txt = ""
-        t0 = time.time()
+        t1 = time.time()
+        if t1 >= tmax:
+            print("\nERROR: server offline for longer than --t-hs; giving up")
+            raise Fatal()
        try:
            zs = json.dumps(req, separators=(",\n", ": "))
            sc, txt = web.req("POST", url, {}, zs.encode("utf-8"), MJ)
@@ -850,13 +882,13 @@ def handshake(ar, file, search):
                return [], False
            elif sc == 409 or "<pre>upload rejected, file already exists" in txt:
                return [], False
-            elif sc == 403:
+            elif sc == 403 or sc == 401:
                print("\nERROR: login required, or wrong password:\n%s" % (txt,))
-                raise BadAuth()
+                raise Fatal()

-            t = "handshake failed, retrying: %s\n  t0=%.3f t1=%.3f td=%.3f\n  %s\n\n"
+            t = "handshake failed, retrying: %s\n  t0=%.3f t1=%.3f t2=%.3f td1=%.3f td2=%.3f\n  %s\n\n"
            now = time.time()
-            eprint(t % (file.name, t0, now, now - t0, em))
+            eprint(t % (file.name, t0, t1, now, now - t0, now - t1, em))
            time.sleep(ar.cd)

    try:
@@ -979,6 +1011,7 @@ class Ctl(object):
        self.nfiles, self.nbytes = self.stats
        self.filegen = walkdirs([], ar.files, ar.x)
        self.recheck = []  # type: list[File]
+        self.last_file = None

        if ar.safe:
            self._safe()
@@ -1015,6 +1048,11 @@ class Ctl(object):

            self._fancy()

+        file = self.last_file
+        if self.up_br and file:
+            zs = quotep(file.name.encode("utf-8", WTF8))
+            web.req("POST", file.url, {}, b"msg=upload-queue-empty;" + zs, MM)
+
        self.ok = not self.errs

    def _safe(self):
@@ -1036,7 +1074,7 @@ class Ctl(object):
                print("  hs...")
                try:
                    hs, _ = handshake(self.ar, file, search)
-                except BadAuth:
+                except Fatal:
                    sys.exit(1)

                if search:
@@ -1164,7 +1202,7 @@ class Ctl(object):
            handshake(self.ar, file, False)

    def cleanup_vt100(self):
-        if VT100:
+        if VT100 and not self.ar.ns:
            ss.scroll_region(None)
        else:
            eprint("\033]9;4;0\033\\")
@@ -1225,9 +1263,7 @@ class Ctl(object):
                            while req:
                                print("DELETING ~%s#%s" % (srd, len(req)))
                                body = json.dumps(req).encode("utf-8")
-                                sc, txt = web.req(
-                                    "POST", self.ar.url + "?delete", {}, body, MJ
-                                )
+                                sc, txt = web.req("POST", "/?delete", {}, body, MJ)
                                if sc == 413 and "json 2big" in txt:
                                    print(" (delete request too big; slicing...)")
                                    req = req[: len(req) // 2]
@@ -1293,7 +1329,8 @@ class Ctl(object):
                if self.ar.jw:
                    print("%s  %s" % (wark, vp))
                else:
-                    zd = datetime.datetime.fromtimestamp(max(0, file.lmod), UTC)
+                    tsdt = datetime.datetime.fromtimestamp
+                    zd = tsdt(max(0, min(2 << 36, file.lmod)), UTC)
                    dt = "%04d-%02d-%02d %02d:%02d:%02d" % (
                        zd.year,
                        zd.month,
@@ -1343,7 +1380,7 @@ class Ctl(object):

            try:
                hs, sprs = handshake(self.ar, file, search)
-            except BadAuth:
+            except Fatal:
                self.panik = 1
                break

@@ -1455,6 +1492,7 @@ class Ctl(object):

            file = fsl.file
            cids = fsl.cids
+            self.last_file = file

            with self.mutex:
                if not self.uploader_busy:
@@ -1538,7 +1576,8 @@ NOTE: if server has --usernames enabled, then password is "username:password"
    ap.add_argument("url", type=unicode, help="server url, including destination folder")
    ap.add_argument("files", type=files_decoder, nargs="+", help="files and/or folders to process")
    ap.add_argument("-v", action="store_true", help="verbose")
-    ap.add_argument("-a", metavar="PASSWD", help="password or $filepath")
+    ap.add_argument("-a", metavar="PASSWD", default="", help="password (or $filepath) for copyparty (is sent in header 'PW'), default is env.var U2C_PW")
+    ap.add_argument("--ba", metavar="PASSWD", default="", help="password (or $filepath) for basic-auth (usually not necessary)")
    ap.add_argument("-s", action="store_true", help="file-search (disables upload)")
    ap.add_argument("-x", type=unicode, metavar="REGEX", action="append", help="skip file if filesystem-abspath matches REGEX (option can be repeated), example: '.*/\\.hist/.*'")
    ap.add_argument("--ok", action="store_true", help="continue even if some local files are inaccessible")
@@ -1566,6 +1605,7 @@ NOTE: if server has --usernames enabled, then password is "username:password"
    ap.add_argument("--wsalt", type=unicode, metavar="S", default="hunter2", help="salt to use when creating warks; must match server config")
    ap.add_argument("--chs", action="store_true", help="verbose (print the hash/offset of each chunk in each file)")
    ap.add_argument("--jw", action="store_true", help="just identifier+filepath, not mtime/size too")
+    ap.add_argument("--stdin", action="store_true", help="calculate file-ID of stdin; u2c.py --stdin - $LEN < a.mkv")

    ap = app.add_argument_group("performance tweaks")
    ap.add_argument("-j", type=int, metavar="CONNS", default=2, help="parallel connections")
@@ -1576,11 +1616,13 @@ NOTE: if server has --usernames enabled, then password is "username:password"
    ap.add_argument("-ns", action="store_true", help="no status panel (for slow consoles and macos)")
    ap.add_argument("--cxp", type=float, metavar="SEC", default=57, help="assume http connections expired after SEConds")
    ap.add_argument("--cd", type=float, metavar="SEC", default=5, help="delay before reattempting a failed handshake/upload")
+    ap.add_argument("--t-hs", type=float, metavar="SEC", default=186, help="crash if handshakes fail due to server-offline for this long")
    ap.add_argument("--safe", action="store_true", help="use simple fallback approach")
    ap.add_argument("-z", action="store_true", help="ZOOMIN' (skip uploading files if they exist at the destination with the ~same last-modified timestamp, so same as yolo / turbo with date-chk but even faster)")

    ap = app.add_argument_group("tls")
    ap.add_argument("-te", metavar="PATH", help="path to ca.pem or cert.pem to expect/verify")
+    ap.add_argument("-teh", action="store_true", help="require correct hostname in -te cert")
    ap.add_argument("-td", action="store_true", help="disable certificate check")
    # fmt: on

@@ -1594,6 +1636,10 @@ NOTE: if server has --usernames enabled, then password is "username:password"
            except:
                pass

+    if ar.stdin:
+        wark_stdin(ar)
+        return
+
    # msys2 doesn't uncygpath absolute paths with whitespace
    if not VT100:
        zsl = []
@@ -1676,11 +1722,18 @@ NOTE: if server has --usernames enabled, then password is "username:password"
            print("\n\n   %s\n\n" % (t,))
            raise

-    if ar.a and ar.a.startswith("$"):
-        fn = ar.a[1:]
-        print("reading password from file [{0}]".format(fn))
-        with open(fn, "rb") as f:
-            ar.a = f.read().decode("utf-8").strip()
+    for k in ("a", "ba"):
+        zs = getattr(ar, k)
+        if zs.startswith("$"):
+            print("reading password from file [%s]" % (zs[1:],))
+            with open(zs[1:], "rb") as f:
+                setattr(ar, k, f.read().decode("utf-8").strip())
+
+    if not ar.a:
+        ar.a = os.environ.get("U2C_PW", "")
+
+    if ar.ba:
+        ar.ba = "Basic " + base64.b64encode(ar.ba.encode("utf-8")).decode("utf-8")

    for n in range(ar.rh):
        try:
@@ -61,6 +61,8 @@ def rep_server():
        print("copyparty says %r" % (sck.recv_string(),))
        reply = b"thx"
        # reply = b"return 1"  # non-zero to block an upload
+        # reply = b'{"rc":1}'  # or as json, that's fine too
+        # reply = b'{"rejectmsg":"naw dude"}'  # or custom message
        sck.send(reply)


@@ -17,8 +17,12 @@ defaults

 listen foo1
  bind *:8081
+  http-request add-header X-Forwarded-Proto https if { ssl_fc }
+  http-request add-header X-Forwarded-Proto http unless { ssl_fc }
  server srv1 127.0.0.1:3923 maxconn 512

 listen foo2
  bind *:8082
+  http-request add-header X-Forwarded-Proto https if { ssl_fc }
+  http-request add-header X-Forwarded-Proto http unless { ssl_fc }
  server srv1 /dev/shm/party.sock maxconn 512
@@ -6,7 +6,7 @@
 #
 # to reverse-proxy a specific path/subpath/location below a domain
 # (rather than a complete subdomain), for example "/qw/er", you must
-# run copyparty with --rp-loc /qw/as and also change the following:
+# run copyparty with --rp-loc /qw/er and also change the following:
 # 	location / {
 # 		proxy_pass http://cpp_tcp;
 # to this:
@@ -75,9 +75,8 @@ server {
 		# recommendation: replace cpp_tcp with cpp_uds below
 		proxy_pass http://cpp_tcp;
 		proxy_redirect off;
-		# disable buffering (next 4 lines)
+		# disable buffering (next 3 lines)
 		proxy_http_version 1.1;
-		client_max_body_size 0;
 		proxy_buffering off;
 		proxy_request_buffering off;
 		# improve download speed from 600 to 1500 MiB/s
@@ -48,6 +48,8 @@ let

  accountsWithPlaceholders = mapAttrs (name: attrs: passwordPlaceholder name);

+  volumesWithoutVariables = filterAttrs (k: v: !(hasInfix "\${" v.path)) cfg.volumes;
+
  configStr = ''
    ${mkSection "global" cfg.settings}
    ${cfg.globalExtraConfig}
@@ -67,11 +69,8 @@ in
  options.services.copyparty = {
    enable = mkEnableOption "web-based file manager";

-    package = mkOption {
-      type = types.package;
-      default = pkgs.copyparty;
-      defaultText = "pkgs.copyparty";
-      description = ''
+    package = mkPackageOption pkgs "copyparty" {
+      extraDescription = ''
        Package of the application to run, exposed for overriding purposes.
      '';
    };
@@ -80,7 +79,7 @@ in
      type = types.bool;
      default = true;
      description = ''
-        Make a shell script wrapper called 'copyparty-hash' with all options set here,
+        Make a shell script wrapper called {command}`copyparty-hash` with all options set here,
        that launches the hashing cli.
      '';
    };
@@ -115,9 +114,9 @@ in
      type = types.attrs;
      description = ''
        Global settings to apply.
-        Directly maps to values in the [global] section of the copyparty config.
+        Directly maps to values in the `[global]` section of the copyparty config.
        Cannot set "c" or "hist", those are set by this module.
-        See `${getExe cfg.package} --help` for more details.
+        See {command}`copyparty --help` for more details.
      '';
      default = {
        i = "127.0.0.1";
@@ -136,7 +135,7 @@ in
    globalExtraConfig = mkOption {
      type = types.str;
      default = "";
-      description = "Appended to the end of the [global] section verbatim. This is useful for flags which are used in a repeating manner (e.g. ipu: 255.255.255.1=user) which can't be repeated in the settings = {} attribute set.";
+      description = "Appended to the end of the `[global]` section verbatim. This is useful for flags which are used in a repeating manner (e.g. `ipu: 255.255.255.1=user`) which can't be repeated in the settings = {} attribute set.";
    };

    accounts = mkOption {
@@ -199,21 +198,21 @@ in
                  Attribute list of permissions and the users to apply them to.

                  The key must be a string containing any combination of allowed permission:
-                    "r" (read):   list folder contents, download files
-                    "w" (write):  upload files; need "r" to see the uploads
-                    "m" (move):   move files and folders; need "w" at destination
-                    "d" (delete): permanently delete files and folders
-                    "g" (get):    download files, but cannot see folder contents
-                    "G" (upget):  "get", but can see filekeys of their own uploads
-                    "h" (html):   "get", but folders return their index.html
-                    "a" (admin):  can see uploader IPs, config-reload
+                  * "r" (read):   list folder contents, download files
+                  * "w" (write):  upload files; need "r" to see the uploads
+                  * "m" (move):   move files and folders; need "w" at destination
+                  * "d" (delete): permanently delete files and folders
+                  * "g" (get):    download files, but cannot see folder contents
+                  * "G" (upget):  "get", but can see filekeys of their own uploads
+                  * "h" (html):   "get", but folders return their index.html
+                  * "a" (admin):  can see uploader IPs, config-reload

                  For example: "rwmd"

                  The value must be one of:
-                    an account name, defined in `accounts`
-                    a list of account names
-                    "*", which means "any account"
+                  * an account name, defined in `accounts`
+                  * a list of account names
+                  * "*", which means "any account"
                '';
                example = literalExpression ''
                  {
@@ -228,7 +227,7 @@ in
                type = types.attrs;
                description = ''
                  Attribute list of volume flags to apply.
-                  See `${getExe cfg.package} --help-flags` for more details.
+                  See {command}`copyparty --help-flags` for more details.
                '';
                example = literalExpression ''
                  {
@@ -286,6 +285,7 @@ in

        environment = {
          PYTHONUNBUFFERED = "true";
+          NIX_SSL_CERT_FILE = "${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt";
          XDG_CONFIG_HOME = externalStateDir;
        };

@@ -325,7 +325,7 @@ in
          BindPaths =
            (if cfg.settings ? hist then [ cfg.settings.hist ] else [ ])
            ++ [ externalStateDir ]
-            ++ (mapAttrsToList (k: v: v.path) cfg.volumes);
+            ++ (mapAttrsToList (k: v: v.path) volumesWithoutVariables);
          # ProtectSystem = "strict";
          # Note that unlike what 'ro' implies,
          # this actually makes it impossible to read anything in the root FS,
@@ -364,10 +364,20 @@ in
              #: in front of things means it wont change it if the directory already exists.
              group = ":${cfg.group}";
              user = ":${cfg.user}";
-              mode = ":755";
+              mode = ":${
+                # Use volume permissions if set
+                if (value.flags ? chmod_d) then
+                  value.flags.chmod_d
+                # Else, use global permission if set
+                else if (cfg.settings ? chmod-d) then
+                  cfg.settings.chmod-d
+                # Else, use the default permission
+                else
+                  "755"
+              }";
            };
          }
-        ) cfg.volumes
+        ) volumesWithoutVariables
      );

      users.groups = lib.mkIf (cfg.group == "copyparty") {
@@ -9,10 +9,19 @@
 # you may want to:
 #   change '/usr/bin/python' to another interpreter
 #   change '/mnt::rw' to another location or permission-set
+#   use a config file instead of command arguments, e.g.:
+#      command_args="-c /etc/copyparty.conf"

 name="$SVCNAME"
 command_background=true
+extra_commands="checkconfig"
 pidfile="/var/run/$SVCNAME.pid"

 command="/usr/bin/python3 /usr/local/bin/copyparty-sfx.py"
 command_args="-q -v /mnt::rw"
+
+checkconfig() {
+        ebegin "Checking $RC_SVCNAME configuration"
+        $command $command_args --exit cfg
+        eend $?
+}
@@ -3,9 +3,9 @@
 # NOTE: You generally shouldn't use this PKGBUILD on Arch, as it is mainly for testing purposes. Install copyparty using pacman instead.

 pkgname=copyparty
-pkgver="1.19.15"
+pkgver="1.20.22"
 pkgrel=1
-pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
+pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++"
 arch=("any")
 url="https://github.com/9001/${pkgname}"
 license=('MIT')
@@ -13,22 +13,22 @@ depends=("bash" "python" "lsof" "python-jinja")
 makedepends=("python-wheel" "python-setuptools" "python-build" "python-installer" "make" "pigz")
 optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tags"
            "cfssl: generate TLS certificates on startup"
-            "python-mutagen: music tags (alternative)"
+            "python-mutagen: music tags (fast/inaccurate)"
            "python-pillow: thumbnails for images"
-            "python-pyvips: thumbnails for images (higher quality, faster, uses more ram)"
+            "python-pyvips: thumbnails for images (eats ram, prefer pillow)"
+            "python-paramiko: sftp server"
+            "python-pyftpdlib: ftp server"
+            "python-pyopenssl: ftps server (needs pyftpdlib too)"
            "libkeyfinder: detection of musical keys"
-            "python-pyopenssl: ftps functionality"
            "python-pyzmq: send zeromq messages from event-hooks"
            "python-argon2-cffi: hashed passwords in config"
 )
 source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
-backup=("etc/${pkgname}/copyparty.conf" )
-sha256sums=("95f4c1e517acaadd5fdc38b47047f16a7cd51495bd94ae9182274f256db59834")
+backup=("etc/copyparty.conf")
+install=copyparty.install
+sha256sums=("14d590644943c3ed14e21093cce79e06f1ae206adeeda16032d15a7241ff4344")

 build() {
-    cd "${srcdir}/${pkgname}-${pkgver}/copyparty/web"
-    make
-
    cd "${srcdir}/${pkgname}-${pkgver}"
    python -m build --wheel --no-isolation
 }
@@ -37,12 +37,11 @@ package() {
    cd "${srcdir}/${pkgname}-${pkgver}"
    python -m installer --destdir="$pkgdir" dist/*.whl

-    install -dm755 "${pkgdir}/etc/${pkgname}"
    install -Dm755 "bin/prisonparty.sh" "${pkgdir}/usr/bin/prisonparty"
-    install -Dm644 "contrib/systemd/${pkgname}.conf" "${pkgdir}/etc/${pkgname}/copyparty.conf"
+    install -Dm644 "contrib/systemd/${pkgname}.conf" "${pkgdir}/etc/copyparty.conf"
    install -Dm644 "contrib/systemd/${pkgname}@.service" "${pkgdir}/usr/lib/systemd/system/${pkgname}@.service"
    install -Dm644 "contrib/systemd/${pkgname}-user.service" "${pkgdir}/usr/lib/systemd/user/${pkgname}.service"
    install -Dm644 "contrib/systemd/prisonparty@.service" "${pkgdir}/usr/lib/systemd/system/prisonparty@.service"
-    install -Dm644 "contrib/systemd/index.md" "${pkgdir}/var/lib/${pkgname}-jail/README.md"
+    install -Dm644 "contrib/systemd/index.md" "${pkgdir}/var/lib/${pkgname}-jail/srv/README.md"
    install -Dm644 "LICENSE" "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE"
 }
@@ -0,0 +1,27 @@
+post_upgrade() {
+
+    # v1.20.22 moved the config-file, affecting the system-service only
+    if [[ $(vercmp "$2" 1.20.22) -lt 0 ]]; then
+
+        oldpath="/etc/copyparty/copyparty.conf"
+        newpath="/etc/copyparty.conf"
+
+        echo "==> Migrating config-file from $oldpath to $newpath"
+
+        if [[ ! -f "$oldpath" ]]; then
+            oldpath="$oldpath.pacsave"
+        fi
+
+        if [[ -f "$newpath.pacnew" ]]; then
+            echo "==> WARNING: A config-file already existed in the new location [$newpath]. The existing file will be used as-is."
+
+        elif [[ -f "$oldpath" ]]; then
+            mv -v "$oldpath" "$newpath"
+            rmdir /etc/copyparty 2>/dev/null || true
+        fi
+
+        systemctl cat copyparty@.service | awk '/=PRTY_CONFIG=/ && !/=\/etc\/copyparty.conf/ {exit 1}' || {
+            echo "==> WARNING: The copyparty system-service has been edited by hand since installation, so it will not read config from the new default config path [$newpath]. Please use PRTY_CONFIG=$newpath if that is correct for your setup."
+        }
+    fi
+}
@@ -2,9 +2,9 @@


 pkgname=copyparty
-pkgver=1.19.15
+pkgver=1.20.22
 pkgrel=1
-pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
+pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++"
 arch=("any")
 url="https://github.com/9001/${pkgname}"
 license=('MIT')
@@ -13,6 +13,7 @@ makedepends=("python3-wheel" "python3-setuptools" "python3-build" "python3-insta
 optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tags"
            "golang-cfssl: generate TLS certificates on startup"
            "python3-mutagen: music tags (alternative)"
+            "python3-paramiko: sftp server"
            "python3-pil: thumbnails for images"
            "python3-openssl: ftps functionality"
            "python3-zmq: send zeromq messages from event-hooks"
@@ -20,12 +21,9 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
 )
 source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
 backup=("/etc/${pkgname}.d/init" )
-sha256sums=("95f4c1e517acaadd5fdc38b47047f16a7cd51495bd94ae9182274f256db59834")
+sha256sums=("14d590644943c3ed14e21093cce79e06f1ae206adeeda16032d15a7241ff4344")

 build() {
-    cd "${srcdir}/${pkgname}-${pkgver}/copyparty/web"
-    make
-
    cd "${srcdir}/${pkgname}-${pkgver}"
    python -m build --wheel --no-isolation
 }
@@ -15,7 +15,7 @@ Description=copyparty file server
 Type=notify
 SyslogIdentifier=copyparty
 Environment=PYTHONUNBUFFERED=x
-WorkingDirectory=/var/lib/copyparty-jail
+WorkingDirectory=/var/lib/copyparty-jail/srv
 ExecReload=/bin/kill -s USR1 $MAINPID

 # user to run as + where the TLS certificate is (if any)
@@ -1,3 +1,13 @@
-this is `/var/lib/copyparty-jail`, the fallback webroot when copyparty has not yet been configured
+this is `/var/lib/copyparty-jail/srv/`, the fallback webroot when copyparty has not yet been configured

-please add some `*.conf` files to `/etc/copyparty.d/`
+please edit the config-file and restart copyparty;
+* if running as a system service: `vim /etc/copyparty.conf`
+* if running as a user service: `vim $HOME/.config/copyparty/copyparty.conf`
+
+some inspiration:
+* a basic example config: https://github.com/9001/copyparty/blob/hovudstraum/contrib/systemd/copyparty.example.conf
+* another example with focus on syntax: https://github.com/9001/copyparty/blob/hovudstraum/docs/example.conf
+* and CTRL-F `config file example` here: https://github.com/9001/copyparty
+
+the full list of configuration options can be seen at https://copyparty.eu/cli/ 
+or by running `copyparty --help`
@@ -17,7 +17,7 @@ Description=copyparty file server
 [Service]
 SyslogIdentifier=prisonparty
 Environment=PYTHONUNBUFFERED=x
-WorkingDirectory=/var/lib/copyparty-jail
+WorkingDirectory=/var/lib/copyparty-jail/srv
 ExecReload=/bin/kill -s USR1 $MAINPID

 # stop systemd-tmpfiles-clean.timer from deleting copyparty while it's running
@@ -15,6 +15,7 @@
  pyzmq,
  ffmpeg,
  mutagen,
+  paramiko,
  pyftpdlib,
  magic,
  partftpy,
@@ -29,7 +30,7 @@
  # create thumbnails with Pillow; faster than FFmpeg / MediaProcessing
  withThumbnails ? true,

-  # create thumbnails with PyVIPS; even faster, uses more memory
+  # create thumbnails with PyVIPS; uses more memory, pillow is preferred
  # -- can be combined with Pillow to support more filetypes
  withFastThumbnails ? false,

@@ -44,6 +45,9 @@
  # send ZeroMQ messages from event-hooks
  withZeroMQ ? true,

+  # enable SFTP server
+  withSFTP ? false,
+
  # enable FTP server
  withFTP ? true,

@@ -67,24 +71,71 @@
  # additional dependencies
  extraPythonPackages ? (_p: [ ]),

+  # to build stable + unstable with the same file
+  stable ? true,
+
+  # for commit date, only used when stable = false
+  copypartyFlake ? null,
+
+  nix-gitignore,
 }:

 let
  pinData = lib.importJSON ./pin.json;
  runtimeDeps = ([ util-linux ] ++ extraPackages ++ lib.optional withMediaProcessing ffmpeg);
+  inherit (copypartyFlake) lastModifiedDate;
+  # ex: "1970" "01" "01"
+  dateStringsZeroPrefixed = {
+    year = builtins.substring 0 4 lastModifiedDate;
+    month = builtins.substring 4 2 lastModifiedDate;
+    day = builtins.substring 6 2 lastModifiedDate;
+  };
+  # ex: "1970" "1" "1"
+  dateStringsShort = builtins.mapAttrs (_: val: toString (lib.toIntBase10 val)) dateStringsZeroPrefixed;
+  unstableVersion =
+    if copypartyFlake == null then
+      "${pinData.version}-unstable"
+    else
+      with dateStringsZeroPrefixed; "${pinData.version}-unstable-${year}-${month}-${day}"
+  ;
+  version = if stable then pinData.version else unstableVersion;
+  stableSrc = fetchurl {
+    inherit (pinData) url hash;
+  };
+  root = ../../../..;
+  unstableSrc = nix-gitignore.gitignoreSource [] root;
+  src = if stable then stableSrc else unstableSrc;
+  rev = copypartyFlake.shortRev or copypartyFlake.dirtyShortRev or "unknown";
+  unstableCodename = "unstable" + (lib.optionalString (copypartyFlake != null) "-${rev}");
 in
 buildPythonApplication {
  pname = "copyparty";
-  inherit (pinData) version;
-  src = fetchurl {
-    inherit (pinData) url hash;
-  };
+  inherit version src;
+  postPatch = lib.optionalString (!stable) ''
+    old_src="$(mktemp -d)"
+    tar -C "$old_src" -xf ${stableSrc}
+    declare -a folders
+    folders=("$old_src"/*)
+    count_folders="''${#folders[@]}"
+    if [[ $count_folders != 1 ]]; then
+      declare -p folders
+      echo "Expected 1 folder, found $count_folders" >&2
+      exit 1
+    fi
+    old_src_folder="''${folders[0]}"
+    cp -r "$old_src_folder"/copyparty/web/deps copyparty/web/deps
+    sed -i 's/^CODENAME =.*$/CODENAME = "${unstableCodename}"/' copyparty/__version__.py
+    ${lib.optionalString (copypartyFlake != null) (with dateStringsShort; ''
+      sed -i 's/^BUILD_DT =.*$/BUILD_DT = (${year}, ${month}, ${day})/' copyparty/__version__.py
+    '')}
+  '';
  dependencies =
    [
      jinja2
      fusepy
    ]
    ++ lib.optional withSMB impacket
+    ++ lib.optional withSFTP paramiko
    ++ lib.optional withFTP pyftpdlib
    ++ lib.optional withFTPS pyopenssl
    ++ lib.optional withTFTP partftpy
@@ -106,7 +157,7 @@ buildPythonApplication {
  meta = {
    description = "Turn almost any device into a file server";
    longDescription = ''
-      Portable file server with accelerated resumable uploads, dedup, WebDAV,
+      Portable file server with accelerated resumable uploads, dedup, WebDAV, SFTP,
      FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps
    '';
    homepage = "https://github.com/9001/copyparty";
@@ -1,5 +1,5 @@
 {
-    "url": "https://github.com/9001/copyparty/releases/download/v1.19.15/copyparty-1.19.15.tar.gz",
-    "version": "1.19.15",
-    "hash": "sha256-lfTB5Resqt1f3Di0cEfxanzVFJW9lK6RgidPJW21mDQ="
+    "url": "https://github.com/9001/copyparty/releases/download/v1.20.22/copyparty-1.20.22.tar.gz",
+    "version": "1.20.22",
+    "hash": "sha256-FNWQZElDw+0U4hCTzOeeBvGuIGre7aFgMtFackH/Q0Q="
 }
@@ -1,8 +1,29 @@
-final: prev: {
-  copyparty = final.python3.pkgs.callPackage ./copyparty {
-    ffmpeg = final.ffmpeg-full;
+final: prev:
+let
+  fullAttrs = {
+    withHashedPasswords = true;
+    withCertgen = true;
+    withThumbnails = true;
+    withFastThumbnails = true;
+    withMediaProcessing = true;
+    withBasicAudioMetadata = true;
+    withZeroMQ = true;
+    withSFTP = true;
+    withFTP = true;
+    withFTPS = true;
+    withTFTP = true;
+    withSMB = true;
+    withMagic = true;
  };

+  call = attrs: final.python3.pkgs.callPackage ./copyparty ({ ffmpeg = final.ffmpeg-headless; } // attrs);
+in
+{
+  copyparty = call { stable = true; };
+  copyparty-unstable = call { stable = false; };
+  copyparty-full = call (fullAttrs // { stable = true; });
+  copyparty-unstable-full = call (fullAttrs // { stable = false; });
+
  python3 = prev.python3.override {
    packageOverrides = pyFinal: pyPrev: {
      partftpy = pyFinal.callPackage ./partftpy { };
@@ -5,15 +5,15 @@ License:        MIT
 Group:          Utilities
 URL:            https://github.com/9001/copyparty
 Source0:        copyparty-$pkgver.tar.gz
-Summary:        File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++
+Summary:        File server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++
 BuildArch:      noarch
 BuildRequires:  python3, python3-devel, pyproject-rpm-macros, python-setuptools, python-wheel, make
 Requires:       python3, (python3-jinja2 or python-jinja2), lsof
 Recommends:     ffmpeg, (golang-github-cloudflare-cfssl or cfssl), python-mutagen, python-pillow, python-pyvips
-Recommends:     qm-vamp-plugins, python-argon2-cffi, (python-pyopenssl or pyopenssl), python-impacket
+Recommends:     qm-vamp-plugins, python-argon2-cffi, (python-pyopenssl or pyopenssl), python-paramiko, python-impacket

 %description
-Portable file server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps
+Portable file server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps

 See release at https://github.com/9001/copyparty/releases

@@ -26,9 +26,6 @@ See release at https://github.com/9001/copyparty/releases
 %setup -q

 %build
-cd "copyparty/web"
-make
-cd -
 %pyproject_wheel

 %install
@@ -39,3 +39,9 @@ point `--css-browser` to one of these by URL:

 * turns copyparty into chromecast just more flexible (and probably way more buggy)
 * usage: put the js somewhere in the webroot and `--js-browser /memes/meadup.js`
+
+
+
+# junk
+
+* [**rave.js**](./rave.js): april-fools joke, [demo (epilepsy warning)](https://cd.ocv.me/b/d2/d21/#af-9b927c42,sorthref), not maintained, very buggy
@@ -36,7 +36,7 @@

 		for (var a = 0; a < files.length; a++) {
 			var file = files[a],
-				is_pic = /\.(jpe?g|png|gif|webp)$/i.exec(file.vp),
+				is_pic = /\.(jpe?g|png|gif|webp|jxl)$/i.exec(file.vp),
 				is_audio = re_au_all.exec(file.vp),
 				basename = file.vp.replace(/\.[^\.]+$/, ""),
 				entry = pairs[basename];
@@ -66,11 +66,14 @@ var u2min = `

 </style>

-<a href="#" onclick="this.parentNode.innerHTML='';">show advanced options</a>
+<a href="#" id="u2min_off">show advanced options</a>
 `;

 if (!has(perms, 'read')) {
  var e2 = mknod('div');
  e2.innerHTML = u2min;
  ebi('wrap').insertBefore(e2, QS('#wfp'));
+  ebi('u2min_off').onclick = function () {
+    this.parentNode.innerHTML='';
+  };
 }
@@ -24,8 +24,8 @@ Note that you can select the owner and group of this volume by changing the `uid
 To install and start copyparty with Podman and systemd as the root user, run the following:

 ```shell
-sudo mkdir -pv /etc/systemd/container/ /etc/copyparty/
-sudo cp -v copyparty.container /etc/systemd/containers/
+sudo mkdir -pv /etc/containers/systemd/ /etc/copyparty/
+sudo cp -v copyparty.container /etc/containers/systemd/
 sudo cp -v copyparty.conf /etc/copyparty/
 sudo systemctl daemon-reload
 sudo systemctl start copyparty
@@ -8,7 +8,7 @@
  # and copyparty replaces %Y-%m%d with Year-MonthDay, so the
  # full path will be something like /var/log/copyparty/2023-1130.txt
  # (note: enable compression by adding .xz at the end)
-  # q, lo: $LOGS_DIRECTORY/%Y-%m%d.log
+  # q, lo: ${LOGS_DIRECTORY}/%Y-%m%d.log

  # p: 80,443,3923   # listen on 80/443 as well (requires CAP_NET_BIND_SERVICE)
  # i: 127.0.0.1     # only allow connections from localhost (reverse-proxies)
@@ -22,6 +22,12 @@
  # stats, nos-dup   # enable the prometheus endpoint, but disable the dupes counter (too slow)
  # no-robots, force-js  # make it harder for search engines to read your server

+  # enable version-checking by uncommenting one of the vc-url lines below;
+  # shows a warning-banner in the controlpanel if your version has a known vulnerability
+  #vc-url: https://api.github.com/repos/9001/copyparty/security-advisories?per_page=9
+  #vc-url: https://api.copyparty.eu/advisories
+  vc-exit  # panic and shutdown instead of just showing the warning
+

 [accounts]
  ed: wark  # username: password
@@ -32,5 +38,7 @@
  accs:
    rw: *      # everyone gets read-write access, but
    rwmda: ed  # the user "ed" gets read-write-move-delete-admin
+  flags:
+    e2ds       # enable filesystem-scanning for this volume only
    # uid: 1000  # If you're running as root, you can change the owner of this volume here
    # gid: 1000  # If you're running as root, you can change the group of this volume here
@@ -1,7 +1,7 @@
 #!/bin/sh
 #
 # PROVIDE: copyparty
-# REQUIRE: networking
+# REQUIRE: networking DAEMON FILESYSTEMS mountd
 # KEYWORD:

 . /etc/rc.subr
@@ -10,7 +10,7 @@ name="copyparty"
 rcvar="copyparty_enable"
 copyparty_user="copyparty"
 copyparty_args="-e2dsa -v /storage:/storage:r" # change as you see fit
-copyparty_command="/usr/local/bin/python3.9 /usr/local/copyparty/copyparty-sfx.py ${copyparty_args}"
+copyparty_command="/usr/local/bin/python3.11 /usr/local/copyparty/copyparty-sfx.py ${copyparty_args}"
 pidfile="/var/run/copyparty/${name}.pid"
 command="/usr/sbin/daemon"
 command_args="-P ${pidfile} -r -f ${copyparty_command}"
@@ -6,7 +6,7 @@
 #   https://apps.apple.com/us/app/a-shell/id1473805438
 #
 # step 2: copypaste the following command into a-Shell:
-#   curl -L https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh
+#   curl -L https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh | sh
 #
 # step 3: launch copyparty with this command: cpp
 #
@@ -1,5 +1,5 @@
 # this will start `/usr/bin/copyparty`
-# and read config from `$HOME/.config/copyparty.conf`
+# and read config from `$HOME/.config/copyparty/copyparty.conf`
 #
 # unless you add -q to disable logging, you may want to remove the
 #   following line to allow buffering (slightly better performance):
@@ -11,13 +11,13 @@ Description=copyparty file server
 [Service]
 Type=notify
 SyslogIdentifier=copyparty
-WorkingDirectory=/var/lib/copyparty-jail
+WorkingDirectory=/var/lib/copyparty-jail/srv
 Environment=PYTHONUNBUFFERED=x
 Environment=PRTY_CONFIG=%h/.config/copyparty/copyparty.conf
 ExecReload=/bin/kill -s USR1 $MAINPID

 # ensure there is a config
-ExecStartPre=/bin/bash -c 'if [[ ! -f %h/.config/copyparty/copyparty.conf ]]; then mkdir -p %h/.config/copyparty; cp /etc/copyparty/copyparty.conf %h/.config/copyparty/copyparty.conf; fi'
+ExecStartPre=/bin/bash -c 'if [[ ! -f %h/.config/copyparty/copyparty.conf ]]; then mkdir -p %h/.config/copyparty; cp -v /etc/copyparty.conf %h/.config/copyparty/; fi'

 # run copyparty
 ExecStart=/usr/bin/python3 /usr/bin/copyparty
@@ -1,13 +1,15 @@
-[global]
-  i: 127.0.0.1
-
-[accounts]
-  user: password
-
 [/]
-  /var/lib/copyparty-jail
+  /var/lib/copyparty-jail/srv/
  accs:
-    r: *
-    rwdma: user
-  flags:
-    grid
+    r: *    # everyone can [r]ead  (browse/download)
+
+
+# above is a very simple config;
+# the webroot is /var/lib/copyparty-jail/srv/
+# readable by everyone, no login required
+#
+# you'll want to replace this file; some inspiration:
+# - a basic example config: https://github.com/9001/copyparty/blob/hovudstraum/contrib/systemd/copyparty.example.conf
+# - another example with focus on syntax: https://github.com/9001/copyparty/blob/hovudstraum/docs/example.conf
+# - and CTRL-F "config file example" here: https://github.com/9001/copyparty
+# - full list of options: https://copyparty.eu/cli/
@@ -3,9 +3,6 @@
 # vim: ft=yaml:


-# put this file in /etc/
-
-
 [global]
  e2dsa  # enable file indexing and filesystem scanning
  e2ts   # and enable multimedia indexing
@@ -16,7 +13,16 @@
  # and copyparty replaces %Y-%m%d with Year-MonthDay, so the
  # full path will be something like /var/log/copyparty/2023-1130.txt
  # (note: enable compression by adding .xz at the end)
-  q, lo: $LOGS_DIRECTORY/%Y-%m%d.log
+  q, lo: ${LOGS_DIRECTORY}/%Y-%m%d.log
+
+  # enable version-checker by uncommenting one of the 'vc-url' lines below; this will
+  # periodically check if your copyparty version has a known security vulnerability,
+  # showing a warning on /?h (control-panel) for all users with permission 'a' or 'A'
+  #vc-url: https://api.github.com/repos/9001/copyparty/security-advisories?per_page=9
+  #vc-url: https://api.copyparty.eu/advisories
+
+  vc-age: 3  # how many hours to wait between each version-check
+  vc-exit  # emergency-exit if a version-check indicates that the current version is vulnerable

  # p: 80,443,3923   # listen on 80/443 as well (requires CAP_NET_BIND_SERVICE)
  # i: 127.0.0.1     # only allow connections from localhost (reverse-proxies)
@@ -40,3 +46,5 @@
  accs:
    rw: *      # everyone gets read-write access, but
    rwmda: ed  # the user "ed" gets read-write-move-delete-admin
+  flags:
+    e2ds       # enable filesystem-scanning for this volume only
@@ -1,5 +1,5 @@
 # this will start `/usr/bin/copyparty`
-# and read config from `/etc/copyparty/copyparty.conf`
+# and read config from `/etc/copyparty.conf`
 #
 # the %i refers to whatever you put after the copyparty@
 #   so with copyparty@foo.service, %i == foo
@@ -14,9 +14,9 @@ Description=copyparty file server
 [Service]
 Type=notify
 SyslogIdentifier=copyparty
-WorkingDirectory=/var/lib/copyparty-jail
+WorkingDirectory=/var/lib/copyparty-jail/srv
 Environment=PYTHONUNBUFFERED=x
-Environment=PRTY_CONFIG=/etc/copyparty/copyparty.conf
+Environment=PRTY_CONFIG=/etc/copyparty.conf
 ExecReload=/bin/kill -s USR1 $MAINPID

 # user to run as + where the TLS certificate is (if any)
@@ -1,10 +1,13 @@
-this is `/var/lib/copyparty-jail`, the fallback webroot when copyparty has not yet been configured
+this is `/var/lib/copyparty-jail/srv/`, the fallback webroot when copyparty has not yet been configured

-please edit `/etc/copyparty/copyparty.conf` (if running as a system service)
-or `$HOME/.config/copyparty/copyparty.conf` if running as a user service
+please edit the config-file and restart copyparty;
+* if running as a system service: `vim /etc/copyparty.conf`
+* if running as a user service: `vim $HOME/.config/copyparty/copyparty.conf`

-a basic configuration example is available at https://github.com/9001/copyparty/blob/hovudstraum/contrib/systemd/copyparty.example.conf
-a configuration example that explains most flags is available at https://github.com/9001/copyparty/blob/hovudstraum/docs/chungus.conf
+some inspiration:
+* a basic example config: https://github.com/9001/copyparty/blob/hovudstraum/contrib/systemd/copyparty.example.conf
+* another example with focus on syntax: https://github.com/9001/copyparty/blob/hovudstraum/docs/example.conf
+* and CTRL-F `config file example` here: https://github.com/9001/copyparty

-the full list of configuration options can be seen at https://ocv.me/copyparty/helptext.html 
+the full list of configuration options can be seen at https://copyparty.eu/cli/ 
 or by running `copyparty --help`
@@ -1,6 +1,6 @@
 # this will start `/usr/bin/copyparty`
 # in a chroot, preventing accidental access elsewhere,
-# and read copyparty config from `/etc/copyparty/copyparty.conf`
+# and read copyparty config from `/etc/copyparty.conf`
 #
 # expose additional filesystem locations to copyparty
 #   by listing them between the last `%i` and `--`
@@ -19,9 +19,9 @@ Description=copyparty file server
 [Service]
 Type=notify
 SyslogIdentifier=prisonparty
-WorkingDirectory=/var/lib/copyparty-jail
+WorkingDirectory=/var/lib/copyparty-jail/srv
 Environment=PYTHONUNBUFFERED=x
-Environment=PRTY_CONFIG=/etc/copyparty/copyparty.conf
+Environment=PRTY_CONFIG=/etc/copyparty.conf
 ExecReload=/bin/kill -s USR1 $MAINPID

 # user to run as + where the TLS certificate is (if any)
@@ -0,0 +1,36 @@
+# docker-compose.yml for copyparty
+services:
+  copyparty:
+    image: copyparty/ac:latest
+    container_name: copyparty
+    user: "1000:1000"
+    volumes:
+      - ./:/cfg:z
+      - /path/to/your/fileshare/top/folder:/w:z
+  
+    environment:
+      LD_PRELOAD: /usr/lib/libmimalloc-secure.so.NOPE
+      # enable mimalloc by replacing "NOPE" with "2" for a nice speed-boost (will use twice as much ram)
+
+      PYTHONUNBUFFERED: 1
+      # ensures log-messages are not delayed (but can reduce speed a tiny bit)
+
+    stop_grace_period: 15s  # thumbnailer is allowed to continue finishing up for 10s after the shutdown signal
+    healthcheck:
+      # hide it from logs with "/._" so it matches the default --lf-url filter 
+      test: ["CMD-SHELL", "wget --spider -q 127.0.0.1:3923/?reset=/._"]
+      interval: 1m
+      timeout: 2s
+      retries: 5
+      start_period: 15s
+    networks:
+      - traefik_default
+    labels:
+      - traefik.http.routers.copyparty.rule=Host(`<DOMAIN>`)
+      - traefik.http.services.copyparty.loadbalancer.server.port=3923
+      - traefik.http.routers.copyparty.tls.certresolver=le
+      - traefik.http.routers.copyparty.entrypoints=websecure
+  
+networks:
+  traefik_default:
+    external: true
@@ -0,0 +1,43 @@
+# note: copied from copyparty/docs/examples/docker/basic-docker-compose/copyparty.conf
+# not actually YAML but lets pretend:
+# -*- mode: yaml -*-
+# vim: ft=yaml:
+
+
+[global]
+  e2dsa  # enable file indexing and filesystem scanning
+  e2ts   # enable multimedia indexing
+  ansi   # enable colors in log messages (both in logfiles and stdout)
+
+  # q, lo: /cfg/log/%Y-%m%d.log   # log to file instead of docker
+
+  # p: 3939          # listen on another port
+  # ipa: 10.89.      # only allow connections from 10.89.*
+  # ipa: 172.16.4.0/23  # ...or only 172.16.4.* and 172.16.5.*
+  # ipa: lan         # ...or allow LAN only; reject internet IPs
+  # df: 16           # stop accepting uploads if less than 16 GB free disk space
+  # ver              # show copyparty version in the controlpanel
+  # grid             # show thumbnails/grid-view by default
+  # theme: 2         # monokai
+  # name: datasaver  # change the server-name that's displayed in the browser
+  # stats, nos-dup   # enable the prometheus endpoint, but disable the dupes counter (too slow)
+  # no-robots, force-js  # make it harder for search engines to read your server
+
+  # enable version-checking by uncommenting one of the vc-url lines below;
+  # shows a warning-banner in the controlpanel if your version has a known vulnerability
+  #vc-url: https://api.github.com/repos/9001/copyparty/security-advisories?per_page=9
+  #vc-url: https://api.copyparty.eu/advisories
+  vc-exit  # panic and shutdown instead of just showing the warning
+
+
+[accounts]
+  ed: wark  # username: password
+
+
+[/]            # create a volume at "/" (the webroot), which will
+  /w           # share /w (the docker data volume)
+  accs:
+    rw: *      # everyone gets read-write access, but
+    rwmda: ed  # the user "ed" gets read-write-move-delete-admin
+  flags:
+    e2ds       # enable filesystem-scanning for this volume only
@@ -0,0 +1,34 @@
+# docker-compose.yml for traefik
+services:
+  traefik:
+    image: traefik:latest
+    command:
+      - --api.insecure=false
+      - --api.dashboard=true
+      - --providers.docker=true
+      - --entrypoints.web.address=:80
+      - --entrypoints.websecure.address=:443
+      - --entryPoints.websecure.http.tls=true
+      - --entryPoints.web.http.redirections.entryPoint.to=websecure
+      - --entryPoints.web.http.redirections.entryPoint.scheme=https
+      - --certificatesresolvers.le.acme.email=<your email here>
+      - --certificatesresolvers.le.acme.storage=/letsencrypt/acme.json
+      - --certificatesresolvers.le.acme.httpchallenge.entrypoint=web
+    ports:
+      - 80:80
+      - 443:443
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock
+      - ./letsencrypt:/letsencrypt
+    labels:
+      # OPTIONAL: host your traefik dashboard on <DOMAIN>
+      # - traefik.http.routers.traefik-dash.rule=Host(`<DOMAIN>`)
+      # - traefik.http.routers.traefik-dash.entrypoints=websecure
+      # - traefik.http.routers.traefik-dash.service=api@internal
+      # - traefik.http.routers.traefik-dash.tls.certresolver=le
+      # - traefik.http.routers.traefik-dash.middlewares=traefik-dash-auth,traefik-dash-redirect
+      # - traefik.http.middlewares.traefik-dash-auth.basicauth.users=<USERNAME>:<HTPASSWD_ENCODED_PASSWD>
+      # - traefik.http.middlewares.traefik-dash-redirect.redirectregex.regex=^https://<SUBDOMAIN>\.<DOMAIN>\.TLD/?$
+      # - traefik.http.middlewares.traefik-dash-redirect.redirectregex.replacement=https://<DOMAIN>/dashboard/
+
+networks: {}
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os
 import platform
@@ -40,10 +40,23 @@ VT100 = "--ansi" in sys.argv or (
 )
 # introduced in anniversary update

+zs = os.environ.get("PRTY_CONFIG", "")
+CFG_DEF = [zs] if zs else []
+
 ANYWIN = WINDOWS or sys.platform in ["msys", "cygwin"]

 MACOS = platform.system() == "Darwin"

+FREEBSD = platform.system() == "FreeBSD"
+
+OPENBSD = platform.system() == "OpenBSD"
+
+ANYBSD = FREEBSD or OPENBSD
+
+UNIX = MACOS or ANYBSD
+
+GRAAL = platform.python_implementation() == "GraalVM"
+
 EXE = bool(getattr(sys, "frozen", False))

 try:
@@ -55,7 +68,7 @@ except:
 zs = """
 web/a/partyfuse.py
 web/a/u2c.py
-web/a/webdav-cfg.bat
+web/a/webdav-cfg.txt
 web/baguettebox.js
 web/browser.css
 web/browser.html
@@ -67,9 +80,10 @@ web/deps/busy.mp3
 web/deps/easymde.css
 web/deps/easymde.js
 web/deps/marked.js
-web/deps/fuse.py
+web/deps/mfusepy.py
 web/deps/mini-fa.css
 web/deps/mini-fa.woff
+web/deps/orbitron.woff2
 web/deps/prism.css
 web/deps/prism.js
 web/deps/prismd.css
@@ -86,7 +100,6 @@ web/md2.js
 web/mde.css
 web/mde.html
 web/mde.js
-web/msg.css
 web/msg.html
 web/opds.xml
 web/rups.css
@@ -100,12 +113,39 @@ web/splash.html
 web/splash.js
 web/svcs.html
 web/svcs.js
+web/tl/chi.js
+web/tl/cze.js
+web/tl/deu.js
+web/tl/epo.js
+web/tl/fin.js
+web/tl/fra.js
+web/tl/grc.js
+web/tl/hun.js
+web/tl/ita.js
+web/tl/jpn.js
+web/tl/kor.js
+web/tl/nld.js
+web/tl/nno.js
+web/tl/nor.js
+web/tl/pol.js
+web/tl/por.js
+web/tl/rus.js
+web/tl/spa.js
+web/tl/swe.js
+web/tl/tur.js
+web/tl/ukr.js
+web/tl/vie.js
 web/ui.css
 web/up2k.js
 web/util.js
-web/w.hash.js
+web/wopi.html
 """
 RES = set(zs.strip().split("\n"))
+RESM = {
+    "web/a/partyfuse.txt": "web/a/partyfuse.py",
+    "web/a/u2c.txt": "web/a/u2c.py",
+    "web/a/webdav-cfg.bat": "web/a/webdav-cfg.txt",
+}


 class EnvParams(object):
@@ -1,8 +1,8 @@
 # coding: utf-8

-VERSION = (1, 19, 16)
-CODENAME = "usernames"
-BUILD_DT = (2025, 10, 5)
+VERSION = (1, 20, 23)
+CODENAME = "sftp is fine too"
+BUILD_DT = (2026, 9, 6)

 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os
 import time
@@ -40,7 +40,7 @@ def makedirs(name: str, vf: dict[str, Any] = MKD_755, exist_ok: bool = True) ->
    todo = []
    bname = fsenc(name)
    while bname:
-        if os.path.isdir(bname):
+        if os.path.isdir(bname) or bname in todo:
            break
        todo.append(bname)
        bname = os.path.dirname(bname)
@@ -104,12 +104,16 @@ def utime(


 def utime_c(
-    log: Union["NamedLogger", Any], p: str, ts: int, follow_symlinks: bool = True, throw: bool = False
-) -> Optional[int]:
+    log: Union["NamedLogger", Any],
+    p: str,
+    ts: float,
+    follow_symlinks: bool = True,
+    throw: bool = False,
+) -> Optional[float]:
    clamp = 0
    ov = ts
    bp = fsenc(p)
-    now = int(time.time())
+    now = time.time()
    while True:
        try:
            if SYMTIME:
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os

@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import threading
 import time
@@ -146,13 +146,13 @@ class BrokerMp(object):
        returns a Queue object which eventually contains the response if want_retval
        (not-impl here since nothing uses it yet)
        """
-        if dest == "httpsrv.listen":
-            for p in self.procs:
-                p.q_pend.put((0, dest, [args[0], len(self.procs)]))
-
-        elif dest == "httpsrv.set_netdevs":
-            for p in self.procs:
-                p.q_pend.put((0, dest, list(args)))
+        if dest.startswith("httpsrv."):
+            if dest == "httpsrv.listen":
+                for p in self.procs:
+                    p.q_pend.put((0, dest, [args[0], len(self.procs)]))
+            else:
+                for p in self.procs:
+                    p.q_pend.put((0, dest, list(args)))

        elif dest == "cb_httpsrv_up":
            self.hub.cb_httpsrv_up()
@@ -160,6 +160,13 @@ class BrokerMp(object):
        else:
            raise Exception("what is " + str(dest))

+    def say1(self, dest: str, *args: Any) -> None:
+        """
+        send message to one lucky recipient
+        """
+        p = self.procs[0]
+        p.q_pend.put((0, dest, list(args)))
+
    def periodic(self) -> None:
        while True:
            time.sleep(1)
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse
 import os
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os
 import threading
@@ -53,12 +53,12 @@ class BrokerThr(BrokerCli):
        return NotExQueue(obj(*args))  # type: ignore

    def say(self, dest: str, *args: Any) -> None:
-        if dest == "httpsrv.listen":
-            self.httpsrv.listen(args[0], 1)
-            return
+        if dest.startswith("httpsrv."):
+            if dest == "httpsrv.listen":
+                self.httpsrv.listen(args[0], 1)
+                return

-        if dest == "httpsrv.set_netdevs":
-            self.httpsrv.set_netdevs(args[0])
+            getattr(self.httpsrv, dest[8:])(*args)
            return

        # new ipc invoking managed service in hub
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse

@@ -16,6 +16,7 @@ if True:  # pylint: disable=using-constant-test

 if TYPE_CHECKING:
    from .httpsrv import HttpSrv
+    from .svchub import SvcHub


 class ExceptionalQueue(Queue, object):
@@ -50,6 +51,7 @@ class BrokerCli(object):
    for example resolving httpconn.* in httpcli -- see lines tagged #mypy404
    """

+    hub: "SvcHub"
    log: "RootLogger"
    args: argparse.Namespace
    asrv: AuthSrv
@@ -1,12 +1,17 @@
+# coding: utf-8
+from __future__ import division, print_function, unicode_literals
+
 import calendar
 import errno
 import json
 import os
+import shutil
 import time

 from .__init__ import ANYWIN
 from .util import Netdev, atomic_move, load_resource, runcmd, wunlink

+NO_TLS = bool(os.environ.get("PRTY_NO_TLS"))
 HAVE_CFSSL = not os.environ.get("PRTY_NO_CFSSL")

 if True:  # pylint: disable=using-constant-test
@@ -19,6 +24,19 @@ else:
    VF = {"mv_re_t": 0, "rm_re_t": 0}


+def _sp_err(exe, what, rc, so, se, sin):
+    try:
+        zs = shutil.which(exe)
+    except:
+        zs = "<?>"
+    try:
+        zi = os.path.getsize(zs)
+    except:
+        zi = 0
+    t = "failed to %s; error %s using %s (%s):\n  STDOUT: %s\n  STDERR: %s\n  STDIN: %s\n"
+    raise Exception(t % (what, rc, zs, zi, so, se, sin.decode("utf-8")))
+
+
 def ensure_cert(log: "RootLogger", args) -> None:
    """
    the default cert (and the entire TLS support) is only here to enable the
@@ -37,9 +55,12 @@ def ensure_cert(log: "RootLogger", args) -> None:
        with open(args.cert, "wb") as f:
            f.write(cert_insec)

+    if args.certkey and not os.path.isfile(args.certkey):
+        raise Exception("certificate-key file does not exist: " + args.certkey)
+
    with open(args.cert, "rb") as f:
        buf = f.read()
-        o1 = buf.find(b" PRIVATE KEY-")
+        o1 = buf.find(b" PRIVATE KEY-") if not args.certkey else 0
        o2 = buf.find(b" CERTIFICATE-")
        m = "unsupported certificate format: "
        if o1 < 0:
@@ -53,8 +74,9 @@ def ensure_cert(log: "RootLogger", args) -> None:
        with open(args.cert, "rb") as f:
            active_cert = f.read()
        if active_cert == cert_insec:
-            t = "using default TLS certificate; https will be insecure:\033[36m {}"
-            log("cert", t.format(args.cert), 3)
+            t = "using default TLS certificate; %s will be insecure:\033[36m %s"
+            t2 = "https" if not args.ftps else "ftps"
+            log("cert", t % (t2, args.cert), 3)
    except:
        pass

@@ -107,13 +129,13 @@ def _gen_ca(log: "RootLogger", args):
    cmd = "cfssl gencert -initca -"
    rc, so, se = runcmd(cmd.split(), 30, sin=sin)
    if rc:
-        raise Exception("failed to create ca-cert: {}, {}".format(rc, se), 3)
+        _sp_err("cfssl", "create ca-cert", rc, so, se, sin)

    cmd = "cfssljson -bare ca"
    sin = so.encode("utf-8")
    rc, so, se = runcmd(cmd.split(), 10, sin=sin, cwd=args.crt_dir)
    if rc:
-        raise Exception("failed to translate ca-cert: {}, {}".format(rc, se), 3)
+        _sp_err("cfssljson", "translate ca-cert", rc, so, se, sin)

    bname = os.path.join(args.crt_dir, "ca")
    try:
@@ -201,13 +223,13 @@ def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
    acmd = cmd.split() + ["-hostname=" + ",".join(names), "-"]
    rc, so, se = runcmd(acmd, 30, sin=sin, cwd=args.crt_dir)
    if rc:
-        raise Exception("failed to create cert: {}, {}".format(rc, se))
+        _sp_err("cfssl", "create cert", rc, so, se, sin)

    cmd = "cfssljson -bare srv"
    sin = so.encode("utf-8")
    rc, so, se = runcmd(cmd.split(), 10, sin=sin, cwd=args.crt_dir)
    if rc:
-        raise Exception("failed to translate cert: {}, {}".format(rc, se))
+        _sp_err("cfssljson", "translate cert", rc, so, se, sin)

    bname = os.path.join(args.crt_dir, "srv")
    try:
@@ -235,10 +257,10 @@ def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
 def gencert(log: "RootLogger", args, netdevs: dict[str, Netdev]):
    global HAVE_CFSSL

-    if args.http_only:
+    if args.http_only and not args.ftps:
        return

-    if args.no_crt or not HAVE_CFSSL:
+    if args.no_crt or args.certkey or not HAVE_CFSSL:
        ensure_cert(log, args)
        return

@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 # awk -F\" '/add_argument\("-[^-]/{print(substr($2,2))}' copyparty/__main__.py | sort | tr '\n' ' '
 zs = "a c e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vp e2vu ed emp i j lo mcr mte mth mtm mtp nb nc nid nih nth nw p q s ss sss v z zv"
@@ -19,16 +19,25 @@ def vf_bmap() -> dict[str, str]:
        "no_clone": "noclone",
        "no_dirsz": "nodirsz",
        "no_dupe": "nodupe",
+        "no_dupe_m": "nodupem",
        "no_forget": "noforget",
+        "no_html": "nohtml",
+        "no_mime": "nomime",
        "no_pipe": "nopipe",
        "no_robots": "norobots",
+        "no_script": "noscript",
        "no_tail": "notail",
        "no_thumb": "dthumb",
        "no_vthumb": "dvthumb",
        "no_athumb": "dathumb",
+        "vol_nospawn": "nospawn",
+        "vol_or_crash": "assert_root",
    }
    for k in (
+        "allow_svg_js",
        "dedup",
+        "dlni",
+        "dothidden",
        "dotsrch",
        "e2d",
        "e2ds",
@@ -43,9 +52,12 @@ def vf_bmap() -> dict[str, str]:
        "grid",
        "gsel",
        "hardlink",
+        "hist_cow",
        "magic",
        "md_no_br",
        "no_db_ip",
+        "no_logues",
+        "no_readme",
        "no_sb_md",
        "no_sb_lg",
        "nsort",
@@ -58,6 +70,16 @@ def vf_bmap() -> dict[str, str]:
        "rm_partial",
        "rmagic",
        "rss",
+        "show_hist",
+        "ui_noacci",
+        "ui_nocpla",
+        "ui_nolbar",
+        "ui_nombar",
+        "ui_nonav",
+        "ui_notree",
+        "ui_norepl",
+        "ui_nosrvi",
+        "ui_noctxb",
        "wo_up_readme",
        "wram",
        "xdev",
@@ -83,14 +105,21 @@ def vf_vmap() -> dict[str, str]:
        "th_x3": "th3x",
    }
    for k in (
+        "apnd_who",
        "bup_ck",
+        "cachectl",
        "casechk",
        "chmod_d",
        "chmod_f",
+        "csp_ui",
+        "csp_dl",
        "dbd",
+        "db_xattr",
        "du_who",
+        "epilogues",
        "ufavico",
        "forget_ip",
+        "fsnt",
        "hsortn",
        "html_head",
        "html_head_s",
@@ -99,6 +128,7 @@ def vf_vmap() -> dict[str, str]:
        "lg_sba",
        "md_sba",
        "md_hist",
+        "md_nhist",
        "nrand",
        "u2ow",
        "og_desc",
@@ -111,10 +141,18 @@ def vf_vmap() -> dict[str, str]:
        "og_tpl",
        "og_ua",
        "opds_exts",
+        "prologues",
+        "preadmes",
        "put_ck",
        "put_name",
+        "readmes",
+        "rotf_tz",
        "mv_retry",
        "rm_retry",
+        "rss_sort",
+        "rss_fmt_t",
+        "rss_fmt_d",
+        "rw_edit",
        "shr_who",
        "sort",
        "tail_fd",
@@ -122,6 +160,10 @@ def vf_vmap() -> dict[str, str]:
        "tail_tmax",
        "tail_who",
        "tcolor",
+        "th_covers",
+        "th_pregen",
+        "th_qv",
+        "th_qvx",
        "th_spec_p",
        "txt_eol",
        "unlist",
@@ -189,10 +231,13 @@ flagcats = {
        "safededup": "verify on-disk data before using it for dedup",
        "noclone": "take dupe data from clients, even if available on HDD",
        "nodupe": "rejects existing files (instead of linking/cloning them)",
+        "nodupem": "rejects existing files during moves as well",
+        "casechk=auto": "actively prevent case-insensitive filesystem? y/n",
        "chmod_d=755": "unix-permission for new dirs/folders",
        "chmod_f=644": "unix-permission for new files",
        "uid=573": "change owner of new files/folders to unix-user 573",
        "gid=999": "change owner of new files/folders to unix-group 999",
+        "fsnt=auto": "filesystem filename traits (lin/win/mac/auto)",
        "wram": "allow uploading into ramdisks",
        "sparse": "force use of sparse files, mainly for s3-backed storage",
        "nosparse": "deny use of sparse files, mainly for slow storage",
@@ -208,6 +253,7 @@ flagcats = {
        "pk": "forces server-side compression, optional arg: xz,9",
    },
    "upload rules": {
+        "apnd_who=dw": "who can append? (aw/dw/w/no)",
        "maxn=250,600": "max 250 uploads over 15min",
        "maxb=1g,300": "max 1 GiB over 5min (suffixes: b, k, m, g, t)",
        "vmaxb=1g": "total volume size max 1 GiB (suffixes: b, k, m, g, t)",
@@ -254,13 +300,14 @@ flagcats = {
        "no_db_ip": "never store uploader-IP in the db; disables unpost",
        "fat32": "avoid excessive reindexing on android sdcardfs",
        "dbd=[acid|swal|wal|yolo]": "database speed-durability tradeoff",
-        "casechk=auto": "actively prevent case-insensitive filesystem? y/n",
+        "hist_cow": "btrfs: keep copy-on-write enabled on db/thumbs",
        "xlink": "cross-volume dupe detection / linking (dangerous)",
        "xdev": "do not descend into other filesystems",
        "xvol": "do not follow symlinks leaving the volume root",
        "dotsrch": "show dotfiles in search results",
        "nodotsrch": "hide dotfiles in search results (default)",
        "srch_excl": "exclude search results with URL matching this regex",
+        "db_xattr=user.foo,user.bar": "index file xattrs as media-tags",
    },
    'database, audio tags\n"mte", "mth", "mtp", "mtm" all work the same as -mte, -mth, ...': {
        "mte=artist,title": "media-tags to index/display",
@@ -273,14 +320,19 @@ flagcats = {
        "dvthumb": "disables video thumbnails",
        "dathumb": "disables audio thumbnails (spectrograms)",
        "dithumb": "disables image thumbnails",
+        "dethumb": "disables custom thumbnails (--th-extract)",
        "pngquant": "compress audio waveforms 33% better",
        "thsize": "thumbnail res; WxH",
        "crop": "center-cropping (y/n/fy/fn)",
        "th3x": "3x resolution (y/n/fy/fn)",
+        "th_qv=40": "webp/jpg thumbnail quality (10~90)",
+        "th_qvx=40": "jxl thumbnail quality (10~90)",
+        "th_covers=a.jpg,a.png": "filenames to prefer as folder-thumbs",
        "convt": "convert-to-image timeout in seconds",
        "aconvt": "convert-to-audio timeout in seconds",
        "th_spec_p=1": "make spectrograms? 0=never 1=fallback 2=always",
        "ext_th=s=/b.png": "use /b.png as thumbnail for file-extension s",
+        "th_pregen=w,wf": "pregenerate thumbs for these formats",
    },
    "handlers\n(better explained in --help-handlers)": {
        "on404=PY": "handle 404s by executing PY file",
@@ -307,6 +359,8 @@ flagcats = {
        "hsortn": "number of sort-rules to add to media URLs",
        "ufavico=URL": "per-volume favicon (.ico/png/gif/svg)",
        "unlist": "dont list files matching REGEX",
+        "dothidden": "enable support for .hidden files",
+        "dlni": "force-download (no-inline) files on click",
        "html_head=TXT": "includes TXT in the <head>, or @PATH for file at PATH",
        "html_head_s=TXT": "additional static text in the html <head>",
        "tcolor=#fc0": "theme color (a hint for webbrowsers, discord, etc.)",
@@ -316,6 +370,14 @@ flagcats = {
        "norobots": "kindly asks search engines to leave",
        "unlistcr": "don't list read-access in controlpanel",
        "unlistcw": "don't list write-access in controlpanel",
+        "prologues=.prologue.html": "files to embed above/before files",
+        "epilogues=.epilogue.html": "files to embed below/after files",
+        "readmes=readme.md,README.md": "files to embed as readmes",
+        "preadmes=preadme.md,PREADME.md": "files to embed as preadmes",
+        "plainreadme": "embed readme/preadme as monospace plaintext",
+        "plainlogues": "embed prologue/epilogue as sans-serif plaintext",
+        "no_readme": "disable embedding readme/preadme",
+        "no_logues": "disable embedding prologue/epilogue",
        "no_sb_md": "disable js sandbox for markdown files",
        "no_sb_lg": "disable js sandbox for prologue/epilogue",
        "sb_md": "enable js sandbox for markdown files (default)",
@@ -324,7 +386,21 @@ flagcats = {
        "lg_sbf": "list of *logue-sandbox safeguards to disable",
        "md_sba": "value of iframe allow-prop for markdown-sandbox",
        "lg_sba": "value of iframe allow-prop for *logue-sandbox",
+        "csp_ui": "content-security-policy for the web-UI",
+        "csp_dl": "content-security-policy for static files",
        "nohtml": "return html and markdown as text/html",
+        "nomime": "disallow ?mime to change response type",
+        "noscript": "disable most javascript by use of CSP",
+        "allow_svg_js": "allow javascript in svg images",
+        "ui_noacci": "hide account-info in the UI",
+        "ui_nocpla": "hide cpanel-link in the UI",
+        "ui_nolbar": "hide link-bar in the UI",
+        "ui_nombar": "hide top-menu in the UI",
+        "ui_nonav": "hide navpane+breadcrumbs in the UI",
+        "ui_notree": "hide navpane in the UI",
+        "ui_norepl": "hide repl-button in the UI",
+        "ui_nosrvi": "hide server-info in the UI",
+        "ui_noctxb": "hide context-buttons in the UI",
    },
    "opengraph (discord embeds)": {
        "og": "enable OG (disables hotlinking)",
@@ -345,8 +421,11 @@ flagcats = {
        "opds_exts": "file formats to list in OPDS feeds; leave empty to show everything",
    },
    "textfiles": {
+        "rw_edit=md,txt": "only require read+write to edit .md and .txt",
        "md_no_br": "newline only on double-newline or two tailing spaces",
-        "md_hist": "where to put markdown backups; s=subfolder, v=volHist, n=nope",
+        "md_hist": "where to put text-editor backups; s=subfolder, v=volHist, n=nope",
+        "md_nhist=99": "how many text-editor backups to keep",
+        "show_hist": "allow browsing the .hist folder (old textfiles)",
        "exp": "enable textfile expansion; see --help-exp",
        "exp_md": "placeholders to expand in markdown files; see --help",
        "exp_lg": "placeholders to expand in prologue/epilogue; see --help",
@@ -359,6 +438,12 @@ flagcats = {
        "tail_tmax=30": "kill connection after 30 sec",
        "tail_who=2": "restrict ?tail access (1=admins,2=authed,3=everyone)",
    },
+    "rss": {
+        "rss": "allow '?rss' URL suffix (experimental)",
+        "rss_sort=m": "default sort-order (m/u/n/s)",
+        "rss_fmt_t={fname}": "default title-format",
+        "rss_fmt_d={album},{.tn}": "default description-format",
+    },
    "others": {
        "dots": "allow all users with read-access to\nenable the option to show dotfiles in listings",
        "fk=8": 'generates per-file accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
@@ -366,7 +451,6 @@ flagcats = {
        "dk=8": 'generates per-directory accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
        "dks": "per-directory accesskeys allow browsing into subdirs",
        "dky": 'allow seeing files (not folders) inside a specific folder\nwith "g" perm, and does not require a valid dirkey to do so',
-        "rss": "allow '?rss' URL suffix (experimental)",
        "rmagic": "expensive analysis for mimetype accuracy",
        "shr_who=auth": "who can create shares? no/auth/a",
        "unp_who=2": "unpost only if same... 1=ip+name, 2=ip, 3=name",
@@ -377,8 +461,11 @@ flagcats = {
        "zipmaxt=no": "reply with 'no' if download-as-zip exceeds max",
        "zipmaxu": "zip-size-limit does not apply to authenticated users",
        "nopipe": "disable race-the-beam (download unfinished uploads)",
+        "cachectl=no-cache": "controls caching in webbrowsers",
        "mv_retry": "ms-windows: timeout for renaming busy files",
        "rm_retry": "ms-windows: timeout for deleting busy files",
+        "nospawn": "don't create volume's folder if not exist",
+        "assert_root": "crash on startup if volume's folder not exist",
        "davauth": "ask webdav clients to login for all folders",
        "davrt": "show lastmod time of symlink destination, not the link itself\n(note: this option is always enabled for recursive listings)",
    },
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import importlib
 import sys
@@ -1,15 +1,16 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse
+import json
 import os
 import re
 import time

-from .__init__ import ANYWIN, MACOS
+from .__init__ import ANYWIN, FREEBSD, MACOS, UNIX
 from .authsrv import AXS, VFS, AuthSrv
 from .bos import bos
-from .util import chkcmd, min_ex, undot
+from .util import chkcmd, json_hesc, min_ex, undot

 if True:  # pylint: disable=using-constant-test
    from typing import Optional, Union
@@ -87,7 +88,7 @@ class Fstab(object):

    def _from_sp_mount(self) -> dict[str, str]:
        sptn = r"^.*? on (.*) type ([^ ]+) \(.*"
-        if MACOS:
+        if MACOS or FREEBSD:
            sptn = r"^.*? on (.*) \(([^ ]+), .*"

        ptn = re.compile(sptn)
@@ -117,7 +118,7 @@ class Fstab(object):

    def build_tab(self) -> None:
        self.log("inspecting mtab for changes")
-        dtab = self._from_sp_mount() if MACOS else self._from_proc()
+        dtab = self._from_sp_mount() if UNIX else self._from_proc()

        # keep empirically-correct values if mounttab unchanged
        srctab = str(sorted(dtab.items()))
@@ -127,6 +128,24 @@ class Fstab(object):

        self.log("mtab has changed; reevaluating support for sparse files")

+        try:
+            fuses = [mp for mp, fs in dtab.items() if fs == "fuseblk"]
+            if not fuses or UNIX:
+                raise Exception()
+            try:
+                so, _ = chkcmd(["lsblk", "-nrfo", "FSTYPE,MOUNTPOINT"])  # centos6
+            except:
+                so, _ = chkcmd(["lsblk", "-nrfo", "FSTYPE,MOUNTPOINTS"])  # future
+            for ln in so.split("\n"):
+                zsl = ln.split(" ", 1)
+                if len(zsl) != 2:
+                    continue
+                fs, mp = zsl
+                if mp in fuses:
+                    dtab[mp] = fs
+        except:
+            pass
+
        tab1 = list(dtab.items())
        tab1.sort(key=lambda x: (len(x[0]), x[0]))
        path1, fs1 = tab1[0]
@@ -194,24 +213,54 @@ class Fstab(object):
        return ret.realpath, ""


+_fstab: Optional[Fstab] = None
+winfs = set(("msdos", "vfat", "ntfs", "exfat"))
+# "msdos" = vfat on macos
+
+
 def ramdisk_chk(asrv: AuthSrv) -> None:
    # should have been in authsrv but that's a circular import
+    global _fstab
    mods = []
    ramfs = ("tmpfs", "overlay")
    log = asrv.log_func or print
-    fstab = Fstab(log, asrv.args, False)
+    if not _fstab:
+        _fstab = Fstab(log, asrv.args, False)
    for vn in asrv.vfs.all_nodes.values():
        if not vn.axs.uwrite or "wram" in vn.flags:
            continue
        ap = vn.realpath
        if not ap or os.path.isfile(ap):
            continue
-        fs, mp = fstab.get(ap)
+        fs, mp = _fstab.get(ap)
        mp = "/" + mp.strip("/")
        if fs == "tmpfs" or (mp == "/" and fs in ramfs):
            mods.append((vn.vpath, ap, fs, mp))
            vn.axs.uwrite.clear()
+            vn.axs.umove.clear()
+            for un, ztsp in list(vn.uaxs.items()):
+                zsl = list(ztsp)
+                zsl[1] = False
+                zsl[2] = False
+                vn.uaxs[un] = tuple(zsl)  # type: ignore
    if mods:
        t = "WARNING: write-access was removed from the following volumes because they are not mapped to an actual HDD for storage! All uploaded data would live in RAM only, and all uploaded files would be LOST on next reboot. To allow uploading and ignore this hazard, enable the 'wram' option (global/volflag). List of affected volumes:"
        t2 = ["\n  volume=[/%s], abspath=%r, type=%s, root=%r" % x for x in mods]
        log("vfs", t + "".join(t2) + "\n", 1)
+
+    assume = "mac" if MACOS else "win" if ANYWIN else "lin"
+    for vol in asrv.vfs.all_nodes.values():
+        if not vol.realpath or vol.flags.get("is_file"):
+            continue
+        zs = vol.flags["fsnt"].strip()[:3].lower()
+        if ANYWIN and not zs:
+            zs = "win"
+        if zs in ("lin", "win", "mac"):
+            vol.flags["fsnt"] = zs
+            continue
+        fs = _fstab.get(vol.realpath)[0]
+        fs = "win" if fs in winfs else assume
+        htm = json.loads(vol.js_htm)
+        vol.flags["fsnt"] = vol.js_ls["fsnt"] = htm["fsnt"] = fs
+        vol.js_htm = json_hesc(json.dumps(htm))
+        # md_htm is dontcare; only relevant for up2k request body
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse
 import errno
@@ -19,15 +19,16 @@ from .__init__ import PY2, TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
 from .util import (
-    FN_EMB,
    VF_CAREFUL,
    Daemon,
    ODict,
    Pebkac,
    exclude_dotfiles,
+    exclude_dothidden,
    fsenc,
    ipnorm,
    pybin,
+    read_utf8,
    relchk,
    runhook,
    sanitize_fn,
@@ -86,7 +87,7 @@ class FtpAuth(DummyAuthorizer):
            if args.usernames:
                alts = ["%s:%s" % (username, password)]
            else:
-                alts = password, username
+                alts = [password, username]

            for zs in alts:
                zs = asrv.iacct.get(asrv.ah.hash(zs), "")
@@ -148,6 +149,7 @@ class FtpFs(AbstractedFS):
        self.hub: "SvcHub" = cmd_channel.hub
        self.args = cmd_channel.args
        self.uname = cmd_channel.uname
+        self.log_src = "%s:FTP" % (cmd_channel.cli_ip,)

        self.cwd = "/"  # pyftpdlib convention of leading slash
        self.root = "/var/lib/empty"
@@ -155,8 +157,8 @@ class FtpFs(AbstractedFS):
        self.listdirinfo = self.listdir
        self.chdir(".")

-    def log(self, msg: str, c: Union[int, str] = 0) -> None:
-        self.hub.log("ftpd", msg, c)
+    def clog(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log(self.log_src, msg, c)

    def v2a(
        self,
@@ -174,12 +176,12 @@ class FtpFs(AbstractedFS):
                t = "Unsupported characters in [{}]"
                raise FSE(t.format(vpath), 1)

-            fn = sanitize_fn(fn or "", "")
+            fn = sanitize_fn(fn or "")
            vpath = vjoin(rd, fn)
            vfs, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
            if (
                w
-                and fn.lower() in FN_EMB
+                and fn.lower() in vfs.flags["emb_all"]
                and self.h.uname not in vfs.axs.uread
                and "wo_up_readme" not in vfs.flags
            ):
@@ -198,14 +200,16 @@ class FtpFs(AbstractedFS):
                if not avfs:
                    raise FSE(t.format(vpath), 1)

-                cr, cw, cm, cd, _, _, _, _ = avfs.can_access("", self.h.uname)
+                cr, cw, cm, cd, _, _, _, _, _ = avfs.uaxs[self.h.uname]
                if r and not cr or w and not cw or m and not cm or d and not cd:
                    raise FSE(t.format(vpath), 1)
+            else:
+                ap = vfs.canonical(rem, False)

            if "bcasechk" in vfs.flags and not vfs.casechk(rem, True):
                raise FSE("No such file or directory", 1)

-            return os.path.join(vfs.realpath, rem), vfs, rem
+            return ap, vfs, rem
        except Pebkac as ex:
            raise FSE(str(ex))

@@ -235,6 +239,8 @@ class FtpFs(AbstractedFS):
        return True

    def open(self, filename: str, mode: str) -> typing.IO[Any]:
+        self.clog("open(%r,%r) @%s" % (filename, mode, self.uname))
+
        r = "r" in mode
        w = "w" in mode or "a" in mode or "+" in mode

@@ -249,9 +255,36 @@ class FtpFs(AbstractedFS):
                need_unlink = False
                td = 0

-        if w and need_unlink:
+            xbu = vfs.flags.get("xbu")
+            if xbu:
+                hr = runhook(
+                    self.clog,
+                    None,
+                    self.hub.up2k,
+                    "xbu.ftp",
+                    xbu,
+                    ap,
+                    filename,
+                    "",
+                    "",
+                    "",
+                    0,
+                    0,
+                    "1.3.8.7",
+                    time.time(),
+                    None,
+                )
+                t = hr.get("rejectmsg") or ""
+                if t or hr.get("rc") != 0:
+                    if not t:
+                        t = "upload blocked by xbu server config: %r" % (filename,)
+                    self.clog(t, 3)
+                    raise FSE(t)
+
+        if w and need_unlink:  # type: ignore  # !rm
+            assert td  # type: ignore  # !rm
            if td >= -1 and td <= self.args.ftp_wt:
-                # within permitted timeframe; unlink and accept
+                # within permitted timeframe; allow overwrite or resume
                do_it = True
            elif self.args.no_del or self.args.ftp_no_ow:
                # file too old, or overwrite not allowed; reject
@@ -268,12 +301,15 @@ class FtpFs(AbstractedFS):
            if not do_it:
                raise FSE("File already exists")

-            wunlink(self.log, ap, VF_CAREFUL)
+            # Don't unlink file for append mode
+            elif "a" not in mode:
+                wunlink(self.clog, ap, VF_CAREFUL)

        ret = open(fsenc(ap), mode, self.args.iobuf)
        if w and "fperms" in vfs.flags:
            set_fperms(ret, vfs.flags)

+        self.clog(" `-- %r" % (ap,))
        return ret

    def chdir(self, path: str) -> None:
@@ -299,10 +335,12 @@ class FtpFs(AbstractedFS):
        self.cwd = nwd

    def mkdir(self, path: str) -> None:
+        self.clog("mkdir(%r) @%s" % (path, self.uname))
        ap, vfs, _ = self.rv2a(path, w=True)
        bos.makedirs(ap, vf=vfs.flags)  # filezilla expects this

    def listdir(self, path: str) -> list[str]:
+        self.clog("ls(%r) @%s" % (path, self.uname))
        vpath = join(self.cwd, path)
        try:
            ap, vfs, rem = self.v2a(vpath, True, False)
@@ -320,7 +358,10 @@ class FtpFs(AbstractedFS):
            vfs_ls.extend(vfs_virt.keys())

            if self.uname not in vfs.axs.udot:
-                vfs_ls = exclude_dotfiles(vfs_ls)
+                if "dothidden" in vfs.flags and ".hidden" in [x[0] for x in vfs_ls]:
+                    vfs_ls = exclude_dothidden(vfs_ls, fsroot)
+                else:
+                    vfs_ls = exclude_dotfiles(vfs_ls)

            vfs_ls.sort()
            return vfs_ls
@@ -349,6 +390,7 @@ class FtpFs(AbstractedFS):
            return ret

    def rmdir(self, path: str) -> None:
+        self.clog("rmdir(%r) @%s" % (path, self.uname))
        ap = self.rv2a(path, d=True)[0]
        try:
            bos.rmdir(ap)
@@ -357,6 +399,7 @@ class FtpFs(AbstractedFS):
                raise

    def remove(self, path: str) -> None:
+        self.clog("rm(%r) @%s" % (path, self.uname))
        if self.args.no_del:
            raise FSE("The delete feature is disabled in server config")

@@ -367,6 +410,7 @@ class FtpFs(AbstractedFS):
            raise FSE(str(ex))

    def rename(self, src: str, dst: str) -> None:
+        self.clog("mv(%r,%r) @%s" % (src, dst, self.uname))
        if self.args.no_mv:
            raise FSE("The rename/move feature is disabled in server config")

@@ -396,6 +440,7 @@ class FtpFs(AbstractedFS):
            return st

    def utime(self, path: str, timeval: float) -> None:
+        self.clog("utime(%r,%r) @%s" % (path, timeval, self.uname))
        ap = self.rv2a(path, w=True)[0]
        bos.utime_c(logging.warning, ap, int(timeval), False)

@@ -403,6 +448,9 @@ class FtpFs(AbstractedFS):
        ap = self.rv2a(path)[0]
        return bos.stat(ap)

+    def readlink(self, *a) -> None:
+        raise OSError(errno.EINVAL, "nope")
+
    def isfile(self, path: str) -> bool:
        try:
            st = self.stat(path)
@@ -445,9 +493,13 @@ class FtpFs(AbstractedFS):
    def get_user_by_uid(self, uid: int) -> str:
        return "root"

-    def get_group_by_uid(self, gid: int) -> str:
+    def get_group_by_gid(self, gid: int) -> str:
        return "root"

+    def mkstemp(self, *a, **ka):
+        self.clog("mkstemp%r @%s" % (a, self.uname))
+        raise FSE("nope")
+

 class FtpHandler(FTPHandler):
    abstracted_fs = FtpFs
@@ -476,6 +528,7 @@ class FtpHandler(FTPHandler):
            return

        self.cli_ip = cip
+        self.log_src = "%s:FTP" % (cip,)

        # abspath->vpath mapping to resolve log_transfer paths
        self.vfs_map: dict[str, str] = {}
@@ -483,6 +536,15 @@ class FtpHandler(FTPHandler):
        # reduce non-debug logging
        self.log_cmds_list = [x for x in self.log_cmds_list if x not in ("CWD", "XCWD")]

+    def clog(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log(self.log_src, msg, c)
+
+    def ftp_NOOP(self, *a) -> None:
+        self.respond("200 pong", logging.debug)
+
+    def ftp_STOU(self, *a) -> None:
+        self.respond("550 nope", logging.warning)
+
    def ftp_STOR(self, file: str, mode: str = "w") -> Any:
        # Optional[str]
        vp = join(self.fs.cwd, file).lstrip("/")
@@ -495,7 +557,7 @@ class FtpHandler(FTPHandler):
        xbu = vfs.flags.get("xbu")
        if xbu:
            hr = runhook(
-                None,
+                self.clog,
                None,
                self.hub.up2k,
                "xbu.ftpd",
@@ -509,10 +571,10 @@ class FtpHandler(FTPHandler):
                0,
                self.cli_ip,
                time.time(),
-                "",
+                None,
            )
            t = hr.get("rejectmsg") or ""
-            if t or not hr:
+            if t or hr.get("rc") != 0:
                if not t:
                    t = "Upload blocked by xbu server config: %r" % (vp,)
                self.respond("550 %s" % (t,), logging.info)
@@ -582,17 +644,25 @@ class Ftpd(object):
                print(t.format(pybin))
                sys.exit(1)

+            if self.args.certkey:
+                h1.keyfile = self.args.certkey
            h1.certfile = self.args.cert
            h1.tls_control_required = True
            h1.tls_data_required = True

            hs.append([h1, self.args.ftps])

+        zs = self.args.ftp_banner.replace("\\n", "\n")
+        if zs.startswith("@"):
+            zs = read_utf8(None, zs[1:], False)
+        banner = zs.replace("\r", "").replace("\n", "\r\n").strip()
+
        for h_lp in hs:
            h2, lp = h_lp
            FtpHandler.hub = h2.hub = hub
            FtpHandler.args = h2.args = hub.args
            FtpHandler.authorizer = h2.authorizer = FtpAuth(hub)
+            FtpHandler.banner = banner

            if self.args.ftp_pr:
                p1, p2 = [int(x) for x in self.args.ftp_pr.split("-")]
@@ -612,7 +682,7 @@ class Ftpd(object):
        lgr = logging.getLogger("pyftpdlib")
        lgr.setLevel(logging.DEBUG if self.args.ftpv else logging.INFO)

-        ips = self.args.i
+        ips = self.args.ftp_i
        if "::" in ips:
            ips.append("0.0.0.0")

@@ -1,32 +1,29 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse  # typechk
-import os
 import re
 import socket
 import threading  # typechk
 import time

-try:
-    if os.environ.get("PRTY_NO_TLS"):
-        raise Exception()
-
-    HAVE_SSL = True
-    import ssl
-except:
-    HAVE_SSL = False
-
 from . import util as Util
 from .__init__ import TYPE_CHECKING, EnvParams
 from .authsrv import AuthSrv  # typechk
+from .cert import NO_TLS
 from .httpcli import HttpCli
-from .ico import Ico
-from .mtag import HAVE_FFMPEG
-from .th_cli import ThumbCli
-from .th_srv import HAVE_PIL, HAVE_VIPS
 from .u2idx import U2idx
-from .util import HMaccas, NetMap, shut_socket
+from .util import HMaccas, NetMap, min_ex, shut_socket
+
+try:
+    if NO_TLS:
+        raise Exception()
+
+    import ssl
+
+    HAVE_SSL = True
+except:
+    HAVE_SSL = False

 if True:  # pylint: disable=using-constant-test
    from typing import Optional, Pattern, Union
@@ -62,27 +59,22 @@ class HttpConn(object):
        self.ipu_iu: Optional[dict[str, str]] = hsrv.ipu_iu
        self.ipu_nm: Optional[NetMap] = hsrv.ipu_nm
        self.ipa_nm: Optional[NetMap] = hsrv.ipa_nm
+        self.ipar_nm: Optional[NetMap] = hsrv.ipar_nm
        self.xff_nm: Optional[NetMap] = hsrv.xff_nm
        self.xff_lan: NetMap = hsrv.xff_lan  # type: ignore
        self.iphash: HMaccas = hsrv.broker.iphash
        self.bans: dict[str, int] = hsrv.bans
        self.aclose: dict[str, int] = hsrv.aclose

-        enth = (HAVE_PIL or HAVE_VIPS or HAVE_FFMPEG) and not self.args.no_thumb
-        self.thumbcli: Optional[ThumbCli] = ThumbCli(hsrv) if enth else None  # mypy404
-        self.ico: Ico = Ico(self.args)  # mypy404
-
        self.t0: float = time.time()  # mypy404
        self.freshen_pwd: float = 0.0
        self.stopping = False
        self.nreq: int = -1  # mypy404
        self.nbyte: int = 0  # mypy404
        self.u2idx: Optional[U2idx] = None
+        self.lf_url: Optional[Pattern[str]] = self.args.lf_url
        self.log_func: "Util.RootLogger" = hsrv.log  # mypy404
        self.log_src: str = "httpconn"  # mypy404
-        self.lf_url: Optional[Pattern[str]] = (
-            re.compile(self.args.lf_url) if self.args.lf_url else None
-        )  # mypy404
        self.set_rproxy()

    def shutdown(self) -> None:
@@ -166,7 +158,7 @@ class HttpConn(object):
            try:
                assert ssl  # type: ignore  # !rm
                ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
-                ctx.load_cert_chain(self.args.cert)
+                ctx.load_cert_chain(self.args.cert, self.args.certkey)
                if self.args.ssl_ver:
                    ctx.options &= ~self.args.ssl_flags_en
                    ctx.options |= self.args.ssl_flags_de
@@ -203,12 +195,12 @@ class HttpConn(object):
            except Exception as ex:
                em = str(ex)

-                if "ALERT_CERTIFICATE_UNKNOWN" in em:
-                    # android-chrome keeps doing this
-                    pass
+                if "ALERT_" in em:
+                    self.log("client refused our TLS cert or config: " + em, c=6)

                else:
-                    self.log("handshake\033[0m " + em, c=5)
+                    t = "https-handshake failed, probably due to client:\n"
+                    self.log(t + min_ex(), c=5)

                return

@@ -221,6 +213,21 @@ class HttpConn(object):
            if not self.cli.run():
                return

+            if self.sr.te == 1:
+                self.log("closing socket (leftover TE)", "90")
+                return
+
+            if (
+                "content-length" in self.cli.headers
+                and int(self.cli.headers["content-length"]) != self.sr.nb
+            ):
+                self.log("closing socket (CL mismatch)", "90")
+                return
+
+            # note: proxies reject PUT sans Content-Length; illegal for HTTP/1.1
+
+            self.sr.nb = self.sr.te = 0
+
            if self.u2idx:
                self.hsrv.put_u2idx(str(self.addr), self.u2idx)
                self.u2idx = None
@@ -1,8 +1,7 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import hashlib
-import math
 import os
 import re
 import socket
@@ -55,8 +54,14 @@ except SyntaxError:
    )
    sys.exit(1)

+from .authsrv import LEELOO_DALLAS
 from .httpconn import HttpConn
+from .ico import Ico
 from .metrics import Metrics
+from .mtag import HAVE_FFMPEG
+from .sutil import gfilter2
+from .th_cli import ThumbCli
+from .th_srv import HAVE_PIL, HAVE_VIPS
 from .u2idx import U2idx
 from .util import (
    E_SCK,
@@ -133,16 +138,20 @@ class HttpSrv(object):
        self.bans: dict[str, int] = {}
        self.aclose: dict[str, int] = {}

+        self.thumbcli: Optional[ThumbCli] = None
+        self.ico: Ico = Ico(self.args)
+
        dli: dict[str, tuple[float, int, "VFS", str, str]] = {}  # info
        dls: dict[str, tuple[float, int]] = {}  # state
        self.dli = self.tdli = dli
        self.dls = self.tdls = dls
-        self.iiam = '<img src="%s.cpr/iiam.gif?cache=i" />' % (self.args.SRS,)
+        self.iiam = '<img src="%s.cpr/w/iiam.gif?cache=i" />' % (self.args.SRS,)

        self.bound: set[tuple[str, int]] = set()
        self.name = "hsrv" + nsuf
        self.mutex = threading.Lock()
        self.u2mutex = threading.Lock()
+        self.bad_ver = False
        self.stopping = False

        self.tp_nthr = 0  # actual
@@ -170,6 +179,8 @@ class HttpSrv(object):
        self.u2idx_free: dict[str, U2idx] = {}
        self.u2idx_n = 0

+        self.wopi_files: dict[str, dict[str, Any]] = {}
+
        assert jinja2  # type: ignore  # !rm
        env = jinja2.Environment()
        env.loader = jinja2.FunctionLoader(lambda f: load_jinja2_resource(self.E, f))
@@ -185,9 +196,11 @@ class HttpSrv(object):
            "shares",
            "splash",
            "svcs",
+            "wopi",
        ]
        self.j2 = {x: env.get_template(x + ".html") for x in jn}
        self.j2["opds"] = env.get_template("opds.xml")
+        self.j2["opds_osd"] = env.get_template("opds_osd.xml")
        self.prism = has_resource(self.E, "web/deps/prism.js.gz")

        if self.args.ipu:
@@ -201,6 +214,7 @@ class HttpSrv(object):
            self.ipr = None

        self.ipa_nm = build_netmap(self.args.ipa)
+        self.ipar_nm = build_netmap(self.args.ipar)
        self.xff_nm = build_netmap(self.args.xff_src)
        self.xff_lan = build_netmap("lan")

@@ -227,16 +241,23 @@ class HttpSrv(object):
            if self.args.log_thrs:
                start_log_thrs(self.log, self.args.log_thrs, nid)

-        self.th_cfg: dict[str, set[str]] = {}
-        Daemon(self.post_init, "hsrv-init2")
+        if (HAVE_PIL or HAVE_VIPS or HAVE_FFMPEG) and not self.args.no_thumb:
+            Daemon(self.post_init, "hsrv-init2")

    def post_init(self) -> None:
        try:
            x = self.broker.ask("thumbsrv.getcfg")
-            self.th_cfg = x.get()
+            self.thumbcli = ThumbCli(self, x.get())
        except:
            pass

+    def set_th_cfg(self, c: dict[str, set[str]], opts: tuple[bool]) -> None:
+        self.args.th_no_jxl = opts[0]
+        self.thumbcli = ThumbCli(self, c)
+
+    def set_bad_ver(self) -> None:
+        self.bad_ver = True
+
    def set_netdevs(self, netdevs: dict[str, Netdev]) -> None:
        ips = set()
        for ip, _ in self.bound:
@@ -296,7 +317,7 @@ class HttpSrv(object):

        self.srvs.append(sck)
        self.bound.add((ip, port))
-        self.nclimax = math.ceil(self.args.nc * 1.0 / nlisteners)
+        self.nclimax = -int(-self.args.nc // nlisteners)
        Daemon(
            self.thr_listen,
            "httpsrv-n{}-listen-{}-{}".format(self.nid or "0", ip, port),
@@ -383,8 +404,8 @@ class HttpSrv(object):
                if nloris < nconn / 2:
                    continue

-                t = "slowloris (idle-conn): {} banned for {} min"
-                self.log(self.name, t.format(ip, self.args.loris, nclose), 1)
+                t = "slow%s (idle-conn): %s banned for %d min"  # slowloris
+                self.log(self.name, t % ("loris", ip, self.args.loris), 1)
                self.bans[ip] = int(time.time() + self.args.loris * 60)

            if self.args.log_conn:
@@ -640,3 +661,73 @@ class HttpSrv(object):

        self.tdli = dli
        self.tdls = dls
+
+    def pregen_thumbs(self) -> None:
+        Daemon(self._pregen_thumbs, "th_pregen")
+
+    def _pregen_thumbs(self) -> None:
+        def log(msg, n):
+            self.log("thumb-pregen", msg, n)
+
+        if getattr(self, "_pregen", False):
+            log("already running", 1)
+            return
+
+        self._pregen = True
+
+        for n in range(9999999):
+            x = self.broker.ask("up2k.is_busy")
+            zb, zi = x.get()
+            if zi:
+                break
+            if not n:
+                log("waiting for up2k to finish initializing", 6)
+            time.sleep(1 if n < 10 else 5 if n < 300 else 15)
+
+        if not self.thumbcli:
+            log("no thumbcli", 1)
+            return
+
+        if self.args.th_pre_rl:
+            try:
+                self.broker.hub.thumbsrv.log = self.broker.hub.thumbsrv._slog
+            except:
+                pass
+
+        nfiles = 0
+        t0 = time.time()
+        scandir = not self.args.no_scandir
+        for vn in self.asrv.vfs.all_nodes.values():
+            fmts = vn.flags.get("th_pregen", "")
+            if not fmts:
+                continue
+            log("starting for volume /%s" % (vn.vpath,), 6)
+            g = vn.walk("", "/", [], LEELOO_DALLAS, [[True]], 2, scandir, False, False)
+            g = gfilter2(g, self, vn.vpath, fmts.split(","))
+            for f in g:
+                nfiles += 1
+                if not nfiles % 256:
+                    now = time.time()
+                    for n in range(9999999):
+                        x = self.broker.ask("up2k.is_busy")
+                        zb, zi = x.get()
+                        if not zb:
+                            if n:
+                                t0 += time.time() - now
+                            break
+                        if not n:
+                            log("waiting for up2k to finish indexing", 6)
+                        time.sleep(5)
+
+        if self.args.th_pre_rl:
+            try:
+                self.broker.hub.thumbsrv.log = self.broker.hub.thumbsrv._log
+            except:
+                pass
+
+        t = "finished; %d files in %d seconds"
+        log(t % (nfiles, time.time() - t0), 6)
+        self._pregen = False
+
+        if self.args.exit == "thgen":
+            self.broker.say("sigterm")
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse  # typechk
 import colorsys
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import errno
 import os
@@ -16,7 +16,7 @@ from .multicast import MC_Sck, MCast
 from .util import IP6_LL, CachedSet, Daemon, Netdev, list_ips, min_ex

 try:
-    if os.getenv("PRTY_SYS_ALL") or os.getenv("PRTY_SYS_DNSLIB"):
+    if os.environ.get("PRTY_SYS_ALL") or os.environ.get("PRTY_SYS_DNSLIB"):
        raise ImportError()
    from .stolen.dnslib import (
        AAAA,
@@ -51,7 +51,6 @@ except ImportError:
        SRV,
        TXT,
        A,
-        Bimap,
        DNSHeader,
        DNSQuestion,
        DNSRecord,
@@ -66,7 +65,7 @@ if TYPE_CHECKING:
 if True:  # pylint: disable=using-constant-test
    from typing import Any, Optional, Union

-if os.getenv("PRTY_MODSPEC"):
+if os.environ.get("PRTY_MODSPEC"):
    from inspect import getsourcefile

    print("PRTY_MODSPEC: dnslib:", getsourcefile(A))
@@ -110,7 +109,7 @@ class MDNS(MCast):
        self.ttl = 300

        if not self.args.zm_nwa_1 and DNS_VND:
-            set_avahi_379()
+            set_avahi_379()  # type: ignore

        zs = self.args.zm_fqdn or (self.args.name + ".local")
        zs = zs.replace("--name", self.args.name).rstrip(".") + "."
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import json
 import time
@@ -17,10 +17,10 @@ class Metrics(object):
        self.hsrv = hsrv

    def tx(self, cli: "HttpCli") -> bool:
-        if not cli.avol:
+        args = cli.args
+        if not cli.avol and cli.uname.lower() not in args.stats_u_set:
            raise Pebkac(403, "'stats' not allowed for user " + cli.uname)

-        args = cli.args
        if not args.stats:
            raise Pebkac(403, "the stats feature is not enabled in server config")

@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse
 import json
@@ -15,8 +15,12 @@ from .authsrv import VFS
 from .bos import bos
 from .util import (
    FFMPEG_URL,
+    LOG,
    REKOBO_LKEY,
+    SCWD,
    VF_CAREFUL,
+    absreal,
+    expand_osenv_c,
    fsenc,
    gzip,
    min_ex,
@@ -26,6 +30,9 @@ from .util import (
    sfsenc,
    uncyg,
    wunlink,
+    zip_fi,
+    zip_lim,
+    zip_readf,
 )

 if True:  # pylint: disable=using-constant-test
@@ -45,29 +52,42 @@ except:
    HAVE_MUTAGEN = False


-def have_ff(scmd: str) -> bool:
-    if ANYWIN:
+def have_ff(name: str) -> list[bytes]:
+    uname = name.upper()
+    if os.environ.get("PRTY_NO_" + uname):
+        return []
+
+    ebin = os.environ.get("PRTY_%s_BIN" % (uname,))
+    try:
+        scmd = (ebin or name).decode("utf-8")
+    except:
+        scmd: str = ebin or name
+
+    if ANYWIN and not ebin:
        scmd += ".exe"

    if PY2:
-        print("# checking {}".format(scmd))
-        acmd = (scmd + " -version").encode("ascii").split(b" ")
+        print("# checking %s" % (scmd,))
+        bcmd = scmd.encode("utf-8")
        try:
-            sp.Popen(acmd, stdout=sp.PIPE, stderr=sp.PIPE).communicate()
-            return True
+            sp.Popen([bcmd, b"-version"], stdout=sp.PIPE, stderr=sp.PIPE).communicate()
+            return [bcmd]
        except:
-            return False
+            return []
    else:
-        return bool(shutil.which(scmd))
+        ret = shutil.which(scmd)
+        return [absreal(ret).encode("utf-8")] if ret else []


-HAVE_FFMPEG = not os.environ.get("PRTY_NO_FFMPEG") and have_ff("ffmpeg")
-HAVE_FFPROBE = not os.environ.get("PRTY_NO_FFPROBE") and have_ff("ffprobe")
+HAVE_FFMPEG = have_ff("ffmpeg")
+HAVE_FFPROBE = have_ff("ffprobe")
+TH_BWRAP = []

-CBZ_PICS = set("png jpg jpeg gif bmp tga tif tiff webp avif".split())
+CBZ_PICS = set("png jpg jpeg gif bmp tga tif tiff webp avif jxl".split())
 CBZ_01 = re.compile(r"(^|[^0-9v])0+[01]\b")

 FMT_AU = set("mp3 ogg flac wav".split())
+M4A = set("aac m4a m4b m4r".split())


 class MParser(object):
@@ -85,7 +105,7 @@ class MParser(object):

        while True:
            try:
-                bp = os.path.expanduser(args)
+                bp = os.path.expanduser(expand_osenv_c(args))
                if WINDOWS:
                    bp = uncyg(bp)

@@ -132,6 +152,7 @@ class MParser(object):
 def au_unpk(
    log: "NamedLogger", fmt_map: dict[str, str], abspath: str, vn: Optional[VFS] = None
 ) -> str:
+    fd = 0
    ret = ""
    maxsz = 1024 * 1024 * 64
    try:
@@ -156,7 +177,7 @@ def au_unpk(
            zil = [x for x in zil if x.filename.lower().split(".")[-1] == au]
            if not zil:
                raise Exception("no audio inside zip")
-            fi = zf.open(zil[0])
+            fi = zf.open(zip_lim(zil[0], maxsz))

        elif pk == "cbz":
            import zipfile
@@ -168,13 +189,20 @@ def au_unpk(
            znil = [x for x in znil if "cover" in x[0]] or znil
            znil = [x for x in znil if CBZ_01.search(x[0])] or znil
            t = "cbz: %d files, %d hits" % (nf, len(znil))
-            using = sorted(znil)[0][1].filename
-            if znil:
-                t += ", using " + using
-            log(t)
            if not znil:
                raise Exception("no images inside cbz")
-            fi = zf.open(using)
+            zi = sorted(znil)[0][1]
+            if znil:
+                t += ", using " + zi.filename
+            log(t)
+            fi = zf.open(zip_lim(zi, maxsz))
+
+        elif pk == "kra" or pk == "ora":
+            import zipfile
+
+            zf = zipfile.ZipFile(abspath, "r")
+            zi = zip_fi(zf, "mergedimage.png", maxsz)
+            fi = zf.open(zi)

        elif pk == "epub":
            fi = get_cover_from_epub(log, abspath)
@@ -185,6 +213,7 @@ def au_unpk(

        fsz = 0
        with os.fdopen(fd, "wb") as fo:
+            fd = 0
            while True:
                buf = fi.read(32768)
                if not buf:
@@ -199,6 +228,8 @@ def au_unpk(
        return ret

    except Exception as ex:
+        if fd:
+            os.close(fd)
        if ret:
            t = "failed to decompress file %r: %r"
            log(t % (abspath, ex))
@@ -208,18 +239,37 @@ def au_unpk(
        return abspath


+def bwrap(prog: bytes, ap_in: bytes, ap_out: bytes) -> list[bytes]:
+    if not TH_BWRAP:
+        return [prog]
+    ret = TH_BWRAP + [b"--ro-bind", prog, prog, b"--ro-bind", ap_in, ap_in]
+    if ap_out:
+        zs = ap_out.rsplit(b"/", 1)[0]
+        ret += [b"--bind", zs, zs]
+    ret.append(prog)
+    return ret
+
+
+def bwrap_fail(serr: str) -> None:
+    if "bwrap:" in serr:
+        LOG[0]("root", "ffmpeg failed due to --th-bwrap;\n  " + serr, 3)
+
+
 def ffprobe(
    abspath: str, timeout: int = 60
 ) -> tuple[dict[str, tuple[int, Any]], dict[str, list[Any]], list[Any], dict[str, Any]]:
-    cmd = [
-        b"ffprobe",
+    # ffprobe -hide_banner -show_streams -show_format --
+    bap = fsenc(abspath)
+    cmd = bwrap(HAVE_FFPROBE[0], bap, b"") + [
        b"-hide_banner",
        b"-show_streams",
        b"-show_format",
        b"--",
-        fsenc(abspath),
+        bap,
    ]
-    rc, so, se = runcmd(cmd, timeout=timeout, nice=True, oom=200)
+    rc, so, se = runcmd(cmd, cwd=SCWD, timeout=timeout, nice=True, oom=200)
+    if rc and TH_BWRAP:
+        bwrap_fail(se)
    retchk(rc, cmd, se)
    return parse_ffprobe(so)

@@ -259,7 +309,7 @@ def parse_ffprobe(
    md: dict[str, list[Any]] = {}  # raw tags

    is_audio = fmt.get("format_name") in FMT_AU
-    if fmt.get("filename", "").split(".")[-1].lower() in ["m4a", "aac"]:
+    if fmt.get("filename", "").split(".")[-1].lower() in M4A:
        is_audio = True

    # if audio file, ensure audio stream appears first
@@ -385,14 +435,15 @@ def get_cover_from_epub(log: "NamedLogger", abspath: str) -> Optional[IO[bytes]]
    from .dxml import parse_xml

    try:
-        from urlparse import urljoin  # Python2
+        from urlparse import urljoin  # type: ignore  # Python2
    except ImportError:
        from urllib.parse import urljoin  # Python3

    with zipfile.ZipFile(abspath, "r") as z:
        # First open the container file to find the package document (.opf file)
        try:
-            container_root = parse_xml(z.read("META-INF/container.xml").decode())
+            zb = zip_readf(z, "META-INF/container.xml", 2 << 20)
+            container_root = parse_xml(zb.decode())
        except KeyError:
            log("epub: no container file found in %s" % (abspath,))
            return None
@@ -406,7 +457,8 @@ def get_cover_from_epub(log: "NamedLogger", abspath: str) -> Optional[IO[bytes]]

        # Then open the first package document to find the path of the cover image
        try:
-            package_root = parse_xml(z.read(rootfile_path).decode())
+            zb = zip_readf(z, rootfile_path, 2 << 20)
+            package_root = parse_xml(zb.decode())
        except KeyError:
            log("epub: no package document found in %s" % (abspath,))
            return None
@@ -430,7 +482,8 @@ def get_cover_from_epub(log: "NamedLogger", abspath: str) -> Optional[IO[bytes]]
        adjusted_cover_path = urljoin(rootfile_path, coverimage_path)

        try:
-            return z.open(adjusted_cover_path)
+            zi = zip_fi(z, adjusted_cover_path, 2 << 24)
+            return z.open(zi)
        except KeyError:
            t = "epub: cover specified in package document, but doesn't exist: %s"
            log(t % (adjusted_cover_path,))
@@ -465,6 +518,8 @@ class MTag(object):
            "ffprobe" if args.no_mutagen or (HAVE_FFPROBE and EXE) else "mutagen"
        )
        self.can_ffprobe = HAVE_FFPROBE and not args.no_mtag_ff
+        self.read_xattrs = args.have_db_xattr
+        self.get = self._get_xattr if self.read_xattrs else self._get_main
        mappings = args.mtm
        or_ffprobe = " or FFprobe"

@@ -486,7 +541,13 @@ class MTag(object):
                msg = "found FFprobe but it was disabled by --no-mtag-ff"
                self.log(msg, c=3)

+        if self.read_xattrs and not self.usable:
+            t = "don't have the necessary dependencies to read conventional media tags, but will read xattrs"
+            self.log(t)
+            self.usable = True
+
        if not self.usable:
+            self._get = None
            if EXE:
                t = "copyparty.exe cannot use mutagen; need ffprobe.exe to read media tags: "
                self.log(t + FFMPEG_URL)
@@ -523,7 +584,7 @@ class MTag(object):
            ],
            ".tn": ["tracknumber", "trck", "trkn", "track"],
            "genre": ["genre", "tcon", "\u00a9gen"],
-            "date": [
+            "tdate": [
                "original-release-date",
                "release-date",
                "date",
@@ -645,7 +706,36 @@ class MTag(object):

        return r1

-    def get(self, abspath: str) -> dict[str, Union[str, float]]:
+    def _get_xattr(
+        self, abspath: str, vf: dict[str, Any]
+    ) -> dict[str, Union[str, float]]:
+        ret = self._get_main(abspath, vf) if self._get else {}
+        if "db_xattr_no" in vf:
+            try:
+                neg = vf["db_xattr_no"]
+                zsl = os.listxattr(abspath)
+                zsl = [x for x in zsl if x not in neg]
+                for xattr in zsl:
+                    zb = os.getxattr(abspath, xattr)
+                    ret[xattr] = zb.decode("utf-8", "replace")
+            except:
+                self.log("failed to read xattrs from [%s]\n%s" % (abspath, min_ex()), 3)
+        elif "db_xattr_yes" in vf:
+            for xattr in vf["db_xattr_yes"]:
+                if "=" in xattr:
+                    xattr, name = xattr.split("=", 1)
+                else:
+                    name = xattr
+                try:
+                    zs = os.getxattr(abspath, xattr)
+                    ret[name] = zs.decode("utf-8", "replace")
+                except:
+                    pass
+        return ret
+
+    def _get_main(
+        self, abspath: str, vf: dict[str, Any]
+    ) -> dict[str, Union[str, float]]:
        ext = abspath.split(".")[-1].lower()
        if ext not in self.args.au_unpk:
            return self._get(abspath)
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import socket
 import time
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import argparse
 import base64
@@ -14,8 +14,9 @@ try:
    if os.environ.get("PRTY_NO_ARGON2"):
        raise Exception()

-    HAVE_ARGON2 = True
    from argon2 import exceptions as argon2ex
+
+    HAVE_ARGON2 = True
 except:
    HAVE_ARGON2 = False

@@ -1,10 +1,10 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os

 try:
-    if os.getenv("PRTY_SYS_ALL") or os.getenv("PRTY_SYS_QRCG"):
+    if os.environ.get("PRTY_SYS_ALL") or os.environ.get("PRTY_SYS_QRCG"):
        raise ImportError()
    from .stolen.qrcodegen import QrCode

@@ -14,7 +14,7 @@ except ImportError:
    VENDORED = False
    from qrcodegen import QrCode

-if os.getenv("PRTY_MODSPEC"):
+if os.environ.get("PRTY_MODSPEC"):
    from inspect import getsourcefile

    print("PRTY_MODSPEC: qrcode:", getsourcefile(QrCode))
@@ -0,0 +1,911 @@
+# coding: utf-8
+from __future__ import division, print_function, unicode_literals
+
+import errno
+import hashlib
+import logging
+import os
+import select
+import socket
+import time
+
+import paramiko
+import paramiko.common
+import paramiko.sftp_attr
+from paramiko.common import AUTH_FAILED, AUTH_SUCCESSFUL
+from paramiko.sftp import (
+    SFTP_FAILURE,
+    SFTP_NO_SUCH_FILE,
+    SFTP_OK,
+    SFTP_OP_UNSUPPORTED,
+    SFTP_PERMISSION_DENIED,
+)
+from queue import Queue
+
+from .__init__ import ANYWIN, TYPE_CHECKING
+from .authsrv import LEELOO_DALLAS, VFS, AuthSrv
+from .bos import bos
+from .util import (
+    VF_CAREFUL,
+    Daemon,
+    ODict,
+    Pebkac,
+    exclude_dotfiles,
+    exclude_dotfiles_ls,
+    exclude_dothidden,
+    exclude_dothidden_ls,
+    ipnorm,
+    min_ex,
+    read_utf8,
+    relchk,
+    runhook,
+    sanitize_fn,
+    ub64enc,
+    undot,
+    vjoin,
+    wunlink,
+)
+
+if TYPE_CHECKING:
+    from .svchub import SvcHub
+
+if True:  # pylint: disable=using-constant-test
+    from typing import Any, BinaryIO, Optional, Union
+
+SATTR = paramiko.sftp_attr.SFTPAttributes
+
+
+class SSH_Srv(paramiko.ServerInterface):
+    def __init__(self, hub: "SvcHub", addr: Any):
+        self.hub = hub
+        self.args = args = hub.args
+        self.log_func = hub.log
+        self.uname = "*"
+
+        self.addr = addr
+        self.ip = addr[0]
+        if self.ip.startswith("::ffff:"):
+            self.ip = self.ip[7:]
+
+        zsl = []
+        if args.sftp_anon:
+            zsl.append("none")
+        if args.sftp_key2u:
+            zsl.append("publickey")
+        if args.sftp_pw or args.sftp_anon:
+            zsl.append("password")
+        self._auths = ",".join(zsl)
+
+    def log(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log("sftp:%s" % (self.ip,), msg, c)
+
+    def get_allowed_auths(self, username: str) -> str:
+        return self._auths
+
+    def get_banner(self) -> tuple[Optional[str], Optional[str]]:
+        if self.args.sftpv:
+            self.log("get_banner")
+        t = self.args.sftp_banner
+        if not t:
+            return (None, None)
+        if t.startswith("@"):
+            t = read_utf8(self.log, t[1:], False)
+        if t and not t.endswith("\n"):
+            t += "\n"
+        return (t, "en-US")
+
+    def check_channel_request(self, kind: str, chanid: int) -> int:
+        if self.args.sftpv:
+            self.log("channel-request: %r, %r" % (kind, chanid))
+        if kind == "session":
+            return paramiko.common.OPEN_SUCCEEDED
+        return paramiko.common.OPEN_FAILED_ADMINISTRATIVELY_PROHIBITED
+
+    def check_auth_none(self, username: str) -> int:
+        try:
+            return self._check_auth_none(username)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return AUTH_FAILED
+
+    def _check_auth_none(self, uname: str) -> int:
+        args = self.args
+        if uname != args.sftp_anon or not uname:
+            return AUTH_FAILED
+
+        ipn = ipnorm(self.ip)
+        bans = self.hub.bans
+        if ipn in bans:
+            rt = bans[ipn] - time.time()
+            if rt < 0:
+                self.log("client unbanned")
+                del bans[ipn]
+            else:
+                self.log("client is banned")
+                return AUTH_FAILED
+
+        self.uname = "*"
+        self.log("auth-none OK: *")
+        return AUTH_SUCCESSFUL
+
+    def check_auth_password(self, username: str, password: str) -> int:
+        try:
+            return self._check_auth_password(username, password)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return AUTH_FAILED
+
+    def _check_auth_password(self, uname: str, pw: str) -> int:
+        args = self.args
+        if args.sftpv:
+            logpw = pw
+            if args.log_badpwd == 0:
+                logpw = ""
+            elif args.log_badpwd == 2:
+                zb = hashlib.sha512(pw.encode("utf-8", "replace")).digest()
+                logpw = "%" + ub64enc(zb[:12]).decode("ascii")
+            self.log("auth-pw: %r, %r" % (uname, logpw))
+
+        ipn = ipnorm(self.ip)
+        bans = self.hub.bans
+        if ipn in bans:
+            rt = bans[ipn] - time.time()
+            if rt < 0:
+                self.log("client unbanned")
+                del bans[ipn]
+            else:
+                self.log("client is banned")
+                return AUTH_FAILED
+
+        anon = args.sftp_anon
+        if anon and uname == anon:
+            self.uname = "*"
+            self.log("auth-pw OK: *")
+            return AUTH_SUCCESSFUL
+
+        if not args.sftp_pw:
+            return AUTH_FAILED
+
+        if args.usernames:
+            alts = ["%s:%s" % (uname, pw)]
+        else:
+            alts = [pw, uname]
+
+        attempt = "%s:%s" % (uname, pw)
+        uname = ""
+        asrv = self.hub.asrv
+        for zs in alts:
+            zs = asrv.iacct.get(asrv.ah.hash(zs), "")
+            if zs:
+                uname = zs
+                break
+
+        if args.ipu and uname == "*":
+            uname = args.ipu_iu[args.ipu_nm.map(self.ip)]
+        if args.ipr and uname in args.ipr_u:
+            if not args.ipr_u[uname].map(self.ip):
+                logging.warning("username [%s] rejected by --ipr", uname)
+                return AUTH_FAILED
+
+        if not uname or not (asrv.vfs.aread.get(uname) or asrv.vfs.awrite.get(uname)):
+            g = self.hub.gpwd
+            if g.lim:
+                bonk, ip = g.bonk(self.ip, attempt)
+                if bonk:
+                    logging.warning("client banned: invalid passwords")
+                    bans[self.ip] = bonk
+                    try:
+                        # only possible if multiprocessing disabled
+                        self.hub.broker.httpsrv.bans[ip] = bonk  # type: ignore
+                        self.hub.broker.httpsrv.nban += 1  # type: ignore
+                    except:
+                        pass
+            return AUTH_FAILED
+
+        self.uname = uname
+        self.log("auth-pw OK: %s" % (uname,))
+        return AUTH_SUCCESSFUL
+
+    def check_auth_publickey(self, username: str, key: paramiko.PKey) -> int:
+        try:
+            return self._check_auth_publickey(username, key)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return AUTH_FAILED
+
+    def _check_auth_publickey(self, uname: str, key: paramiko.PKey) -> int:
+        args = self.args
+        if args.sftpv:
+            zs = key.get_name() + "," + key.get_base64()[:32]
+            self.log("auth-key: %r, %r" % (uname, zs))
+
+        ipn = ipnorm(self.ip)
+        bans = self.hub.bans
+        if ipn in bans:
+            rt = bans[ipn] - time.time()
+            if rt < 0:
+                self.log("client unbanned")
+                del bans[ipn]
+            else:
+                self.log("client is banned")
+                return AUTH_FAILED
+
+        anon = args.sftp_anon
+        if anon and uname == anon:
+            self.uname = "*"
+            self.log("auth-key OK: *")
+            return AUTH_SUCCESSFUL
+
+        attempt = "%s %s" % (key.get_name(), key.get_base64())
+        ok = args.sftp_key2u.get(attempt) == uname
+
+        if ok and args.ipr and uname in args.ipr_u:
+            if not args.ipr_u[uname].map(self.ip):
+                logging.warning("username [%s] rejected by --ipr", uname)
+                return AUTH_FAILED
+
+        asrv = self.hub.asrv
+        if not ok or not (asrv.vfs.aread.get(uname) or asrv.vfs.awrite.get(uname)):
+            self.log("auth-key REJECTED: %s" % (uname,))
+            return AUTH_FAILED
+
+        self.uname = uname
+        self.log("auth-key OK: %s" % (uname,))
+        return AUTH_SUCCESSFUL
+
+
+class SFTP_FH(paramiko.SFTPHandle):
+    def __init__(self, flags: int = 0) -> None:
+        self.filename = ""
+        self.readfile: Optional[BinaryIO] = None
+        self.writefile: Optional[BinaryIO] = None
+        super(SFTP_FH, self).__init__(flags)
+
+    def stat(self):
+        try:
+            f = self.readfile or self.writefile
+            return SATTR.from_stat(os.fstat(f.fileno()))
+        except OSError as ex:
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def chattr(self, attr):
+        # python doesn't have equivalents to fchown or fchmod, so we have to
+        # use the stored filename
+        if not self.writefile:
+            return SFTP_PERMISSION_DENIED
+        try:
+            paramiko.SFTPServer.set_file_attr(self.filename, attr)
+            return SFTP_OK
+        except OSError as ex:
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+
+class SFTP_Srv(paramiko.SFTPServerInterface):
+    def __init__(self, ssh: paramiko.ServerInterface, *a, **ka):
+        super(SFTP_Srv, self).__init__(ssh, *a, **ka)
+        self.ssh = ssh
+        self.ip: str = ssh.ip  # type: ignore
+        self.hub: "SvcHub" = ssh.hub  # type: ignore
+        self.uname: str = ssh.uname  # type: ignore
+        self.args = self.hub.args
+        self.asrv: "AuthSrv" = self.hub.asrv
+        self.v = self.args.sftpv
+        self.vv = self.args.sftpvv
+
+        if self.uname == LEELOO_DALLAS:
+            raise Exception("send her back")
+
+        self.vols = [
+            vp
+            for vp, vn in self.asrv.vfs.all_vols.items()
+            if self.uname in vn.axs.uread
+            or self.uname in vn.axs.uwrite
+            or self.uname in vn.axs.uget
+        ]
+        self.vis = set()
+        for zs in self.vols:
+            self.vis.add(zs)
+            while zs:
+                zs = zs.rsplit("/", 1)[0] if "/" in zs else ""
+                self.vis.add(zs)
+
+    def log(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log("sftp:%s" % (self.ip,), msg, c)
+
+    def v2a(
+        self,
+        vpath: str,
+        r: bool = False,
+        w: bool = False,
+        m: bool = False,
+        d: bool = False,
+    ) -> tuple[str, VFS, str]:
+        vpath = vpath.replace(os.sep, "/").strip("/")
+        rd, fn = os.path.split(vpath)
+        if relchk(rd):
+            self.log("malicious vpath: %s", vpath)
+            raise Exception("Unsupported characters in [%s]" % (vpath,))
+
+        fn = sanitize_fn(fn or "")
+        vpath = vjoin(rd, fn)
+        vn, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
+        if (
+            w
+            and fn.lower() in vn.flags["emb_all"]
+            and self.uname not in vn.axs.uread
+            and "wo_up_readme" not in vn.flags
+        ):
+            fn = "_wo_" + fn
+            vpath = vjoin(rd, fn)
+            vn, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
+
+        if not vn.realpath:
+            # return "", vn, rem
+            raise OSError(errno.ENOENT, "no filesystem mounted at [/%s]" % (vpath,))
+
+        if "xdev" in vn.flags or "xvol" in vn.flags:
+            ap = vn.canonical(rem)
+            avn = vn.chk_ap(ap)
+            t = "Permission denied in [{}]"
+            if not avn:
+                raise OSError(errno.EPERM, "permission denied in [/%s]" % (vpath,))
+
+            cr, cw, cm, cd, _, _, _, _, _ = avn.uaxs[self.uname]
+            if r and not cr or w and not cw or m and not cm or d and not cd:
+                raise OSError(errno.EPERM, "permission denied in [/%s]" % (vpath,))
+        else:
+            ap = vn.canonical(rem, False)
+
+        if "bcasechk" in vn.flags and not vn.casechk(rem, True):
+            raise OSError(errno.ENOENT, "file does not exist case-sensitively")
+
+        return ap, vn, rem
+
+    def list_folder(self, path: str) -> list[SATTR] | int:
+        try:
+            return self._list_folder(path)
+        except Pebkac as ex:
+            if ex.code == 404:
+                self.log("folder 404: %s" % (path,))
+                return SFTP_NO_SUCH_FILE
+            return SFTP_PERMISSION_DENIED
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _list_folder(self, path: str) -> list[SATTR] | int:
+        if self.v:
+            self.log("ls(%s):" % (path,))
+        path = path.strip("/")
+        try:
+            ap, vn, rem = self.v2a(path, r=True)
+        except Pebkac:
+            try:
+                self.v2a(path, w=True)
+                self.log("ls(%s): [] (write-only)" % (path,))
+                return []  # display write-only folders as empty
+            except:
+                pass
+            if path not in self.vis:
+                self.log("ls(%s): EPERM" % (path,))
+                return SFTP_PERMISSION_DENIED
+            # list of accessible volumes
+            ret = []
+            zi = int(time.time())
+            vst = os.stat_result((16877, -1, -1, 1, 1000, 1000, 8, zi, zi, zi))
+            prefix = path + "/"
+            for vn in self.asrv.vfs.all_nodes.values():
+                if path and not vn.vpath.startswith(prefix):
+                    continue  # vn is parent
+                vname = vn.vpath[len(prefix) :]
+                if "/" in vname or not vname:
+                    continue  # only include vols at current level
+                ret.append(SATTR.from_stat(vst, filename=vn.vpath))
+            ret.sort(key=lambda x: x.filename)
+            self.log("ls(%s): vfs-vols; |%d|" % (path, len(ret)))
+            return ret
+
+        fsroot, vfs_ls, vfs_virt = vn.ls(
+            rem,
+            self.uname,
+            not self.args.no_scandir,
+            [[True, False], [False, True]],
+            throw=True,
+        )
+        vnames = list(vfs_virt)
+        if self.uname not in vn.axs.udot:
+            if "dothidden" in vn.flags and ".hidden" in [x[0] for x in vfs_ls]:
+                vfs_ls = exclude_dothidden_ls(vfs_ls, fsroot)
+                vnames = exclude_dothidden(vnames, fsroot)
+            else:
+                vfs_ls = exclude_dotfiles_ls(vfs_ls)
+                vnames = exclude_dotfiles(vnames)
+        ret = [SATTR.from_stat(x[1], filename=x[0]) for x in vfs_ls]
+        for zs, vn2 in vfs_virt.items():
+            if not vn2.realpath or zs not in vnames:
+                continue
+            st = bos.stat(vn2.realpath)
+            ret.append(SATTR.from_stat(st, filename=zs))
+        ret.sort(key=lambda x: x.filename)
+        self.log("ls(%s): |%d|" % (path, len(ret)))
+        return ret
+
+    def stat(self, path: str) -> SATTR | int:
+        try:
+            return self._stat(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def lstat(self, path: str) -> SATTR | int:
+        try:
+            return self._stat(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _stat(self, vp: str) -> SATTR | int:
+        vp = vp.strip("/")
+        try:
+            ap, vn, _ = self.v2a(vp)
+            if (
+                self.uname not in vn.axs.uread
+                and self.uname not in vn.axs.uwrite
+                and self.uname not in vn.axs.uget
+            ):
+                self.log("stat(%s): EPERM" % (vp,))
+                return SFTP_PERMISSION_DENIED
+            st = bos.stat(ap)
+            self.log("stat(%s): %s" % (vp, st))
+        except:
+            if vp not in self.vis:
+                self.log("stat(%s): ENOENT" % (vp,))
+                return SFTP_NO_SUCH_FILE
+            zi = int(time.time())
+            st = os.stat_result((16877, -1, -1, 1, 1000, 1000, 8, zi, zi, zi))
+            self.log("stat(%s): vfs-vols")
+        return SATTR.from_stat(st)
+
+    def open(self, path: str, flags: int, attr: SATTR) -> paramiko.SFTPHandle | int:
+        try:
+            return self._open(path, flags, attr)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _open(self, vp: str, iflag: int, attr: SATTR) -> paramiko.SFTPHandle | int:
+        if ANYWIN:
+            iflag |= os.O_BINARY
+        if iflag & os.O_WRONLY:
+            rd = False
+            wr = True
+            if iflag & os.O_APPEND:
+                smode = "ab"
+            else:
+                smode = "wb"
+        elif iflag & os.O_RDWR:
+            rd = wr = True
+            if iflag & os.O_APPEND:
+                smode = "a+b"
+            else:
+                smode = "r+b"
+        else:
+            rd = True
+            wr = False
+            smode = "rb"
+
+        try:
+            ap, vn, rem = self.v2a(vp, rd, wr)
+            vf = vn.flags
+        except Pebkac as ex:
+            t = "denied open file [%s], iflag=%s, read=%s, write=%s: %s"
+            self.log(t % (vp, iflag, rd, wr, ex))
+            return SFTP_PERMISSION_DENIED
+
+        self.log("open(%s, %x, %s)" % (vp, iflag, smode))
+
+        if wr:
+            try:
+                st = bos.stat(ap)
+                td = time.time() - st.st_mtime
+                need_unlink = True
+            except:
+                need_unlink = False
+                td = 0
+
+            xbu = vn.flags.get("xbu")
+            if xbu:
+                hr = runhook(
+                    self.log,
+                    None,
+                    self.hub.up2k,
+                    "xbu.sftp",
+                    xbu,
+                    ap,
+                    vp,
+                    "",
+                    "",
+                    "",
+                    0,
+                    0,
+                    "7.3.8.7",
+                    time.time(),
+                    None,
+                )
+                t = hr.get("rejectmsg") or ""
+                if t or hr.get("rc") != 0:
+                    if not t:
+                        t = "upload blocked by xbu server config: %r" % (vp,)
+                    self.log(t, 3)
+                    return SFTP_PERMISSION_DENIED
+
+            self.log("writing to [%s] => [%s]" % (vp, ap))
+
+        if wr and need_unlink:  # type: ignore  # !rm
+            assert td  # type: ignore  # !rm
+            if td >= -1 and td <= self.args.ftp_wt:
+                # within permitted timeframe; allow overwrite or resume
+                do_it = True
+            elif self.args.no_del or self.args.ftp_no_ow:
+                # file too old, or overwrite not allowed; reject
+                do_it = False
+            else:
+                # allow overwrite if user has delete permission
+                do_it = self.uname in vn.axs.udel
+
+            if not do_it:
+                t = "file already exists and no permission to overwrite: %s"
+                self.log(t % (vp,))
+                return SFTP_PERMISSION_DENIED
+
+            # Don't unlink file for append mode
+            elif "a" not in smode:
+                wunlink(self.log, ap, VF_CAREFUL)
+
+        chmod = getattr(attr, "st_mode", None)
+        if chmod is None:
+            chmod = vf.get("chmod_f", 0o644)
+            self.log("open(%s, %x): client did not chmod" % (vp, iflag))
+        else:
+            self.log("open(%s, %x): client set chmod 0%o" % (vp, iflag, chmod))
+
+        try:
+            fd = os.open(ap, iflag, chmod)
+        except OSError as ex:
+            t = "failed to os.open [%s] -> [%s] with iflag [%s] and chmod [%s]: %r"
+            self.log(t % (vp, ap, iflag, chmod, ex), 3)
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+        if iflag & os.O_CREAT:
+            paramiko.SFTPServer.set_file_attr(ap, attr)
+
+        try:
+            f = os.fdopen(fd, smode)
+        except OSError as ex:
+            t = "failed to os.fdpen [%s] -> [%s] with smode [%s]: %r"
+            self.log(t % (vp, ap, smode, ex), 3)
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+        ret = SFTP_FH(iflag)
+        ret.filename = ap
+        ret.readfile = f if rd else None
+        ret.writefile = f if wr else None
+        return ret
+
+    def remove(self, path: str) -> int:
+        try:
+            return self._remove(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _remove(self, vp: str) -> int:
+        self.log("rm(%s)" % (vp,))
+        if self.args.no_del:
+            self.log("The delete feature is disabled in server config")
+            return SFTP_PERMISSION_DENIED
+        try:
+            self.hub.up2k.handle_rm(self.uname, self.ip, [vp], [], False, False)
+            self.log("rm(%s): ok" % (vp,))
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied delete [%s]: %s"
+            self.log(t % (vp, ex))
+            if str(ex).startswith("file not found"):
+                return SFTP_NO_SUCH_FILE
+            try:
+                # write-only client trying to rm before upload?
+                ap, vn, _ = self.v2a(vp)
+                if (
+                    self.uname not in vn.axs.uread
+                    and self.uname not in vn.axs.uwrite
+                    and self.uname not in vn.axs.uget
+                ):
+                    self.log("rm(%s): EPERM" % (vp,))
+                    return SFTP_PERMISSION_DENIED
+                if not bos.path.exists(ap):
+                    self.log(" `- file didn't exist; returning ENOENT")
+                    return SFTP_NO_SUCH_FILE
+            except:
+                pass
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            self.log("failed: rm(%s): %r" % (vp, ex))
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def rename(self, oldpath: str, newpath: str) -> int:
+        try:
+            return self._rename(oldpath, newpath)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _rename(self, svp: str, dvp: str) -> int:
+        self.log("mv(%s, %s)" % (svp, dvp))
+        if self.args.no_mv:
+            self.log("The rename/move feature is disabled in server config")
+        svp = svp.strip("/")
+        dvp = dvp.strip("/")
+        try:
+            self.hub.up2k.handle_mv("", self.uname, self.ip, svp, dvp)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied rename [%s] to [%s]: %s"
+            self.log(t % (svp, dvp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            self.log("mv(%s, %s): %r" % (svp, dvp, ex))
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def mkdir(self, path: str, attr: SATTR) -> int:
+        try:
+            return self._mkdir(path, attr)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _mkdir(self, vp: str, attr: SATTR) -> int:
+        self.log("mkdir(%s)" % (vp,))
+        try:
+            vn, rem = self.asrv.vfs.get(vp, self.uname, False, True)
+            ap = vn.canonical(rem, False)
+            bos.makedirs(ap, vf=vn.flags)  # filezilla expects this
+            if attr is not None:
+                paramiko.SFTPServer.set_file_attr(ap, attr)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied mkdir [%s]: %s"
+            self.log(t % (vp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            self.log("mkdir(%s): %r" % (vp, ex))
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def rmdir(self, path: str) -> int:
+        try:
+            return self._rmdir(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _rmdir(self, vp: str) -> int:
+        self.log("rmdir(%s)" % (vp,))
+        try:
+            vn, rem = self.asrv.vfs.get(vp, self.uname, False, False, will_del=True)
+            ap = os.path.join(vn.realpath, rem)
+            bos.rmdir(ap)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied rmdir [%s]: %s"
+            self.log(t % (vp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            self.log("rmdir(%s): %r" % (vp, ex))
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def chattr(self, path: str, attr: SATTR) -> int:
+        try:
+            return self._chattr(path, attr)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _chattr(self, vp: str, attr: SATTR) -> int:
+        self.log("chattr(%s, %s)" % (vp, attr))
+        try:
+            vn, rem = self.asrv.vfs.get(vp, self.uname, False, True, will_del=True)
+            ap = os.path.join(vn.realpath, rem)
+            paramiko.SFTPServer.set_file_attr(ap, attr)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied chattr [%s]: %s"
+            self.log(t % (vp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            self.log("chattr(%s): %r" % (vp, ex))
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def symlink(self, target_path: str, path: str) -> int:
+        return SFTP_OP_UNSUPPORTED
+
+    def readlink(self, path: str) -> str | int:
+        return path
+
+    def canonicalize(self, path: str) -> str:
+        return "/%s" % (undot(path),)
+
+
+class Transport2(paramiko.Transport):
+    def __init__(self, cli, addr):
+        self.addr = addr
+        super(Transport2, self).__init__(cli)
+
+    def _check_banner(self):
+        try:
+            return super()._check_banner()
+        except Exception as ex:
+            if "Error reading SSH protocol bann" in str(ex):
+                self.logger.warning("%s rejected: %s" % (self.addr, ex))
+                self.addr = None
+            else:
+                raise
+
+
+class Sftpd(object):
+    def __init__(self, hub: "SvcHub") -> None:
+        self.hub = hub
+        self.args = args = hub.args
+        self.log_func = hub.log
+        self.srv: list[socket.socket] = []
+        self.bound: list[str] = []
+        self.hs_q = Queue()
+        self.sessions = {}
+
+        ips = args.sftp_i
+        if "::" in ips:
+            ips.append("0.0.0.0")
+
+        ips = [x for x in ips if not x.startswith(("unix:", "fd:"))]
+
+        if args.sftp4:
+            ips = [x for x in ips if ":" not in x]
+
+        if not ips:
+            self.log("cannot start sftp-server; no compatible IPs in -i", 1)
+            return
+
+        self.hostkeys = []
+        hostkeytypes = (
+            ("ed25519", "Ed25519Key", {}),  # best
+            ("ecdsa", "ECDSAKey", {"bits": 384}),
+            ("rsa", "RSAKey", {"bits": 4096}),
+            ("dsa", "DSSKey", {}),  # worst
+        )
+        for fname, aname, opts in hostkeytypes:
+            fpath = "%s/ssh_host_%s_key" % (args.sftp_hostk, fname.lower())
+            try:
+                pkey = getattr(paramiko, aname).from_private_key_file(fpath)
+            except Exception as ex:
+                try:
+                    genfun = getattr(paramiko, aname).generate
+                except Exception as ex2:
+                    if args.sftpv or fname not in ("dsa", "ed25519"):
+                        # dsa dropped in 4.0
+                        # ed25519 not supported yet
+                        self.log("cannot generate %s hostkey: %r" % (aname, ex2), 3)
+                    continue
+                self.log("generating hostkey [%s] due to %r" % (fpath, ex))
+                pkey = genfun(**opts)
+                pkey.write_private_key_file(fpath)
+                pkey = getattr(paramiko, aname).from_private_key_file(fpath)
+            self.hostkeys.append(pkey)
+            if args.sftpv:
+                self.log("loaded hostkey %r" % (pkey,))
+
+        ips = list(ODict.fromkeys(ips))  # dedup
+
+        for ip in ips:
+            self._bind(ip)
+
+        self.log("listening @ %s port %s" % (self.bound, args.sftp))
+
+        for n in range(self.args.sftp_hs_n):
+            Daemon(self.acceptor, "sftp-hs-%d" % (n,))
+
+    def log(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log("sftp", msg, c)
+
+    def _bind(self, ip: str) -> None:
+        port = self.args.sftp
+        try:
+            ipv = socket.AF_INET6 if ":" in ip else socket.AF_INET
+            srv = socket.socket(ipv, socket.SOCK_STREAM)
+            if not ANYWIN or self.args.reuseaddr:
+                srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            srv.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+            srv.settimeout(0)  # == srv.setblocking(False)
+            try:
+                srv.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, False)
+            except:
+                pass  # will create another ipv4 socket instead
+            if getattr(self.args, "freebind", False):
+                srv.setsockopt(socket.SOL_IP, socket.IP_FREEBIND, 1)
+            srv.bind((ip, port))
+            srv.listen(10)
+            self.srv.append(srv)
+            self.bound.append(ip)
+        except Exception as ex:
+            if ip == "0.0.0.0" and "::" in self.bound:
+                try:
+                    srv.close()  # type: ignore
+                except:
+                    pass
+                return  # dualstack
+            self.log("could not listen on (%s,%s): %r" % (ip, port, ex), 3)
+
+    def _accept(self, srv: socket.socket) -> None:
+        cli, addr = srv.accept()
+        # cli.settimeout(0)  # == srv.setblocking(False)
+        self.log("%r is connecting" % (addr,))
+        self.hs_q.put((cli, addr))
+
+    def acceptor(self) -> None:
+        while True:
+            try:
+                cli, addr = self.hs_q.get()
+                self._accept2(cli, addr)
+            except:
+                time.sleep(1)
+
+    def _accept2(self, cli, addr) -> None:
+        tra = Transport2(cli, addr)
+        for hkey in self.hostkeys:
+            tra.add_server_key(hkey)
+        tra.set_subsystem_handler("sftp", paramiko.SFTPServer, SFTP_Srv)
+        psrv = SSH_Srv(self.hub, addr)
+        try:
+            tra.start_server(server=psrv)
+        except Exception as ex:
+            if tra.addr:
+                self.log("%r could not establish connection: %r" % (addr, ex), 3)
+            cli.close()
+            return
+
+        chan = tra.accept(timeout=self.args.sftp_hs_t)
+        if chan is None:
+            self.log("%r did not open an sftp channel" % (addr,), 3)
+            cli.close()
+            return
+
+        self.sessions[addr] = (chan, tra, psrv)
+        # tra.join()
+        # self.log("%r disconnected" % (addr,))
+
+    def run(self):
+        lgr = logging.getLogger("paramiko.transport")
+        lgr.setLevel(logging.DEBUG if self.args.sftpvv else logging.INFO)
+
+        if self.args.no_poll:
+            fun = self._run_select
+        else:
+            fun = self._run_poll
+        Daemon(fun, "sftpd")
+
+    def _run_select(self):
+        while not self.hub.stopping:
+            rx, _, _ = select.select(self.srv, [], [], 180)
+            for sck in rx:
+                self._accept(sck)
+
+    def _run_poll(self):
+        fd2sck = {}
+        poll = select.poll()
+        for sck in self.srv:
+            fd = sck.fileno()
+            fd2sck[fd] = sck
+            poll.register(fd, select.POLLIN)
+        while not self.hub.stopping:
+            pr = poll.poll(180 * 1000)
+            rx = [fd2sck[x[0]] for x in pr if x[1] & select.POLLIN]
+            for sck in rx:
+                self._accept(sck)
@@ -12,7 +12,7 @@ from types import SimpleNamespace
 from .__init__ import ANYWIN, EXE, TYPE_CHECKING
 from .authsrv import LEELOO_DALLAS, VFS
 from .bos import bos
-from .util import Daemon, absreal, min_ex, pybin, runhook, vjoin
+from .util import Daemon, absreal, min_ex, pybin, runhook

 if True:  # pylint: disable=using-constant-test
    from typing import Any, Union
@@ -89,13 +89,15 @@ class SMB(object):
        smbserver.isInFileJail = self._is_in_file_jail
        self._disarm()

-        ip = next((x for x in self.args.i if ":" not in x), None)
+        zs = " " if self.args.smb6 else ":"
+        ip = next((x for x in self.args.smb_i if zs not in x), None)
        if not ip:
-            self.log("smb", "IPv6 not supported for SMB; listening on 0.0.0.0", 3)
+            self.log("smb", "IPv6 not enabled with --smb6; listening on 0.0.0.0", 3)
            ip = "0.0.0.0"

        port = int(self.args.smb_port)
-        srv = smbserver.SimpleSMBServer(listenAddress=ip, listenPort=port)
+        kw = {"ipv6": True} if ":" in ip else {}
+        srv = smbserver.SimpleSMBServer(listenAddress=ip, listenPort=port, **kw)
        try:
            if self.accs:
                srv.setAuthCallback(self._auth_cb)
@@ -121,6 +123,7 @@ class SMB(object):

        self.srv = srv
        self.stop = srv.stop
+        ip = "[%s]" % (ip,) if kw else ip
        self.log("smb", "listening @ {}:{}".format(ip, port))

    def nlog(self, msg: str, c: Union[int, str] = 0) -> None:
@@ -191,7 +194,7 @@ class SMB(object):
        vfs, rem = self.asrv.vfs.get(vpath, uname, *perms)
        if not vfs.realpath:
            raise Exception("unmapped vfs")
-        return vfs, vjoin(vfs.realpath, rem)
+        return vfs, vfs.canonical(rem, False)

    def _listdir(self, vpath: str, *a: Any, **ka: Any) -> list[str]:
        vpath = vpath.replace("\\", "/").lstrip("/")
@@ -262,10 +265,10 @@ class SMB(object):
                    0,
                    "1.7.6.2",
                    time.time(),
-                    "",
+                    None,
                )
                t = hr.get("rejectmsg") or ""
-                if t or not hr:
+                if t or hr.get("rc") != 0:
                    if not t:
                        t = "blocked by xbu server config: %r" % (vpath,)
                    yeet(t)
@@ -1,11 +1,10 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import errno
 import re
 import select
 import socket
-import time

 from .__init__ import TYPE_CHECKING
 from .multicast import MC_Sck, MCast
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import re
 import stat
@@ -8,6 +8,7 @@ https://github.com/pydron/ifaddr/tree/0.2.0
 """

 import os
+import platform

 from ._shared import IP, Adapter

@@ -16,20 +17,28 @@ def nope(include_unconfigured=False):
    return []


-try:
-    S390X = os.uname().machine == "s390x"
-except:
-    S390X = False
+host_os = platform.system()
+machine = platform.machine()
+py_impl = platform.python_implementation()


-if os.environ.get("PRTY_NO_IFADDR") or S390X:
+if os.environ.get("PRTY_NO_IFADDR"):
+    get_adapters = nope
+elif machine in ("s390x",) or host_os in ("IRIX32",):
    # s390x deadlocks at libc.getifaddrs
+    # irix libc does not have getifaddrs at all
+    print("ifaddr unavailable; can't determine LAN IP: unsupported OS")
+    get_adapters = nope
+elif py_impl in ("GraalVM",):
+    print("ifaddr unavailable; can't determine LAN IP: unsupported interpreter")
    get_adapters = nope
 elif os.name == "nt":
    from ._win32 import get_adapters
 elif os.name == "posix":
    from ._posix import get_adapters
 else:
-    raise RuntimeError("Unsupported Operating System: %s" % os.name)
+    print("ifaddr unavailable; can't determine LAN IP: unsupported OS")
+    get_adapters = nope
+

 __all__ = ["Adapter", "IP", "get_adapters"]
@@ -1,23 +1,27 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os
 import tempfile
 from datetime import datetime

-from .__init__ import CORES
-from .authsrv import VFS, AuthSrv
+from .__init__ import CORES, TYPE_CHECKING
+from .authsrv import LEELOO_DALLAS, VFS, AuthSrv
 from .bos import bos
 from .th_cli import ThumbCli
-from .util import UTC, vjoin, vol_san
+from .th_srv import TH_CH
+from .util import UTC, sigblock, vjoin, vol_san

 if True:  # pylint: disable=using-constant-test
    from typing import Any, Generator, Optional

    from .util import NamedLogger

+if TYPE_CHECKING:
+    from httpsrv import HttpSrv

-TAR_NO_OPUS = set("aac|m4a|mp3|oga|ogg|opus|wma".split("|"))
+
+TAR_NO_OPUS = set("aac|m4a|m4b|m4r|mp3|oga|ogg|opus|wma".split("|"))


 class StreamArc(object):
@@ -41,20 +45,33 @@ class StreamArc(object):
        self.stopped = True


+_pools = {}
+
+
+def close_pools() -> None:
+    for p in list(_pools):
+        try:
+            p.shutdown(wait=False, cancel_futures=True)
+        except:
+            pass
+
+
 def gfilter(
    fgen: Generator[dict[str, Any], None, None],
    thumbcli: ThumbCli,
    uname: str,
    vtop: str,
+    vname: str,
    fmt: str,
 ) -> Generator[dict[str, Any], None, None]:
    from concurrent.futures import ThreadPoolExecutor

    pend = []
-    with ThreadPoolExecutor(max_workers=CORES) as tp:
+    with ThreadPoolExecutor(max_workers=CORES, initializer=sigblock) as tp:
+        _pools[tp] = 1
        try:
            for f in fgen:
-                task = tp.submit(enthumb, thumbcli, uname, vtop, f, fmt)
+                task = tp.submit(enthumb, thumbcli, uname, vtop, vname, f, fmt)
                pend.append((task, f))
                if pend[0][0].done() or len(pend) > CORES * 4:
                    task, f = pend.pop(0)
@@ -78,24 +95,82 @@ def gfilter(
                except:
                    pass
            thumbcli.log("gfilter flushed")
+        _pools.pop(tp, None)
+
+
+def gfilter2(
+    fgen: Generator[
+        tuple[
+            "VFS",
+            str,
+            str,
+            str,
+            list[tuple[str, os.stat_result]],
+            list[tuple[str, os.stat_result]],
+            dict[str, "VFS"],
+        ],
+        None,
+        None,
+    ],
+    hsrv: "HttpSrv",
+    vtop: str,
+    fmts: list[str],
+) -> Generator[dict[str, Any], None, None]:
+    from concurrent.futures import ThreadPoolExecutor
+
+    pend = []
+    with ThreadPoolExecutor(max_workers=CORES, initializer=sigblock) as tp:
+        _pools[tp] = 1
+        for _, _, vpath, apath, files, rd, vd in fgen:
+            if "/.hist/" in vpath:
+                continue
+            fnames = [n[0] for n in files]
+            vpaths = [vpath + "/" + n for n in fnames] if vpath else fnames
+            for vp, fi in zip(vpaths, files):
+                for fmt in fmts:
+                    try:
+                        f = {"vp": vp, "st": fi[1]}
+                        task = tp.submit(
+                            enthumb, hsrv.thumbcli, LEELOO_DALLAS, vtop, "", f, fmt
+                        )
+                        pend.append((task, f))
+                        if pend[0][0].done() or len(pend) > CORES * 4:
+                            task, f = pend.pop(0)
+                            try:
+                                f = task.result(600)
+                            except:
+                                pass
+                            yield f
+                    except:
+                        pass
+        for task, f in pend:
+            try:
+                f = task.result(600)
+            except:
+                pass
+            yield f
+        _pools.pop(tp, None)


 def enthumb(
-    thumbcli: ThumbCli, uname: str, vtop: str, f: dict[str, Any], fmt: str
+    thumbcli: ThumbCli, uname: str, vtop: str, vname: str, f: dict[str, Any], fmt: str
 ) -> dict[str, Any]:
    rem = f["vp"]
    ext = rem.rsplit(".", 1)[-1].lower()
    if (fmt == "mp3" and ext == "mp3") or (fmt == "opus" and ext in TAR_NO_OPUS):
        raise Exception()

-    vp = vjoin(vtop, rem.split("/", 1)[1])
+    if vname:
+        vp = vjoin(vtop, rem.split("/", 1)[1])
+    else:
+        vp = vjoin(vtop, rem)
    vn, rem = thumbcli.asrv.vfs.get(vp, uname, True, False)
    dbv, vrem = vn.get_dbv(rem)
    thp = thumbcli.get(dbv, vrem, f["st"].st_mtime, fmt)
    if not thp:
        raise Exception()

-    ext = "jpg" if fmt == "j" else "webp" if fmt == "w" else fmt
+    ext = fmt if fmt == "wav" else TH_CH.get(fmt[:1], fmt)
    sz = bos.path.getsize(thp)
    st: os.stat_result = f["st"]
    ts = st.st_mtime
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import calendar
 import stat
@@ -8,7 +8,7 @@ import time
 from .authsrv import AuthSrv
 from .bos import bos
 from .sutil import StreamArc, errdesc
-from .util import min_ex, sanitize_fn, spack, sunpack, yieldfile, zlib
+from .util import VPTL_WIN, min_ex, sanitize_to, spack, sunpack, yieldfile, zlib

 if True:  # pylint: disable=using-constant-test
    from typing import Any, Generator, Optional
@@ -104,7 +104,7 @@ def gen_hdr(
    ret += spack(b"<LL", vsz, vsz)

    # windows support (the "?" replace below too)
-    fn = sanitize_fn(fn, "/")
+    fn = sanitize_to(fn, VPTL_WIN)
    bfn = fn.encode("utf-8" if utf8 else "cp437", "replace").replace(b"?", b"_")

    # add ntfs (0x24) and/or unix (0x10) extrafields for utc, add z64 if requested
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import os
 import re
@@ -7,7 +7,7 @@ import socket
 import sys
 import time

-from .__init__ import ANYWIN, PY2, TYPE_CHECKING, unicode
+from .__init__ import ANYWIN, OPENBSD, PY2, TYPE_CHECKING, UNIX, unicode
 from .cert import gencert
 from .qrkode import QrCode, qr2png, qr2svg, qr2txt, qrgen
 from .util import (
@@ -21,7 +21,8 @@ from .util import (
    VF_CAREFUL,
    Netdev,
    atomic_move,
-    get_adapters,
+    chkcmd,
+    list_nics,
    min_ex,
    sunpack,
    termsize,
@@ -422,6 +423,7 @@ class TcpSrv(object):
        self.hub.broker.say("httpsrv.set_netdevs", self.netdevs)
        self.hub.start_zeroconf()
        gencert(self.log, self.args, self.netdevs)
+        self.hub.restart_sftpd()
        self.hub.restart_ftpd()
        self.hub.restart_tftpd()

@@ -459,23 +461,7 @@ class TcpSrv(object):
    def detect_interfaces(self, listen_ips: list[str]) -> dict[str, Netdev]:
        listen_ips = [x for x in listen_ips if not x.startswith(("unix:", "fd:"))]

-        nics = get_adapters(True)
-        eps: dict[str, Netdev] = {}
-        for nic in nics:
-            for nip in nic.ips:
-                ipa = nip.ip[0] if ":" in str(nip.ip) else nip.ip
-                sip = "{}/{}".format(ipa, nip.network_prefix)
-                nd = Netdev(sip, nic.index or 0, nic.nice_name, "")
-                eps[sip] = nd
-                try:
-                    idx = socket.if_nametoindex(nd.name)
-                    if idx and idx != nd.idx:
-                        t = "netdev idx mismatch; ifaddr={} cpython={}"
-                        self.log("tcpsrv", t.format(nd.idx, idx), 3)
-                        nd.idx = idx
-                except:
-                    pass
-
+        eps = list_nics()
        netlist = str(sorted(eps.items()))
        if netlist == self.netlist and self.netdevs:
            return {}
@@ -509,6 +495,13 @@ class TcpSrv(object):
        return eps

    def _extdevs_nix(self) -> Generator[str, None, None]:
+        if UNIX:
+            so, _ = chkcmd(["netstat", "-nrf", "inet"])
+            for ln in so.split("\n"):
+                if not ln.startswith("default"):
+                    continue
+                yield ln.split()[7] if OPENBSD else ln.split()[3]
+            return
        with open("/proc/net/route", "rb") as f:
            next(f)
            for ln in f:
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 try:
    from types import SimpleNamespace
@@ -37,12 +37,12 @@ from .__init__ import EXE, PY2, TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
 from .util import (
-    FN_EMB,
    UTC,
    BytesIO,
    Daemon,
    ODict,
    exclude_dotfiles,
+    exclude_dothidden,
    min_ex,
    runhook,
    set_fperms,
@@ -175,7 +175,7 @@ class Tftpd(object):
            p1, p2 = [int(x) for x in self.args.tftp_pr.split("-")]
            ports = list(range(p1, p2 + 1))

-        ips = self.args.i
+        ips = self.args.tftp_i
        if "::" in ips:
            ips.append("0.0.0.0")

@@ -268,7 +268,7 @@ class Tftpd(object):
        vfs, rem = self.asrv.vfs.get(vpath, "*", *perms)
        if perms[1] and "*" not in vfs.axs.uread and "wo_up_readme" not in vfs.flags:
            zs, fn = vsplit(vpath)
-            if fn.lower() in FN_EMB:
+            if fn.lower() in vfs.flags["emb_all"]:
                vpath = vjoin(zs, "_wo_" + fn)
                vfs, rem = self.asrv.vfs.get(vpath, "*", *perms)

@@ -319,7 +319,11 @@ class Tftpd(object):
        ls = virs + reals

        if "*" not in vn.axs.udot:
-            names = set(exclude_dotfiles([x[2] for x in ls]))
+            zsl = [x[2] for x in ls]
+            if "dothidden" in vn.flags and ".hidden" in zsl:
+                names = set(exclude_dothidden(zsl, fsroot))
+            else:
+                names = set(exclude_dotfiles(zsl))
            ls = [x for x in ls if x[2] in names]

        try:
@@ -379,10 +383,10 @@ class Tftpd(object):
                    0,
                    "8.3.8.7",
                    time.time(),
-                    "",
+                    None,
                )
                t = hr.get("rejectmsg") or ""
-                if t or not hr:
+                if t or hr.get("rc") != 0:
                    if not t:
                        t = "upload blocked by xbu server config: %r" % (vpath,)
                    yeet(t)
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import errno
 import os
@@ -8,7 +8,7 @@ import stat
 from .__init__ import TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
-from .th_srv import EXTS_AC, HAVE_WEBP, thumb_path
+from .th_srv import EXTS_AC, H_PIL_JXL, H_PIL_WEBP, thumb_path
 from .util import Cooldown, Pebkac

 if True:  # pylint: disable=using-constant-test
@@ -20,28 +20,21 @@ if TYPE_CHECKING:

 IOERROR = "reading the file was denied by the server os; either due to filesystem permissions, selinux, apparmor, or similar:\n%r"

+IMG_EXTS = set(["webp", "jpg", "png", "jxl"])
+

 class ThumbCli(object):
-    def __init__(self, hsrv: "HttpSrv") -> None:
+    def __init__(self, hsrv: "HttpSrv", c: dict[str, set[str]]) -> None:
        self.broker = hsrv.broker
        self.log_func = hsrv.log
        self.args = hsrv.args
        self.asrv = hsrv.asrv

        # cache on both sides for less broker spam
-        self.cooldown = Cooldown(self.args.th_poke)
-
-        try:
-            c = hsrv.th_cfg
-            if not c:
-                raise Exception()
-        except:
-            c = {
-                k: set()
-                for k in ["thumbable", "pil", "vips", "raw", "ffi", "ffv", "ffa"]
-            }
+        self.cooldown = Cooldown(self.args.th_poke) if self.args.th_poke else None

        self.thumbable = c["thumbable"]
+        self.thumbable_native = c["thumbable_native"]
        self.fmt_pil = c["pil"]
        self.fmt_vips = c["vips"]
        self.fmt_raw = c["raw"]
@@ -50,8 +43,9 @@ class ThumbCli(object):
        self.fmt_ffa = c["ffa"]

        # defer args.th_ff_jpg, can change at runtime
-        d = next((x for x in self.args.th_dec if x in ("vips", "pil")), None)
-        self.can_webp = HAVE_WEBP or d == "vips"
+        nonpil = next((x for x in self.args.th_dec if x in ("vips", "ff")), None)
+        self.can_webp = (H_PIL_WEBP or nonpil) and not self.args.th_no_webp
+        self.can_jxl = (H_PIL_JXL or nonpil) and not self.args.th_no_jxl

    def log(self, msg: str, c: Union[int, str] = 0) -> None:
        self.log_func("thumbcli", msg, c)
@@ -66,6 +60,10 @@ class ThumbCli(object):
        if is_vid and "dvthumb" in dbv.flags:
            return None

+        is_custom = ext not in self.thumbable_native
+        if is_custom and "dethumb" in dbv.flags:
+            return None
+
        want_opus = fmt in EXTS_AC
        is_au = ext in self.fmt_ffa
        is_vau = want_opus and ext in self.fmt_ffv
@@ -83,29 +81,30 @@ class ThumbCli(object):
        elif want_opus:
            return None

-        is_img = not is_vid and not is_au
+        is_img = not is_vid and not is_au and not is_custom
        if is_img and "dithumb" in dbv.flags:
            return None

        preferred = self.args.th_dec[0] if self.args.th_dec else ""

-        if rem.startswith(".hist/th/") and rem.split(".")[-1] in ["webp", "jpg", "png"]:
+        if rem.startswith(".hist/th/") and rem.split(".")[-1] in IMG_EXTS:
            return os.path.join(ptop, rem)

-        if fmt[:1] in "jw" and fmt != "wav":
-            sfmt = fmt[:1]
+        sfmt = fmt[:1]
+        if sfmt in "jwx" and fmt != "wav":

            if sfmt == "j" and self.args.th_no_jpg:
                sfmt = "w"

            if sfmt == "w":
-                if (
-                    self.args.th_no_webp
-                    or (is_img and not self.can_webp)
-                    or (self.args.th_ff_jpg and (not is_img or preferred == "ff"))
+                if not self.can_webp or (
+                    self.args.th_ff_jpg and (not is_img or preferred == "ff")
                ):
                    sfmt = "j"

+            if sfmt == "x" and not self.can_jxl:
+                sfmt = "w"
+
            vf_crop = dbv.flags["crop"]
            vf_th3x = dbv.flags["th3x"]

@@ -121,7 +120,7 @@ class ThumbCli(object):

            fmt = sfmt

-        elif fmt[:1] == "p" and not is_au and not is_vid:
+        elif sfmt == "p" and not is_au and not is_vid:
            t = "cannot thumbnail %r: png only allowed for waveforms"
            self.log(t % (rem,), 6)
            return None
@@ -133,9 +132,12 @@ class ThumbCli(object):

        tpath = thumb_path(histpath, rem, mtime, fmt, self.fmt_ffa)
        tpaths = [tpath]
-        if fmt[:1] == "w" and fmt != "wav":
+        fmtc = fmt[:1]
+        if fmtc == "w" and fmt != "wav":
            # also check for jpg (maybe webp is unavailable)
            tpaths.append(tpath.rsplit(".", 1)[0] + ".jpg")
+        elif fmtc == "x":
+            tpaths.append(tpath.rsplit(".", 1)[0] + ".webp")

        ret = None
        abort = False
@@ -152,14 +154,15 @@ class ThumbCli(object):
                pass

        if ret:
-            tdir = os.path.dirname(tpath)
-            if self.cooldown.poke(tdir):
-                self.broker.say("thumbsrv.poke", tdir)
+            if self.cooldown:
+                tdir = os.path.dirname(tpath)
+                if self.cooldown.poke(tdir):
+                    self.broker.say("thumbsrv.poke", tdir)

-            if want_opus:
-                # audio files expire individually
-                if self.cooldown.poke(tpath):
-                    self.broker.say("thumbsrv.poke", tpath)
+                if want_opus:
+                    # audio files expire individually
+                    if self.cooldown.poke(tpath):
+                        self.broker.say("thumbsrv.poke", tpath)

            return ret

@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import calendar
 import os
@@ -22,11 +22,20 @@ from .util import (
    quotep,
    s3dec,
    vjoin,
+    vjoins,
 )

 if HAVE_SQLITE3:
    import sqlite3

+try:
+    if os.environ.get("PRTY_NO_UNIDATA"):
+        raise Exception()
+
+    import unicodedata
+except:
+    pass
+
 try:
    from pathlib import Path
 except:
@@ -42,6 +51,14 @@ if PY2:
    range = xrange  # type: ignore


+def norm_icase(txt: str) -> str:
+    return txt.casefold() if txt else txt
+
+
+def norm_nfkc(txt: str) -> str:
+    return unicodedata.normalize("NFKC", txt).casefold() if txt else txt
+
+
 class U2idx(object):
    def __init__(self, hsrv: "HttpSrv") -> None:
        self.log_func = hsrv.log
@@ -54,6 +71,10 @@ class U2idx(object):
            return

        if self.args.srch_icase:
+            if self.args.srch_nfkc:
+                self.normfun = norm_nfkc
+            else:
+                self.normfun = norm_icase
            self._open_db = self._open_db_icase
        else:
            self._open_db = self._open_db_std
@@ -81,7 +102,7 @@ class U2idx(object):

    def _open_db_icase(self, *args, **kwargs):
        db = self._open_db_std(*args, **kwargs)
-        db.create_function("casefold", 1, lambda x: x.casefold() if x else x)
+        db.create_function("casefold", 1, self.normfun)
        return db

    def shutdown(self) -> None:
@@ -322,7 +343,7 @@ class U2idx(object):

            if icase and "casefold(" in q:
                try:
-                    v = unicode(v).casefold()
+                    v = self.normfun(unicode(v))
                except:
                    v = unicode(v).lower()

@@ -437,7 +458,7 @@ class U2idx(object):
                if rd.startswith("//") or fn.startswith("//"):
                    rd, fn = s3dec(rd, fn)

-                vp = vjoin(vjoin(vtop, rd), fn)
+                vp = vjoins(vtop, rd, fn)

                if vp in seen_rps:
                    continue
@@ -1,15 +1,13 @@
 # coding: utf-8
-from __future__ import print_function, unicode_literals
+from __future__ import division, print_function, unicode_literals

 import errno
 import hashlib
 import json
-import math
 import os
 import re
-import shutil
 import stat
-import subprocess as sp
+import sys
 import tempfile
 import threading
 import time
@@ -18,7 +16,7 @@ from copy import deepcopy

 from queue import Queue

-from .__init__ import ANYWIN, MACOS, PY2, TYPE_CHECKING, WINDOWS, E
+from .__init__ import ANYWIN, PY2, TYPE_CHECKING, UNIX, WINDOWS, E
 from .authsrv import LEELOO_DALLAS, SEESLOG, VFS, AuthSrv
 from .bos import bos
 from .cfg import vf_bmap, vf_cmap, vf_vmap
@@ -27,6 +25,7 @@ from .mtag import MParser, MTag
 from .util import (
    E_FS_CRIT,
    E_FS_MEH,
+    HAVE_FICLONE,
    HAVE_SQLITE3,
    SYMTIME,
    VF_CAREFUL,
@@ -53,12 +52,15 @@ from .util import (
    ren_open,
    rmdirs,
    rmdirs_up,
+    runcmd,
    runhook,
    runihook,
    s2hms,
    s3dec,
    s3enc,
    sanitize_fn,
+    set_ap_perms,
+    set_fperms,
    sfsenc,
    spack,
    statdir,
@@ -66,9 +68,11 @@ from .util import (
    ub64enc,
    unhumanize,
    vjoin,
+    vjoins,
    vsplit,
    w8b64dec,
    w8b64enc,
+    winsparse,
    wunlink,
 )

@@ -88,13 +92,17 @@ if True:  # pylint: disable=using-constant-test
 if TYPE_CHECKING:
    from .svchub import SvcHub

+USE_FICLONE = HAVE_FICLONE and sys.version_info < (3, 14)
+if USE_FICLONE:
+    import fcntl
+
 zsg = "avif,avifs,bmp,gif,heic,heics,heif,heifs,ico,j2p,j2k,jp2,jpeg,jpg,jpx,png,tga,tif,tiff,webp"
 ICV_EXTS = set(zsg.split(","))

 zsg = "3gp,asf,av1,avc,avi,flv,m4v,mjpeg,mjpg,mkv,mov,mp4,mpeg,mpeg2,mpegts,mpg,mpg2,mts,nut,ogm,ogv,rm,vob,webm,wmv"
 VCV_EXTS = set(zsg.split(","))

-zsg = "aif,aiff,alac,ape,flac,m4a,mp3,oga,ogg,opus,tak,tta,wav,wma,wv,cbz,epub"
+zsg = "aif,aiff,alac,ape,flac,m4a,m4b,m4r,mka,mp3,oga,ogg,opus,tak,tta,wav,wma,wv,cbz,epub"
 ACV_EXTS = set(zsg.split(","))

 zsg = "nohash noidx xdev xvol"
@@ -124,6 +132,7 @@ class Mpqe(object):
        self,
        mtp: dict[str, MParser],
        entags: set[str],
+        vf: dict[str, Any],
        w: str,
        abspath: str,
        oth_tags: dict[str, Any],
@@ -131,6 +140,7 @@ class Mpqe(object):
        # mtp empty = mtag
        self.mtp = mtp
        self.entags = entags
+        self.vf = vf
        self.w = w
        self.abspath = abspath
        self.oth_tags = oth_tags
@@ -156,6 +166,7 @@ class Up2k(object):
        self.gt0 = 0
        self.gt1 = 0
        self.stop = False
+        self.fika = ""
        self.mutex = threading.Lock()
        self.reload_mutex = threading.Lock()
        self.reload_flag = 0
@@ -164,6 +175,7 @@ class Up2k(object):
        self.pp: Optional[ProgressPrinter] = None
        self.rescan_cond = threading.Condition()
        self.need_rescan: set[str] = set()
+        self.have_e2d = False
        self.db_act = 0.0

        self.reg_mutex = threading.Lock()
@@ -243,6 +255,10 @@ class Up2k(object):
        with self.reload_mutex:
            if self.reload_flag < n:
                self.reload_flag = n
+            if not self.have_e2d and not self.reloading:
+                self.reloading = True
+                Daemon(self._reload_thr)
+                return
        with self.rescan_cond:
            self.rescan_cond.notify_all()

@@ -280,7 +296,7 @@ class Up2k(object):

    def deferred_init(self) -> None:
        all_vols = self.asrv.vfs.all_vols
-        have_e2d = self.init_indexes(all_vols, [], False)
+        self.init_indexes(all_vols, [], False)

        if self.stop:
            # up-mt consistency not guaranteed if init is interrupted;
@@ -298,7 +314,7 @@ class Up2k(object):
            return

        Daemon(self._snapshot, "up2k-snapshot")
-        if have_e2d:
+        if self.have_e2d:
            Daemon(self._hasher, "up2k-hasher")
            Daemon(self._sched_rescan, "up2k-rescan")
            if self.mtag:
@@ -326,6 +342,10 @@ class Up2k(object):
            if not self.stop:
                self.log("uploads are now possible", 2)

+    def is_busy(self) -> tuple[bool, float]:
+        # returns ( currently-busy , have-finished-at-least-once )
+        return bool(self.pp), self.gt1
+
    def get_state(self, get_q: bool, uname: str) -> str:
        mtpq: Union[int, str] = 0
        ups = []
@@ -650,19 +670,20 @@ class Up2k(object):

            for _ in range(2):
                q = "select ip, at from up where ip > '' order by +at limit 1"
-                hits = cur.execute(q).fetchall()
-                if not hits:
-                    break
+                with self.mutex:
+                    hits = cur.execute(q).fetchall()
+                    if not hits:
+                        break

-                remains = hits[0][1] - cutoff
-                if remains > 0:
-                    timeout = min(timeout, now + remains)
-                    break
+                    remains = hits[0][1] - cutoff
+                    if remains > 0:
+                        timeout = min(timeout, now + remains)
+                        break

-                q = "update up set ip = '' where ip > '' and at <= %d"
-                cur.execute(q % (cutoff,))
-                zi = cur.rowcount
-                cur.connection.commit()
+                    q = "update up set ip = '' where ip > '' and at <= %d"
+                    cur.execute(q % (cutoff,))
+                    zi = cur.rowcount
+                    cur.connection.commit()

                t = "forget-ip(%d) removed %d IPs from db [/%s]"
                self.log(t % (maxage, zi, vol.vpath))
@@ -876,7 +897,7 @@ class Up2k(object):

    def init_indexes(
        self, all_vols: dict[str, VFS], scan_vols: list[str], fscan: bool, gid: int = 0
-    ) -> bool:
+    ) -> None:
        if not gid:
            with self.mutex:
                gid = self.gid
@@ -1113,7 +1134,7 @@ class Up2k(object):
        else:
            self.unpp()

-        return have_e2d
+        self.have_e2d = self.have_e2d or have_e2d

    def register_vpath(
        self, ptop: str, flags: dict[str, Any]
@@ -1142,7 +1163,7 @@ class Up2k(object):
        ft = "\033[0;32m{}{:.0}"
        ff = "\033[0;35m{}{:.0}"
        fv = "\033[0;36m{}:\033[90m{}"
-        zs = "bcasechk du_iwho ext_th_d html_head html_head_d html_head_s put_name2 mv_re_r mv_re_t rm_re_r rm_re_t srch_re_dots srch_re_nodot zipmax zipmaxn_v zipmaxs_v"
+        zs = "bcasechk du_iwho emb_all emb_lgs emb_mds ext_th_d html_head html_head_d html_head_s ls_q_m put_name2 mv_re_r mv_re_t rm_re_r rm_re_t oh_f oh_g rw_edit_set srch_re_dots srch_re_nodot th_coversd th_coversl th_covers_set th_coversd_set zipmax zipmaxn_v zipmaxs_v"
        fx = set(zs.split())
        fd = vf_bmap()
        fd.update(vf_cmap())
@@ -1276,6 +1297,14 @@ class Up2k(object):
            self.log(t % (vpath, ex), 1)
            return None

+        if dir_is_empty(self.log_func, not self.args.no_scandir, histpath) and not (
+            ANYWIN or UNIX or "hist_cow" in flags
+        ):
+            try:
+                runcmd([b"chattr", b"+C", fsenc(histpath)], 1)
+            except:
+                pass
+
        try:
            cur = self._open_db_wd(db_path)

@@ -1414,6 +1443,8 @@ class Up2k(object):
                    rtop,
                    rei,
                    reh,
+                    vol.flags["th_coversd"],
+                    vol.flags["th_coversd_set"],
                    n4g,
                    ffat,
                    [],
@@ -1465,6 +1496,23 @@ class Up2k(object):

            return True, bool(n_add or n_rm or do_vac)

+    def _fika(self, db: Dbw) -> None:
+        zs = self.fika
+        self.fika = ""
+        if zs not in self.args.fika:
+            return
+
+        t = "fika(%s); commit %d new files; %d updates"
+        self.log(t % (zs, db.nf, db.n))
+        db.c.connection.commit()
+        db.n = db.nf = 0
+        db.t = time.time()
+
+        self.mutex.release()
+        time.sleep(0.5)
+        self.mutex.acquire()
+        db.t = time.time()
+
    def _build_dir(
        self,
        db: Dbw,
@@ -1474,6 +1522,8 @@ class Up2k(object):
        rcdir: str,
        rei: Optional[Pattern[str]],
        reh: Optional[Pattern[str]],
+        th_cvd: list[str],
+        th_cvds: set[str],
        n4g: bool,
        ffat: bool,
        seen: list[str],
@@ -1499,8 +1549,6 @@ class Up2k(object):
        cv = vcv = acv = ""
        e_d = {}

-        th_cvd = self.args.th_coversd
-        th_cvds = self.args.th_coversd_set
        scan_pr_s = self.args.scan_pr_s

        assert self.pp and self.mem_cur  # !rm
@@ -1517,8 +1565,12 @@ class Up2k(object):
        gl = sorted(g)
        partials = set([x[0] for x in gl if "PARTIAL" in x[0]])
        for iname, inf in gl:
-            if self.stop:
-                return -1, 0, 0
+            if self.fika:
+                if not self.stop:
+                    self._fika(db)
+                if self.stop:
+                    self.fika = "f"
+                    return -1, 0, 0

            rp = rds + iname
            abspath = cdirs + iname
@@ -1563,6 +1615,8 @@ class Up2k(object):
                        rap,
                        rei,
                        reh,
+                        th_cvd,
+                        th_cvds,
                        n4g,
                        fat32,
                        seen,
@@ -1593,7 +1647,7 @@ class Up2k(object):

                rsz += sz
                files.append((sz, lmod, iname))
-                if sz:
+                if sz and th_cvd:
                    liname = iname.lower()
                    ext = liname.rsplit(".", 1)[-1]
                    if (
@@ -1665,8 +1719,12 @@ class Up2k(object):

        seen_files = set([x[2] for x in files])  # for dropcheck
        for sz, lmod, fn in files:
-            if self.stop:
-                return -1, 0, 0
+            if self.fika:
+                if not self.stop:
+                    self._fika(db)
+                if self.stop:
+                    self.fika = "f"
+                    return -1, 0, 0

            rp = rds + fn
            abspath = cdirs + fn
@@ -2193,7 +2251,7 @@ class Up2k(object):
            abspath = djoin(ptop, rd, fn)
            self.pp.msg = "c%d %s" % (nq, abspath)
            if not mpool:
-                n_tags = self._tagscan_file(cur, entags, w, abspath, ip, at, un)
+                n_tags = self._tagscan_file(cur, entags, flags, w, abspath, ip, at, un)
            else:
                oth_tags = {}
                if ip:
@@ -2203,7 +2261,7 @@ class Up2k(object):
                if un:
                    oth_tags["up_by"] = un

-                mpool.put(Mpqe({}, entags, w, abspath, oth_tags))
+                mpool.put(Mpqe({}, entags, flags, w, abspath, oth_tags))
                with self.mutex:
                    n_tags = len(self._flush_mpool(cur))

@@ -2310,9 +2368,10 @@ class Up2k(object):
            return

        entags = self.entags[ptop]
+        vf = self.flags[ptop]

        parsers = {}
-        for parser in self.flags[ptop]["mtp"]:
+        for parser in vf["mtp"]:
            try:
                parser = MParser(parser)
            except:
@@ -2387,7 +2446,7 @@ class Up2k(object):
                    if un:
                        oth_tags["up_by"] = un

-                    jobs.append(Mpqe(parsers, set(), w, abspath, oth_tags))
+                    jobs.append(Mpqe(parsers, set(), vf, w, abspath, oth_tags))
                    in_progress[w] = True

            with self.mutex:
@@ -2518,7 +2577,7 @@ class Up2k(object):
            return

        for _ in range(mpool.maxsize):
-            mpool.put(Mpqe({}, set(), "", "", {}))
+            mpool.put(Mpqe({}, set(), {}, "", "", {}))

        mpool.join()

@@ -2537,7 +2596,7 @@ class Up2k(object):
                        t = "tag-thr: {}({})"
                        self.log(t.format(self.mtag.backend, qe.abspath), "90")

-                    tags = self.mtag.get(qe.abspath) if st.st_size else {}
+                    tags = self.mtag.get(qe.abspath, qe.vf) if st.st_size else {}
                else:
                    if self.args.mtag_vv:
                        t = "tag-thr: {}({})"
@@ -2570,6 +2629,7 @@ class Up2k(object):
        self,
        write_cur: "sqlite3.Cursor",
        entags: set[str],
+        vf: dict[str, Any],
        wark: str,
        abspath: str,
        ip: str,
@@ -2588,7 +2648,7 @@ class Up2k(object):
            return 0

        try:
-            tags = self.mtag.get(abspath) if st.st_size else {}
+            tags = self.mtag.get(abspath, vf) if st.st_size else {}
        except Exception as ex:
            self._log_tag_err("", abspath, ex)
            return 0
@@ -2687,6 +2747,17 @@ class Up2k(object):
            t = "-" * 72
            raise Exception("%s\n%s\n%s" % (t, txt, t))

+    def _sql_ok(self, cur: "sqlite3.Cursor", q: str) -> bool:
+        try:
+            cur.execute(q).fetchone()
+            return True
+        except Exception as ex:
+            if "no such table" not in str(ex):
+                t = "WARNING: database issues, maybe due to server filesystem or hardware (check %sOS-logs):\n  %r -- \033[31m%r\n"
+                t2 = "" if ANYWIN or UNIX else "dmesg / "
+                self.log(t % (t2, q, ex), 3)
+        return False
+
    def _orz(self, db_path: str) -> "sqlite3.Cursor":
        assert sqlite3  # type: ignore  # !rm
        c = sqlite3.connect(
@@ -2860,11 +2931,8 @@ class Up2k(object):

    def _add_dhash_tab(self, cur: "sqlite3.Cursor") -> None:
        # v5 -> v5a
-        try:
-            cur.execute("select d, h from dh limit 1").fetchone()
+        if self._sql_ok(cur, "select d, h from dh limit 1"):
            return
-        except:
-            pass

        for cmd in [
            r"create table dh (d text, h text)",
@@ -2878,11 +2946,8 @@ class Up2k(object):
    def _add_xiu_tab(self, cur: "sqlite3.Cursor") -> None:
        # v5a -> v5b
        # store rd+fn rather than warks to support nohash vols
-        try:
-            cur.execute("select c, w, rd, fn from iu limit 1").fetchone()
+        if self._sql_ok(cur, "select c, w, rd, fn from iu limit 1"):
            return
-        except:
-            pass

        try:
            cur.execute("drop table iu")
@@ -2900,11 +2965,8 @@ class Up2k(object):

    def _add_cv_tab(self, cur: "sqlite3.Cursor") -> None:
        # v5b -> v5c
-        try:
-            cur.execute("select rd, dn, fn from cv limit 1").fetchone()
+        if self._sql_ok(cur, "select rd, dn, fn from cv limit 1"):
            return
-        except:
-            pass

        for cmd in [
            r"create table cv (rd text, dn text, fn text)",
@@ -2939,11 +3001,8 @@ class Up2k(object):

    def _add_ds_tab(self, cur: "sqlite3.Cursor") -> None:
        # v5d -> v5e
-        try:
-            cur.execute("select rd, sz from ds limit 1").fetchone()
+        if self._sql_ok(cur, "select rd, sz from ds limit 1"):
            return
-        except:
-            pass

        for cmd in [
            r"create table ds (rd text, sz int, nf int)",
@@ -2966,6 +3025,7 @@ class Up2k(object):
            raise Pebkac(503, SBUSY % ("fs-reload",))

        got_lock = False
+        self.fika = "u"
        try:
            # bit expensive; 3.9=10x 3.11=2x
            if self.mutex.acquire(timeout=10):
@@ -2993,7 +3053,7 @@ class Up2k(object):
            raise Pebkac(500, "too many xbu relocs, giving up")

        ptop = cj["ptop"]
-        if not self.register_vpath(ptop, cj["vcfg"]):
+        if not self.register_vpath(ptop, cj.pop("vcfg")):
            if ptop not in self.registry:
                raise Pebkac(410, "location unavailable")

@@ -3033,7 +3093,7 @@ class Up2k(object):
                )

            zi = cj["lmod"]
-            bad_mt = zi <= 0 or zi > 0xAAAAAAAA
+            bad_mt = zi <= 0 or zi > (2 << 36)
            if bad_mt or vfs.flags.get("up_ts", "") == "fu":
                # force upload time rather than last-modified
                cj["lmod"] = int(time.time())
@@ -3146,7 +3206,7 @@ class Up2k(object):
                c2 = None
                for cur, dp_dir, dp_fn in lost:
                    t = "forgetting desynced db entry: %r"
-                    self.log(t % ("/" + vjoin(vjoin(vfs.vpath, dp_dir), dp_fn)))
+                    self.log(t % ("/" + vjoins(vfs.vpath, dp_dir, dp_fn)))
                    self.db_rm(cur, vfs.flags, dp_dir, dp_fn, cj["size"])
                    if c2 and c2 != cur:
                        c2.connection.commit()
@@ -3221,7 +3281,25 @@ class Up2k(object):
                    dst = djoin(cj["ptop"], cj["prel"], cj["name"])
                    vsrc = djoin(job["vtop"], job["prel"], job["name"])
                    vsrc = vsrc.replace("\\", "/")  # just for prints anyways
-                    if "done" not in job:
+                    if vfs.lim:
+                        dst, cj["prel"] = vfs.lim.all(
+                            cj["addr"],
+                            cj["prel"],
+                            cj["size"],
+                            cj["ptop"],
+                            djoin(cj["ptop"], cj["prel"]),
+                            self.hub.broker,
+                            reg,
+                            "up2k._get_volsize",
+                        )
+                        bos.makedirs(dst, vf=vfs.flags)
+                        vfs.lim.nup(cj["addr"])
+                        vfs.lim.bup(cj["addr"], cj["size"])
+
+                    if "rvp0" in job and wark in reg:
+                        # xbu reloc; accept wrong path
+                        job["addr"] = cj["addr"]
+                    elif "done" not in job:
                        self.log("unfinished:\n  %r\n  %r" % (src, dst))
                        err = "partial upload exists at a different location; please resume uploading here instead:\n"
                        err += "/" + quotep(vsrc) + " "
@@ -3298,10 +3376,10 @@ class Up2k(object):
                                job["size"],
                                job["addr"],
                                job["at"],
-                                "",
+                                [dwark],
                            )
                            t = hr.get("rejectmsg") or ""
-                            if t or not hr:
+                            if t or hr.get("rc") != 0:
                                if not t:
                                    t = "upload blocked by xbu server config: %r"
                                    t = t % (vp,)
@@ -3311,18 +3389,29 @@ class Up2k(object):
                                x = pathmod(self.vfs, dst, vp, hr["reloc"])
                                if x:
                                    ud1 = (vfs.vpath, job["prel"], job["name"])
+                                    job["rvp0"] = vjoins(*ud1)
                                    pdir, _, job["name"], (vfs, rem) = x
                                    dst = os.path.join(pdir, job["name"])
-                                    job["vcfg"] = vfs.flags
                                    job["ptop"] = vfs.realpath
                                    job["vtop"] = vfs.vpath
                                    job["prel"] = rem
-                                    job["name"] = sanitize_fn(job["name"], "")
+                                    job["name"] = sanitize_fn(job["name"])
                                    ud2 = (vfs.vpath, job["prel"], job["name"])
                                    if ud1 != ud2:
                                        # print(json.dumps(job, sort_keys=True, indent=4))
+                                        job["vcfg"] = vfs.flags
                                        job["hash"] = cj["hash"]
-                                        self.log("xbu reloc1:%d..." % (depth,), 6)
+                                        t = "xbu reloc1=%d ptop=%r vtop=%r prel=%r name=%r"
+                                        t = t % (
+                                            depth,
+                                            job["ptop"],
+                                            job["vtop"],
+                                            job["prel"],
+                                            job["name"],
+                                        )
+                                        self.log(t, 6)
+                                        zs = djoin(job["ptop"], job["prel"])
+                                        bos.makedirs(zs, vf=vfs.flags)
                                        return self._handle_json(job, depth + 1)

                        job["name"] = self._untaken(pdir, job, now)
@@ -3462,7 +3551,15 @@ class Up2k(object):
        fp = djoin(fdir, fname)

        ow = job.get("replace") and bos.path.exists(fp)
-        if ow and "mt" in str(job["replace"]).lower():
+        if ow:
+            replace_arg = str(job["replace"]).lower()
+
+        if ow and "skip" in replace_arg:  # type: ignore
+            self.log("skipping upload, filename already exists: %r" % fp)
+            err = "upload rejected, a file with that name already exists"
+            raise Pebkac(409, err)
+
+        if ow and "mt" in replace_arg:  # type: ignore
            mts = bos.stat(fp).st_mtime
            mtc = job["lmod"]
            if mtc < mts:
@@ -3528,13 +3625,30 @@ class Up2k(object):
        if self.args.nw:
            return

-        linked = False
+        linked = 0
        try:
-            if "reflink" in flags:
-                raise Exception("reflink")
+            if rm and bos.path.exists(dst):
+                wunlink(self.log, dst, flags)
+
            if not is_mv and not flags.get("dedup"):
                raise Exception("dedup is disabled in config")

+            if "reflink" in flags:
+                if not USE_FICLONE:
+                    raise Exception("reflink")  # python 3.14 or newer; no need
+                try:
+                    with open(fsenc(src), "rb") as fi, open(fsenc(dst), "wb") as fo:
+                        fcntl.ioctl(fo.fileno(), fcntl.FICLONE, fi.fileno())
+                        if "fperms" in flags:
+                            set_fperms(fo, flags)
+                except:
+                    if bos.path.exists(dst):
+                        wunlink(self.log, dst, flags)
+                    raise
+                if lmod:
+                    bos.utime_c(self.log, dst, int(lmod), False)
+                return
+
            lsrc = src
            ldst = dst
            fs1 = bos.stat(os.path.dirname(src)).st_dev
@@ -3561,13 +3675,10 @@ class Up2k(object):
                lsrc = lsrc.replace("/", "\\")
                ldst = ldst.replace("/", "\\")

-            if rm and bos.path.exists(dst):
-                wunlink(self.log, dst, flags)
-
            try:
                if "hardlink" in flags:
                    os.link(fsenc(absreal(src)), fsenc(dst))
-                    linked = True
+                    linked = 2
            except Exception as ex:
                self.log("cannot hardlink: " + repr(ex))
                if "hardlinkonly" in flags:
@@ -3586,7 +3697,7 @@ class Up2k(object):
                else:
                    os.symlink(fsenc(lsrc), fsenc(ldst))

-                linked = True
+                linked = 1
        except Exception as ex:
            if str(ex) != "reflink":
                self.log("cannot link; creating copy: " + repr(ex))
@@ -3600,12 +3711,16 @@ class Up2k(object):
                raise Exception(t % (src, fsrc, dst))
            trystat_shutil_copy2(self.log, fsenc(csrc), fsenc(dst))

-        if lmod and (not linked or SYMTIME):
-            bos.utime_c(self.log, dst, int(lmod), False)
+        if linked < 2 and (linked < 1 or SYMTIME):
+            if lmod:
+                bos.utime_c(self.log, dst, int(lmod), False)
+            if "fperms" in flags:
+                set_ap_perms(dst, flags)

    def handle_chunks(
        self, ptop: str, wark: str, chashes: list[str]
    ) -> tuple[list[str], int, list[list[int]], str, float, int, bool]:
+        self.fika = "u"
        with self.mutex, self.reg_mutex:
            self.db_act = self.vol_act[ptop] = time.time()
            job = self.registry[ptop].get(wark)
@@ -3708,6 +3823,7 @@ class Up2k(object):
    def confirm_chunks(
        self, ptop: str, wark: str, written: list[str], locked: list[str]
    ) -> tuple[int, str]:
+        self.fika = "u"
        with self.mutex, self.reg_mutex:
            return self._confirm_chunks(ptop, wark, written, locked, True)

@@ -3747,6 +3863,7 @@ class Up2k(object):

    def finish_upload(self, ptop: str, wark: str, busy_aps: dict[str, int]) -> None:
        self.busy_aps = busy_aps
+        self.fika = "u"
        with self.mutex, self.reg_mutex:
            self._finish_upload(ptop, wark)

@@ -3982,10 +4099,10 @@ class Up2k(object):
                sz,
                ip,
                at or time.time(),
-                "",
+                [dwark],
            )
            t = hr.get("rejectmsg") or ""
-            if t or not hr:
+            if t or hr.get("rc") != 0:
                if not t:
                    t = "upload blocked by xau server config: %r"
                    t = t % (djoin(vtop, rd, fn),)
@@ -4016,31 +4133,7 @@ class Up2k(object):
                with self.rescan_cond:
                    self.rescan_cond.notify_all()

-        if rd and sz and fn.lower() in self.args.th_coversd_set:
-            # wasteful; db_add will re-index actual covers
-            # but that won't catch existing files
-            crd, cdn = rd.rsplit("/", 1) if "/" in rd else ("", rd)
-            try:
-                q = "select fn from cv where rd=? and dn=?"
-                db_cv = db.execute(q, (crd, cdn)).fetchone()[0]
-                db_lcv = db_cv.lower()
-                if db_lcv in self.args.th_coversd_set:
-                    idx_db = self.args.th_coversd.index(db_lcv)
-                    idx_fn = self.args.th_coversd.index(fn.lower())
-                    add_cv = idx_fn < idx_db
-                else:
-                    add_cv = True
-            except:
-                add_cv = True
-
-            if add_cv:
-                try:
-                    db.execute("delete from cv where rd=? and dn=?", (crd, cdn))
-                    db.execute("insert into cv values (?,?,?)", (crd, cdn, fn))
-                except:
-                    pass
-
-        if "nodirsz" not in vflags:
+        if vflags and "nodirsz" not in vflags:
            try:
                q = "update ds set nf=nf+1, sz=sz+? where rd=?"
                q2 = "insert into ds values(?,?,1)"
@@ -4054,6 +4147,9 @@ class Up2k(object):
                pass

    def handle_fs_abrt(self, akey: str) -> None:
+        if akey == "ping":
+            with self.mutex, self.reg_mutex:
+                return
        self.abrt_key = akey

    def handle_rm(
@@ -4101,6 +4197,7 @@ class Up2k(object):
            vn0, rem0 = self.vfs.get(vpath, uname, *permsets[0])
            vn, rem = vn0.get_dbv(rem0)
            ptop = vn.realpath
+            self.fika = "d"
            with self.mutex, self.reg_mutex:
                abrt_cfg = vn.flags.get("u2abort", 1)
                addr = (ip or "\n") if abrt_cfg in (1, 2) else ""
@@ -4156,6 +4253,7 @@ class Up2k(object):
            st = bos.lstat(atop)
            is_dir = stat.S_ISDIR(st.st_mode)
        except:
+            # NOTE: "file not found" *sftpd
            raise Pebkac(400, "file not found on disk (already deleted?)")

        if "bcasechk" in vn.flags and not vn.casechk(rem, False):
@@ -4203,7 +4301,7 @@ class Up2k(object):
                    _ = dbv.get(volpath, uname, *permsets[0])

                if xbd:
-                    if not runhook(
+                    hr = runhook(
                        self.log,
                        None,
                        self,
@@ -4218,13 +4316,17 @@ class Up2k(object):
                        st.st_size,
                        ip,
                        time.time(),
-                        "",
-                    ):
-                        t = "delete blocked by xbd server config: %r"
-                        self.log(t % (abspath,), 1)
+                        None,
+                    )
+                    t = hr.get("rejectmsg") or ""
+                    if t or hr.get("rc") != 0:
+                        if not t:
+                            t = "delete blocked by xbd server config: %r" % (abspath,)
+                        self.log(t, 1)
                        continue

                n_files += 1
+                self.fika = "d"
                with self.mutex, self.reg_mutex:
                    cur = None
                    try:
@@ -4258,7 +4360,7 @@ class Up2k(object):
                        st.st_size,
                        ip,
                        time.time(),
-                        "",
+                        None,
                    )

        if is_dir:
@@ -4289,6 +4391,7 @@ class Up2k(object):
            raise Pebkac(400, "file does not exist case-sensitively")

        if stat.S_ISREG(st.st_mode) or stat.S_ISLNK(st.st_mode):
+            self.fika = "c"
            with self.mutex:
                try:
                    ret = self._cp_file(uname, ip, svp, dvp, curs)
@@ -4310,6 +4413,7 @@ class Up2k(object):

        # don't use svn_dbv; would skip subvols due to _ls `if not rem:`
        g = svn.walk("", srem, [], uname, permsets, dots, scandir, True)
+        self.fika = "c"
        with self.mutex:
            try:
                for dbv, vrem, _, atop, files, rd, vd in g:
@@ -4371,7 +4475,7 @@ class Up2k(object):
        xbc = svn.flags.get("xbc")
        xac = dvn.flags.get("xac")
        if xbc:
-            if not runhook(
+            hr = runhook(
                self.log,
                None,
                self,
@@ -4386,9 +4490,12 @@ class Up2k(object):
                fsize,
                ip,
                time.time(),
-                "",
-            ):
-                t = "copy blocked by xbr server config: %r" % (svp,)
+                None,
+            )
+            t = hr.get("rejectmsg") or ""
+            if t or hr.get("rc") != 0:
+                if not t:
+                    t = "copy blocked by xbr server config: %r" % (svp,)
                self.log(t, 1)
                raise Pebkac(405, t)

@@ -4470,6 +4577,8 @@ class Up2k(object):
                    bos.utime(dabs, times, False)
                except:
                    pass
+            if "fperms" in dvn.flags:
+                set_ap_perms(dabs, dvn.flags)

        if xac:
            runhook(
@@ -4487,7 +4596,7 @@ class Up2k(object):
                fsize,
                ip,
                time.time(),
-                "",
+                None,
            )

        return "k"
@@ -4510,6 +4619,7 @@ class Up2k(object):
            raise Pebkac(400, "file does not exist case-sensitively")

        if stat.S_ISREG(st.st_mode) or stat.S_ISLNK(st.st_mode):
+            self.fika = "m"
            with self.mutex:
                try:
                    ret = self._mv_file(uname, ip, svp, dvp, curs)
@@ -4533,6 +4643,7 @@ class Up2k(object):
                raise Pebkac(400, "mv: source folder contains other volumes")

        g = svn.walk("", srem, [], uname, permsets, 2, scandir, True)
+        self.fika = "m"
        with self.mutex:
            try:
                for dbv, vrem, _, atop, files, rd, vd in g:
@@ -4623,7 +4734,7 @@ class Up2k(object):
        xbr = svn.flags.get("xbr")
        xar = dvn.flags.get("xar")
        if xbr:
-            if not runhook(
+            hr = runhook(
                self.log,
                None,
                self,
@@ -4638,9 +4749,12 @@ class Up2k(object):
                fsize,
                ip,
                time.time(),
-                "",
-            ):
-                t = "move blocked by xbr server config: %r" % (svp,)
+                [dabs],
+            )
+            t = hr.get("rejectmsg") or ""
+            if t or hr.get("rc") != 0:
+                if not t:
+                    t = "move blocked by xbr server config: %r" % (svp,)
                self.log(t, 1)
                raise Pebkac(405, t)

@@ -4678,7 +4792,7 @@ class Up2k(object):
                    fsize,
                    ip,
                    time.time(),
-                    "",
+                    [sabs],
                )

            return "k"
@@ -4690,6 +4804,12 @@ class Up2k(object):
        if w:
            assert c1  # !rm
            if c2 and c2 != c1:
+                if "nodupem" in dvn.flags:
+                    q = "select w from up where substr(w,1,16) = ?"
+                    for (w2,) in c2.execute(q, (w[:16],)):
+                        if w == w2:
+                            t = "file exists in target volume, and dupes are forbidden in config"
+                            raise Pebkac(400, t)
                self._copy_tags(c1, c2, w)

            xlink = bool(svn.flags.get("xlink"))
@@ -4746,6 +4866,8 @@ class Up2k(object):
                wunlink(self.log, sabs, svn.flags)
            else:
                atomic_move(self.log, sabs, dabs, svn.flags)
+                if svn != dvn and "fperms" in dvn.flags:
+                    set_ap_perms(dabs, dvn.flags)

        except OSError as ex:
            if ex.errno != errno.EXDEV:
@@ -4779,6 +4901,8 @@ class Up2k(object):
                    bos.utime(dabs, times, False)
                except:
                    pass
+            if "fperms" in dvn.flags:
+                set_ap_perms(dabs, dvn.flags)

            wunlink(self.log, sabs, svn.flags)

@@ -4798,7 +4922,7 @@ class Up2k(object):
                fsize,
                ip,
                time.time(),
-                "",
+                [sabs],
            )

        return "k"
@@ -5102,7 +5226,7 @@ class Up2k(object):
        if eno not in E_FS_CRIT:
            return self.log("hashing failed; %r @ %r\n%s" % (ex, ap, min_ex()), 3)
        t = "hashing failed; %r @ %r\n%s\nWARNING: This MAY indicate a serious issue with your harddisk or filesystem! Please investigate %sOS-logs\n"
-        t2 = "" if ANYWIN or MACOS else "dmesg and "
+        t2 = "" if ANYWIN or UNIX else "dmesg and "
        return self.log(t % (ex, ap, min_ex(), t2), 1)

    def _new_upload(self, job: dict[str, Any], vfs: VFS, depth: int) -> dict[str, str]:
@@ -5136,10 +5260,10 @@ class Up2k(object):
                job["size"],
                job["addr"],
                job["t0"],
-                "",
+                [job["dwrk"]],
            )
            t = hr.get("rejectmsg") or ""
-            if t or not hr:
+            if t or hr.get("rc") != 0:
                if not t:
                    t = "upload blocked by xbu server config: %r" % (vp_chk,)
                self.log(t, 1)
@@ -5148,15 +5272,25 @@ class Up2k(object):
                x = pathmod(self.vfs, ap_chk, vp_chk, hr["reloc"])
                if x:
                    ud1 = (vfs.vpath, job["prel"], job["name"])
+                    job["rvp0"] = vjoins(*ud1)
                    pdir, _, job["name"], (vfs, rem) = x
-                    job["vcfg"] = vf = vfs.flags
+                    vf = vfs.flags
                    job["ptop"] = vfs.realpath
                    job["vtop"] = vfs.vpath
                    job["prel"] = rem
-                    job["name"] = sanitize_fn(job["name"], "")
+                    job["name"] = sanitize_fn(job["name"])
                    ud2 = (vfs.vpath, job["prel"], job["name"])
                    if ud1 != ud2:
-                        self.log("xbu reloc2:%d..." % (depth,), 6)
+                        job["vcfg"] = vf
+                        t = "xbu reloc2=%d ptop=%r vtop=%r prel=%r name=%r" % (
+                            depth,
+                            job["ptop"],
+                            job["vtop"],
+                            job["prel"],
+                            job["name"],
+                        )
+                        self.log(t, 6)
+                        bos.makedirs(djoin(job["ptop"], job["prel"]), vf=vf)
                        return self._handle_json(job, depth + 1)

        job["name"] = self._untaken(pdir, job, job["t0"])
@@ -5196,9 +5330,12 @@ class Up2k(object):
                and self.args.sparse * 1024 * 1024 <= sz
            ):
                try:
-                    sp.check_call(["fsutil", "sparse", "setflag", abspath])
-                except:
-                    self.log("could not sparse %r" % (abspath,), 3)
+                    winsparse(f)
+                    # some kernels/etc are buggy; force sync:
+                    f.close()
+                    f = open(abspath, "rb+")
+                except Exception as ex:
+                    self.log("could not sparse %r: %s" % (abspath, ex), 3)
                    relabel = True
                    sprs = False

@@ -5258,6 +5395,7 @@ class Up2k(object):
                self.do_snapshot()

    def do_snapshot(self) -> None:
+        self.fika = "u"
        with self.mutex, self.reg_mutex:
            for k, reg in self.registry.items():
                self._snap_reg(k, reg)
@@ -5362,7 +5500,7 @@ class Up2k(object):
            # self.log("\n  " + repr([ptop, rd, fn]))
            abspath = djoin(ptop, rd, fn)
            try:
-                tags = self.mtag.get(abspath) if sz else {}
+                tags = self.mtag.get(abspath, self.flags[ptop]) if sz else {}
                ntags1 = len(tags)
                parsers = self._get_parsers(ptop, tags, abspath)
                if self.args.mtag_vv:
@@ -5538,6 +5676,7 @@ class Up2k(object):

    def shutdown(self) -> None:
        self.stop = True
+        self.fika = "f"

        if self.mth:
            self.mth.stop = True
@@ -5583,7 +5722,7 @@ def up2k_chunksize(filesize: int) -> int:
    stepsize = 512 * 1024
    while True:
        for mul in [1, 2]:
-            nchunks = math.ceil(filesize * 1.0 / chunksize)
+            nchunks = -int(-filesize // chunksize)
            if nchunks <= 256 or (chunksize >= 32 * 1024 * 1024 and nchunks <= 4096):
                return chunksize

@@ -5591,6 +5730,31 @@ def up2k_chunksize(filesize: int) -> int:
            stepsize *= mul


+def up2k_hashlist_from_file(path: str) -> tuple[list[str], os.stat_result]:
+    """not used by copyparty itself, only by some hooks"""
+    st = bos.stat(path)
+    fsz = st.st_size
+    csz = up2k_chunksize(fsz)
+    ret = []
+    with open(fsenc(path), "rb", 256 * 1024) as f:
+        while fsz > 0:
+            hashobj = hashlib.sha512()
+            rem = min(csz, fsz)
+            fsz -= rem
+            while rem > 0:
+                buf = f.read(min(rem, 64 * 1024))
+                if not buf:
+                    raise Exception("EOF at " + str(f.tell()))
+
+                hashobj.update(buf)
+                rem -= len(buf)
+
+            digest = hashobj.digest()[:33]
+            ret.append(ub64enc(digest).decode("ascii"))
+
+    return ret, st
+
+
 def up2k_wark_from_hashlist(salt: str, filesize: int, hashes: list[str]) -> str:
    """server-reproducible file identifier, independent of name or location"""
    values = [salt, str(filesize)] + hashes
@@ -2,11 +2,20 @@
 # which should help on really slow connections
 # but then why are you using copyparty in the first place

-pk: $(addsuffix .gz, $(wildcard *.js *.css))
-un: $(addsuffix .un, $(wildcard *.gz))
+pk: $(addsuffix .gz, $(wildcard tl/*.js *.js *.css) \
+	a/webdav-cfg.txt )
+un: $(addsuffix .un, $(wildcard tl/*.gz *.gz a/*.gz))
+
+tl/%.js.gz: tl/%.js
+	sh Makefile.s1 <$< | pigz -c11 -J 34 -I 573 >$@
+	pigz -dc <$@ | grep -q ^Ls
+	touch -r $< $@
+	rm $<

 %.gz: %
-	pigz -11 -J 34 -I 573 $<
+	pigz -c11 -J 34 -I 573 <$< >$@
+	touch -r $< $@
+	rm $<

 %.un: %
 	pigz -d $<