Stop non-admins injecting nginx directives through host fields

advanced_config is admin-only, but other fields are written into nginx
config as-is and the schema barely limited them. A non-admin could add
any directive through them:

- location path, forward_host and forward_path (any non-empty string)
- proxy host forward_host (a quote breaks out of set $server "...")
- redirection forward_domain_name and stream forwarding_host (their
  pattern allowed ";", e.g. "example.com; return 200; #")
- access list IPv6 addresses ("%.+" zone id allowed "::1%x; allow all")
- domain names (a newline got out of the header comment)

Validation
- New lib/nginx-values.js holds the rules. Values can't contain
  characters that end or change an nginx token: whitespace, control
  characters, ; { } " ' # \, and no $ in a host. Location paths allow a
  modifier and an unquoted or double-quoted URI, so =, ~, ~* and ^~
  locations and quoted regexes keep working.
- The API schemas use the same patterns, including the certificate
  schema's own copy of the domain rule. A test fails if they drift.
- generateConfig checks stored values again, so rows saved before these
  rules can't inject either. The check is about nginx structure only:
  a stored value the API would now reject, but which can't add
  directives, still renders. If a value is unsafe, nothing is written
  and the host is marked offline with the reason in nginx_err. The same
  check covers per-location access lists and the Let's Encrypt request
  config.
- htpasswd user names can't contain ":" or control characters, which
  corrupted the file.

Fixes along the way
- IPv6 client addresses: drop zone ids, which nginx's allow/deny
  rejects anyway, and fix broken escapes (literal "d" for digits, an
  unescaped ".").
- Rewrite the domain rule used by redirection and stream hosts so it
  can't backtrack exponentially. It accepts the same values.
- Pattern errors still say "must match pattern", but no longer quote
  the regex.

Tests
- backend/test/nginx-values.test.js, run by `yarn test` and in
  scripts/ci/test-and-build. Real request schemas reject an injection
  set and accept valid values on every endpoint. Accepted values,
  rendered through the real templates and parsed using nginx's
  tokenizer rules, keep the same statement structure. The check
  refuses unsafe stored rows.
- NginxValues.cy.js: injection attempts from a non-admin get 400, and
  hosts using valid edge-case values come up with nginx_online true.

.gitattributes

@@ -0,0 +1,7 @@
+# Force LF everywhere: these files run inside Linux containers
+* text=auto eol=lf
+*.png binary
+*.jpg binary
+*.ico binary
+*.woff binary
+*.woff2 binary

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,40 @@
+<!--
+
+WARNINGS:
+
+1. An AI bot will comment on this PR! If you don't get a perfect score or don't
+address any issues raised, your PR won't get merged!
+
+2. Any PR without a description will be closed. The title is not enough!
+
+3. Don't go creating a duplicate PR! Check that someone hasn't already created
+something that tackles your fix and if so, help them first
+
+-->
+
+## Why
+
+<!-- Provide a brief description of WHY you are making your changes -->
+
+<!-- Consider if you are changing the API, then go in to detail why and/or if
+you change will break API for existing users -->
+
+## Type of Change
+
+<!-- Mark the relevant options with an "x" -->
+
+- [ ] Bug fix (non-breaking change that fixes an issue)
+- [ ] New feature (non-breaking change that adds functionality)
+- [ ] Breaking change (fix or feature that would cause existing functionality to change)
+- [ ] Documentation update
+- [ ] Code refactoring
+- [ ] API changes
+- [ ] Performance improvement
+- [ ] Test addition or update
+
+## AI Usage
+
+<!-- Mark the relevant options with an "x" -->
+
+- [ ] AI was used to write this
+- [ ] AI was used to review this

.github/workflows/stale.yml

@@ -8,7 +8,7 @@ jobs:
   stale:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/stale@v10
+      - uses: actions/stale@v11
         with:
           stale-issue-label: 'stale'
           stale-pr-label: 'stale'

.gitignore

@@ -8,3 +8,6 @@ test/node_modules
 */node_modules
 docker/dev/dnsrouter-config.json.tmp
 docker/dev/resolv.conf
+.claude
+test/__pycache__
+

.version

@@ -1 +1 @@
-2.14.0
+2.17.0

README.md

@@ -1,7 +1,7 @@
 <p align="center">
 	<img src="https://nginxproxymanager.com/github.png">
 	<br><br>
-	<img src="https://img.shields.io/badge/version-2.14.0-green.svg?style=for-the-badge">
+	<img src="https://img.shields.io/badge/version-2.17.0-green.svg?style=for-the-badge">
 	<a href="https://hub.docker.com/repository/docker/jc21/nginx-proxy-manager">
 		<img src="https://img.shields.io/docker/stars/jc21/nginx-proxy-manager.svg?style=for-the-badge">
 	</a>
@@ -10,7 +10,7 @@
 	</a>
 </p>
 
-This project comes as a pre-built docker image that enables you to easily forward to your websites
+This project comes as a pre-built Docker image that enables you to easily forward to your websites
 running at home or otherwise, including free SSL, without having to know too much about Nginx or Letsencrypt.
 
 - [Quick Setup](#quick-setup)
@@ -20,9 +20,9 @@ running at home or otherwise, including free SSL, without having to know too muc
 ## Project Goal
 
 I created this project to fill a personal need to provide users with an easy way to accomplish reverse
-proxying hosts with SSL termination and it had to be so easy that a monkey could do it. This goal hasn't changed.
-While there might be advanced options they are optional and the project should be as simple as possible
-so that the barrier for entry here is low.
+proxying hosts with SSL termination, and it had to be so easy that a monkey could do it. This goal hasn't changed.
+While there might be advanced options, they are optional, and the project should be as simple as possible
+so that the barrier to entry here is low.
 
 <a href="https://www.buymeacoffee.com/jc21" target="_blank"><img src="http://public.jc21.com/github/by-me-a-coffee.png" alt="Buy Me A Coffee" style="height: 51px !important;width: 217px !important;" ></a>
 
@@ -30,11 +30,11 @@ so that the barrier for entry here is low.
 ## Features
 
 - Beautiful and Secure Admin Interface based on [Tabler](https://tabler.github.io/)
-- Easily create forwarding domains, redirections, streams and 404 hosts without knowing anything about Nginx
+- Easily create forwarding domains, redirections, streams, and 404 hosts without knowing anything about Nginx
 - Free SSL using Let's Encrypt or provide your own custom SSL certificates
 - Access Lists and basic HTTP Authentication for your hosts
 - Advanced Nginx configuration available for super users
-- User management, permissions and audit log
+- User management, permissions, and audit log
 
 ::: warning
 `armv7` is no longer supported in version 2.14+. This is due to Nodejs dropping support for armhf. Please
@@ -43,15 +43,15 @@ use the `2.13.7` image tag if this applies to you.
 
 ## Hosting your home network
 
-I won't go in to too much detail here but here are the basics for someone new to this self-hosted world.
+I won't go into too much detail here, but here are the basics for someone new to this self-hosted world.
 
 1. Your home router will have a Port Forwarding section somewhere. Log in and find it
-2. Add port forwarding for port 80 and 443 to the server hosting this project
+2. Add port forwarding for ports 80 and 443 to the server hosting this project
 3. Configure your domain name details to point to your home, either with a static ip or a service like
    - DuckDNS
    - [Amazon Route53](https://github.com/jc21/route53-ddns)
    - [Cloudflare](https://github.com/jc21/cloudflare-ddns)
-4. Use the Nginx Proxy Manager as your gateway to forward to your other web based services
+4. Use the Nginx Proxy Manager as your gateway to forward to your other web-based services
 
 ## Quick Setup
 

SECURITY.md

@@ -0,0 +1,30 @@
+# Security Policy
+
+## Supported Versions
+
+Only the latest stable release receives security updates.
+Older versions are not actively maintained.
+
+| Version | Supported |
+| ------- | --------- |
+| 2.16.x (latest) | :white_check_mark: |
+| < 2.16.0 | :x: |
+
+Docker images: `jc21/nginx-proxy-manager:latest`, `jc21/nginx-proxy-manager:2`
+
+See all releases: https://github.com/NginxProxyManager/nginx-proxy-manager/releases
+
+## Reporting a Vulnerability
+
+**Do NOT open a public GitHub Issue to report a security vulnerability.**
+
+Use GitHub's private vulnerability reporting:
+https://github.com/NginxProxyManager/nginx-proxy-manager/security/advisories/new
+
+Please include:
+- Affected version (Docker image tag or release)
+- Description of the vulnerability
+- Steps to reproduce
+- Potential impact
+
+Once a fix is available, a public GitHub Security Advisory will be published.

backend/app.js

@@ -62,7 +62,7 @@ app.use("/", mainRoutes);
 app.use((err, req, res, _) => {
 	const payload = {
 		error: {
-			code: err.status,
+			code: err.status || 500,
 			message: err.public ? err.message : "Internal Error",
 		},
 	};

backend/biome.json

@@ -1,91 +1,74 @@
 {
-    "$schema": "https://biomejs.dev/schemas/2.3.14/schema.json",
-    "vcs": {
-        "enabled": true,
-        "clientKind": "git",
-        "useIgnoreFile": true
-    },
-    "files": {
-        "ignoreUnknown": false,
-        "includes": [
-            "**/*.ts",
-            "**/*.tsx",
-            "**/*.js",
-            "**/*.jsx",
-            "!**/dist/**/*"
-        ]
-    },
-    "formatter": {
-        "enabled": true,
-        "indentStyle": "tab",
-        "indentWidth": 4,
-        "lineWidth": 120,
-        "formatWithErrors": true
-    },
-    "assist": {
-        "actions": {
-            "source": {
-                "organizeImports": {
-                    "level": "on",
-                    "options": {
-                        "groups": [
-                            ":BUN:",
-                            ":NODE:",
-                            [
-                                "npm:*",
-                                "npm:*/**"
-                            ],
-                            ":PACKAGE_WITH_PROTOCOL:",
-                            ":URL:",
-                            ":PACKAGE:",
-                            [
-                                "/src/*",
-                                "/src/**"
-                            ],
-                            [
-                                "/**"
-                            ],
-                            [
-                                "#*",
-                                "#*/**"
-                            ],
-                            ":PATH:"
-                        ]
-                    }
-                }
-            }
-        }
-    },
-    "linter": {
-        "enabled": true,
-        "rules": {
-            "recommended": true,
-            "correctness": {
-                "useUniqueElementIds": "off"
-            },
-            "suspicious": {
-                "noExplicitAny": "off"
-            },
-            "performance": {
-                "noDelete": "off"
-            },
-            "nursery": "off",
-            "a11y": {
-                "useSemanticElements": "off",
-                "useValidAnchor": "off"
-            },
-            "style": {
-                "noParameterAssign": "error",
-                "useAsConstAssertion": "error",
-                "useDefaultParameterLast": "error",
-                "useEnumInitializers": "error",
-                "useSelfClosingElements": "error",
-                "useSingleVarDeclarator": "error",
-                "noUnusedTemplateLiteral": "error",
-                "useNumberNamespace": "error",
-                "noInferrableTypes": "error",
-                "noUselessElse": "error"
-            }
-        }
-    }
+	"$schema": "https://biomejs.dev/schemas/2.6.6/schema.json",
+	"vcs": {
+		"enabled": true,
+		"clientKind": "git",
+		"useIgnoreFile": true
+	},
+	"files": {
+		"ignoreUnknown": false,
+		"includes": ["**/*.ts", "**/*.tsx", "**/*.js", "**/*.jsx", "!**/dist/**/*"]
+	},
+	"formatter": {
+		"enabled": true,
+		"indentStyle": "tab",
+		"indentWidth": 4,
+		"lineWidth": 120,
+		"formatWithErrors": true
+	},
+	"assist": {
+		"actions": {
+			"source": {
+				"organizeImports": {
+					"level": "on",
+					"options": {
+						"groups": [
+							":BUN:",
+							":NODE:",
+							["npm:*", "npm:*/**"],
+							":PACKAGE_WITH_PROTOCOL:",
+							":URL:",
+							":PACKAGE:",
+							["/src/*", "/src/**"],
+							["/**"],
+							["#*", "#*/**"],
+							":PATH:"
+						]
+					}
+				}
+			}
+		}
+	},
+	"linter": {
+		"enabled": true,
+		"rules": {
+			"preset": "recommended",
+			"correctness": {
+				"useUniqueElementIds": "off"
+			},
+			"suspicious": {
+				"noExplicitAny": "off"
+			},
+			"performance": {
+				"noDelete": "off"
+			},
+			"nursery": "off",
+			"a11y": {
+				"useSemanticElements": "off",
+				"useValidAnchor": "off"
+			},
+			"style": {
+				"noParameterAssign": "error",
+				"useAsConstAssertion": "error",
+				"useDefaultParameterLast": "error",
+				"useEnumInitializers": "error",
+				"useSelfClosingElements": "error",
+				"useSingleVarDeclarator": "error",
+				"noUnusedTemplateLiteral": "error",
+				"useNumberNamespace": "error",
+				"noInferrableTypes": "error",
+				"noUselessElse": "error"
+			}
+		}
+	}
 }

backend/db.js

@@ -1,5 +1,5 @@
 import knex from "knex";
-import {configGet, configHas} from "./lib/config.js";
+import { configGet, configHas } from "./lib/config.js";
 
 let instance = null;
 
@@ -23,8 +23,8 @@ const generateDbConfig = () => {
 			user: cfg.user,
 			password: cfg.password,
 			database: cfg.name,
-			port:     cfg.port,
-			...(cfg.ssl ? { ssl: cfg.ssl } : {})
+			port: cfg.port,
+			...(cfg.ssl ? { ssl: cfg.ssl } : {}),
 		},
 		migrations: {
 			tableName: "migrations",
@@ -37,6 +37,6 @@ const getInstance = () => {
 		instance = knex(generateDbConfig());
 	}
 	return instance;
-}
+};
 
 export default getInstance;

backend/index.js

@@ -26,7 +26,9 @@ async function appStart() {
 		})
 		.then(() => {
 			internalCertificate.initTimer();
-			internalIpRanges.initTimer();
+			if (IP_RANGES_FETCH_ENABLED) {
+				internalIpRanges.initTimer();
+			}
 
 			const server = app.listen(3000, () => {
 				logger.info(`Backend PID ${process.pid} listening on port 3000 ...`);

backend/internal/2fa.js

@@ -161,12 +161,12 @@ const internal2fa = {
 		}
 
 		const result = await verify({
-            token: code,
-            secret: auth.meta.totp_secret,
-            guardrails: createGuardrails({
-                MIN_SECRET_BYTES: 10,
-            }),
-        });
+			token: code,
+			secret: auth.meta.totp_secret,
+			guardrails: createGuardrails({
+				MIN_SECRET_BYTES: 10,
+			}),
+		});
 
 		if (!result.valid) {
 			throw new errs.AuthError("Invalid verification code");
@@ -288,11 +288,7 @@ const internal2fa = {
 	},
 
 	getUserPasswordAuth: async (userId) => {
-		const auth = await authModel
-			.query()
-			.where("user_id", userId)
-			.andWhere("type", "password")
-			.first();
+		const auth = await authModel.query().where("user_id", userId).andWhere("type", "password").first();
 
 		if (!auth) {
 			throw new errs.ItemNotFoundError("Auth not found");

backend/internal/access-list.js

@@ -2,7 +2,10 @@ import fs from "node:fs";
 import batchflow from "batchflow";
 import _ from "lodash";
 import errs from "../lib/error.js";
+import { isValid, unsafeValueError } from "../lib/nginx-values.js";
+import { isMysql, isPostgres } from "../lib/config.js";
 import utils from "../lib/utils.js";
+import db from "../db.js";
 import { access as logger } from "../logger.js";
 import accessListModel from "../models/access_list.js";
 import accessListAuthModel from "../models/access_list_auth.js";
@@ -15,6 +18,36 @@ const omissions = () => {
 	return ["is_deleted"];
 };
 
+/**
+ * Find proxy hosts that reference an access list in their locations JSON.
+ *
+ * @param   {Integer}  accessListId
+ * @returns {Promise<Array>}
+ */
+const getProxyHostsUsingAccessListInLocations = async (accessListId) => {
+	let result;
+	if (isMysql()) {
+		const searchObj = JSON.stringify([{ access_list_id: accessListId }]);
+		result = await db().raw("SELECT id FROM proxy_host WHERE is_deleted = 0 AND JSON_CONTAINS(locations, ?, ?)", [
+			searchObj,
+			"$",
+		]);
+	} else if (isPostgres()) {
+		result = await db().raw("SELECT id FROM proxy_host WHERE is_deleted = 0 AND locations::jsonb @> ?::jsonb", [
+			JSON.stringify([{ access_list_id: accessListId }]),
+		]);
+	} else {
+		result = await db().raw("SELECT id FROM proxy_host WHERE is_deleted = 0 AND locations LIKE ?", [
+			`%"access_list_id":${accessListId}%`,
+		]);
+	}
+	// knex raw() returns [rows, metadata] for MySQL, { rows } for Postgres and rows for SQLite
+	if (!Array.isArray(result)) {
+		return result?.rows || [];
+	}
+	return (Array.isArray(result[0]) ? result[0] : result) || [];
+};
+
 const internalAccessList = {
 	/**
 	 * @param   {Access}  access
@@ -48,20 +81,17 @@ const internalAccessList = {
 			return true;
 		});
 
-		// Clients
-		data.clients?.map((client) => {
-			promises.push(
-				accessListClientModel.query().insert({
-					access_list_id: row.id,
-					address: client.address,
-					directive: client.directive,
-				}),
-			);
-			return true;
-		});
-
 		await Promise.all(promises);
 
+		// Clients
+		for (const client of data.clients ?? []) {
+			await accessListClientModel.query().insert({
+				access_list_id: row.id,
+				address: client.address,
+				directive: client.directive,
+			});
+		}
+
 		// re-fetch with expansions
 		const freshRow = await internalAccessList.get(
 			access,
@@ -69,7 +99,7 @@ const internalAccessList = {
 				id: data.id,
 				expand: ["owner", "items", "clients", "proxy_hosts.access_list.[clients,items]"],
 			},
-			true // skip masking
+			true, // skip masking
 		);
 
 		// Audit log
@@ -154,25 +184,17 @@ const internalAccessList = {
 
 		// Check for clients and add/update/remove them
 		if (typeof data.clients !== "undefined" && data.clients) {
-			const clientPromises = [];
-			data.clients.map((client) => {
-				if (client.address) {
-					clientPromises.push(
-						accessListClientModel.query().insert({
-							access_list_id: data.id,
-							address: client.address,
-							directive: client.directive,
-						}),
-					);
-				}
-				return true;
-			});
-
 			const query = accessListClientModel.query().delete().where("access_list_id", data.id);
 			await query;
-			// Add new clitens
-			if (clientPromises.length) {
-				await Promise.all(clientPromises);
+
+			for (const client of data.clients) {
+				if (client.address) {
+					await accessListClientModel.query().insert({
+						access_list_id: data.id,
+						address: client.address,
+						directive: client.directive,
+					});
+				}
 			}
 		}
 
@@ -191,13 +213,34 @@ const internalAccessList = {
 				id: data.id,
 				expand: ["owner", "items", "clients", "proxy_hosts.[certificate,access_list.[clients,items]]"],
 			},
-			true // skip masking
+			true, // skip masking
 		);
 
-		await internalAccessList.build(freshRow)
+		await internalAccessList.build(freshRow);
 		if (Number.parseInt(freshRow.proxy_host_count, 10)) {
 			await internalNginx.bulkGenerateConfigs("proxy_host", freshRow.proxy_hosts);
 		}
+
+		// Also regenerate configs for proxy hosts that reference this access list in their locations
+		const locationHostRows = await getProxyHostsUsingAccessListInLocations(data.id);
+		if (locationHostRows?.length) {
+			const locationHostIds = locationHostRows
+				.map((r) => r.id)
+				.filter((id) => {
+					// Exclude hosts already regenerated above
+					return !freshRow.proxy_hosts?.find((h) => h.id === id);
+				});
+			if (locationHostIds.length) {
+				const locationHosts = await proxyHostModel
+					.query()
+					.where("is_deleted", 0)
+					.whereIn("id", locationHostIds)
+					.allowGraph(proxyHostModel.defaultAllowGraph)
+					.withGraphFetched("[owner, certificate, access_list.[clients,items]]");
+				await internalNginx.bulkGenerateConfigs("proxy_host", locationHosts);
+			}
+		}
+
 		await internalNginx.reload();
 		return internalAccessList.maskItems(freshRow);
 	},
@@ -213,17 +256,13 @@ const internalAccessList = {
 	 */
 	get: async (access, data, skipMasking) => {
 		const thisData = data || {};
-		const accessData = await access.can("access_lists:get", thisData.id)
+		const accessData = await access.can("access_lists:get", thisData.id);
 
 		const query = accessListModel
 			.query()
 			.select("access_list.*", accessListModel.raw("COUNT(proxy_host.id) as proxy_host_count"))
 			.leftJoin("proxy_host", function () {
-				this.on("proxy_host.access_list_id", "=", "access_list.id").andOn(
-					"proxy_host.is_deleted",
-					"=",
-					0,
-				);
+				this.on("proxy_host.access_list_id", "=", "access_list.id").andOn("proxy_host.is_deleted", "=", 0);
 			})
 			.where("access_list.is_deleted", 0)
 			.andWhere("access_list.id", thisData.id)
@@ -241,7 +280,7 @@ const internalAccessList = {
 
 		let row = await query.then(utils.omitRow(omissions()));
 
-		if (!row || !row.id) {
+		if (!row?.id) {
 			throw new errs.ItemNotFoundError(thisData.id);
 		}
 		if (!skipMasking && typeof row.items !== "undefined" && row.items) {
@@ -268,7 +307,7 @@ const internalAccessList = {
 			expand: ["proxy_hosts", "items", "clients"],
 		});
 
-		if (!row || !row.id) {
+		if (!row?.id) {
 			throw new errs.ItemNotFoundError(data.id);
 		}
 
@@ -278,28 +317,41 @@ const internalAccessList = {
 		// 4. audit log
 
 		// 1. update row to be deleted
-		await accessListModel
-			.query()
-			.where("id", row.id)
-			.patch({
-				is_deleted: 1,
-			});
+		await accessListModel.query().where("id", row.id).patch({
+			is_deleted: 1,
+		});
 
 		// 2. update any proxy hosts that were using it (ignoring permissions)
-		if (row.proxy_hosts) {
-			await proxyHostModel
+		const affectedHostIds = new Set((row.proxy_hosts || []).map((h) => h.id));
+		if (affectedHostIds.size) {
+			await proxyHostModel.query().where("access_list_id", "=", row.id).patch({ access_list_id: 0 });
+		}
+
+		// Also clear it from any proxy host locations using it, these will then inherit the host's access list
+		const locationHostRows = await getProxyHostsUsingAccessListInLocations(row.id);
+		for (const { id: hostId } of locationHostRows) {
+			const host = await proxyHostModel.query().where("id", hostId).first();
+			if (host?.locations?.some((loc) => loc.access_list_id === row.id)) {
+				const updatedLocations = host.locations.map((loc) => {
+					if (loc.access_list_id === row.id) {
+						return { ...loc, access_list_id: 0 };
+					}
+					return loc;
+				});
+				await proxyHostModel.query().where("id", hostId).patch({ locations: updatedLocations });
+				affectedHostIds.add(hostId);
+			}
+		}
+
+		// 3. reconfigure those hosts from fresh rows, then reload nginx
+		if (affectedHostIds.size) {
+			const affectedHosts = await proxyHostModel
 				.query()
-				.where("access_list_id", "=", row.id)
-				.patch({ access_list_id: 0 });
-
-			// 3. reconfigure those hosts, then reload nginx
-			// set the access_list_id to zero for these items
-			row.proxy_hosts.map((_val, idx) => {
-				row.proxy_hosts[idx].access_list_id = 0;
-				return true;
-			});
-
-			await internalNginx.bulkGenerateConfigs("proxy_host", row.proxy_hosts);
+				.where("is_deleted", 0)
+				.whereIn("id", [...affectedHostIds])
+				.allowGraph(proxyHostModel.defaultAllowGraph)
+				.withGraphFetched("[owner, certificate, access_list.[clients,items]]");
+			await internalNginx.bulkGenerateConfigs("proxy_host", affectedHosts);
 		}
 
 		await internalNginx.reload();
@@ -336,11 +388,7 @@ const internalAccessList = {
 			.query()
 			.select("access_list.*", accessListModel.raw("COUNT(proxy_host.id) as proxy_host_count"))
 			.leftJoin("proxy_host", function () {
-				this.on("proxy_host.access_list_id", "=", "access_list.id").andOn(
-					"proxy_host.is_deleted",
-					"=",
-					0,
-				);
+				this.on("proxy_host.access_list_id", "=", "access_list.id").andOn("proxy_host.is_deleted", "=", 0);
 			})
 			.where("access_list.is_deleted", 0)
 			.groupBy("access_list.id")
@@ -382,10 +430,7 @@ const internalAccessList = {
 	 * @returns {Promise}
 	 */
 	getCount: async (userId, visibility) => {
-		const query = accessListModel
-			.query()
-			.count("id as count")
-			.where("is_deleted", 0);
+		const query = accessListModel.query().count("id as count").where("is_deleted", 0);
 
 		if (visibility !== "all") {
 			query.andWhere("owner_user_id", userId);
@@ -447,20 +492,29 @@ const internalAccessList = {
 		}
 
 		// 2. create empty access file
-		fs.writeFileSync(htpasswdFile, '', {encoding: 'utf8'});
+		fs.writeFileSync(htpasswdFile, "", { encoding: "utf8" });
 
 		// 3. generate password for each user
 		if (list.items.length) {
 			await new Promise((resolve, reject) => {
-				batchflow(list.items).sequential()
+				batchflow(list.items)
+					.sequential()
 					.each((_i, item, next) => {
 						if (item.password?.length) {
+							// One "user:hash" line each; a ":" or line break in the name would corrupt the file
+							if (!isValid("accessUsername", item.username)) {
+								reject(unsafeValueError(`Access List #${list.id}: user name is not allowed`));
+								return;
+							}
 							logger.info(`Adding: ${item.username}`);
 
-							utils.execFile('openssl', ['passwd', '-apr1', item.password])
+							utils
+								.execFile("openssl", ["passwd", "-apr1", item.password])
 								.then((res) => {
 									try {
-										fs.appendFileSync(htpasswdFile, `${item.username}:${res}\n`, {encoding: 'utf8'});
+										fs.appendFileSync(htpasswdFile, `${item.username}:${res}\n`, {
+											encoding: "utf8",
+										});
 									} catch (err) {
 										reject(err);
 									}
@@ -482,7 +536,7 @@ const internalAccessList = {
 					});
 			});
 		}
-	}
-}
+	},
+};
 
 export default internalAccessList;

backend/internal/audit-log.js

@@ -3,7 +3,6 @@ import { castJsonIfNeed } from "../lib/helpers.js";
 import auditLogModel from "../models/audit-log.js";
 
 const internalAuditLog = {
-
 	/**
 	 * All logs
 	 *
@@ -46,11 +45,7 @@ const internalAuditLog = {
 	get: async (access, data) => {
 		await access.can("auditlog:list");
 
-		const query = auditLogModel
-			.query()
-			.andWhere("id", data.id)
-			.allowGraph("[user]")
-			.first();
+		const query = auditLogModel.query().andWhere("id", data.id).allowGraph("[user]").first();
 
 		if (typeof data.expand !== "undefined" && data.expand !== null) {
 			query.withGraphFetched(`[${data.expand.join(", ")}]`);

backend/internal/certificate.js

@@ -1,7 +1,7 @@
 import fs from "node:fs";
 import https from "node:https";
 import path from "path";
-import archiver from "archiver";
+import { ZipArchive } from "archiver";
 import _ from "lodash";
 import moment from "moment";
 import { ProxyAgent } from "proxy-agent";
@@ -24,8 +24,163 @@ const certbotCommand = "certbot";
 const certbotLogsDir = "/data/logs";
 const certbotWorkDir = "/tmp/letsencrypt-lib";
 
+// Certbot output lines that say nothing about why a request failed
+const certbotNoise = [
+	/^Saving debug log/,
+	/^Ask for help/,
+	/^See the logfile/,
+	/^(- )+-?$/,
+	/^Processing \//,
+	/^Renewing an existing/,
+	/^Requesting a certificate/,
+	/^Plugins selected/,
+	/^Account registered/,
+	/^Waiting \d+ seconds/,
+	/^All renewals failed/,
+	/^\S+\/fullchain\.pem \(failure\)$/,
+	/^\d+ renew failure\(s\)/,
+];
+
+// Plain-language hints for failures that are common and fixable outside NPM.
+// [pattern, hint, dnsProvider?]: a hint with a dnsProvider only applies to certificates using that
+// provider, so a generic message (e.g. "Authentication error" from another DNS plugin) isn't
+// reported as a Cloudflare problem.
+const certbotHints = [
+	// Checked first: a rate limit is unambiguous and the most useful thing to report
+	[
+		/too many (certificates|failed authorizations|new orders)|rateLimited/i,
+		() => "Let's Encrypt rate limit reached. Wait before retrying (see https://letsencrypt.org/docs/rate-limits/).",
+	],
+	[
+		/expected \S+ to be a symlink/i,
+		() =>
+			"This certificate's files in /etc/letsencrypt were copied without their symlinks (live/ must link to archive/). Restore /etc/letsencrypt from a copy made with tar or rsync -a, or delete this certificate and create it again.",
+	],
+	[
+		/Renewal configuration file \S+ is broken/i,
+		() =>
+			"This certificate's files in /etc/letsencrypt are damaged or missing. Restore /etc/letsencrypt from a backup, or delete this certificate and create it again.",
+	],
+	[
+		/\b9109\b.*?location: ([0-9a-f.:]+)/i,
+		(m) => {
+			const ip = m[1].replace(/\.+$/, "");
+			return `Cloudflare refused the API token from this server's public IP (${ip}). The token has Client IP Address Filtering: add ${ip} to the token in Cloudflare (My Profile > API Tokens), or remove the filter, then renew again.`;
+		},
+		"cloudflare",
+	],
+	[
+		// certbot-dns-cloudflare errors read "<context>: <code> <message>", e.g.
+		// "Error determining zone_id: 9109 Invalid access token."
+		/: (6003|6111|9103|9109|10000) [A-Za-z]|Invalid request headers|Invalid access token|Authentication error/i,
+		() =>
+			"Cloudflare rejected the API token. Check the token value and that it has Zone > DNS > Edit for this domain's zone.",
+		"cloudflare",
+	],
+	[
+		/NXDOMAIN|DNS problem/i,
+		() => "Let's Encrypt could not look up the domain. Check the DNS records of every domain on the certificate.",
+	],
+	[
+		/Timeout during connect|Connection refused|Invalid response from/i,
+		() =>
+			"Let's Encrypt could not reach this server over HTTP (port 80) for the challenge. Check that the domain points here and port 80 is open, or use a DNS challenge.",
+	],
+];
+
+/**
+ * The values in a DNS credentials file ("key = value" lines), for redacting from certbot output.
+ *
+ * @param   {String} [credentials]
+ * @returns {String[]}
+ */
+const credentialValues = (credentials) =>
+	`${credentials || ""}`
+		.split("\n")
+		.map((line) => line.replace(/^[^=:]*[=:]/, "").trim())
+		.filter((value) => value.length >= 4);
+
+/**
+ * Removes anything that could be a secret from certbot output before it leaves the server.
+ * DNS plugins can echo provider errors verbatim, including request URLs that carry API keys in
+ * the query string (e.g. "401 Client Error ... for url: https://...?token=..."), so this strips
+ * the given credential values, URL query strings and userinfo, and key=value pairs whose name
+ * looks secret.
+ *
+ * @param   {String}   text
+ * @param   {String[]} [secrets]
+ * @returns {String}
+ */
+const redactCertbotOutput = (text, secrets = []) => {
+	let out = `${text || ""}`;
+	for (const secret of [...secrets].sort((a, b) => b.length - a.length)) {
+		out = out.split(secret).join("[REDACTED]");
+	}
+	return out
+		.replace(/\b(Bearer|Basic)\s+[A-Za-z0-9._~+/=-]+/gi, "$1 [REDACTED]")
+		.replace(/(\b[a-z][a-z0-9+.-]*:\/\/)[^/\s@]*@/gi, "$1[REDACTED]@")
+		.replace(/(\b[a-z][a-z0-9+.-]*:\/\/[^\s?#]*)\?[^\s#]*/gi, "$1?[REDACTED]")
+		.replace(
+			/\b([\w-]*(?:token|key|secret|password|passwd|pass|auth|credential|signature|sig)[\w-]*)(["']?\s*[=:]\s*["']?)[^\s"'&,;]+/gi,
+			"$1$2[REDACTED]",
+		);
+};
+
+/**
+ * Turns certbot's stderr into a message the UI can show: a hint for common causes plus certbot's own reason.
+ * The output is redacted first (see redactCertbotOutput), since DNS plugins can echo credentials.
+ *
+ * @param   {String}   stderr
+ * @param   {String}   [dnsProvider]  The certificate's DNS provider, if it uses a DNS challenge
+ * @param   {String[]} [secrets]      Values that must never appear in the message
+ * @returns {String}
+ */
+const certbotFailureMessage = (stderr, dnsProvider, secrets) => {
+	const reason = redactCertbotOutput(stderr, secrets)
+		.split("\n")
+		.map((line) => line.trim())
+		.filter((line) => line && !certbotNoise.some((re) => re.test(line)))
+		.join(" ")
+		.slice(0, 800);
+
+	for (const [pattern, hint, provider] of certbotHints) {
+		if (provider && provider !== dnsProvider) {
+			continue;
+		}
+		const match = reason.match(pattern);
+		if (match) {
+			return `${hint(match)} (Certbot: ${reason})`;
+		}
+	}
+	return reason ? `Certbot failed: ${reason}` : "Certbot failed. See /data/logs/letsencrypt.log for details.";
+};
+
+/**
+ * Runs certbot; a failure becomes a public error with the real reason instead of "Internal Error".
+ *
+ * @param   {Array}  args
+ * @param   {Object} [opts]
+ * @param   {String} [dnsProvider]  Passed through so provider-specific hints only show for that provider
+ * @param   {String} [credentials]  The DNS credentials file contents, so its values are redacted from errors
+ * @returns {Promise}
+ */
+const runCertbot = async (args, opts, dnsProvider, credentials) => {
+	try {
+		return await utils.execFile(certbotCommand, args, opts);
+	} catch (err) {
+		if (err instanceof error.CommandError) {
+			const secrets = credentialValues(credentials);
+			logger.error(redactCertbotOutput(err.message, secrets));
+			// The raw error is not attached as `previous`: the API error handler returns
+			// `previous` to the client for certificate routes, and it holds unredacted output.
+			throw new error.ValidationError(certbotFailureMessage(err.message, dnsProvider, secrets));
+		}
+		throw err;
+	}
+};
+
 const omissions = () => {
-	return ["is_deleted", "owner.is_deleted", "meta.dns_provider_credentials"];
+	return ["is_deleted", "owner.is_deleted", "meta.dns_provider_credentials", "meta.certificate_key"];
 };
 
 const internalCertificate = {
@@ -66,7 +221,7 @@ const internalCertificate = {
 				.andWhere("provider", "letsencrypt")
 				.andWhere("expires_on", "<", expirationThreshold)
 				.then((certificates) => {
-					if (!certificates || !certificates.length) {
+					if (!certificates?.length) {
 						return null;
 					}
 
@@ -75,6 +230,7 @@ const internalCertificate = {
 					 * instance of Certbot is already running.'
 					 */
 					let sequence = Promise.resolve();
+					let renewed = 0;
 
 					certificates.forEach((certificate) => {
 						sequence = sequence.then(() =>
@@ -88,7 +244,11 @@ const internalCertificate = {
 										token: tokenModel(),
 									},
 									{ id: certificate.id },
+									{ reload: false }, // one reload for the whole batch, below
 								)
+								.then(() => {
+									renewed++;
+								})
 								.catch((err) => {
 									// Don't want to stop the train here, just log the error
 									logger.error(err.message);
@@ -96,7 +256,11 @@ const internalCertificate = {
 						);
 					});
 
-					return sequence;
+					return sequence.then(() => {
+						if (renewed) {
+							return internalCertificate.reloadAfterRenew();
+						}
+					});
 				})
 				.then(() => {
 					logger.info("Completed SSL cert renew process");
@@ -143,7 +307,7 @@ const internalCertificate = {
 				await internalCertificate.disableInUseHosts(inUseResult);
 
 				const user = await userModel.query().where("is_deleted", 0).andWhere("id", data.owner_user_id).first();
-				if (!user || !user.email) {
+				if (!user?.email) {
 					throw new error.ValidationError(
 						"A valid email address must be set on your user account to use Let's Encrypt",
 					);
@@ -305,7 +469,7 @@ const internalCertificate = {
 		}
 
 		const row = await query.then(utils.omitRow(omissions()));
-		if (!row || !row.id) {
+		if (!row?.id) {
 			throw new error.ItemNotFoundError(data.id);
 		}
 		// Custom omissions
@@ -370,7 +534,7 @@ const internalCertificate = {
 	 * @returns {Promise}
 	 */
 	zipFiles: async (source, out) => {
-		const archive = archiver("zip", { zlib: { level: 9 } });
+		const archive = new ZipArchive({ zlib: { level: 9 } });
 		const stream = fs.createWriteStream(out);
 
 		return new Promise((resolve, reject) => {
@@ -395,12 +559,35 @@ const internalCertificate = {
 	 */
 	delete: async (access, data) => {
 		await access.can("certificates:delete", data.id);
-		const row = await internalCertificate.get(access, { id: data.id });
+		const hostRelations = ["proxy_hosts", "redirection_hosts", "dead_hosts", "streams"];
+		const row = await internalCertificate.get(access, { id: data.id, expand: hostRelations });
 
-		if (!row || !row.id) {
+		if (!row?.id) {
 			throw new error.ItemNotFoundError(data.id);
 		}
 
+		// A certificate that hosts still use can't be deleted: their rendered configs include its
+		// files, and revoking it would leave them serving a revoked certificate. Hosts may belong
+		// to other users, so only counts are reported.
+		const labels = {
+			proxy_hosts: "proxy host",
+			redirection_hosts: "redirection host",
+			dead_hosts: "404 host",
+			streams: "stream",
+		};
+		const usage = hostRelations
+			.map((rel) => ({ rel, count: row[rel]?.length || 0 }))
+			.filter((u) => u.count > 0)
+			.map((u) => `${u.count} ${labels[u.rel]}${u.count === 1 ? "" : "s"}`);
+		if (usage.length) {
+			throw new error.ValidationError(
+				`Certificate is in use by ${usage.join(", ")}. Remove it from them before deleting it.`,
+			);
+		}
+		for (const rel of hostRelations) {
+			delete row[rel];
+		}
+
 		await certificateModel.query().where("id", row.id).patch({
 			is_deleted: 1,
 		});
@@ -416,12 +603,47 @@ const internalCertificate = {
 		});
 
 		if (row.provider === "letsencrypt") {
-			// Revoke the cert
+			// Revoke the cert. Errors are logged by revokeLetsEncryptSsl.
 			await internalCertificate.revokeLetsEncryptSsl(row);
+			// Certbot leaves the files behind when it can't revoke (e.g. a broken renewal config),
+			// so remove them either way. No host uses them; that was checked above.
+			// The row is already deleted and audited, so leftover files are logged, not thrown:
+			// an error here would report a completed delete as failed, and a retry would 404.
+			const leftover = internalCertificate.removeLetsEncryptFiles(row.id);
+			if (leftover.length) {
+				logger.warn(
+					`Certificate #${row.id} was deleted, but these files could not be removed and should be deleted by hand: ${leftover.join(", ")}`,
+				);
+			}
 		}
 		return true;
 	},
 
+	/**
+	 * Removes a certificate's certbot files, as `certbot delete` would. Never throws.
+	 *
+	 * @param   {Number}  certificateId
+	 * @returns {String[]}  paths that could not be removed
+	 */
+	removeLetsEncryptFiles: (certificateId) => {
+		const paths = [
+			internalCertificate.getLiveCertPath(certificateId),
+			`/etc/letsencrypt/archive/npm-${certificateId}`,
+			`/etc/letsencrypt/renewal/npm-${certificateId}.conf`,
+			`/etc/letsencrypt/credentials/credentials-${certificateId}`,
+		];
+		const failed = [];
+		for (const p of paths) {
+			try {
+				fs.rmSync(p, { recursive: true, force: true });
+			} catch (err) {
+				logger.error(`Could not remove ${p}: ${err.message}`);
+				failed.push(p);
+			}
+		}
+		return failed;
+	},
+
 	/**
 	 * All Certs
 	 *
@@ -545,13 +767,15 @@ const internalCertificate = {
 
 	/**
 	 * Validates that the certs provided are good.
-	 * No access required here, nothing is changed or stored.
+	 * Nothing is changed or stored, but the files go through openssl, so it needs the same access as creating one.
 	 *
+	 * @param   {Access}  access
 	 * @param   {Object}  data
 	 * @param   {Object}  data.files
 	 * @returns {Promise}
 	 */
-	validate: (data) => {
+	validate: async (access, data) => {
+		await access.can("certificates:create");
 		// Put file contents into an object
 		const files = {};
 		_.map(data.files, (file, name) => {
@@ -600,27 +824,75 @@ const internalCertificate = {
 			throw new error.ValidationError("Cannot upload certificates for this type of provider");
 		}
 
-		const validations = await internalCertificate.validate(data);
+		const validations = await internalCertificate.validate(access, data);
 		if (typeof validations.certificate === "undefined") {
 			throw new error.ValidationError("Certificate file was not provided");
 		}
 
+		// row came through get(), which omits meta.certificate_key so it never goes out over
+		// the API. update() replaces meta wholesale, so build on the stored meta (read from the
+		// model directly) or an upload without a key would erase the stored one.
+		const stored = await certificateModel.query().where("id", row.id).first();
+		const meta = { ...(stored?.meta || {}) };
+		const uploaded = [];
+
 		_.map(data.files, (file, name) => {
 			if (internalCertificate.allowedSslFiles.indexOf(name) !== -1) {
-				row.meta[name] = file.data.toString();
+				meta[name] = file.data.toString();
+				uploaded.push(name);
 			}
 		});
 
+		// Fail before anything is written: without a key, writeCustomCert would overwrite
+		// fullchain.pem and then fail on privkey.pem, leaving nginx with a mismatched pair.
+		if (!meta.certificate_key) {
+			throw new error.ValidationError("Certificate Key file was not provided");
+		}
+		// The key may be the stored one (a certificate-only upload), so check the pair belongs
+		// together; a mismatch would make nginx -t fail for every host on the next reload.
+		await internalCertificate.checkKeyMatchesCertificate(meta.certificate, meta.certificate_key);
+
 		const certificate = await internalCertificate.update(access, {
 			id: data.id,
 			expires_on: moment(validations.certificate.dates.to, "X").format("YYYY-MM-DD HH:mm:ss"),
-			domain_names: [validations.certificate.cn],
-			meta: _.clone(row.meta), // Prevent the update method from changing this value that we'll use later
+			domain_names: validations.certificate.cn ? [validations.certificate.cn] : [],
+			meta: _.clone(meta), // Prevent the update method from changing this value that we'll use later
 		});
 
-		certificate.meta = row.meta;
+		certificate.meta = meta;
 		await internalCertificate.writeCustomCert(certificate);
-		return _.pick(row.meta, internalCertificate.allowedSslFiles);
+		// Only echo back what was uploaded in this request, never the stored key
+		return _.pick(meta, uploaded);
+	},
+
+	/**
+	 * Throws unless the private key is the one the certificate was issued for, by comparing the
+	 * public key derived from each.
+	 *
+	 * @param   {String}  certificate  PEM certificate (the leaf comes first)
+	 * @param   {String}  privateKey   PEM private key
+	 * @returns {Promise}
+	 */
+	checkKeyMatchesCertificate: async (certificate, privateKey) => {
+		const certPath = await tempWrite(certificate);
+		const keyPath = await tempWrite(privateKey);
+		try {
+			const [certPub, keyPub] = await Promise.all([
+				utils.execFile("openssl", ["x509", "-in", certPath, "-pubkey", "-noout"]),
+				utils.execFile("openssl", ["pkey", "-in", keyPath, "-pubout"]),
+			]);
+			if (!certPub.trim() || certPub.trim() !== keyPub.trim()) {
+				throw new error.ValidationError("Certificate does not match the Certificate Key");
+			}
+		} catch (err) {
+			if (err instanceof error.ValidationError) {
+				throw err;
+			}
+			throw new error.ValidationError("Could not check that the Certificate matches the Certificate Key");
+		} finally {
+			fs.rmSync(certPath, { force: true });
+			fs.rmSync(keyPath, { force: true });
+		}
 	},
 
 	/**
@@ -683,13 +955,15 @@ const internalCertificate = {
 
 		try {
 			const result = await utils.execFile("openssl", ["x509", "-in", certificateFile, "-subject", "-noout"]);
+
 			// Examples:
 			// subject=CN = *.jc21.com
 			// subject=CN = something.example.com
-			const regex = /(?:subject=)?[^=]+=\s+(\S+)/gim;
+			// subject=CN=*.jc21.com
+			const regex = /(?:subject=)?[^=]+=\s*(\S+)/gim;
 			const match = regex.exec(result);
 			if (match && typeof match[1] !== "undefined") {
-				certData.cn = match[1];
+				certData.cn = match[1].trim();
 			}
 
 			const result2 = await utils.execFile("openssl", ["x509", "-in", certificateFile, "-issuer", "-noout"]);
@@ -779,6 +1053,7 @@ const internalCertificate = {
 
 		const args = [
 			"certonly",
+			"-n", // non-interactive
 			"--config",
 			letsencryptConfig,
 			"--work-dir",
@@ -808,7 +1083,7 @@ const internalCertificate = {
 
 		logger.info(`Command: ${certbotCommand} ${args ? args.join(" ") : ""}`);
 
-		const result = await utils.execFile(certbotCommand, args, adds.opts);
+		const result = await runCertbot(args, adds.opts);
 		logger.success(result);
 		return result;
 	},
@@ -834,6 +1109,7 @@ const internalCertificate = {
 
 		const args = [
 			"certonly",
+			"-n", // non-interactive
 			"--config",
 			letsencryptConfig,
 			"--work-dir",
@@ -856,7 +1132,8 @@ const internalCertificate = {
 		if (hasConfigArg) {
 			args.push(`--${dnsPlugin.full_plugin_name}-credentials`, credentialsLocation);
 		}
-		if (certificate.meta.propagation_seconds !== undefined) {
+		// The route53 plugin waits for the change through the AWS API and has no propagation option
+		if (certificate.meta.propagation_seconds !== undefined && certificate.meta.dns_provider !== "route53") {
 			args.push(
 				`--${dnsPlugin.full_plugin_name}-propagation-seconds`,
 				certificate.meta.propagation_seconds.toString(),
@@ -874,13 +1151,28 @@ const internalCertificate = {
 		logger.info(`Command: ${certbotCommand} ${args ? args.join(" ") : ""}`);
 
 		try {
-			const result = await utils.execFile(certbotCommand, args, adds.opts);
+			const result = await runCertbot(
+				args,
+				adds.opts,
+				certificate.meta.dns_provider,
+				certificate.meta.dns_provider_credentials,
+			);
 			logger.info(result);
 			return result;
-		} catch (err) {
-			// Don't fail if file does not exist, so no need for action in the callback
+		} finally {
+			// Remove the credentials file whether certbot succeeded or failed.
+			//
+			// This cleanup used to sit in a catch block, so it only ran when issuance FAILED.
+			// A certificate that issued successfully left its DNS provider API credentials in
+			// /etc/letsencrypt/credentials for the entire life of that certificate. Nothing
+			// reads the file between certbot runs, so there is no reason to keep it:
+			// renewLetsEncryptSslWithDnsChallenge() writes it again immediately before each
+			// renewal.
+			//
+			// unlink is fire-and-forget with an empty callback. If the file is already gone
+			// that is the end state we wanted anyway, and a missing file must never turn a
+			// successful issuance into a failure.
 			fs.unlink(credentialsLocation, () => {});
-			throw err;
 		}
 	},
 
@@ -890,7 +1182,8 @@ const internalCertificate = {
 	 * @param   {Number}  data.id
 	 * @returns {Promise}
 	 */
-	renew: async (access, data) => {
+	renew: async (access, data, options = {}) => {
+		const reload = options.reload !== false;
 		await access.can("certificates:update", data);
 		const certificate = await internalCertificate.get(access, data);
 
@@ -916,12 +1209,34 @@ const internalCertificate = {
 				meta: updatedCertificate,
 			});
 
+			// The renewal has succeeded and been recorded at this point. A reload failure
+			// (e.g. an unrelated host's config is broken) must not turn it into an error;
+			// nginx picks the new files up on the next successful reload.
+			if (reload) {
+				await internalCertificate.reloadAfterRenew();
+			}
+
 			return updatedCertificate;
 		}
 
 		throw new error.ValidationError("Only Let'sEncrypt certificates can be renewed");
 	},
 
+	/**
+	 * Reloads nginx so it serves renewed certificate files. Errors are logged, not thrown.
+	 *
+	 * @returns {Promise<Boolean>}  whether the reload succeeded
+	 */
+	reloadAfterRenew: async () => {
+		try {
+			await internalNginx.reload();
+			return true;
+		} catch (err) {
+			logger.error(`Certificate renewed but nginx could not be reloaded: ${err.message}`);
+			return false;
+		}
+	},
+
 	/**
 	 * @param   {Object}  certificate   the certificate row
 	 * @returns {Promise}
@@ -958,7 +1273,7 @@ const internalCertificate = {
 
 		logger.info(`Command: ${certbotCommand} ${args ? args.join(" ") : ""}`);
 
-		const result = await utils.execFile(certbotCommand, args, adds.opts);
+		const result = await runCertbot(args, adds.opts);
 		logger.info(result);
 		return result;
 	},
@@ -977,6 +1292,43 @@ const internalCertificate = {
 			`Renewing LetsEncrypt certificates via ${dnsPlugin.name} for Cert #${certificate.id}: ${certificate.domain_names.join(", ")}`,
 		);
 
+		// certbot reads the DNS credentials back from the path recorded in the renewal config
+		// it wrote at issuance time, for example:
+		//
+		//     authenticator = dns-cloudflare
+		//     dns_cloudflare_credentials = /etc/letsencrypt/credentials/credentials-27
+		//
+		// so the file has to be present for the duration of this run. Write it here and remove
+		// it again below rather than leaving it on disk between renewals.
+		//
+		// Leaving it is an avoidable exposure. Anything running as root - a compromised
+		// process, a script, malware - can read the token and use it to issue valid Let's
+		// Encrypt certificates for the domain. Those certificates are genuinely trusted, so
+		// traffic presented with them passes TLS inspection, IDS/IPS and DLP that would
+		// otherwise flag it, and an exfiltration path built on them looks like ordinary
+		// HTTPS. The exposure window should be one certbot run, not the life of the
+		// certificate.
+		//
+		// The value is not on the certificate object we were handed: renew() sources that from
+		// internalCertificate.get(), which pipes the row through utils.omitRow(omissions()) so
+		// meta.dns_provider_credentials can never travel out over the API. Read the row from
+		// the model directly to get at it.
+		const row = await certificateModel.query().where("id", certificate.id).first();
+		const credentials = row?.meta?.dns_provider_credentials;
+		const credentialsLocation = `/etc/letsencrypt/credentials/credentials-${certificate.id}`;
+
+		if (credentials) {
+			fs.mkdirSync("/etc/letsencrypt/credentials", { recursive: true });
+			fs.writeFileSync(credentialsLocation, credentials, { mode: 0o600 });
+		} else {
+			// Nothing stored to write. A certificate issued under the previous behaviour may
+			// still have its file on disk; leave it be and let certbot decide. Throwing here
+			// would break a renewal that would otherwise have succeeded.
+			logger.warn(
+				`No stored DNS credentials for Cert #${certificate.id}; relying on any existing ${credentialsLocation}`,
+			);
+		}
+
 		const args = [
 			"renew",
 			"--force-renewal",
@@ -1004,9 +1356,19 @@ const internalCertificate = {
 
 		logger.info(`Command: ${certbotCommand} ${args ? args.join(" ") : ""}`);
 
-		const result = await utils.execFile(certbotCommand, args, adds.opts);
-		logger.info(result);
-		return result;
+		try {
+			const result = await runCertbot(args, adds.opts, certificate.meta.dns_provider, credentials);
+			logger.info(result);
+			return result;
+		} finally {
+			// Only clean up a file we put there ourselves. If `credentials` came back empty we
+			// wrote nothing, and an older file left on disk by the previous behaviour is the
+			// only thing keeping that certificate renewable - deleting it would break the next
+			// run for no gain.
+			if (credentials) {
+				fs.unlink(credentialsLocation, () => {});
+			}
+		}
 	},
 
 	/**
@@ -1038,7 +1400,7 @@ const internalCertificate = {
 		logger.info(`Command: ${certbotCommand} ${args ? args.join(" ") : ""}`);
 
 		try {
-			const result = await utils.execFile(certbotCommand, args, adds.opts);
+			const result = await runCertbot(args, adds.opts);
 			await utils.exec(`rm -f '/etc/letsencrypt/credentials/credentials-${certificate.id}' || true`);
 			logger.info(result);
 			return result;

backend/internal/dead-host.js

@@ -54,9 +54,9 @@ const internalDeadHost = {
 			thisData.advanced_config = "";
 		}
 
-		const row = await deadHostModel.query()
-			.insertAndFetch(thisData)
-			.then(utils.omitRow(omissions()));
+		internalHost.assertAdvancedConfigAllowed(access, thisData);
+
+		const row = await deadHostModel.query().insertAndFetch(thisData).then(utils.omitRow(omissions()));
 
 		// Add to audit log
 		await internalAuditLog.add(access, {
@@ -132,6 +132,9 @@ const internalDeadHost = {
 			);
 		}
 
+		// Before the certificate is created, so a refused request doesn't leave one behind
+		internalHost.assertAdvancedConfigAllowed(access, data, row);
+
 		if (createCertificate) {
 			const cert = await internalCertificate.createQuickCertificate(access, {
 				domain_names: data.domain_names || row.domain_names,
@@ -153,12 +156,8 @@ const internalDeadHost = {
 
 		thisData = internalHost.cleanSslHstsData(thisData, row);
 
-
 		// do the row update
-		await deadHostModel
-			.query()
-			.where({id: data.id})
-			.patch(data);
+		await deadHostModel.query().where({ id: data.id }).patch(data);
 
 		// Add to audit log
 		await internalAuditLog.add(access, {
@@ -168,15 +167,18 @@ const internalDeadHost = {
 			meta: thisData,
 		});
 
-		const thisRow = await internalDeadHost
-			.get(access, {
-				id: thisData.id,
-				expand: ["owner", "certificate"],
-			});
+		const thisRow = await internalDeadHost.get(access, {
+			id: thisData.id,
+			expand: ["owner", "certificate"],
+		});
 
 		// Configure nginx
-		const newMeta = await internalNginx.configure(deadHostModel, "dead_host", row);
-		row.meta = newMeta;
+		if (!thisRow.enabled) {
+			// No need to add nginx config if host is disabled
+			return _.omit(internalHost.cleanRowCertificateMeta(thisRow), omissions());
+		}
+		const newMeta = await internalNginx.configure(deadHostModel, "dead_host", thisRow);
+		thisRow.meta = newMeta;
 		return _.omit(internalHost.cleanRowCertificateMeta(thisRow), omissions());
 	},
 
@@ -194,7 +196,7 @@ const internalDeadHost = {
 			.query()
 			.where("is_deleted", 0)
 			.andWhere("id", data.id)
-			.allowGraph("[owner,certificate]")
+			.allowGraph(deadHostModel.defaultAllowGraph)
 			.first();
 
 		if (accessData.permission_visibility !== "all") {
@@ -206,7 +208,7 @@ const internalDeadHost = {
 		}
 
 		const row = await query.then(utils.omitRow(omissions()));
-		if (!row || !row.id) {
+		if (!row?.id) {
 			throw new errs.ItemNotFoundError(data.id);
 		}
 		// Custom omissions
@@ -224,18 +226,15 @@ const internalDeadHost = {
 	 * @returns {Promise}
 	 */
 	delete: async (access, data) => {
-		await access.can("dead_hosts:delete", data.id)
+		await access.can("dead_hosts:delete", data.id);
 		const row = await internalDeadHost.get(access, { id: data.id });
-		if (!row || !row.id) {
+		if (!row?.id) {
 			throw new errs.ItemNotFoundError(data.id);
 		}
 
-		await deadHostModel
-			.query()
-			.where("id", row.id)
-			.patch({
-				is_deleted: 1,
-			});
+		await deadHostModel.query().where("id", row.id).patch({
+			is_deleted: 1,
+		});
 
 		// Delete Nginx Config
 		await internalNginx.deleteConfig("dead_host", row);
@@ -259,12 +258,12 @@ const internalDeadHost = {
 	 * @returns {Promise}
 	 */
 	enable: async (access, data) => {
-		await access.can("dead_hosts:update", data.id)
+		await access.can("dead_hosts:update", data.id);
 		const row = await internalDeadHost.get(access, {
 			id: data.id,
 			expand: ["certificate", "owner"],
 		});
-		if (!row || !row.id) {
+		if (!row?.id) {
 			throw new errs.ItemNotFoundError(data.id);
 		}
 		if (row.enabled) {
@@ -273,12 +272,9 @@ const internalDeadHost = {
 
 		row.enabled = 1;
 
-		await deadHostModel
-			.query()
-			.where("id", row.id)
-			.patch({
-				enabled: 1,
-			});
+		await deadHostModel.query().where("id", row.id).patch({
+			enabled: 1,
+		});
 
 		// Configure nginx
 		await internalNginx.configure(deadHostModel, "dead_host", row);
@@ -301,9 +297,9 @@ const internalDeadHost = {
 	 * @returns {Promise}
 	 */
 	disable: async (access, data) => {
-		await access.can("dead_hosts:update", data.id)
+		await access.can("dead_hosts:update", data.id);
 		const row = await internalDeadHost.get(access, { id: data.id });
-		if (!row || !row.id) {
+		if (!row?.id) {
 			throw new errs.ItemNotFoundError(data.id);
 		}
 		if (!row.enabled) {
@@ -312,12 +308,9 @@ const internalDeadHost = {
 
 		row.enabled = 0;
 
-		await deadHostModel
-			.query()
-			.where("id", row.id)
-			.patch({
-				enabled: 0,
-			});
+		await deadHostModel.query().where("id", row.id).patch({
+			enabled: 0,
+		});
 
 		// Delete Nginx Config
 		await internalNginx.deleteConfig("dead_host", row);
@@ -342,12 +335,12 @@ const internalDeadHost = {
 	 * @returns {Promise}
 	 */
 	getAll: async (access, expand, searchQuery) => {
-		const accessData = await access.can("dead_hosts:list")
+		const accessData = await access.can("dead_hosts:list");
 		const query = deadHostModel
 			.query()
 			.where("is_deleted", 0)
 			.groupBy("id")
-			.allowGraph("[owner,certificate]")
+			.allowGraph(deadHostModel.defaultAllowGraph)
 			.orderBy(castJsonIfNeed("domain_names"), "ASC");
 
 		if (accessData.permission_visibility !== "all") {

backend/internal/host.js

@@ -1,10 +1,110 @@
 import _ from "lodash";
+import errs from "../lib/error.js";
 import { castJsonIfNeed } from "../lib/helpers.js";
 import deadHostModel from "../models/dead_host.js";
 import proxyHostModel from "../models/proxy_host.js";
 import redirectionHostModel from "../models/redirection_host.js";
 
 const internalHost = {
+	/**
+	 * Only admins may set or change advanced_config (raw nginx config). Throws for anyone else.
+	 * On create (no `row`), any non-empty advanced_config is refused, including on locations.
+	 * On update, the host's advanced_config must be unchanged (or omitted), and locations must
+	 * only carry advanced configs the host already has (see assertLocationAdvancedConfigUnchanged).
+	 *
+	 * @param   {Access}  access
+	 * @param   {Object}  data    incoming host data; data.locations may be filled in on update
+	 * @param   {Object}  [row]   the existing host, when updating
+	 * @throws  {PermissionError}
+	 */
+	assertAdvancedConfigAllowed: (access, data, row) => {
+		if (access.isAdmin?.()) {
+			return;
+		}
+		const deny = () => {
+			throw new errs.PermissionError("You do not have permission to modify advanced configuration");
+		};
+
+		if (!row) {
+			const hasConfig = (v) => typeof v === "string" && v.trim() !== "";
+			if (hasConfig(data.advanced_config)) {
+				deny();
+			}
+			if (Array.isArray(data.locations) && data.locations.some((loc) => hasConfig(loc?.advanced_config))) {
+				deny();
+			}
+			return;
+		}
+
+		if (typeof data.advanced_config !== "undefined" && data.advanced_config !== row.advanced_config) {
+			deny();
+		}
+		if (Array.isArray(data.locations)) {
+			internalHost.assertLocationAdvancedConfigUnchanged(data.locations, row.locations);
+		}
+	},
+
+	/**
+	 * For non-admins: every location's advanced_config must be empty or one that already exists
+	 * on this host. Matching is by value, not path, so renaming or reordering a location that
+	 * carries admin-written config is allowed, but new or edited config is not. Each existing
+	 * config can be used once (no duplicating it onto other paths), and same-path matches are
+	 * taken first. Clearing a config is allowed: deleting the location is, and has the same
+	 * effect, and a location deleted and re-added at the same path arrives with an empty config.
+	 * A location that omits advanced_config keeps the one at its path, unless another location
+	 * explicitly carries it. Mutates `locations`.
+	 *
+	 * @param   {Array}  locations          incoming locations
+	 * @param   {Array}  [existingLocations] the host's current locations
+	 * @throws  {PermissionError}
+	 */
+	assertLocationAdvancedConfigUnchanged: (locations, existingLocations) => {
+		const existing = Array.isArray(existingLocations) ? existingLocations : [];
+		const advAtPath = (path) => existing.find((l) => l.path === path)?.advanced_config || "";
+		const pool = existing.map((l) => l.advanced_config || "").filter(Boolean);
+		const take = (value) => {
+			const i = pool.indexOf(value);
+			if (i === -1) {
+				return false;
+			}
+			pool.splice(i, 1);
+			return true;
+		};
+		const deny = () => {
+			throw new errs.PermissionError("You do not have permission to modify advanced configuration");
+		};
+
+		// Pass 1: locations that keep the config at their own path
+		const unresolved = [];
+		const omitted = [];
+		for (const loc of locations) {
+			const atPath = advAtPath(loc.path);
+			if (typeof loc.advanced_config === "undefined") {
+				omitted.push(loc);
+			} else if ((loc.advanced_config || "") === atPath && (!atPath || take(atPath))) {
+				// unchanged
+			} else {
+				unresolved.push(loc);
+			}
+		}
+
+		// Pass 2: renamed/moved locations may carry an existing config not already claimed, or
+		// none at all
+		for (const loc of unresolved) {
+			if (loc.advanced_config && !take(loc.advanced_config)) {
+				deny();
+			}
+		}
+
+		// Pass 3: locations that omit advanced_config keep the one at their path, if no location
+		// that names it explicitly has claimed it. Running this last stops a new location at an
+		// old path taking the config away from the location it was renamed to.
+		for (const loc of omitted) {
+			const atPath = advAtPath(loc.path);
+			loc.advanced_config = atPath && take(atPath) ? atPath : "";
+		}
+	},
+
 	/**
 	 * Makes sure that the ssl_* and hsts_* fields play nicely together.
 	 * ie: if there is no cert, then force_ssl is off.
@@ -79,6 +179,9 @@ const internalHost = {
 			redirection_hosts: [],
 		};
 
+		// No relations are eager-loaded here: this scans every host just to match domains, and
+		// the few that match are rendered through internalNginx.generateConfig(), which loads
+		// their certificate and access list itself (loadRenderRelations).
 		const proxyRes = await proxyHostModel.query().where("is_deleted", 0);
 		responseObject.proxy_hosts = internalHost._getHostsWithDomains(proxyRes, domainNames);
 		responseObject.total_count += responseObject.proxy_hosts.length;

backend/internal/log-viewer.js

@@ -0,0 +1,229 @@
+import fs from "node:fs";
+import errs from "../lib/error.js";
+import internalDeadHost from "./dead-host.js";
+import internalProxyHost from "./proxy-host.js";
+import internalRedirectionHost from "./redirection-host.js";
+import internalStream from "./stream.js";
+
+const SYSTEM_LOG_FILE = "/data/logs/backend.log";
+const LETSENCRYPT_LOG_FILE = "/data/logs/letsencrypt.log";
+
+// Matches the access_log/error_log paths written by the nginx templates
+// (see backend/templates/{proxy_host,redirection_host,dead_host,stream}.conf).
+// This is a fixed, server-side lookup table - a host log path is always derived
+// from a validated `host_type` enum + numeric `host_id`, never from a client-supplied
+// path or filename, so there is no path-traversal surface here.
+const HOST_FILE_PREFIX = {
+	proxy: "proxy-host",
+	redirection: "redirection-host",
+	dead: "dead-host",
+	stream: "stream",
+};
+
+const DEFAULT_LINES = 200;
+const MAX_LINES = 1000;
+const CHUNK_SIZE = 64 * 1024;
+// Never scan further back than this, regardless of how many lines were requested,
+// so a huge or pathological log file can't turn a single request into unbounded I/O.
+const MAX_SCAN_BYTES = 5 * 1024 * 1024;
+
+/**
+ * Reads at most `maxLines` lines from the end of a file, without loading the
+ * whole file into memory. Reads backwards in fixed-size chunks until enough
+ * newlines have been seen, the start of the file is reached, or the hard
+ * MAX_SCAN_BYTES ceiling is hit.
+ *
+ * @param   {String} filePath
+ * @param   {Number} maxLines
+ * @returns {Promise<{lines: String[], size: Number, truncated: Boolean}>}
+ */
+const readLastLines = async (filePath, maxLines) => {
+	let handle;
+	try {
+		handle = await fs.promises.open(filePath, "r");
+		const stat = await handle.stat();
+		const { size } = stat;
+
+		if (size === 0) {
+			return { lines: [], size: 0, truncated: false };
+		}
+
+		let position = size;
+		let scanned = 0;
+		let newlineCount = 0;
+		const chunks = [];
+
+		while (position > 0 && newlineCount <= maxLines && scanned < MAX_SCAN_BYTES) {
+			const readSize = Math.min(CHUNK_SIZE, position);
+			position -= readSize;
+			const buffer = Buffer.alloc(readSize);
+			await handle.read(buffer, 0, readSize, position);
+			scanned += readSize;
+			for (let i = buffer.length - 1; i >= 0; i--) {
+				if (buffer[i] === 0x0a) newlineCount++;
+			}
+			chunks.unshift(buffer);
+		}
+
+		const truncated = position > 0 && scanned >= MAX_SCAN_BYTES;
+
+		// If we didn't start reading from byte 0, the first line in our buffer is only
+		// partial *unless* it happens that `position` landed exactly on a line boundary
+		// (the byte right before it is a newline) - check that one byte to avoid
+		// silently dropping a perfectly valid line.
+		let startsOnLineBoundary = position === 0;
+		if (position > 0) {
+			const boundaryByte = Buffer.alloc(1);
+			await handle.read(boundaryByte, 0, 1, position - 1);
+			startsOnLineBoundary = boundaryByte[0] === 0x0a;
+		}
+
+		const text = Buffer.concat(chunks).toString("utf8");
+		const allLines = text.split("\n");
+
+		if (!startsOnLineBoundary && allLines.length > 0) {
+			allLines.shift();
+		}
+		// Drop the trailing empty element caused by a final trailing newline.
+		if (allLines.length > 0 && allLines[allLines.length - 1] === "") {
+			allLines.pop();
+		}
+
+		return { lines: allLines.slice(-maxLines), size, truncated };
+	} finally {
+		if (handle) {
+			await handle.close();
+		}
+	}
+};
+
+/**
+ * Resolves a validated {type, host_type, host_id, channel} selection to the
+ * fixed, absolute path of the log file on disk. Throws if the combination
+ * isn't a recognised source.
+ *
+ * `channel` ("access" | "error") picks which of the two log files nginx writes
+ * per host - it's deliberately not named "stream" to avoid confusion with the
+ * "stream" host_type (TCP/UDP stream hosts).
+ *
+ * @param   {Object} data
+ * @returns {String}
+ */
+const resolveFilePath = (data) => {
+	switch (data.type) {
+		case "system":
+			return SYSTEM_LOG_FILE;
+		case "letsencrypt":
+			return LETSENCRYPT_LOG_FILE;
+		case "host": {
+			const prefix = HOST_FILE_PREFIX[data.host_type];
+			if (!prefix || !data.host_id || !["access", "error"].includes(data.channel)) {
+				throw new errs.ValidationError("Invalid host log source");
+			}
+			return `/data/logs/${prefix}-${data.host_id}_${data.channel}.log`;
+		}
+		default:
+			throw new errs.ItemNotFoundError(data.type);
+	}
+};
+
+/**
+ * @param   {Array} rows
+ * @returns {Array}
+ */
+const toHostOptions = (rows) =>
+	rows.map((row) => ({
+		id: row.id,
+		label: Array.isArray(row.domain_names) ? row.domain_names.join(", ") : `Host #${row.id}`,
+	}));
+
+const internalLogViewer = {
+	/**
+	 * Lists the log sources available for the log viewer: the system (backend)
+	 * log, the Let's Encrypt (certbot) log, and one entry per host the caller
+	 * can see, for each host type.
+	 *
+	 * @param   {Access} access
+	 * @returns {Promise}
+	 */
+	listSources: async (access) => {
+		await access.can("logs:list");
+
+		const [proxyHosts, redirectionHosts, deadHosts, streams] = await Promise.all([
+			internalProxyHost.getAll(access),
+			internalRedirectionHost.getAll(access),
+			internalDeadHost.getAll(access),
+			internalStream.getAll(access),
+		]);
+
+		return {
+			system: { label: "System" },
+			letsencrypt: { label: "Let's Encrypt" },
+			hosts: {
+				proxy: toHostOptions(proxyHosts),
+				redirection: toHostOptions(redirectionHosts),
+				dead: toHostOptions(deadHosts),
+				stream: streams.map((row) => ({
+					id: row.id,
+					label: `Port ${row.incoming_port} → ${row.forwarding_host}:${row.forwarding_port}`,
+				})),
+			},
+		};
+	},
+
+	/**
+	 * Returns the last N lines of the requested log source, optionally
+	 * filtered by level and/or a plain-text search term.
+	 *
+	 * @param   {Access} access
+	 * @param   {Object} data
+	 * @param   {String} data.type        "system" | "letsencrypt" | "host"
+	 * @param   {String} [data.host_type] "proxy" | "redirection" | "dead" | "stream"
+	 * @param   {Number} [data.host_id]
+	 * @param   {String} [data.channel]   "access" | "error"
+	 * @param   {Number} [data.lines]
+	 * @param   {String} [data.level]
+	 * @param   {String} [data.search]
+	 * @returns {Promise}
+	 */
+	tail: async (access, data) => {
+		await access.can("logs:list");
+
+		const filePath = resolveFilePath(data);
+		const lines = Math.min(Math.max(data.lines || DEFAULT_LINES, 1), MAX_LINES);
+
+		let result;
+		try {
+			result = await readLastLines(filePath, lines);
+		} catch (err) {
+			if (err.code === "ENOENT") {
+				return { lines: [], size: 0, truncated: false, exists: false };
+			}
+			throw err;
+		}
+
+		let outputLines = result.lines;
+
+		// Only the system log is written in our own "LEVEL   [scope]" format - level
+		// filtering on nginx/certbot lines would just match nothing and look like an
+		// empty log, so the filter is a no-op for any other source.
+		if (data.type === "system" && data.level) {
+			const needle = ` ${data.level.toUpperCase().padEnd(7)} `;
+			outputLines = outputLines.filter((line) => line.includes(needle));
+		}
+
+		if (data.search) {
+			const needle = data.search.toLowerCase();
+			outputLines = outputLines.filter((line) => line.toLowerCase().includes(needle));
+		}
+
+		return {
+			lines: outputLines,
+			size: result.size,
+			truncated: result.truncated,
+			exists: true,
+		};
+	},
+};
+
+export default internalLogViewer;

backend/internal/nginx.js

@@ -1,10 +1,14 @@
 import fs from "node:fs";
+import net from "node:net";
 import { dirname } from "node:path";
 import { fileURLToPath } from "node:url";
 import _ from "lodash";
 import errs from "../lib/error.js";
+import { assertAccessListRenderable, assertHostRenderable, isTokenSafe, unsafeValueError } from "../lib/nginx-values.js";
 import utils from "../lib/utils.js";
 import { debug, nginx as logger } from "../logger.js";
+import accessListModel from "../models/access_list.js";
+import certificateModel from "../models/certificate.js";
 
 const __filename = fileURLToPath(import.meta.url);
 const __dirname = dirname(__filename);
@@ -34,12 +38,33 @@ const internalNginx = {
 				// We're deleting this config regardless.
 				// Don't throw errors, as the file may not exist at all
 				// Delete the .err file too
-				return internalNginx.deleteConfig(host_type, host, false, true);
+				return internalNginx.deleteConfig(host_type, host, true);
 			})
 			.then(() => {
-				return internalNginx.generateConfig(host_type, host);
+				return internalNginx.generateConfig(host_type, host).then(
+					() => true,
+					(err) => {
+						if (!err.unsafeNginxValue) {
+							throw err;
+						}
+						// Nothing was written. Report it on the host like a failed nginx test.
+						logger.error(err.message);
+						combined_meta = _.assign({}, host.meta, {
+							nginx_online: false,
+							nginx_err: err.message,
+						});
+						return model
+							.query()
+							.where("id", host.id)
+							.patch({ meta: combined_meta })
+							.then(() => false);
+					},
+				);
 			})
-			.then(() => {
+			.then((generated) => {
+				if (!generated) {
+					return;
+				}
 				// Test nginx again and update meta with result
 				return internalNginx
 					.test()
@@ -83,10 +108,12 @@ const internalNginx = {
 								meta: combined_meta,
 							})
 							.then(() => {
-								internalNginx.renameConfigAsError(host_type, host);
+								// Keep the failed config as a .err file for inspection
+								return internalNginx.renameConfigAsError(host_type, host);
 							})
 							.then(() => {
-								return internalNginx.deleteConfig(host_type, host, true);
+								// The rename removed the live config already, don't touch the .err file
+								return internalNginx.deleteConfig(host_type, host, false);
 							});
 					});
 			})
@@ -165,6 +192,38 @@ const internalNginx = {
 						host.locations[i],
 					);
 
+					// A location with its own access list overrides the host's,
+					// otherwise it inherits the host's access list
+					let locationAccessList = null;
+					if (locationCopy.access_list_id > 0 && locationCopy.access_list_id !== host.access_list?.id) {
+						locationAccessList = await accessListModel
+							.query()
+							.where("is_deleted", 0)
+							.andWhere("id", locationCopy.access_list_id)
+							.withGraphFetched("[clients,items]")
+							.first();
+					}
+					if (locationAccessList) {
+						assertAccessListRenderable(
+							`Proxy Host #${host.id} location ${locationCopy.path}`,
+							locationAccessList,
+						);
+						locationCopy.access_list = locationAccessList;
+					} else if (
+						locationCopy.access_list_id > 0 &&
+						locationCopy.access_list_id !== host.access_list?.id
+					) {
+						// The location names its own list but it's gone. Don't fall back to the
+						// host's (possibly less restrictive) list; _access.conf denies all instead.
+						logger.error(
+							`Proxy Host #${host.id} location "${locationCopy.path}" references Access List #${locationCopy.access_list_id} which does not exist, all access will be denied`,
+						);
+						locationCopy.access_list = null;
+					} else {
+						locationCopy.access_list_id = host.access_list_id;
+						locationCopy.access_list = host.access_list;
+					}
+
 					if (locationCopy.forward_host.indexOf("/") > -1) {
 						const splitted = locationCopy.forward_host.split("/");
 
@@ -176,7 +235,9 @@ const internalNginx = {
 				}
 			};
 
-			locationRendering().then(() => resolve(renderedLocations));
+			locationRendering()
+				.then(() => resolve(renderedLocations))
+				.catch(reject);
 		});
 	},
 
@@ -185,11 +246,16 @@ const internalNginx = {
 	 * @param   {Object}  host
 	 * @returns {Promise}
 	 */
-	generateConfig: (host_type, host_row) => {
+	generateConfig: async (host_type, host_row) => {
 		// Prevent modifying the original object:
 		const host = JSON.parse(JSON.stringify(host_row));
 		const nice_host_type = internalNginx.getFileFriendlyHostType(host_type);
 
+		await internalNginx.loadRenderRelations(nice_host_type, host);
+
+		// Values are written into the config as-is; refuse any that could add directives
+		assertHostRenderable(nice_host_type, host);
+
 		debug(logger, `Generating ${nice_host_type} Config:`, JSON.stringify(host, null, 2));
 
 		const renderEngine = utils.getRenderEngine();
@@ -217,10 +283,21 @@ const internalNginx = {
 			}
 
 			// For redirection hosts, if the scheme is not http or https, set it to $scheme
-			if (nice_host_type === "redirection_host" && ['http', 'https'].indexOf(host.forward_scheme.toLowerCase()) === -1) {
+			if (
+				nice_host_type === "redirection_host" &&
+				["http", "https"].indexOf(host.forward_scheme.toLowerCase()) === -1
+			) {
 				host.forward_scheme = "$scheme";
 			}
 
+			// A stream forwarding to an IPv6 literal must have the address wrapped in
+			// square brackets before nginx appends ":<port>". Without the brackets nginx
+			// reads the trailing ":<port>" as part of the address and rejects the upstream
+			// ("invalid port in upstream"), so the stream saves but never activates (#5740).
+			if (nice_host_type === "stream" && net.isIPv6(host.forwarding_host)) {
+				host.forwarding_host = `[${host.forwarding_host}]`;
+			}
+
 			if (host.locations) {
 				//logger.info ('host.locations = ' + JSON.stringify(host.locations, null, 2));
 				origLocations = [].concat(host.locations);
@@ -241,26 +318,79 @@ const internalNginx = {
 			// Set the IPv6 setting for the host
 			host.ipv6 = internalNginx.ipv6Enabled();
 
-			locationsPromise.then(() => {
-				renderEngine
-					.parseAndRender(template, host)
-					.then((config_text) => {
-						fs.writeFileSync(filename, config_text, { encoding: "utf8" });
-						debug(logger, "Wrote config:", filename, config_text);
+			locationsPromise
+				.then(() => {
+					renderEngine
+						.parseAndRender(template, host)
+						.then((config_text) => {
+							fs.writeFileSync(filename, config_text, { encoding: "utf8" });
+							debug(logger, "Wrote config:", filename, config_text);
 
-						// Restore locations array
-						host.locations = origLocations;
+							// Restore locations array
+							host.locations = origLocations;
 
-						resolve(true);
-					})
-					.catch((err) => {
-						debug(logger, `Could not write ${filename}:`, err.message);
-						reject(new errs.ConfigurationError(err.message));
-					});
-			});
+							resolve(true);
+						})
+						.catch((err) => {
+							debug(logger, `Could not write ${filename}:`, err.message);
+							reject(new errs.ConfigurationError(err.message));
+						});
+				})
+				.catch((err) => {
+					debug(logger, `Could not render locations for ${filename}:`, err.message);
+					reject(new errs.ConfigurationError(err.message));
+				});
 		});
 	},
 
+	/**
+	 * The templates rely on the certificate and access list relations being loaded.
+	 * Not every caller fetches them (or fetches them fully), and when they're missing
+	 * the config silently renders without SSL or without any access rules (#5919).
+	 * Load anything that is referenced but not loaded onto the given host object.
+	 *
+	 * @param   {String}  host_type
+	 * @param   {Object}  host
+	 * @returns {Promise}
+	 */
+	loadRenderRelations: async (host_type, host) => {
+		if (host.certificate_id > 0 && host.certificate?.id !== host.certificate_id) {
+			const certificate = await certificateModel
+				.query()
+				.where("is_deleted", 0)
+				.andWhere("id", host.certificate_id)
+				.first();
+			host.certificate = certificate ? JSON.parse(JSON.stringify(certificate)) : null;
+		}
+
+		if (host_type === "proxy_host" && host.access_list_id > 0) {
+			if (
+				host.access_list?.id !== host.access_list_id ||
+				!Array.isArray(host.access_list.clients) ||
+				!Array.isArray(host.access_list.items)
+			) {
+				const accessList = await accessListModel
+					.query()
+					.where("is_deleted", 0)
+					.andWhere("id", host.access_list_id)
+					.withGraphFetched("[clients,items]")
+					.first();
+				host.access_list = accessList ? JSON.parse(JSON.stringify(accessList)) : null;
+			}
+
+			if (!host.access_list) {
+				// _access.conf renders "deny all" when the list is missing, so fail closed
+				logger.error(
+					`Proxy Host #${host.id} references Access List #${host.access_list_id} which does not exist, all access will be denied`,
+				);
+			} else if (!host.access_list.clients.length && !host.access_list.items.length) {
+				logger.warn(
+					`Proxy Host #${host.id} uses Access List #${host.access_list_id} which has no clients or authorizations, no access restrictions will apply`,
+				);
+			}
+		}
+	},
+
 	/**
 	 * This generates a temporary nginx config listening on port 80 for the domain names listed
 	 * in the certificate setup. It allows the letsencrypt acme challenge to be requested by letsencrypt
@@ -273,6 +403,15 @@ const internalNginx = {
 		debug(logger, "Generating LetsEncrypt Request Config:", certificate);
 		const renderEngine = utils.getRenderEngine();
 
+		const unsafe = (certificate.domain_names || []).find((name) => !isTokenSafe(name));
+		if (typeof unsafe !== "undefined") {
+			return Promise.reject(
+				unsafeValueError(
+					`Certificate #${certificate.id}: domain name ${JSON.stringify(unsafe)} is not allowed`,
+				),
+			);
+		}
+
 		return new Promise((resolve, reject) => {
 			let template = null;
 			const filename = `/data/nginx/temp/letsencrypt_${certificate.id}.conf`;
@@ -375,8 +514,8 @@ const internalNginx = {
 		const config_file_err = `${config_file}.err`;
 
 		return new Promise((resolve /*, reject*/) => {
-			fs.unlink(config_file, () => {
-				// ignore result, continue
+			fs.unlink(config_file_err, () => {
+				// ignore result, a previous .err file may not exist
 				fs.rename(config_file, config_file_err, () => {
 					// also ignore result, as this is a debugging informative file anyway
 					resolve();

backend/internal/proxy-host.js

@@ -2,7 +2,7 @@ import _ from "lodash";
 import errs from "../lib/error.js";
 import { castJsonIfNeed } from "../lib/helpers.js";
 import utils from "../lib/utils.js";
-import proxyHostModel from "../models/proxy_host.js";
+import proxyHostModel, { scopeToOwnerOrAllowed } from "../models/proxy_host.js";
 import internalAuditLog from "./audit-log.js";
 import internalCertificate from "./certificate.js";
 import internalHost from "./host.js";
@@ -57,6 +57,8 @@ const internalProxyHost = {
 					thisData.advanced_config = "";
 				}
 
+				internalHost.assertAdvancedConfigAllowed(access, thisData);
+
 				return proxyHostModel.query().insertAndFetch(thisData).then(utils.omitRow(omissions()));
 			})
 			.then((row) => {
@@ -83,7 +85,7 @@ const internalProxyHost = {
 					expand: ["certificate", "owner", "access_list.[clients,items]"],
 				});
 			})
-			.then((row) => {
+			.then(async (row) => {
 				// Configure nginx
 				return internalNginx.configure(proxyHostModel, "proxy_host", row).then(() => {
 					return row;
@@ -115,9 +117,9 @@ const internalProxyHost = {
 	 */
 	update: (access, data) => {
 		let thisData = data;
-		const create_certificate = thisData.certificate_id === "new";
+		const createCertificate = thisData.certificate_id === "new";
 
-		if (create_certificate) {
+		if (createCertificate) {
 			delete thisData.certificate_id;
 		}
 
@@ -155,7 +157,10 @@ const internalProxyHost = {
 					);
 				}
 
-				if (create_certificate) {
+				// Before the certificate is created, so a refused request doesn't leave one behind
+				internalHost.assertAdvancedConfigAllowed(access, thisData, row);
+
+				if (createCertificate) {
 					return internalCertificate
 						.createQuickCertificate(access, {
 							domain_names: thisData.domain_names || row.domain_names,
@@ -208,7 +213,7 @@ const internalProxyHost = {
 						id: thisData.id,
 						expand: ["owner", "certificate", "access_list.[clients,items]"],
 					})
-					.then((row) => {
+					.then(async (row) => {
 						if (!row.enabled) {
 							// No need to add nginx config if host is disabled
 							return row;
@@ -232,7 +237,6 @@ const internalProxyHost = {
 	 */
 	get: (access, data) => {
 		const thisData = data || {};
-
 		return access
 			.can("proxy_hosts:get", thisData.id)
 			.then((access_data) => {
@@ -240,11 +244,11 @@ const internalProxyHost = {
 					.query()
 					.where("is_deleted", 0)
 					.andWhere("id", thisData.id)
-					.allowGraph("[owner,access_list.[clients,items],certificate]")
+					.allowGraph(proxyHostModel.defaultAllowGraph)
 					.first();
 
 				if (access_data.permission_visibility !== "all") {
-					query.andWhere("owner_user_id", access.token.getUserId(1));
+					scopeToOwnerOrAllowed(query, access.token.getUserId(1), access.getPermissions?.()?.meta);
 				}
 
 				if (typeof thisData.expand !== "undefined" && thisData.expand !== null) {
@@ -254,7 +258,7 @@ const internalProxyHost = {
 				return query.then(utils.omitRow(omissions()));
 			})
 			.then((row) => {
-				if (!row || !row.id) {
+				if (!row?.id) {
 					throw new errs.ItemNotFoundError(thisData.id);
 				}
 				const thisRow = internalHost.cleanRowCertificateMeta(row);
@@ -280,7 +284,7 @@ const internalProxyHost = {
 				return internalProxyHost.get(access, { id: data.id });
 			})
 			.then((row) => {
-				if (!row || !row.id) {
+				if (!row?.id) {
 					throw new errs.ItemNotFoundError(data.id);
 				}
 
@@ -324,11 +328,11 @@ const internalProxyHost = {
 			.then(() => {
 				return internalProxyHost.get(access, {
 					id: data.id,
-					expand: ["certificate", "owner", "access_list"],
+					expand: ["certificate", "owner", "access_list.[clients,items]"],
 				});
 			})
-			.then((row) => {
-				if (!row || !row.id) {
+			.then(async (row) => {
+				if (!row?.id) {
 					throw new errs.ItemNotFoundError(data.id);
 				}
 				if (row.enabled) {
@@ -337,27 +341,21 @@ const internalProxyHost = {
 
 				row.enabled = 1;
 
-				return proxyHostModel
-					.query()
-					.where("id", row.id)
-					.patch({
-						enabled: 1,
-					})
-					.then(() => {
-						// Configure nginx
-						return internalNginx.configure(proxyHostModel, "proxy_host", row);
-					})
-					.then(() => {
-						// Add to audit log
-						return internalAuditLog.add(access, {
-							action: "enabled",
-							object_type: "proxy-host",
-							object_id: row.id,
-							meta: _.omit(row, omissions()),
-						});
-					});
-			})
-			.then(() => {
+				await proxyHostModel.query().where("id", row.id).patch({
+					enabled: 1,
+				});
+
+				// Configure nginx
+				await internalNginx.configure(proxyHostModel, "proxy_host", row);
+
+				// Add to audit log
+				await internalAuditLog.add(access, {
+					action: "enabled",
+					object_type: "proxy-host",
+					object_id: row.id,
+					meta: _.omit(row, omissions()),
+				});
+
 				return true;
 			});
 	},
@@ -376,7 +374,7 @@ const internalProxyHost = {
 				return internalProxyHost.get(access, { id: data.id });
 			})
 			.then((row) => {
-				if (!row || !row.id) {
+				if (!row?.id) {
 					throw new errs.ItemNotFoundError(data.id);
 				}
 				if (!row.enabled) {
@@ -422,15 +420,16 @@ const internalProxyHost = {
 	 */
 	getAll: async (access, expand, searchQuery) => {
 		const accessData = await access.can("proxy_hosts:list");
+
 		const query = proxyHostModel
 			.query()
 			.where("is_deleted", 0)
 			.groupBy("id")
-			.allowGraph("[owner,access_list,certificate]")
+			.allowGraph(proxyHostModel.defaultAllowGraph)
 			.orderBy(castJsonIfNeed("domain_names"), "ASC");
 
 		if (accessData.permission_visibility !== "all") {
-			query.andWhere("owner_user_id", access.token.getUserId(1));
+			scopeToOwnerOrAllowed(query, access.token.getUserId(1), access.getPermissions?.()?.meta);
 		}
 
 		// Query is used for searching
@@ -456,18 +455,18 @@ const internalProxyHost = {
 	 *
 	 * @param   {Number}  user_id
 	 * @param   {String}  visibility
+	 * @param   {Object}  [permissionsMeta]  the user's permissions meta, as already loaded by access
 	 * @returns {Promise}
 	 */
-	getCount: (user_id, visibility) => {
+	getCount: async (user_id, visibility, permissionsMeta) => {
 		const query = proxyHostModel.query().count("id as count").where("is_deleted", 0);
 
 		if (visibility !== "all") {
-			query.andWhere("owner_user_id", user_id);
+			scopeToOwnerOrAllowed(query, user_id, permissionsMeta);
 		}
 
-		return query.first().then((row) => {
-			return Number.parseInt(row.count, 10);
-		});
+		const row = await query.first();
+		return Number.parseInt(row.count, 10);
 	},
 };
 

backend/internal/redirection-host.js

@@ -53,10 +53,12 @@ const internalRedirectionHost = {
 
 				// Fix for db field not having a default value
 				// for this optional field.
-				if (typeof data.advanced_config === "undefined") {
-					data.advanced_config = "";
+				if (typeof thisData.advanced_config === "undefined") {
+					thisData.advanced_config = "";
 				}
 
+				internalHost.assertAdvancedConfigAllowed(access, thisData);
+
 				return redirectionHostModel.query().insertAndFetch(thisData).then(utils.omitRow(omissions()));
 			})
 			.then((row) => {
@@ -155,6 +157,9 @@ const internalRedirectionHost = {
 					);
 				}
 
+				// Before the certificate is created, so a refused request doesn't leave one behind
+				internalHost.assertAdvancedConfigAllowed(access, thisData, row);
+
 				if (createCertificate) {
 					return internalCertificate
 						.createQuickCertificate(access, {
@@ -229,7 +234,6 @@ const internalRedirectionHost = {
 	 */
 	get: (access, data) => {
 		const thisData = data || {};
-
 		return access
 			.can("redirection_hosts:get", thisData.id)
 			.then((access_data) => {
@@ -237,7 +241,7 @@ const internalRedirectionHost = {
 					.query()
 					.where("is_deleted", 0)
 					.andWhere("id", thisData.id)
-					.allowGraph("[owner,certificate]")
+					.allowGraph(redirectionHostModel.defaultAllowGraph)
 					.first();
 
 				if (access_data.permission_visibility !== "all") {
@@ -252,7 +256,7 @@ const internalRedirectionHost = {
 			})
 			.then((row) => {
 				let thisRow = row;
-				if (!thisRow || !thisRow.id) {
+				if (!thisRow?.id) {
 					throw new errs.ItemNotFoundError(thisData.id);
 				}
 				thisRow = internalHost.cleanRowCertificateMeta(thisRow);
@@ -278,7 +282,7 @@ const internalRedirectionHost = {
 				return internalRedirectionHost.get(access, { id: data.id });
 			})
 			.then((row) => {
-				if (!row || !row.id) {
+				if (!row?.id) {
 					throw new errs.ItemNotFoundError(data.id);
 				}
 
@@ -326,7 +330,7 @@ const internalRedirectionHost = {
 				});
 			})
 			.then((row) => {
-				if (!row || !row.id) {
+				if (!row?.id) {
 					throw new errs.ItemNotFoundError(data.id);
 				}
 				if (row.enabled) {
@@ -374,7 +378,7 @@ const internalRedirectionHost = {
 				return internalRedirectionHost.get(access, { id: data.id });
 			})
 			.then((row) => {
-				if (!row || !row.id) {
+				if (!row?.id) {
 					throw new errs.ItemNotFoundError(data.id);
 				}
 				if (!row.enabled) {
@@ -426,7 +430,7 @@ const internalRedirectionHost = {
 					.query()
 					.where("is_deleted", 0)
 					.groupBy("id")
-					.allowGraph("[owner,certificate]")
+					.allowGraph(redirectionHostModel.defaultAllowGraph)
 					.orderBy(castJsonIfNeed("domain_names"), "ASC");
 
 				if (access_data.permission_visibility !== "all") {

backend/internal/report.js

@@ -15,7 +15,11 @@ const internalReport = {
 				const userId = access.token.getUserId(1);
 
 				const promises = [
-					internalProxyHost.getCount(userId, access_data.permission_visibility),
+					internalProxyHost.getCount(
+						userId,
+						access_data.permission_visibility,
+						access.getPermissions?.()?.meta,
+					),
 					internalRedirectionHost.getCount(userId, access_data.permission_visibility),
 					internalStream.getCount(userId, access_data.permission_visibility),
 					internalDeadHost.getCount(userId, access_data.permission_visibility),

backend/internal/stream.js

@@ -178,7 +178,6 @@ const internalStream = {
 	 */
 	get: (access, data) => {
 		const thisData = data || {};
-
 		return access
 			.can("streams:get", thisData.id)
 			.then((access_data) => {
@@ -186,7 +185,7 @@ const internalStream = {
 					.query()
 					.where("is_deleted", 0)
 					.andWhere("id", thisData.id)
-					.allowGraph("[owner,certificate]")
+					.allowGraph(streamModel.defaultAllowGraph)
 					.first();
 
 				if (access_data.permission_visibility !== "all") {
@@ -201,7 +200,7 @@ const internalStream = {
 			})
 			.then((row) => {
 				let thisRow = row;
-				if (!thisRow || !thisRow.id) {
+				if (!thisRow?.id) {
 					throw new errs.ItemNotFoundError(thisData.id);
 				}
 				thisRow = internalHost.cleanRowCertificateMeta(thisRow);
@@ -227,7 +226,7 @@ const internalStream = {
 				return internalStream.get(access, { id: data.id });
 			})
 			.then((row) => {
-				if (!row || !row.id) {
+				if (!row?.id) {
 					throw new errs.ItemNotFoundError(data.id);
 				}
 
@@ -275,7 +274,7 @@ const internalStream = {
 				});
 			})
 			.then((row) => {
-				if (!row || !row.id) {
+				if (!row?.id) {
 					throw new errs.ItemNotFoundError(data.id);
 				}
 				if (row.enabled) {
@@ -323,7 +322,7 @@ const internalStream = {
 				return internalStream.get(access, { id: data.id });
 			})
 			.then((row) => {
-				if (!row || !row.id) {
+				if (!row?.id) {
 					throw new errs.ItemNotFoundError(data.id);
 				}
 				if (!row.enabled) {
@@ -375,7 +374,7 @@ const internalStream = {
 					.query()
 					.where("is_deleted", 0)
 					.groupBy("id")
-					.allowGraph("[owner,certificate]")
+					.allowGraph(streamModel.defaultAllowGraph)
 					.orderBy("incoming_port", "ASC");
 
 				if (access_data.permission_visibility !== "all") {

backend/internal/token.js

@@ -34,26 +34,21 @@ export default {
 			.andWhere("is_disabled", 0)
 			.first();
 
-		if (!user) {
-			throw new errs.AuthError(ERROR_MESSAGE_INVALID_AUTH);
-		}
-
+		// An unknown, disabled or passwordless account goes through the same queries and the same password check
+		// as a real one, and fails with the same error: answering any sooner or any differently would tell anyone
+		// trying addresses which ones have an account. There is no user 0, so its lookup finds nothing. An empty
+		// secret, which creating a user through the API can store, is no hash: bcrypt would reject it at once.
 		const auth = await authModel
 			.query()
-			.where("user_id", "=", user.id)
+			.where("user_id", "=", user?.id ?? 0)
 			.where("type", "=", "password")
 			.first();
 
-		if (!auth) {
-			throw new errs.AuthError(ERROR_MESSAGE_INVALID_AUTH);
-		}
-
-		const valid = await auth.verifyPassword(data.secret);
-		if (!valid) {
-			throw new errs.AuthError(
-				ERROR_MESSAGE_INVALID_AUTH,
-				ERROR_MESSAGE_INVALID_AUTH_I18N,
-			);
+		const valid = auth?.secret
+			? await auth.verifyPassword(data.secret)
+			: await authModel.verifyDummyPassword(data.secret);
+		if (!user || !valid) {
+			throw new errs.AuthError(ERROR_MESSAGE_INVALID_AUTH, ERROR_MESSAGE_INVALID_AUTH_I18N);
 		}
 
 		if (data.scope !== "user" && _.indexOf(user.roles, data.scope) === -1) {
@@ -171,7 +166,7 @@ export default {
 		}
 
 		// Check scope
-		if (!tokenData.scope || tokenData.scope[0] !== "2fa-challenge") {
+		if (tokenData.scope?.[0] !== "2fa-challenge") {
 			throw new errs.AuthError("Invalid challenge token");
 		}
 
@@ -183,10 +178,7 @@ export default {
 		// Verify 2FA code
 		const valid = await twoFactor.verifyForLogin(userId, code);
 		if (!valid) {
-			throw new errs.AuthError(
-				ERROR_MESSAGE_INVALID_2FA,
-				ERROR_MESSAGE_INVALID_2FA_I18N,
-			);
+			throw new errs.AuthError(ERROR_MESSAGE_INVALID_2FA, ERROR_MESSAGE_INVALID_2FA_I18N);
 		}
 
 		// Create full token

backend/internal/user.js

@@ -3,6 +3,7 @@ import _ from "lodash";
 import errs from "../lib/error.js";
 import utils from "../lib/utils.js";
 import authModel from "../models/auth.js";
+import proxyHostModel from "../models/proxy_host.js";
 import userModel from "../models/user.js";
 import userPermissionModel from "../models/user_permission.js";
 import internalAuditLog from "./audit-log.js";
@@ -87,7 +88,13 @@ const internalUser = {
 		}
 
 		return access
-			.can("users:update", data.id)
+			.can("users:permissions", data.id)
+			.catch(() => {
+				delete data.roles;
+			})
+			.then(() => {
+				return access.can("users:update", data.id);
+			})
 			.then(() => {
 				// Make sure that the user being updated doesn't change their email to another user that is already using it
 				// 1. get user we want to update
@@ -171,7 +178,7 @@ const internalUser = {
 				return query.then(utils.omitRow(omissions()));
 			})
 			.then((row) => {
-				if (!row || !row.id) {
+				if (!row?.id) {
 					throw new errs.ItemNotFoundError(thisData.id);
 				}
 				// Custom omissions
@@ -251,11 +258,9 @@ const internalUser = {
 	},
 
 	deleteAll: async () => {
-		await userModel
-			.query()
-			.patch({
-				is_deleted: 1,
-			});
+		await userModel.query().patch({
+			is_deleted: 1,
+		});
 	},
 
 	/**
@@ -384,11 +389,21 @@ const internalUser = {
 					.andWhere("type", data.type)
 					.first()
 					.then((existing_auth) => {
+						// Stamped here rather than read off modified_on, because it is compared against a
+						// token's `iat` and the two only line up when the same clock writes both. The
+						// database clock is a different one: with the app on one timezone and the database
+						// on another, its timestamps come back hours away from where Node thinks it is.
+						const password_changed_at = Math.floor(Date.now() / 1000);
+
 						if (existing_auth) {
 							// patch
+							const meta = existing_auth.meta || {};
+							meta.password_changed_at = password_changed_at;
+
 							return authModel.query().where("user_id", user.id).andWhere("type", data.type).patch({
 								type: data.type, // This is required for the model to encrypt on save
 								secret: data.secret,
+								meta,
 							});
 						}
 						// insert
@@ -396,7 +411,7 @@ const internalUser = {
 							user_id: user.id,
 							type: data.type,
 							secret: data.secret,
-							meta: {},
+							meta: { password_changed_at },
 						});
 					})
 					.then(() => {
@@ -430,7 +445,7 @@ const internalUser = {
 				return internalUser.get(access, { id: data.id });
 			})
 			.then((user) => {
-				if (user.id !== data.id) {
+				if (Number(user.id) !== Number(data.id)) {
 					// Sanity check that something crazy hasn't happened
 					throw new errs.InternalValidationError(
 						`User could not be updated, IDs do not match: ${user.id} !== ${data.id}`,
@@ -440,21 +455,23 @@ const internalUser = {
 				return user;
 			})
 			.then((user) => {
+				const permData = _.omit(data, ["id"]);
 				// Get perms row, patch if it exists
 				return userPermissionModel
 					.query()
 					.where("user_id", user.id)
 					.first()
-					.then((existing_auth) => {
+					.then(async (existing_auth) => {
+						permData.meta = await internalUser.normalisePermissionMeta(existing_auth, permData);
 						if (existing_auth) {
 							// patch
 							return userPermissionModel
 								.query()
 								.where("user_id", user.id)
-								.patchAndFetchById(existing_auth.id, _.assign({ user_id: user.id }, data));
+								.patchAndFetchById(existing_auth.id, _.assign({ user_id: user.id }, permData));
 						}
 						// insert
-						return userPermissionModel.query().insertAndFetch(_.assign({ user_id: user.id }, data));
+						return userPermissionModel.query().insertAndFetch(_.assign({ user_id: user.id }, permData));
 					})
 					.then((permissions) => {
 						// Add to Audit Log
@@ -474,6 +491,36 @@ const internalUser = {
 			});
 	},
 
+	/**
+	 * The permission meta to store. Proxy host grants (meta.proxy_host_ids) only apply while
+	 * visibility is "user" and proxy hosts are not hidden, so they're cleared otherwise; kept,
+	 * they would silently come back when proxy hosts are shown again. Ids of hosts that no
+	 * longer exist are dropped. `permData` may be partial, so the stored values fill the gaps.
+	 *
+	 * @param   {Object}  [existing]  the stored user_permission row
+	 * @param   {Object}  permData    incoming permission fields
+	 * @returns {Promise<Object>}
+	 */
+	normalisePermissionMeta: async (existing, permData) => {
+		const effective = _.assign(
+			{ visibility: "user", proxy_hosts: "hidden" },
+			existing || {},
+			_.omit(permData, ["meta"]),
+		);
+		const meta = _.assign({}, existing?.meta || {}, permData.meta || {});
+		const requested = Array.isArray(meta.proxy_host_ids) ? _.uniq(meta.proxy_host_ids) : [];
+
+		if (effective.visibility !== "user" || effective.proxy_hosts === "hidden" || !requested.length) {
+			meta.proxy_host_ids = [];
+			return meta;
+		}
+
+		const rows = await proxyHostModel.query().select("id").where("is_deleted", 0).whereIn("id", requested);
+		const found = new Set(rows.map((r) => r.id));
+		meta.proxy_host_ids = requested.filter((id) => found.has(id));
+		return meta;
+	},
+
 	/**
 	 * @param {Access}   access
 	 * @param {Object}   data

backend/knexfile.js

@@ -1,19 +1,19 @@
 module.exports = {
 	development: {
-		client:     'mysql2',
+		client: "mysql2",
 		migrations: {
-			tableName: 'migrations',
-			stub:      'lib/migrate_template.js',
-			directory: 'migrations'
-		}
+			tableName: "migrations",
+			stub: "lib/migrate_template.js",
+			directory: "migrations",
+		},
 	},
 
 	production: {
-		client:     'mysql2',
+		client: "mysql2",
 		migrations: {
-			tableName: 'migrations',
-			stub:      'lib/migrate_template.js',
-			directory: 'migrations'
-		}
-	}
+			tableName: "migrations",
+			stub: "lib/migrate_template.js",
+			directory: "migrations",
+		},
+	},
 };

backend/lib/access.js

@@ -12,7 +12,8 @@ import { fileURLToPath } from "node:url";
 import Ajv from "ajv/dist/2020.js";
 import _ from "lodash";
 import { access as logger } from "../logger.js";
-import proxyHostModel from "../models/proxy_host.js";
+import authModel from "../models/auth.js";
+import proxyHostModel, { scopeToOwnerOrAllowed } from "../models/proxy_host.js";
 import TokenModel from "../models/token.js";
 import userModel from "../models/user.js";
 import permsSchema from "./access/permissions.json" with { type: "json" };
@@ -80,6 +81,22 @@ export default function (tokenString) {
 				if (!ok) {
 					throw new errs.AuthError("Invalid token scope for User");
 				}
+
+				// A token issued before the password was last changed is no longer valid: taking an account
+				// back from whoever has the old password has to end the sessions that password opened.
+				const auth = await authModel
+					.query()
+					.where("user_id", "=", user.id)
+					.where("type", "=", "password")
+					.first();
+
+				// Both sides come from the same clock and in the same unit, whole seconds since
+				// the epoch: `setPassword` stamps the marker and `jsonwebtoken` stamps `iat`.
+				const changedAt = auth?.meta?.password_changed_at;
+				if (changedAt && typeof tokenData.iat === "number" && tokenData.iat < changedAt) {
+					throw new errs.TokenRevokedError("Token was issued before the password was changed");
+				}
+
 				initialised = true;
 				userRoles = user.roles;
 				permissions = user.permissions;
@@ -119,13 +136,10 @@ export default function (tokenString) {
 
 					// Proxy Hosts
 					case "proxy_hosts": {
-						const query = proxyHostModel
-							.query()
-							.select("id")
-							.andWhere("is_deleted", 0);
+						const query = proxyHostModel.query().select("id").andWhere("is_deleted", 0);
 
 						if (permissions.visibility === "user") {
-							query.andWhere("owner_user_id", tokenUserId);
+							scopeToOwnerOrAllowed(query, tokenUserId, permissions.meta);
 						}
 
 						const rows = await query;
@@ -199,6 +213,11 @@ export default function (tokenString) {
 	return {
 		token: Token,
 
+		getPermissions: () => permissions,
+		getRoles: () => userRoles,
+		hasRole: (role) => userRoles.includes(role),
+		isAdmin: () => userRoles.includes("admin") || Token.hasScope("admin"),
+
 		/**
 		 *
 		 * @param   {Boolean}  [allowInternal]
@@ -206,7 +225,8 @@ export default function (tokenString) {
 		 */
 		load: async (allowInternal) => {
 			if (tokenString) {
-				return await Token.load(tokenString);
+				await this.init();
+				return tokenData;
 			}
 			allowInternalAccess = allowInternal;
 			return allowInternal || null;
@@ -241,6 +261,7 @@ export default function (tokenString) {
 						permission_streams: permissions.streams,
 						permission_access_lists: permissions.access_lists,
 						permission_certificates: permissions.certificates,
+						permission_meta: permissions.meta,
 					},
 				};
 
@@ -271,7 +292,12 @@ export default function (tokenString) {
 				err.permission = permission;
 				err.permission_data = data;
 				logger.error(permission, data, err.message);
-				throw errs.PermissionError("Permission Denied", err);
+				// A revoked token is not a permission problem, and the client can tell: the frontend
+				// clears the session on a 401 and on nothing else.
+				if (err instanceof errs.TokenRevokedError) {
+					throw err;
+				}
+				throw new errs.PermissionError("Permission Denied", err);
 			}
 		},
 	};

backend/lib/access/logs-list.json

@@ -0,0 +1,7 @@
+{
+	"anyOf": [
+		{
+			"$ref": "roles#/definitions/admin"
+		}
+	]
+}

backend/lib/certbot.js

@@ -4,8 +4,6 @@ import { certbot as logger } from "../logger.js";
 import errs from "./error.js";
 import utils from "./utils.js";
 
-const CERTBOT_VERSION_REPLACEMENT = "$(certbot --version | grep -Eo '[0-9](\\.[0-9]+)+')";
-
 /**
  * Installs a cerbot plugin given the key for the object from
  * ../certbot/dns-plugins.json
@@ -15,24 +13,32 @@ const CERTBOT_VERSION_REPLACEMENT = "$(certbot --version | grep -Eo '[0-9](\\.[0
  */
 const installPlugin = async (pluginKey) => {
 	if (typeof dnsPlugins[pluginKey] === "undefined") {
-		// throw Error(`Certbot plugin ${pluginKey} not found`);
 		throw new errs.ItemNotFoundError(pluginKey);
 	}
 
 	const plugin = dnsPlugins[pluginKey];
 	logger.start(`Installing ${pluginKey}...`);
 
-	plugin.version = plugin.version.replace(/{{certbot-version}}/g, CERTBOT_VERSION_REPLACEMENT);
-	plugin.dependencies = plugin.dependencies.replace(/{{certbot-version}}/g, CERTBOT_VERSION_REPLACEMENT);
+	plugin.version = plugin.version.replace(/{{certbot-version}}/g, process.env.CERTBOT_VERSION);
+	plugin.dependencies = plugin.dependencies.replace(/{{certbot-version}}/g, process.env.CERTBOT_VERSION);
 
-	// SETUPTOOLS_USE_DISTUTILS is required for certbot plugins to install correctly
-	// in new versions of Python
-	let env = Object.assign({}, process.env, { SETUPTOOLS_USE_DISTUTILS: "stdlib" });
+	// SETUPTOOLS_USE_DISTUTILS=local uses setuptools' own bundled distutils.
+	// "stdlib" breaks Python 3.13+ where distutils was removed from the standard library.
+	let env = Object.assign({}, process.env, { SETUPTOOLS_USE_DISTUTILS: "local" });
 	if (typeof plugin.env === "object") {
 		env = Object.assign(env, plugin.env);
 	}
 
-	const cmd = `. /opt/certbot/bin/activate && pip install --no-cache-dir ${plugin.dependencies} ${plugin.package_name}${plugin.version}  && deactivate`;
+	const quotedDeps = plugin.dependencies.trim()
+		? plugin.dependencies
+				.trim()
+				.split(/\s+/)
+				.filter(Boolean)
+				.map((d) => `'${d}'`)
+				.join(" ")
+		: "";
+
+	const cmd = `. /opt/certbot/bin/activate && pip install --no-cache-dir ${quotedDeps} '${plugin.package_name}${plugin.version}' && deactivate`;
 	return utils
 		.exec(cmd, { env })
 		.then((result) => {
@@ -73,9 +79,7 @@ const installPlugins = async (pluginKeys) => {
 			})
 			.end(() => {
 				if (hasErrors) {
-					reject(
-						new errs.CommandError("Some plugins failed to install. Please check the logs above", 1),
-					);
+					reject(new errs.CommandError("Some plugins failed to install. Please check the logs above", 1));
 				} else {
 					resolve();
 				}
@@ -83,4 +87,4 @@ const installPlugins = async (pluginKeys) => {
 	});
 };
 
-export { installPlugins, installPlugin };
+export { installPlugin, installPlugins };

backend/lib/config.js

@@ -2,10 +2,13 @@ import fs from "node:fs";
 import NodeRSA from "node-rsa";
 import { global as logger } from "../logger.js";
 
-const keysFile         = '/data/keys.json';
-const mysqlEngine      = 'mysql2';
-const postgresEngine   = 'pg';
-const sqliteClientName = 'better-sqlite3';
+const keysFile = "/data/keys.json";
+const mysqlEngine = "mysql2";
+const postgresEngine = "pg";
+const sqliteClientName = "better-sqlite3";
+
+// Not used for new setups anymore but may exist in legacy setups
+const legacySqliteClientName = "sqlite3";
 
 let instance = null;
 
@@ -37,14 +40,20 @@ const configure = () => {
 		}
 	}
 
-	const toBool = (v) => /^(1|true|yes|on)$/i.test((v || '').trim());
+	const toBool = (v) => /^(1|true|yes|on)$/i.test((v || "").trim());
 
-	const envMysqlHost                  = process.env.DB_MYSQL_HOST || null;
-	const envMysqlUser                  = process.env.DB_MYSQL_USER || null;
-	const envMysqlName                  = process.env.DB_MYSQL_NAME || null;
-	const envMysqlSSL                   = toBool(process.env.DB_MYSQL_SSL);
-	const envMysqlSSLRejectUnauthorized	= process.env.DB_MYSQL_SSL_REJECT_UNAUTHORIZED === undefined ? true : toBool(process.env.DB_MYSQL_SSL_REJECT_UNAUTHORIZED);
-	const envMysqlSSLVerifyIdentity		= process.env.DB_MYSQL_SSL_VERIFY_IDENTITY === undefined ? true : toBool(process.env.DB_MYSQL_SSL_VERIFY_IDENTITY);
+	const envMysqlHost = process.env.DB_MYSQL_HOST || null;
+	const envMysqlUser = process.env.DB_MYSQL_USER || null;
+	const envMysqlName = process.env.DB_MYSQL_NAME || null;
+	const envMysqlSSL = toBool(process.env.DB_MYSQL_SSL);
+	const envMysqlSSLRejectUnauthorized =
+		process.env.DB_MYSQL_SSL_REJECT_UNAUTHORIZED === undefined
+			? true
+			: toBool(process.env.DB_MYSQL_SSL_REJECT_UNAUTHORIZED);
+	const envMysqlSSLVerifyIdentity =
+		process.env.DB_MYSQL_SSL_VERIFY_IDENTITY === undefined
+			? true
+			: toBool(process.env.DB_MYSQL_SSL_VERIFY_IDENTITY);
 	if (envMysqlHost && envMysqlUser && envMysqlName) {
 		// we have enough mysql creds to go with mysql
 		logger.info("Using MySQL configuration");
@@ -55,8 +64,10 @@ const configure = () => {
 				port: process.env.DB_MYSQL_PORT || 3306,
 				user: envMysqlUser,
 				password: process.env.DB_MYSQL_PASSWORD,
-				name:     envMysqlName,
-				ssl:      envMysqlSSL ? { rejectUnauthorized: envMysqlSSLRejectUnauthorized, verifyIdentity: envMysqlSSLVerifyIdentity } : false,
+				name: envMysqlName,
+				ssl: envMysqlSSL
+					? { rejectUnauthorized: envMysqlSSLRejectUnauthorized, verifyIdentity: envMysqlSSLVerifyIdentity }
+					: false,
 			},
 			keys: getKeys(),
 		};
@@ -134,7 +145,7 @@ const generateKeys = () => {
 
 	// Write keys config
 	try {
-		fs.writeFileSync(keysFile, JSON.stringify(keys, null, 2));
+		fs.writeFileSync(keysFile, JSON.stringify(keys, null, 2), { mode: 0o600 });
 	} catch (err) {
 		logger.error(`Could not write JWT key pair to config file: ${keysFile}: ${err.message}`);
 		process.exit(1);
@@ -184,7 +195,7 @@ const configGet = (key) => {
  */
 const isSqlite = () => {
 	instance === null && configure();
-	return instance.database.knex && instance.database.knex.client === sqliteClientName;
+	return instance.database.knex && [sqliteClientName, legacySqliteClientName].includes(instance.database.knex.client);
 };
 
 /**
@@ -208,7 +219,7 @@ const isPostgres = () => {
 };
 
 /**
- * Are we running in debug mdoe?
+ * Are we running in debug mode?
  *
  * @returns {boolean}
  */
@@ -219,7 +230,7 @@ const isDebugMode = () => !!process.env.DEBUG;
  *
  * @returns {boolean}
  */
-const isCI = () => process.env.CI === 'true' && process.env.DEBUG === 'true';
+const isCI = () => process.env.CI === "true" && process.env.DEBUG === "true";
 
 /**
  * Returns a public key
@@ -256,4 +267,16 @@ const useLetsencryptServer = () => {
 	return null;
 };
 
-export { isCI, configHas, configGet, isSqlite, isMysql, isPostgres, isDebugMode, getPrivateKey, getPublicKey, useLetsencryptStaging, useLetsencryptServer };
+export {
+	isCI,
+	configHas,
+	configGet,
+	isSqlite,
+	isMysql,
+	isPostgres,
+	isDebugMode,
+	getPrivateKey,
+	getPublicKey,
+	useLetsencryptStaging,
+	useLetsencryptServer,
+};

backend/lib/error.js

@@ -22,6 +22,15 @@ const errs = {
 		this.status = 404;
 	},
 
+	TokenRevokedError: function (message, previous) {
+		Error.captureStackTrace(this, this.constructor);
+		this.name = this.constructor.name;
+		this.previous = previous;
+		this.message = message;
+		this.public = true;
+		this.status = 401;
+	},
+
 	AuthError: function (message, messageI18n, previous) {
 		Error.captureStackTrace(this, this.constructor);
 		this.name = this.constructor.name;

backend/lib/express/cors.js

@@ -3,7 +3,7 @@ export default (req, res, next) => {
 		res.set({
 			"Access-Control-Allow-Origin": req.headers.origin,
 			"Access-Control-Allow-Credentials": true,
-			"Access-Control-Allow-Methods": "OPTIONS, GET, POST",
+			"Access-Control-Allow-Methods": "OPTIONS, GET, POST, PUT, DELETE",
 			"Access-Control-Allow-Headers":
 				"Content-Type, Cache-Control, Pragma, Expires, Authorization, X-Dataset-Total, X-Dataset-Offset, X-Dataset-Limit",
 			"Access-Control-Max-Age": 5 * 60,

backend/lib/express/log-request.js

@@ -0,0 +1,7 @@
+import chalk from "chalk";
+import { debug, express as logger } from "../../logger.js";
+
+export default (req, _res, next) => {
+	debug(logger, `[${chalk.yellow(req.method.toUpperCase())}] ${chalk.green(req.path)}`);
+	next();
+};

backend/lib/express/pagination.js

@@ -1,4 +1,4 @@
-import _  from "lodash";
+import _ from "lodash";
 
 export default (default_sort, default_offset, default_limit, max_limit) => {
 	/**

backend/lib/express/rate-limit.js

@@ -0,0 +1,155 @@
+import crypto from "node:crypto";
+import { isCI } from "../config.js";
+
+/**
+ * In-memory fixed window rate limiter for authentication endpoints
+ */
+
+// Keys longer than this are hashed, so a client can't make the store hold arbitrarily large
+// strings (keys can include request body values such as the login identity).
+const MAX_KEY_LENGTH = 128;
+// Entries per limiter. A flood of unique keys sweeps expired entries, then evicts the oldest,
+// instead of growing the store until the periodic cleanup runs.
+const MAX_STORE_SIZE = 50000;
+
+// One store per limiter, so separate endpoints never share buckets
+const stores = new Set();
+
+// Clean up old entries every 2 minutes
+const cleanupTimer = setInterval(() => {
+	const now = Date.now();
+	for (const store of stores) {
+		for (const [key, record] of store.entries()) {
+			if (now - record.startTime > record.windowMs * 2) {
+				store.delete(key);
+			}
+		}
+	}
+}, 120000);
+cleanupTimer.unref();
+
+// Generated at container start (prepare/45-admin-port.sh) and sent by the admin nginx as
+// X-NPM-Peer-Secret alongside X-NPM-Peer-Addr. Unset (development, or a mounted production.conf)
+// means the header is never trusted.
+const peerSecret = Buffer.from(process.env.NPM_PEER_SECRET || "");
+
+const hasPeerSecret = (value) => {
+	if (!peerSecret.length || typeof value !== "string") {
+		return false;
+	}
+	const given = Buffer.from(value);
+	return given.length === peerSecret.length && crypto.timingSafeEqual(given, peerSecret);
+};
+
+/**
+ * The address to rate limit on: the TCP peer of the admin nginx, which it passes as
+ * X-NPM-Peer-Addr ($realip_remote_addr). That header is only honoured from a loopback socket
+ * carrying the startup secret, so it can only have come from the admin nginx: a proxy host or
+ * stream forwarding to the backend port is also a loopback peer, but can't know the secret.
+ * The realip-resolved client address is deliberately not used: nginx trusts X-Real-IP from
+ * private ranges, so a client on the LAN or Docker network could pick a new address, and a fresh
+ * bucket, on every request. Behind another proxy (including an NPM proxy host in front of the
+ * admin UI) this resolves to that proxy's address.
+ *
+ * @param   {Object} req
+ * @returns {String}
+ */
+export const getClientIp = (req) => {
+	const socketAddr = req.socket?.remoteAddress || "unknown";
+	const isLoopback = socketAddr === "127.0.0.1" || socketAddr === "::1" || socketAddr === "::ffff:127.0.0.1";
+	const trusted = isLoopback && hasPeerSecret(req.headers["x-npm-peer-secret"]);
+	const peerAddr = trusted ? req.headers["x-npm-peer-addr"]?.trim() : null;
+	return peerAddr || socketAddr;
+};
+
+const makeRoom = (store, now) => {
+	for (const [k, r] of store.entries()) {
+		if (now - r.startTime > r.windowMs) {
+			store.delete(k);
+		}
+	}
+	// Maps iterate in insertion order, so this drops the oldest windows first
+	for (const k of store.keys()) {
+		if (store.size < MAX_STORE_SIZE) {
+			break;
+		}
+		store.delete(k);
+	}
+};
+
+const boundedKey = (key) =>
+	key.length > MAX_KEY_LENGTH ? `sha256:${crypto.createHash("sha256").update(key).digest("hex")}` : key;
+
+/**
+ * @param {Object}   [options]
+ * @param {Number}   [options.windowMs]
+ * @param {Number}   [options.max]
+ * @param {String}   [options.message]
+ * @param {Boolean}  [options.skipSuccessfulRequests]
+ * @param {Function} [options.keyGenerator]  (req, clientIp) => String. Defaults to the client IP.
+ *                                           Including the target account in the key stops every
+ *                                           user being locked out when many clients share an IP
+ *                                           (e.g. the admin UI behind another reverse proxy).
+ */
+export default function createRateLimiter(options = {}) {
+	const windowMs = options.windowMs || 60 * 1000; // 1 minute default
+	const max = options.max || 10; // 10 requests default
+	const message = options.message || "Too many attempts from this IP, please try again later.";
+	const skipSuccessfulRequests = options.skipSuccessfulRequests !== false;
+	const keyGenerator = options.keyGenerator || ((_req, clientIp) => clientIp);
+
+	const store = new Map();
+	stores.add(store);
+
+	return (req, res, next) => {
+		// Bypass rate limiting in CI, test environments, or when explicitly disabled
+		if (
+			isCI() ||
+			process.env.CI === "true" ||
+			process.env.NODE_ENV === "test" ||
+			process.env.DISABLE_RATE_LIMIT === "true"
+		) {
+			return next();
+		}
+
+		const key = boundedKey(String(keyGenerator(req, getClientIp(req))));
+
+		const now = Date.now();
+		let record = store.get(key);
+
+		if (!record || now - record.startTime > windowMs) {
+			record = {
+				count: 0,
+				startTime: now,
+				windowMs,
+			};
+			if (!store.has(key) && store.size >= MAX_STORE_SIZE) {
+				makeRoom(store, now);
+			}
+			store.set(key, record);
+		}
+
+		record.count++;
+
+		if (skipSuccessfulRequests) {
+			const currentRecord = record;
+			res.on("finish", () => {
+				if (res.statusCode < 400 && currentRecord.count > 0) {
+					currentRecord.count--;
+				}
+			});
+		}
+
+		if (record.count > max) {
+			res.setHeader("Retry-After", Math.ceil((record.startTime + windowMs - now) / 1000));
+			return res.status(429).json({
+				error: {
+					code: 429,
+					message,
+				},
+			});
+		}
+
+		next();
+	};
+}

backend/lib/express/user-id-from-me.js

@@ -1,6 +1,6 @@
 export default (req, res, next) => {
-	if (req.params.user_id === 'me' && res.locals.access) {
-		req.params.user_id = res.locals.access.token.get('attrs').id;
+	if (req.params.user_id === "me" && res.locals.access) {
+		req.params.user_id = res.locals.access.token.get("attrs").id;
 	} else {
 		req.params.user_id = Number.parseInt(req.params.user_id, 10);
 	}

backend/lib/nginx-values.js

@@ -0,0 +1,134 @@
+import net from "node:net";
+import errs from "./error.js";
+
+/**
+ * Rules for user-supplied values that the templates write into nginx config as-is.
+ *
+ * Anything that ends or changes an nginx token (whitespace, ";", "{", "}", quotes, "#" which
+ * starts a comment, and "\" which escapes the next character) would let the value add its own
+ * directives, which only admins may do (advanced_config). The API schema rejects these values
+ * with the same patterns (a test keeps the two in step). generateConfig() checks stored values
+ * again, so rows saved before these rules can't add directives either. That check is only about
+ * nginx structure, not format: a stored value in a format the API no longer accepts, but which
+ * can't add directives, still renders.
+ *
+ * Patterns are strings so they can be compared with the JSON schema; compile them with the "u"
+ * flag, as ajv does.
+ */
+
+// One character that can't end or change an unquoted nginx token
+const TOKEN_CHAR = String.raw`[^\s\p{Cc};{}"'#\\]`;
+// Host part of a forward host: also no "$" (no variables in the upstream address) and no "/"
+const HOST_CHARS = String.raw`[^\s\p{Cc};{}"'#\\$/]+`;
+// The existing domain name rule ("a.b.c", no "*", no empty labels), written so it can't
+// backtrack exponentially
+const DOMAIN = String.raw`[^.*]+(?:\.[^.*]+)*\.?[^.]`;
+
+export const patterns = {
+	// server_name entries. The old character blacklist plus whitespace and control characters.
+	domainName: "^[^&| @!#%^();:/\\\\}{=+?<>,~`'\"\\s\\p{Cc}]+$",
+	// proxy_host forward_host and location forward_host: a host, optionally followed by a path
+	// (locations split "host/path" at render time)
+	forwardHost: String.raw`^${HOST_CHARS}(?:/${TOKEN_CHAR}*)?$`,
+	// location forward_path, appended straight after the port
+	forwardPath: String.raw`^${TOKEN_CHAR}*$`,
+	// location path: an optional modifier, then one token, either unquoted (a "\" must escape a
+	// visible character, so it can't swallow the space before "{") or double quoted. A modifier
+	// on its own isn't a location.
+	locationPath: String.raw`^(?!(?:=|~\*?|\^~)$)(?:(?:=|~\*?|\^~) +)?(?:"(?:[^"\\\p{Cc}]|\\[^\p{Cc}])+"|(?:[^\s\p{Cc};{}"'#\\]|\\[^\s\p{Cc}])+)$`,
+	// redirection forward_domain_name, e.g. "example.com:8443/path"
+	redirectDomain: String.raw`^(?=${TOKEN_CHAR}+$)${DOMAIN}$`,
+	// stream forwarding_host when it is a domain name (IPs are matched separately)
+	streamDomain: String.raw`^(?=${HOST_CHARS}$)${DOMAIN}$`,
+	// access list client address: IPv6 with an optional prefix. No zone id ("%eth0"): nginx's
+	// allow/deny rejects them, and the old "%.+" let anything through, including "; allow all"
+	clientIpv6: String.raw`^(?:(?:[0-9A-Fa-f]{1,4}:){7}(?:[0-9A-Fa-f]{1,4}|:)|(?:[0-9A-Fa-f]{1,4}:){6}(?::[0-9A-Fa-f]{1,4}|(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|(?:[0-9A-Fa-f]{1,4}:){5}(?:(?::[0-9A-Fa-f]{1,4}){1,2}|:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|(?:[0-9A-Fa-f]{1,4}:){4}(?:(?::[0-9A-Fa-f]{1,4}){1,3}|(?::[0-9A-Fa-f]{1,4})?:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|(?:[0-9A-Fa-f]{1,4}:){3}(?:(?::[0-9A-Fa-f]{1,4}){1,4}|(?::[0-9A-Fa-f]{1,4}){0,2}:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|(?:[0-9A-Fa-f]{1,4}:){2}(?:(?::[0-9A-Fa-f]{1,4}){1,5}|(?::[0-9A-Fa-f]{1,4}){0,3}:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|(?:[0-9A-Fa-f]{1,4}:)(?:(?::[0-9A-Fa-f]{1,4}){1,6}|(?::[0-9A-Fa-f]{1,4}){0,4}:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|(?::[0-9A-Fa-f]{1,4}){0,5}:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:))(?:/(?:[0-9]|[1-9][0-9]|1[01][0-9]|12[0-8]))?$`,
+	// htpasswd user name: a ":" or a line break would corrupt the file
+	accessUsername: String.raw`^[^:\p{Cc}]+$`,
+};
+
+const re = Object.fromEntries(Object.entries(patterns).map(([k, v]) => [k, new RegExp(v, "u")]));
+const unsafeToken = /[\s\p{Cc};{}"'#\\]/u;
+const ipv4WithPrefix = /^(?:[0-9]{1,3}\.){3}[0-9]{1,3}(?:\/(?:[0-9]|[1-2][0-9]|3[0-2]))?$/;
+
+export const isValid = (name, value) => typeof value === "string" && re[name].test(value);
+
+export const isValidClientAddress = (value) =>
+	typeof value === "string" && (value === "all" || ipv4WithPrefix.test(value) || re.clientIpv6.test(value));
+
+export const isValidStreamHost = (value) =>
+	typeof value === "string" && (net.isIP(value) !== 0 || re.streamDomain.test(value));
+
+/**
+ * Thrown when a stored value can't be written into nginx config safely
+ *
+ * @param {String} message
+ */
+export const unsafeValueError = (message) => {
+	const err = new errs.ConfigurationError(message);
+	err.unsafeNginxValue = true;
+	return err;
+};
+
+const describe = (value) => JSON.stringify(String(value)).slice(0, 80);
+
+/**
+ * @param   {String}  label    e.g. "Proxy Host #3"
+ * @param   {Object}  [accessList]
+ * @throws  {ConfigurationError}
+ */
+export const assertAccessListRenderable = (label, accessList) => {
+	for (const client of accessList?.clients || []) {
+		// "address" is rendered unquoted ("allow <address>;")
+		if (typeof client.address !== "string" || unsafeToken.test(client.address)) {
+			throw unsafeValueError(`${label}: access list client address ${describe(client.address)} is not allowed`);
+		}
+	}
+};
+
+// Can be written into config without ending or changing a token
+export const isTokenSafe = (value) => typeof value === "string" && value.length > 0 && !unsafeToken.test(value);
+
+/**
+ * Checks every user-supplied value the host's template renders. Call before rendering.
+ *
+ * @param   {String}  hostType  proxy_host, redirection_host, dead_host, stream or default
+ * @param   {Object}  host      with access_list loaded, for proxy hosts
+ * @throws  {ConfigurationError}
+ */
+export const assertHostRenderable = (hostType, host) => {
+	const label = `${hostType.replace(/_/g, " ")} #${host.id}`;
+	const check = (ok, what, value) => {
+		if (!ok) {
+			throw unsafeValueError(`${label}: ${what} ${describe(value)} is not allowed in nginx config`);
+		}
+	};
+
+	if (hostType !== "stream" && hostType !== "default") {
+		for (const name of host.domain_names || []) {
+			check(isTokenSafe(name), "domain name", name);
+		}
+	}
+
+	if (hostType === "proxy_host") {
+		check(isTokenSafe(host.forward_host), "forward host", host.forward_host);
+		assertAccessListRenderable(label, host.access_list);
+		for (const loc of host.locations || []) {
+			// The path is several tokens (modifier, then a quoted or unquoted URI): its rule is the
+			// structural one
+			check(isValid("locationPath", loc.path), "location path", loc.path);
+			check(isTokenSafe(loc.forward_host), "location forward host", loc.forward_host);
+			if (loc.forward_path) {
+				check(isTokenSafe(loc.forward_path), "location forward path", loc.forward_path);
+			}
+		}
+	}
+
+	if (hostType === "redirection_host") {
+		check(isTokenSafe(host.forward_domain_name), "forward domain", host.forward_domain_name);
+	}
+
+	if (hostType === "stream") {
+		check(isTokenSafe(host.forwarding_host), "forwarding host", host.forwarding_host);
+	}
+};

backend/lib/utils.js

@@ -3,14 +3,12 @@ import { dirname } from "node:path";
 import { fileURLToPath } from "node:url";
 import { Liquid } from "liquidjs";
 import _ from "lodash";
-import { debug, global as logger } from "../logger.js";
 import errs from "./error.js";
 
 const __filename = fileURLToPath(import.meta.url);
 const __dirname = dirname(__filename);
 
 const exec = async (cmd, options = {}) => {
-	debug(logger, "CMD:", cmd);
 	const { stdout, stderr } = await new Promise((resolve, reject) => {
 		const child = nodeExec(cmd, options, (isError, stdout, stderr) => {
 			if (isError) {
@@ -34,7 +32,6 @@ const exec = async (cmd, options = {}) => {
  * @returns {Promise}
  */
 const execFile = (cmd, args, options) => {
-	debug(logger, `CMD: ${cmd} ${args ? args.join(" ") : ""}`);
 	const opts = options || {};
 
 	return new Promise((resolve, reject) => {

backend/lib/validator/api.js

@@ -1,3 +1,4 @@
+import net from "node:net";
 import Ajv from "ajv/dist/2020.js";
 import errs from "../error.js";
 
@@ -9,6 +10,8 @@ const ajv = new Ajv({
 	coerceTypes: true,
 });
 
+ajv.addFormat("ipv6", { type: "string", validate: (value) => net.isIPv6(value) });
+
 /**
  * @param {Object} schema
  * @param {Object} payload
@@ -24,21 +27,24 @@ const apiValidator = async (schema, payload /*, description*/) => {
 		throw new errs.ValidationError("Payload is undefined");
 	}
 
-
 	const validate = ajv.compile(schema);
 
 	const valid = validate(payload);
 
-
 	if (valid && !validate.errors) {
 		return payload;
 	}
 
-
-
+	// Keep ajv's wording, which clients may match on, but not the regex it quotes: some are long
+	// and mean nothing to a user
+	for (const e of validate.errors || []) {
+		if (e.keyword === "pattern") {
+			e.message = "must match pattern";
+		}
+	}
 	const message = ajv.errorsText(validate.errors);
 	const err = new errs.ValidationError(message);
-	err.debug = {validationErrors: validate.errors, payload};
+	err.debug = { validationErrors: validate.errors, payload };
 	throw err;
 };
 

backend/lib/validator/index.js

@@ -1,4 +1,5 @@
-import Ajv from 'ajv/dist/2020.js';
+import net from "node:net";
+import Ajv from "ajv/dist/2020.js";
 import _ from "lodash";
 import commonDefinitions from "../../schema/common.json" with { type: "json" };
 import errs from "../error.js";
@@ -14,6 +15,8 @@ const ajv = new Ajv({
 	schemas: [commonDefinitions],
 });
 
+ajv.addFormat("ipv6", { type: "string", validate: (value) => net.isIPv6(value) });
+
 /**
  *
  * @param   {Object} schema

backend/logger.js

@@ -1,3 +1,5 @@
+import fs from "node:fs";
+import path from "node:path";
 import signale from "signale";
 import { isDebugMode } from "./lib/config.js";
 
@@ -5,17 +7,115 @@ const opts = {
 	logLevel: "info",
 };
 
-const global = new signale.Signale({ scope: "Global   ", ...opts });
-const migrate = new signale.Signale({ scope: "Migrate  ", ...opts });
-const express = new signale.Signale({ scope: "Express  ", ...opts });
-const access = new signale.Signale({ scope: "Access   ", ...opts });
-const nginx = new signale.Signale({ scope: "Nginx    ", ...opts });
-const ssl = new signale.Signale({ scope: "SSL      ", ...opts });
-const certbot = new signale.Signale({ scope: "Certbot  ", ...opts });
-const importer = new signale.Signale({ scope: "Importer ", ...opts });
-const setup = new signale.Signale({ scope: "Setup    ", ...opts });
-const ipRanges = new signale.Signale({ scope: "IP Ranges", ...opts });
-const remoteVersion = new signale.Signale({ scope: "Remote Version", ...opts });
+// Methods that are actually used across the codebase (see grep of `.info(`, `.warn(`, etc).
+// Only these are mirrored to the log file - decorative signale methods (star, note, watch, ...)
+// are left console-only since they carry no diagnostic value worth persisting.
+const PERSISTED_METHODS = ["info", "warn", "error", "debug", "success", "fatal", "complete"];
+
+const LOG_FILE = "/data/logs/backend.log";
+// biome-ignore lint/suspicious/noControlCharactersInRegex: stripping ANSI colour codes before writing to disk
+const ANSI_PATTERN = /\x1b\[[0-9;]*m/g;
+
+let fileStream = null;
+let lastOpenAttempt = 0;
+// If the file can't be opened (eg. running outside the standard Docker image, or in
+// CI without /data), don't retry on every single log call - but do retry periodically
+// so a transient issue (disk full, permissions fixed later) recovers without a restart.
+const REOPEN_COOLDOWN_MS = 30 * 1000;
+
+/**
+ * Lazily opens the backend log file for appending. If the directory isn't writable,
+ * file logging is silently disabled and console logging continues unaffected.
+ *
+ * @returns {import('node:fs').WriteStream|null}
+ */
+const getFileStream = () => {
+	if (fileStream) {
+		return fileStream;
+	}
+
+	const now = Date.now();
+	if (now - lastOpenAttempt < REOPEN_COOLDOWN_MS) {
+		return null;
+	}
+	lastOpenAttempt = now;
+
+	try {
+		fs.mkdirSync(path.dirname(LOG_FILE), { recursive: true });
+		const stream = fs.createWriteStream(LOG_FILE, { flags: "a" });
+		stream.on("error", () => {
+			fileStream = null;
+		});
+		fileStream = stream;
+	} catch (_err) {
+		fileStream = null;
+	}
+	return fileStream;
+};
+
+/**
+ * Formats and appends a single log line to the backend log file.
+ * Never throws - a failure here must never take down the application.
+ *
+ * @param {String} level
+ * @param {String} scope
+ * @param {Array}  args
+ */
+const writeToFile = (level, scope, args) => {
+	const stream = getFileStream();
+	if (!stream) {
+		return;
+	}
+
+	const message = args
+		.map((arg) => {
+			if (typeof arg === "string") return arg;
+			if (arg instanceof Error) return arg.stack || arg.message;
+			try {
+				return JSON.stringify(arg);
+			} catch (_err) {
+				return String(arg);
+			}
+		})
+		.join(" ")
+		.replace(ANSI_PATTERN, "");
+
+	const line = `${new Date().toISOString()} ${level.toUpperCase().padEnd(7)} [${scope.trim()}] ${message}\n`;
+	stream.write(line);
+};
+
+/**
+ * Wraps a Signale instance so that every call to one of PERSISTED_METHODS is also
+ * appended to the backend log file, in addition to its normal console output.
+ *
+ * @param   {Signale} instance
+ * @param   {String}  scope
+ * @returns {Signale}
+ */
+const withFileSink = (instance, scope) => {
+	for (const method of PERSISTED_METHODS) {
+		const original = instance[method].bind(instance);
+		instance[method] = (...args) => {
+			writeToFile(method, scope, args);
+			return original(...args);
+		};
+	}
+	return instance;
+};
+
+const createLogger = (scope) => withFileSink(new signale.Signale({ scope, ...opts }), scope);
+
+const global = createLogger("Global   ");
+const migrate = createLogger("Migrate  ");
+const express = createLogger("Express  ");
+const access = createLogger("Access   ");
+const nginx = createLogger("Nginx    ");
+const ssl = createLogger("SSL      ");
+const certbot = createLogger("Certbot  ");
+const importer = createLogger("Importer ");
+const setup = createLogger("Setup    ");
+const ipRanges = createLogger("IP Ranges");
+const remoteVersion = createLogger("Remote Version");
 
 const debug = (logger, ...args) => {
 	if (isDebugMode()) {

backend/migrations/20190227065017_settings.js

@@ -13,13 +13,14 @@ const migrateName = "settings";
 const up = (knex) => {
 	logger.info(`[${migrateName}] Migrating Up...`);
 
-	return knex.schema.createTable('setting', (table) => {
-		table.string('id').notNull().primary();
-		table.string('name', 100).notNull();
-		table.string('description', 255).notNull();
-		table.string('value', 255).notNull();
-		table.json('meta').notNull();
-	})
+	return knex.schema
+		.createTable("setting", (table) => {
+			table.string("id").notNull().primary();
+			table.string("name", 100).notNull();
+			table.string("description", 255).notNull();
+			table.string("value", 255).notNull();
+			table.json("meta").notNull();
+		})
 		.then(() => {
 			logger.info(`[${migrateName}] setting Table created`);
 		});

backend/migrations/20251111090000_redirect_auto_scheme.js

@@ -17,9 +17,7 @@ const up = (knex) => {
 		.table("redirection_host", async (table) => {
 			// change the column default from $scheme to auto
 			await table.string("forward_scheme").notNull().defaultTo("auto").alter();
-			await knex('redirection_host')
-				.where('forward_scheme', '$scheme')
-				.update({ forward_scheme: 'auto' });
+			await knex("redirection_host").where("forward_scheme", "$scheme").update({ forward_scheme: "auto" });
 		})
 		.then(() => {
 			logger.info(`[${migrateName}] redirection_host Table altered`);
@@ -38,9 +36,7 @@ const down = (knex) => {
 	return knex.schema
 		.table("redirection_host", async (table) => {
 			await table.string("forward_scheme").notNull().defaultTo("$scheme").alter();
-			await knex('redirection_host')
-				.where('forward_scheme', 'auto')
-				.update({ forward_scheme: '$scheme' });
+			await knex("redirection_host").where("forward_scheme", "auto").update({ forward_scheme: "$scheme" });
 		})
 		.then(() => {
 			logger.info(`[${migrateName}] redirection_host Table altered`);

backend/migrations/20260131163528_trust_forwarded_proto.js

@@ -10,16 +10,16 @@ const migrateName = "trust_forwarded_proto";
  * @param   {Object} knex
  * @returns {Promise}
  */
-const up = function (knex) {
-    logger.info(`[${migrateName}] Migrating Up...`);
+const up = (knex) => {
+	logger.info(`[${migrateName}] Migrating Up...`);
 
-    return knex.schema
-        .alterTable('proxy_host', (table) => {
-            table.tinyint('trust_forwarded_proto').notNullable().defaultTo(0);
-        })
-        .then(() => {
-            logger.info(`[${migrateName}] proxy_host Table altered`);
-        });
+	return knex.schema
+		.alterTable("proxy_host", (table) => {
+			table.tinyint("trust_forwarded_proto").notNullable().defaultTo(0);
+		})
+		.then(() => {
+			logger.info(`[${migrateName}] proxy_host Table altered`);
+		});
 };
 
 /**
@@ -28,16 +28,16 @@ const up = function (knex) {
  * @param   {Object} knex
  * @returns {Promise}
  */
-const down = function (knex) {
-    logger.info(`[${migrateName}] Migrating Down...`);
+const down = (knex) => {
+	logger.info(`[${migrateName}] Migrating Down...`);
 
-    return knex.schema
-        .alterTable('proxy_host', (table) => {
-            table.dropColumn('trust_forwarded_proto');
-        })
-        .then(() => {
-            logger.info(`[${migrateName}] proxy_host Table altered`);
-        });
+	return knex.schema
+		.alterTable("proxy_host", (table) => {
+			table.dropColumn("trust_forwarded_proto");
+		})
+		.then(() => {
+			logger.info(`[${migrateName}] proxy_host Table altered`);
+		});
 };
 
-export { up, down };
+export { up, down };

backend/migrations/20260906220000_user_permission_meta.js

@@ -0,0 +1,41 @@
+import { migrate as logger } from "../logger.js";
+
+const migrateName = "user_permission_meta";
+
+/**
+ * Migrate
+ *
+ * @param   {Object} knex
+ * @returns {Promise}
+ */
+const up = (knex) => {
+	logger.info(`[${migrateName}] Migrating Up...`);
+
+	return knex.schema
+		.alterTable("user_permission", (table) => {
+			table.json("meta").nullable();
+		})
+		.then(() => {
+			logger.info(`[${migrateName}] user_permission Table altered`);
+		});
+};
+
+/**
+ * Undo Migrate
+ *
+ * @param   {Object} knex
+ * @returns {Promise}
+ */
+const down = (knex) => {
+	logger.info(`[${migrateName}] Migrating Down...`);
+
+	return knex.schema
+		.alterTable("user_permission", (table) => {
+			table.dropColumn("meta");
+		})
+		.then(() => {
+			logger.info(`[${migrateName}] user_permission Table altered`);
+		});
+};
+
+export { up, down };

backend/models/access_list.js

@@ -31,6 +31,10 @@ class AccessList extends Model {
 
 	$parseDatabaseJson(json) {
 		const thisJson = super.$parseDatabaseJson(json);
+		// Postgres returns COUNT() as a string
+		if (typeof thisJson.proxy_host_count === "string") {
+			thisJson.proxy_host_count = Number.parseInt(thisJson.proxy_host_count, 10);
+		}
 		return convertIntFieldsToBool(thisJson, boolFields);
 	}
 

backend/models/auth.js

@@ -1,6 +1,7 @@
 // Objection Docs:
 // http://vincit.github.io/objection.js/
 
+import crypto from "node:crypto";
 import bcrypt from "bcrypt";
 import { Model } from "objection";
 import db from "../db.js";
@@ -11,10 +12,15 @@ import User from "./user.js";
 Model.knex(db());
 
 const boolFields = ["is_deleted"];
+const saltRounds = 13;
+
+// A hash of a random value, for a login that names an account with no password to check it against. Hashed
+// with the same cost as real passwords, and made at startup so the first such login takes no longer either.
+const dummyHash = bcrypt.hash(crypto.randomBytes(32).toString("hex"), saltRounds);
 
 function encryptPassword() {
 	if (this.type === "password" && this.secret) {
-		return bcrypt.hash(this.secret, 13).then((hash) => {
+		return bcrypt.hash(this.secret, saltRounds).then((hash) => {
 			this.secret = hash;
 		});
 	}
@@ -60,6 +66,18 @@ class Auth extends Model {
 		return bcrypt.compare(password, this.secret);
 	}
 
+	/**
+	 * Take as long as verifyPassword does, then fail. bcrypt is nearly all of the time a login takes, so a login
+	 * that skipped it for an unknown account would answer far sooner, and say which accounts exist.
+	 *
+	 * @param {String} password
+	 * @returns {Promise<false>}
+	 */
+	static async verifyDummyPassword(password) {
+		await bcrypt.compare(password, await dummyHash);
+		return false;
+	}
+
 	static get name() {
 		return "Auth";
 	}

backend/models/certificate.js

@@ -15,6 +15,16 @@ Model.knex(db());
 
 const boolFields = ["is_deleted"];
 
+const cleanDomainNames = (domainNames) => {
+	// Sort domain_names
+	if (typeof domainNames !== "undefined") {
+		const newDomainNames = domainNames.filter((name) => name != null);
+		newDomainNames.sort();
+		return newDomainNames;
+	}
+	return [];
+};
+
 class Certificate extends Model {
 	$beforeInsert() {
 		this.created_on = now();
@@ -26,7 +36,9 @@ class Certificate extends Model {
 		}
 
 		// Default for domain_names
-		if (typeof this.domain_names === "undefined") {
+		if (typeof this.domain_names !== "undefined") {
+			this.domain_names = cleanDomainNames(this.domain_names);
+		} else {
 			this.domain_names = [];
 		}
 
@@ -34,16 +46,12 @@ class Certificate extends Model {
 		if (typeof this.meta === "undefined") {
 			this.meta = {};
 		}
-
-		this.domain_names.sort();
 	}
 
 	$beforeUpdate() {
 		this.modified_on = now();
-
-		// Sort domain_names
 		if (typeof this.domain_names !== "undefined") {
-			this.domain_names.sort();
+			this.domain_names = cleanDomainNames(this.domain_names);
 		}
 	}
 

backend/models/dead_host.js

@@ -3,7 +3,7 @@
 
 import { Model } from "objection";
 import db from "../db.js";
-import { convertBoolFieldsToInt, convertIntFieldsToBool } from "../lib/helpers.js";
+import { castJsonIfNeed, convertBoolFieldsToInt, convertIntFieldsToBool } from "../lib/helpers.js";
 import Certificate from "./certificate.js";
 import now from "./now_helper.js";
 import User from "./user.js";
@@ -61,6 +61,18 @@ class DeadHost extends Model {
 		return ["domain_names", "meta"];
 	}
 
+	static get defaultAllowGraph() {
+		return "[owner,certificate]";
+	}
+
+	static get defaultExpand() {
+		return ["certificate", "owner"];
+	}
+
+	static get defaultOrder() {
+		return [castJsonIfNeed("domain_names"), "ASC"];
+	}
+
 	static get relationMappings() {
 		return {
 			owner: {

backend/models/proxy_host.js

@@ -3,7 +3,7 @@
 
 import { Model } from "objection";
 import db from "../db.js";
-import { convertBoolFieldsToInt, convertIntFieldsToBool } from "../lib/helpers.js";
+import { castJsonIfNeed, convertBoolFieldsToInt, convertIntFieldsToBool } from "../lib/helpers.js";
 import AccessList from "./access_list.js";
 import Certificate from "./certificate.js";
 import now from "./now_helper.js";
@@ -73,6 +73,18 @@ class ProxyHost extends Model {
 		return ["domain_names", "meta", "locations"];
 	}
 
+	static get defaultAllowGraph() {
+		return "[owner,access_list.[clients,items],certificate]";
+	}
+
+	static get defaultExpand() {
+		return ["owner", "certificate", "access_list.[clients,items]"];
+	}
+
+	static get defaultOrder() {
+		return [castJsonIfNeed("domain_names"), "ASC"];
+	}
+
 	static get relationMappings() {
 		return {
 			owner: {
@@ -112,4 +124,23 @@ class ProxyHost extends Model {
 	}
 }
 
+/**
+ * Restricts a proxy host query to hosts the user owns or has been granted by id
+ * (user permission meta.proxy_host_ids).
+ *
+ * @param   {Object}  query           objection query on proxy_host
+ * @param   {Number}  userId
+ * @param   {Object}  [permissionMeta] the user's permission meta
+ * @returns {Object}  the query
+ */
+export const scopeToOwnerOrAllowed = (query, userId, permissionMeta) => {
+	const allowedIds = permissionMeta?.proxy_host_ids || [];
+	return query.andWhere((builder) => {
+		builder.where("owner_user_id", userId);
+		if (allowedIds.length > 0) {
+			builder.orWhereIn("id", allowedIds);
+		}
+	});
+};
+
 export default ProxyHost;

backend/models/redirection_host.js

@@ -3,7 +3,7 @@
 
 import { Model } from "objection";
 import db from "../db.js";
-import { convertBoolFieldsToInt, convertIntFieldsToBool } from "../lib/helpers.js";
+import { castJsonIfNeed, convertBoolFieldsToInt, convertIntFieldsToBool } from "../lib/helpers.js";
 import Certificate from "./certificate.js";
 import now from "./now_helper.js";
 import User from "./user.js";
@@ -70,6 +70,18 @@ class RedirectionHost extends Model {
 		return ["domain_names", "meta"];
 	}
 
+	static get defaultAllowGraph() {
+		return "[owner,certificate]";
+	}
+
+	static get defaultExpand() {
+		return ["certificate", "owner"];
+	}
+
+	static get defaultOrder() {
+		return [castJsonIfNeed("domain_names"), "ASC"];
+	}
+
 	static get relationMappings() {
 		return {
 			owner: {

backend/models/setting.js

@@ -7,23 +7,23 @@ import db from "../db.js";
 Model.knex(db());
 
 class Setting extends Model {
-	$beforeInsert () {
+	$beforeInsert() {
 		// Default for meta
-		if (typeof this.meta === 'undefined') {
+		if (typeof this.meta === "undefined") {
 			this.meta = {};
 		}
 	}
 
-	static get name () {
-		return 'Setting';
+	static get name() {
+		return "Setting";
 	}
 
-	static get tableName () {
-		return 'setting';
+	static get tableName() {
+		return "setting";
 	}
 
-	static get jsonAttributes () {
-		return ['meta'];
+	static get jsonAttributes() {
+		return ["meta"];
 	}
 }
 

backend/models/stream.js

@@ -1,6 +1,6 @@
 import { Model } from "objection";
 import db from "../db.js";
-import { convertBoolFieldsToInt, convertIntFieldsToBool } from "../lib/helpers.js";
+import { castJsonIfNeed, convertBoolFieldsToInt, convertIntFieldsToBool } from "../lib/helpers.js";
 import Certificate from "./certificate.js";
 import now from "./now_helper.js";
 import User from "./user.js";
@@ -46,6 +46,18 @@ class Stream extends Model {
 		return ["meta"];
 	}
 
+	static get defaultAllowGraph() {
+		return "[owner,certificate]";
+	}
+
+	static get defaultExpand() {
+		return ["certificate", "owner"];
+	}
+
+	static get defaultOrder() {
+		return [castJsonIfNeed("incoming_port"), "ASC"];
+	}
+
 	static get relationMappings() {
 		return {
 			owner: {

backend/models/user_permission.js

@@ -8,21 +8,25 @@ import now from "./now_helper.js";
 Model.knex(db());
 
 class UserPermission extends Model {
-	$beforeInsert () {
-		this.created_on  = now();
+	$beforeInsert() {
+		this.created_on = now();
 		this.modified_on = now();
 	}
 
-	$beforeUpdate () {
+	$beforeUpdate() {
 		this.modified_on = now();
 	}
 
-	static get name () {
-		return 'UserPermission';
+	static get name() {
+		return "UserPermission";
 	}
 
-	static get tableName () {
-		return 'user_permission';
+	static get tableName() {
+		return "user_permission";
+	}
+
+	static get jsonAttributes() {
+		return ["meta"];
 	}
 }
 

backend/package.json

@@ -8,42 +8,44 @@
 	"type": "module",
 	"scripts": {
 		"lint": "biome lint",
+		"test": "node --test test/",
 		"prettier": "biome format --write .",
-		"validate-schema": "node validate-schema.js"
+		"validate-schema": "node validate-schema.js",
+		"regenerate-config": "node scripts/regenerate-config"
 	},
 	"dependencies": {
-		"@apidevtools/json-schema-ref-parser": "^14.1.1",
-		"ajv": "^8.18.0",
-		"archiver": "^7.0.1",
+		"@apidevtools/json-schema-ref-parser": "^16.0.6",
+		"ajv": "^8.20.0",
+		"archiver": "^8.0.0",
 		"batchflow": "^0.4.0",
 		"bcrypt": "^6.0.0",
-		"better-sqlite3": "^12.6.2",
-		"body-parser": "^2.2.2",
-		"compression": "^1.7.4",
-		"express": "^5.2.1",
-		"express-fileupload": "^1.5.2",
+		"better-sqlite3": "^13.0.3",
+		"body-parser": "^2.4.0",
+		"chalk": "6.0.1",
+		"compression": "^1.8.2",
+		"express": "^5.3.0",
+		"express-fileupload": "^1.6.0",
 		"gravatar": "^1.8.2",
 		"jsonwebtoken": "^9.0.3",
-		"knex": "3.1.0",
-		"liquidjs": "10.24.0",
-		"lodash": "^4.17.23",
-		"moment": "^2.30.1",
-		"mysql2": "^3.17.1",
-		"node-rsa": "^1.1.1",
-		"objection": "3.1.5",
-		"otplib": "^13.3.0",
+		"knex": "3.3.0",
+		"liquidjs": "10.30.1",
+		"lodash": "^4.18.1",
+		"moment": "^2.31.0",
+		"mysql2": "^3.24.6",
+		"node-rsa": "^2.0.0",
+		"objection": "4.4.3",
+		"otplib": "^13.6.0",
 		"path": "^0.12.7",
-		"pg": "^8.18.0",
-		"proxy-agent": "^6.5.0",
+		"pg": "^8.23.1",
+		"proxy-agent": "^8.0.2",
 		"signale": "1.4.0",
-		"sqlite3": "^5.1.7",
-		"temp-write": "^4.0.0"
+		"sqlite3": "^6.0.1",
+		"temp-write": "^6.0.1"
 	},
 	"devDependencies": {
-		"@apidevtools/swagger-parser": "^12.1.0",
-		"@biomejs/biome": "^2.3.14",
-		"chalk": "5.6.2",
-		"nodemon": "^3.1.11"
+		"@apidevtools/swagger-parser": "^13.1.0",
+		"@biomejs/biome": "^2.6.6",
+		"nodemon": "^3.1.14"
 	},
 	"signale": {
 		"displayDate": true,

backend/routes/audit-log.js

@@ -86,10 +86,7 @@ router
 				},
 				{
 					event_id: req.params.event_id,
-					expand:
-						typeof req.query.expand === "string"
-							? req.query.expand.split(",")
-							: null,
+					expand: typeof req.query.expand === "string" ? req.query.expand.split(",") : null,
 				},
 			);
 

backend/routes/ci.js

@@ -0,0 +1,59 @@
+import express from "express";
+import dnsPlugins from "../certbot/dns-plugins.json" with { type: "json" };
+import { installPlugin } from "../lib/certbot.js";
+import { debug, express as logger } from "../logger.js";
+
+const router = express.Router({
+	caseSensitive: true,
+	strict: true,
+	mergeParams: true,
+});
+
+/**
+ * ONLY AVAILABLE IN CI ENVIRONMENT!
+ */
+
+/**
+ * /api/ci/certbot-plugins
+ */
+router
+	.route("/certbot-plugins")
+	.options((_, res) => {
+		res.sendStatus(204);
+	})
+
+	// Return all certbot plugins
+	.get(async (_req, res, _next) => {
+		res.status(200).send(dnsPlugins);
+	});
+
+/**
+ * /api/ci/certbot-plugins/{plugin}
+ */
+router
+	.route("/certbot-plugins/:plugin")
+	.options((_, res) => {
+		res.sendStatus(204);
+	})
+
+	// Install a certbot plugin
+	.post(async (req, res, next) => {
+		try {
+			const pluginName = req.params.plugin;
+			// check if plugin exists
+			if (!dnsPlugins[pluginName]) {
+				return res.status(404).send({
+					error: "Plugin not found",
+				});
+			}
+
+			await installPlugin(pluginName);
+			res.status(200).send(true);
+		} catch (err) {
+			debug(logger, `${req.method.toUpperCase()} ${req.path}: ${err}`);
+			next(err);
+		}
+		return;
+	});
+
+export default router;

backend/routes/logs.js

@@ -0,0 +1,112 @@
+import express from "express";
+import internalLogViewer from "../internal/log-viewer.js";
+import jwtdecode from "../lib/express/jwt-decode.js";
+import validator from "../lib/validator/index.js";
+import { debug, express as logger } from "../logger.js";
+
+const router = express.Router({
+	caseSensitive: true,
+	strict: true,
+	mergeParams: true,
+});
+
+/**
+ * /api/logs/sources
+ */
+router
+	.route("/sources")
+	.options((_, res) => {
+		res.sendStatus(204);
+	})
+	.all(jwtdecode())
+
+	/**
+	 * GET /api/logs/sources
+	 *
+	 * Lists the log sources available for the log viewer
+	 */
+	.get(async (req, res, next) => {
+		try {
+			const data = await internalLogViewer.listSources(res.locals.access);
+			res.status(200).send(data);
+		} catch (err) {
+			debug(logger, `${req.method.toUpperCase()} ${req.path}: ${err}`);
+			next(err);
+		}
+	});
+
+/**
+ * /api/logs/tail
+ */
+router
+	.route("/tail")
+	.options((_, res) => {
+		res.sendStatus(204);
+	})
+	.all(jwtdecode())
+
+	/**
+	 * GET /api/logs/tail
+	 *
+	 * Retrieve the last N lines of a log source
+	 */
+	.get(async (req, res, next) => {
+		try {
+			const data = await validator(
+				{
+					required: ["type"],
+					additionalProperties: false,
+					properties: {
+						type: {
+							type: "string",
+							enum: ["system", "letsencrypt", "host"],
+						},
+						host_type: {
+							anyOf: [
+								{ type: "null" },
+								{ type: "string", enum: ["proxy", "redirection", "dead", "stream"] },
+							],
+						},
+						host_id: {
+							anyOf: [{ type: "null" }, { type: "integer", minimum: 1 }],
+						},
+						channel: {
+							anyOf: [{ type: "null" }, { type: "string", enum: ["access", "error"] }],
+						},
+						lines: {
+							anyOf: [{ type: "null" }, { type: "integer", minimum: 1, maximum: 1000 }],
+						},
+						level: {
+							anyOf: [
+								{ type: "null" },
+								{
+									type: "string",
+									enum: ["INFO", "WARN", "ERROR", "DEBUG", "SUCCESS", "FATAL", "COMPLETE"],
+								},
+							],
+						},
+						search: {
+							anyOf: [{ type: "null" }, { type: "string", minLength: 1, maxLength: 200 }],
+						},
+					},
+				},
+				{
+					type: req.query.type,
+					host_type: typeof req.query.host_type === "string" ? req.query.host_type : null,
+					host_id: typeof req.query.host_id !== "undefined" ? req.query.host_id : null,
+					channel: typeof req.query.channel === "string" ? req.query.channel : null,
+					lines: typeof req.query.lines !== "undefined" ? req.query.lines : null,
+					level: typeof req.query.level === "string" ? req.query.level : null,
+					search: typeof req.query.search === "string" ? req.query.search : null,
+				},
+			);
+
+			const result = await internalLogViewer.tail(res.locals.access, data);
+			res.status(200).send(result);
+		} catch (err) {
+			debug(logger, `${req.method.toUpperCase()} ${req.path}: ${err}`);
+			next(err);
+		}
+	});
+
+export default router;

backend/routes/main.js

@@ -1,8 +1,12 @@
 import express from "express";
+import { isCI } from "../lib/config.js";
 import errs from "../lib/error.js";
+import logRequest from "../lib/express/log-request.js";
 import pjson from "../package.json" with { type: "json" };
 import { isSetup } from "../setup.js";
 import auditLogRoutes from "./audit-log.js";
+import ciRoutes from "./ci.js";
+import logsRoutes from "./logs.js";
 import accessListsRoutes from "./nginx/access_lists.js";
 import certificatesHostsRoutes from "./nginx/certificates.js";
 import deadHostsRoutes from "./nginx/dead_hosts.js";
@@ -22,6 +26,8 @@ const router = express.Router({
 	mergeParams: true,
 });
 
+router.use(logRequest);
+
 /**
  * Health Check
  * GET /api
@@ -45,6 +51,7 @@ router.use("/schema", schemaRoutes);
 router.use("/tokens", tokensRoutes);
 router.use("/users", usersRoutes);
 router.use("/audit-log", auditLogRoutes);
+router.use("/logs", logsRoutes);
 router.use("/reports", reportsRoutes);
 router.use("/settings", settingsRoutes);
 router.use("/version", versionRoutes);
@@ -55,6 +62,11 @@ router.use("/nginx/streams", streamsRoutes);
 router.use("/nginx/access-lists", accessListsRoutes);
 router.use("/nginx/certificates", certificatesHostsRoutes);
 
+// Only include CI routes if we're in a CI environment
+if (isCI()) {
+	router.use("/ci", ciRoutes);
+}
+
 /**
  * API 404 for all other routes
  *

backend/routes/nginx/certificates.js

@@ -44,18 +44,11 @@ router
 					},
 				},
 				{
-					expand:
-						typeof req.query.expand === "string"
-							? req.query.expand.split(",")
-							: null,
+					expand: typeof req.query.expand === "string" ? req.query.expand.split(",") : null,
 					query: typeof req.query.query === "string" ? req.query.query : null,
 				},
 			);
-			const rows = await internalCertificate.getAll(
-				res.locals.access,
-				data.expand,
-				data.query,
-			);
+			const rows = await internalCertificate.getAll(res.locals.access, data.expand, data.query);
 			res.status(200).send(rows);
 		} catch (err) {
 			debug(logger, `${req.method.toUpperCase()} ${req.path}: ${err}`);
@@ -70,15 +63,9 @@ router
 	 */
 	.post(async (req, res, next) => {
 		try {
-			const payload = await apiValidator(
-				getValidationSchema("/nginx/certificates", "post"),
-				req.body,
-			);
+			const payload = await apiValidator(getValidationSchema("/nginx/certificates", "post"), req.body);
 			req.setTimeout(900000); // 15 minutes timeout
-			const result = await internalCertificate.create(
-				res.locals.access,
-				payload,
-			);
+			const result = await internalCertificate.create(res.locals.access, payload);
 			res.status(201).send(result);
 		} catch (err) {
 			debug(logger, `${req.method.toUpperCase()} ${req.path}: ${err}`);
@@ -139,16 +126,10 @@ router
 	 */
 	.post(async (req, res, next) => {
 		try {
-			const payload = await apiValidator(
-				getValidationSchema("/nginx/certificates/test-http", "post"),
-				req.body,
-			);
+			const payload = await apiValidator(getValidationSchema("/nginx/certificates/test-http", "post"), req.body);
 			req.setTimeout(60000); // 1 minute timeout
 
-			const result = await internalCertificate.testHttpsChallenge(
-				res.locals.access,
-				payload,
-			);
+			const result = await internalCertificate.testHttpsChallenge(res.locals.access, payload);
 			res.status(200).send(result);
 		} catch (err) {
 			debug(logger, `${req.method.toUpperCase()} ${req.path}: ${err}`);
@@ -180,7 +161,7 @@ router
 		}
 
 		try {
-			const result = await internalCertificate.validate({
+			const result = await internalCertificate.validate(res.locals.access, {
 				files: req.files,
 			});
 			res.status(200).send(result);
@@ -224,10 +205,7 @@ router
 				},
 				{
 					certificate_id: req.params.certificate_id,
-					expand:
-						typeof req.query.expand === "string"
-							? req.query.expand.split(",")
-							: null,
+					expand: typeof req.query.expand === "string" ? req.query.expand.split(",") : null,
 				},
 			);
 			const row = await internalCertificate.get(res.locals.access, {

backend/routes/nginx/dead_hosts.js

@@ -194,9 +194,11 @@ router
 	/**
 	 * POST /api/nginx/dead-hosts/123/disable
 	 */
-	.post((req, res, next) => {
+	.post(async (req, res, next) => {
 		try {
-			const result = internalDeadHost.disable(res.locals.access, { id: Number.parseInt(req.params.host_id, 10) });
+			const result = await internalDeadHost.disable(res.locals.access, {
+				id: Number.parseInt(req.params.host_id, 10),
+			});
 			res.status(200).send(result);
 		} catch (err) {
 			debug(logger, `${req.method.toUpperCase()} ${req.path}: ${err}`);

backend/routes/nginx/proxy_hosts.js

@@ -1,4 +1,5 @@
 import express from "express";
+import fs from "node:fs";
 import internalProxyHost from "../../internal/proxy-host.js";
 import jwtdecode from "../../lib/express/jwt-decode.js";
 import apiValidator from "../../lib/validator/api.js";
@@ -206,4 +207,70 @@ router
 		}
 	});
 
+/**
+ * Proxy-host logs
+ *
+ * /api/nginx/proxy-hosts/123/logs
+ */
+router
+	.route("/:host_id/logs")
+	.options((_, res) => {
+		res.sendStatus(204);
+	})
+	.all(jwtdecode())
+
+	/**
+	 * GET /api/nginx/proxy-hosts/123/logs
+	 *
+	 * Retrieve logs for a specific proxy-host
+	 */
+	.get(async (req, res, next) => {
+		try {
+			const data = await validator(
+				{
+					required: ["host_id"],
+					additionalProperties: false,
+					properties: {
+						host_id: {
+							$ref: "common#/properties/id",
+						},
+						type: {
+							type: "string",
+							enum: ["access", "error"],
+						},
+					},
+				},
+				{
+					host_id: req.params.host_id,
+					type: req.query.type || "access",
+				},
+			);
+
+			const hostId = Number.parseInt(data.host_id, 10);
+			const logType = data.type === "error" ? "error" : "access";
+			const logFile = `/data/logs/proxy-host-${hostId}_${logType}.log`;
+
+			// Check access permission
+			await res.locals.access.can("proxy_hosts:get", hostId);
+
+			let logs = "";
+			if (fs.existsSync(logFile)) {
+				const content = fs.readFileSync(logFile, { encoding: "utf8" });
+				const lines = content.split("\n");
+				// Return last 1000 lines to avoid huge payloads
+				const maxLines = 1000;
+				if (lines.length > maxLines) {
+					logs = lines.slice(-maxLines).join("\n");
+				} else {
+					logs = content;
+				}
+			}
+
+			res.status(200).send({ logs });
+		} catch (err) {
+			debug(logger, `${req.method.toUpperCase()} ${req.path}: ${err}`);
+			next(err);
+		}
+	});
+
 export default router;

backend/routes/schema.js

@@ -18,7 +18,7 @@ router
 	/**
 	 * GET /schema
 	 */
-	.get(async (req, res) => {
+	.get(async (req, res, next) => {
 		try {
 			const swaggerJSON = await getCompiledSchema();
 

backend/routes/tokens.js

@@ -1,10 +1,56 @@
 import express from "express";
 import internalToken from "../internal/token.js";
 import jwtdecode from "../lib/express/jwt-decode.js";
+import createRateLimiter from "../lib/express/rate-limit.js";
 import apiValidator from "../lib/validator/api.js";
 import { debug, express as logger } from "../logger.js";
 import { getValidationSchema } from "../schema/index.js";
 
+// Two limits apply to logins. Per IP + target account: when the admin UI sits behind another
+// proxy every client can resolve to the same IP, and an IP-only bucket at this size would let
+// anyone lock out every user with a handful of bad logins.
+const tokenRateLimiter = createRateLimiter({
+	windowMs: 60 * 1000,
+	max: 10,
+	message: "Too many login attempts. Please try again in 1 minute.",
+	skipSuccessfulRequests: true,
+	keyGenerator: (req, clientIp) => {
+		const identity = typeof req.body?.identity === "string" ? req.body.identity.trim().toLowerCase() : "";
+		return `${clientIp}|${identity}`;
+	},
+});
+
+// And per IP across all accounts, so one address can't spray passwords over many accounts at
+// 10 a minute each. Only failures count. Behind a shared proxy this cap is shared by everyone
+// using it, which is why it's higher than the per-account one.
+const tokenIpRateLimiter = createRateLimiter({
+	windowMs: 60 * 1000,
+	max: 30,
+	message: "Too many login attempts. Please try again in 1 minute.",
+	skipSuccessfulRequests: true,
+});
+
+// Keyed on the user the challenge belongs to, not the challenge token itself, so fetching
+// fresh challenge tokens doesn't reset the code-guessing budget. The payload is read without
+// verifying the signature; verify2FA does that. A forged token only affects buckets for the
+// sender's own IP.
+const challengeUserId = (token) => {
+	try {
+		const payload = JSON.parse(Buffer.from(String(token).split(".")[1], "base64url").toString());
+		return String(payload?.attrs?.id ?? "");
+	} catch {
+		return "";
+	}
+};
+
+const twoFactorRateLimiter = createRateLimiter({
+	windowMs: 60 * 1000,
+	max: 10,
+	message: "Too many login attempts. Please try again in 1 minute.",
+	skipSuccessfulRequests: true,
+	keyGenerator: (req, clientIp) => `${clientIp}|${challengeUserId(req.body?.challenge_token)}`,
+});
+
 const router = express.Router({
 	caseSensitive: true,
 	strict: true,
@@ -42,7 +88,7 @@ router
 	 *
 	 * Create a new Token
 	 */
-	.post(async (req, res, next) => {
+	.post(tokenIpRateLimiter, tokenRateLimiter, async (req, res, next) => {
 		try {
 			const data = await apiValidator(getValidationSchema("/tokens", "post"), req.body);
 			const result = await internalToken.getTokenFromEmail(data);
@@ -64,7 +110,7 @@ router
 	 *
 	 * Verify 2FA code and get full token
 	 */
-	.post(async (req, res, next) => {
+	.post(twoFactorRateLimiter, async (req, res, next) => {
 		try {
 			const { challenge_token, code } = await apiValidator(getValidationSchema("/tokens/2fa", "post"), req.body);
 			const result = await internalToken.verify2FA(challenge_token, code);

backend/routes/users.js

@@ -48,18 +48,11 @@ router
 					},
 				},
 				{
-					expand:
-						typeof req.query.expand === "string"
-							? req.query.expand.split(",")
-							: null,
+					expand: typeof req.query.expand === "string" ? req.query.expand.split(",") : null,
 					query: typeof req.query.query === "string" ? req.query.query : null,
 				},
 			);
-			const users = await internalUser.getAll(
-				res.locals.access,
-				data.expand,
-				data.query,
-			);
+			const users = await internalUser.getAll(res.locals.access, data.expand, data.query);
 			res.status(200).send(users);
 		} catch (err) {
 			debug(logger, `${req.method.toUpperCase()} ${req.path}: ${err}`);
@@ -95,10 +88,7 @@ router
 				}
 			}
 
-			const payload = await apiValidator(
-				getValidationSchema("/users", "post"),
-				body,
-			);
+			const payload = await apiValidator(getValidationSchema("/users", "post"), body);
 			const user = await internalUser.create(res.locals.access, payload);
 			res.status(201).send(user);
 		} catch (err) {
@@ -169,20 +159,14 @@ router
 				},
 				{
 					user_id: req.params.user_id,
-					expand:
-						typeof req.query.expand === "string"
-							? req.query.expand.split(",")
-							: null,
+					expand: typeof req.query.expand === "string" ? req.query.expand.split(",") : null,
 				},
 			);
 
 			const user = await internalUser.get(res.locals.access, {
 				id: data.user_id,
 				expand: data.expand,
-				omit: internalUser.getUserOmisionsByAccess(
-					res.locals.access,
-					data.user_id,
-				),
+				omit: internalUser.getUserOmisionsByAccess(res.locals.access, data.user_id),
 			});
 			res.status(200).send(user);
 		} catch (err) {
@@ -198,10 +182,7 @@ router
 	 */
 	.put(async (req, res, next) => {
 		try {
-			const payload = await apiValidator(
-				getValidationSchema("/users/{userID}", "put"),
-				req.body,
-			);
+			const payload = await apiValidator(getValidationSchema("/users/{userID}", "put"), req.body);
 			payload.id = req.params.user_id;
 			const result = await internalUser.update(res.locals.access, payload);
 			res.status(200).send(result);
@@ -248,10 +229,7 @@ router
 	 */
 	.put(async (req, res, next) => {
 		try {
-			const payload = await apiValidator(
-				getValidationSchema("/users/{userID}/auth", "put"),
-				req.body,
-			);
+			const payload = await apiValidator(getValidationSchema("/users/{userID}/auth", "put"), req.body);
 			payload.id = req.params.user_id;
 			const result = await internalUser.setPassword(res.locals.access, payload);
 			res.status(200).send(result);
@@ -281,15 +259,9 @@ router
 	 */
 	.put(async (req, res, next) => {
 		try {
-			const payload = await apiValidator(
-				getValidationSchema("/users/{userID}/permissions", "put"),
-				req.body,
-			);
-			payload.id = req.params.user_id;
-			const result = await internalUser.setPermissions(
-				res.locals.access,
-				payload,
-			);
+			const payload = await apiValidator(getValidationSchema("/users/{userID}/permissions", "put"), req.body);
+			payload.id = Number.parseInt(req.params.user_id, 10);
+			const result = await internalUser.setPermissions(res.locals.access, payload);
 			res.status(200).send(result);
 		} catch (err) {
 			debug(logger, `${req.method.toUpperCase()} ${req.path}: ${err}`);
@@ -408,10 +380,7 @@ router
 	 */
 	.post(async (req, res, next) => {
 		try {
-			const { code } = await apiValidator(
-				getValidationSchema("/users/{userID}/2fa/enable", "post"),
-				req.body,
-			);
+			const { code } = await apiValidator(getValidationSchema("/users/{userID}/2fa/enable", "post"), req.body);
 			const result = await internal2FA.enable(res.locals.access, req.params.user_id, code);
 			res.status(200).send(result);
 		} catch (err) {

backend/schema/common.json

@@ -84,7 +84,7 @@
 			"uniqueItems": true,
 			"items": {
 				"type": "string",
-				"pattern": "^[^&| @!#%^();:/\\\\}{=+?<>,~`'\"]+$"
+				"pattern": "^[^&| @!#%^();:/\\\\}{=+?<>,~`'\"\\s\\p{Cc}]+$"
 			},
 			"example": ["example.com", "www.example.com"]
 		},
@@ -147,7 +147,7 @@
 				},
 				{
 					"type": "string",
-					"pattern": "^s*((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]d|1dd|[1-9]?d)(.(25[0-5]|2[0-4]d|1dd|[1-9]?d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]d|1dd|[1-9]?d)(.(25[0-5]|2[0-4]d|1dd|[1-9]?d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]d|1dd|[1-9]?d)(.(25[0-5]|2[0-4]d|1dd|[1-9]?d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]d|1dd|[1-9]?d)(.(25[0-5]|2[0-4]d|1dd|[1-9]?d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]d|1dd|[1-9]?d)(.(25[0-5]|2[0-4]d|1dd|[1-9]?d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]d|1dd|[1-9]?d)(.(25[0-5]|2[0-4]d|1dd|[1-9]?d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]d|1dd|[1-9]?d)(.(25[0-5]|2[0-4]d|1dd|[1-9]?d)){3}))|:)))(%.+)?s*(/([0-9]|[1-9][0-9]|1[0-1][0-9]|12[0-8]))?$"
+					"pattern": "^(?:(?:[0-9A-Fa-f]{1,4}:){7}(?:[0-9A-Fa-f]{1,4}|:)|(?:[0-9A-Fa-f]{1,4}:){6}(?::[0-9A-Fa-f]{1,4}|(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|(?:[0-9A-Fa-f]{1,4}:){5}(?:(?::[0-9A-Fa-f]{1,4}){1,2}|:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|(?:[0-9A-Fa-f]{1,4}:){4}(?:(?::[0-9A-Fa-f]{1,4}){1,3}|(?::[0-9A-Fa-f]{1,4})?:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|(?:[0-9A-Fa-f]{1,4}:){3}(?:(?::[0-9A-Fa-f]{1,4}){1,4}|(?::[0-9A-Fa-f]{1,4}){0,2}:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|(?:[0-9A-Fa-f]{1,4}:){2}(?:(?::[0-9A-Fa-f]{1,4}){1,5}|(?::[0-9A-Fa-f]{1,4}){0,3}:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|(?:[0-9A-Fa-f]{1,4}:)(?:(?::[0-9A-Fa-f]{1,4}){1,6}|(?::[0-9A-Fa-f]{1,4}){0,4}:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:)|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|(?::[0-9A-Fa-f]{1,4}){0,5}:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])(?:\\.(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])){3}|:))(?:/(?:[0-9]|[1-9][0-9]|1[01][0-9]|12[0-8]))?$"
 				},
 				{
 					"type": "string",
@@ -164,7 +164,8 @@
 				"properties": {
 					"username": {
 						"type": "string",
-						"minLength": 1
+						"minLength": 1,
+						"pattern": "^[^:\\p{Cc}]+$"
 					},
 					"password": {
 						"type": "string"

backend/schema/components/certificate-object.json

@@ -31,7 +31,7 @@
 			"uniqueItems": true,
 			"items": {
 				"type": "string",
-				"pattern": "^[^&| @!#%^();:/\\\\}{=+?<>,~`'\"]+$"
+				"pattern": "^[^&| @!#%^();:/\\\\}{=+?<>,~`'\"\\s\\p{Cc}]+$"
 			},
 			"example": ["example.com", "www.example.com"]
 		},

backend/schema/components/log-sources-object.json

@@ -0,0 +1,69 @@
+{
+	"type": "object",
+	"description": "Available log sources for the log viewer",
+	"required": ["system", "letsencrypt", "hosts"],
+	"additionalProperties": false,
+	"properties": {
+		"system": {
+			"type": "object",
+			"properties": {
+				"label": {
+					"type": "string",
+					"example": "System"
+				}
+			}
+		},
+		"letsencrypt": {
+			"type": "object",
+			"properties": {
+				"label": {
+					"type": "string",
+					"example": "Let's Encrypt"
+				}
+			}
+		},
+		"hosts": {
+			"type": "object",
+			"required": ["proxy", "redirection", "dead", "stream"],
+			"additionalProperties": false,
+			"properties": {
+				"proxy": {
+					"$ref": "#/$defs/host-option-list"
+				},
+				"redirection": {
+					"$ref": "#/$defs/host-option-list"
+				},
+				"dead": {
+					"$ref": "#/$defs/host-option-list"
+				},
+				"stream": {
+					"$ref": "#/$defs/host-option-list"
+				}
+			},
+			"example": {
+				"proxy": [{ "id": 1, "label": "example.com" }],
+				"redirection": [],
+				"dead": [],
+				"stream": [{ "id": 1, "label": "Port 5432 → 10.0.0.5:5432" }]
+			}
+		}
+	},
+	"$defs": {
+		"host-option-list": {
+			"type": "array",
+			"items": {
+				"type": "object",
+				"required": ["id", "label"],
+				"properties": {
+					"id": {
+						"$ref": "../common.json#/properties/id"
+					},
+					"label": {
+						"type": "string",
+						"example": "example.com"
+					}
+				}
+			}
+		}
+	}
+}

backend/schema/components/log-tail-object.json

@@ -0,0 +1,30 @@
+{
+	"type": "object",
+	"description": "The last N lines of a log source",
+	"required": ["lines", "size", "truncated", "exists"],
+	"additionalProperties": false,
+	"properties": {
+		"lines": {
+			"type": "array",
+			"items": {
+				"type": "string"
+			},
+			"example": ["2026-01-01T00:00:00.000Z INFO    [Global   ] Backend PID 1 listening on port 3000 ..."]
+		},
+		"size": {
+			"type": "integer",
+			"description": "Size in bytes of the underlying log file",
+			"example": 4096
+		},
+		"truncated": {
+			"type": "boolean",
+			"description": "True when the file is larger than the maximum amount of data scanned per request",
+			"example": false
+		},
+		"exists": {
+			"type": "boolean",
+			"description": "False when the underlying log file does not exist yet",
+			"example": true
+		}
+	}
+}

backend/schema/components/permission-object.json

@@ -1,6 +1,7 @@
 {
 	"type": "object",
 	"minProperties": 1,
+	"additionalProperties": false,
 	"properties": {
 		"visibility": {
 			"type": "string",
@@ -43,6 +44,25 @@
 			"description": "Certificates Permissions",
 			"enum": ["hidden", "view", "manage"],
 			"example": "hidden"
+		},
+		"meta": {
+			"type": "object",
+			"description": "Permission metadata",
+			"example": {
+				"proxy_host_ids": [1]
+			},
+			"properties": {
+				"proxy_host_ids": {
+					"type": "array",
+					"description": "Allowed Proxy Host IDs",
+					"example": [1],
+					"items": {
+						"type": "integer",
+						"description": "Proxy Host ID",
+						"example": 1
+					}
+				}
+			}
 		}
 	}
 }

backend/schema/components/proxy-host-object.json

@@ -45,6 +45,7 @@
 			"type": "string",
 			"minLength": 1,
 			"maxLength": 255,
+			"pattern": "^[^\\s\\p{Cc};{}\"'#\\\\$/]+(?:/[^\\s\\p{Cc};{}\"'#\\\\]*)?$",
 			"example": "127.0.0.1"
 		},
 		"forward_port": {
@@ -108,7 +109,8 @@
 					},
 					"path": {
 						"type": "string",
-						"minLength": 1
+						"minLength": 1,
+						"pattern": "^(?!(?:=|~\\*?|\\^~)$)(?:(?:=|~\\*?|\\^~) +)?(?:\"(?:[^\"\\\\\\p{Cc}]|\\\\[^\\p{Cc}])+\"|(?:[^\\s\\p{Cc};{}\"'#\\\\]|\\\\[^\\s\\p{Cc}])+)$"
 					},
 					"forward_scheme": {
 						"$ref": "#/properties/forward_scheme"
@@ -120,10 +122,14 @@
 						"$ref": "#/properties/forward_port"
 					},
 					"forward_path": {
-						"type": "string"
+						"type": "string",
+						"pattern": "^[^\\s\\p{Cc};{}\"'#\\\\]*$"
 					},
 					"advanced_config": {
 						"type": "string"
+					},
+					"access_list_id": {
+						"$ref": "../common.json#/properties/access_list_id"
 					}
 				}
 			},
@@ -132,7 +138,8 @@
 					"path": "/app",
 					"forward_scheme": "http",
 					"forward_host": "example.com",
-					"forward_port": 80
+					"forward_port": 80,
+					"access_list_id": 0
 				}
 			]
 		},

backend/schema/components/redirection-host-object.json

@@ -57,7 +57,7 @@
 		"forward_domain_name": {
 			"description": "Domain Name",
 			"type": "string",
-			"pattern": "^(?:[^.*]+\\.?)+[^.]$",
+			"pattern": "^(?=[^\\s\\p{Cc};{}\"'#\\\\]+$)[^.*]+(?:\\.[^.*]+)*\\.?[^.]$",
 			"example": "jc21.com"
 		},
 		"preserve_path": {

backend/schema/components/stream-object.json

@@ -39,12 +39,12 @@
 				{
 					"description": "Domain Name",
 					"type": "string",
-					"pattern": "^(?:[^.*]+\\.?)+[^.]$",
+					"pattern": "^(?=[^\\s\\p{Cc};{}\"'#\\\\$/]+$)[^.*]+(?:\\.[^.*]+)*\\.?[^.]$",
 					"example": "example.com"
 				},
 				{
 					"type": "string",
-					"format": "^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}$"
+					"pattern": "^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}$"
 				},
 				{
 					"type": "string",

backend/schema/paths/logs/sources/get.json

@@ -0,0 +1,36 @@
+{
+	"operationId": "getLogSources",
+	"summary": "Get available log sources",
+	"tags": ["logs"],
+	"security": [
+		{
+			"bearerAuth": ["admin"]
+		}
+	],
+	"responses": {
+		"200": {
+			"description": "200 response",
+			"content": {
+				"application/json": {
+					"examples": {
+						"default": {
+							"value": {
+								"system": { "label": "System" },
+								"letsencrypt": { "label": "Let's Encrypt" },
+								"hosts": {
+									"proxy": [{ "id": 1, "label": "example.com" }],
+									"redirection": [],
+									"dead": [],
+									"stream": [{ "id": 1, "label": "Port 5432 → 10.0.0.5:5432" }]
+								}
+							}
+						}
+					},
+					"schema": {
+						"$ref": "../../../components/log-sources-object.json"
+					}
+				}
+			}
+		}
+	}
+}

backend/schema/paths/logs/tail/get.json

@@ -0,0 +1,100 @@
+{
+	"operationId": "getLogTail",
+	"summary": "Get the last N lines of a log source",
+	"tags": ["logs"],
+	"security": [
+		{
+			"bearerAuth": ["admin"]
+		}
+	],
+	"parameters": [
+		{
+			"in": "query",
+			"name": "type",
+			"required": true,
+			"description": "Which log source to read",
+			"schema": {
+				"type": "string",
+				"enum": ["system", "letsencrypt", "host"]
+			}
+		},
+		{
+			"in": "query",
+			"name": "host_type",
+			"description": "Required when type=host",
+			"schema": {
+				"type": "string",
+				"enum": ["proxy", "redirection", "dead", "stream"]
+			}
+		},
+		{
+			"in": "query",
+			"name": "host_id",
+			"description": "Required when type=host",
+			"schema": {
+				"type": "integer",
+				"minimum": 1
+			}
+		},
+		{
+			"in": "query",
+			"name": "channel",
+			"description": "Required when type=host - which log file to read (not to be confused with host_type=stream)",
+			"schema": {
+				"type": "string",
+				"enum": ["access", "error"]
+			}
+		},
+		{
+			"in": "query",
+			"name": "lines",
+			"description": "Number of lines to return from the end of the file",
+			"schema": {
+				"type": "integer",
+				"minimum": 1,
+				"maximum": 1000,
+				"default": 200
+			}
+		},
+		{
+			"in": "query",
+			"name": "level",
+			"description": "Only applicable to type=system",
+			"schema": {
+				"type": "string",
+				"enum": ["INFO", "WARN", "ERROR", "DEBUG", "SUCCESS", "FATAL", "COMPLETE"]
+			}
+		},
+		{
+			"in": "query",
+			"name": "search",
+			"description": "Case-insensitive plain-text search",
+			"schema": {
+				"type": "string",
+				"maxLength": 200
+			}
+		}
+	],
+	"responses": {
+		"200": {
+			"description": "200 response",
+			"content": {
+				"application/json": {
+					"examples": {
+						"default": {
+							"value": {
+								"lines": ["2026-01-01T00:00:00.000Z INFO    [Global   ] Backend PID 1 listening on port 3000 ..."],
+								"size": 4096,
+								"truncated": false,
+								"exists": true
+							}
+						}
+					},
+					"schema": {
+						"$ref": "../../../components/log-tail-object.json"
+					}
+				}
+			}
+		}
+	}
+}

backend/schema/paths/nginx/certificates/certID/delete.json

@@ -35,6 +35,26 @@
 					}
 				}
 			}
+		},
+		"400": {
+			"description": "400 response",
+			"content": {
+				"application/json": {
+					"examples": {
+						"default": {
+							"value": {
+								"error": {
+									"code": 400,
+									"message": "Certificate is in use by 1 proxy host. Remove it from them before deleting it."
+								}
+							}
+						}
+					},
+					"schema": {
+						"$ref": "../../../../components/error.json"
+					}
+				}
+			}
 		}
 	}
 }

backend/schema/paths/users/userID/permissions/put.json

@@ -35,7 +35,10 @@
 					"dead_hosts": "hidden",
 					"proxy_hosts": "manage",
 					"redirection_hosts": "hidden",
-					"streams": "hidden"
+					"streams": "hidden",
+					"meta": {
+						"proxy_host_ids": [1]
+					}
 				}
 			}
 		}

backend/schema/swagger.json

@@ -24,6 +24,10 @@
 			"name": "audit-log",
 			"description": "Endpoints related to Audit Logs"
 		},
+		{
+			"name": "logs",
+			"description": "Endpoints for viewing system, Let's Encrypt and per-host logs"
+		},
 		{
 			"name": "access-lists",
 			"description": "Endpoints related to Access Lists"
@@ -81,6 +85,16 @@
 				"$ref": "./paths/audit-log/id/get.json"
 			}
 		},
+		"/logs/sources": {
+			"get": {
+				"$ref": "./paths/logs/sources/get.json"
+			}
+		},
+		"/logs/tail": {
+			"get": {
+				"$ref": "./paths/logs/tail/get.json"
+			}
+		},
 		"/nginx/access-lists": {
 			"get": {
 				"$ref": "./paths/nginx/access-lists/get.json"

backend/scripts/regenerate-config

@@ -0,0 +1,76 @@
+#!/usr/bin/env node
+
+import * as process from "node:process"; // Use the node: protocol for built-ins
+import internalNginx from "../internal/nginx.js";
+import { global as logger } from "../logger.js";
+import deadHostModel from "../models/dead_host.js";
+import proxyHostModel from "../models/proxy_host.js";
+import redirectionHostModel from "../models/redirection_host.js";
+import streamModel from "../models/stream.js";
+
+const args = process.argv.slice(2);
+const UNATTENDED = args.includes("-y") || args.includes("--yes");
+const DRY_RUN = args.includes("--dry-run");
+
+if (args.includes("--help") || args.includes("-h")) {
+	console.log("\nThis will iterate over all Hosts and regnerate their Nginx configs.\n")
+	console.log("Usage: ./regenerate-config [-h|--help] [-y|--yes] [--dry-run]\n");
+	process.exit(0);
+}
+
+// ask for the user to confirm the action if not in unattended mode
+if (!UNATTENDED && !DRY_RUN) {
+	const readline = await import("node:readline");
+	const rl = readline.createInterface({
+		input: process.stdin,
+		output: process.stdout,
+	});
+
+	const question = (query) =>
+		new Promise((resolve) => rl.question(query, resolve));
+
+	const answer = await question(
+		"This will iterate over all Hosts and regnerate their Nginx configs.\n\nAre you sure you want to proceed? (y/N) ",
+	);
+	rl.close();
+
+	if (answer.toLowerCase() !== "y") {
+		console.log("Aborting.");
+		process.exit(0);
+	}
+}
+
+const logIt = (msg, type = "info") => logger[type](
+	`${DRY_RUN ? '[DRY RUN] ' : ''}${msg}`,
+);
+
+// Let's do it.
+
+const processItems = async (model, type, hostType) => {
+	const rows = await model
+		.query()
+		.where("is_deleted", 0)
+		.andWhere("enabled", 1)
+		.groupBy("id")
+		.allowGraph(model.defaultAllowGraph)
+		.withGraphFetched(`[${model.defaultExpand.join(", ")}]`)
+		.orderBy(...model.defaultOrder);
+
+	logIt(`[${type}] Found ${rows.length} rows to process...`);
+	for (const row of rows) {
+		if (!DRY_RUN) {
+			logIt(`[${type}] Regenerating config #${row.id}: ${row.domain_names ? row.domain_names.join(", ") : 'port ' + row.incoming_port}`);
+			await internalNginx.configure(model, hostType, row);
+		} else {
+			logIt(`[${type}] Skipping generation of config #${row.id}: ${row.domain_names ? row.domain_names.join(", ") : 'port ' + row.incoming_port}`);
+		}
+	}
+};
+
+await processItems(proxyHostModel, "Proxy Host", "proxy_host");
+await processItems(redirectionHostModel, "Redirection Host", "redirection_host");
+await processItems(deadHostModel, "404 Host", "dead_host");
+await processItems(streamModel, "Stream", "stream");
+
+logIt("Completed", "success");
+process.exit(0);

backend/setup.js

@@ -6,11 +6,12 @@ import certificateModel from "./models/certificate.js";
 import settingModel from "./models/setting.js";
 import userModel from "./models/user.js";
 import userPermissionModel from "./models/user_permission.js";
+import fs from "node:fs/promises";
 
 export const isSetup = async () => {
 	const row = await userModel.query().select("id").where("is_deleted", 0).first();
 	return row?.id > 0;
-}
+};
 
 /**
  * Creates a default admin users if one doesn't already exist in the database
@@ -44,18 +45,14 @@ const setupDefaultUser = async () => {
 			roles: ["admin"],
 		};
 
-		const user = await userModel
-			.query()
-			.insertAndFetch(data);
+		const user = await userModel.query().insertAndFetch(data);
 
-		await authModel
-			.query()
-			.insert({
-				user_id: user.id,
-				type: "password",
-				secret: initialAdminPassword,
-				meta: {},
-			});
+		await authModel.query().insert({
+			user_id: user.id,
+			type: "password",
+			secret: initialAdminPassword,
+			meta: {},
+		});
 
 		await userPermissionModel.query().insert({
 			user_id: user.id,
@@ -77,22 +74,16 @@ const setupDefaultUser = async () => {
  * @returns {Promise}
  */
 const setupDefaultSettings = async () => {
-	const row = await settingModel
-		.query()
-		.select("id")
-		.where({ id: "default-site" })
-		.first();
+	const row = await settingModel.query().select("id").where({ id: "default-site" }).first();
 
 	if (!row?.id) {
-		await settingModel
-			.query()
-			.insert({
-				id: "default-site",
-				name: "Default Site",
-				description: "What to show when Nginx is hit with an unknown Host",
-				value: "congratulations",
-				meta: {},
-			});
+		await settingModel.query().insert({
+			id: "default-site",
+			name: "Default Site",
+			description: "What to show when Nginx is hit with an unknown Host",
+			value: "congratulations",
+			meta: {},
+		});
 		logger.info("Default settings added");
 	}
 };
@@ -103,14 +94,10 @@ const setupDefaultSettings = async () => {
  * @returns {Promise}
  */
 const setupCertbotPlugins = async () => {
-	const certificates = await certificateModel
-		.query()
-		.where("is_deleted", 0)
-		.andWhere("provider", "letsencrypt");
+	const certificates = await certificateModel.query().where("is_deleted", 0).andWhere("provider", "letsencrypt");
 
 	if (certificates?.length) {
 		const plugins = [];
-		const promises = [];
 
 		certificates.map((certificate) => {
 			if (certificate.meta && certificate.meta.dns_challenge === true) {
@@ -118,24 +105,23 @@ const setupCertbotPlugins = async () => {
 					plugins.push(certificate.meta.dns_provider);
 				}
 
-				// Make sure credentials file exists
-				const credentials_loc = `/etc/letsencrypt/credentials/credentials-${certificate.id}`;
-				// Escape single quotes and backslashes
-				if (typeof certificate.meta.dns_provider_credentials === "string") {
-					const escapedCredentials = certificate.meta.dns_provider_credentials
-						.replaceAll("'", "\\'")
-						.replaceAll("\\", "\\\\");
-					const credentials_cmd = `[ -f '${credentials_loc}' ] || { mkdir -p /etc/letsencrypt/credentials 2> /dev/null; echo '${escapedCredentials}' > '${credentials_loc}' && chmod 600 '${credentials_loc}'; }`;
-					promises.push(utils.exec(credentials_cmd));
-				}
+				// Deliberately does NOT write the DNS credentials file here any more.
+				//
+				// It used to, so that a later `certbot renew` would find the path recorded in its
+				// renewal config. The effect was that every backend restart rewrote a plaintext
+				// DNS provider API token for every DNS-01 certificate, and left it there.
+				//
+				// internalCertificate now writes that file immediately before it runs certbot and
+				// removes it again afterwards, so there is exactly one writer and the credential
+				// is on disk only for the length of a certbot run. Recreating the files at boot
+				// would put every one of them straight back.
 			}
 			return true;
 		});
 
 		await installPlugins(plugins);
 
-		if (promises.length) {
-			await Promise.all(promises);
+		if (plugins.length) {
 			logger.info(`Added Certbot plugins ${plugins.join(", ")}`);
 		}
 	}

backend/templates/_access.conf

@@ -1,4 +1,5 @@
 {% if access_list_id > 0 %}
+  {% if access_list %}
     {% if access_list.items.length > 0 %}
     # Authorization
     auth_basic            "Authorization required";
@@ -10,11 +11,13 @@
 
     {% endif %}
 
+    {% if access_list.clients.length > 0 %}
     # Access Rules: {{ access_list.clients | size }} total
     {% for client in access_list.clients %}
     {{client | nginxAccessRule}}
     {% endfor %}
     deny all;
+    {% endif %}
 
     # Access checks must...
     {% if access_list.satisfy_any == 1 or access_list.satisfy_any == true %}
@@ -22,4 +25,8 @@
     {% else %}
     satisfy all;
     {% endif %}
+  {% else %}
+    # Access List #{{ access_list_id }} could not be loaded: deny rather than serve unrestricted
+    deny all;
+  {% endif %}
 {% endif %}

backend/templates/_hsts.conf

@@ -2,7 +2,11 @@
 {% if ssl_forced == 1 or ssl_forced == true %}
 {% if hsts_enabled == 1 or hsts_enabled == true %}
   # HSTS (ngx_http_headers_module is required) (63072000 seconds = 2 years)
-  add_header Strict-Transport-Security $hsts_header always;
+{% if hsts_subdomains == 1 or hsts_subdomains == true %}
+  add_header Strict-Transport-Security $hsts_header_subdomains always;
+{% else %}
+  add_header Strict-Transport-Security $hsts_header_plain always;
+{% endif %}
 {% endif %}
 {% endif %}
 {% endif %}

backend/templates/_hsts_map.conf

@@ -1,3 +1,6 @@
-map $scheme $hsts_header {
-    https   "max-age=63072000;{% if hsts_subdomains == 1 or hsts_subdomains == true -%} includeSubDomains;{% endif %} preload";
+map $scheme $hsts_header_plain {
+    https   "max-age=63072000; preload";
+}
+map $scheme $hsts_header_subdomains {
+    https   "max-age=63072000;includeSubDomains; preload";
 }

backend/templates/_location.conf

@@ -1,7 +1,7 @@
   location {{ path }} {
     {{ advanced_config }}
 
-    proxy_set_header Host $host;
+    proxy_set_header Host $forward_host;
     proxy_set_header X-Forwarded-Scheme $scheme;
     proxy_set_header X-Forwarded-Proto  $scheme;
     proxy_set_header X-Forwarded-For    $remote_addr;

backend/templates/stream.conf

@@ -5,8 +5,8 @@
 {% if enabled %}
 {% if tcp_forwarding == 1 or tcp_forwarding == true -%}
 server {
-  listen {{ incoming_port }} {%- if certificate %} ssl {%- endif %};
-  {% unless ipv6 -%} # {%- endunless -%} listen [::]:{{ incoming_port }} {%- if certificate %} ssl {%- endif %};
+  listen {{ incoming_port }} reuseport {%- if certificate %} ssl {%- endif %};
+  {% unless ipv6 -%} # {%- endunless -%} listen [::]:{{ incoming_port }} reuseport {%- if certificate %} ssl {%- endif %};
 
   {%- include "_certificates_stream.conf" %}
 
@@ -23,8 +23,8 @@ server {
 
 {% if udp_forwarding == 1 or udp_forwarding == true -%}
 server {
-  listen {{ incoming_port }} udp;
-  {% unless ipv6 -%} # {%- endunless -%} listen [::]:{{ incoming_port }} udp;
+  listen {{ incoming_port }} udp reuseport;
+  {% unless ipv6 -%} # {%- endunless -%} listen [::]:{{ incoming_port }} udp reuseport;
 
   proxy_pass {{ forwarding_host }}:{{ forwarding_port }};
 

backend/test/nginx-values.test.js

@@ -0,0 +1,550 @@
+/**
+ * Regression tests: values a non-admin can set must never add nginx directives.
+ *
+ * Templates write these values into config as-is, so each one is checked three ways:
+ * - the API rejects injection attempts (real request schemas, real validator)
+ * - accepted values, rendered through the real templates, produce exactly the same nginx
+ *   statement structure as a plain value (parsed with nginx's tokenizer rules)
+ * - generateConfig's render-time check refuses unsafe values already stored in the database
+ */
+import assert from "node:assert/strict";
+import fs from "node:fs";
+import { describe, it } from "node:test";
+import {
+	assertHostRenderable,
+	isValid,
+	isValidClientAddress,
+	isValidStreamHost,
+	patterns,
+} from "../lib/nginx-values.js";
+import utils from "../lib/utils.js";
+import apiValidator from "../lib/validator/api.js";
+import { getCompiledSchema, getValidationSchema } from "../schema/index.js";
+
+const TEMPLATES = new URL("../templates/", import.meta.url);
+
+// ---------------------------------------------------------------------------------------------
+// A tokenizer following ngx_conf_read_token: whitespace separates, ";" ends a statement, "{"
+// opens a block, "}" closes one, "#" at the start of a token comments to end of line, quotes
+// group, "\" escapes the next character, and "${" doesn't open a block.
+// ---------------------------------------------------------------------------------------------
+const parseNginx = (text) => {
+	const root = [];
+	const stack = [root];
+	let args = [];
+	let i = 0;
+	const n = text.length;
+	const isSpace = (c) => c === " " || c === "\t" || c === "\r" || c === "\n";
+
+	while (i < n) {
+		const c = text[i];
+		if (isSpace(c)) {
+			i++;
+		} else if (c === "#") {
+			while (i < n && text[i] !== "\n") i++;
+		} else if (c === ";") {
+			if (!args.length) throw new Error(`unexpected ";" at ${i}`);
+			stack.at(-1).push({ name: args[0], args: args.slice(1) });
+			args = [];
+			i++;
+		} else if (c === "{") {
+			if (!args.length) throw new Error(`unexpected "{" at ${i}`);
+			const block = { name: args[0], args: args.slice(1), block: [] };
+			stack.at(-1).push(block);
+			stack.push(block.block);
+			args = [];
+			i++;
+		} else if (c === "}") {
+			if (args.length) throw new Error(`unexpected "}" at ${i}`);
+			if (stack.length === 1) throw new Error(`unbalanced "}" at ${i}`);
+			stack.pop();
+			i++;
+		} else if (c === '"' || c === "'") {
+			let tok = c;
+			i++;
+			while (i < n && text[i] !== c) {
+				if (text[i] === "\\" && i + 1 < n) {
+					tok += text[i] + text[i + 1];
+					i += 2;
+				} else {
+					tok += text[i++];
+				}
+			}
+			if (i >= n) throw new Error("unterminated quoted string");
+			tok += c;
+			i++;
+			if (i < n && !isSpace(text[i]) && text[i] !== ";" && text[i] !== "{") {
+				throw new Error(`unexpected "${text[i]}" after quoted string`);
+			}
+			args.push(tok);
+		} else {
+			let tok = "";
+			while (i < n) {
+				const d = text[i];
+				if (d === "\\" && i + 1 < n) {
+					tok += d + text[i + 1];
+					i += 2;
+					continue;
+				}
+				if (d === "$" && text[i + 1] === "{") {
+					tok += "${";
+					i += 2;
+					continue;
+				}
+				if (isSpace(d) || d === ";" || d === "{") break;
+				tok += d;
+				i++;
+			}
+			args.push(tok);
+		}
+	}
+	if (args.length) throw new Error("statement without ;");
+	if (stack.length !== 1) throw new Error("unclosed block");
+	return root;
+};
+
+// Statement names and argument counts, nested; values themselves don't matter
+const shape = (stmts) =>
+	stmts.map((s) => `${s.name}/${s.args.length}${s.block ? `{${shape(s.block).join(",")}}` : ""}`);
+
+const find = (stmts, name) => {
+	for (const s of stmts) {
+		if (s.name === name) return s;
+		if (s.block) {
+			const f = find(s.block, name);
+			if (f) return f;
+		}
+	}
+	return null;
+};
+
+const render = (file, ctx) =>
+	utils.getRenderEngine().parseAndRender(fs.readFileSync(new URL(file, TEMPLATES), "utf8"), ctx);
+
+// Same structure as a plain value, or throws (a value that breaks parsing changes it too)
+const sameShapeAs = async (file, makeCtx, plain, value) => {
+	const base = shape(parseNginx(await render(file, makeCtx(plain))));
+	let got;
+	try {
+		got = shape(parseNginx(await render(file, makeCtx(value))));
+	} catch {
+		return false;
+	}
+	return JSON.stringify(got) === JSON.stringify(base);
+};
+
+// ---------------------------------------------------------------------------------------------
+// Corpora. "injections" add or change directives when rendered unchecked; "rejected" are other
+// values the rules refuse (they would break or confuse the config without injecting).
+// ---------------------------------------------------------------------------------------------
+const cases = {
+	locationPath: {
+		valid: [
+			"/",
+			"/api",
+			"/api/",
+			"= /exact",
+			"~ \\.php$",
+			"~* \\.(jpg|png|gif)$",
+			"^~ /static/",
+			'~ "^/v[0-9]{1,2}/"',
+			'"/with space"',
+			"/über",
+			"/a$b",
+			"/a\\;b",
+		],
+		injections: [
+			"/x { return 200; } location /y",
+			"/x {} include /etc/passwd; location /y",
+			"/a; } server { listen 8080",
+			"/a\nreturn 200 x;\nlocation /b",
+			'"/a" { return 200; } location "/b"',
+			"/a {",
+		],
+		rejected: ["", " /a", "/a b", "/a#", "/a\\", '"/a', '"/a\\"', "'/a'", "~", "~  /a\tb", "/a}", "/a\u0000"],
+	},
+	forwardHost: {
+		valid: ["127.0.0.1", "backend", "my_service.internal", "10.0.0.1/path/", "backend/$1", "[::1]", "host:8080"],
+		injections: [
+			"a; return 200; #",
+			'a"; return 200; set $x "',
+			"a:80; } location /x { return 200",
+			"a\nreturn 200;",
+		],
+		rejected: ["", "a b", "a'", "a{", "a}", "a#b", "a\\b", "$http_host", "a$b", "/path", "a/b c"],
+	},
+	forwardPath: {
+		valid: ["", "/", "/x", "/x/$1/"],
+		injections: ["/x; return 200", "/x\nreturn 200"],
+		rejected: ["/x y", "/x#", "/x{", "/x\\"],
+	},
+	redirectDomain: {
+		valid: [
+			"example.com",
+			"example.com:8443",
+			"example.com/path",
+			"sub.example.co.uk/a?b=c",
+			"example.com/$1",
+			"ab",
+		],
+		injections: ["example.com; return 200 x; #", "example.com;\nreturn 200", "a.com { return 200; }"],
+		rejected: ["a b.com", "a.com#x", 'a.com"', "a.com\\", ".com", "a..b", "a", "a.com."],
+	},
+	streamHost: {
+		valid: ["example.com", "1.2.3.4", "::1", "backend.internal"],
+		injections: ["a.com:80; } server { listen 9999", "a.com;\nlisten 1"],
+		rejected: ["a.com/x", "$host", "a b", "a.com#"],
+	},
+	domainName: {
+		valid: ["example.com", "*.example.com", "xn--bcher-kva.example", "a-b.example"],
+		injections: ["a.com\nreturn 200 x", "a.com\n} server {"],
+		rejected: ["a\tb.com", "a;b", "a{b", "a b"],
+	},
+	clientAddress: {
+		valid: ["192.168.0.1", "10.0.0.0/8", "all", "::1", "2001:db8::/32", "::ffff:192.168.1.1", "fe80::1"],
+		injections: ["::1%x; allow all", "::1%x\nallow all", "1.2.3.4; allow all"],
+		rejected: ["fe80::1%eth0", "s::1", "1.2.3.4/33", "::ffff:1d.1.1.1"],
+	},
+	accessUsername: {
+		valid: ["admin", "john smith", "ü"],
+		injections: [],
+		rejected: ["a:b", "a\nroot:$apr1$x$y", "\u0000"],
+	},
+};
+
+const checkers = {
+	locationPath: (v) => isValid("locationPath", v),
+	forwardHost: (v) => isValid("forwardHost", v),
+	forwardPath: (v) => isValid("forwardPath", v),
+	redirectDomain: (v) => isValid("redirectDomain", v),
+	streamHost: isValidStreamHost,
+	domainName: (v) => isValid("domainName", v),
+	clientAddress: isValidClientAddress,
+	accessUsername: (v) => isValid("accessUsername", v),
+};
+
+describe("nginx value rules", () => {
+	for (const [name, { valid, injections, rejected }] of Object.entries(cases)) {
+		it(`${name}: accepts valid values`, () => {
+			for (const v of valid) assert.ok(checkers[name](v), `should accept ${JSON.stringify(v)}`);
+		});
+		it(`${name}: rejects injections and other unsafe values`, () => {
+			for (const v of [...injections, ...rejected]) {
+				assert.ok(!checkers[name](v), `should reject ${JSON.stringify(v)}`);
+			}
+		});
+	}
+
+	it("redirect domain rule matches the old rule for token-safe values", () => {
+		const old = /^(?:[^.*]+\.?)+[^.]$/u;
+		const corpus = [
+			"ab",
+			"a.b",
+			"a.b.c",
+			"a..b",
+			".a",
+			"a.",
+			"a*",
+			"a.*",
+			"*a",
+			"a*b",
+			"a.b:80/x",
+			"x",
+			"ab.",
+			"a.b.",
+		];
+		for (const v of corpus) {
+			assert.equal(isValid("redirectDomain", v), old.test(v), `differs for ${JSON.stringify(v)}`);
+		}
+	});
+
+	it("domain rules don't backtrack exponentially", () => {
+		const evil = `${"a".repeat(30000)}.`;
+		const start = process.hrtime.bigint();
+		isValid("redirectDomain", evil);
+		isValidStreamHost(evil);
+		const ms = Number(process.hrtime.bigint() - start) / 1e6;
+		assert.ok(ms < 1000, `took ${ms}ms`);
+	});
+});
+
+describe("API request validation", async () => {
+	await getCompiledSchema();
+
+	const location = (o) => ({ path: "/api", forward_scheme: "http", forward_host: "backend", forward_port: 80, ...o });
+	const proxyHost = (o) => ({
+		domain_names: ["example.com"],
+		forward_scheme: "http",
+		forward_host: "backend",
+		forward_port: 80,
+		...o,
+	});
+	const redirection = (o) => ({
+		domain_names: ["example.com"],
+		forward_scheme: "http",
+		forward_http_code: 301,
+		forward_domain_name: "example.org",
+		...o,
+	});
+	const stream = (o) => ({
+		incoming_port: 9000,
+		forwarding_host: "backend",
+		forwarding_port: 80,
+		tcp_forwarding: true,
+		udp_forwarding: false,
+		...o,
+	});
+
+	// [field, endpoint, method, payload builder]
+	const endpoints = {
+		locationPath: [
+			["/nginx/proxy-hosts", "post", (v) => proxyHost({ locations: [location({ path: v })] })],
+			["/nginx/proxy-hosts/{hostID}", "put", (v) => ({ locations: [location({ path: v })] })],
+		],
+		forwardHost: [
+			["/nginx/proxy-hosts", "post", (v) => proxyHost({ forward_host: v })],
+			["/nginx/proxy-hosts/{hostID}", "put", (v) => ({ forward_host: v })],
+			["/nginx/proxy-hosts/{hostID}", "put", (v) => ({ locations: [location({ forward_host: v })] })],
+		],
+		forwardPath: [["/nginx/proxy-hosts/{hostID}", "put", (v) => ({ locations: [location({ forward_path: v })] })]],
+		redirectDomain: [
+			["/nginx/redirection-hosts", "post", (v) => redirection({ forward_domain_name: v })],
+			["/nginx/redirection-hosts/{hostID}", "put", (v) => ({ forward_domain_name: v })],
+		],
+		streamHost: [
+			["/nginx/streams", "post", (v) => stream({ forwarding_host: v })],
+			["/nginx/streams/{streamID}", "put", (v) => ({ forwarding_host: v })],
+		],
+		domainName: [
+			["/nginx/proxy-hosts", "post", (v) => proxyHost({ domain_names: [v] })],
+			["/nginx/redirection-hosts", "post", (v) => redirection({ domain_names: [v] })],
+			["/nginx/dead-hosts", "post", (v) => ({ domain_names: [v] })],
+			["/nginx/certificates", "post", (v) => ({ provider: "letsencrypt", domain_names: [v] })],
+		],
+		clientAddress: [
+			["/nginx/access-lists", "post", (v) => ({ name: "x", clients: [{ directive: "allow", address: v }] })],
+			["/nginx/access-lists/{listID}", "put", (v) => ({ clients: [{ directive: "allow", address: v }] })],
+		],
+		accessUsername: [
+			["/nginx/access-lists", "post", (v) => ({ name: "x", items: [{ username: v, password: "pw" }] })],
+			["/nginx/access-lists/{listID}", "put", (v) => ({ items: [{ username: v, password: "pw" }] })],
+		],
+	};
+
+	const validate = (path, method, payload) => {
+		const schema = getValidationSchema(path, method);
+		assert.ok(schema, `no request schema for ${method} ${path}`);
+		return apiValidator(schema, payload);
+	};
+
+	for (const [name, list] of Object.entries(endpoints)) {
+		for (const [path, method, build] of list) {
+			it(`${method.toUpperCase()} ${path} ${name}: rejects injections`, async () => {
+				for (const v of [...cases[name].injections, ...cases[name].rejected]) {
+					if (v === "") continue; // minLength cases, covered by the rules above
+					await assert.rejects(validate(path, method, build(v)), /must/, `accepted ${JSON.stringify(v)}`);
+				}
+			});
+			it(`${method.toUpperCase()} ${path} ${name}: accepts valid values`, async () => {
+				for (const v of cases[name].valid) {
+					if (name === "forwardHost" && v === "") continue;
+					await validate(path, method, build(v));
+				}
+			});
+		}
+	}
+
+	it("schema patterns are the ones the render-time check uses", async () => {
+		const s = await getCompiledSchema();
+		const body = (path, method) => s.paths[path][method].requestBody.content["application/json"].schema;
+		const ph = body("/nginx/proxy-hosts", "post").properties;
+		const loc = ph.locations.items.properties;
+		const al = body("/nginx/access-lists", "post").properties;
+		assert.equal(ph.domain_names.items.pattern, patterns.domainName);
+		assert.equal(body("/nginx/certificates", "post").properties.domain_names.items.pattern, patterns.domainName);
+		assert.equal(ph.forward_host.pattern, patterns.forwardHost);
+		assert.equal(loc.path.pattern, patterns.locationPath);
+		assert.equal(loc.forward_host.pattern, patterns.forwardHost);
+		assert.equal(loc.forward_path.pattern, patterns.forwardPath);
+		assert.equal(
+			body("/nginx/redirection-hosts", "post").properties.forward_domain_name.pattern,
+			patterns.redirectDomain,
+		);
+		assert.equal(body("/nginx/streams", "post").properties.forwarding_host.anyOf[0].pattern, patterns.streamDomain);
+		assert.equal(al.clients.items.properties.address.oneOf[1].pattern, patterns.clientIpv6);
+		assert.equal(al.items.items.properties.username.pattern, patterns.accessUsername);
+	});
+});
+
+describe("rendered templates", () => {
+	const locationCtx = (field) => (v) => ({
+		path: "/api",
+		forward_scheme: "http",
+		forward_host: "backend",
+		forward_port: 8080,
+		forward_path: "",
+		advanced_config: "",
+		access_list_id: 0,
+		[field]: v,
+	});
+	const proxyCtx = (field) => (v) => ({
+		enabled: true,
+		id: 1,
+		domain_names: ["example.com"],
+		forward_scheme: "http",
+		forward_host: "backend",
+		forward_port: 80,
+		certificate_id: 0,
+		locations: "",
+		advanced_config: "",
+		use_default_location: true,
+		ipv6: true,
+		[field]: v,
+	});
+	const redirectCtx = (v) => ({
+		enabled: true,
+		id: 1,
+		domain_names: ["example.com"],
+		forward_scheme: "http",
+		forward_http_code: 301,
+		forward_domain_name: v,
+		preserve_path: true,
+		advanced_config: "",
+		use_default_location: true,
+		ipv6: true,
+	});
+	const streamCtx = (v) => ({
+		enabled: true,
+		id: 1,
+		incoming_port: 9000,
+		forwarding_host: v,
+		forwarding_port: 80,
+		tcp_forwarding: true,
+		udp_forwarding: true,
+		ipv6: true,
+	});
+	const accessCtx = (v) => ({
+		access_list_id: 1,
+		access_list: { items: [], clients: [{ directive: "allow", address: v }], satisfy_any: false },
+	});
+
+	// A location modifier is a separate argument, so compare with a plain path using the same one
+	const plainPath = (v) => {
+		const m = v.match(/^(=|~\*?|\^~) +/);
+		return m ? `${m[1]} /api` : "/api";
+	};
+
+	// [template, context builder, plain value (or builder), injections that apply to it]
+	const targets = {
+		locationPath: [["_location.conf", locationCtx("path"), plainPath]],
+		forwardHost: [
+			["_location.conf", locationCtx("forward_host"), "backend"],
+			// Rendered inside double quotes here, so only a quote can break out
+			["proxy_host.conf", proxyCtx("forward_host"), "backend", ['a"; return 200; set $x "', 'a" ; } server { "']],
+		],
+		forwardPath: [["_location.conf", locationCtx("forward_path"), "/x"]],
+		redirectDomain: [["redirection_host.conf", redirectCtx, "example.org"]],
+		streamHost: [["stream.conf", streamCtx, "backend"]],
+		domainName: [["proxy_host.conf", (v) => proxyCtx("domain_names")([v]), "example.com"]],
+		clientAddress: [["_access.conf", accessCtx, "10.0.0.1"]],
+	};
+
+	for (const [name, list] of Object.entries(targets)) {
+		for (const [file, ctx, plain, injections = cases[name].injections] of list) {
+			const plainFor = (v) => (typeof plain === "function" ? plain(v) : plain);
+			it(`${file} ${name}: accepted values keep the statement structure`, async () => {
+				for (const v of cases[name].valid) {
+					if (v === "" && name !== "forwardPath") continue;
+					// generateConfig brackets IPv6 stream hosts before rendering
+					const value = name === "streamHost" && v.includes(":") ? `[${v}]` : v;
+					assert.ok(
+						await sameShapeAs(file, ctx, plainFor(v), value),
+						`${JSON.stringify(v)} changed the config`,
+					);
+				}
+			});
+			it(`${file} ${name}: the injection corpus would change it unchecked`, async () => {
+				assert.ok(injections.length > 0);
+				for (const v of injections) {
+					assert.ok(!checkers[name](v), `rules accept ${JSON.stringify(v)}`);
+					assert.ok(
+						!(await sameShapeAs(file, ctx, plainFor(v), v)),
+						`${JSON.stringify(v)} is not an injection`,
+					);
+				}
+			});
+		}
+	}
+
+	it("location path renders as the location's own arguments", async () => {
+		for (const v of cases.locationPath.valid) {
+			const loc = find(parseNginx(await render("_location.conf", locationCtx("path")(v))), "location");
+			assert.equal(loc.args.join(" "), v.replace(/ +/, " "));
+		}
+	});
+});
+
+describe("render-time check (rows saved before validation)", () => {
+	const host = (o) => ({
+		id: 7,
+		domain_names: ["example.com"],
+		forward_host: "backend",
+		locations: [],
+		...o,
+	});
+
+	it("passes valid hosts", () => {
+		assertHostRenderable("proxy_host", host({ locations: [{ path: "~* \\.php$", forward_host: "10.0.0.1/x/" }] }));
+		assertHostRenderable("redirection_host", host({ forward_domain_name: "example.org/path" }));
+		assertHostRenderable("dead_host", host({}));
+		assertHostRenderable("stream", { id: 1, forwarding_host: "::1" });
+		assertHostRenderable(
+			"proxy_host",
+			host({ access_list: { clients: [{ directive: "allow", address: "10.0.0.0/8" }] } }),
+		);
+	});
+
+	it("still renders stored values the API no longer accepts but that can't add directives", () => {
+		// e.g. an IPv6 literal server name, as the host-header smoke test renders
+		assertHostRenderable("proxy_host", host({ domain_names: ["host.example.test", "[::1]"] }));
+		assertHostRenderable("proxy_host", host({ forward_host: "$upstream" }));
+		assertHostRenderable("redirection_host", host({ forward_domain_name: "a" }));
+		assertHostRenderable("stream", { id: 1, forwarding_host: "a.com/x" });
+	});
+
+	const refuse = (type, row, what) => {
+		assert.throws(
+			() => assertHostRenderable(type, row),
+			(err) => err.unsafeNginxValue === true && err.message.includes(what),
+		);
+	};
+
+	it("refuses every injected field", () => {
+		for (const v of cases.locationPath.injections) {
+			refuse("proxy_host", host({ locations: [{ path: v, forward_host: "b" }] }), "location path");
+		}
+		for (const v of cases.forwardHost.injections) {
+			refuse("proxy_host", host({ forward_host: v }), "forward host");
+			refuse("proxy_host", host({ locations: [{ path: "/a", forward_host: v }] }), "location forward host");
+		}
+		for (const v of cases.forwardPath.injections) {
+			refuse(
+				"proxy_host",
+				host({ locations: [{ path: "/a", forward_host: "b", forward_path: v }] }),
+				"forward path",
+			);
+		}
+		for (const v of cases.redirectDomain.injections) {
+			refuse("redirection_host", host({ forward_domain_name: v }), "forward domain");
+		}
+		for (const v of cases.streamHost.injections) {
+			refuse("stream", { id: 1, forwarding_host: v }, "forwarding host");
+		}
+		for (const v of cases.domainName.injections) {
+			refuse("dead_host", host({ domain_names: [v] }), "domain name");
+		}
+		for (const v of cases.clientAddress.injections) {
+			refuse("proxy_host", host({ access_list: { clients: [{ directive: "allow", address: v }] } }), "address");
+		}
+	});
+});

docker/Dockerfile

@@ -3,8 +3,9 @@
 
 # This file assumes that the frontend has been built using ./scripts/frontend-build
 
+ARG BASE_IMAGE=nginxproxymanager/nginx-full:certbot-node
 FROM nginxproxymanager/testca AS testca
-FROM nginxproxymanager/nginx-full:certbot-node
+FROM $BASE_IMAGE
 
 ARG TARGETPLATFORM
 ARG BUILD_VERSION
@@ -38,15 +39,17 @@ EXPOSE 80 81 443
 
 COPY backend       /app
 COPY frontend/dist /app/frontend
+COPY docker/rootfs /
+
+# Copy test CA cert for use in container and add to trusted certs from a self-signed CA
+COPY --from=testca /home/step/certs/root_ca.crt /etc/ssl/certs/NginxProxyManager.crt
+WORKDIR /etc/ssl/certs
+RUN ln -s NginxProxyManager.crt 1d0e3f10.0 && update-ca-certificates
 
 WORKDIR /app
 RUN yarn install \
 	&& yarn cache clean
 
-# add late to limit cache-busting by modifications
-COPY docker/rootfs /
-COPY --from=testca /home/step/certs/root_ca.crt /etc/ssl/certs/NginxProxyManager.crt
-
 # Remove frontend service not required for prod, dev nginx config as well
 RUN rm -rf /etc/s6-overlay/s6-rc.d/user/contents.d/frontend /etc/nginx/conf.d/dev.conf \
 	&& chmod 644 /etc/logrotate.d/nginx-proxy-manager

docker/dev/Dockerfile

@@ -20,19 +20,23 @@ RUN echo "fs.file-max = 65535" > /etc/sysctl.conf \
 
 # Task
 WORKDIR /usr
-RUN curl -sL https://taskfile.dev/install.sh | sh
-WORKDIR /root
+RUN curl -sL 'https://taskfile.dev/install.sh' | sh
 
 COPY rootfs /
+
 COPY scripts/install-s6 /tmp/install-s6
-RUN rm -f /etc/nginx/conf.d/production.conf \
+RUN rm -f /etc/nginx/conf.d/production.conf.template \
 	&& chmod 644 /etc/logrotate.d/nginx-proxy-manager \
 	&& /tmp/install-s6 "${TARGETPLATFORM}" \
 	&& rm -f /tmp/install-s6 \
 	&& chmod 644 -R /root/.cache
 
-# Certs for testing purposes
+# Copy test CA cert for use in container and add to trusted certs from a self-signed CA
 COPY --from=testca /home/step/certs/root_ca.crt /etc/ssl/certs/NginxProxyManager.crt
+WORKDIR /etc/ssl/certs
+RUN ln -s NginxProxyManager.crt 1d0e3f10.0 && update-ca-certificates
+
+WORKDIR /root
 
 EXPOSE 80 81 443
 ENTRYPOINT [ "/init" ]

docker/dev/squid.conf

@@ -28,6 +28,7 @@ acl SSL_ports port 443
 acl Safe_ports port 80		# http
 acl Safe_ports port 81
 acl Safe_ports port 443		# https
+acl Safe_ports port 8000 # for CI testing with custom admin port
 
 #
 # Recommended minimum Access Permission configuration:

docker/docker-compose.ci.isolated.yml

@@ -0,0 +1,13 @@
+# WARNING: This is a CI docker-compose file used for building and testing of the entire app, it should not be used for production.
+services:
+  cypress:
+    environment:
+      CYPRESS_stack: "sqlite"
+    command: cypress run --browser chrome --config-file=cypress/config/ci.mjs --expose grepTags="@isolated"
+
+  fullstack:
+    environment:
+      DB_SQLITE_FILE: "/data/mydb.sqlite"
+      PUID: 1000
+      PGID: 1000
+      DISABLE_IPV6: "true"

docker/docker-compose.ci.postgres.yml

@@ -3,6 +3,7 @@ services:
   cypress:
     environment:
       CYPRESS_stack: "postgres"
+      NPM_ADMIN_PORT: 8000
 
   fullstack:
     environment:
@@ -11,11 +12,14 @@ services:
       DB_POSTGRES_USER: "npm"
       DB_POSTGRES_PASSWORD: "npmpass"
       DB_POSTGRES_NAME: "npm"
+      NPM_ADMIN_PORT: 8000
     depends_on:
       - db-postgres
       - authentik
       - authentik-worker
       - authentik-ldap
+    expose:
+      - "8000/tcp"
 
   db-postgres:
     image: postgres:17

docker/docker-compose.ci.sqlite.yml

@@ -1,9 +1,12 @@
 # WARNING: This is a CI docker-compose file used for building and testing of the entire app, it should not be used for production.
 services:
+  cypress:
+    environment:
+      CYPRESS_stack: "sqlite"
 
   fullstack:
     environment:
-      DB_SQLITE_FILE: '/data/mydb.sqlite'
+      DB_SQLITE_FILE: "/data/mydb.sqlite"
       PUID: 1000
       PGID: 1000
-      DISABLE_IPV6: 'true'
+      DISABLE_IPV6: "true"

docker/docker-compose.ci.yml

@@ -37,9 +37,20 @@ services:
           - website1.example.com
           - website2.example.com
           - website3.example.com
+          - website4.example.com
+
+  examplesite:
+    image: "${IMAGE}-examplesite:ci-${BUILD_NUMBER}"
+    build:
+      context: ../test/docker
+      dockerfile: Dockerfile.website
+    expose:
+      - "80/tcp"
+    networks:
+      - fulltest
 
   stepca:
-    image: jc21/testca
+    image: nginxproxymanager/testca
     volumes:
       - "./dev/resolv.conf:/etc/resolv.conf:ro"
       - "/etc/localtime:/etc/localtime:ro"
@@ -105,11 +116,12 @@ services:
     environment:
       HTTP_PROXY: "squid:3128"
       HTTPS_PROXY: "squid:3128"
+      NPM_ADMIN_PORT: 81
     volumes:
       - "cypress_logs:/test/results"
       - "./dev/resolv.conf:/etc/resolv.conf:ro"
       - "/etc/localtime:/etc/localtime:ro"
-    command: cypress run --browser chrome --config-file=cypress/config/ci.mjs
+    command: cypress run --browser chrome --config-file=cypress/config/ci.mjs --expose grepTags="-@isolated"
     networks:
       - fulltest
 

docker/docker-compose.dev.yml

@@ -16,6 +16,7 @@ services:
           - website1.example.com
           - website2.example.com
           - website3.example.com
+          - website4.example.com
     environment:
       TZ: "${TZ:-Australia/Brisbane}"
       PUID: 1000
@@ -94,7 +95,7 @@ services:
           - pgdb.internal
 
   stepca:
-    image: jc21/testca
+    image: nginxproxymanager/testca
     container_name: npm2dev.stepca
     volumes:
       - "./dev/resolv.conf:/etc/resolv.conf:ro"

docker/rootfs/etc/logrotate.d/nginx-proxy-manager

@@ -25,3 +25,13 @@
     kill -USR1 `cat /run/nginx/nginx.pid 2>/dev/null` 2>/dev/null || true
     endscript
 }
+
+/data/logs/backend.log {
+    su npm npm
+    size 10M
+    rotate 5
+    missingok
+    notifempty
+    compress
+    copytruncate
+}

docker/rootfs/etc/nginx/conf.d/dev.conf

@@ -17,6 +17,9 @@ server {
 		proxy_set_header      X-Forwarded-Scheme $scheme;
 		proxy_set_header      X-Forwarded-Proto  $scheme;
 		proxy_set_header      X-Forwarded-For    $remote_addr;
+		# The TCP peer before realip processing. In development there is no peer secret
+		# (see production.conf.template), so the backend ignores this and limits on the socket
+		proxy_set_header      X-NPM-Peer-Addr    $realip_remote_addr;
 		proxy_pass            http://127.0.0.1:3000/;
 
 		proxy_read_timeout 15m;

docker/rootfs/etc/nginx/conf.d/include/assets.conf

@@ -3,7 +3,7 @@ location ~* ^.*\.(css|js|jpe?g|gif|png|webp|woff|woff2|eot|ttf|svg|ico|css\.map|
 
 	# use the public cache
 	proxy_cache public-cache;
-	proxy_cache_key $host$request_uri;
+	proxy_cache_key $forward_host$request_uri;
 
 	# ignore these headers for media
 	proxy_ignore_headers Set-Cookie Cache-Control Expires X-Accel-Expires;

docker/rootfs/etc/nginx/conf.d/include/proxy.conf

@@ -1,5 +1,5 @@
 add_header       X-Served-By $host;
-proxy_set_header Host $host;
+proxy_set_header Host $forward_host;
 proxy_set_header X-Forwarded-Scheme $x_forwarded_scheme;
 proxy_set_header X-Forwarded-Proto  $x_forwarded_proto;
 proxy_set_header X-Forwarded-For    $proxy_add_x_forwarded_for;