polyunfill: remove unix_chkpwd from /run/wrappers

note that it doesnt show in `webm` exports

README.md

@@ -1,3 +1,7 @@
+![hello](doc/hello.gif)
+
+# .❄️≡We|_c0m3 7o m`/ f14k≡❄️.
+
 ## What's Here
 
 this is the top-level repo from which i configure/deploy all my NixOS machines:
@@ -6,18 +10,19 @@ this is the top-level repo from which i configure/deploy all my NixOS machines:
 - server
 - mobile phone (Pinephone)
 
-everything outside of <./hosts/> and <./secrets/> is intended for export, to be importable for use by 3rd parties.
+everything outside of [hosts/](./hosts/) and [secrets/](./secrets/) is intended for export, to be importable for use by 3rd parties.
 the only hard dependency for my exported pkgs/modules should be [nixpkgs][nixpkgs].
-building <./hosts/> will require [sops][sops].
+building [hosts/](./hosts/) will require [sops][sops].
 
 you might specifically be interested in these files (elaborated further in #key-points-of-interest):
-- [`sxmo-utils`](./pkgs/additional/sxmo-utils/default.nix)
-  - [example SXMO deployment](./hosts/modules/gui/sxmo/default.nix)
+- ~~[`sxmo-utils`](./pkgs/additional/sxmo-utils/default.nix)~~
+  - ~~[example SXMO deployment](./hosts/modules/gui/sxmo/default.nix)~~
+  - these files will remain until my config settles down, but i no longer use or maintain SXMO.
 - [my implementation of impermanence](./modules/persist/default.nix)
 - my way of deploying dotfiles/configuring programs per-user:
-  - <./modules/fs/default.nix>
-  - <./modules/programs.nix>
-  - <./modules/users.nix>
+  - [modules/fs/](./modules/fs/default.nix)
+  - [modules/programs/](./modules/programs/default.nix)
+  - [modules/users/](./modules/users/default.nix)
 
 [nixpkgs]: https://github.com/NixOS/nixpkgs
 [sops]: https://github.com/Mic92/sops-nix
@@ -35,37 +40,37 @@ or follow the instructions [here][NUR] to use it via the Nix User Repositories.
 
 ## Layout
 - `doc/`
-    - instructions for tasks i find myself doing semi-occasionally in this repo.
+  - instructions for tasks i find myself doing semi-occasionally in this repo.
 - `hosts/`
-    - the bulk of config which isn't factored with external use in mind.
-    - that is, if you were to add this repo to a flake.nix for your own use,
-      you won't likely be depending on anything in this directory.
+  - the bulk of config which isn't factored with external use in mind.
+  - that is, if you were to add this repo to a flake.nix for your own use,
+    you won't likely be depending on anything in this directory.
 - `integrations/`
-    - code intended for consumption by external tools (e.g. the Nix User Repos)
+  - code intended for consumption by external tools (e.g. the Nix User Repos)
 - `modules/`
-    - config which is gated behind `enable` flags, in similar style to nixpkgs'
-      `nixos/` directory.
-    - if you depend on this repo, it's most likely for something in this directory.
+  - config which is gated behind `enable` flags, in similar style to nixpkgs'
+    `nixos/` directory.
+  - if you depend on this repo, it's most likely for something in this directory.
 - `nixpatches/`
-    - literally, diffs i apply atop upstream nixpkgs before performing further eval.
+  - literally, diffs i apply atop upstream nixpkgs before performing further eval.
 - `overlays/`
-    - exposed via the `overlays` output in `flake.nix`.
-    - predominantly a list of `callPackage` directives.
+  - exposed via the `overlays` output in `flake.nix`.
+  - predominantly a list of `callPackage` directives.
 - `pkgs/`
-    - derivations for things not yet packaged in nixpkgs.
-    - derivations for things from nixpkgs which i need to `override` for some reason.
-    - inline code for wholly custom packages (e.g. `pkgs/additional/sane-scripts/` for CLI tools
-      that are highly specific to my setup).
+  - derivations for things not yet packaged in nixpkgs.
+  - derivations for things from nixpkgs which i need to `override` for some reason.
+  - inline code for wholly custom packages (e.g. `pkgs/additional/sane-scripts/` for CLI tools
+    that are highly specific to my setup).
 - `scripts/`
-    - scripts which aren't reachable on a deployed system, but may aid manual deployments
+  - scripts which aren't reachable on a deployed system, but may aid manual deployments
 - `secrets/`
-    - encrypted keys, API tokens, anything which one or more of my machines needs
-      read access to but shouldn't be world-readable.
-    - not much to see here
+  - encrypted keys, API tokens, anything which one or more of my machines needs
+    read access to but shouldn't be world-readable.
+  - not much to see here
 - `templates/`
-    - exposed via the `templates` output in `flake.nix`.
-    - used to instantiate short-lived environments.
-    - used to auto-fill the boiler-plate portions of new packages.
+  - exposed via the `templates` output in `flake.nix`.
+  - used to instantiate short-lived environments.
+  - used to auto-fill the boiler-plate portions of new packages.
 
 
 ## Key Points of Interest
@@ -73,35 +78,41 @@ or follow the instructions [here][NUR] to use it via the Nix User Repositories.
 i.e. you might find value in using these in your own config:
 
 - `modules/fs/`
-    - use this to statically define leafs and nodes anywhere in the filesystem,
-      not just inside `/nix/store`.
-    - e.g. specify that `/var/www` should be:
-        - owned by a specific user/group
-        - set to a specific mode
-        - symlinked to some other path
-        - populated with some statically-defined data
-        - populated according to some script
-        - created as a dependency of some service (e.g. `nginx`)
-    - values defined here are applied neither at evaluation time _nor_ at activation time.
-        - rather, they become systemd services.
-        - systemd manages dependencies
-        - e.g. link `/var/www -> /mnt/my-drive/www` only _after_ `/mnt/my-drive/www` appears)
-    - this is akin to using [Home Manager's][home-manager] file API -- the part which lets you
-      statically define `~/.config` files -- just with a different philosophy.
+  - use this to statically define leafs and nodes anywhere in the filesystem,
+    not just inside `/nix/store`.
+  - e.g. specify that `/var/www` should be:
+    - owned by a specific user/group
+    - set to a specific mode
+    - symlinked to some other path
+    - populated with some statically-defined data
+    - populated according to some script
+    - created as a dependency of some service (e.g. `nginx`)
+  - values defined here are applied neither at evaluation time _nor_ at activation time.
+    - rather, they become systemd services.
+    - systemd manages dependencies
+    - e.g. link `/var/www -> /mnt/my-drive/www` only _after_ `/mnt/my-drive/www` appears)
+  - this is akin to using [Home Manager's][home-manager] file API -- the part which lets you
+    statically define `~/.config` files -- just with a different philosophy.
 - `modules/persist/`
-    - my alternative to the Impermanence module.
-    - this builds atop `modules/fs/` to achieve things stock impermanence can't:
-        - persist things to encrypted storage which is unlocked at login time (pam_mount).
-        - "persist" cache directories -- to free up RAM -- but auto-wipe them on mount
-          and encrypt them to ephemeral keys so they're unreadable post shutdown/unmount.
-- `modules/programs.nix`
-    - like nixpkgs' `programs` options, but allows both system-wide or per-user deployment.
-    - allows `fs` and `persist` config values to be gated behind program deployment:
-        - e.g. `/home/<user>/.mozilla/firefox` is persisted only for users who
-          `sane.programs.firefox.enableFor.user."<user>" = true;`
-- `modules/users.nix`
-    - convenience layer atop the above modules so that you can just write
-      `fs.".config/git"` instead of `fs."/home/colin/.config/git"`
+  - my alternative to the Impermanence module.
+  - this builds atop `modules/fs/` to achieve things stock impermanence can't:
+    - persist things to encrypted storage which is unlocked at login time (pam_mount).
+    - "persist" cache directories -- to free up RAM -- but auto-wipe them on mount
+      and encrypt them to ephemeral keys so they're unreadable post shutdown/unmount.
+- `modules/programs/`
+  - like nixpkgs' `programs` options, but allows both system-wide or per-user deployment.
+  - allows `fs` and `persist` config values to be gated behind program deployment:
+    - e.g. `/home/<user>/.mozilla/firefox` is persisted only for users who
+      `sane.programs.firefox.enableFor.user."<user>" = true;`
+  - allows aggressive sandboxing any program:
+    - `sane.programs.firefox.sandbox.method = "bwrap";  # sandbox with bubblewrap`
+    - `sane.programs.firefox.sandbox.whitelistWayland = true;  # allow it to render a wayland window`
+    - `sane.programs.firefox.sandbox.extraHomePaths = [ "Downloads" ];  # allow it read/write access to ~/Downloads`
+    - integrated with `fs` and `persist` modules so that programs' config files and persisted data stores are linked into the sandbox w/o any extra involvement.
+- `modules/users/`
+  - convenience layer atop the above modules so that you can just write
+    `fs.".config/git"` instead of `fs."/home/colin/.config/git"`
+  - per-user services managed by [s6-rc](https://www.skarnet.org/software/s6-rc/)
 
 some things in here could easily find broader use. if you would find benefit in
 them being factored out of my config, message me and we could work to make that happen.

TODO.md

@@ -1,57 +1,93 @@
 ## BUGS
-- nixpkgs date is incorrect (1970.01.01...)
+- moby: megapixels doesn't load in sandbox
+- when moby wlan is explicitly set down (via ip link set wlan0 down), /var/lib/trust-dns/dhcp-configs doesn't get reset
+  - `ip monitor` can detect those manual link state changes (NM-dispatcher it seems cannot)
+  - or try dnsmasq?
+- trust-dns: can't recursively resolve api.mangadex.org
+  - and *sometimes* apple.com fails
+- wg-ovpnd-* interfaces don't work, because i use the same keys across all hosts...
+  - and if i had them differ and simultaneously online, then i'd exceed the OVPN machine count.
+  - i should at least have them be up'd only on-demand.
+- sandbox: link cache means that if i update ~/.config/... files inline, sandboxed programs still see the old version
+- mpv: audiocast has mpv sending its output to the builtin speakers unless manually changed
+- mpv: no way to exit fullscreen video on moby
+  - uosc hides controls on FS, and touch doesn't support unhiding
+- Signal restart loop drains battery
+  - decrease s6 restart time?
+- `ssh` access doesn't grant same linux capabilities as login
 - ringer (i.e. dino incoming call) doesn't prevent moby from sleeping
-- `nix` operations from lappy hang when `desko` is unreachable
-  - could at least direct the cache to `http://desko-hn:5001`
+- sway mouse/kb hotplug doesn't work
+- sysvol (volume overlay): when casting with `blast`, sysvol doesn't react to volume changes
+- moby: kaslr is effectively disabled
+  - `dmesg | grep "KASLR disabled due to lack of seed"`
+  - fix by adding `kaslrseed` to uboot script before `booti`
+    - <https://github.com/armbian/build/pull/4352>
+    - not sure how that's supposed to work with tow-boot; maybe i should just update tow-boot
+- moby: bpf is effectively disabled?
+  - `dmesg | grep 'systemd[1]: bpf-lsm: Failed to load BPF object: No such process'`
+  - `dmesg | grep 'hid_bpf: error while preloading HID BPF dispatcher: -22'`
 
 ## REFACTORING:
-
+- REMOVE DEPRECATED `crypt` from sftpgo_auth_hook
+- consolidate ~/dev and ~/ref
+  - ~/dev becomes a link to ~/ref/cat/mine
 - fold hosts/common/home/ssh.nix -> hosts/common/users/colin.nix
 
 ### sops/secrets
-- attach secrets to the thing they're used by (sane.programs)
 - rework secrets to leverage `sane.fs`
 - remove sops activation script as it's covered by my systemd sane.fs impl
+- user secrets could just use `gocryptfs`, like with ~/private?
+  - can gocryptfs support nested filesystems, each with different perms (for desko, moby, etc)?
 
 ### roles
 - allow any host to take the role of `uninsane.org`
   - will make it easier to test new services?
 
 ### upstreaming
-- split out a sxmo module usable by NUR consumers
-- bump nodejs version in lemmy-ui
 - add updateScripts to all my packages in nixpkgs
-- fix lightdm-mobile-greeter for newer libhandy
-- port zecwallet-lite to a from-source build
-- REVIEW/integrate jellyfin dataDir config: <https://github.com/NixOS/nixpkgs/pull/233617>
 
 #### upstreaming to non-nixpkgs repos
 - gtk: build schemas even on cross compilation: <https://github.com/NixOS/nixpkgs/pull/247844>
 
 
 ## IMPROVEMENTS:
+- systemd/journalctl: use a less shit pager
+  - there's an env var for it: SYSTEMD_PAGER? and a flag for journalctl
+
 ### security/resilience
 - validate duplicity backups!
 - encrypt more ~ dirs (~/archives, ~/records, ..?)
   - best to do this after i know for sure i have good backups
-- have `sane.programs` be wrapped such that they run in a cgroup?
-  - at least, only give them access to the portion of the fs they *need*.
-  - Android takes approach of giving each app its own user: could hack that in here.
-  - **systemd-run** takes a command and runs it in a temporary scope (cgroup)
-    - presumably uses the same options as systemd services
-    - see e.g. <https://github.com/NixOS/nixpkgs/issues/113903#issuecomment-857296349>
-  - flatpak does this, somehow
-  - apparmor?  SElinux?  (desktop) "portals"?
-  - see Spectrum OS; Alyssa Ross; etc
-  - bubblewrap-based sandboxing: <https://github.com/nixpak/nixpak>
+- /mnt/desko/home, etc, shouldn't include secrets (~/private)
+  - 95% of its use is for remote media access and stuff which isn't in VCS (~/records)
+- port all sane.programs to be sandboxed
+  - enforce that all `environment.packages` has a sandbox profile (or explicitly opts out)
+  - revisit "non-sandboxable" apps and check that i'm not actually just missing mountpoints
+    - LL_FS_RW=/ isn't enough -- need all mount points like `=/:/proc:/sys:...`.
+  - ensure non-bin package outputs are linked for sandboxed apps
+    - i.e. `outputs.man`, `outputs.debug`, `outputs.doc`, ...
+  - lock down dbus calls within the sandbox
+    - otherwise anyone can `systemd-run --user ...` to potentially escape a sandbox
+    - <https://github.com/flatpak/xdg-dbus-proxy>
+  - remove `.ssh` access from Firefox!
+    - limit access to `~/knowledge/secrets` through an agent that requires GUI approval, so a firefox exploit can't steal all my logins
+  - port sanebox to a compiled language (hare?)
+    - it adds like 50-70ms launch time _on my laptop_. i'd hate to know how much that is on the pinephone.
+  - remove /run/wrappers from the sandbox path
+    - they're mostly useless when using no-new-privs, just an opportunity to forget to specify deps
+- make dconf stuff less monolithic
+  - i.e. per-app dconf profiles for those which need it. possible static config.
 - canaries for important services
   - e.g. daily email checks; daily backup checks
   - integrate `nix check` into Gitea actions?
 
-### faster/better deployments
-- remove audacity's dependency on webkitgtk (via wxwidgets)
-
 ### user experience
+- rofi: sort items case-insensitively
+- xdg-desktop-portal shouldn't kill children on exit
+  - *maybe* a job for `setsid -f`?
+- replace starship prompt with something more efficient
+  - watch `forkstat`: it does way too much
+- cleanup waybar so that it's not invoking playerctl every 2 seconds
 - install apps:
   - display QR codes for WiFi endpoints: <https://linuxphoneapps.org/apps/noappid.wisperwind.wifi2qr/>
   - shopping list (not in nixpkgs): <https://linuxphoneapps.org/apps/ro.hume.cosmin.shoppinglist/>
@@ -70,9 +106,13 @@
   - Shootin Stars  (Godot; not in nixpkgs) <https://gitlab.com/greenbeast/shootin-stars>
   - numberlink (generic name for Flow Free). not packaged in Nix
   - Neverball (https://neverball.org/screenshots.php). nix: as `neverball`
+  - blurble (https://linuxphoneapps.org/games/app.drey.blurble/). nix: not as of 2024-02-05
+  - Trivia Quiz (https://linuxphoneapps.org/games/io.github.nokse22.trivia-quiz/)
+- sane-sync-music: remove empty dirs
 
 #### moby
 - fix cpuidle (gets better power consumption): <https://xnux.eu/log/077.html>
+- moby: tune keyboard layout
 - SwayNC:
   - don't show MPRIS if no players detected
     - this is a problem of playerctld, i guess
@@ -89,12 +129,14 @@
 - moby: theme GTK apps (i.e. non-adwaita styles)
   - especially, make the menubar collapsible
   - try Gradience tool specifically for theming adwaita? <https://linuxphoneapps.org/apps/com.github.gradienceteam.gradience/>
-- phog: remove the gnome-shell runtime dependency to save hella closure size
 
 #### non-moby
+- sane-tag-music: integrate `beets`/<https://beets.io/>
+  - this should be able to auto-tag a large part of my library
 - RSS: integrate a paywall bypass
   - e.g. self-hosted [ladder](https://github.com/everywall/ladder) (like 12ft.io)
 - neovim: set up language server (lsp; rnix-lsp; nvim-lspconfig)
+- neovim: integrate LLMs
 - Helix: make copy-to-system clipboard be the default
 - firefox/librewolf: persist history
   - just not cookies or tabs
@@ -111,13 +153,12 @@
   - could change junk filter from "no DKIM success" to explicit "DKIM failed"
 
 ### perf
+- debug nixos-rebuild times
 - add `pkgs.impure-cached.<foo>` package set to build things with ccache enabled
   - every package here can be auto-generated, and marked with some env var so that it doesn't pollute the pure package set
   - would be super handy for package prototyping!
-- fix desko so it doesn't dispatch so many build jobs to servo by default
 
 ## NEW FEATURES:
 - migrate MAME cabinet to nix
   - boot it from PXE from servo?
 - enable IPv6
-- package lemonade lemmy app: <https://linuxphoneapps.org/apps/ml.mdwalters.lemonade/>

flake.lock

@@ -1,5 +1,79 @@
 {
   "nodes": {
+    "flake-compat": {
+      "locked": {
+        "lastModified": 1688025799,
+        "narHash": "sha256-ktpB4dRtnksm9F5WawoIkEneh1nrEvuxb5lJFt1iOyw=",
+        "owner": "nix-community",
+        "repo": "flake-compat",
+        "rev": "8bf105319d44f6b9f0d764efa4fdef9f1cc9ba1c",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-community",
+        "repo": "flake-compat",
+        "type": "github"
+      }
+    },
+    "flake-parts": {
+      "inputs": {
+        "nixpkgs-lib": [
+          "nixpkgs-wayland",
+          "nix-eval-jobs",
+          "nixpkgs"
+        ]
+      },
+      "locked": {
+        "lastModified": 1712014858,
+        "narHash": "sha256-sB4SWl2lX95bExY2gMFG5HIzvva5AVMJd4Igm+GpZNw=",
+        "owner": "hercules-ci",
+        "repo": "flake-parts",
+        "rev": "9126214d0a59633752a136528f5f3b9aa8565b7d",
+        "type": "github"
+      },
+      "original": {
+        "owner": "hercules-ci",
+        "repo": "flake-parts",
+        "type": "github"
+      }
+    },
+    "flake-utils": {
+      "inputs": {
+        "systems": "systems"
+      },
+      "locked": {
+        "lastModified": 1710146030,
+        "narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
+        "owner": "numtide",
+        "repo": "flake-utils",
+        "rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
+        "type": "github"
+      },
+      "original": {
+        "owner": "numtide",
+        "repo": "flake-utils",
+        "type": "github"
+      }
+    },
+    "lib-aggregate": {
+      "inputs": {
+        "flake-utils": "flake-utils",
+        "nixpkgs-lib": "nixpkgs-lib"
+      },
+      "locked": {
+        "lastModified": 1716120557,
+        "narHash": "sha256-rvNq9YolMY1DRMgwdAti8qwNDjkhTsotSWa15/Ch7+A=",
+        "owner": "nix-community",
+        "repo": "lib-aggregate",
+        "rev": "5fa64b174daa22fe0d20ebbcc0ec2c7905b503f1",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-community",
+        "repo": "lib-aggregate",
+        "type": "github"
+      }
+    },
     "mobile-nixos": {
       "flake": false,
       "locked": {
@@ -17,13 +91,87 @@
         "type": "github"
       }
     },
+    "nix-eval-jobs": {
+      "inputs": {
+        "flake-parts": "flake-parts",
+        "nix-github-actions": "nix-github-actions",
+        "nixpkgs": "nixpkgs",
+        "treefmt-nix": "treefmt-nix"
+      },
+      "locked": {
+        "lastModified": 1715804156,
+        "narHash": "sha256-GtIHP86Cz1kD9xZO/cKbNQACHKdoT9WFbLJAq6W2EDY=",
+        "owner": "nix-community",
+        "repo": "nix-eval-jobs",
+        "rev": "bb95091f6c6f38f6cfc215a1797a2dd466312c8b",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-community",
+        "repo": "nix-eval-jobs",
+        "type": "github"
+      }
+    },
+    "nix-github-actions": {
+      "inputs": {
+        "nixpkgs": [
+          "nixpkgs-wayland",
+          "nix-eval-jobs",
+          "nixpkgs"
+        ]
+      },
+      "locked": {
+        "lastModified": 1703863825,
+        "narHash": "sha256-rXwqjtwiGKJheXB43ybM8NwWB8rO2dSRrEqes0S7F5Y=",
+        "owner": "nix-community",
+        "repo": "nix-github-actions",
+        "rev": "5163432afc817cf8bd1f031418d1869e4c9d5547",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-community",
+        "repo": "nix-github-actions",
+        "type": "github"
+      }
+    },
+    "nixpkgs": {
+      "locked": {
+        "lastModified": 1715037484,
+        "narHash": "sha256-OUt8xQFmBU96Hmm4T9tOWTu4oCswCzoVl+pxSq/kiFc=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "ad7efee13e0d216bf29992311536fce1d3eefbef",
+        "type": "github"
+      },
+      "original": {
+        "owner": "NixOS",
+        "ref": "nixpkgs-unstable",
+        "repo": "nixpkgs",
+        "type": "github"
+      }
+    },
+    "nixpkgs-lib": {
+      "locked": {
+        "lastModified": 1716079763,
+        "narHash": "sha256-DGRfb7fO7c3XDS3twmuaV5NAGPPdU3W7Q35fjIZc8iY=",
+        "owner": "nix-community",
+        "repo": "nixpkgs.lib",
+        "rev": "0df131b5ee4d928a4b664b6d0cd99cf134d6ab6b",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-community",
+        "repo": "nixpkgs.lib",
+        "type": "github"
+      }
+    },
     "nixpkgs-next-unpatched": {
       "locked": {
-        "lastModified": 1705233677,
-        "narHash": "sha256-eq3VE8QGJsunqqF/BlLslWE1gASp4Hlgp0c78coxat0=",
+        "lastModified": 1716573668,
+        "narHash": "sha256-K+8vYJtu7X+qQ1Q66kYTheq0IKKVyPMOMclY9oJYnS8=",
         "owner": "nixos",
         "repo": "nixpkgs",
-        "rev": "724e39ebb9b8eda97f17d423f66fbc5a991f4f8d",
+        "rev": "cc5c0d369b5e8f49705e2a2d7464e4b162804805",
         "type": "github"
       },
       "original": {
@@ -35,27 +183,27 @@
     },
     "nixpkgs-stable": {
       "locked": {
-        "lastModified": 1705033721,
-        "narHash": "sha256-K5eJHmL1/kev6WuqyqqbS1cdNnSidIZ3jeqJ7GbrYnQ=",
+        "lastModified": 1716061101,
+        "narHash": "sha256-H0eCta7ahEgloGIwE/ihkyGstOGu+kQwAiHvwVoXaA0=",
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "a1982c92d8980a0114372973cbdfe0a307f1bdea",
+        "rev": "e7cc61784ddf51c81487637b3031a6dd2d6673a2",
         "type": "github"
       },
       "original": {
         "owner": "NixOS",
-        "ref": "release-23.05",
+        "ref": "release-23.11",
         "repo": "nixpkgs",
         "type": "github"
       }
     },
     "nixpkgs-unpatched": {
       "locked": {
-        "lastModified": 1705254014,
-        "narHash": "sha256-4RrVNEqxeji4vqDgzSl7JoCD6a0ag5LF9zXFndtqrpE=",
+        "lastModified": 1716592782,
+        "narHash": "sha256-aGafFFAkQwkiBXo+ocx4sCco+L233JqlUZhVfXceaQY=",
         "owner": "nixos",
         "repo": "nixpkgs",
-        "rev": "6c08fe3ccf437d8b26bec010fd925ddd6bb0d0d5",
+        "rev": "2baa58d3488bd9cc4d53d6812509edc34a1c7e2a",
         "type": "github"
       },
       "original": {
@@ -65,11 +213,35 @@
         "type": "github"
       }
     },
+    "nixpkgs-wayland": {
+      "inputs": {
+        "flake-compat": "flake-compat",
+        "lib-aggregate": "lib-aggregate",
+        "nix-eval-jobs": "nix-eval-jobs",
+        "nixpkgs": [
+          "nixpkgs-unpatched"
+        ]
+      },
+      "locked": {
+        "lastModified": 1716589021,
+        "narHash": "sha256-m7+ogvgw6tCk5P1H9OqS2yTFlZOus9I+Genv3PUWkno=",
+        "owner": "nix-community",
+        "repo": "nixpkgs-wayland",
+        "rev": "8746004cd97164c89f0997ea06642b819e5bc3fb",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-community",
+        "repo": "nixpkgs-wayland",
+        "type": "github"
+      }
+    },
     "root": {
       "inputs": {
         "mobile-nixos": "mobile-nixos",
         "nixpkgs-next-unpatched": "nixpkgs-next-unpatched",
         "nixpkgs-unpatched": "nixpkgs-unpatched",
+        "nixpkgs-wayland": "nixpkgs-wayland",
         "sops-nix": "sops-nix",
         "uninsane-dot-org": "uninsane-dot-org"
       }
@@ -82,11 +254,11 @@
         "nixpkgs-stable": "nixpkgs-stable"
       },
       "locked": {
-        "lastModified": 1705201153,
-        "narHash": "sha256-y0/a4IMDZrc7lAkR7Gcm5R3W2iCBiARHnYZe6vkmiNE=",
+        "lastModified": 1716400300,
+        "narHash": "sha256-0lMkIk9h3AzOHs1dCL9RXvvN4PM8VBKb+cyGsqOKa4c=",
         "owner": "Mic92",
         "repo": "sops-nix",
-        "rev": "70dd0d521f7849338e487a219c1a07c429a66d77",
+        "rev": "b549832718b8946e875c016a4785d204fcfc2e53",
         "type": "github"
       },
       "original": {
@@ -95,6 +267,43 @@
         "type": "github"
       }
     },
+    "systems": {
+      "locked": {
+        "lastModified": 1681028828,
+        "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
+        "owner": "nix-systems",
+        "repo": "default",
+        "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-systems",
+        "repo": "default",
+        "type": "github"
+      }
+    },
+    "treefmt-nix": {
+      "inputs": {
+        "nixpkgs": [
+          "nixpkgs-wayland",
+          "nix-eval-jobs",
+          "nixpkgs"
+        ]
+      },
+      "locked": {
+        "lastModified": 1711963903,
+        "narHash": "sha256-N3QDhoaX+paWXHbEXZapqd1r95mdshxToGowtjtYkGI=",
+        "owner": "numtide",
+        "repo": "treefmt-nix",
+        "rev": "49dc4a92b02b8e68798abd99184f228243b6e3ac",
+        "type": "github"
+      },
+      "original": {
+        "owner": "numtide",
+        "repo": "treefmt-nix",
+        "type": "github"
+      }
+    },
     "uninsane-dot-org": {
       "inputs": {
         "nixpkgs": [
@@ -102,11 +311,11 @@
         ]
       },
       "locked": {
-        "lastModified": 1704082841,
-        "narHash": "sha256-4g3lePnUALb8B1m3rEDD6rrZAq2pTN4qaSnStbd676U=",
+        "lastModified": 1715894399,
+        "narHash": "sha256-h1EdA/h74zgNPNEYbH+0mgOMlJgLVcxuZ8/ewsZlgEc=",
         "ref": "refs/heads/master",
-        "rev": "4a1fa488e64e6c87c6c951e3fafb2684692f64d3",
-        "revCount": 234,
+        "rev": "e6f88f563bdd1700c04018951de4f69862646dd1",
+        "revCount": 240,
         "type": "git",
         "url": "https://git.uninsane.org/colin/uninsane"
       },

flake.nix

@@ -48,6 +48,11 @@
     # nixpkgs-unpatched.url = "github:nixos/nixpkgs?ref=nixos-staging-next";
     nixpkgs-next-unpatched.url = "github:nixos/nixpkgs?ref=staging-next";
 
+    nixpkgs-wayland = {
+      url = "github:nix-community/nixpkgs-wayland";
+      inputs.nixpkgs.follows = "nixpkgs-unpatched";
+    };
+
     mobile-nixos = {
       # <https://github.com/nixos/mobile-nixos>
       # only used for building disk images, not relevant after deployment
@@ -76,6 +81,7 @@
     self,
     nixpkgs-unpatched,
     nixpkgs-next-unpatched ? nixpkgs-unpatched,
+    nixpkgs-wayland,
     mobile-nixos,
     sops-nix,
     uninsane-dot-org,
@@ -97,38 +103,27 @@
       patchNixpkgs = variant: nixpkgs: (import ./nixpatches/flake.nix).outputs {
         inherit variant nixpkgs;
         self = patchNixpkgs variant nixpkgs;
-      } // {
-        # provide values that nixpkgs ordinarily sources from the flake.lock file,
-        # inaccessible to it here because of the import-from-derivation.
-        # rev and shortRev seem to not always exist (e.g. if the working tree is dirty),
-        # so those are made conditional.
-        #
-        # these values impact the name of a produced nixos system. having date/rev in the
-        # `readlink /run/current-system` store path helps debuggability.
-        inherit (self) lastModifiedDate lastModified;
-      } // optionalAttrs (self ? rev) {
-        inherit (self) rev;
-      } // optionalAttrs (self ? shortRev) {
-        inherit (self) shortRev;
       };
 
       nixpkgs' = patchNixpkgs "master" nixpkgs-unpatched;
       nixpkgsCompiledBy = system: nixpkgs'.legacyPackages."${system}";
 
-      evalHost = { name, local, target, light ? false, nixpkgs ? nixpkgs' }: nixpkgs.lib.nixosSystem {
+      evalHost = { name, local, target, variant ? null, nixpkgs ? nixpkgs' }: nixpkgs.lib.nixosSystem {
         system = target;
         modules = [
           {
             nixpkgs.buildPlatform.system = local;
-            # nixpkgs.config.replaceStdenv = { pkgs }: pkgs.ccacheStdenv;
           }
           (optionalAttrs (local != target) {
             # XXX(2023/12/11): cache.nixos.org uses `system = ...` instead of `hostPlatform.system`, and that choice impacts the closure of every package.
             # so avoid specifying hostPlatform.system on non-cross builds, so i can use upstream caches.
             nixpkgs.hostPlatform.system = target;
           })
-          (optionalAttrs light {
-            sane.enableSlowPrograms = false;
+          (optionalAttrs (variant == "light") {
+            sane.maxBuildCost = 2;
+          })
+          (optionalAttrs (variant == "min") {
+            sane.maxBuildCost = 0;
           })
           (import ./hosts/instantiate.nix { hostName = name; })
           self.nixosModules.default
@@ -146,11 +141,13 @@
         hosts = {
           servo       = { name = "servo";  local = "x86_64-linux"; target = "x86_64-linux";  };
           desko       = { name = "desko";  local = "x86_64-linux"; target = "x86_64-linux";  };
-          desko-light = { name = "desko";  local = "x86_64-linux"; target = "x86_64-linux";  light = true; };
+          desko-light = { name = "desko";  local = "x86_64-linux"; target = "x86_64-linux";  variant = "light"; };
           lappy       = { name = "lappy";  local = "x86_64-linux"; target = "x86_64-linux";  };
-          lappy-light = { name = "lappy";  local = "x86_64-linux"; target = "x86_64-linux";  light = true; };
+          lappy-light = { name = "lappy";  local = "x86_64-linux"; target = "x86_64-linux";  variant = "light"; };
+          lappy-min =   { name = "lappy";  local = "x86_64-linux"; target = "x86_64-linux";  variant = "min"; };
           moby        = { name = "moby";   local = "x86_64-linux"; target = "aarch64-linux"; };
-          moby-light  = { name = "moby";   local = "x86_64-linux"; target = "aarch64-linux"; light = true; };
+          moby-light  = { name = "moby";   local = "x86_64-linux"; target = "aarch64-linux"; variant = "light"; };
+          moby-min  =   { name = "moby";   local = "x86_64-linux"; target = "aarch64-linux"; variant = "min"; };
           rescue      = { name = "rescue"; local = "x86_64-linux"; target = "x86_64-linux";  };
         };
         hostsNext = mapAttrs' (h: v: {
@@ -190,18 +187,25 @@
         #   hence the weird redundancy.
         default = final: prev: self.overlays.pkgs final prev;
         sane-all = final: prev: import ./overlays/all.nix final prev;
-        disable-flakey-tests = final: prev: import ./overlays/disable-flakey-tests.nix final prev;
         pkgs = final: prev: import ./overlays/pkgs.nix final prev;
         pins = final: prev: import ./overlays/pins.nix final prev;
         preferences = final: prev: import ./overlays/preferences.nix final prev;
-        optimizations = final: prev: import ./overlays/optimizations.nix final prev;
         passthru = final: prev:
           let
             mobile = (import "${mobile-nixos}/overlay/overlay.nix");
             uninsane = uninsane-dot-org.overlays.default;
+            wayland = final: prev: {
+              # default is to dump the packages into `waylandPkgs` *and* the toplevel.
+              # but i just want the `waylandPkgs` set
+              inherit (nixpkgs-wayland.overlays.default final prev)
+                waylandPkgs
+                new-wayland-protocols  #< 2024/03/10: nixpkgs-wayland assumes this will be in the toplevel
+              ;
+            };
           in
             (mobile final prev)
             // (uninsane final prev)
+            // (wayland final prev)
           ;
       };
 
@@ -257,18 +261,45 @@
           pkgs = self.legacyPackages."x86_64-linux";
           sanePkgs = import ./pkgs { inherit pkgs; };
           deployScript = host: addr: action: pkgs.writeShellScript "deploy-${host}" ''
-            nix build '.#nixosConfigurations.${host}.config.system.build.toplevel' --out-link ./result-${host} "$@"
-            sudo nix sign-paths -r -k /run/secrets/nix_serve_privkey $(readlink ./result-${host})
+            set -e
 
-            # XXX: this triggers another config eval & (potentially) build.
-            # if the config changed between these invocations, the above signatures might not apply to the deployed config.
-            # let the user handle that edge case by re-running this whole command.
-            # N.B.: `--fast` option here is critical to cross-compiled deployments: without it the build machine will try to invoke the host machine's `nix` binary.
-            # TODO: solve this by replacing the nixos-build invocation with:
-            # - nix-copy-closure --to $host $result
-            # - on target: nix-env set -p /nix/var/nix/profiles/system $result
-            # - on target: $result/bin/switch-to-configuration
-            nixos-rebuild --flake '.#${host}' ${action} --target-host colin@${addr} --use-remote-sudo "$@" --fast
+            host="${host}"
+            addr="${addr}"
+            action="${if action != null then action else ""}"
+            runOnTarget() {
+              # run the command ($@) on the machine we're deploying to.
+              # if that's a remote machine, then do it via ssh, else local shell.
+              if [ -n "$addr" ]; then
+                ssh "$addr" "$@"
+              else
+                "$@"
+              fi
+            }
+
+            nix build ".#nixosConfigurations.$host.config.system.build.toplevel" --out-link "./build/result-$host" "$@"
+            storePath="$(readlink ./build/result-$host)"
+
+            # mimic `nixos-rebuild --target-host`, in effect:
+            # - nix-copy-closure ...
+            # - nix-env --set ...
+            # - switch-to-configuration <boot|dry-activate|switch|test|>
+            # avoid the actual `nixos-rebuild` for a few reasons:
+            # - fewer nix evals
+            # - more introspectability and debuggability
+            # - sandbox friendliness (especially: `git` doesn't have to be run as root)
+
+            if [ -n "$addr" ]; then
+              sudo nix store sign -r -k /run/secrets/nix_signing_key "$storePath"
+              # add more `-v` for more verbosity (up to 5).
+              # builders-use-substitutes false: optimizes so that the remote machine doesn't try to get paths from its substituters.
+              #   we already have all paths here, and the remote substitution is slow to check and SERIOUSLY flaky on moby in particular.
+              nix copy -vv --option builders-use-substitutes false --to "ssh-ng://$addr" "$storePath"
+            fi
+
+            if [ -n "$action" ]; then
+              runOnTarget sudo nix-env -p /nix/var/nix/profiles/system --set "$storePath"
+              runOnTarget sudo "$storePath/bin/switch-to-configuration" "$action"
+            fi
           '';
           deployApp = host: addr: action: {
             type = "app";
@@ -342,9 +373,15 @@
                 - `nix run '.#update.feeds'`
                   - updates metadata for all feeds
                 - `nix run '.#init-feed' <url>`
-                - `nix run '.#deploy.{desko,lappy,moby,servo}[-light][.test]' [nixos-rebuild args ...]`
+                - `nix run '.#deploy.{desko,lappy,moby,servo}[-light|-test]' [nix args ...]`
+                  - build and deploy the host
+                - `nix run '.#preDeploy.{desko,lappy,moby,servo}[-light]' [nix args ...]`
+                  - copy closures to a host, but don't activate it
+                  - or `nix run '.#preDeploy'` to target all hosts
                 - `nix run '.#check'`
                   - make sure all systems build; NUR evaluates
+                - `nix run '.#bench'`
+                  - benchmark the eval time of common targets this flake provides
 
                 specific build targets of interest:
                 - `nix build '.#imgs.rescue'`
@@ -371,12 +408,54 @@
           };
 
           deploy = {
+            desko       = deployApp "desko"       "desko" "switch";
+            desko-light = deployApp "desko-light" "desko" "switch";
             lappy       = deployApp "lappy"       "lappy" "switch";
             lappy-light = deployApp "lappy-light" "lappy" "switch";
+            lappy-min   = deployApp "lappy-min"   "lappy" "switch";
             moby        = deployApp "moby"        "moby"  "switch";
             moby-light  = deployApp "moby-light"  "moby"  "switch";
+            moby-min  =   deployApp "moby-min"    "moby"  "switch";
             moby-test   = deployApp "moby"        "moby"  "test";
             servo       = deployApp "servo"       "servo" "switch";
+
+            # like `nixos-rebuild --flake . switch`
+            self        = deployApp "$(hostname)"       "" "switch";
+            self-light  = deployApp "$(hostname)-light" "" "switch";
+            self-min    = deployApp "$(hostname)-min"   "" "switch";
+
+            type = "app";
+            program = builtins.toString (pkgs.writeShellScript "deploy-all" ''
+              nix run '.#deploy.lappy'
+              nix run '.#deploy.moby'
+              nix run '.#deploy.desko'
+              nix run '.#deploy.servo'
+            '');
+          };
+          preDeploy = {
+            # build the host and copy the runtime closure to that host, but don't activate it.
+            desko       = deployApp "desko"       "desko" null;
+            desko-light = deployApp "desko-light" "desko" null;
+            lappy       = deployApp "lappy"       "lappy" null;
+            lappy-light = deployApp "lappy-light" "lappy" null;
+            lappy-min   = deployApp "lappy-min"   "lappy" null;
+            moby        = deployApp "moby"        "moby"  null;
+            moby-light  = deployApp "moby-light"  "moby"  null;
+            moby-min    = deployApp "moby-min"    "moby"  null;
+            servo       = deployApp "servo"       "servo" null;
+            type = "app";
+            program = builtins.toString (pkgs.writeShellScript "predeploy-all" ''
+              # copy the -min/-light variants first; this might be run while waiting on a full build. or the full build failed.
+              nix run '.#preDeploy.moby-min' -- "$@"
+              nix run '.#preDeploy.lappy-min' -- "$@"
+              nix run '.#preDeploy.moby-light' -- "$@"
+              nix run '.#preDeploy.lappy-light' -- "$@"
+              nix run '.#preDeploy.desko-light' -- "$@"
+              nix run '.#preDeploy.lappy' -- "$@"
+              nix run '.#preDeploy.servo' -- "$@"
+              nix run '.#preDeploy.moby' -- "$@"
+              nix run '.#preDeploy.desko' -- "$@"
+            '');
           };
 
           sync = {
@@ -397,8 +476,8 @@
             # can run this from any device that has ssh access to desko and servo
             type = "app";
             program = builtins.toString (pkgs.writeShellScript "sync-to-desko" ''
-              sudo mount /mnt/desko-home
-              ${pkgs.sane-scripts.sync-music}/bin/sane-sync-music --compat /mnt/servo-media/Music /mnt/desko-home/Music "$@"
+              sudo mount /mnt/desko/home
+              ${pkgs.sane-scripts.sync-music}/bin/sane-sync-music --compat /mnt/servo/media/Music /mnt/desko/home/Music "$@"
             '');
           };
 
@@ -407,8 +486,8 @@
             # can run this from any device that has ssh access to lappy and servo
             type = "app";
             program = builtins.toString (pkgs.writeShellScript "sync-to-lappy" ''
-              sudo mount /mnt/lappy-home
-              ${pkgs.sane-scripts.sync-music}/bin/sane-sync-music --compress --compat /mnt/servo-media/Music /mnt/lappy-home/Music "$@"
+              sudo mount /mnt/lappy/home
+              ${pkgs.sane-scripts.sync-music}/bin/sane-sync-music --compress --compat /mnt/servo/media/Music /mnt/lappy/home/Music "$@"
             '');
           };
 
@@ -417,9 +496,11 @@
             # can run this from any device that has ssh access to moby and servo
             type = "app";
             program = builtins.toString (pkgs.writeShellScript "sync-to-moby" ''
-              sudo mount /mnt/moby-home
+              sudo mount /mnt/moby/home
+              sudo mount /mnt/desko/home
+              ${pkgs.rsync}/bin/rsync -arv --exclude servo-macros /mnt/moby/home/Pictures/ /mnt/desko/home/Pictures/moby/
               # N.B.: limited by network/disk -> reduce job count to improve pause/resume behavior
-              ${pkgs.sane-scripts.sync-music}/bin/sane-sync-music --compress --compat --jobs 4 /mnt/servo-media/Music /mnt/moby-home/Music "$@"
+              ${pkgs.sane-scripts.sync-music}/bin/sane-sync-music --compress --compat --jobs 4 /mnt/servo/media/Music /mnt/moby/home/Music "$@"
             '');
           };
 
@@ -451,6 +532,7 @@
                 --option allow-import-from-derivation true \
                 --drv-path --show-trace \
                 -I nixpkgs=${nixpkgs-unpatched} \
+                -I nixpkgs-overlays=${./.}/hosts/common/nix/overlay \
                 -I ../../ \
                 | tee  # tee to prevent interactive mode
             '');
@@ -462,7 +544,7 @@
               checkHost = host: let
                 shellHost = pkgs.lib.replaceStrings [ "-" ] [ "_" ] host;
               in ''
-                nix build -v '.#nixosConfigurations.${host}.config.system.build.toplevel' --out-link ./result-${host} -j2 "$@"
+                nix build -v '.#nixosConfigurations.${host}.config.system.build.toplevel' --out-link ./build/result-${host} -j2 "$@"
                 RC_${shellHost}=$?
               '';
             in builtins.toString (pkgs.writeShellScript
@@ -470,6 +552,9 @@
               ''
                 # build minimally-usable hosts first, then their full image.
                 # this gives me a minimal image i can deploy or copy over, early.
+                ${checkHost "lappy-min"}
+                ${checkHost "moby-min"}
+
                 ${checkHost "desko-light"}
                 ${checkHost "moby-light"}
                 ${checkHost "lappy-light"}
@@ -511,7 +596,31 @@
           check.rescue = {
             type = "app";
             program = builtins.toString (pkgs.writeShellScript "check-rescue" ''
-              nix build -v '.#imgs.rescue' --out-link ./result-rescue-img -j2
+              nix build -v '.#imgs.rescue' --out-link ./build/result-rescue-img -j2
+            '');
+          };
+
+          bench = {
+            type = "app";
+            program = builtins.toString (pkgs.writeShellScript "bench" ''
+              doBench() {
+                attrPath="$1"
+                shift
+                echo -n "benchmarking eval of '$attrPath'... "
+                /run/current-system/sw/bin/time -f "%e sec" -o /dev/stdout \
+                  nix eval --no-eval-cache --quiet --raw ".#$attrPath" --apply 'result: if result != null then "" else "unexpected null"' $@ 2> /dev/null
+              }
+
+              if [ -n "$1" ]; then
+                doBench "$@"
+              else
+                doBench hostConfigs
+                doBench hostConfigs.lappy
+                doBench hostConfigs.lappy.sane.programs
+                doBench hostConfigs.lappy.sane.users.colin
+                doBench hostConfigs.lappy.sane.fs
+                doBench hostConfigs.lappy.environment.systemPackages
+              fi
             '');
           };
         };

hosts/by-name/desko/default.nix

@@ -4,18 +4,16 @@
     ./fs.nix
   ];
 
+  sane.services.trust-dns.asSystemResolver = false;  # TEMPORARY: TODO: re-enable trust-dns
+  # sane.programs.devPkgs.enableFor.user.colin = true;
   # sane.guest.enable = true;
 
-  # services.distccd.enable = true;
-  # sane.programs.distcc.enableFor.user.guest = true;
-
-  # TODO: remove emulation, but need to fix nixos-rebuild to moby for that.
-  # sane.roles.build-machine.emulation = true;
+  # don't enable wifi by default: it messes with connectivity.
+  # systemd.services.iwd.enable = false;
+  # systemd.services.wpa_supplicant.enable = false;
 
   sops.secrets.colin-passwd.neededForUsers = true;
 
-  sane.ports.openFirewall = true;  # for e.g. nix-serve
-
   sane.roles.build-machine.enable = true;
   sane.roles.client = true;
   sane.roles.dev-machine = true;
@@ -23,19 +21,16 @@
   sane.services.wg-home.enable = true;
   sane.services.wg-home.ip = config.sane.hosts.by-name."desko".wg-home.ip;
   sane.services.duplicity.enable = true;
-  sane.services.nixserve.secretKeyFile = config.sops.secrets.nix_serve_privkey.path;
 
-  sane.nixcache.substituters.desko = false;
   sane.nixcache.remote-builders.desko = false;
 
-  sane.gui.sway.enable = true;
+  sane.programs.cups.enableFor.user.colin = true;
+  sane.programs.sway.enableFor.user.colin = true;
   sane.programs.iphoneUtils.enableFor.user.colin = true;
   sane.programs.steam.enableFor.user.colin = true;
 
-  # sane.programs.devPkgs.enableFor.user.colin = true;
-
-  sane.programs.signal-desktop.config.autostart = true;
   sane.programs."gnome.geary".config.autostart = true;
+  sane.programs.signal-desktop.config.autostart = true;
 
   boot.loader.efi.canTouchEfiVariables = false;
   sane.image.extraBootFiles = [ pkgs.bootpart-uefi-x86_64 ];
@@ -43,10 +38,6 @@
   # needed to use libimobiledevice/ifuse, for iphone sync
   services.usbmuxd.enable = true;
 
-  # don't enable wifi by default: it messes with connectivity.
-  systemd.services.iwd.enable = false;
-  systemd.services.wpa_supplicant.enable = false;
-
   # default config: https://man.archlinux.org/man/snapper-configs.5
   # defaults to something like:
   #   - hourly snapshots

hosts/by-name/desko/fs.nix

@@ -6,7 +6,6 @@
   fileSystems."/tmp".options = [ "size=64G" ];
 
   fileSystems."/nix" = {
-    # device = "/dev/disk/by-uuid/0ab0770b-7734-4167-88d9-6e4e20bb2a56";
     device = "/dev/disk/by-uuid/845d85bf-761d-431b-a406-e6f20909154f";
     fsType = "btrfs";
     options = [
@@ -16,7 +15,6 @@
   };
 
   fileSystems."/boot" = {
-    # device = "/dev/disk/by-uuid/41B6-BAEF";
     device = "/dev/disk/by-uuid/5049-9AFD";
     fsType = "vfat";
   };

hosts/by-name/lappy/default.nix

@@ -2,7 +2,6 @@
 {
   imports = [
     ./fs.nix
-    ./polyfill.nix
   ];
 
   sane.roles.client = true;
@@ -12,11 +11,15 @@
   sane.services.wg-home.ip = config.sane.hosts.by-name."lappy".wg-home.ip;
 
   # sane.guest.enable = true;
-  sane.gui.sway.enable = true;
   boot.loader.efi.canTouchEfiVariables = false;
   sane.image.extraBootFiles = [ pkgs.bootpart-uefi-x86_64 ];
 
+  sane.programs.cups.enableFor.user.colin = true;
   sane.programs.stepmania.enableFor.user.colin = true;
+  sane.programs.sway.enableFor.user.colin = true;
+
+  sane.programs."gnome.geary".config.autostart = true;
+  sane.programs.signal-desktop.config.autostart = true;
 
   sops.secrets.colin-passwd.neededForUsers = true;
 

hosts/by-name/lappy/fs.nix

@@ -14,24 +14,4 @@
     device = "/dev/disk/by-uuid/BD79-D6BB";
     fsType = "vfat";
   };
-
-  # fileSystems."/nix" = {
-  #   device = "/dev/disk/by-uuid/5a7fa69c-9394-8144-a74c-6726048b129f";
-  #   fsType = "btrfs";
-  # };
-
-  # fileSystems."/boot" = {
-  #   device = "/dev/disk/by-uuid/4302-1685";
-  #   fsType = "vfat";
-  # };
-
-  # fileSystems."/" = {
-  #   device = "none";
-  #   fsType = "tmpfs";
-  #   options = [
-  #     "mode=755"
-  #     "size=1G"
-  #     "defaults"
-  #   ];
-  # };
 }

hosts/by-name/lappy/polyfill.nix

@@ -1,42 +0,0 @@
-# doesn't actually *enable* anything,
-# but sets up any modules such that if they *were* enabled, they'll act as expected.
-{ pkgs, ... }:
-{
-  sane.gui.sxmo = {
-    greeter = "greetd-sway-gtkgreet";
-    noidle = true;  #< power button requires 1s hold, which makes it impractical to be dealing with.
-    settings = {
-      # XXX: make sure the user is part of the `input` group!
-      SXMO_LISGD_INPUT_DEVICE = "/dev/input/by-id/usb-Wacom_Co._Ltd._Pen_and_multitouch_sensor-event-if00";
-      # these identifiers are from `swaymsg -t get_inputs`
-      SXMO_VOLUME_BUTTON = "1:1:AT_Translated_Set_2_keyboard";
-      # SXMO_VOLUME_BUTTON = "none";
-      # N.B.: thinkpad's power button requires a full second press to do anything
-      SXMO_POWER_BUTTON = "0:1:Power_Button";
-      # SXMO_POWER_BUTTON = "none";
-      SXMO_DISABLE_LEDS = "1";
-      SXMO_UNLOCK_IDLE_TIME = "120";  # default
-      # sxmo tries to determine device type from /proc/device-tree/compatible,
-      # but that doesn't seem to exist on NixOS?  (or maybe it just doesn't exist
-      # on non-aarch64 builds).
-      # the device type informs (at least):
-      # - SXMO_WIFI_MODULE
-      # - SXMO_RTW_SCAN_INTERVAL
-      # - SXMO_TOUCHSCREEN_ID
-      # - SXMO_MONITOR
-      # - SXMO_ALSA_CONTROL_NAME
-      # - SXMO_SWAY_SCALE
-      # see <repo:mil/sxmo-utils:scripts/deviceprofiles>
-      # SXMO_DEVICE_NAME = "pine64,pinephone-1.2";
-      # if sxmo doesn't know the device, it can't decide whether to use one_button or three_button mode
-      #   and so it just wouldn't handle any button inputs (sxmo_hook_inputhandler.sh not on path)
-      SXMO_DEVICE_NAME = "three_button_touchscreen";
-    };
-    package = (pkgs.sxmo-utils.override { preferSystemd = true; }).overrideAttrs (base: {
-      postPatch = (base.postPatch or "") + ''
-        # after volume-button navigation mode, restore full keyboard functionality
-        cp ${./xkb_mobile_normal_buttons} ./configs/xkb/xkb_mobile_normal_buttons
-      '';
-    });
-  };
-}

hosts/by-name/moby/bootloader.nix

@@ -1,12 +1,22 @@
+# tow-boot: <https://tow-boot.org>
+# docs (pinephone specific): <https://github.com/Tow-Boot/Tow-Boot/tree/development/boards/pine64-pinephoneA64>
+# LED and button behavior is defined here: <https://github.com/Tow-Boot/Tow-Boot/blob/development/modules/tow-boot/phone-ux.nix>
+# - hold VOLDOWN: enter recovery mode
+#   - LED will turn aqua instead of yellow
+#   - recovery mode would ordinarily allow a selection of entries, but for pinephone i guess it doesn't do anything?
+# - hold VOLUP: force it to load the OS from eMMC?
+#   - LED will turn blue instead of yellow
+# boot LEDs:
+# - yellow = entered tow-boot
+# - 10 red flashes => poweroff means tow-boot couldn't boot into the next stage (i.e. distroboot)
+#   - distroboot: <https://source.denx.de/u-boot/u-boot/-/blob/v2022.04/doc/develop/distro.rst>)
 { config, pkgs, ... }:
 {
   # we need space in the GPT header to place tow-boot.
   # only actually need 1 MB, but better to over-allocate than under-allocate
   sane.image.extraGPTPadding = 16 * 1024 * 1024;
   sane.image.firstPartGap = 0;
-  system.build.img = pkgs.runCommand "nixos_full-disk-image.img" {} ''
-    cp -v ${config.system.build.img-without-firmware}/nixos.img $out
-    chmod +w $out
-    dd if=${pkgs.tow-boot-pinephone}/Tow-Boot.noenv.bin of=$out bs=1024 seek=8 conv=notrunc
+  sane.image.installBootloader = ''
+    dd if=${pkgs.tow-boot-pinephone}/Tow-Boot.noenv.bin of=$out/nixos.img bs=1024 seek=8 conv=notrunc
   '';
 }

hosts/by-name/moby/default.nix

@@ -21,21 +21,26 @@
 
   sane.roles.client = true;
   sane.roles.handheld = true;
-  sane.zsh.showDeadlines = false;  # unlikely to act on them when in shell
+  sane.programs.zsh.config.showDeadlines = false;  # unlikely to act on them when in shell
   sane.services.wg-home.enable = true;
   sane.services.wg-home.ip = config.sane.hosts.by-name."moby".wg-home.ip;
 
   # XXX colin: phosh doesn't work well with passwordless login,
   # so set this more reliable default password should anything go wrong
   users.users.colin.initialPassword = "147147";
-  services.getty.autologinUser = "root";  # allows for emergency maintenance?
+  # services.getty.autologinUser = "root";  # allows for emergency maintenance?
 
   sops.secrets.colin-passwd.neededForUsers = true;
 
-  sane.gui.sxmo.enable = true;
-  # sane.programs.consoleUtils.enableFor.user.colin = false;
-  # sane.programs.guiApps.enableFor.user.colin = false;
+  # sane.gui.sxmo.enable = true;
+  sane.programs.sway.enableFor.user.colin = true;
+  sane.programs.swaylock.enableFor.user.colin = false;  #< not usable on touch
+  sane.programs.schlock.enableFor.user.colin = true;
+  sane.programs.swayidle.config.actions.screenoff.delay = 300;
+  sane.programs.swayidle.config.actions.screenoff.enable = true;
+  sane.programs.sane-input-handler.enableFor.user.colin = true;
   sane.programs.blueberry.enableFor.user.colin = false;  # bluetooth manager: doesn't cross compile!
+  sane.programs.fcitx5.enableFor.user.colin = false;  # does not cross compile
   sane.programs.mercurial.enableFor.user.colin = false;  # does not cross compile
   sane.programs.nvme-cli.enableFor.system = false;  # does not cross compile (libhugetlbfs)
 
@@ -45,27 +50,16 @@
 
   # sane.programs.ntfy-sh.config.autostart = true;
   sane.programs.dino.config.autostart = true;
-  sane.programs.signal-desktop.config.autostart = true;
+  # sane.programs.signal-desktop.config.autostart = true;  # TODO: enable once electron stops derping.
   # sane.programs."gnome.geary".config.autostart = true;
   # sane.programs.calls.config.autostart = true;
-  sane.programs.mpv.config.vo = "wlshm";  #< see hosts/common/programs/mpv.nix for details
 
   sane.programs.firefox.mime.priority = 300;  # prefer other browsers when possible
   # HACK/TODO: make `programs.P.env.VAR` behave according to `mime.priority`
   sane.programs.firefox.env = lib.mkForce {};
   sane.programs.epiphany.env.BROWSER = "epiphany";
-
-  # note the .conf.d approach: using ~/.config/pipewire/pipewire.conf directly breaks all audio,
-  # presumably because that deletes the defaults entirely whereas the .conf.d approach selectively overrides defaults
-  sane.user.fs.".config/pipewire/pipewire.conf.d/10-fix-dino-mic-cutout.conf".symlink.text = ''
-    # config docs: <https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Config-PipeWire#properties>
-    # useful to run `pw-top` to see that these settings are actually having effect,
-    # and `pw-metadata` to see if any settings conflict (e.g. max-quantum < min-quantum)
-    #
-    # restart pipewire after editing these files:
-    # - `systemctl --user restart pipewire`
-    # - pipewire users will likely stop outputting audio until they are also restarted
-    #
+  sane.programs.pipewire.config = {
+    # tune so Dino doesn't drop audio
     # there's seemingly two buffers for the mic (see: <https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/FAQ#pipewire-buffering-explained>)
     # 1. Pipewire buffering out of the driver and into its own member.
     # 2. Pipewire buffering into Dino.
@@ -76,11 +70,9 @@
     # `pw-metadata -n settings 0 clock.force-quantum 1024` reduces to about 1 error per second.
     # `pw-metadata -n settings 0 clock.force-quantum 2048` reduces to 1 error every < 10s.
     # pipewire default config includes `clock.power-of-two-quantum = true`
-    context.properties = {
-      default.clock.min-quantum = 2048
-      default.clock.max-quantum = 8192
-    }
-  '';
+    min-quantum = 2048;
+    max-quantum = 8192;
+  };
 
   boot.loader.efi.canTouchEfiVariables = false;
   # /boot space is at a premium. default was 20.
@@ -118,45 +110,7 @@
   '';
 
   # enable rotation sensor
-  hardware.sensor.iio.enable = true;
-
-  # inject specialized alsa configs via the environment.
-  # specifically, this gets the pinephone headphones & internal earpiece working.
-  # see pkgs/patched/alsa-ucm-conf for more info.
-  environment.variables.ALSA_CONFIG_UCM2 = "/run/current-system/sw/share/alsa/ucm2";
-  environment.pathsToLink = [ "/share/alsa/ucm2" ];
-  environment.systemPackages = [
-    (pkgs.alsa-ucm-conf-sane.override {
-      # internal speaker has a tendency to break :(
-      preferEarpiece = true;
-    })
-  ];
-  systemd = let
-    ucm-env = config.environment.variables.ALSA_CONFIG_UCM2;
-  in {
-    # cribbed from <repo:nixos/mobile-nixos:modules/quirks/audio.nix>
-
-    # pipewire
-    user.services.pipewire.environment.ALSA_CONFIG_UCM2       = ucm-env;
-    user.services.pipewire-pulse.environment.ALSA_CONFIG_UCM2 = ucm-env;
-    user.services.wireplumber.environment.ALSA_CONFIG_UCM2    = ucm-env;
-    services.pipewire.environment.ALSA_CONFIG_UCM2            = ucm-env;
-    services.pipewire-pulse.environment.ALSA_CONFIG_UCM2      = ucm-env;
-    services.wireplumber.environment.ALSA_CONFIG_UCM2         = ucm-env;
-
-    # pulseaudio
-    # user.services.pulseaudio.environment.ALSA_CONFIG_UCM2 = ucm-env;
-    # services.pulseaudio.environment.ALSA_CONFIG_UCM2      = ucm-env;
-
-
-    # TODO: move elsewhere...
-    services.ModemManager.serviceConfig = {
-      # N.B.: the extra "" in ExecStart serves to force upstream ExecStart to be ignored
-      ExecStart = [ "" "${pkgs.modemmanager}/bin/ModemManager --debug" ];
-      # --debug sets DEBUG level logging: so reset
-      ExecStartPost = [ "${pkgs.modemmanager}/bin/mmcli --set-logging=INFO" ];
-    };
-  };
+  # hardware.sensor.iio.enable = true;
 
   services.udev.extraRules = let
     chmod = "${pkgs.coreutils}/bin/chmod";

hosts/by-name/moby/gps.nix

@@ -64,6 +64,5 @@
     "dialout"  # TODO: figure out if dialout is required. that's for /dev/ttyUSB1, but geoclue probably doesn't read that?
   ];
 
-  sane.services.eg25-control.enable = true;
   sane.programs.where-am-i.enableFor.user.colin = true;
 }

hosts/by-name/moby/kernel.nix

@@ -40,35 +40,213 @@ let
   '';
 in
 {
-  boot.kernelPackages = pkgs.linuxPackagesFor pkgs.linux-megous;
+  # kernel compatibility (2024/05/22: 03dab630)
+  # - linux-megous: boots to ssh, desktop
+  #   - camera apps: megapixels (no cameras found), snapshot (no cameras found)
+  # - linux-postmarketos: boots to ssh. desktop ONLY if "anx7688" is in the initrd.availableKernelModules.
+  #   - camera apps: megapixels (both rear and front cameras work), `cam -l` (finds only the rear camera), snapshot (no cameras found)
+  # - linux-megous.override { withMegiPinephoneConfig = true; }: NO SSH, NO SIGNS OF LIFE
+  # - linux-megous.override { withFullConfig = false; }: boots to ssh, no desktop
+  #
+  boot.kernelPackages = pkgs.linuxPackagesFor (pkgs.linux-postmarketos.override {
+    withModemPower = true;
+  });
+  # boot.kernelPackages = pkgs.linuxPackagesFor pkgs.linux-megous;
+  # boot.kernelPackages = pkgs.linuxPackagesFor (pkgs.linux-megous.override {
+  #   withFullConfig = false;
+  # });
+  # boot.kernelPackages = pkgs.linuxPackagesFor (pkgs.linux-megous.override {
+  #   withMegiPinephoneConfig = true;  #< N.B.: does not boot as of 2024/05/22!
+  # });
   # boot.kernelPackages = pkgs.linuxPackagesFor pkgs.linux-manjaro;
   # boot.kernelPackages = pkgs.linuxPackagesFor pkgs.linux_latest;
 
-  # alternatively, apply patches directly to stock nixos kernel:
-  # boot.kernelPatches = manjaroPatches ++ [
-  #   (patchDefconfig kernelConfig)
-  # ];
-
-  # configure nixos to build a compressed kernel image, since it doesn't usually do that for aarch64 target.
-  # without this i run out of /boot space in < 10 generations
+  # nixpkgs.hostPlatform.linux-kernel becomes stdenv.hostPlatform.linux-kernel
   nixpkgs.hostPlatform.linux-kernel = {
     # defaults:
     name = "aarch64-multiplatform";
-    baseConfig = "defconfig";
-    DTB = true;
-    autoModules = true;
-    preferBuiltin = true;
-    # extraConfig = ...
-    # ^-- raspberry pi stuff: we don't need it.
+    # baseConfig: defaults to "defconfig";
+    # baseConfig = "pinephone_defconfig";  #< N.B.: ignored by `pkgs.linux-megous`
+    DTB = true;  #< DTB: compile device tree blobs
+    # autoModules (default: true): for config options not manually specified, answer `m` to anything which supports it.
+    # - this effectively builds EVERY MODULE SUPPORTED.
+    autoModules = true;  #< N.B.: ignored by `pkgs.linux-megous`
+    # preferBuiltin (default: false; true for rpi): for config options which default to `Y` upstream, build them as `Y` (overriding `autoModules`)
+    # preferBuiltin = false;
 
+    # build a compressed kernel image: without this i run out of /boot space in < 10 generations
     # target = "Image";  # <-- default
     target = "Image.gz";  # <-- compress the kernel image
     # target = "zImage";  # <-- confuses other parts of nixos :-(
   };
 
+  # boot.initrd.kernelModules = [
+  #   "drm"  #< force drm to be plugged
+  # ];
+  boot.initrd.availableKernelModules = [
+    # see <repo:postmarketOS/pmaports:device/main/device-pine64-pinephone/modules-initfs>
+    # - they include sun6i_mipi_dsi sun4i_drm pwm_sun4i sun8i_mixer anx7688 gpio_vibra pinephone_keyboard
+    "anx7688" #< required for display initialization and functional cameras
+    # full list of modules active post-boot with the linux-megous kernel + autoModules=true:
+    # - `lsmod | sort | cut -d ' ' -f 1`
+    # "8723cs"
+    # "axp20x_adc"  #< NOT FOUND in megous-no-autoModules
+    # "axp20x_battery"
+    # "axp20x_pek"
+    # "axp20x_usb_power"
+    # "backlight"
+    # "blake2b_generic"
+    # "bluetooth"
+    # "bridge"
+    # "btbcm"
+    # "btqca"
+    # "btrfs"
+    # "btrtl"
+    # "cec"
+    # "cfg80211"
+    # "chacha_neon"
+    # "crc_ccitt"
+    # "crct10dif_ce"
+    # "crypto_engine"
+    # "display_connector"  #< NOT FOUND in pmos
+    # "drm"
+    # "drm_display_helper"
+    # "drm_dma_helper"
+    # "drm_kms_helper"
+    # "drm_shmem_helper"
+    # "dw_hdmi"
+    # "dw_hdmi_cec"  #< NOT FOUND in pmos
+    # "dw_hdmi_i2s_audio"
+    # "ecc"
+    # "ecdh_generic"
+    # "fuse"
+    # "gc2145"  #< NOT FOUND in megous-no-autoModules
+    # "goodix_ts"
+    # "gpio_vibra"  #< NOT FOUND in megous-no-autoModules
+    # "gpu_sched"
+    # "hci_uart"
+    # "i2c_gpio"
+    # "inv_mpu6050"  #< NOT FOUND in megous-no-autoModules
+    # "inv_mpu6050_i2c"  #< NOT FOUND in megous-no-autoModules
+    # "inv_sensors_timestamp"  #< NOT FOUND in megous-no-autoModules
+    # "ip6t_rpfilter"
+    # "ip6_udp_tunnel"
+    # "ip_set"
+    # "ip_set_hash_ipport"
+    # "ip_tables"
+    # "ipt_rpfilter"
+    # "joydev"
+    # "led_class_flash"  #< NOT FOUND in megous-no-autoModules
+    # "leds_sgm3140"  #< NOT FOUND in megous-no-autoModules
+    # "ledtrig_pattern"  #< NOT FOUND in megous-no-autoModules
+    # "libarc4"
+    # "libchacha"
+    # "libchacha20poly1305"
+    # "libcrc32c"
+    # "libcurve25519_generic"
+    # "lima"
+    # "llc"
+    # "mac80211"
+    # "macvlan"
+    # "mc"
+    # "modem_power"
+    # "mousedev"
+    # "nf_conntrack"
+    # "nf_defrag_ipv4"
+    # "nf_defrag_ipv6"
+    # "nf_log_syslog"
+    # "nf_nat"
+    # "nfnetlink"
+    # "nf_tables"
+    # "nft_chain_nat"
+    # "nft_compat"
+    # "nls_cp437"
+    # "nls_iso8859_1"
+    # "nvmem_reboot_mode"
+    # "ov5640"
+    # "panel_sitronix_st7703"
+    # "phy_sun6i_mipi_dphy"
+    # "pinctrl_axp209"  #< NOT FOUND in pmos
+    # "pinephone_keyboard"  #< NOT FOUND in megous-no-autoModules
+    # "poly1305_neon"
+    # "polyval_ce"
+    # "polyval_generic"
+    # "ppkb_manager"  #< NOT FOUND in megous-no-autoModules
+    # "pwm_bl"
+    # "pwm_sun4i"
+    # "qrtr"
+    # "raid6_pq"
+    # "rfkill"
+    # "rtw88_8703b"
+    # "rtw88_8723cs"
+    # "rtw88_8723x"
+    # "rtw88_core"
+    # "rtw88_sdio"
+    # "sch_fq_codel"
+    # "sm4"
+    # "snd_soc_bt_sco"
+    # "snd_soc_ec25"  #< NOT FOUND in megous-no-autoModules
+    # "snd_soc_hdmi_codec"
+    # "snd_soc_simple_amplifier"
+    # "snd_soc_simple_card"
+    # "snd_soc_simple_card_utils"
+    # "stk3310"  #< NOT FOUND in megous-no-autoModules
+    # "st_magn"
+    # "st_magn_i2c"
+    # "st_magn_spi"  #< NOT FOUND in pmos
+    # "stp"
+    # "st_sensors"
+    # "st_sensors_i2c"
+    # "st_sensors_spi"  #< NOT FOUND in pmos
+    # "sun4i_drm"
+    # "sun4i_i2s"
+    # "sun4i_lradc_keys"  #< NOT FOUND in megous-no-autoModules
+    # "sun4i_tcon"
+    # "sun50i_codec_analog"
+    # "sun6i_csi"
+    # "sun6i_dma"
+    # "sun6i_mipi_dsi"
+    # "sun8i_a33_mbus"  #< NOT FOUND in megous-no-autoModules
+    # "sun8i_adda_pr_regmap"
+    # "sun8i_ce"  #< NOT FOUND in pmos
+    # "sun8i_codec"  #< NOT FOUND in megous-no-autoModules
+    # "sun8i_di"  #< NOT FOUND in megous-no-autoModules
+    # "sun8i_drm_hdmi"
+    # "sun8i_mixer"
+    # "sun8i_rotate"  #< NOT FOUND in megous-no-autoModules
+    # "sun8i_tcon_top"
+    # "sun9i_hdmi_audio"  #< NOT FOUND in megous-no-autoModules
+    # "sunxi_wdt"  #< NOT FOUND in pmos
+    # "tap"
+    # "typec"  #< NOT FOUND in pmos
+    # "udp_tunnel"
+    # "uio"  #< NOT FOUND in pmos
+    # "uio_pdrv_genirq"
+    # "v4l2_async"
+    # "v4l2_cci" #< NOT FOUND in pmos
+    # "v4l2_flash_led_class"  #< NOT FOUND in megous-no-autoModules
+    # "v4l2_fwnode"
+    # "v4l2_mem2mem"
+    # "videobuf2_common"
+    # "videobuf2_dma_contig"
+    # "videobuf2_memops"
+    # "videobuf2_v4l2"
+    # "videodev"
+    # "wireguard"
+    # "xor"
+    # "x_tables"
+    # "xt_conntrack"
+    # "xt_LOG"
+    # "xt_nat"
+    # "xt_pkttype"
+    # "xt_set"
+    # "xt_tcpudp"
+    # "zram"
+  ];
+
   # disable proximity sensor.
   # the filtering/calibration is bad that it causes the screen to go fully dark at times.
-  boot.blacklistedKernelModules = [ "stk3310" ];
+  # boot.blacklistedKernelModules = [ "stk3310" ];
 
   boot.kernelParams = [
     # without this some GUI apps fail: `DRM_IOCTL_MODE_CREATE_DUMB failed: Cannot allocate memory`
@@ -78,13 +256,18 @@ in
     #
     # the default CMA seems to be 32M.
     # i was running fine with 256MB from 2022/07-ish through 2022/12-ish, but then the phone quit reliably coming back from sleep (phosh): maybe a memory leak?
+    # bumped to 512M on 2023/01
+    # bumped to 1536M on 2024/05
     # `cat /proc/meminfo` to see CmaTotal/CmaFree if interested in tuning this.
-    "cma=512M"
+    # kernel param mentioned here: <https://cateee.net/lkddb/web-lkddb/CMA_SIZE_PERCENTAGE.html>
+    # i think cma mem isn't exclusive -- it can be used as ordinary `malloc`, still. i heard someone suggest the OS default should just be 50% memory to CMA.
+    "cma=1536M"
     # 2023/10/20: potential fix for the lima (GPU) timeout bugs:
     # - <https://gitlab.com/postmarketOS/pmaports/-/issues/805#note_890467824>
     "lima.sched_timeout_ms=2000"
   ];
 
-  services.xserver.displayManager.job.preStart = ensureHWReady;
-  systemd.services.greetd.preStart = ensureHWReady;
+  # services.xserver.displayManager.job.preStart = ensureHWReady;
+  # systemd.services.greetd.preStart = ensureHWReady;
+  systemd.services.unl0kr.preStart = ensureHWReady;
 }

hosts/by-name/moby/polyfill.nix

@@ -24,73 +24,22 @@
     backlight = "backlight";  # /sys/class/backlight/*backlight*/brightness
   };
 
-  sane.gui.sxmo = {
-    nogesture = true;
-    settings = {
-      ### hardware: touch screen
-      SXMO_LISGD_INPUT_DEVICE = "/dev/input/by-path/platform-1c2ac00.i2c-event";
-      # vol and power are detected correctly by upstream
+  sane.programs.alacritty.config.fontSize = 9;
 
-      ### preferences
-      DEFAULT_COUNTRY = "US";
+  sane.programs.sway.config = {
+    font = "pango:monospace 10";
+    mod = "Mod1";  # prefer Alt
+    workspace_layout = "tabbed";
+  };
 
-      SXMO_AUTOROTATE = "1";  # enable auto-rotation at launch. has no meaning in stock/upstream sxmo-utils
-
-      # BEMENU lines (wayland DMENU):
-      # - camera is 9th entry
-      # - flashlight is 10th entry
-      # - config is 14th entry. inside that:
-      #   - autorotate is 11th entry
-      #   - system menu is 19th entry
-      #   - close is 20th entry
-      # - power is 15th entry
-      # - close is 16th entry
-      SXMO_BEMENU_LANDSCAPE_LINES = "11";  # default 8
-      SXMO_BEMENU_PORTRAIT_LINES = "16";  # default 16
-      SXMO_LOCK_IDLE_TIME = "15";  # how long between screenoff -> lock -> back to screenoff (default: 8)
-      # gravity: how far to tilt the device before the screen rotates
-      # for a given setting, normal <-> invert requires more movement then left <-> right
-      # i.e. the settingd doesn't feel completely symmetric
-      # SXMO_ROTATION_GRAVITY default is 16374
-      # SXMO_ROTATION_GRAVITY = "12800";  # uncomfortably high
-      # SXMO_ROTATION_GRAVITY = "12500";    # kinda uncomfortable when walking
-      SXMO_ROTATION_GRAVITY = "12000";
-      SXMO_SCREENSHOT_DIR = "/home/colin/Pictures";  # default: "$HOME"
-
-      # sway/wayland scaling:
-      # - conflicting info out there on how scaling actually works
-      #   at the least, for things where it matters (mpv), it seems like scale settings have 0 effect on perf
-      # ways to enforce scaling:
-      # - <https://wiki.archlinux.org/title/HiDPI>
-      # - `swaymsg -- output DSI-1 scale 2.0`  (scales everything)
-      # - `dconf write /org/gnome/desktop/interface/text-scaling-factor 2.0`  (scales ONLY TEXT)
-      # - `GDK_DPI_SCALE=2.0`  (scales ONLY TEXT)
-      #
-      # application notes:
-      # - cozy: in landscape, playback position is not visible unless scale <= 1.7
-      #   - if in a tab, then scale 1.6 is the max
-      # SXMO_SWAY_SCALE = "1.5";  # hard to press gPodder icons
-      SXMO_SWAY_SCALE = "1.6";
-      # SXMO_SWAY_SCALE = "1.8";
-      # SXMO_SWAY_SCALE = "2";
-      SXMO_WORKSPACE_WRAPPING = "5";  # how many workspaces. default: 4
-
-      # wvkbd layers:
-      # - full
-      # - landscape
-      # - special  (e.g. coding symbols like ~)
-      # - emoji
-      # - nav
-      # - simple  (like landscape, but no parens/tab/etc; even fewer chars)
-      # - simplegrid  (simple, but grid layout)
-      # - dialer  (digits)
-      # - cyrillic
-      # - arabic
-      # - persian
-      # - greek
-      # - georgian
-      WVKBD_LANDSCAPE_LAYERS = "landscape,special,emoji";
-      WVKBD_LAYERS = "full,special,emoji";
-    };
+  sane.programs.waybar.config = {
+    fontSize = 14;
+    height = 26;
+    persistWorkspaces = [ "1" "2" "3" "4" "5" ];
+    modules.media = false;
+    modules.network = false;
+    modules.perf = false;
+    modules.windowTitle = false;
+    # TODO: show modem state
   };
 }

hosts/by-name/rescue/default.nix

@@ -6,7 +6,7 @@
 
   boot.loader.efi.canTouchEfiVariables = false;
   sane.image.extraBootFiles = [ pkgs.bootpart-uefi-x86_64 ];
-  sane.persist.enable = false;
+  sane.persist.enable = false;  # what we mean here is that the image is immutable; `/` is still tmpfs.
   sane.nixcache.enable = false;  # don't want to be calling out to dead machines that we're *trying* to rescue
 
   # auto-login at shell

hosts/by-name/servo/default.nix

@@ -15,20 +15,19 @@
   };
 
   sane.roles.build-machine.enable = true;
-  sane.zsh.showDeadlines = false;  # ~/knowledge doesn't always exist
+  sane.programs.zsh.config.showDeadlines = false;  # ~/knowledge doesn't always exist
   sane.programs.consoleUtils.suggestedPrograms = [
     "consoleMediaUtils"  # notably, for go2tv / casting
     "pcConsoleUtils"
     "sane-scripts.stop-all-servo"
   ];
   sane.services.dyn-dns.enable = true;
+  sane.services.trust-dns.asSystemResolver = false;  # TODO: enable once it's all working well
   sane.services.wg-home.enable = true;
   sane.services.wg-home.visibleToWan = true;
   sane.services.wg-home.forwardToWan = true;
   sane.services.wg-home.routeThroughServo = false;
   sane.services.wg-home.ip = config.sane.hosts.by-name."servo".wg-home.ip;
-  sane.nixcache.substituters.servo = false;
-  sane.nixcache.substituters.desko = false;
   sane.nixcache.remote-builders.desko = false;
   sane.nixcache.remote-builders.servo = false;
   # sane.services.duplicity.enable = true;  # TODO: re-enable after HW upgrade

hosts/by-name/servo/fs.nix

@@ -8,6 +8,7 @@
 # - 1. identify disk IDs: `ls -l /dev/disk/by-id`
 # - 2. pool these disks: `zpool create -f -m legacy pool raidz ata-ST4000VN008-2DR166_WDH0VB45 ata-ST4000VN008-2DR166_WDH17616 ata-ST4000VN008-2DR166_WDH0VC8Q ata-ST4000VN008-2DR166_WDH17680`
 #   - legacy documented: <https://superuser.com/questions/790036/what-is-a-zfs-legacy-mount-point>
+# - 3. enable acl support: `zfs set acltype=posixacl pool`
 #
 # import pools: `zpool import pool`
 # show zfs datasets: `zfs list` (will be empty if haven't imported)
@@ -25,6 +26,7 @@
   # scrub all zfs pools weekly:
   services.zfs.autoScrub.enable = true;
   boot.extraModprobeConfig = ''
+    ### zfs_arc_max tunable:
     # ZFS likes to use half the ram for its own cache and let the kernel push everything else to swap.
     # so, reduce its cache size
     # see: <https://askubuntu.com/a/1290387>
@@ -33,7 +35,13 @@
     # for all tunables, see: `man 4 zfs`
     # to update these parameters without rebooting:
     # - `echo '4294967296' | sane-sudo-redirect /sys/module/zfs/parameters/zfs_arc_max`
-    options zfs zfs_arc_max=4294967296
+    ### zfs_bclone_enabled tunable
+    # this allows `cp --reflink=always FOO BAR` to work. i.e. shallow copies.
+    # it's unstable as of 2.2.3. led to *actual* corruption in 2.2.1, but hopefully better by now.
+    # - <https://github.com/openzfs/zfs/issues/405>
+    # note that `du -h` won't *always* show the reduced size for reflink'd files (?).
+    # `zpool get all | grep clone` seems to be the way to *actually* see how much data is being deduped
+    options zfs zfs_arc_max=4294967296 zfs_bclone_enabled=1
   '';
   # to be able to mount the pool like this, make sure to tell zfs to NOT manage it itself.
   # otherwise local-fs.target will FAIL and you will be dropped into a rescue shell.
@@ -43,6 +51,7 @@
   fileSystems."/mnt/pool" = {
     device = "pool";
     fsType = "zfs";
+    options = [ "acl" ];  #< not sure if this `acl` flag is actually necessary. it mounts without it.
   };
   # services.zfs.zed = ... # TODO: zfs can send me emails when disks fail
   sane.programs.sysadminUtils.suggestedPrograms = [ "zfs" ];
@@ -50,6 +59,7 @@
   sane.persist.stores."ext" = {
     origin = "/mnt/pool/persist";
     storeDescription = "external HDD storage";
+    defaultMethod = "bind";  #< TODO: change to "symlink"?
   };
 
   # increase /tmp space (defaults to 50% of RAM) for building large nix things.
@@ -81,59 +91,45 @@
   };
   sane.fs."/mnt/usb-hdd".mount = {};
 
-  sane.persist.sys.byStore.plaintext = [
-    # TODO: this is overly broad; only need media and share directories to be persisted
-    { user = "colin"; group = "users"; path = "/var/lib/uninsane"; }
-  ];
-  # force some problematic directories to always get correct permissions:
-  sane.fs."/var/lib/uninsane/media".dir.acl = {
-    user = "colin"; group = "media"; mode = "0775";
-  };
-  sane.fs."/var/lib/uninsane/media/archive".dir = {};
-  sane.fs."/var/lib/uninsane/media/archive/README.md".file.text = ''
+  # FIRST TIME SETUP FOR MEDIA DIRECTORY:
+  # - set the group stick bit: `sudo find /var/media -type d -exec chmod g+s {} +`
+  #   - this ensures new files/dirs inherit the group of their parent dir (instead of the user who creates them)
+  # - ensure everything under /var/media is mounted with `-o acl`, to support acls
+  # - ensure all files are rwx by group: `setfacl --recursive --modify d:g::rwx /var/media`
+  #   - alternatively, `d:g:media:rwx` to grant `media` group even when file has a different owner, but that's a bit complex
+  sane.persist.sys.byStore.ext = [{
+    path = "/var/media";
+    user = "colin";
+    group = "media";
+    mode = "0775";
+  }];
+  sane.fs."/var/media/archive".dir = {};
+  # this is file.text instead of symlink.text so that it may be read over a remote mount (where consumers might not have any /nix/store/.../README.md path)
+  sane.fs."/var/media/archive/README.md".file.text = ''
     this directory is for media i wish to remove from my library,
     but keep for a short time in case i reverse my decision.
     treat it like a system trash can.
   '';
-  sane.fs."/var/lib/uninsane/media/Books".dir = {};
-  sane.fs."/var/lib/uninsane/media/Books/Audiobooks".dir = {};
-  sane.fs."/var/lib/uninsane/media/Books/Books".dir = {};
-  sane.fs."/var/lib/uninsane/media/Books/Visual".dir = {};
-  sane.fs."/var/lib/uninsane/media/collections".dir = {};
-  sane.fs."/var/lib/uninsane/media/datasets".dir = {};
-  sane.fs."/var/lib/uninsane/media/freeleech".dir = {};
-  sane.fs."/var/lib/uninsane/media/Music".dir = {};
-  sane.fs."/var/lib/uninsane/media/Pictures".dir = {};
-  sane.fs."/var/lib/uninsane/media/Videos".dir = {};
-  sane.fs."/var/lib/uninsane/media/Videos/Film".dir = {};
-  sane.fs."/var/lib/uninsane/media/Videos/Shows".dir = {};
-  sane.fs."/var/lib/uninsane/media/Videos/Talks".dir = {};
+  sane.fs."/var/media/Books".dir = {};
+  sane.fs."/var/media/Books/Audiobooks".dir = {};
+  sane.fs."/var/media/Books/Books".dir = {};
+  sane.fs."/var/media/Books/Visual".dir = {};
+  sane.fs."/var/media/collections".dir = {};
+  # sane.fs."/var/media/datasets".dir = {};
+  sane.fs."/var/media/freeleech".dir = {};
+  sane.fs."/var/media/Music".dir = {};
+  sane.fs."/var/media/Pictures".dir = {};
+  sane.fs."/var/media/Videos".dir = {};
+  sane.fs."/var/media/Videos/Film".dir = {};
+  sane.fs."/var/media/Videos/Shows".dir = {};
+  sane.fs."/var/media/Videos/Talks".dir = {};
+
+  # this is file.text instead of symlink.text so that it may be read over a remote mount (where consumers might not have any /nix/store/.../README.md path)
   sane.fs."/var/lib/uninsane/datasets/README.md".file.text = ''
     this directory may seem redundant with ../media/datasets. it isn't.
     this directory exists on SSD, allowing for speedy access to specific datasets when necessary.
     the contents should be a subset of what's in ../media/datasets.
   '';
-  # make sure large media is stored to the HDD
-  sane.persist.sys.byStore.ext = [
-    {
-      user = "colin";
-      group = "users";
-      mode = "0777";
-      path = "/var/lib/uninsane/media/Videos";
-    }
-    {
-      user = "colin";
-      group = "users";
-      mode = "0777";
-      path = "/var/lib/uninsane/media/freeleech";
-    }
-    {
-      user = "colin";
-      group = "users";
-      mode = "0777";
-      path = "/var/lib/uninsane/media/datasets";
-    }
-  ];
 
   # btrfs doesn't easily support swapfiles
   # swapDevices = [

hosts/by-name/servo/net.nix

@@ -24,18 +24,6 @@ in
     sane.ports.openFirewall = true;
     sane.ports.openUpnp = true;
 
-    # view refused packets with: `sudo journalctl -k`
-    # networking.firewall.logRefusedPackets = true;
-
-    # these useDHCP lines are legacy from the auto-generated config. might be safe to remove now?
-    networking.useDHCP = false;
-    networking.interfaces.eth0.useDHCP = true;
-    # XXX colin: probably don't need this. wlan0 won't be populated unless i touch a value in networking.interfaces.wlan0
-    networking.wireless.enable = false;
-
-    # this is needed to forward packets from the VPN to the host
-    boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
-
     # unless we add interface-specific settings for each VPN, we have to define nameservers globally.
     # networking.nameservers = [
     #   "1.1.1.1"
@@ -50,6 +38,39 @@ in
     #   FallbackDNS=1.1.1.1 9.9.9.9
     # '';
 
+    # tun-sea config
+    sane.dns.zones."uninsane.org".inet.A."doof.tunnel" = "205.201.63.12";
+    sane.dns.zones."uninsane.org".inet.AAAA."doof.tunnel" = "2602:fce8:106::51";
+    networking.wireguard.interfaces.wg-doof = let
+      ip = "${pkgs.iproute2}/bin/ip";
+    in {
+      privateKeyFile = config.sops.secrets.wg_doof_privkey.path;
+      # wg is active only in this namespace.
+      # run e.g. ip netns exec doof <some command like ping/curl/etc, it'll go through wg>
+      #   sudo ip netns exec doof ping www.google.com
+      interfaceNamespace = "doof";
+      ips = [
+        "205.201.63.12/32"
+        "2602:fce8:106::51/128"
+      ];
+      peers = [
+        {
+          publicKey = "nuESyYEJ3YU0hTZZgAd7iHBz1ytWBVM5PjEL1VEoTkU=";
+          # TODO: configure DNS within the doof ns and use tun-sea.doof.net endpoint
+          # endpoint = "tun-sea.doof.net:53263";
+          endpoint = "205.201.63.44:53263";
+          allowedIPs = [ "0.0.0.0/0" "::/0" ];
+          persistentKeepalive = 25; #< keep the NAT alive
+        }
+      ];
+      preSetup = ''
+        ${ip} netns add doof || (test -e /run/netns/doof && echo "doof already exists")
+      '';
+      postShutdown = ''
+        ${ip} netns delete doof || echo "couldn't delete doof"
+      '';
+    };
+
     # OVPN CONFIG (https://www.ovpn.com):
     # DOCS: https://nixos.wiki/wiki/WireGuard
     # if you `systemctl restart wireguard-wg-ovpns`, make sure to also restart any other services in `NetworkNamespacePath = .../ovpns`.
@@ -99,7 +120,7 @@ in
         }
       ];
       preSetup = ''
-        ${ip} netns add ovpns || echo "ovpns already exists"
+        ${ip} netns add ovpns || (test -e /run/netns/ovpns && echo "ovpns already exists")
       '';
       postShutdown = ''
         ${in-ns} ip link del ovpns-veth-b || echo "couldn't delete ovpns-veth-b"

hosts/by-name/servo/services/calibre.nix

@@ -13,7 +13,7 @@ in
 lib.mkIf false
 {
   sane.persist.sys.byStore.plaintext = [
-    { inherit user group; mode = "0700"; path = svc-dir; }
+    { inherit user group; mode = "0700"; path = svc-dir; method = "bind"; }
   ];
 
   services.calibre-web.enable = true;
@@ -24,7 +24,7 @@ lib.mkIf false
   # services.calibre-web.options.calibreLibrary = svc-dir;
 
   services.nginx.virtualHosts."calibre.uninsane.org" = {
-    addSSL = true;
+    forceSSL = true;
     enableACME = true;
     locations."/" = {
       proxyPass = "http://${ip}:${builtins.toString port}";

hosts/by-name/servo/services/coturn.nix

@@ -24,50 +24,64 @@
 #   that is NOT the case when the STUN server and client A are on the same LAN
 #   even if client A contacts the STUN server via its WAN address with port reflection enabled.
 #   hence, there's no obvious way to put the STUN server on the same LAN as either client and expect the rest to work.
+# - there an old version which *half worked*, which is:
+#   - run the turn server in the root namespace.
+#   - bind the turn server to the veth connecting it to the VPN namespace (so it sends outgoing traffic to the right place).
+#   - NAT the turn port range from VPN into root namespace (so it receives incomming traffic).
+#   - this approach would fail the prosody conversations.im check, but i didn't notice *obvious* call routing errors.
+#
+# debugging:
+# - log messages like 'usage: realm=<turn.uninsane.org>, username=<1715915193>, rp=14, rb=1516, sp=8, sb=684'
+#   - rp = received packets
+#   - rb = received bytes
+#   - sp = sent packets
+#   - sb = sent bytes
 { lib, ... }:
 let
-  # TODO: this range could be larger, but right now that's costly because each element is its own UPnP forward
-  # TURN port range (inclusive)
-  turnPortLow = 49152;
-  turnPortHigh = 49167;
+  # TURN port range (inclusive).
+  # default coturn behavior is to use the upper quarter of all ports. i.e. 49152 - 65535.
+  # i believe TURN allocations expire after either 5 or 10 minutes of inactivity.
+  turnPortLow = 49152; # 49152 = 0xc000
+  turnPortHigh = turnPortLow + 256;
   turnPortRange = lib.range turnPortLow turnPortHigh;
 in
 {
-  sane.ports.ports = lib.mkMerge ([
-    {
-      "3478" = {
-        # this is the "control" port.
-        # i.e. no client data is forwarded through it, but it's where clients request tunnels.
-        protocol = [ "tcp" "udp" ];
-        # visibleTo.lan = true;
-        # visibleTo.wan = true;
-        visibleTo.ovpn = true;
-        description = "colin-stun-turn";
-      };
-      "5349" = {
-        # the other port 3478 also supports TLS/DTLS, but presumably clients wanting TLS will default 5349
-        protocol = [ "tcp" ];
-        # visibleTo.lan = true;
-        # visibleTo.wan = true;
-        visibleTo.ovpn = true;
-        description = "colin-stun-turn-over-tls";
-      };
-    }
-  ] ++ (builtins.map
-    (port: {
-      "${builtins.toString port}" = let
-        count = port - turnPortLow + 1;
-        numPorts = turnPortHigh - turnPortLow + 1;
-      in {
-        protocol = [ "tcp" "udp" ];
-        # visibleTo.lan = true;
-        # visibleTo.wan = true;
-        visibleTo.ovpn = true;
-        description = "colin-turn-${builtins.toString count}-of-${builtins.toString numPorts}";
-      };
-    })
-    turnPortRange
-  ));
+  # the port definitions are only needed if running in the root net namespace
+  # sane.ports.ports = lib.mkMerge ([
+  #   {
+  #     "3478" = {
+  #       # this is the "control" port.
+  #       # i.e. no client data is forwarded through it, but it's where clients request tunnels.
+  #       protocol = [ "tcp" "udp" ];
+  #       # visibleTo.lan = true;
+  #       # visibleTo.wan = true;
+  #       visibleTo.ovpn = true;  # forward traffic from the VPN to the root NS
+  #       description = "colin-stun-turn";
+  #     };
+  #     "5349" = {
+  #       # the other port 3478 also supports TLS/DTLS, but presumably clients wanting TLS will default 5349
+  #       protocol = [ "tcp" ];
+  #       # visibleTo.lan = true;
+  #       # visibleTo.wan = true;
+  #       visibleTo.ovpn = true;
+  #       description = "colin-stun-turn-over-tls";
+  #     };
+  #   }
+  # ] ++ (builtins.map
+  #   (port: {
+  #     "${builtins.toString port}" = let
+  #       count = port - turnPortLow + 1;
+  #       numPorts = turnPortHigh - turnPortLow + 1;
+  #     in {
+  #       protocol = [ "tcp" "udp" ];
+  #       # visibleTo.lan = true;
+  #       # visibleTo.wan = true;
+  #       visibleTo.ovpn = true;
+  #       description = "colin-turn-${builtins.toString count}-of-${builtins.toString numPorts}";
+  #     };
+  #   })
+  #   turnPortRange
+  # ));
 
   services.nginx.virtualHosts."turn.uninsane.org" = {
     # allow ACME to procure a cert via nginx for this domain
@@ -103,22 +117,28 @@ in
   services.coturn.realm = "turn.uninsane.org";
   services.coturn.cert = "/var/lib/acme/turn.uninsane.org/fullchain.pem";
   services.coturn.pkey = "/var/lib/acme/turn.uninsane.org/key.pem";
+
+  #v disable to allow unauthenticated access (or set `services.coturn.no-auth = true`)
   services.coturn.use-auth-secret = true;
   services.coturn.static-auth-secret-file = "/var/lib/coturn/shared_secret.bin";
-  services.coturn.lt-cred-mech = true;
+  services.coturn.lt-cred-mech = true; #< XXX: use-auth-secret overrides lt-cred-mech
+
   services.coturn.min-port = turnPortLow;
   services.coturn.max-port = turnPortHigh;
   # services.coturn.secure-stun = true;
   services.coturn.extraConfig = lib.concatStringsSep "\n" [
     "verbose"
-    # "Verbose"  #< even MORE verbosity than "verbose"
-    # "no-multicast-peers"  # disables sending to IPv4 broadcast addresses (e.g. 224.0.0.0/3)
-    "listening-ip=10.0.1.5"
+    # "Verbose"  #< even MORE verbosity than "verbose"  (it's TOO MUCH verbosity really)
+    "no-multicast-peers"  # disables sending to IPv4 broadcast addresses (e.g. 224.0.0.0/3)
+    # "listening-ip=10.0.1.5" "external-ip=185.157.162.178"  #< 2024/04/25: works, if running in root namespace
+    "listening-ip=185.157.162.178" "external-ip=185.157.162.178"
+
+    # old attempts:
     # "external-ip=185.157.162.178/10.0.1.5"
-    "external-ip=185.157.162.178"
     # "listening-ip=10.78.79.51"  # can be specified multiple times; omit for *
     # "external-ip=97.113.128.229/10.78.79.51"
     # "external-ip=97.113.128.229"
     # "mobility"  # "mobility with ICE (MICE) specs support" (?)
   ];
+  systemd.services.coturn.serviceConfig.NetworkNamespacePath = "/run/netns/ovpns";
 }

hosts/by-name/servo/services/cryptocurrencies/bitcoin.nix

@@ -30,8 +30,7 @@ let
 in
 {
   sane.persist.sys.byStore.ext = [
-    # /var/lib/monero/lmdb is what consumes most of the space
-    { user = "bitcoind-mainnet"; group = "bitcoind-mainnet"; path = "/var/lib/bitcoind-mainnet"; }
+    { user = "bitcoind-mainnet"; group = "bitcoind-mainnet"; path = "/var/lib/bitcoind-mainnet"; method = "bind"; }
   ];
 
   # sane.ports.ports."8333" = {

hosts/by-name/servo/services/cryptocurrencies/clightning-sane/clightning-sane

@@ -1,6 +1,22 @@
 #!/usr/bin/env nix-shell
 #!nix-shell -i python3 -p "python3.withPackages (ps: [ ps.pyln-client ])"
 
+"""
+clightning-sane: helper to perform common Lightning node admin operations:
+- view channel balances
+- rebalance channels
+
+COMMON OPERATIONS:
+- view channel balances: `clightning-sane status`
+- rebalance channels to improve routability (without paying any fees): `clightning-sane autobalance`
+
+FULL OPERATION:
+- `clightning-sane status --full`
+  - `P$`: represents how many msats i've captured in fees from this channel.
+  - `COST`: rough measure of how much it's "costing" me to let my channel partner hold funds on his side of the channel.
+            this is based on the notion that i only capture fees from outbound transactions, and so the channel partner holding all liquidity means i can't capture fees on that liquidity.
+"""
+
 # pyln-client docs: <https://github.com/ElementsProject/lightning/tree/master/contrib/pyln-client>
 # terminology:
 # - "scid": "Short Channel ID", e.g. 123456x7890x0
@@ -726,7 +742,7 @@ def main():
     logging.basicConfig()
     logger.setLevel(logging.INFO)
 
-    parser = argparse.ArgumentParser(description="rebalance lightning channel balances")
+    parser = argparse.ArgumentParser(description=__doc__)
     parser.add_argument("--verbose", action="store_true", help="more logging")
     parser.add_argument("--min-msat", default="999", help="min transaction size")
     parser.add_argument("--max-msat", default="1000000", help="max transaction size")

hosts/by-name/servo/services/cryptocurrencies/clightning.nix

@@ -73,7 +73,7 @@
 { config, pkgs, ... }:
 {
   sane.persist.sys.byStore.ext = [
-    { user = "clightning"; group = "clightning"; mode = "0710"; path = "/var/lib/clightning"; }
+    { user = "clightning"; group = "clightning"; mode = "0710"; path = "/var/lib/clightning"; method = "bind"; }
   ];
 
   # `lightning-cli` finds its RPC file via `~/.lightning/bitcoin/lightning-rpc`, to message the daemon

hosts/by-name/servo/services/cryptocurrencies/monero.nix

@@ -3,7 +3,7 @@
 {
   sane.persist.sys.byStore.ext = [
     # /var/lib/monero/lmdb is what consumes most of the space
-    { user = "monero"; group = "monero"; path = "/var/lib/monero"; }
+    { user = "monero"; group = "monero"; path = "/var/lib/monero"; method = "bind"; }
   ];
 
   services.monero.enable = true;

hosts/by-name/servo/services/cryptocurrencies/tor.nix

@@ -4,7 +4,7 @@
   # tor hidden service hostnames aren't deterministic, so persist.
   # might be able to get away with just persisting /var/lib/tor/onion, not sure.
   sane.persist.sys.byStore.plaintext = [
-    { user = "tor"; group = "tor"; mode = "0710"; path = "/var/lib/tor"; }
+    { user = "tor"; group = "tor"; mode = "0710"; path = "/var/lib/tor"; method = "bind"; }
   ];
 
   # tor: `tor.enable` doesn't start a relay, exit node, proxy, etc. it's minimal.

hosts/by-name/servo/services/default.nix

@@ -20,7 +20,6 @@
     ./navidrome.nix
     ./nginx.nix
     ./nixos-prebuild.nix
-    ./nixserve.nix
     ./ntfy
     ./pict-rs.nix
     ./pleroma.nix

hosts/by-name/servo/services/ejabberd.nix

@@ -45,7 +45,7 @@ in
 lib.mkIf false
 {
   sane.persist.sys.byStore.plaintext = [
-    { user = "ejabberd"; group = "ejabberd"; path = "/var/lib/ejabberd"; }
+    { user = "ejabberd"; group = "ejabberd"; path = "/var/lib/ejabberd"; method = "bind"; }
   ];
   sane.ports.ports = lib.mkMerge ([
     {

hosts/by-name/servo/services/email/dovecot.nix

@@ -127,10 +127,11 @@
   services.dovecot2.modules = [
     pkgs.dovecot_pigeonhole  # enables sieve execution (?)
   ];
-  services.dovecot2.sieveScripts = {
+  services.dovecot2.sieve = {
+    extensions = [ "fileinto" ];
     # if any messages fail to pass (or lack) DKIM, move them to Junk
     # XXX the key name ("after") is only used to order sieve execution/ordering
-    after = builtins.toFile "ensuredkim.sieve" ''
+    scripts.after = builtins.toFile "ensuredkim.sieve" ''
       require "fileinto";
 
       if not header :contains "Authentication-Results" "dkim=pass" {
@@ -139,4 +140,6 @@
       }
     '';
   };
+
+  systemd.services.dovecot2.serviceConfig.RestartSec = lib.mkForce "15s";  # nixos defaults this to 1s
 }

hosts/by-name/servo/services/email/postfix.nix

@@ -20,9 +20,9 @@ in
 {
   sane.persist.sys.byStore.plaintext = [
     # TODO: mode? could be more granular
-    { user = "opendkim"; group = "opendkim"; path = "/var/lib/opendkim"; }
-    { user = "root"; group = "root"; path = "/var/lib/postfix"; }
-    { user = "root"; group = "root"; path = "/var/spool/mail"; }
+    { user = "opendkim"; group = "opendkim"; path = "/var/lib/opendkim"; method = "bind"; }
+    { user = "root"; group = "root"; path = "/var/lib/postfix"; method = "bind"; }
+    { user = "root"; group = "root"; path = "/var/spool/mail"; method = "bind"; }
     # *probably* don't need these dirs:
     # "/var/lib/dhparams"          # https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/security/dhparams.nix
     # "/var/lib/dovecot"

hosts/by-name/servo/services/export/default.nix

@@ -2,14 +2,14 @@
 {
   imports = [
     ./nfs.nix
-    ./sftpgo.nix
+    ./sftpgo
   ];
 
   users.groups.export = {};
 
   fileSystems."/var/export/media" = {
     # everything in here could be considered publicly readable (based on the viewer's legal jurisdiction)
-    device = "/var/lib/uninsane/media";
+    device = "/var/media";
     options = [ "rbind" ];
   };
   # fileSystems."/var/export/playground" = {
@@ -30,7 +30,7 @@
   # to query the quota/status:
   # - `sudo btrfs qgroup show -re /var/export/playground`
   sane.persist.sys.byStore.ext = [
-    { user = "root"; group = "export"; mode = "0775"; path = "/var/export/playground"; }
+    { user = "root"; group = "export"; mode = "0775"; path = "/var/export/playground"; method = "bind"; }
   ];
 
   sane.fs."/var/export/README.md" = {

hosts/by-name/servo/services/export/nfs.nix

@@ -26,7 +26,7 @@
     description = "NFS server portmapper";
   };
   sane.ports.ports."2049" = {
-    protocol = [ "tcp" ];
+    protocol = [ "tcp" "udp" ];
     visibleTo.lan = true;
     description = "NFS server";
   };
@@ -51,6 +51,23 @@
   services.nfs.server.mountdPort = 4002;
   services.nfs.server.statdPort = 4000;
 
+  services.nfs.extraConfig = ''
+    [nfsd]
+    # XXX: NFS over UDP REQUIRES SPECIAL CONFIG TO AVOID DATA LOSS.
+    # see `man 5 nfs`: "Using NFS over UDP on high-speed links".
+    # it's actually just a general property of UDP over IPv4 (IPv6 fixes it).
+    # both the client and the server should configure a shorter-than-default IPv4 fragment reassembly window to mitigate.
+    # OTOH, tunneling NFS over Wireguard also bypasses this weakness, because a mis-assembled packet would not have a valid signature.
+    udp=y
+
+    [exports]
+    # all export paths are relative to rootdir.
+    # for NFSv4, the export with fsid=0 behaves as `/` publicly,
+    # but NFSv3 implements no such feature.
+    # using `rootdir` instead of relying on `fsid=0` allows consistent export paths regardless of NFS proto version
+    rootdir=/var/export
+  '';
+
   # format:
   #   fspoint	visibility(options)
   # options:
@@ -85,13 +102,20 @@
       in "${export} 10.78.79.0/22(${lib.concatStringsSep "," lanOpts}) 10.0.10.0/24(${lib.concatStringsSep "," vpnOpts})";
   in lib.concatStringsSep "\n" [
     (fmtExport {
-      export = "/var/export";
+      export = "/";
       baseOpts = [ "crossmnt" "fsid=root" ];
       extraLanOpts = [ "ro" ];
       extraVpnOpts = [ "rw" "no_root_squash" ];
     })
     (fmtExport {
-      export = "/var/export/playground";
+      # provide /media as an explicit export. NFSv4 can transparently mount a subdir of an export, but NFSv3 can only mount paths which are exports.
+      export = "/media";
+      baseOpts = [ "crossmnt" ];  # TODO: is crossmnt needed here?
+      extraLanOpts = [ "ro" ];
+      extraVpnOpts = [ "rw" "no_root_squash" ];
+    })
+    (fmtExport {
+      export = "/playground";
       baseOpts = [
         "mountpoint"
         "all_squash"

hosts/by-name/servo/services/export/sftpgo.nix

@@ -1,186 +0,0 @@
-# docs:
-# - <https://github.com/drakkan/sftpgo>
-# - config options: <https://github.com/drakkan/sftpgo/blob/main/docs/full-configuration.md>
-# - config defaults: <https://github.com/drakkan/sftpgo/blob/main/sftpgo.json>
-# - nixos options: <repo:nixos/nixpkgs:nixos/modules/services/web-apps/sftpgo.nix>
-# - nixos example: <repo:nixos/nixpkgs:nixos/tests/sftpgo.nix>
-#
-# sftpgo is a FTP server that also supports WebDAV, SFTP, and web clients.
-#
-# TODO: change umask so sftpgo-created files default to 644.
-# - it does indeed appear that the 600 is not something sftpgo is explicitly doing.
-
-
-{ config, lib, pkgs, sane-lib, ... }:
-let
-  # user permissions:
-  # - see <repo:drakkan/sftpgo:internal/dataprovider/user.go>
-  # - "*" = grant all permissions
-  # - read-only perms:
-  #   - "list" = list files and directories
-  #   - "download"
-  # - rw perms:
-  #   - "upload"
-  #   - "overwrite" = allow uploads to replace existing files
-  #   - "delete" = delete files and directories
-  #     - "delete_files"
-  #     - "delete_dirs"
-  #   - "rename" = rename files and directories
-  #     - "rename_files"
-  #     - "rename_dirs"
-  #   - "create_dirs"
-  #   - "create_symlinks"
-  #   - "chmod"
-  #   - "chown"
-  #   - "chtimes" = change atime/mtime (access and modification times)
-  #
-  # home_dir:
-  # - it seems (empirically) that a user can't cd above their home directory.
-  #   though i don't have a reference for that in the docs.
-  authResponseSuccess = {
-    status = 1;
-    username = "anonymous";
-    expiration_date = 0;
-    home_dir = "/var/export";
-    # uid/gid 0 means to inherit sftpgo uid.
-    # - i.e. users can't read files which Linux user `sftpgo` can't read
-    # - uploaded files belong to Linux user `sftpgo`
-    # other uid/gid values aren't possible for localfs backend, unless i let sftpgo use `sudo`.
-    uid = 0;
-    gid = 0;
-    # uid = 65534;
-    # gid = 65534;
-    max_sessions = 0;
-    # quota_*: 0 means to not use SFTP's quota system
-    quota_size = 0;
-    quota_files = 0;
-    permissions = {
-      "/" = [ "list" "download" ];
-      "/playground" = [
-        # read-only:
-        "list"
-        "download"
-        # write:
-        "upload"
-        "overwrite"
-        "delete"
-        "rename"
-        "create_dirs"
-        "create_symlinks"
-        # intentionally omitted:
-        # "chmod"
-        # "chown"
-        # "chtimes"
-      ];
-    };
-    upload_bandwidth = 0;
-    download_bandwidth = 0;
-    filters = {
-      allowed_ip = [];
-      denied_ip = [];
-    };
-    public_keys = [];
-    # other fields:
-    # ? groups
-    # ? virtual_folders
-  };
-  authResponseFail = {
-    username = "";
-  };
-  authSuccessJson = pkgs.writeText "sftp-auth-success.json" (builtins.toJSON authResponseSuccess);
-  authFailJson = pkgs.writeText "sftp-auth-fail.json" (builtins.toJSON authResponseFail);
-  unwrappedAuthProgram = pkgs.static-nix-shell.mkBash {
-    pname = "sftpgo_external_auth_hook";
-    src = ./.;
-    pkgs = [ "coreutils" ];
-  };
-  authProgram = pkgs.writeShellScript "sftpgo-auth-hook" ''
-    ${unwrappedAuthProgram}/bin/sftpgo_external_auth_hook ${authFailJson} ${authSuccessJson}
-  '';
-in
-{
-  # Client initiates a FTP "control connection" on port 21.
-  # - this handles the client -> server commands, and the server -> client status, but not the actual data
-  # - file data, directory listings, etc need to be transferred on an ephemeral "data port".
-  # - 50000-50100 is a common port range for this.
-  sane.ports.ports = {
-    "21" = {
-      protocol = [ "tcp" ];
-      visibleTo.lan = true;
-      description = "colin-FTP server";
-    };
-  } // (sane-lib.mapToAttrs
-    (port: {
-      name = builtins.toString port;
-      value = {
-        protocol = [ "tcp" ];
-        visibleTo.lan = true;
-        description = "colin-FTP server data port range";
-      };
-    })
-    (lib.range 50000 50100)
-  );
-
-  services.sftpgo = {
-    enable = true;
-    group = "export";
-    settings = {
-      ftpd = {
-        bindings = [
-          {
-            # binding this means any wireguard client can connect
-            address = "10.0.10.5";
-            port = 21;
-            debug = true;
-          }
-          {
-            # binding this means any LAN client can connect
-            address = "10.78.79.51";
-            port = 21;
-            debug = true;
-          }
-        ];
-
-        # active mode is susceptible to "bounce attacks", without much benefit over passive mode
-        disable_active_mode = true;
-        hash_support = true;
-        passive_port_range = {
-          start = 50000;
-          end = 50100;
-        };
-
-        banner = ''
-          Welcome, friends, to Colin's read-only FTP server! Also available via NFS on the same host.
-          Username: "anonymous"
-          Password: "anonymous"
-          CONFIGURE YOUR CLIENT FOR "PASSIVE" mode, e.g. `ftp --passive uninsane.org`
-          Please let me know if anything's broken or not as it should be. Otherwise, browse and DL freely :)
-        '';
-
-      };
-      data_provider = {
-        driver = "memory";
-        external_auth_hook = "${authProgram}";
-        # track_quota:
-        # - 0: disable quota tracking
-        # - 1: quota is updated on every upload/delete, even if user has no quota restriction
-        # - 2: quota is updated on every upload/delete, but only if user/folder has a quota restriction  (default, i think)
-        # track_quota = 2;
-      };
-    };
-  };
-
-  users.users.sftpgo.extraGroups = [ "export" ];
-
-  systemd.services.sftpgo = {
-    after = [ "network-online.target" ];
-    wants = [ "network-online.target" ];
-    serviceConfig = {
-      ReadOnlyPaths = [ "/var/export" ];
-      ReadWritePaths = [ "/var/export/playground" ];
-
-      Restart = "always";
-      RestartSec = "20s";
-    };
-  };
-}

hosts/by-name/servo/services/export/sftpgo/default.nix

@@ -0,0 +1,167 @@
+# docs:
+# - <https://github.com/drakkan/sftpgo>
+# - config options: <https://github.com/drakkan/sftpgo/blob/main/docs/full-configuration.md>
+# - config defaults: <https://github.com/drakkan/sftpgo/blob/main/sftpgo.json>
+# - nixos options: <repo:nixos/nixpkgs:nixos/modules/services/web-apps/sftpgo.nix>
+# - nixos example: <repo:nixos/nixpkgs:nixos/tests/sftpgo.nix>
+#
+# sftpgo is a FTP server that also supports WebDAV, SFTP, and web clients.
+
+{ config, lib, pkgs, sane-lib, ... }:
+let
+  external_auth_hook = pkgs.static-nix-shell.mkPython3Bin {
+    pname = "external_auth_hook";
+    srcRoot = ./.;
+  };
+  # Client initiates a FTP "control connection" on port 21.
+  # - this handles the client -> server commands, and the server -> client status, but not the actual data
+  # - file data, directory listings, etc need to be transferred on an ephemeral "data port".
+  # - 50000-50100 is a common port range for this.
+  #   50000 is used by soulseek.
+  passiveStart = 50050;
+  passiveEnd   = 50070;
+in
+{
+  sane.ports.ports = {
+    "21" = {
+      protocol = [ "tcp" ];
+      visibleTo.lan = true;
+      # visibleTo.wan = true;
+      description = "colin-FTP server";
+    };
+    "990" = {
+      protocol = [ "tcp" ];
+      visibleTo.lan = true;
+      visibleTo.wan = true;
+      description = "colin-FTPS server";
+    };
+  } // (sane-lib.mapToAttrs
+    (port: {
+      name = builtins.toString port;
+      value = {
+        protocol = [ "tcp" ];
+        visibleTo.lan = true;
+        visibleTo.wan = true;
+        description = "colin-FTP server data port range";
+      };
+    })
+    (lib.range passiveStart passiveEnd)
+  );
+
+  # use nginx/acme to produce a cert for FTPS
+  services.nginx.virtualHosts."ftp.uninsane.org" = {
+    addSSL = true;
+    enableACME = true;
+  };
+  sane.dns.zones."uninsane.org".inet.CNAME."ftp" = "native";
+
+  services.sftpgo = {
+    enable = true;
+    group = "export";
+
+    package = lib.warnIf (lib.versionOlder "2.5.6" pkgs.sftpgo.version) "sftpgo update: safe to use nixpkgs' sftpgo but keep my own `patches`" pkgs.buildGoModule {
+      inherit (pkgs.sftpgo) name ldflags nativeBuildInputs doCheck subPackages postInstall passthru meta;
+      version = "2.5.6-unstable-2024-04-18";
+      src = pkgs.fetchFromGitHub {
+        # need to use > 2.5.6 for sftpgo_safe_fileinfo.patch to apply
+        owner = "drakkan";
+        repo = "sftpgo";
+        rev = "950cf67e4c03a12c7e439802cabbb0b42d4ee5f5";
+        hash = "sha256-UfiFd9NK3DdZ1J+FPGZrM7r2mo9xlKi0dsSlLEinYXM=";
+      };
+      vendorHash = "sha256-n1/9A2em3BCtFX+132ualh4NQwkwewMxYIMOphJEamg=";
+      patches = (pkgs.sftpgo.patches or []) ++ [
+        # fix for compatibility with kodi:
+        # ftp LIST operation returns entries over-the-wire like:
+        # - dgrwxrwxr-x 1 ftp ftp            9 Apr  9 15:05 Videos
+        # however not all clients understand all mode bits (like that `g`, indicating SGID / group sticky bit).
+        # instead, only send mode bits which are well-understood.
+        # the full set of bits, from which i filter, is found here: <https://pkg.go.dev/io/fs#FileMode>
+        ./safe_fileinfo.patch
+      ];
+    };
+
+    settings = {
+      ftpd = {
+        bindings = [
+          {
+            # binding this means any wireguard client can connect
+            address = "10.0.10.5";
+            port = 21;
+            debug = true;
+          }
+          {
+            # binding this means any LAN client can connect (also WAN traffic forwarded from the gateway)
+            address = "10.78.79.51";
+            port = 21;
+            debug = true;
+          }
+          {
+            # binding this means any wireguard client can connect
+            address = "10.0.10.5";
+            port = 990;
+            debug = true;
+            tls_mode = 2;  # 2 = "implicit FTPS": client negotiates TLS before any FTP command.
+          }
+          {
+            # binding this means any LAN client can connect (also WAN traffic forwarded from the gateway)
+            address = "10.78.79.51";
+            port = 990;
+            debug = true;
+            tls_mode = 2;  # 2 = "implicit FTPS": client negotiates TLS before any FTP command.
+          }
+        ];
+
+        # active mode is susceptible to "bounce attacks", without much benefit over passive mode
+        disable_active_mode = true;
+        hash_support = true;
+        passive_port_range = {
+          start = passiveStart;
+          end = passiveEnd;
+        };
+
+        certificate_file = "/var/lib/acme/ftp.uninsane.org/full.pem";
+        certificate_key_file = "/var/lib/acme/ftp.uninsane.org/key.pem";
+
+        banner = ''
+          Welcome, friends, to Colin's FTP server! Also available via NFS on the same host, but LAN-only.
+
+          Read-only access (LAN-restricted):
+          Username: "anonymous"
+          Password: "anonymous"
+
+          CONFIGURE YOUR CLIENT FOR "PASSIVE" MODE, e.g. `ftp --passive ftp.uninsane.org`.
+          Please let me know if anything's broken or not as it should be. Otherwise, browse and transfer freely :)
+        '';
+
+      };
+      data_provider = {
+        driver = "memory";
+        external_auth_hook = "${external_auth_hook}/bin/external_auth_hook";
+        # track_quota:
+        # - 0: disable quota tracking
+        # - 1: quota is updated on every upload/delete, even if user has no quota restriction
+        # - 2: quota is updated on every upload/delete, but only if user/folder has a quota restriction  (default, i think)
+        # track_quota = 2;
+      };
+    };
+  };
+
+  users.users.sftpgo.extraGroups = [
+    "export"
+    "media"
+    "nginx"  # to access certs
+  ];
+
+  systemd.services.sftpgo = {
+    after = [ "network-online.target" ];
+    wants = [ "network-online.target" ];
+    serviceConfig = {
+      ReadWritePaths = [ "/var/export" ];
+
+      Restart = "always";
+      RestartSec = "20s";
+      UMask = lib.mkForce "0002";
+    };
+  };
+}

hosts/by-name/servo/services/export/sftpgo/external_auth_hook

@@ -0,0 +1,157 @@
+#!/usr/bin/env nix-shell
+#!nix-shell -i python3 -p "python3.withPackages (ps: [  ])"
+# vim: set filetype=python :
+#
+# available environment variables:
+# - SFTPGO_AUTHD_USERNAME
+# - SFTPGO_AUTHD_USER
+# - SFTPGO_AUTHD_IP
+# - SFTPGO_AUTHD_PROTOCOL  = { "DAV", "FTP", "HTTP", "SSH" }
+# - SFTPGO_AUTHD_PASSWORD
+# - SFTPGO_AUTHD_PUBLIC_KEY
+# - SFTPGO_AUTHD_KEYBOARD_INTERACTIVE
+# - SFTPGO_AUTHD_TLS_CERT
+#
+# user permissions:
+# - see <repo:drakkan/sftpgo:internal/dataprovider/user.go>
+# - "*" = grant all permissions
+# - read-only perms:
+#   - "list" = list files and directories
+#   - "download"
+# - rw perms:
+#   - "upload"
+#   - "overwrite" = allow uploads to replace existing files
+#   - "delete" = delete files and directories
+#     - "delete_files"
+#     - "delete_dirs"
+#   - "rename" = rename files and directories
+#     - "rename_files"
+#     - "rename_dirs"
+#   - "create_dirs"
+#   - "create_symlinks"
+#   - "chmod"
+#   - "chown"
+#   - "chtimes" = change atime/mtime (access and modification times)
+#
+# home_dir:
+# - it seems (empirically) that a user can't cd above their home directory.
+#   though i don't have a reference for that in the docs.
+
+import crypt
+import json
+import os
+
+from hmac import compare_digest
+
+authFail = dict(username="")
+
+PERM_RO = [ "list", "download" ]
+PERM_RW = [
+  # read-only:
+  "list",
+  "download",
+  # write:
+  "upload",
+  "overwrite",
+  "delete",
+  "rename",
+  "create_dirs",
+  "create_symlinks",
+  # intentionally omitted:
+  # "chmod",
+  # "chown",
+  # "chtimes",
+]
+
+TRUSTED_CREDS = [
+  # /etc/shadow style creds.
+  # mkpasswd -m sha-512
+  # $<method>$<salt>$<hash>
+  "$6$Zq3c2u4ghUH4S6EP$pOuRt13sEKfX31OqPbbd1LuhS21C9MICMc94iRdTAgdAcJ9h95gQH/6Jf6Ie4Obb0oxQtojRJ1Pd/9QHOlFMW."  #< m. rocket boy
+]
+
+def mkAuthOk(username: str, permissions: dict[str, list[str]]) -> dict:
+  return dict(
+    status = 1,
+    username = username,
+    expiration_date = 0,
+    home_dir = "/var/export",
+    # uid/gid 0 means to inherit sftpgo uid.
+    # - i.e. users can't read files which Linux user `sftpgo` can't read
+    # - uploaded files belong to Linux user `sftpgo`
+    # other uid/gid values aren't possible for localfs backend, unless i let sftpgo use `sudo`.
+    uid = 0,
+    gid = 0,
+    # uid = 65534,
+    # gid = 65534,
+    max_sessions = 0,
+    # quota_*: 0 means to not use SFTP's quota system
+    quota_size = 0,
+    quota_files = 0,
+    permissions = permissions,
+    upload_bandwidth = 0,
+    download_bandwidth = 0,
+    filters = dict(
+      allowed_ip = [],
+      denied_ip = [],
+    ),
+    public_keys = [],
+    # other fields:
+    # ? groups
+    # ? virtual_folders
+  )
+
+def isLan(ip: str) -> bool:
+  return ip.startswith("10.78.76.") \
+    or ip.startswith("10.78.77.") \
+    or ip.startswith("10.78.78.") \
+    or ip.startswith("10.78.79.")
+
+def isWireguard(ip: str) -> bool:
+  return ip.startswith("10.0.10.")
+
+def isTrustedCred(password: str) -> bool:
+  for cred in TRUSTED_CREDS:
+    _, method, salt, hash_ = cred.split("$")
+    # assert method == "6", f"unrecognized crypt entry: {cred}"
+    if crypt.crypt(password, f"${method}${salt}") == cred:
+        return True
+
+  return False
+
+def getAuthResponse(ip: str, username: str, password: str) -> dict:
+  """
+  return a sftpgo auth response either denying the user or approving them
+  with a set of permissions.
+  """
+  if isTrustedCred(password) and username != "colin":
+    # allow r/w access from those with a special token
+    return mkAuthOk(username, permissions = {
+      "/": PERM_RW,
+      "/playground": PERM_RW,
+    })
+  if isWireguard(ip):
+    # allow any user from wireguard
+    return mkAuthOk(username, permissions = {
+      "/": PERM_RW,
+      "/playground": PERM_RW,
+    })
+  if isLan(ip):
+    if username == "anonymous":
+      # allow anonymous users on the LAN
+      return mkAuthOk("anonymous", permissions = {
+        "/": PERM_RO,
+        "/playground": PERM_RW,
+      })
+
+  return authFail
+
+def main():
+  ip = os.environ.get("SFTPGO_AUTHD_IP", "")
+  username = os.environ.get("SFTPGO_AUTHD_USERNAME", "")
+  password = os.environ.get("SFTPGO_AUTHD_PASSWORD", "")
+  resp = getAuthResponse(ip, username, password)
+  print(json.dumps(resp))
+
+if __name__ == "__main__":
+  main()

hosts/by-name/servo/services/export/sftpgo/safe_fileinfo.patch

@@ -0,0 +1,32 @@
+diff --git a/internal/ftpd/handler.go b/internal/ftpd/handler.go
+index 036c3977..33211261 100644
+--- a/internal/ftpd/handler.go
++++ b/internal/ftpd/handler.go
+@@ -169,7 +169,7 @@ func (c *Connection) Stat(name string) (os.FileInfo, error) {
+ 		}
+ 		return nil, err
+ 	}
+-	return fi, nil
++	return vfs.NewFileInfo(name, fi.IsDir(), fi.Size(), fi.ModTime(), false), nil
+ }
+
+ // Name returns the name of this connection
+@@ -315,7 +315,17 @@ func (c *Connection) ReadDir(name string) (ftpserver.DirLister, error) {
+ 		}, nil
+ 	}
+
+-	return c.ListDir(name)
++	lister, err := c.ListDir(name)
++	if err != nil {
++		return nil, err
++	}
++	return &patternDirLister{
++		DirLister:      lister,
++		pattern:        "*",
++		lastCommand:    c.clientContext.GetLastCommand(),
++		dirName:        name,
++		connectionPath: c.clientContext.Path(),
++	}, nil
+ }
+
+ // GetHandle implements ClientDriverExtentionFileTransfer

hosts/by-name/servo/services/export/sftpgo_external_auth_hook

@@ -1,23 +0,0 @@
-#!/usr/bin/env nix-shell
-#!nix-shell -i bash -p coreutils
-# vim: set filetype=bash :
-#
-# available environment variables:
-# - SFTPGO_AUTHD_USERNAME
-# - SFTPGO_AUTHD_USER
-# - SFTPGO_AUTHD_IP
-# - SFTPGO_AUTHD_PROTOCOL  = { "DAV", "FTP", "HTTP", "SSH" }
-# - SFTPGO_AUTHD_PASSWORD
-# - SFTPGO_AUTHD_PUBLIC_KEY
-# - SFTPGO_AUTHD_KEYBOARD_INTERACTIVE
-# - SFTPGO_AUTHD_TLS_CERT
-#
-#
-# call with <script_name> /path/to/fail/response.json /path/to/success/response.json
-
-
-if [ "$SFTPGO_AUTHD_USERNAME" = "anonymous" ]; then
-  cat "$2"
-else
-  cat "$1"
-fi

hosts/by-name/servo/services/freshrss.nix

@@ -16,7 +16,7 @@
     mode = "0400";
   };
   sane.persist.sys.byStore.plaintext = [
-    { user = "freshrss"; group = "freshrss"; path = "/var/lib/freshrss"; }
+    { user = "freshrss"; group = "freshrss"; path = "/var/lib/freshrss"; method = "bind"; }
   ];
 
   services.freshrss.enable = true;

hosts/by-name/servo/services/gitea.nix

@@ -4,7 +4,7 @@
 {
   sane.persist.sys.byStore.plaintext = [
     # TODO: mode? could be more granular
-    { user = "git"; group = "gitea"; path = "/var/lib/gitea"; }
+    { user = "git"; group = "gitea"; path = "/var/lib/gitea"; method = "bind"; }
   ];
   services.gitea.enable = true;
   services.gitea.user = "git";  # default is 'gitea'
@@ -100,6 +100,24 @@
     locations."/" = {
       proxyPass = "http://127.0.0.1:3000";
     };
+    # gitea serves all `raw` files as content-type: plain, but i'd like to serve them as their actual content type.
+    # or at least, enough to make specific pages viewable (serving unoriginal content as arbitrary content type is dangerous).
+    locations."~ ^/colin/phone-case-cq/raw/.*.html" = {
+      proxyPass = "http://127.0.0.1:3000";
+      extraConfig = ''
+        proxy_hide_header Content-Type;
+        default_type text/html;
+        add_header Content-Type text/html;
+      '';
+    };
+    locations."~ ^/colin/phone-case-cq/raw/.*.js" = {
+      proxyPass = "http://127.0.0.1:3000";
+      extraConfig = ''
+        proxy_hide_header Content-Type;
+        default_type text/html;
+        add_header Content-Type text/javascript;
+      '';
+    };
   };
 
   sane.dns.zones."uninsane.org".inet.CNAME."git" = "native";

hosts/by-name/servo/services/goaccess.nix

@@ -20,7 +20,7 @@
           --ignore-panel=HOSTS \
           --ws-url=wss://sink.uninsane.org:443/ws \
           --port=7890 \
-          -o /var/lib/uninsane/sink/index.html
+          -o /var/lib/goaccess/index.html
       '';
       ExecReload = "${pkgs.coreutils}/bin/kill -HUP $MAINPID";
       Type = "simple";
@@ -28,17 +28,19 @@
       RestartSec = "10s";
 
       # hardening
-      WorkingDirectory = "/tmp";
+      # TODO: run as `goaccess` user and add `goaccess` user to group `nginx`.
       NoNewPrivileges = true;
+      PrivateDevices = "yes";
       PrivateTmp = true;
       ProtectHome = "read-only";
-      ProtectSystem = "strict";
-      SystemCallFilter = "~@clock @cpu-emulation @debug @keyring @memlock @module @mount @obsolete @privileged @reboot @resources @setuid @swap @raw-io";
-      ReadOnlyPaths = "/";
-      ReadWritePaths = [ "/proc/self" "/var/lib/uninsane/sink" ];
-      PrivateDevices = "yes";
       ProtectKernelModules = "yes";
       ProtectKernelTunables = "yes";
+      ProtectSystem = "strict";
+      ReadOnlyPaths = [ "/var/log/nginx" ];
+      ReadWritePaths = [ "/proc/self" "/var/lib/goaccess" ];
+      StateDirectory = "goaccess";
+      SystemCallFilter = "~@clock @cpu-emulation @debug @keyring @memlock @module @mount @obsolete @privileged @reboot @resources @setuid @swap @raw-io";
+      WorkingDirectory = "/var/lib/goaccess";
     };
     after = [ "network.target" ];
     wantedBy = [ "multi-user.target" ];
@@ -49,7 +51,7 @@
     addSSL = true;
     enableACME = true;
     # inherit kTLS;
-    root = "/var/lib/uninsane/sink";
+    root = "/var/lib/goaccess";
 
     locations."/ws" = {
       proxyPass = "http://127.0.0.1:7890";

hosts/by-name/servo/services/ipfs.nix

@@ -12,7 +12,7 @@ lib.mkIf false # i don't actively use ipfs anymore
 {
   sane.persist.sys.byStore.plaintext = [
     # TODO: mode? could be more granular
-    { user = "261"; group = "261"; path = "/var/lib/ipfs"; }
+    { user = "261"; group = "261"; path = "/var/lib/ipfs"; method = "bind"; }
   ];
 
   networking.firewall.allowedTCPPorts = [ 4001 ];

hosts/by-name/servo/services/jackett.nix

@@ -1,9 +1,10 @@
-{ ... }:
+{ lib, pkgs, ... }:
 
+lib.mkIf false  #< TODO: re-enable once confident of sandboxing
 {
   sane.persist.sys.byStore.plaintext = [
     # TODO: mode? we only need this to save Indexer creds ==> migrate to config?
-    { user = "root"; group = "root"; path = "/var/lib/jackett"; }
+    { user = "root"; group = "root"; path = "/var/lib/jackett"; method = "bind"; }
   ];
   services.jackett.enable = true;
 
@@ -12,6 +13,8 @@
   systemd.services.jackett.serviceConfig = {
     # run this behind the OVPN static VPN
     NetworkNamespacePath = "/run/netns/ovpns";
+    ExecStartPre = [ "${lib.getExe pkgs.sane-scripts.ip-check} --no-upnp --expect 185.157.162.178" ];  # abort if public IP is not as expected
+
     # patch jackett to listen on the public interfaces
     # ExecStart = lib.mkForce "${pkgs.jackett}/bin/Jackett --NoUpdates --DataFolder /var/lib/jackett/.config/Jackett --ListenPublic";
   };

hosts/by-name/servo/services/jellyfin.nix

@@ -41,7 +41,7 @@
   };
 
   sane.persist.sys.byStore.plaintext = [
-    { user = "jellyfin"; group = "jellyfin"; mode = "0700"; path = "/var/lib/jellyfin"; }
+    { user = "jellyfin"; group = "jellyfin"; mode = "0700"; path = "/var/lib/jellyfin"; method = "bind"; }
   ];
   sane.fs."/var/lib/jellyfin/config/logging.json" = {
     # "Emby.Dlna" logging: <https://jellyfin.org/docs/general/networking/dlna>
@@ -75,7 +75,7 @@
   # Jellyfin multimedia server
   # this is mostly taken from the official jellfin.org docs
   services.nginx.virtualHosts."jelly.uninsane.org" = {
-    addSSL = true;
+    forceSSL = true;
     enableACME = true;
     # inherit kTLS;
 

hosts/by-name/servo/services/kiwix-serve.nix

@@ -7,7 +7,7 @@
 { ... }:
 {
   sane.persist.sys.byStore.ext = [
-    { user = "colin"; group = "users"; path = "/var/lib/kiwix"; }
+    { user = "colin"; group = "users"; path = "/var/lib/kiwix"; method = "bind"; }
   ];
 
   sane.services.kiwix-serve = {

hosts/by-name/servo/services/komga.nix

@@ -5,14 +5,14 @@ let
 in
 {
   sane.persist.sys.byStore.plaintext = [
-    { inherit user group; mode = "0700"; path = stateDir; }
+    { inherit user group; mode = "0700"; path = stateDir; method = "bind"; }
   ];
 
   services.komga.enable = true;
   services.komga.port = 11319;  # chosen at random
 
   services.nginx.virtualHosts."komga.uninsane.org" = {
-    addSSL = true;
+    forceSSL = true;
     enableACME = true;
     locations."/" = {
       proxyPass = "http://127.0.0.1:${builtins.toString port}";

hosts/by-name/servo/services/lemmy.nix

@@ -10,16 +10,21 @@ let
   uiPort = 1234;  # default ui port is 1234
   backendPort = 8536; # default backend port is 8536
   #^ i guess the "backend" port is used for federation?
-  pict-rs = pkgs.pict-rs.overrideAttrs (upstream: {
-    # as of v 0.4.2, all non-GIF video is forcibly transcoded.
-    # that breaks lemmy, because of the request latency.
-    # and it eats up hella CPU.
-    # pict-rs is iffy around video altogether: mp4 seems the best supported.
-    postPatch = (upstream.postPatch or "") + ''
-      substituteInPlace src/validate.rs \
-        --replace 'if transcode_options.needs_reencode() {' 'if false {'
-    '';
-  });
+  pict-rs = pkgs.pict-rs;
+  # pict-rs = pkgs.pict-rs.overrideAttrs (upstream: {
+  #   # as of v0.4.2, all non-GIF video is forcibly transcoded.
+  #   # that breaks lemmy, because of the request latency.
+  #   # and it eats up hella CPU.
+  #   # pict-rs is iffy around video altogether: mp4 seems the best supported.
+  #   # XXX: this patch no longer applies after 0.5.10 -> 0.5.11 update.
+  #   #      git log is hard to parse, but *suggests* that video is natively supported
+  #   #      better than in the 0.4.2 days, e.g. 5fd59fc5b42d31559120dc28bfef4e5002fb509e
+  #   #      "Change commandline flag to allow disabling video, since it is enabled by default"
+  #   postPatch = (upstream.postPatch or "") + ''
+  #     substituteInPlace src/validate.rs \
+  #       --replace-fail 'if transcode_options.needs_reencode() {' 'if false {'
+  #   '';
+  # });
 in {
   services.lemmy = {
     enable = true;
@@ -78,8 +83,8 @@ in {
   # CLI args: <https://git.asonix.dog/asonix/pict-rs#user-content-running>
   systemd.services.pict-rs.serviceConfig.ExecStart = lib.mkForce (lib.concatStringsSep " " [
     "${lib.getBin pict-rs}/bin/pict-rs run"
-    "--media-max-frame-count" (builtins.toString (30*60*60))
+    "--media-video-max-frame-count" (builtins.toString (30*60*60))
     "--media-process-timeout 120"
-    "--media-enable-full-video true"  # allow audio
+    "--media-video-allow-audio"  # allow audio
   ]);
 }

hosts/by-name/servo/services/matrix/default.nix

@@ -21,7 +21,7 @@
   ];
 
   sane.persist.sys.byStore.plaintext = [
-    { user = "matrix-synapse"; group = "matrix-synapse"; path = "/var/lib/matrix-synapse"; }
+    { user = "matrix-synapse"; group = "matrix-synapse"; path = "/var/lib/matrix-synapse"; method = "bind"; }
   ];
   services.matrix-synapse.enable = true;
   services.matrix-synapse.settings = {

hosts/by-name/servo/services/matrix/discord-puppet.nix

@@ -6,7 +6,7 @@
 lib.mkIf false
 {
   sane.persist.sys.byStore.plaintext = [
-    { user = "matrix-synapse"; group = "matrix-synapse"; path = "/var/lib/mx-puppet-discord"; }
+    { user = "matrix-synapse"; group = "matrix-synapse"; path = "/var/lib/mx-puppet-discord"; method = "bind"; }
   ];
 
   services.matrix-synapse.settings.app_service_config_files = [

hosts/by-name/servo/services/matrix/irc.nix

@@ -103,7 +103,7 @@ in
 
   sane.persist.sys.byStore.plaintext = [
     # TODO: mode?
-    { user = "matrix-appservice-irc"; group = "matrix-appservice-irc"; path = "/var/lib/matrix-appservice-irc"; }
+    { user = "matrix-appservice-irc"; group = "matrix-appservice-irc"; path = "/var/lib/matrix-appservice-irc"; method = "bind"; }
   ];
 
   # XXX: matrix-appservice-irc PreStart tries to chgrp the registration.yml to matrix-synapse,

hosts/by-name/servo/services/matrix/signal.nix

@@ -5,8 +5,8 @@
 lib.mkIf false  # disabled 2024/01/11: i don't use it, and pkgs.mautrix-signal had some API changes
 {
   sane.persist.sys.byStore.plaintext = [
-    { user = "mautrix-signal"; group = "mautrix-signal"; path = "/var/lib/mautrix-signal"; }
-    { user = "signald"; group = "signald"; path = "/var/lib/signald"; }
+    { user = "mautrix-signal"; group = "mautrix-signal"; path = "/var/lib/mautrix-signal"; method = "bind"; }
+    { user = "signald"; group = "signald"; path = "/var/lib/signald"; method = "bind"; }
   ];
 
   # allow synapse to read the registration file

hosts/by-name/servo/services/navidrome.nix

@@ -1,15 +1,16 @@
 { lib, ... }:
 
+lib.mkIf false  #< i don't actively use navidrome
 {
   sane.persist.sys.byStore.plaintext = [
-    { user = "navidrome"; group = "navidrome"; path = "/var/lib/navidrome"; }
+    { user = "navidrome"; group = "navidrome"; path = "/var/lib/navidrome"; method = "bind"; }
   ];
   services.navidrome.enable = true;
   services.navidrome.settings = {
     # docs: https://www.navidrome.org/docs/usage/configuration-options/
     Address = "127.0.0.1";
     Port = 4533;
-    MusicFolder = "/var/lib/uninsane/media/Music";
+    MusicFolder = "/var/media/Music";
     CovertArtPriority = "*.jpg, *.JPG, *.png, *.PNG, embedded";
     AutoImportPlaylists = false;
     ScanSchedule = "@every 1h";

hosts/by-name/servo/services/nginx.nix

@@ -55,8 +55,8 @@ in
 
   # web blog/personal site
   # alternative way to link stuff into the share:
-  # sane.fs."/var/lib/uninsane/share/Ubunchu".mount.bind = "/var/lib/uninsane/media/Books/Visual/HiroshiSeo/Ubunchu";
-  # sane.fs."/var/lib/uninsane/media/Books/Visual/HiroshiSeo/Ubunchu".dir = {};
+  # sane.fs."/var/www/sites/uninsane.org/share/Ubunchu".mount.bind = "/var/media/Books/Visual/HiroshiSeo/Ubunchu";
+  # sane.fs."/var/media/Books/Visual/HiroshiSeo/Ubunchu".dir = {};
   services.nginx.virtualHosts."uninsane.org" = publog {
     # a lot of places hardcode https://uninsane.org,
     # and then when we mix http + non-https, we get CORS violations
@@ -89,6 +89,16 @@ in
         disable_symlinks on;
       '';
     };
+    locations."/share/Milkbags/" = {
+      alias = "/var/media/Videos/Milkbags/";
+      extraConfig = ''
+        # autoindex => render directory listings
+        autoindex on;
+        # don't follow any symlinks when serving files
+        # otherwise it allows a directory escape
+        disable_symlinks on;
+      '';
+    };
 
     # allow matrix users to discover that @user:uninsane.org is reachable via matrix.uninsane.org
     locations."= /.well-known/matrix/server".extraConfig =
@@ -169,8 +179,8 @@ in
   security.acme.defaults.email = "admin.acme@uninsane.org";
 
   sane.persist.sys.byStore.plaintext = [
-    { user = "acme"; group = "acme"; path = "/var/lib/acme"; }
-    { user = "colin"; group = "users"; path = "/var/www/sites"; }
+    { user = "acme"; group = "acme"; path = "/var/lib/acme"; method = "bind"; }
+    { user = "colin"; group = "users"; path = "/var/www/sites"; method = "bind"; }
   ];
 
   # let's encrypt default chain looks like:

hosts/by-name/servo/services/nixserve.nix

@@ -1,21 +0,0 @@
-{ config, ... }:
-
-{
-  services.nginx.virtualHosts."nixcache.uninsane.org" = {
-    addSSL = true;
-    enableACME = true;
-    # inherit kTLS;
-    # serverAliases = [ "nixcache" ];
-    locations."/".extraConfig = ''
-      proxy_pass http://localhost:${toString config.services.nix-serve.port};
-      proxy_set_header Host $host;
-      proxy_set_header X-Real-IP $remote_addr;
-      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-    '';
-  };
-
-  sane.dns.zones."uninsane.org".inet.CNAME."nixcache" = "native";
-
-  sane.services.nixserve.enable = true;
-  sane.services.nixserve.secretKeyFile = config.sops.secrets.nix_serve_privkey.path;
-}

hosts/by-name/servo/services/ntfy/ntfy-sh.nix

@@ -34,7 +34,7 @@ in
     # not 100% necessary to persist this, but ntfy does keep a 12hr (by default) cache
     # for pushing notifications to users who become offline.
     # ACLs also live here.
-    { user = "ntfy-sh"; group ="ntfy-sh"; path = "/var/lib/ntfy-sh"; }
+    { user = "ntfy-sh"; group ="ntfy-sh"; path = "/var/lib/ntfy-sh"; method = "bind"; }
   ];
 
   services.ntfy-sh.enable = true;

hosts/by-name/servo/services/ntfy/ntfy-waiter.nix

@@ -49,7 +49,7 @@ in
       type = types.package;
       default = pkgs.static-nix-shell.mkPython3Bin {
         pname = "ntfy-waiter";
-        src = ./.;
+        srcRoot = ./.;
         pkgs = [ "ntfy-sh" ];
       };
       description = ''

hosts/by-name/servo/services/pict-rs.nix

@@ -6,7 +6,7 @@ let
 in
 {
   sane.persist.sys.byStore.plaintext = lib.mkIf cfg.enable [
-    { user = "pict-rs"; group = "pict-rs"; path = cfg.dataDir; }
+    { user = "pict-rs"; group = "pict-rs"; path = cfg.dataDir; method = "bind"; }
   ];
 
   systemd.services.pict-rs.serviceConfig = {

hosts/by-name/servo/services/pleroma.nix

@@ -15,7 +15,7 @@ let
 in
 {
   sane.persist.sys.byStore.plaintext = [
-    { user = "pleroma"; group = "pleroma"; path = "/var/lib/pleroma"; }
+    { user = "pleroma"; group = "pleroma"; path = "/var/lib/pleroma"; method = "bind"; }
   ];
   services.pleroma.enable = true;
   services.pleroma.secretConfigFile = config.sops.secrets.pleroma_secrets.path;
@@ -25,7 +25,7 @@ in
 
     config :pleroma, Pleroma.Web.Endpoint,
       url: [host: "fed.uninsane.org", scheme: "https", port: 443],
-      http: [ip: {127, 0, 0, 1}, port: 4000]
+      http: [ip: {127, 0, 0, 1}, port: 4040]
     #   secret_key_base: "{secrets.pleroma.secret_key_base}",
     #   signing_salt: "{secrets.pleroma.signing_salt}"
 
@@ -167,7 +167,7 @@ in
     enableACME = true;
     # inherit kTLS;
     locations."/" = {
-      proxyPass = "http://127.0.0.1:4000";
+      proxyPass = "http://127.0.0.1:4040";
       recommendedProxySettings = true;
       # documented: https://git.pleroma.social/pleroma/pleroma/-/blob/develop/installation/pleroma.nginx
       extraConfig = ''

hosts/by-name/servo/services/postgres.nix

@@ -8,7 +8,7 @@ in
 {
   sane.persist.sys.byStore.plaintext = [
     # TODO: mode?
-    { user = "postgres"; group = "postgres"; path = "/var/lib/postgresql"; }
+    { user = "postgres"; group = "postgres"; path = "/var/lib/postgresql"; method = "bind"; }
   ];
   services.postgresql.enable = true;
 

hosts/by-name/servo/services/prosody/default.nix

@@ -57,7 +57,7 @@ let
 in
 {
   sane.persist.sys.byStore.plaintext = [
-    { user = "prosody"; group = "prosody"; path = "/var/lib/prosody"; }
+    { user = "prosody"; group = "prosody"; path = "/var/lib/prosody"; method = "bind"; }
   ];
   sane.ports.ports."5000" = {
     protocol = [ "tcp" ];

hosts/by-name/servo/services/slskd.nix

@@ -3,10 +3,15 @@
 #
 # config precedence (higher precedence overrules lower precedence):
 # - Default Values < Environment Variables < YAML Configuraiton File < Command Line Arguments
-{ config, lib, ... }:
+#
+# debugging:
+# - soulseek is just *flaky*. if you see e.g. DNS errors, even though you can't replicate them via `dig` or `getent ahostsv4`, just give it 10 minutes to work out:
+#   - "Soulseek.AddressException: Failed to resolve address 'vps.slsknet.org': Resource temporarily unavailable"
+{ config, lib, pkgs, ... }:
+
 {
   sane.persist.sys.byStore.plaintext = [
-    { user = "slskd"; group = "slskd"; path = "/var/lib/slskd"; }
+    { user = "slskd"; group = "media"; path = "/var/lib/slskd"; method = "bind"; }
   ];
   sops.secrets."slskd_env" = {
     owner = config.users.users.slskd.name;
@@ -15,7 +20,7 @@
 
   users.users.slskd.extraGroups = [ "media" ];
 
-  sane.ports.ports."50000" = {
+  sane.ports.ports."50300" = {
     protocol = [ "tcp" ];
     # not visible to WAN: i run this in a separate netns
     visibleTo.ovpn = true;
@@ -28,12 +33,14 @@
     forceSSL = true;
     enableACME = true;
     locations."/" = {
-      proxyPass = "http://10.0.1.6:5001";
+      proxyPass = "http://10.0.1.6:5030";
       proxyWebsockets = true;
     };
   };
 
   services.slskd.enable = true;
+  services.slskd.domain = null;  # i'll manage nginx for it
+  services.slskd.group = "media";
   # env file, for auth (SLSKD_SLSK_PASSWORD, SLSKD_SLSK_USERNAME)
   services.slskd.environmentFile = config.sops.secrets.slskd_env.path;
   services.slskd.settings = {
@@ -44,13 +51,13 @@
       # [Alias]/path/on/disk
       # NOTE: Music library is quick to scan; videos take a solid 10min to scan.
       # TODO: re-enable the other libraries
-      # "[Audioooks]/var/lib/uninsane/media/Books/Audiobooks"
-      # "[Books]/var/lib/uninsane/media/Books/Books"
-      # "[Manga]/var/lib/uninsane/media/Books/Visual"
-      # "[games]/var/lib/uninsane/media/games"
-      "[Music]/var/lib/uninsane/media/Music"
-      # "[Film]/var/lib/uninsane/media/Videos/Film"
-      # "[Shows]/var/lib/uninsane/media/Videos/Shows"
+      # "[Audioooks]/var/media/Books/Audiobooks"
+      # "[Books]/var/media/Books/Books"
+      # "[Manga]/var/media/Books/Visual"
+      # "[games]/var/media/games"
+      "[Music]/var/media/Music"
+      # "[Film]/var/media/Videos/Film"
+      # "[Shows]/var/media/Videos/Shows"
     ];
     # directories.downloads = "..." # TODO
     # directories.incomplete = "..." # TODO
@@ -62,13 +69,12 @@
     # flags.volatile = true;  # store searches and active transfers in RAM (completed transfers still go to disk). rec for btrfs/zfs
   };
 
-  systemd.services.slskd = {
-    serviceConfig = {
-      # run this behind the OVPN static VPN
-      NetworkNamespacePath = "/run/netns/ovpns";
-      Restart = lib.mkForce "always";  # exits "success" when it fails to connect to soulseek server
-      RestartSec = "60s";
-      Group = "media";
-    };
+  systemd.services.slskd.serviceConfig = {
+    # run this behind the OVPN static VPN
+    NetworkNamespacePath = "/run/netns/ovpns";
+    ExecStartPre = [ "${lib.getExe pkgs.sane-scripts.ip-check} --no-upnp --expect 185.157.162.178" ];  # abort if public IP is not as expected
+
+    Restart = lib.mkForce "always";  # exits "success" when it fails to connect to soulseek server
+    RestartSec = "60s";
   };
 }

hosts/by-name/servo/services/transmission.nix

@@ -1,14 +1,97 @@
-{ config, pkgs, ... }:
+{ config, lib, pkgs, ... }:
 
+let
+  # 2023/09/06: nixpkgs `transmission` defaults to old 3.00
+  # 2024/02/15: some torrent trackers whitelist clients; everyone is still on 3.00 for some reason :|
+  #             some do this via peer-id (e.g. baka); others via user-agent (e.g. MAM).
+  #             peer-id format is essentially the same between 3.00 and 4.x (just swap the MAJOR/MINOR/PATCH numbers).
+  #             user-agent format has changed. `Transmission/3.00` (old) v.s. `TRANSMISSION/MAJ.MIN.PATCH` (new).
+  realTransmission = pkgs.transmission_4;
+  realVersion = {
+    major = lib.versions.major realTransmission.version;
+    minor = lib.versions.minor realTransmission.version;
+    patch = lib.versions.patch realTransmission.version;
+  };
+  package = realTransmission.overrideAttrs (upstream: {
+    # `cmakeFlags = [ "-DTR_VERSION_MAJOR=3" ]`, etc, doesn't seem to take effect.
+    postPatch = (upstream.postPatch or "") + ''
+      substituteInPlace CMakeLists.txt \
+        --replace-fail 'TR_VERSION_MAJOR "${realVersion.major}"' 'TR_VERSION_MAJOR "3"' \
+        --replace-fail 'TR_VERSION_MINOR "${realVersion.minor}"' 'TR_VERSION_MINOR "0"' \
+        --replace-fail 'TR_VERSION_PATCH "${realVersion.patch}"' 'TR_VERSION_PATCH "0"' \
+        --replace-fail 'set(TR_USER_AGENT_PREFIX "''${TR_SEMVER}")' 'set(TR_USER_AGENT_PREFIX "3.00")'
+    '';
+  });
+  download-dir = "/var/media/torrents";
+  torrent-done = pkgs.writeShellApplication {
+    name = "torrent-done";
+    runtimeInputs = with pkgs; [
+      acl
+      coreutils
+      findutils
+      rsync
+      util-linux
+    ];
+    text = ''
+      destructive() {
+        if [ -n "''${TR_DRY_RUN-}" ]; then
+          echo "$*"
+        else
+          "$@"
+        fi
+      }
+      if [[ "$TR_TORRENT_DIR" =~ ^.*freeleech.*$ ]]; then
+        # freeleech torrents have no place in my permanent library
+        echo "freeleech: nothing to do"
+        exit 0
+      fi
+      if ! [[ "$TR_TORRENT_DIR" =~ ^${download-dir}/.*$ ]]; then
+        echo "unexpected torrent dir, aborting: $TR_TORRENT_DIR"
+        exit 0
+      fi
+
+      REL_DIR="''${TR_TORRENT_DIR#${download-dir}/}"
+      MEDIA_DIR="/var/media/$REL_DIR"
+
+      destructive mkdir -p "$(dirname "$MEDIA_DIR")"
+      destructive rsync -arv "$TR_TORRENT_DIR/" "$MEDIA_DIR/"
+      # make the media rwx by anyone in the group
+      destructive find "$MEDIA_DIR" -type d -exec setfacl --recursive --modify d:g::rwx,o::rx {} \;
+      destructive find "$MEDIA_DIR" -type d -exec chmod g+rw,a+rx {} \;
+
+      # if there's a single directory inside the media dir, then inline that
+      subdirs=("$MEDIA_DIR"/*)
+      if [ ''${#subdirs} -eq 1 ]; then
+        dirname="''${subdirs[0]}"
+        if [ -d "$dirname" ]; then
+          mv "$dirname"/* "$MEDIA_DIR/" && rmdir "$dirname"
+        fi
+      fi
+
+      # remove noisy files:
+      find "$MEDIA_DIR/" -type f \(\
+           -iname 'www.YTS.*.jpg' \
+        -o -iname 'WWW.YIFY*.COM.jpg' \
+        -o -iname 'YIFY*.com.txt' \
+        -o -iname 'YTS*.com.txt' \
+        \) -exec rm {} \;
+
+      # dedupe the whole media library.
+      # yeah, a bit excessive: move this to a cron job if that's problematic.
+      destructive hardlink /var/media --reflink=always --ignore-time --verbose
+    '';
+  };
+in
+lib.mkIf false  #< TODO: re-enable once confident of sandboxing
 {
   sane.persist.sys.byStore.plaintext = [
     # TODO: mode? we need this specifically for the stats tracking in .config/
-    { user = "transmission"; group = config.users.users.transmission.group; path = "/var/lib/transmission"; }
+    { user = "transmission"; group = config.users.users.transmission.group; path = "/var/lib/transmission"; method = "bind"; }
   ];
   users.users.transmission.extraGroups = [ "media" ];
 
   services.transmission.enable = true;
-  services.transmission.package = pkgs.transmission_4;  #< 2023/09/06: nixpkgs `transmission` defaults to old 3.00
+  services.transmission.package = package;
   #v setting `group` this way doesn't tell transmission to `chown` the files it creates
   #  it's a nixpkgs setting which just runs the transmission daemon as this group
   services.transmission.group = "media";
@@ -20,9 +103,11 @@
   ];
 
   services.transmission.settings = {
+    # DOCUMENTATION/options list: <https://github.com/transmission/transmission/blob/main/docs/Editing-Configuration-Files.md#options>
+
     # message-level = 3;  #< enable for debug logging. 0-3, default is 2.
-    # 0.0.0.0 => allow rpc from any host: we gate it via firewall and auth requirement
-    rpc-bind-address = "0.0.0.0";
+    # 10.0.1.6 => allow rpc only from the root servo ns. it'll tunnel things to the net, if need be.
+    rpc-bind-address = "10.0.1.6";
     #rpc-host-whitelist = "bt.uninsane.org";
     #rpc-whitelist = "*.*.*.*";
     rpc-authentication-required = true;
@@ -32,6 +117,10 @@
     rpc-password = "{503fc8928344f495efb8e1f955111ca5c862ce0656SzQnQ5";
     rpc-whitelist-enabled = false;
 
+    # force behind ovpns in case the NetworkNamespace fails somehow
+    bind-address-ipv4 = "185.157.162.178";
+    port-forwarding-enabled = false;
+
     # hopefully, make the downloads world-readable
     # umask = 0;  #< default is 2: i.e. deny writes from world
 
@@ -47,11 +136,23 @@
     # see: https://git.zknt.org/mirror/transmission/commit/cfce6e2e3a9b9d31a9dafedd0bdc8bf2cdb6e876?lang=bg-BG
     anti-brute-force-enabled = false;
 
-    download-dir = "/var/lib/uninsane/media";
-    incomplete-dir = "/var/lib/uninsane/media/incomplete";
+    inherit download-dir;
+    incomplete-dir = "${download-dir}/incomplete";
     # transmission regularly fails to move stuff from the incomplete dir to the main one, so disable:
-    # TODO: uncomment this line!
     incomplete-dir-enabled = false;
+
+    # env vars available in script:
+    # - TR_APP_VERSION - Transmission's short version string, e.g. `4.0.0`
+    # - TR_TIME_LOCALTIME
+    # - TR_TORRENT_BYTES_DOWNLOADED - Number of bytes that were downloaded for this torrent
+    # - TR_TORRENT_DIR - Location of the downloaded data
+    # - TR_TORRENT_HASH - The torrent's info hash
+    # - TR_TORRENT_ID
+    # - TR_TORRENT_LABELS - A comma-delimited list of the torrent's labels
+    # - TR_TORRENT_NAME - Name of torrent (not filename)
+    # - TR_TORRENT_TRACKERS - A comma-delimited list of the torrent's trackers' announce URLs
+    script-torrent-done-enabled = true;
+    script-torrent-done-filename = "${torrent-done}/bin/torrent-done";
   };
 
   systemd.services.transmission.after = [ "wireguard-wg-ovpns.service" ];
@@ -59,8 +160,11 @@
   systemd.services.transmission.serviceConfig = {
     # run this behind the OVPN static VPN
     NetworkNamespacePath = "/run/netns/ovpns";
+    ExecStartPre = [ "${lib.getExe pkgs.sane-scripts.ip-check} --no-upnp --expect 185.157.162.178" ];  # abort if public IP is not as expected
+
     Restart = "on-failure";
     RestartSec = "30s";
+    BindPaths = [ "/var/media" ];  #< so it can move completed torrents into the media library
   };
 
   # service to automatically backup torrents i add to transmission

hosts/by-name/servo/services/trust-dns.nix

@@ -2,19 +2,11 @@
 { config, lib, pkgs, ... }:
 
 let
+  dyn-dns = config.sane.services.dyn-dns;
   nativeAddrs = lib.mapAttrs (_name: builtins.head) config.sane.dns.zones."uninsane.org".inet.A;
   bindOvpn = "10.0.1.5";
-in lib.mkMerge [
+in
 {
-  services.trust-dns.enable = true;
-
-  # don't bind to IPv6 until i explicitly test that stack
-  services.trust-dns.settings.listen_addrs_ipv6 = [];
-  services.trust-dns.quiet = true;
-  # FIXME(2023/11/26): services.trust-dns.debug doesn't log requests: use RUST_LOG=debug env for that.
-  # - see: <https://github.com/hickory-dns/hickory-dns/issues/2082>
-  # services.trust-dns.debug = true;
-
   sane.ports.ports."53" = {
     protocol = [ "udp" "tcp" ];
     visibleTo.lan = true;
@@ -66,23 +58,6 @@ in lib.mkMerge [
 
   services.trust-dns.settings.zones = [ "uninsane.org" ];
 
-  # TODO: can i transform this into some sort of service group?
-  # have `systemctl restart trust-dns.service` restart all the individual services?
-  systemd.services.trust-dns.serviceConfig = {
-    DynamicUser = lib.mkForce false;
-    User = "trust-dns";
-    Group = "trust-dns";
-    wantedBy = lib.mkForce [];
-  };
-  systemd.services.trust-dns.enable = false;
-
-  users.groups.trust-dns = {};
-  users.users.trust-dns = {
-    group = "trust-dns";
-    isSystemUser = true;
-  };
-
-  # sane.services.dyn-dns.restartOnChange = [ "trust-dns.service" ];
 
   networking.nat.enable = true;
   networking.nat.extraCommands = ''
@@ -107,98 +82,73 @@ in lib.mkMerge [
     visibleTo.lan = true;
     description = "colin-redirected-dns-for-lan-namespace";
   };
-}
-{
-  systemd.services =
-    let
-      sed = "${pkgs.gnused}/bin/sed";
-      stateDir = "/var/lib/trust-dns";
-      zoneTemplate = pkgs.writeText "uninsane.org.zone.in" config.sane.dns.zones."uninsane.org".rendered;
 
-      zoneDirFor = flavor: "${stateDir}/${flavor}";
-      zoneFor = flavor: "${zoneDirFor flavor}/uninsane.org.zone";
-      mkTrustDnsService = opts: flavor: let
-        flags = let baseCfg = config.services.trust-dns; in
-          (lib.optional baseCfg.debug "--debug") ++ (lib.optional baseCfg.quiet "--quiet");
-        flagsStr = builtins.concatStringsSep " " flags;
 
-        anative = nativeAddrs."servo.${flavor}";
-
-        toml = pkgs.formats.toml { };
-        configTemplate = opts.config or (toml.generate "trust-dns-${flavor}.toml" (
-          (
-            lib.filterAttrsRecursive (_: v: v != null) config.services.trust-dns.settings
-          ) // {
-            listen_addrs_ipv4 = opts.listen or [ anative ];
-          }
-        ));
-        configFile = "${stateDir}/${flavor}-config.toml";
-
-        port = opts.port or 53;
-      in {
-        description = "trust-dns Domain Name Server (serving ${flavor})";
-        unitConfig.Documentation = "https://trust-dns.org/";
-
-        preStart = ''
-          wan=$(cat '${config.sane.services.dyn-dns.ipPath}')
-          ${sed} s/%AWAN%/$wan/ ${configTemplate} > ${configFile}
-        '' + lib.optionalString (!opts ? config) ''
-          mkdir -p ${zoneDirFor flavor}
-          ${sed} \
-            -e s/%CNAMENATIVE%/servo.${flavor}/ \
-            -e s/%ANATIVE%/${anative}/ \
-            -e s/%AWAN%/$wan/ \
-            -e s/%AOVPNS%/185.157.162.178/ \
-            ${zoneTemplate} > ${zoneFor flavor}
-        '';
-        serviceConfig = config.systemd.services.trust-dns.serviceConfig // {
-          ExecStart = ''
-            ${pkgs.trust-dns}/bin/${pkgs.trust-dns.meta.mainProgram} \
-            --port ${builtins.toString port} \
-            --zonedir ${zoneDirFor flavor}/ \
-            --config ${configFile} ${flagsStr}
-          '';
-        };
-
-        after = [ "network.target" ];
-        wantedBy = [ "multi-user.target" ];
-      };
-    in {
-      trust-dns-wan = mkTrustDnsService { listen = [ nativeAddrs."servo.lan" bindOvpn ]; } "wan";
-      trust-dns-lan = mkTrustDnsService { port = 1053; } "lan";
-      trust-dns-hn = mkTrustDnsService { port = 1053; } "hn";
-      trust-dns-hn-resolver = mkTrustDnsService {
-        config = pkgs.writeText "hn-resolver-config.toml" ''
-          # i host a resolver in the wireguard VPN so that clients can resolve DNS through the VPN.
-          # (that's what this file achieves).
-          #
-          # one would expect this resolver could host the authoritative zone for `uninsane.org`, and then forward everything else to the system resolver...
-          # and while that works for `dig`, it breaks for `nslookup` (and so `ssh`, etc).
-          #
-          # DNS responses include a flag for if the responding server is the authority of the zone queried.
-          # it seems that default Linux stub resolvers either:
-          # - expect DNSSEC when the response includes that bit, or
-          # - expect A records to be in the `answer` section instead of `additional` section.
-          # or perhaps something more nuanced. but for `nslookup` to be reliable, it has to talk to an
-          # instance of trust-dns which is strictly a resolver, with no authority.
-          # hence, this config: a resolver which forwards to the actual authority.
-
-          listen_addrs_ipv4 = ["${nativeAddrs."servo.hn"}"]
-          listen_addrs_ipv6 = []
-
-          [[zones]]
-          zone = "uninsane.org"
-          zone_type = "Forward"
-          stores = { type = "forward", name_servers = [{ socket_addr = "${nativeAddrs."servo.hn"}:1053", protocol = "udp", trust_nx_responses = true }] }
-
-          [[zones]]
-          # forward the root zone to the local DNS resolver
-          zone = "."
-          zone_type = "Forward"
-          stores = { type = "forward", name_servers = [{ socket_addr = "127.0.0.53:53", protocol = "udp", trust_nx_responses = true }] }
-        '';
-      } "hn-resolver";
+  sane.services.trust-dns.enable = true;
+  sane.services.trust-dns.instances = let
+    mkSubstitutions = flavor: {
+      "%AWAN%" = "$(cat '${dyn-dns.ipPath}')";
+      "%CNAMENATIVE%" = "servo.${flavor}";
+      "%ANATIVE%" = nativeAddrs."servo.${flavor}";
+      "%AOVPNS%" = "185.157.162.178";
     };
+  in
+  {
+    wan = {
+      substitutions = mkSubstitutions "wan";
+      listenAddrsIpv4 = [
+        nativeAddrs."servo.lan"
+        bindOvpn
+      ];
+    };
+    lan = {
+      substitutions = mkSubstitutions "lan";
+      listenAddrsIpv4 = [ nativeAddrs."servo.lan" ];
+      port = 1053;
+    };
+    hn = {
+      substitutions = mkSubstitutions "hn";
+      listenAddrsIpv4 = [ nativeAddrs."servo.hn" ];
+      port = 1053;
+    };
+    # hn-resolver = {
+    #   # don't need %AWAN% here because we forward to the hn instance.
+    #   listenAddrsIpv4 = [ nativeAddrs."servo.hn" ];
+    #   extraConfig = {
+    #     zones = [
+    #       {
+    #         zone = "uninsane.org";
+    #         zone_type = "Forward";
+    #         stores = {
+    #           type = "forward";
+    #           name_servers = [
+    #             {
+    #               socket_addr = "${nativeAddrs."servo.hn"}:1053";
+    #               protocol = "udp";
+    #               trust_nx_responses = true;
+    #             }
+    #           ];
+    #         };
+    #       }
+    #       {
+    #         # forward the root zone to the local DNS resolver
+    #         zone = ".";
+    #         zone_type = "Forward";
+    #         stores = {
+    #           type = "forward";
+    #           name_servers = [
+    #             {
+    #               socket_addr = "127.0.0.53:53";
+    #               protocol = "udp";
+    #               trust_nx_responses = true;
+    #             }
+    #           ];
+    #         };
+    #       }
+    #     ];
+    #   };
+    # };
+  };
 
   sane.services.dyn-dns.restartOnChange = [
     "trust-dns-wan.service"
@@ -207,4 +157,3 @@ in lib.mkMerge [
     # "trust-dns-hn-resolver.service"  # doesn't need restart because it doesn't know about WAN IP
   ];
 }
-]

hosts/common/default.nix

@@ -1,4 +1,4 @@
-{ lib, pkgs, ... }:
+{ config, lib, pkgs, ... }:
 {
   imports = [
     ./feeds.nix
@@ -9,11 +9,13 @@
     ./ids.nix
     ./machine-id.nix
     ./net
-    ./nix-path
+    ./nix
     ./persist.nix
+    ./polyunfill.nix
     ./programs
     ./secrets.nix
     ./ssh.nix
+    ./systemd.nix
     ./users
   ];
 
@@ -30,63 +32,6 @@
   # time.timeZone = "America/Los_Angeles";
   time.timeZone = "Etc/UTC";  # DST is too confusing for me => use a stable timezone
 
-  nix.extraOptions = ''
-    # see: `man nix.conf`
-    # useful when a remote builder has a faster internet connection than me
-    builders-use-substitutes = true  # default: false
-    # maximum seconds to wait when connecting to binary substituter
-    connect-timeout = 3  # default: 0
-    # download-attempts = 5  # default: 5
-    # allow `nix flake ...` command
-    experimental-features = nix-command flakes
-    # whether to build from source when binary substitution fails
-    fallback = true  # default: false
-    # whether to keep building dependencies if any other one fails
-    keep-going = true  # default: false
-    # whether to keep build-only dependencies of GC roots (e.g. C compiler) when doing GC
-    keep-outputs = true  # default: false
-    # how many lines to show from failed build
-    log-lines = 30  # default: 10
-    # narinfo-cache-negative-ttl = 3600  # default: 3600
-    # whether to use ~/.local/state/nix/profile instead of ~/.nix-profile, etc
-    use-xdg-base-directories = true  # default: false
-    # whether to warn if repository has uncommited changes
-    warn-dirty = false  # default: true
-  '';
-  # hardlinks identical files in the nix store to save 25-35% disk space.
-  # unclear _when_ this occurs. it's not a service.
-  # does the daemon continually scan the nix store?
-  # does the builder use some content-addressed db to efficiently dedupe?
-  nix.settings.auto-optimise-store = true;
-  # TODO: see if i can remove this?
-  nix.settings.trusted-users = [ "root" ];
-
-  services.journald.extraConfig = ''
-    # docs: `man journald.conf`
-    # merged journald config is deployed to /etc/systemd/journald.conf
-    [Journal]
-    # disable journal compression because the underlying fs is compressed
-    Compress=no
-  '';
-
-  systemd.services.nix-daemon.serviceConfig = {
-    # the nix-daemon manages nix builders
-    # kill nix-daemon subprocesses when systemd-oomd detects an out-of-memory condition
-    # see:
-    # - nixos PR that enabled systemd-oomd: <https://github.com/NixOS/nixpkgs/pull/169613>
-    # - systemd's docs on these properties: <https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html#ManagedOOMSwap=auto%7Ckill>
-    #
-    # systemd's docs warn that without swap, systemd-oomd might not be able to react quick enough to save the system.
-    # see `man oomd.conf` for further tunables that may help.
-    #
-    # alternatively, apply this more broadly with `systemd.oomd.enableSystemSlice = true` or `enableRootSlice`
-    # TODO: also apply this to the guest user's slice (user-1100.slice)
-    # TODO: also apply this to distccd
-    ManagedOOMMemoryPressure = "kill";
-    ManagedOOMSwap = "kill";
-  };
-
-
   system.activationScripts.nixClosureDiff = {
     supportsDryActivation = true;
     text = ''
@@ -98,37 +43,6 @@
       fi
     '';
   };
-  system.activationScripts.notifyActive = {
-    text = ''
-      # send a notification to any sway users logged in, that the system has been activated/upgraded.
-      # this probably doesn't work if more than one sway session exists on the system.
-      _notifyActiveSwaySock="$(echo /run/user/*/sway-ipc.*.sock)"
-      if [ -e "$_notifyActiveSwaySock" ]; then
-        SWAYSOCK="$_notifyActiveSwaySock" ${pkgs.sway}/bin/swaymsg -- exec \
-          "${pkgs.libnotify}/bin/notify-send 'nixos activated' 'version: $(cat $systemConfig/nixos-version)'"
-      fi
-    '';
-  };
-
-  # disable non-required packages like nano, perl, rsync, strace
-  environment.defaultPackages = [];
-
-  # dconf docs: <https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/desktop_migration_and_administration_guide/profiles>
-  # this lets programs temporarily write user-level dconf settings (aka gsettings).
-  # they're written to ~/.config/dconf/user, unless `DCONF_PROFILE` is set to something other than the default of /etc/dconf/profile/user
-  # find keys/values with `dconf dump /`
-  programs.dconf.enable = true;
-  programs.dconf.packages = [
-    (pkgs.writeTextFile {
-      name = "dconf-user-profile";
-      destination = "/etc/dconf/profile/user";
-      text = ''
-        user-db:user
-        system-db:site
-      '';
-    })
-  ];
-  # sane.programs.glib.enableFor.user.colin = true;  # for `gsettings`
 
   # link debug symbols into /run/current-system/sw/lib/debug
   # hopefully picked up by gdb automatically?

hosts/common/feeds.nix

@@ -1,4 +1,5 @@
 # where to find good stuff?
+# - universal search/directory: <https://podcastindex.org>
 # - podcasts w/ a community: <https://lemmyverse.net/communities?query=podcast>
 # - podcast rec thread: <https://lemmy.ml/post/1565858>
 #
@@ -72,24 +73,25 @@ let
     (fromDb "feeds.feedburner.com/80000HoursPodcast" // rat)
     (fromDb "feeds.feedburner.com/dancarlin/history" // rat)
     (fromDb "feeds.feedburner.com/radiolab" // pol)  # Radiolab -- also available here, but ONLY OVER HTTP: <http://feeds.wnyc.org/radiolab>
-    (fromDb "feeds.libsyn.com/421877" // rat)  # Less Wrong Curated
     (fromDb "feeds.megaphone.fm/behindthebastards" // pol)  # also Maggie Killjoy
-    # (fromDb "feeds.megaphone.fm/hubermanlab" // uncat)  # Daniel Huberman on sleep
     (fromDb "feeds.megaphone.fm/recodedecode" // tech)  # The Verge - Decoder
     (fromDb "feeds.simplecast.com/54nAGcIl" // pol)  # The Daily
     (fromDb "feeds.simplecast.com/82FI35Px" // pol)  # Ezra Klein Show
     (fromDb "feeds.simplecast.com/wgl4xEgL" // rat)  # Econ Talk
     (fromDb "feeds.simplecast.com/xKJ93w_w" // uncat)  # Atlas Obscura
-    # (fromDb "feeds.simplecast.com/l2i9YnTd" // tech // pol)  # Hard Fork (NYtimes tech)
     (fromDb "feeds.transistor.fm/acquired" // tech)
+    (fromDb "fulltimenix.com" // tech)
+    (fromDb "futureofcoding.org/episodes" // tech)
+    (fromDb "hackerpublicradio.org" // tech)
     (fromDb "lexfridman.com/podcast" // rat)
     (fromDb "mapspodcast.libsyn.com" // uncat)  # Multidisciplinary Association for Psychedelic Studies
+    (fromDb "microarch.club" // tech)
     (fromDb "omegataupodcast.net" // tech)  # 3/4 German; 1/4 eps are English
     (fromDb "omny.fm/shows/cool-people-who-did-cool-stuff" // pol)  # Maggie Killjoy -- referenced by Cory Doctorow
+    (fromDb "omny.fm/shows/money-stuff-the-podcast")  # Matt Levine
     (fromDb "omny.fm/shows/the-dollop-with-dave-anthony-and-gareth-reynolds")  # The Dollop history/comedy
     (fromDb "originstories.libsyn.com" // uncat)
     (fromDb "podcast.posttv.com/itunes/post-reports.xml" // pol)
-    # (fromDb "podcast.thelinuxexp.com" // tech)  # low-brow linux/foss PR announcements
     (fromDb "politicalorphanage.libsyn.com" // pol)
     (fromDb "reverseengineering.libsyn.com/rss" // tech)  # UnNamed Reverse Engineering Podcast
     (fromDb "rss.acast.com/deconstructed")  # The Intercept - Deconstructed
@@ -102,14 +104,21 @@ let
     (fromDb "sharkbytes.transistor.fm" // tech)  # Wireshark Podcast o_0
     (fromDb "sscpodcast.libsyn.com" // rat)  # Astral Codex Ten
     (fromDb "talesfromthebridge.buzzsprout.com" // tech)  # Sci-Fi? has Peter Watts; author of No Moods, Ads or Cutesy Fucking Icons (rifters.com)
+    (fromDb "theamphour.com" // tech)
+    (fromDb "techtalesshow.com" // tech)  # Corbin Davenport
     (fromDb "techwontsave.us" // pol)  # rec by Cory Doctorow
-    # (fromDb "trashfuturepodcast.podbean.com" // pol)  # rec by Cory Doctorow, but way rambly
     (fromDb "wakingup.libsyn.com" // pol)  # Sam Harris
     (fromDb "werenotwrong.fireside.fm" // pol)
+    (mkPod "https://sfconservancy.org/casts/the-corresponding-source/feeds/ogg/" // tech)
 
+    # (fromDb "feeds.libsyn.com/421877" // rat)  # Less Wrong Curated
+    # (fromDb "feeds.megaphone.fm/hubermanlab" // uncat)  # Daniel Huberman on sleep
+    # (fromDb "feeds.simplecast.com/l2i9YnTd" // tech // pol)  # Hard Fork (NYtimes tech)
+    # (fromDb "podcast.thelinuxexp.com" // tech)  # low-brow linux/foss PR announcements
     # (fromDb "rss.art19.com/your-welcome" // pol)  # Michael Malice - Your Welcome -- also available here: <https://origin.podcastone.com/podcast?categoryID2=2232>
     # (fromDb "rss.prod.firstlook.media/deconstructed/podcast.rss" // pol)  #< possible URL rot
     # (fromDb "rss.prod.firstlook.media/intercepted/podcast.rss" // pol)  #< possible URL rot
+    # (fromDb "trashfuturepodcast.podbean.com" // pol)  # rec by Cory Doctorow, but way rambly
     # (mkPod "https://anchor.fm/s/21bc734/podcast/rss" // pol // infrequent)  # Emerge: making sense of what's next -- <https://www.whatisemerging.com/emergepodcast>
     # (mkPod "https://audioboom.com/channels/5097784.rss" // tech)  # Lateral with Tom Scott
     # (mkPod "https://feeds.megaphone.fm/RUNMED9919162779" // pol // infrequent)  # The Witch Trials of J.K. Rowling: <https://www.thefp.com/witchtrials>
@@ -117,12 +126,14 @@ let
   ];
 
   texts = [
+    (fromDb "acoup.blog/feed")  # history, states. author: <https://historians.social/@bretdevereaux/following>
     (fromDb "amosbbatto.wordpress.com" // tech)
+    (fromDb "anish.lakhwara.com" // tech)
+    (fromDb "apenwarr.ca/log/rss.php" // tech)  # CEO of tailscale
     (fromDb "applieddivinitystudies.com" // rat)
     (fromDb "artemis.sh" // tech)
     (fromDb "ascii.textfiles.com" // tech)  # Jason Scott
     (fromDb "austinvernon.site" // tech)
-    (fromDb "balajis.com" // pol)  # Balaji
     (fromDb "ben-evans.com/benedictevans" // pol)
     (fromDb "bitbashing.io" // tech)
     (fromDb "bitsaboutmoney.com" // uncat)
@@ -133,19 +144,19 @@ let
     (fromDb "blog.thalheim.io" // tech)  # Mic92
     (fromDb "bunniestudios.com" // tech)  # Bunnie Juang
     (fromDb "capitolhillseattle.com" // pol)
-    # (fromDb "drewdevault.com" // tech)
-    # (fromDb "econlib.org" // pol)
     (fromDb "edwardsnowden.substack.com" // pol // text)
     (fromDb "fasterthanli.me" // tech)
     (fromDb "gwern.net" // rat)
+    (fromDb "hardcoresoftware.learningbyshipping.com" // tech)  # Steven Sinofsky
     (fromDb "harihareswara.net" // tech // pol)  # rec by Cory Doctorow
     (fromDb "ianthehenry.com" // tech)
     (fromDb "idiomdrottning.org" // uncat)
     (fromDb "interconnected.org/home/feed" // rat)  # Matt Webb -- engineering-ish, but dreamy
     (fromDb "jeffgeerling.com" // tech)
     (fromDb "jefftk.com" // tech)
+    (fromDb "jwz.org/blog" // tech // pol)  # DNA lounge guy, loooong-time blogger
+    (fromDb "kill-the-newsletter.com/feeds/joh91bv7am2pnznv.xml" // pol)  # Matt Levine - Money Stuff
     (fromDb "kosmosghost.github.io/index.xml" // tech)
-    # (fromDb "lesswrong.com" // rat)
     (fromDb "linmob.net" // tech)
     (fromDb "lwn.net" // tech)
     (fromDb "lynalden.com" // pol)
@@ -153,52 +164,63 @@ let
     (fromDb "mg.lol" // tech)
     (fromDb "mindingourway.com" // rat)
     (fromDb "morningbrew.com/feed" // pol)
+    (fromDb "nixpkgs.news" // tech)
     (fromDb "overcomingbias.com" // rat)  # Robin Hanson
     (fromDb "palladiummag.com" // uncat)
     (fromDb "philosopher.coach" // rat)  # Peter Saint-Andre -- side project of stpeter.im
     (fromDb "pomeroyb.com" // tech)
+    (fromDb "postmarketos.org/blog" // tech)
     (fromDb "preposterousuniverse.com" // rat)  # Sean Carroll
-    (fromDb "profectusmag.com" // uncat)
     (fromDb "project-insanity.org" // tech)  # shared blog by a few NixOS devs, notably onny
     (fromDb "putanumonit.com" // rat)  # mostly dating topics. not advice, or humor, but looking through a social lens
     (fromDb "richardcarrier.info" // rat)
     (fromDb "rifters.com/crawl" // uncat)  # No Moods, Ads or Cutesy Fucking Icons
     (fromDb "righto.com" // tech)  # Ken Shirriff
     (fromDb "rootsofprogress.org" // rat)  # Jason Crawford
+    (fromDb "samuel.dionne-riel.com" // tech)  # SamuelDR
     (fromDb "sagacioussuricata.com" // tech)  # ian (Sanctuary)
     (fromDb "semiaccurate.com" // tech)
     (fromDb "sideways-view.com" // rat)  # Paul Christiano
+    (fromDb "slatecave.net" // tech)
     (fromDb "slimemoldtimemold.com" // rat)
     (fromDb "spectrum.ieee.org" // tech)
     (fromDb "stpeter.im/atom.xml" // pol)
-    # (fromDb "theregister.com" // tech)
+    (fromDb "thediff.co" // pol)  # Byrne Hobart
     (fromDb "thisweek.gnome.org" // tech)
     (fromDb "tuxphones.com" // tech)
     (fromDb "uninsane.org" // tech)
     (fromDb "unintendedconsequenc.es" // rat)
-    # (fromDb "vitalik.ca" // tech)  # moved to vitalik.eth.limo
     (fromDb "vitalik.eth.limo" // tech)  # Vitalik Buterin
-    (fromDb "webcurious.co.uk" // uncat)
+    (fromDb "weekinethereumnews.com" // tech)
+    (fromDb "willow.phantoma.online")  # wizard@xyzzy.link
     (fromDb "xn--gckvb8fzb.com" // tech)
     (mkSubstack "astralcodexten" // rat // daily)  # Scott Alexander
-    (mkSubstack "byrnehobart" // pol // infrequent)
-    # (mkSubstack "doomberg" // tech // weekly)  # articles are all pay-walled
     (mkSubstack "eliqian" // rat // weekly)
     (mkSubstack "oversharing" // pol // daily)
     (mkSubstack "samkriss" // humor // infrequent)
     (mkText "http://benjaminrosshoffman.com/feed" // pol // weekly)
     (mkText "http://boginjr.com/feed" // tech // infrequent)
-    (mkText "https://acoup.blog/feed" // rat // weekly)
-    (mkText "https://anish.lakhwara.com/home.html" // tech // weekly)
     (mkText "https://forum.merveilles.town/rss.xml" // pol // infrequent)  #quality RSS list here: <https://forum.merveilles.town/thread/57/share-your-rss-feeds%21-6/>
-    # (mkText "https://github.com/Kaiteki-Fedi/Kaiteki/commits/master.atom" // tech // infrequent)
     (mkText "https://jvns.ca/atom.xml" // tech // weekly)  # Julia Evans
     (mkText "https://linuxphoneapps.org/blog/atom.xml" // tech // infrequent)
     (mkText "https://nixos.org/blog/announcements-rss.xml" // tech // infrequent)  # more nixos stuff here, but unclear how to subscribe: <https://nixos.org/blog/categories.html>
     (mkText "https://nixos.org/blog/stories-rss.xml" // tech // weekly)
-    # (mkText "https://til.simonwillison.net/tils/feed.atom" // tech // weekly)
-    (mkText "https://www.bloomberg.com/opinion/authors/ARbTQlRLRjE/matthew-s-levine.rss" // pol // weekly)  # Matt Levine
+    (mkText "https://solar.lowtechmagazine.com/posts/index.xml" // tech // weekly)
     (mkText "https://www.stratechery.com/rss" // pol // weekly)  # Ben Thompson
+
+    # (fromDb "balajis.com" // pol)  # Balaji
+    # (fromDb "drewdevault.com" // tech)
+    # (fromDb "econlib.org" // pol)
+    # (fromDb "lesswrong.com" // rat)
+    # (fromDb "profectusmag.com" // pol)  # some conservative/libertarian think tank
+    # (fromDb "thesideview.co" // uncat)  # spiritual journal; RSS items are stubs
+    # (fromDb "theregister.com" // tech)
+    # (fromDb "vitalik.ca" // tech)  # moved to vitalik.eth.limo
+    # (fromDb "webcurious.co.uk" // uncat)  # link aggregator; defunct?
+    # (mkSubstack "doomberg" // tech // weekly)  # articles are all pay-walled
+    # (mkText "https://github.com/Kaiteki-Fedi/Kaiteki/commits/master.atom" // tech // infrequent)
+    # (mkText "https://til.simonwillison.net/tils/feed.atom" // tech // weekly)
+    # (mkText "https://www.bloomberg.com/opinion/authors/ARbTQlRLRjE/matthew-s-levine.rss" // pol // weekly)  # Matt Levine (preview/paywalled)
   ];
 
   videos = [
@@ -208,17 +230,22 @@ let
     (fromDb "youtube.com/@Exurb1a")
     (fromDb "youtube.com/@hbomberguy")
     (fromDb "youtube.com/@JackStauber")
+    (fromDb "youtube.com/@NativLang")
     (fromDb "youtube.com/@PolyMatter")
-    # (fromDb "youtube.com/@rossmanngroup" // pol // tech)  # Louis Rossmann
     (fromDb "youtube.com/@TechnologyConnections" // tech)
     (fromDb "youtube.com/@TheB1M")
     (fromDb "youtube.com/@TomScottGo")
     (fromDb "youtube.com/@Vihart")
     (fromDb "youtube.com/@Vox")
     (fromDb "youtube.com/@Vsauce")
+
+    # (fromDb "youtube.com/@rossmanngroup" // pol // tech)  # Louis Rossmann
   ];
 
   images = [
+    (fromDb "catandgirl.com" // img // humor)
+    (fromDb "davidrevoy.com" // img // art)
+    (fromDb "grumpy.website" // img // humor)
     (fromDb "miniature-calendar.com" // img // art // daily)
     (fromDb "pbfcomics.com" // img // humor)
     (fromDb "poorlydrawnlines.com/feed" // img // humor)

hosts/common/fs.nix

@@ -1,40 +1,73 @@
 # docs
 # - x-systemd options: <https://www.freedesktop.org/software/systemd/man/systemd.mount.html>
+# - fuse options: `man mount.fuse`
 
-{ lib, pkgs, sane-lib, ... }:
+{ config, lib, pkgs, sane-lib, utils, ... }:
 
 let
   fsOpts = rec {
     common = [
       "_netdev"
       "noatime"
-      "user"  # allow any user with access to the device to mount the fs
+      # user: allow any user with access to the device to mount the fs.
+      #       note that this requires a suid `mount` binary; see: <https://zameermanji.com/blog/2022/8/5/using-fuse-without-root-on-linux/>
+      "user"
       "x-systemd.requires=network-online.target"
       "x-systemd.after=network-online.target"
       "x-systemd.mount-timeout=10s"  # how long to wait for mount **and** how long to wait for unmount
     ];
-    auto = [ "x-systemd.automount" ];
-    noauto = [ "noauto" ];  # don't mount as part of remote-fs.target
+    # x-systemd.automount: mount the fs automatically *on first access*.
+    # creates a `path-to-mount.automount` systemd unit.
+    automount = [ "x-systemd.automount" ];
+    # noauto: don't mount as part of remote-fs.target.
+    # N.B.: `remote-fs.target` is a dependency of multi-user.target, itself of graphical.target.
+    # hence, omitting `noauto` can slow down boots.
+    noauto = [ "noauto" ];
+    # lazyMount: defer mounting until first access from userspace.
+    # see: `man systemd.automount`, `man automount`, `man autofs`
+    lazyMount = noauto ++ automount;
     wg = [
       "x-systemd.requires=wireguard-wg-home.service"
       "x-systemd.after=wireguard-wg-home.service"
     ];
 
-    ssh = common ++ [
-      "identityfile=/home/colin/.ssh/id_ed25519"
-      "allow_other"
+    fuse = [
+      "allow_other"  # allow users other than the one who mounts it to access it. needed, if systemd is the one mounting this fs (as root)
+      # allow_root: allow root to access files on this fs (if mounted by non-root, else it can always access them).
+      #             N.B.: if both allow_root and allow_other are specified, then only allow_root takes effect.
+      # "allow_root"
+      # default_permissions: enforce local permissions check. CRUCIAL if using `allow_other`.
+      # w/o this, permissions mode of sshfs is like:
+      # - sshfs runs all remote commands as the remote user.
+      # - if a local user has local permissions to the sshfs mount, then their file ops are sent blindly across the tunnel.
+      # - `allow_other` allows *any* local user to access the mount, and hence any local user can now freely become the remote mapped user.
+      # with default_permissions, sshfs doesn't tunnel file ops from users until checking that said user could perform said op on an equivalent local fs.
       "default_permissions"
     ];
-    sshColin = ssh ++ [
-      "transform_symlinks"
-      "idmap=user"
+    fuseColin = fuse ++ [
       "uid=1000"
       "gid=100"
     ];
-    sshRoot = ssh ++ [
-      # we don't transform_symlinks because that breaks the validity of remote /nix stores
-      "sftp_server=/run/wrappers/bin/sudo\\040/run/current-system/sw/libexec/sftp-server"
+
+    ssh = common ++ fuse ++ [
+      "identityfile=/home/colin/.ssh/id_ed25519"
+      # i *think* idmap=user means that `colin` on `localhost` and `colin` on the remote are actually treated as the same user, even if their uid/gid differs?
+      # i.e., local colin's id is translated to/from remote colin's id on every operation?
+      "idmap=user"
     ];
+    sshColin = ssh ++ fuseColin ++ [
+      # follow_symlinks: remote files which are symlinks are presented to the local system as ordinary files (as the target of the symlink).
+      #                  if the symlink target does not exist, the presentation is unspecified.
+      #                  symlinks which point outside the mount ARE followed. so this is more capable than `transform_symlinks`
+      "follow_symlinks"
+      # symlinks on the remote fs which are absolute paths are presented to the local system as relative symlinks pointing to the expected data on the remote fs.
+      # only symlinks which would point inside the mountpoint are translated.
+      "transform_symlinks"
+    ];
+    # sshRoot = ssh ++ [
+    #   # we don't transform_symlinks because that breaks the validity of remote /nix stores
+    #   "sftp_server=/run/wrappers/bin/sudo\\040/run/current-system/sw/libexec/sftp-server"
+    # ];
     # in the event of hunt NFS mounts, consider:
     # - <https://unix.stackexchange.com/questions/31979/stop-broken-nfs-mounts-from-locking-a-directory>
 
@@ -42,28 +75,103 @@ let
     # actimeo=n  = how long (in seconds) to cache file/dir attributes (default: 3-60s)
     # bg         = retry failed mounts in the background
     # retry=n    = for how many minutes `mount` will retry NFS mount operation
+    # intr       = allow Ctrl+C to abort I/O (it will error with `EINTR`)
     # soft       = on "major timeout", report I/O error to userspace
+    # softreval  = on "major timeout", service the request using known-stale cache results instead of erroring -- if such cache data exists
     # retrans=n  = how many times to retry a NFS request before giving userspace a "server not responding" error (default: 3)
     # timeo=n    = number of *deciseconds* to wait for a response before retrying it (default: 600)
     #              note: client uses a linear backup, so the second request will have double this timeout, then triple, etc.
+    # proto=udp  = encapsulate protocol ops inside UDP packets instead of a TCP session.
+    #              requires `nfsvers=3` and a kernel compiled with `NFS_DISABLE_UDP_SUPPORT=n`.
+    #              UDP might be preferable to TCP because the latter is liable to hang for ~100s (kernel TCP timeout) after a link drop.
+    #              however, even UDP has issues with `umount` hanging.
+    #
+    # N.B.: don't change these without first testing the behavior of sandboxed apps on a flaky network.
     nfs = common ++ [
-      # "actimeo=10"
-      "bg"
-      "retrans=4"
+      # "actimeo=5"
+      # "bg"
+      "retrans=1"
       "retry=0"
+      # "intr"
       "soft"
-      "timeo=15"
+      "softreval"
+      "timeo=30"
       "nofail"  # don't fail remote-fs.target when this mount fails  (not an option for sshfs else would be common)
+      # "proto=udp"  # default kernel config doesn't support NFS over UDP: <https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1964093> (see comment 11).
+      # "nfsvers=3"  # NFSv4+ doesn't support UDP at *all*. it's ok to omit nfsvers -- server + client will negotiate v3 based on udp requirement. but omitting causes confusing mount errors when the server is *offline*, because the client defaults to v4 and thinks the udp option is a config error.
+      # "x-systemd.idle-timeout=10"  # auto-unmount after this much inactivity
+    ];
+
+    # manually perform a ftp mount via e.g.
+    #   curlftpfs -o ftpfs_debug=2,user=anonymous:anonymous,connect_timeout=10 -f -s ftp://servo-hn /mnt/my-ftp
+    ftp = common ++ fuseColin ++ [
+      # "ftpfs_debug=2"
+      "user=colin:ipauth"
+      # connect_timeout=10: casting shows to T.V. fails partway through about half the time
+      "connect_timeout=20"
     ];
   };
   remoteHome = host: {
-    fileSystems."/mnt/${host}-home" = {
+    sane.programs.sshfs-fuse.enableFor.system = true;
+    fileSystems."/mnt/${host}/home" = {
       device = "colin@${host}:/home/colin";
       fsType = "fuse.sshfs";
-      options = fsOpts.sshColin ++ fsOpts.noauto;
+      options = fsOpts.sshColin ++ fsOpts.lazyMount;
       noCheck = true;
     };
-    sane.fs."/mnt/${host}-home" = sane-lib.fs.wantedDir;
+    sane.fs."/mnt/${host}/home" = sane-lib.fs.wanted {
+      dir.acl.user = "colin";
+      dir.acl.group = "users";
+      dir.acl.mode = "0700";
+    };
+  };
+  remoteServo = subdir: {
+    sane.programs.curlftpfs.enableFor.system = true;
+    sane.fs."/mnt/servo/${subdir}" = sane-lib.fs.wanted {
+      dir.acl.user = "colin";
+      dir.acl.group = "users";
+      dir.acl.mode = "0750";
+    };
+    fileSystems."/mnt/servo/${subdir}" = {
+      device = "ftp://servo-hn:/${subdir}";
+      noCheck = true;
+      fsType = "fuse.curlftpfs";
+      options = fsOpts.ftp ++ fsOpts.noauto ++ fsOpts.wg;
+      # fsType = "nfs";
+      # options = fsOpts.nfs ++ fsOpts.lazyMount ++ fsOpts.wg;
+    };
+    systemd.services."automount-servo-${utils.escapeSystemdPath subdir}" = let
+      fs = config.fileSystems."/mnt/servo/${subdir}";
+    in {
+      # this is a *flaky* network mount, especially on moby.
+      # if done as a normal autofs mount, access will eternally block when network is dropped.
+      # notably, this would block *any* sandboxed app which allows media access, whether they actually try to use that media or not.
+      # a practical solution is this: mount as a service -- instead of autofs -- and unmount on timeout error, in a restart loop.
+      # until the ftp handshake succeeds, nothing is actually mounted to the vfs, so this doesn't slow down any I/O when network is down.
+      description = "automount /mnt/servo/${subdir} in a fault-tolerant and non-blocking manner";
+      after = [ "network-online.target" ];
+      requires = [ "network-online.target" ];
+      wantedBy = [ "default.target" ];
+
+      serviceConfig.Type = "simple";
+      serviceConfig.ExecStart = lib.escapeShellArgs [
+        "/usr/bin/env"
+        "PATH=/run/current-system/sw/bin"
+        "mount.${fs.fsType}"
+        "-f"  # foreground (i.e. don't daemonize)
+        "-s"  # single-threaded (TODO: it's probably ok to disable this?)
+        "-o"
+        (lib.concatStringsSep "," (lib.filter (o: !lib.hasPrefix "x-systemd." o) fs.options))
+        fs.device
+        "/mnt/servo/${subdir}"
+      ];
+      # not sure if this configures a linear, or exponential backoff.
+      # but the first restart will be after `RestartSec`, and the n'th restart (n = RestartSteps) will be RestartMaxDelaySec after the n-1'th exit.
+      serviceConfig.Restart = "always";
+      serviceConfig.RestartSec = "10s";
+      serviceConfig.RestartMaxDelaySec = "120s";
+      serviceConfig.RestartSteps = "5";
+    };
   };
 in
 lib.mkMerge [
@@ -99,45 +207,30 @@ lib.mkMerge [
     # but it decreases working memory under the heaviest of loads by however much space the compressed memory occupies (e.g. 50% if 2:1; 25% if 4:1)
     zramSwap.memoryPercent = 100;
 
-    # fileSystems."/mnt/servo-nfs" = {
-    #   device = "servo-hn:/";
-    #   noCheck = true;
-    #   fsType = "nfs";
-    #   options = fsOpts.nfs ++ fsOpts.auto ++ fsOpts.wg;
-    # };
-    fileSystems."/mnt/servo-nfs/media" = {
-      device = "servo-hn:/media";
-      noCheck = true;
-      fsType = "nfs";
-      options = fsOpts.nfs ++ fsOpts.auto ++ fsOpts.wg;
-    };
-    fileSystems."/mnt/servo-nfs/playground" = {
-      device = "servo-hn:/playground";
-      noCheck = true;
-      fsType = "nfs";
-      options = fsOpts.nfs ++ fsOpts.auto ++ fsOpts.wg;
-    };
-    # fileSystems."/mnt/servo-media-nfs" = {
-    #   device = "servo-hn:/media";
-    #   noCheck = true;
-    #   fsType = "nfs";
-    #   options = fsOpts.common ++ fsOpts.auto;
-    # };
-    sane.fs."/mnt/servo-media" = sane-lib.fs.wantedSymlinkTo "/mnt/servo-nfs/media";
+    # environment.pathsToLink = [
+    #   # needed to achieve superuser access for user-mounted filesystems (see sshRoot above)
+    #   # we can only link whole directories here, even though we're only interested in pkgs.openssh
+    #   "/libexec"
+    # ];
 
-    environment.pathsToLink = [
-      # needed to achieve superuser access for user-mounted filesystems (see optionsRoot above)
-      # we can only link whole directories here, even though we're only interested in pkgs.openssh
-      "/libexec"
-    ];
-
-    environment.systemPackages = [
-      pkgs.sshfs-fuse
-    ];
+    programs.fuse.userAllowOther = true;  #< necessary for `allow_other` or `allow_root` options.
   }
 
   (remoteHome "desko")
   (remoteHome "lappy")
   (remoteHome "moby")
+  # this granularity of servo media mounts is necessary to support sandboxing:
+  # for flaky mounts, we can only bind the mountpoint itself into the sandbox,
+  # so it's either this or unconditionally bind all of media/.
+  (remoteServo "media/archive")
+  (remoteServo "media/Books")
+  (remoteServo "media/collections")
+  # (remoteServo "media/datasets")
+  (remoteServo "media/games")
+  (remoteServo "media/Music")
+  (remoteServo "media/Pictures/macros")
+  (remoteServo "media/torrents")
+  (remoteServo "media/Videos")
+  (remoteServo "playground")
 ]
 

hosts/common/hardware/default.nix

@@ -28,6 +28,23 @@
   #   "systemd.log_level=debug"
   #   "systemd.log_target=console"
 
+  # moby has to run recent kernels (defined elsewhere).
+  # meanwhile, kernel variation plays some minor role in things like sandboxing (landlock) and capabilities.
+  # simpler to keep near the latest kernel on all devices,
+  # and also makes certain that any weird system-level bugs i see aren't likely to be stale kernel bugs.
+  # servo needs zfs though, which doesn't support every kernel.
+  boot.kernelPackages = lib.mkDefault pkgs.zfs.latestCompatibleLinuxPackages;
+
+  # TODO: remove after linux 6.9. see: <https://github.com/axboe/liburing/issues/1113>
+  # - <https://github.com/neovim/neovim/issues/28149>
+  # - <https://git.kernel.dk/cgit/linux/commit/?h=io_uring-6.9&id=e5444baa42e545bb929ba56c497e7f3c73634099>
+  # when removing, try starting and suspending (ctrl+z) two instances of neovim simultaneously.
+  # if the system doesn't freeze, then this is safe to remove.
+  # added 2024-04-04
+  sane.user.fs.".profile".symlink.text = lib.mkBefore ''
+    export UV_USE_IO_URING=0
+  '';
+
   # hack in the `boot.shell_on_fail` arg since that doesn't always seem to work.
   boot.initrd.preFailCommands = "allowShell=1";
 
@@ -63,19 +80,12 @@
   # - query details with `sudo cpupower frequency-info`
   powerManagement.cpuFreqGovernor = "ondemand";
 
-  services.logind.extraConfig = ''
-    # see: `man logind.conf`
-    # don’t shutdown when power button is short-pressed (commonly done an accident, or by cats).
-    #   but do on long-press: useful to gracefully power-off server.
-    HandlePowerKey=lock
-    HandlePowerKeyLongPress=poweroff
-    HandleLidSwitch=lock
-  '';
-
-  # some packages build only if binfmt *isn't* present
-  nix.settings.system-features = lib.mkIf (config.boot.binfmt.emulatedSystems == []) [
-    "no-binfmt"
-  ];
+  # see: `man logind.conf`
+  # don’t shutdown when power button is short-pressed (commonly done an accident, or by cats).
+  #   but do on long-press: useful to gracefully power-off server.
+  services.logind.powerKey = "lock";
+  services.logind.powerKeyLongPress = "poweroff";
+  services.logind.lidSwitch = "lock";
 
   # services.snapper.configs = {
   #   root = {

hosts/common/hardware/x86_64.nix

@@ -7,6 +7,17 @@
       "usb_storage"   # rpi needed this to boot from usb storage, i think.
       "nvme"  # to boot from nvme devices
       # efi_pstore evivars
+
+      # added (speculatively) 2024/05/21; these were implicitly being added by nixos/modules/system/boot/kernel.nix
+      # i've copied not all of them, but most
+      "mmc_block"
+      "dm_mod"
+      # USB keyboards
+      "uhci_hcd" "ehci_hcd" "ehci_pci" "ohci_hcd" "ohci_pci" "xhci_hcd" "xhci_pci" "usbhid" "hid_generic" "hid_lenovo" "hid_apple" "hid_roccat" "hid_logitech_hidpp" "hid_logitech_dj" "hid_microsoft" "hid_cherry" "hid_corsair"
+      # x86 keyboard stuff
+      "pcips2" "atkbd" "i8042"
+      # stage-2 init needs rtc?
+      "rtc_cmos"
     ];
 
     hardware.cpu.amd.updateMicrocode = true;    # desktop

hosts/common/home/default.nix

@@ -1,7 +1,7 @@
 { ... }:
 {
   imports = [
-    ./keyring
+    ./fs.nix
     ./mime.nix
     ./ssh.nix
     ./xdg-dirs.nix

hosts/common/home/fs.nix

@@ -0,0 +1,45 @@
+{ config, lib, ... }:
+{
+  sane.user.persist.byStore.plaintext = [
+    "archive"
+    "dev"
+    # TODO: records should be private
+    "records"
+    "ref"
+    "tmp"
+    "use"
+    "Books/local"
+    "Music"
+    "Pictures/albums"
+    "Pictures/cat"
+    "Pictures/from"
+    "Pictures/Screenshots"  #< XXX: something is case-sensitive about this?
+    "Pictures/Photos"
+    "Videos/local"
+
+    # these are persisted simply to save on RAM.
+    # ~/.cache/nix can become several GB.
+    # mesa_shader_cache is < 10 MB.
+    # TODO: integrate with sane.programs.sandbox?
+    ".cache/mesa_shader_cache"
+    ".cache/nix"
+  ];
+  sane.user.persist.byStore.private = [
+    "knowledge"
+  ];
+
+  # convenience
+  sane.user.fs = let
+    persistEnabled = config.sane.persist.enable;
+  in {
+    ".persist/private" = lib.mkIf persistEnabled { symlink.target = config.sane.persist.stores.private.origin; };
+    ".persist/plaintext" = lib.mkIf persistEnabled { symlink.target = config.sane.persist.stores.plaintext.origin; };
+    ".persist/ephemeral" = lib.mkIf persistEnabled { symlink.target = config.sane.persist.stores.cryptClearOnBoot.origin; };
+
+    "nixos".symlink.target = "dev/nixos";
+
+    "Books/servo".symlink.target = "/mnt/servo/media/Books";
+    "Videos/servo".symlink.target = "/mnt/servo/media/Videos";
+    "Pictures/servo-macros".symlink.target = "/mnt/servo/media/Pictures/macros";
+  };
+}

hosts/common/home/keyring/default.nix

@@ -1,17 +0,0 @@
-{ config, pkgs, sane-lib, ... }:
-
-let
-  init-keyring = pkgs.static-nix-shell.mkBash {
-    pname = "init-keyring";
-    src = ./.;
-  };
-in
-{
-  sane.user.persist.byStore.private = [ ".local/share/keyrings" ];
-
-  sane.user.fs."private/.local/share/keyrings/default" = {
-    generated.command = [ "${init-keyring}/bin/init-keyring" ];
-    wantedBy = [ config.sane.fs."/home/colin/private".unit ];
-    wantedBeforeBy = [ ];  # don't created this as part of `multi-user.target`
-  };
-}

hosts/common/home/keyring/init-keyring

@@ -1,21 +0,0 @@
-#!/usr/bin/env nix-shell
-#!nix-shell -i bash
-# initializes the default libsecret keyring (used by gnome-keyring) if not already initialized.
-# this initializes it to be plaintext/unencrypted.
-
-ringdir=/home/colin/private/.local/share/keyrings
-if test -f "$ringdir/default"
-then
-  echo 'keyring already initialized: not doing anything'
-else
-  keyring="$ringdir/Default_keyring.keyring"
-
-  echo 'initializing default user keyring:' "$keyring.new"
-  echo '[keyring]' > "$keyring.new"
-  echo 'display-name=Default keyring' >> "$keyring.new"
-  echo 'lock-on-idle=false' >> "$keyring.new"
-  echo 'lock-after=false' >> "$keyring.new"
-  chown colin:users "$keyring.new"
-  # closest to an atomic update we can achieve
-  mv "$keyring.new" "$keyring" && echo -n "Default_keyring" > "$ringdir/default"
-fi

hosts/common/home/mime.nix

@@ -1,4 +1,5 @@
-{ config, lib, ...}:
+# TODO: move into modules/users.nix
+{ config, lib, pkgs, ...}:
 
 let
   # [ ProgramConfig ]
@@ -6,6 +7,9 @@ let
     (p: p.enabled)
     (builtins.attrValues config.sane.programs);
 
+  # [ ProgramConfig ]
+  enabledProgramsWithPackage = builtins.filter (p: p.package != null) enabledPrograms;
+
   # [ { "<mime-type>" = { prority, desktop } ]
   enabledWeightedMimes = builtins.map weightedMimes enabledPrograms;
 
@@ -21,23 +25,70 @@ let
 
   # [ { priority, desktop } ... ] -> Self
   sortOneMimeType = associations: builtins.sort
-    (l: r: assert l.priority != r.priority; l.priority < r.priority)
+    (l: r: lib.throwIf
+      (l.priority == r.priority)
+      "${l.desktop} and ${r.desktop} share a preferred mime type with identical priority ${builtins.toString l.priority} (and so the desired association is ambiguous)"
+      (l.priority < r.priority)
+    )
     associations;
   sortMimes = mimes: builtins.mapAttrs (_k: sortOneMimeType) mimes;
+  # { "<mime-type>"} = [ { priority, desktop } ... ]; } -> { "<mime-type>" = [ "<desktop>" ... ]; }
   removePriorities = mimes: builtins.mapAttrs
     (_k: associations: builtins.map (a: a.desktop) associations)
     mimes;
+  # { "<mime-type>" = [ "<desktop>" ... ]; } -> { "<mime-type>" = "<desktop1>;<desktop2>;..."; }
+  formatDesktopLists = mimes: builtins.mapAttrs
+    (_k: desktops: lib.concatStringsSep ";" desktops)
+    mimes;
+
+  mimeappsListPkg = pkgs.writeTextDir "share/applications/mimeapps.list" (
+    lib.generators.toINI { } {
+      "Default Applications" = formatDesktopLists (removePriorities (sortMimes (mergeMimes enabledWeightedMimes)));
+    }
+  );
+
+  localShareApplicationsPkg = (pkgs.symlinkJoin {
+    name = "user-local-share-applications";
+    paths = builtins.map
+      (p: builtins.toString p.package)
+      (enabledProgramsWithPackage ++ [ { package=mimeappsListPkg; } ]);
+  }).overrideAttrs (orig: {
+    # like normal symlinkJoin, but don't error if the path doesn't exist
+    buildCommand = ''
+      mkdir -p $out/share/applications
+      for i in $(cat $pathsPath); do
+        if [ -e "$i/share/applications" ]; then
+          ${pkgs.buildPackages.xorg.lndir}/bin/lndir -silent $i/share/applications $out/share/applications
+        fi
+      done
+      runHook postBuild
+    '';
+    postBuild = ''
+      # rebuild `mimeinfo.cache`, used by file openers to show the list of *all* apps, not just the user's defaults.
+      ${pkgs.buildPackages.desktop-file-utils}/bin/update-desktop-database $out/share/applications
+    '';
+  });
+
 in
 {
   # the xdg mime type for a file can be found with:
   # - `xdg-mime query filetype path/to/thing.ext`
   # the default handler for a mime type can be found with:
   # - `xdg-mime query default <mimetype>`  (e.g. x-scheme-handler/http)
+  # the nix-configured handler can be found `nix-repl > :lf . > hostConfigs.desko.xdg.mime.defaultApplications`
+  #
+  # glib/gio is queried via glib.bin output:
+  # - `gio mime x-scheme-handler/https`
+  # - `gio open <path_or_url>`
+  # - `gio launch </path/to/app.desktop>`
   #
   # we can have single associations or a list of associations.
   # there's also options to *remove* [non-default] associations from specific apps
-  xdg.mime.enable = true;
-  xdg.mime.defaultApplications = removePriorities (sortMimes (mergeMimes enabledWeightedMimes));
-
+  # N.B.: don't use nixos' `xdg.mime` option becaue that caues `/share/applications` to be linked into the whole system,
+  # which limits what i can do around sandboxing. getting the default associations to live in ~/ makes it easier to expose
+  # the associations to apps selectively.
+  # xdg.mime.enable = true;
+  # xdg.mime.defaultApplications = removePriorities (sortMimes (mergeMimes enabledWeightedMimes));
 
+  sane.user.fs.".local/share/applications".symlink.target = "${localShareApplicationsPkg}/share/applications";
 }

hosts/common/home/xdg-dirs.nix

@@ -3,13 +3,18 @@
 {
   # XDG defines things like ~/Desktop, ~/Downloads, etc.
   # these clutter the home, so i mostly don't use them.
+  # note that several of these are not actually standardized anywhere.
+  # some are even non-conventional, like:
+  # - XDG_PHOTOS_DIR: only works because i patch e.g. megapixels
   sane.user.fs.".config/user-dirs.dirs".symlink.text = ''
     XDG_DESKTOP_DIR="$HOME/.xdg/Desktop"
     XDG_DOCUMENTS_DIR="$HOME/dev"
     XDG_DOWNLOAD_DIR="$HOME/tmp"
     XDG_MUSIC_DIR="$HOME/Music"
+    XDG_PHOTOS_DIR="$HOME/Pictures/Photos"
     XDG_PICTURES_DIR="$HOME/Pictures"
     XDG_PUBLICSHARE_DIR="$HOME/.xdg/Public"
+    XDG_SCREENSHOTS_DIR="$HOME/Pictures/Screenshots"
     XDG_TEMPLATES_DIR="$HOME/.xdg/Templates"
     XDG_VIDEOS_DIR="$HOME/Videos"
   '';
@@ -17,4 +22,12 @@
   # prevent `xdg-user-dirs-update` from overriding/updating our config
   # see <https://manpages.ubuntu.com/manpages/bionic/man5/user-dirs.conf.5.html>
   sane.user.fs.".config/user-dirs.conf".symlink.text = "enabled=False";
+
+  sane.user.fs.".profile".symlink.text = ''
+    # configure XDG_<type>_DIR preferences (e.g. for downloads, screenshots, etc)
+    # surround with `set -o allexport` since user-dirs.dirs doesn't `export` its vars
+    set -a
+    source $HOME/.config/user-dirs.dirs
+    set +a
+  '';
 }

hosts/common/hosts.nix

@@ -36,10 +36,4 @@
     wg-home.endpoint = "uninsane.org:51820";
     lan-ip = "10.78.79.51";
   };
-
-  sane.hosts.by-name."supercap" = {
-    ssh.authorized = false;
-    ssh.host_pubkey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHf/mqqkX45EWAcquV04MC3SUljTApdclH1gjI19F+PA";
-    lan-ip = "10.78.79.232";
-  };
 }

hosts/common/ids.nix

@@ -57,6 +57,8 @@
   sane.ids.bitcoind-mainnet.gid = 2418;
   sane.ids.clightning.uid = 2419;
   sane.ids.clightning.gid = 2419;
+  sane.ids.nix-serve.uid = 2420;
+  sane.ids.nix-serve.gid = 2420;
 
   sane.ids.colin.uid = 1000;
   sane.ids.guest.uid = 1100;

hosts/common/net/default.nix

@@ -7,6 +7,24 @@
     ./upnp.nix
     ./vpn.nix
   ];
+
+  systemd.network.enable = true;
+  networking.useNetworkd = true;
+
+  # view refused/dropped packets with: `sudo journalctl -k`
+  # networking.firewall.logRefusedPackets = true;
+  # networking.firewall.logRefusedUnicastsOnly = false;
+  networking.firewall.logReversePathDrops = true;
+  # linux will drop inbound packets if it thinks a reply to that packet wouldn't exit via the same interface (rpfilter).
+  #   that heuristic fails for complicated VPN-style routing, especially with SNAT.
+  # networking.firewall.checkReversePath = false;  # or "loose" to keep it partially.
+  # networking.firewall.enable = false;  #< set false to debug
+
+  # this is needed to forward packets from the VPN to the host.
+  # this is required separately by servo and by any `sane-vpn` users,
+  # however Nix requires this be set centrally, in only one location (i.e. here)
+  boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
+
   # the default backend is "wpa_supplicant".
   # wpa_supplicant reliably picks weak APs to connect to.
   # see: <https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues/474>
@@ -42,8 +60,5 @@
   networking.networkmanager.plugins = lib.mkForce [];
 
   # keyfile.path = where networkmanager should look for connection credentials
-  networking.networkmanager.extraConfig = ''
-    [keyfile]
-    path=/var/lib/NetworkManager/system-connections
-  '';
+  networking.networkmanager.settings.keyfile.path = "/var/lib/NetworkManager/system-connections";
 }

hosts/common/net/dns.nix

@@ -1,15 +1,39 @@
-{ ... }:
+# things to consider when changing these parameters:
+# - temporary VPN access (`sane-vpn up ...`)
+# - servo `ovpns` namespace (it *relies* on /etc/resolv.conf mentioning 127.0.0.53)
+#
+# components:
+# - /etc/nsswitch.conf:
+#   - glibc uses this to provide `getaddrinfo`, i.e. host -> ip address lookup
+#     call directly with `getent ahostsv4 www.google.com`
+#   - `nss` (a component of glibc) is modular: names mentioned in that file are `dlopen`'d (i think that's the mechanism)
+#     in NixOS, that means _they have to be on LDPATH_.
+#   - `nscd` is used by NixOS simply to proxy nss requests.
+#      here, /etc/nsswitch.conf consumers contact nscd via /var/run/nscd/socket.
+#      in this way, only `nscd` needs to have the nss modules on LDPATH.
+# - /etc/resolv.conf
+#   - contains the DNS servers for a system.
+#   - historically, NetworkManager would update this file as you switch networks.
+#   - modern implementations hardcodes `127.0.0.53` and then systemd-resolved proxies everything (and caches).
+#
+# namespacing:
+# - each namespace may use a different /etc/resolv.conf to specify different DNS servers
+# - nscd breaks namespacing: the host nscd is unaware of the guest's /etc/resolv.conf, and so directs the guest's DNS requests to the host's servers.
+#   - this is fixed by either removing `/var/run/nscd/socket` from the namespace, or disabling nscd altogether.
+{ config, lib, ... }:
+lib.mkMerge [
 {
+  sane.services.trust-dns.enable = lib.mkDefault config.sane.services.trust-dns.asSystemResolver;
+  sane.services.trust-dns.asSystemResolver = lib.mkDefault true;
+}
+(lib.mkIf (!config.sane.services.trust-dns.asSystemResolver) {
   # use systemd's stub resolver.
   # /etc/resolv.conf isn't sophisticated enough to use different servers per net namespace (or link).
   # instead, running the stub resolver on a known address in the root ns lets us rewrite packets
-  # in the ovnps namespace to use the provider's DNS resolvers.
+  # in servo's ovnps namespace to use the provider's DNS resolvers.
   # a weakness is we can only query 1 NS at a time (unless we were to clone the packets?)
-  # there also seems to be some cache somewhere that's shared between the two namespaces.
-  # i think this is a libc thing. might need to leverage proper cgroups to _really_ kill it.
-  # - getent ahostsv4 www.google.com
-  # - try fix: <https://serverfault.com/questions/765989/connect-to-3rd-party-vpn-server-but-dont-use-it-as-the-default-route/766290#766290>
-  services.resolved.enable = true;
+  # TODO: improve trust-dns recursive resolver and then remove this
+  services.resolved.enable = true;  #< to disable, set ` = lib.mkForce false`, as other systemd features default to enabling `resolved`.
   # without DNSSEC:
   # - dig matrix.org => works
   # - curl https://matrix.org => works
@@ -24,7 +48,8 @@
     # stub resolver (just forwards upstream) lives on 127.0.0.54
     "127.0.0.53"
   ];
-
+})
+{
   # nscd -- the Name Service Caching Daemon -- caches DNS query responses
   # in a way that's unaware of my VPN routing, so routes are frequently poor against
   # services which advertise different IPs based on geolocation.
@@ -32,6 +57,17 @@
   # nsncd is the Name Service NON-Caching Daemon. it's a drop-in that doesn't cache;
   # this is OK on the host -- because systemd-resolved caches. it's probably sub-optimal
   # in the netns and we query upstream DNS more often than needed. hm.
-  # TODO: run a separate recursive resolver in each namespace.
-  services.nscd.enableNsncd = true;
+  # services.nscd.enableNsncd = true;
+
+  # disabling nscd LOSES US SOME FUNCTIONALITY. in particular, only the glibc-builtin modules are accessible via /etc/resolv.conf.
+  # - dns: glibc-bultin
+  # - files: glibc-builtin
+  # - myhostname: systemd
+  # - mymachines: systemd
+  # - resolve: systemd
+  # in practice, i see no difference with nscd disabled.
+  # disabling nscd VASTLY simplifies netns and process isolation. see explainer at top of file.
+  services.nscd.enable = false;
+  system.nssModules = lib.mkForce [];
 }
+]

hosts/common/net/vpn.nix

@@ -1,89 +1,56 @@
-{ config, lib, pkgs, ... }:
-
 # to add a new OVPN VPN:
 # - generate a privkey `wg genkey`
 # - add this key to `sops secrets/universal.yaml`
-# - upload pubkey to OVPN.com
+# - upload pubkey to OVPN.com (`cat wg.priv | wg pubkey`)
 # - generate config @ OVPN.com
 # - copy the Address, PublicKey, Endpoint from OVPN's config
-# N.B.: maximum interface name in Linux is 15 characters.
+
+{ config, lib, pkgs, ... }:
 let
-  def-wg-vpn = name: { endpoint, publicKey, address, dns, privateKeyFile }: {
-    # networking.wg-quick.interfaces."${name}" = {
-    #   inherit address privateKeyFile dns;
-    #   peers = [
-    #     {
-    #       allowedIPs = [
-    #         "0.0.0.0/0"
-    #         "::/0"
-    #       ];
-    #       inherit endpoint publicKey;
-    #     }
-    #   ];
-    #   # to start: `systemctl start wg-quick-${name}`
-    #   autostart = false;
-    # };
-    systemd.network.netdevs."${name}" = {
-      # see: `man 5 systemd.netdev`
-      wireguardConfig = {
-        PrivateKeyFile = privateKeyFile;
-      };
-      wireguardPeers = [{
-        AllowedIPs = [
-          "0.0.0.0/0"
-          "::/0"
-        ];
-        Endpoint = endpoint;
-        PublicKey = publicKey;
-      }];
+  def-ovpn = name: { endpoint, publicKey, addrV4, id }: {
+    sane.vpn."ovpnd-${name}" = {
+      inherit endpoint publicKey addrV4 id;
+      privateKeyFile = config.sops.secrets."wg/ovpnd_${name}_privkey".path;
+      dns = [
+        "46.227.67.134"
+        "192.165.9.158"
+      ];
     };
-    systemd.network.networks."${name}" = {
-      # see: `man 5 systemd.network`
-      matchConfig.Name = name;
-      networkConfig.Address = address;
-      networkConfig.DNS = dns;
+
+    sops.secrets."wg/ovpnd_${name}_privkey" = {
+      # needs to be readable by systemd-network or else it says "Ignoring network device" and doesn't expose it to networkctl.
+      owner = "systemd-network";
     };
   };
-  def-ovpn = name: { endpoint, publicKey, address }: def-wg-vpn "ovpnd-${name}" {
-    inherit endpoint publicKey address;
-    privateKeyFile = config.sops.secrets."wg/ovpnd_${name}_privkey".path;
-    dns = [
-      "46.227.67.134"
-      "192.165.9.158"
-    ];
-  };
 in lib.mkMerge [
   (def-ovpn "us" {
     endpoint = "vpn31.prd.losangeles.ovpn.com:9929";
     publicKey = "VW6bEWMOlOneta1bf6YFE25N/oMGh1E1UFBCfyggd0k=";
-    address = [
-      "172.27.237.218/32"
-      "fd00:0000:1337:cafe:1111:1111:ab00:4c8f/128"
-    ];
-  })
-  # NB: us-* share the same wg key and link-local addrs, but distinct public addresses
-  (def-ovpn "us-atl" {
-    endpoint = "vpn18.prd.atlanta.ovpn.com:9929";
-    publicKey = "Dpg/4v5s9u0YbrXukfrMpkA+XQqKIFpf8ZFgyw0IkE0=";
-    address = [
-      "172.21.182.178/32"
-      "fd00:0000:1337:cafe:1111:1111:cfcb:27e3/128"
-    ];
+    id = 1;
+    addrV4 = "172.27.237.218";
+    # addrV6 = "fd00:0000:1337:cafe:1111:1111:ab00:4c8f";
   })
+  # TODO: us-atl disabled until i can give it a different link-local address and wireguard key than us-mi
+  # (def-ovpn "us-atl" {
+  #   endpoint = "vpn18.prd.atlanta.ovpn.com:9929";
+  #   publicKey = "Dpg/4v5s9u0YbrXukfrMpkA+XQqKIFpf8ZFgyw0IkE0=";
+  #   address = [
+  #     "172.21.182.178/32"
+  #     "fd00:0000:1337:cafe:1111:1111:cfcb:27e3/128"
+  #   ];
+  # })
   (def-ovpn "us-mi" {
     endpoint = "vpn34.prd.miami.ovpn.com:9929";
     publicKey = "VtJz2irbu8mdkIQvzlsYhU+k9d55or9mx4A2a14t0V0=";
-    address = [
-      "172.21.182.178/32"
-      "fd00:0000:1337:cafe:1111:1111:cfcb:27e3/128"
-    ];
+    id = 2;
+    addrV4 = "172.21.182.178";
+    # addrV6 = "fd00:0000:1337:cafe:1111:1111:cfcb:27e3";
   })
   (def-ovpn "ukr" {
     endpoint = "vpn96.prd.kyiv.ovpn.com:9929";
     publicKey = "CjZcXDxaaKpW8b5As1EcNbI6+42A6BjWahwXDCwfVFg=";
-    address = [
-      "172.18.180.159/32"
-      "fd00:0000:1337:cafe:1111:1111:ec5c:add3/128"
-    ];
+    id = 3;
+    addrV4 = "172.18.180.159";
+    # addrV6 = "fd00:0000:1337:cafe:1111:1111:ec5c:add3";
   })
 ]

hosts/common/nix-path/default.nix

@@ -1,16 +0,0 @@
-{ pkgs, sane-lib, ... }:
-
-{
-  # allow `nix-shell` (and probably nix-index?) to locate our patched and custom packages
-  nix.nixPath = [
-    "nixpkgs=${pkgs.path}"
-    # note the import starts at repo root: this allows `./overlay/default.nix` to access the stuff at the root
-    # "nixpkgs-overlays=${../../..}/hosts/common/nix-path/overlay"
-    # as long as my system itself doesn't rely on NIXPKGS at runtime, we can point the overlays to git
-    # to avoid switching so much during development
-    "nixpkgs-overlays=/home/colin/dev/nixos/hosts/common/nix-path/overlay"
-  ];
-
-  # ensure new deployments have a source of this repo with which they can bootstrap.
-  environment.etc."nixos".source = ../../..;
-}

hosts/common/nix/default.nix

@@ -0,0 +1,91 @@
+{ config, lib, pkgs, ... }:
+
+{
+  nix.settings = {
+    # see: `man nix.conf`
+
+    # useful when a remote builder has a faster internet connection than me.
+    # note that this also applies to `nix copy --to`, though.
+    # i think any time a remote machine wants a path, this means we ask them to try getting it themselves before we supply it.
+    builders-use-substitutes = true;  # default: false
+
+    # maximum seconds to wait when connecting to binary substituter
+    connect-timeout = 3;  # default: 0
+
+    # download-attempts = 5;  # default: 5
+
+    # allow `nix flake ...` command
+    experimental-features = [ "nix-command" "flakes "];
+
+    # whether to build from source when binary substitution fails
+    fallback = true;  # default: false
+
+    # whether to keep building dependencies if any other one fails
+    keep-going = true;  # default: false
+
+    # whether to keep build-only dependencies of GC roots (e.g. C compiler) when doing GC
+    keep-outputs = true;  # default: false
+
+    # how many lines to show from failed build
+    log-lines = 30;  # default: 10
+
+    # how many substitution downloads to perform in parallel.
+    # i wonder if parallelism is causing moby's substitutions to fail?
+    max-substitution-jobs = 6;  # default: 16
+
+    # narinfo-cache-negative-ttl = 3600  # default: 3600
+    # whether to use ~/.local/state/nix/profile instead of ~/.nix-profile, etc
+    use-xdg-base-directories = true;  # default: false
+
+    # whether to warn if repository has uncommited changes
+    warn-dirty = false;  # default: true
+
+    # hardlinks identical files in the nix store to save 25-35% disk space.
+    # unclear _when_ this occurs. it's not a service.
+    # does the daemon continually scan the nix store?
+    # does the builder use some content-addressed db to efficiently dedupe?
+    auto-optimise-store = true;
+
+    # allow #!nix-shell scripts to locate my patched nixpkgs & custom packages.
+    # this line might become unnecessary: see <https://github.com/NixOS/nixpkgs/pull/273170>
+    nix-path = config.nix.nixPath;
+  };
+
+  # allow `nix-shell` (and probably nix-index?) to locate our patched and custom packages.
+  # this is actually a no-op, and the real action happens in assigning `nix.settings.nix-path`.
+  nix.nixPath = (lib.optionals (config.sane.maxBuildCost >= 2) [
+    "nixpkgs=${pkgs.path}"
+  ]) ++ [
+    # note the import starts at repo root: this allows `./overlay/default.nix` to access the stuff at the root
+    # "nixpkgs-overlays=${../../..}/hosts/common/nix-path/overlay"
+    # as long as my system itself doesn't rely on NIXPKGS at runtime, we can point the overlays to git
+    # to avoid switching so much during development
+    "nixpkgs-overlays=/home/colin/dev/nixos/hosts/common/nix/overlay"
+  ];
+
+  # ensure new deployments have a source of this repo with which they can bootstrap.
+  # this however changes on every commit and can be slow to copy for e.g. `moby`.
+  environment.etc."nixos" = lib.mkIf (config.sane.maxBuildCost >= 3) {
+    source = ../../..;
+  };
+  environment.etc."nix/registry.json" = lib.mkIf (config.sane.maxBuildCost < 3) {
+    enable = false;
+  };
+
+  systemd.services.nix-daemon.serviceConfig = {
+    # the nix-daemon manages nix builders
+    # kill nix-daemon subprocesses when systemd-oomd detects an out-of-memory condition
+    # see:
+    # - nixos PR that enabled systemd-oomd: <https://github.com/NixOS/nixpkgs/pull/169613>
+    # - systemd's docs on these properties: <https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html#ManagedOOMSwap=auto%7Ckill>
+    #
+    # systemd's docs warn that without swap, systemd-oomd might not be able to react quick enough to save the system.
+    # see `man oomd.conf` for further tunables that may help.
+    #
+    # alternatively, apply this more broadly with `systemd.oomd.enableSystemSlice = true` or `enableRootSlice`
+    # TODO: also apply this to the guest user's slice (user-1100.slice)
+    # TODO: also apply this to distccd
+    ManagedOOMMemoryPressure = "kill";
+    ManagedOOMSwap = "kill";
+  };
+}

hosts/common/persist.nix

@@ -1,13 +1,14 @@
 { ... }:
 
 {
-  sane.persist.stores.private.origin = "/home/colin/private";
-  # store /home/colin/a/b in /home/private/a/b instead of /home/private/home/colin/a/b
+  # store /home/colin/a/b in /mnt/persist/private/a/b instead of /mnt/persist/private/home/colin/a/b
   sane.persist.stores.private.prefix = "/home/colin";
 
+  sane.persist.sys.byStore.initrd = [
+    "/var/log"
+  ];
   sane.persist.sys.byStore.plaintext = [
     # TODO: these should be private.. somehow
-    "/var/log"
     "/var/backup"  # for e.g. postgres dumps
   ];
   sane.persist.sys.byStore.cryptClearOnBoot = [

hosts/common/polyunfill.nix

@@ -0,0 +1,114 @@
+# strictly *decrease* the scope of the default nixos installation/config
+
+{ lib, ... }:
+{
+  # remove a few items from /run/wrappers we don't need.
+  options.security.wrappers = lib.mkOption {
+    apply = lib.filterAttrs (name: _: !(builtins.elem name [
+      # wrappers from <repo:nixos/nixpkgs:nixos/modules/programs/shadow.nix>
+      "newgidmap"
+      "newgrp"
+      "newuidmap"
+      # "sg"
+      # "su"
+      # wrappers from <repo:nixos/nixpkgs:nixos/modules/security/pam.nix>
+      # may need to patch e.g. `pam` package (pam_unix) to not refer to unix_chkpwd by path
+      "unix_chkpwd"
+    ]));
+  };
+
+  config = {
+    nixpkgs.overlays = [(self: super: {
+      pam = super.pam.overrideAttrs (upstream: {
+        postPatch = (if upstream.postPatch != null then upstream.postPatch else "") + ''
+          substituteInPlace modules/pam_unix/Makefile.am --replace-fail \
+            "/run/wrappers/bin/unix_chkpwd" "$out"
+        '';
+      });
+    })];
+
+    # disable non-required packages like nano, perl, rsync, strace
+    environment.defaultPackages = [];
+
+    # remove all the non-existent default directories from XDG_DATA_DIRS, XDG_CONFIG_DIRS to simplify debugging.
+    # this is defaulted in <repo:nixos/nixpkgs:nixos/modules/programs/environment.nix>,
+    # without being gated by any higher config.
+    environment.profiles = lib.mkForce [
+      "/etc/profiles/per-user/$USER"
+      "/run/current-system/sw"
+    ];
+
+    # NIXPKGS_CONFIG defaults to "/etc/nix/nixpkgs-config.nix" in <nixos/modules/programs/environment.nix>.
+    # that's never existed on my system and everything does fine without it set empty (no nixpkgs API to forcibly *unset* it).
+    environment.variables.NIXPKGS_CONFIG = lib.mkForce "";
+    # XDG_CONFIG_DIRS defaults to "/etc/xdg", which doesn't exist.
+    # in practice, pam appends the values i want to XDG_CONFIG_DIRS, though this approach causes an extra leading `:`
+    environment.sessionVariables.XDG_CONFIG_DIRS = lib.mkForce [];
+    # XCURSOR_PATH: defaults to `[ "$HOME/.icons" "$HOME/.local/share/icons" ]`, neither of which i use, just adding noise.
+    # see: <repo:nixos/nixpkgs:nixos/modules/config/xdg/icons.nix>
+    environment.sessionVariables.XCURSOR_PATH = lib.mkForce [];
+
+    # disable nixos' portal module, otherwise /share/applications gets linked into the system and complicates things (sandboxing).
+    # instead, i manage portals myself via the sane.programs API (e.g. sane.programs.xdg-desktop-portal).
+    xdg.portal.enable = false;
+    xdg.menus.enable = false;  #< links /share/applications, and a bunch of other empty (i.e. unused) dirs
+
+    # xdg.autostart.enable defaults to true, and links /etc/xdg/autostart into the environment, populated with .desktop files.
+    # see: <repo:nixos/nixpkgs:nixos/modules/config/xdg/autostart.nix>
+    # .desktop files are a questionable way to autostart things: i generally prefer a service manager for that.
+    xdg.autostart.enable = false;
+
+    # nix.channel.enable: populates `/nix/var/nix/profiles/per-user/root/channels`, `/root/.nix-channels`, `$HOME/.nix-defexpr/channels`
+    #   <repo:nixos/nixpkgs:nixos/modules/config/nix-channel.nix>
+    # TODO: may want to recreate NIX_PATH, nix.settings.nix-path
+    nix.channel.enable = false;
+
+    # environment.stub-ld: populate /lib/ld-linux.so with an object that unconditionally errors on launch,
+    # so as to inform when trying to run a non-nixos binary?
+    # IMO that's confusing: i thought /lib/ld-linux.so was some file actually required by nix.
+    environment.stub-ld.enable = false;
+
+    # `less.enable` sets LESSKEYIN_SYSTEM, LESSOPEN, LESSCLOSE env vars, which does confusing "lesspipe" things, so disable that.
+    # it's enabled by default from `<nixos/modules/programs/environment.nix>`, who also sets `PAGER="less"` and `EDITOR="nano"` (keep).
+    programs.less.enable = lib.mkForce false;
+    environment.variables.PAGER = lib.mkOverride 900 "";  # mkDefault sets 1000. non-override is 100. 900 will beat the nixpkgs `mkDefault` but not anyone else.
+    environment.variables.EDITOR = lib.mkOverride 900 "";
+
+    # several packages (dconf, modemmanager, networkmanager, gvfs, polkit, udisks, bluez/blueman, feedbackd, etc)
+    # will add themselves to the dbus search path.
+    # i prefer dbus to only search XDG paths (/share/dbus-1) for service files, as that's more introspectable.
+    # see: <repo:nixos/nixpkgs:nixos/modules/services/system/dbus.nix>
+    # TODO: sandbox dbus? i pretty explicitly don't want to use it as a launcher.
+    services.dbus.packages = lib.mkForce [
+      "/run/current-system/sw"
+      # config.system.path
+      # pkgs.dbus
+      # pkgs.polkit.out
+      # pkgs.modemmanager
+      # pkgs.networkmanager
+      # pkgs.udisks
+      # pkgs.wpa_supplicant
+    ];
+
+    # systemd by default forces shitty defaults for e.g. /tmp/.X11-unix.
+    # nixos propagates those in: <nixos/modules/system/boot/systemd/tmpfiles.nix>
+    # by overwriting this with an empty file, we can effectively remove it.
+    environment.etc."tmpfiles.d/x11.conf".text = "# (removed by Colin)";
+
+    # see: <nixos/modules/tasks/swraid.nix>
+    # it was enabled by default before 23.11
+    boot.swraid.enable = lib.mkDefault false;
+
+    # see: <nixos/modules/system/boot/kernel.nix>
+    # by default, it adds to boot.initrd.availableKernelModules:
+    # - SATA: "ahci" "sata_nv" "sata_via" "sata_sis" "sata_uli" "ata_piix" "pata_marvell"
+    # - "nvme"
+    # - scsi: "sd_mod" "sr_mod"
+    # - SD/eMMC: "mmc_block"
+    # - USB keyboards: "uhci_hcd" "ehci_hcd" "ehci_pci" "ohci_hcd" "ohci_pci" "xhci_hcd" "xhci_pci" "usbhid" "hid_generic" "hid_lenovo" "hid_apple" "hid_roccat" "hid_logitech_hidpp" "hid_logitech_dj" "hid_microsoft" "hid_cherry" "hid_corsair"
+    # - LVM: "dm_mod"
+    # - on x86 only: more keyboard stuff: "pcips2" "atkbd" "i8042"
+
+    boot.initrd.includeDefaultModules = lib.mkDefault false;
+  };
+}

hosts/common/programs/abaddon.nix

@@ -15,7 +15,7 @@ in
       };
     };
 
-    package = pkgs.abaddon.overrideAttrs (upstream: {
+    packageUnwrapped = pkgs.abaddon.overrideAttrs (upstream: {
       patches = (upstream.patches or []) ++ [
         (pkgs.fetchpatch {
           url = "https://git.uninsane.org/colin/abaddon/commit/eb551f188d34679f75adcbc83cb8d5beb4d19fd6.patch";
@@ -87,13 +87,8 @@ in
 
     services.abaddon = {
       description = "unofficial Discord chat client";
-      wantedBy = lib.mkIf cfg.config.autostart [ "default.target" ];
-      serviceConfig = {
-        ExecStart = "${cfg.package}/bin/abaddon";
-        Type = "simple";
-        Restart = "always";
-        RestartSec = "20s";
-      };
+      partOf = lib.mkIf cfg.config.autostart [ "graphical-session" ];
+      command = "abaddon";
     };
   };
 }

hosts/common/programs/aerc.nix

@@ -3,6 +3,9 @@
 
 {
   sane.programs.aerc = {
+    sandbox.method = "bwrap";
+    sandbox.wrapperType = "inplace";  #< /share/aerc/aerc.conf refers to other /share files by absolute path
+    sandbox.net = "clearnet";
     secrets.".config/aerc/accounts.conf" = ../../../secrets/common/aerc_accounts.conf.bin;
     mime.associations."x-scheme-handler/mailto" = "aerc.desktop";
   };

hosts/common/programs/alacritty.nix

@@ -3,13 +3,28 @@
 #   - `man 5 alacritty`
 #   - defaults: <https://github.com/alacritty/alacritty/releases>  -> alacritty.yml
 # - irc: #alacritty on libera.chat
-{ lib, ... }:
+{ config, lib, ... }:
+let
+  cfg = config.sane.programs.alacritty;
+in
 {
   sane.programs.alacritty = {
+    configOption = with lib; mkOption {
+      default = {};
+      type = types.submodule {
+        options.fontSize = mkOption {
+          type = types.int;
+          default = 14;
+        };
+      };
+    };
+
+    sandbox.enable = false;
     env.TERMINAL = lib.mkDefault "alacritty";
+
     fs.".config/alacritty/alacritty.toml".symlink.text = ''
       [font]
-      size = 14
+      size = ${builtins.toString cfg.config.fontSize}
 
       [[keyboard.bindings]]
       mods = "Control"
@@ -35,6 +50,21 @@
       mods = "Control|Shift"
       key = "PageDown"
       action = "ScrollPageDown"
+
+      # disable OS shortcuts which leak through...
+      # see sway config or sane-input-handler for more info on why these leak through
+      [[keyboard.bindings]]
+      key = "AudioVolumeUp"
+      action = "None"
+      [[keyboard.bindings]]
+      key = "AudioVolumeDown"
+      action = "None"
+      [[keyboard.bindings]]
+      key = "Power"
+      action = "None"
+      [[keyboard.bindings]]
+      key = "PowerOff"
+      action = "None"
     '';
   };
 }

hosts/common/programs/alsa-ucm-conf/default.nix

@@ -0,0 +1,65 @@
+{ config, lib, pkgs, ... }:
+let
+  cfg = config.sane.programs.alsa-ucm-conf;
+in
+{
+  sane.programs.alsa-ucm-conf = {
+    configOption = with lib; mkOption {
+      default = {};
+      type = types.submodule {
+        options.preferEarpiece = mkOption {
+          type = types.bool;
+          default = true;
+        };
+      };
+    };
+
+    # upstream alsa ships with PinePhone audio configs, but they don't actually produce sound.
+    # see: <https://github.com/alsa-project/alsa-ucm-conf/pull/134>
+    # these audio files come from some revision of:
+    # - <https://gitlab.manjaro.org/manjaro-arm/packages/community/phosh/alsa-ucm-pinephone>
+    #
+    # alternative to patching is to plumb `ALSA_CONFIG_UCM2 = "${./ucm2}"` environment variable into the relevant places
+    # e.g. `systemd.services.pulseaudio.environment`.
+    # that leaves more opportunity for gaps (i.e. missing a service),
+    # on the other hand this method causes about 500 packages to be rebuilt (including qt5 and webkitgtk).
+    #
+    # note that with these files, the following audio device support:
+    # - headphones work.
+    # - "internal earpiece" works.
+    # - "internal speaker" doesn't work (but that's probably because i broke the ribbon cable)
+    # - "analog output" doesn't work.
+    packageUnwrapped = pkgs.alsa-ucm-conf.overrideAttrs (upstream: {
+      postPatch = (upstream.postPatch or "") + ''
+        cp ${./ucm2/PinePhone}/* ucm2/Allwinner/A64/PinePhone/
+
+        # fix the self-contained ucm files i source from to have correct path within the alsa-ucm-conf source tree
+        substituteInPlace ucm2/Allwinner/A64/PinePhone/PinePhone.conf \
+          --replace-fail 'HiFi.conf' '/Allwinner/A64/PinePhone/HiFi.conf'
+        substituteInPlace ucm2/Allwinner/A64/PinePhone/PinePhone.conf \
+          --replace-fail 'VoiceCall.conf' '/Allwinner/A64/PinePhone/VoiceCall.conf'
+      '' + lib.optionalString cfg.config.preferEarpiece ''
+        # decrease the priority of the internal speaker so that sounds are routed
+        # to the earpiece by default.
+        # this is just personal preference.
+        substituteInPlace ucm2/Allwinner/A64/PinePhone/{HiFi.conf,VoiceCall.conf} \
+          --replace-fail 'PlaybackPriority 300' 'PlaybackPriority 100'
+      '';
+    });
+
+    sandbox.enable = false;  #< only provides #out/share/alsa
+
+    # alsa-lib package only looks in its $out/share/alsa to find runtime config data, by default.
+    # but ALSA_CONFIG_UCM2 is an env var that can override that.
+    # this is particularly needed by wireplumber;
+    #   also *maybe* pipewire and pipewire-pulse.
+    # taken from <repo:nixos/mobile-nixos:modules/quirks/audio.nix>
+    env.ALSA_CONFIG_UCM2 = "/run/current-system/sw/share/alsa/ucm2";
+
+    enableFor.system = lib.mkIf (builtins.any (en: en) (builtins.attrValues cfg.enableFor.user)) true;
+  };
+
+  environment.pathsToLink = lib.mkIf cfg.enabled [
+    "/share/alsa/ucm2"
+  ];
+}

hosts/common/programs/animatch.nix

@@ -1,10 +1,43 @@
-{ ... }:
+# debug with:
+# - `animatch --debug`
+# - `gdb animatch`
+# try:
+# - `animatch --fullscreen`
+# - `animatch --windowed`
+# the other config options (e.g. verbose logging -- which doesn't seem to do anything) have to be configured via .ini file
+# ```ini
+# # ~/.config/Holy Pangolin/Animatch/SuperDerpy.ini
+# [SuperDerpy]
+# debug=1
+# disableTouch=1
+# [game]
+# verbose=1
+# ```
+{ pkgs, ... }:
 {
   sane.programs.animatch = {
+    packageUnwrapped = with pkgs; animatch.override {
+      # allegro has no native wayland support, and so by default crashes when run without Xwayland.
+      # enable the allegro SDL backend, and achieve Wayland support via SDL's Wayland support.
+      # TODO: see about upstreaming this to nixpkgs?
+      allegro5 = allegro5.overrideAttrs (upstream: {
+        buildInputs = upstream.buildInputs ++ [
+          SDL2
+        ];
+        cmakeFlags = upstream.cmakeFlags ++ [
+          "-DALLEGRO_SDL=on"
+        ];
+      });
+    };
+
+    buildCost = 1;
+
+    sandbox.method = "bwrap";
+    sandbox.whitelistWayland = true;
+
     persist.byStore.plaintext = [
-      # game progress
-      ".config/Holy Pangolin/Animatch"
-      ".local/share/Holy Pangolin/Animatch"  # i think this one might be wrong
+      # ".config/Holy Pangolin/Animatch"  #< used for SuperDerpy config (e.g. debug, disableTouch, fullscreen, enable sound, etc). SuperDerpy.ini
+      ".local/share/Holy Pangolin/Animatch"  #< used for game state (level clears). SuperDerpy.ini
     ];
   };
 }

hosts/common/programs/audacity.nix

@@ -1,7 +1,12 @@
+# tips/tricks
+# - audio recording
+#   - default recording input will be silent, on lappy.
+#   - Audio Setup -> Rescan Audio Devices ...
+#   - Audio Setup -> Recording device -> sysdefault
 { pkgs, ... }:
 {
   sane.programs.audacity = {
-    package = pkgs.audacity.override {
+    packageUnwrapped = pkgs.audacity.override {
       # wxGTK32 uses webkitgtk-4.0.
       # audacity doesn't actually need webkit though, so diable to reduce closure
       wxGTK32 = pkgs.wxGTK32.override {
@@ -9,6 +14,23 @@
       };
     };
 
+    buildCost = 1;
+
+    sandbox.method = "bwrap";
+    sandbox.whitelistAudio = true;
+    sandbox.whitelistWayland = true;
+    sandbox.autodetectCliPaths = true;
+    sandbox.extraHomePaths = [
+      # support media imports via file->open dir to some common media directories
+      "tmp"
+      "Music"
+      # audacity needs the entire config dir mounted if running in a sandbox
+      ".config/audacity"
+    ];
+    sandbox.extraPaths = [
+      "/dev/snd"  # for recording audio inputs to work
+    ];
+
     # disable first-run splash screen
     fs.".config/audacity/audacity.cfg".file.text = ''
       PrefsVersion=1.1.1r1

hosts/common/programs/bemenu.nix

@@ -87,7 +87,13 @@ let
 in
 {
   sane.programs.bemenu = {
-    package = pkgs.bemenu.overrideAttrs (upstream: {
+    sandbox.method = "bwrap";  # landlock works, but requires *all* of /run/user/$ID to be granted.
+    sandbox.whitelistWayland = true;
+    sandbox.extraHomePaths = [
+      ".cache/fontconfig"  #< else it complains, and is *way* slower
+    ];
+
+    packageUnwrapped = pkgs.bemenu.overrideAttrs (upstream: {
       nativeBuildInputs = (upstream.nativeBuildInputs or []) ++ [
         pkgs.makeWrapper
       ];

hosts/common/programs/blast-ugjka/blast-to-default

@@ -0,0 +1,163 @@
+#!/usr/bin/env nix-shell
+#!nix-shell -i python3 -p "python3.withPackages (ps: [  ])" -p blast-ugjka
+# vim: set filetype=python :
+
+import logging
+import socket
+import subprocess
+
+from enum import Enum
+
+logger = logging.getLogger(__name__)
+
+# map from known devices -> required flags
+DEVICE_MAP = {
+  "Theater TV": [],
+  "Cuddlevision": [ "-usewav" ],
+}
+
+def get_ranked_ip_addrs():
+    """
+    return the IP addresses most likely to be LAN addresses
+    based on: <https://stackoverflow.com/a/1267524>
+    """
+    _name, _aliases, static_addrs = socket.gethostbyname_ex(socket.gethostname())
+    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+    s.connect(("1", 53))
+    con_addr, _port = s.getsockname()
+    return sorted(set(static_addrs + [ con_addr ]), key=lambda a: (a.startswith("127"), a))
+
+
+class ParserState(Enum):
+    Break = "break"
+    Receiver = "receiver"
+    Ips = "ip"
+
+class Status(Enum):
+    Continue = "continue"
+    Error = "error"
+    RedoWithFlags = "redo_with_flags"
+    Launched = "launched"
+
+class BlastDriver:
+    parsing: ParserState | None = None
+    last_write: str | None = None
+    def __init__(self, blast_flags: list[str] = []):
+        self.ranked_ips = get_ranked_ip_addrs()
+        self.blast = subprocess.Popen(
+            ["blast", "-source", "blast.monitor"] + blast_flags,
+            stdin=subprocess.PIPE,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+        )
+        self.blast_flags = list(blast_flags)
+        self.receiver_names = []
+        self.ips = []
+
+    def writeline(self, line: str) -> None:
+        logger.debug("[send] %s", line)
+        self.blast.stdin.write(f"{line}\n".encode())
+        self.blast.stdin.flush()
+        self.last_write = line
+
+    def readline(self) -> str:
+        line = self.blast.stdout.readline().decode('utf-8').strip()
+        line = line.replace('\x1b[1A\x1b[K', '')  #< escape codes
+        logger.debug("[recv] %r", line)
+        return line
+
+    def set_state(self, state: ParserState):
+        logger.debug("[pars] %s", state)
+        self.parsing = state
+
+    def feedline(self, line: str) -> (Status, str|None):
+        """
+        apply a line from blast's stdout to modify parser state.
+        returns a status code (e.g. Status.Continue), and optionally a reply to send back to blast.
+        """
+        if line == "Loading...":
+            return Status.Continue, None
+        elif line == "----------":
+            self.set_state(ParserState.Break)
+            return Status.Continue, None
+        elif line == "DLNA receivers":
+            self.set_state(ParserState.Receiver)
+            return Status.Continue, None
+        elif line == "Your LAN ip addresses":
+            self.set_state(ParserState.Ips)
+            return Status.Continue, None
+        elif line == "Select the DLNA device:":
+            assert len(self.receiver_names) == 1, self.receiver_names
+            name = self.receiver_names[0]
+            if name in DEVICE_MAP and DEVICE_MAP[name] != self.blast_flags:
+                return Status.RedoWithFlags, None
+            return Status.Continue, "0"
+        elif line == "Select the lan IP address for the stream:":
+            for r in self.ranked_ips:
+                if r in self.ips:
+                    return Status.Launched, str(self.ips.index(r))
+            # fallback: just guess the best IP
+            return Status.Launched, "0"
+        elif self.parsing == ParserState.Receiver:
+            id_, name = line.split(": ")
+            assert id_ == str(len(self.receiver_names)), (id_, self.receiver_names)
+            self.receiver_names.append(name)
+            return Status.Continue, None
+        elif self.parsing == ParserState.Ips:
+            id_, ip = line.split(": ")
+            assert id_ == str(len(self.ips)), (id_, self.ips)
+            self.ips.append(ip)
+            return Status.Continue, None
+        elif line == f"[{self.last_write}]":
+            # it's echoing to us what we wrote
+            return Status.Continue, None
+        # elif line == "":
+        #     return Status.Continue, None
+        else:
+            logger.info("unrecognized output (state=%s): %r", self.parsing, line)
+            return Status.Error, None
+
+    def step(self) -> Status:
+        """
+        advance the interaction between us and blast.
+        reads a line from blast, modifies internal state, maybe sends a reply.
+        could block indefinitely.
+        """
+        line = self.readline()
+        status, reply = self.feedline(line)
+        if reply is not None:
+            self.writeline(reply)
+        return status
+
+def try_blast(*args, **kwargs) -> BlastDriver | None:
+    blast = BlastDriver(*args, **kwargs)
+    status = Status.Continue
+    while status == Status.Continue:
+        status = blast.step()
+
+    if status == Status.RedoWithFlags:
+        dev = blast.receiver_names[0]
+        blast_flags = DEVICE_MAP[dev]
+        logger.info("re-exec blast for %s with flags: %r", dev, blast_flags)
+        blast.blast.terminate()
+        return try_blast(blast_flags=blast_flags)
+    elif status == Status.Error:
+        logger.info("blast error => terminating")
+        blast.blast.terminate()
+    else:
+        # successfully launched
+        return blast
+
+
+def main():
+    logging.basicConfig()
+    logging.getLogger().setLevel(logging.DEBUG)
+
+    blast = try_blast()
+
+    if blast is not None:
+        logger.info("waiting until blast exits")
+        blast.blast.wait()
+
+if __name__ == "__main__":
+    main()

hosts/common/programs/blast-ugjka/default.nix

@@ -0,0 +1,51 @@
+# blast: tunnel audio from a pulseaudio sink to a UPnP/DLNA device (like a TV).
+# - expect 7s of latency
+# - can cast the default sink, or create a new one "blast.monitor"
+#   and either assign that to default or assign apps to it.
+# compatibility:
+# - there is no single invocation which will be compatible with all known devices.
+# - sony tv:
+#  - `blast` (default): WORKS
+#  - `-usewav`: FAILS
+# - LG TV:
+#   - `-usewav`: WORKS!
+#   - `-useaac`: FAILS
+#   - `-useflac`: FAILS
+#   - `-uselpcm`: FAILS
+#   - `-uselpcmle`: FAILS
+#   - `-format aac`: FAILS
+#   - `-bitrate 128`: FAILS
+#   - `-nochunked`: FAILS
+#   - `-format "ogg" -mime 'audio/x-opus+ogg'`: FAILS
+#   - `-mime audio/ac3 -format ac3`: FAILS
+{ config, lib, pkgs, ... }:
+let
+  cfg = config.sane.programs.blast-ugjka;
+in
+{
+  sane.programs.blast-ugjka = {
+    sandbox.method = "bwrap";
+    sandbox.whitelistAudio = true;
+    sandbox.net = "clearnet";
+  };
+
+  sane.programs.blast-to-default = {
+    # helper to deal with blast's interactive CLI
+    packageUnwrapped = pkgs.static-nix-shell.mkPython3Bin {
+      pname = "blast-to-default";
+      pkgs = [ "blast-ugjka" ];
+      srcRoot = ./.;
+    };
+    sandbox.method = "bwrap";
+    sandbox.whitelistAudio = true;
+    sandbox.net = "clearnet";
+    sandbox.extraConfig = [
+      # else it fails to reap its children (or, maybe, it fails to hook its parent's death signal?)
+      # might be possible to remove this, but kinda hard to see a clean way.
+      "--sanebox-keep-namespace" "pid"
+    ];
+    suggestedPrograms = [ "blast-ugjka" "sane-die-with-parent" ];
+  };
+
+  networking.firewall.allowedTCPPorts = lib.mkIf cfg.enabled [ 9000 ];
+}

hosts/common/programs/bonsai.nix

@@ -1,7 +1,7 @@
 # bonsai docs: <https://sr.ht/~stacyharper/bonsai/>
 { config, lib, pkgs, ... }:
 let
-  cfg = config.sane.gui.sxmo.bonsaid;
+  cfg = config.sane.programs.bonsai;
 
   delayType = with lib; types.submodule {
     options = {
@@ -88,74 +88,34 @@ let
         mergeOneOption loc defs
     ;
   };
-
-  # transitionType = with lib; types.submodule {
-  #   options = {
-  #     type = mkOption {
-  #       type = types.enum [ "delay" "event" "exec" ];
-  #     };
-  #     delay_duration = mkOption {
-  #       type = types.nullOr types.int;
-  #       default = null;
-  #       description = ''
-  #         used for "delay" types only.
-  #         nanoseconds until the event is finalized.
-  #       '';
-  #     };
-  #     event_name = mkOption {
-  #       type = types.nullOr types.str;
-  #       default = null;
-  #       description = ''
-  #         name of event which this transition applies to.
-  #       '';
-  #     };
-  #     transitions = mkOption {
-  #       type = types.nullOr (types.listOf transitionType);
-  #       default = null;
-  #       description = ''
-  #         list of transitions out of this state.
-  #       '';
-  #     };
-  #     command = mkOption {
-  #       type = types.nullOr (types.listOf types.str);
-  #       default = null;
-  #       description = ''
-  #         used for "exec" types only.
-  #         command to run when the event is triggered.
-  #       '';
-  #     };
-  #   };
-  # };
 in
 {
-  options = with lib; {
-    sane.gui.sxmo.bonsaid.package = mkOption {
-      type = types.package;
-      default = pkgs.bonsai;
+  sane.programs.bonsai = {
+    configOption = with lib; mkOption {
+      default = {};
+      type = types.submodule {
+        options = {
+          transitions = mkOption {
+            type = types.listOf transitionType;
+            default = [];
+          };
+        };
+      };
     };
-    sane.gui.sxmo.bonsaid.transitions = mkOption {
-      type = types.listOf transitionType;
-      default = [];
-    };
-    sane.gui.sxmo.bonsaid.configFile = mkOption {
-      type = types.path;
-      default = pkgs.writeText "bonsai_tree.json" (builtins.toJSON cfg.transitions);
-      description = ''
-        configuration file to pass to bonsai.
-        usually auto-generated from the sibling options; exposed mainly for debugging or convenience.
-      '';
-    };
-  };
-  config = lib.mkIf config.sane.gui.sxmo.enable {
-    sane.user.services.bonsaid = {
-      description = "programmable input dispatcher";
-      script = ''
-        ${pkgs.coreutils}/bin/rm -f $XDG_RUNTIME_DIR/bonsai
-        exec ${cfg.package}/bin/bonsaid -t ${cfg.configFile}
-      '';
-      serviceConfig.Type = "simple";
-      serviceConfig.Restart = "always";
-      serviceConfig.RestartSec = "5s";
+
+    fs.".config/bonsai/bonsai_tree.json".symlink.text = builtins.toJSON cfg.config.transitions;
+
+    sandbox.method = "bwrap";
+    sandbox.extraRuntimePaths = [
+      "/"  #< just needs "bonsai", but needs to create it first...
+    ];
+
+    services.bonsaid = {
+      description = "bonsai: programmable input dispatcher";
+      partOf = [ "graphical-session" ];
+      # nice -n -11 chosen arbitrarily. i hope this will allow for faster response to inputs, but without audio underruns (pipewire is -21, dino -15-ish)
+      command = "nice -n -11 bonsaid -t $HOME/.config/bonsai/bonsai_tree.json";
+      cleanupCommand = "rm -f $XDG_RUNTIME_DIR/bonsai";
     };
   };
 }

hosts/common/programs/brave.nix

@@ -1,6 +1,17 @@
 { ... }:
 {
   sane.programs.brave = {
+    sandbox.method = "bwrap";
+    sandbox.wrapperType = "inplace";  # /opt/share/brave.com vendor-style packaging
+    sandbox.net = "all";
+    sandbox.extraHomePaths = [
+      "dev"  # for developing anything web-related
+      "tmp"
+    ];
+    sandbox.whitelistAudio = true;
+    sandbox.whitelistDri = true;
+    sandbox.whitelistWayland = true;
+
     persist.byStore.cryptClearOnBoot = [
       ".cache/BraveSoftware"
       ".config/BraveSoftware"

hosts/common/programs/bubblewrap.nix

@@ -0,0 +1,35 @@
+{ pkgs, ... }:
+{
+  sane.programs.bubblewrap = {
+    sandbox.enable = false;  # don't sandbox the sandboxer :)
+    packageUnwrapped = pkgs.bubblewrap.overrideAttrs (base: {
+      # patches = (base.patches or []) ++ [
+      #   (pkgs.fetchpatch {
+      #     url = "https://git.uninsane.org/colin/bubblewrap/commit/9843f9b2b5f086563fd37250658d69350a2939be.patch";
+      #     name = "enable debug logging and add a bunch more tracing";
+      #     hash = "sha256-AlDsqddaBahhqGibZlCjgmChuK7mmxDt0aYHNgY05OI=";
+      #   })
+      # ];
+      postPatch = (base.postPatch or "") + ''
+        # bwrap doesn't like to be invoked with any capabilities, which is troublesome if i
+        # want to do things like ship CAP_NET_ADMIN,CAP_NET_RAW in the ambient set for tools like Wireshark.
+        # but this limitation of bwrap is artificial and at first look is just a scenario the author probably
+        # never expected: patch out the guard check.
+        #
+        # see: <https://github.com/containers/bubblewrap/issues/397>
+        #
+        # note that invoking bwrap with capabilities in the 'init' namespace does NOT grant the sandboxed process
+        # capabilities in the 'init' namespace. it's a limitation of namespaces that namespaced processes can
+        # never receive capabilities in their parent namespace.
+        substituteInPlace bubblewrap.c --replace-fail \
+          'die ("Unexpected capabilities but not setuid, old file caps config?");' \
+          '// die ("Unexpected capabilities but not setuid, old file caps config?");'
+
+        # enable debug printing
+        # substituteInPlace utils.h --replace-fail \
+        #   '#define __debug__(x)' \
+        #   '#define __debug__(x) printf x'
+      '';
+    });
+  };
+}