WIP: mootube: init at 2022-08-28

note that this doesn't cross compile, because of stupid fucking rust build.rs scripts

thanks, openssl-sys

.gitignore

@@ -1,5 +1,4 @@
-/keep
+.working
 result
 result-*
 /secrets/local.nix
-/working

README.md

@@ -11,7 +11,7 @@ the only hard dependency for my exported pkgs/modules should be [nixpkgs][nixpkg
 building <./hosts/> will require [sops][sops].
 
 you might specifically be interested in these files (elaborated further in #key-points-of-interest):
-- [`sxmo-utils-latest`](./pkgs/additional/sxmo-utils/default.nix)
+- [`sxmo-utils`](./pkgs/additional/sxmo-utils/default.nix)
   - [example SXMO deployment](./hosts/modules/gui/sxmo/default.nix)
 - [my implementation of impermanence](./modules/persist/default.nix)
 - my way of deploying dotfiles/configuring programs per-user:

TODO.md

@@ -1,10 +1,14 @@
 ## BUGS
-- why i need to manually restart `wireguard-wg-ovpns` on servo periodically
+- ringer (i.e. dino incoming call) doesn't prevent moby from sleeping
-  - else DNS fails
+- Fractal opens links with non-preferred web browser
-- fix epiphany URL bar input on moby
+- `nix` operations from lappy hang when `desko` is unreachable
+  - could at least direct the cache to `http://desko-hn:5001`
+- waybar isn't visible on moby until after `swaymsg reload`
 
 ## REFACTORING:
 
+- fold hosts/common/home/ssh.nix -> hosts/common/users/colin.nix
+
 ### sops/secrets
 - attach secrets to the thing they're used by (sane.programs)
 - rework secrets to leverage `sane.fs`
@@ -21,7 +25,6 @@
 - fix lightdm-mobile-greeter for newer libhandy
 - port zecwallet-lite to a from-source build
 - REVIEW/integrate jellyfin dataDir config: <https://github.com/NixOS/nixpkgs/pull/233617>
-- remove `libsForQt5.callPackage` broadly: <https://github.com/NixOS/nixpkgs/issues/180841>
 
 #### upstreaming to non-nixpkgs repos
 - gtk: build schemas even on cross compilation: <https://github.com/NixOS/nixpkgs/pull/247844>
@@ -47,16 +50,34 @@
   - e.g. daily email checks; daily backup checks
   - integrate `nix check` into Gitea actions?
 
+### faster/better deployments
+- remove audacity's dependency on webkitgtk (via wxwidgets)
+
 ### user experience
-#### moby
-- fix cpuidle (gets better power consumption): <https://xnux.eu/log/077.html>
 - install apps:
   - display QR codes for WiFi endpoints: <https://linuxphoneapps.org/apps/noappid.wisperwind.wifi2qr/>
-  - shopping list: <https://linuxphoneapps.org/apps/ro.hume.cosmin.shoppinglist/>
+  - shopping list (not in nixpkgs): <https://linuxphoneapps.org/apps/ro.hume.cosmin.shoppinglist/>
-  - offline Wikipedia
+  - offline Wikipedia (or, add to `wike`)
+  - offline docs viewer (gtk): <https://github.com/workbenchdev/Biblioteca>
+  - some type of games manager/launcher
+    - Gnome Highscore (retro games)?: <https://gitlab.gnome.org/World/highscore>
+  - better maps for mobile (Osmin (QtQuick)? Pure Maps (Qt/Kirigami)? Gnome Maps is improved in 45)
+  - note-taking app: <https://linuxphoneapps.org/categories/note-taking/>
+  - OSK overlay specifically for mobile gaming
+    - i.e. mock joysticks, for use with SuperTux and SuperTuxKart
+- install mobile-friendly games:
+  - Shattered Pixel Dungeon (nixpkgs `shattered-pixel-dungeon`; doesn't cross-compile b/c openjdk/libIDL) <https://github.com/ebolalex/shattered-pixel-dungeon>
+  - UnCiv (Civ V clone; nixpkgs `unciv`; doesn't cross-compile):  <https://github.com/yairm210/UnCiv>
+  - Simon Tatham's Puzzle Collection (not in nixpkgs) <https://git.tartarus.org/?p=simon/puzzles.git>
+  - Shootin Stars  (Godot; not in nixpkgs) <https://gitlab.com/greenbeast/shootin-stars>
+
+#### moby
+- fix cpuidle (gets better power consumption): <https://xnux.eu/log/077.html>
 - SwayNC:
   - don't show MPRIS if no players detected
     - this is a problem of playerctld, i guess
+    - also, the album icon when "Not playing" doesn't follow the size we give in the config
+      - that means mpris always takes up excessive space on moby
   - add option to change audio output
   - fix colors (red alert) to match overall theme
 - moby: tune GPS
@@ -70,12 +91,6 @@
 - sxmo: port to swaybar like i use on desktop
   - users in #sxmo claim it's way better perf
 - sxmo: fix youtube scripts (package youtube-cli)
-- sxmo: don't put all deps on PATH
-  - maybe: use resholve to hard-code them
-    - this is the most "correct", but least patchable
-  - maybe: express each invocation as a function in sxmo_common.sh
-    - this will require some patching to handle `exec <foo>` style
-  - maybe: save original PATH and reset it before invoking user files
 - moby: theme GTK apps (i.e. non-adwaita styles)
   - combine multiple icon themes to get one which has the full icon set?
   - get adwaita-icon-theme to ship everything even when cross-compiled?
@@ -84,6 +99,8 @@
 - phog: remove the gnome-shell runtime dependency to save hella closure size
 
 #### non-moby
+- RSS: integrate a paywall bypass
+  - e.g. self-hosted [ladder](https://github.com/everywall/ladder) (like 12ft.io)
 - neovim: set up language server (lsp; rnix-lsp; nvim-lspconfig)
 - Helix: make copy-to-system clipboard be the default
 - firefox/librewolf: persist history
@@ -104,10 +121,6 @@
 - add `pkgs.impure-cached.<foo>` package set to build things with ccache enabled
   - every package here can be auto-generated, and marked with some env var so that it doesn't pollute the pure package set
   - would be super handy for package prototyping!
-- why does nixos-rebuild switch take 5 minutes when net is flakey?
-  - trying to auto-mount servo?
-  - something to do with systemd services restarting/stalling
-  - maybe wireguard & its refresh operation, specifically?
 - get moby to build without binfmt emulation (i.e. make all emulation explicit)
   - then i can distribute builds across servo + desko, and also allow servo to pull packages from desko w/o worrying about purity
 

flake.lock

@@ -5,11 +5,11 @@
         "systems": "systems"
       },
       "locked": {
-        "lastModified": 1687709756,
+        "lastModified": 1694529238,
-        "narHash": "sha256-Y5wKlQSkgEK2weWdOu4J3riRd+kV/VCgHsqLNTTWQ/0=",
+        "narHash": "sha256-zsNZZGTGnMOf9YpHKJqMSsa0dXbfmxeoJ7xHlrt+xmY=",
         "owner": "numtide",
         "repo": "flake-utils",
-        "rev": "dbabf0ca0c0c4bce6ea5eaf65af5cb694d2082c7",
+        "rev": "ff7b65b44d01cf9ba6a71320833626af21126384",
         "type": "github"
       },
       "original": {
@@ -21,26 +21,27 @@
     "mobile-nixos": {
       "flake": false,
       "locked": {
-        "lastModified": 1696124168,
+        "lastModified": 1694749521,
-        "narHash": "sha256-EzGHYAR7rozQQLZEHbKEcb5VpUFGoxwEsM0OWfW4wqU=",
+        "narHash": "sha256-MiVokKlpcJmfoGuWAMeW1En7gZ5hk0rCQArYm6P9XCc=",
         "owner": "nixos",
         "repo": "mobile-nixos",
-        "rev": "7cee346c3f8e73b25b1cfbf7a086a7652c11e0f3",
+        "rev": "d25d3b87e7f300d8066e31d792337d9cd7ecd23b",
         "type": "github"
       },
       "original": {
         "owner": "nixos",
+        "ref": "d25d3b87e7f300d8066e31d792337d9cd7ecd23b",
         "repo": "mobile-nixos",
         "type": "github"
       }
     },
     "nixpkgs-stable": {
       "locked": {
-        "lastModified": 1696123266,
+        "lastModified": 1700905716,
-        "narHash": "sha256-S6MZEneQeE4M/E/C8SMnr7B7oBnjH/hbm96Kak5hAAI=",
+        "narHash": "sha256-w1vHn2MbGfdC+CrP3xLZ3scsI06N0iQLU7eTHIVEFGw=",
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "dbe90e63a36762f1fbde546e26a84af774a32455",
+        "rev": "dfb95385d21475da10b63da74ae96d89ab352431",
         "type": "github"
       },
       "original": {
@@ -52,16 +53,16 @@
     },
     "nixpkgs-unpatched": {
       "locked": {
-        "lastModified": 1696375444,
+        "lastModified": 1701180790,
-        "narHash": "sha256-Sv0ICt/pXfpnFhTGYTsX6lUr1SljnuXWejYTI2ZqHa4=",
+        "narHash": "sha256-kYWcHsk2A1VUpiOvSo7Pq175WnSVeltspTGM2q+Cr3U=",
         "owner": "nixos",
         "repo": "nixpkgs",
-        "rev": "81e8f48ebdecf07aab321182011b067aafc78896",
+        "rev": "c9702bf40b036c0f1d3d5b0aaf3eee2bf920124c",
         "type": "github"
       },
       "original": {
         "owner": "nixos",
-        "ref": "nixos-unstable",
+        "ref": "master",
         "repo": "nixpkgs",
         "type": "github"
       }
@@ -82,11 +83,11 @@
         "nixpkgs-stable": "nixpkgs-stable"
       },
       "locked": {
-        "lastModified": 1696320910,
+        "lastModified": 1701127353,
-        "narHash": "sha256-fbuEc6wylH+0VxG48lhPBK+SQJHfo2lusUwWHZNipIM=",
+        "narHash": "sha256-qVNX0wOl0b7+I35aRu78xUphOyELh+mtUp1KBx89K1Q=",
         "owner": "Mic92",
         "repo": "sops-nix",
-        "rev": "746c7fa1a64c1671a4bf287737c27fdc7101c4c2",
+        "rev": "b1edbf5c0464b4cced90a3ba6f999e671f0af631",
         "type": "github"
       },
       "original": {
@@ -118,11 +119,11 @@
         ]
       },
       "locked": {
-        "lastModified": 1696306988,
+        "lastModified": 1699515935,
-        "narHash": "sha256-I/OyJxIxu0n5h1eFqwVw0C6wTN3ewBXp2lGAdo1ur70=",
+        "narHash": "sha256-cJIuVrYorhIzG5pRFZb+ZtaKhTFD92ThC42SaxvSe/E=",
         "ref": "refs/heads/master",
-        "rev": "1f588493031168d92a1e60705f26aaf4b2cdc07e",
+        "rev": "8a4273489d945f21d7e0ca6aac952460c7d4c391",
-        "revCount": 208,
+        "revCount": 216,
         "type": "git",
         "url": "https://git.uninsane.org/colin/uninsane"
       },

flake.nix

@@ -36,20 +36,24 @@
     #   - staging-next merged into master.
     #
     # which branch to source from?
-    # - for everyday development, prefer `nixos-unstable` branch, as it provides good caching.
+    # - nixos-unstable: for everyday development; it provides good caching
-    # - if need to test bleeding updates (e.g. if submitting code into staging):
+    # - master: temporarily if i'm otherwise cherry-picking lots of already-applied patches
-    #   - use `staging-next` if it's been cut (i.e. if there's an active staging-next -> master PR)
+    # - staging-next: if testing stuff that's been PR'd into staging, i.e. base library updates.
-    #   - use `staging` if no staging-next branch has been cut.
+    # - staging: maybe if no staging-next -> master PR has been cut yet?
     #
     # <https://github.com/nixos/nixpkgs/tree/nixos-unstable>
-    nixpkgs-unpatched.url = "github:nixos/nixpkgs?ref=nixos-unstable";
+    # nixpkgs-unpatched.url = "github:nixos/nixpkgs?ref=nixos-unstable";
+    nixpkgs-unpatched.url = "github:nixos/nixpkgs?ref=master";
     # nixpkgs-unpatched.url = "github:nixos/nixpkgs?ref=staging-next";
     # nixpkgs-unpatched.url = "github:nixos/nixpkgs?ref=staging";
 
     mobile-nixos = {
       # <https://github.com/nixos/mobile-nixos>
       # only used for building disk images, not relevant after deployment
-      url = "github:nixos/mobile-nixos";
+      # TODO: replace with something else. commit `0f3ac0bef1aea70254a3bae35e3cc2561623f4c1`
+      # replaces the imageBuilder with a "new implementation from celun" and wildly breaks my use.
+      # pinning to d25d3b... is equivalent to holding at 2023-09-15
+      url = "github:nixos/mobile-nixos?ref=d25d3b87e7f300d8066e31d792337d9cd7ecd23b";
       flake = false;
     };
     sops-nix = {
@@ -108,7 +112,7 @@
 
       nixpkgsCompiledBy = system: nixpkgs.legacyPackages."${system}";
 
-      evalHost = { name, local, target }: nixpkgs.lib.nixosSystem {
+      evalHost = { name, local, target, light ? false }: nixpkgs.lib.nixosSystem {
         system = target;
         modules = [
           {
@@ -121,6 +125,9 @@
             # nixpkgs.buildPlatform = local;  # set by instantiate.nix instead
             # nixpkgs.config.replaceStdenv = { pkgs }: pkgs.ccacheStdenv;
           }
+          (optionalAttrs light {
+            sane.enableSlowPrograms = false;
+          })
           (import ./hosts/instantiate.nix { hostName = name; })
           self.nixosModules.default
           self.nixosModules.passthru
@@ -136,21 +143,21 @@
       nixosConfigurations =
         let
           hosts = {
-            servo =  { name = "servo"; local = "x86_64-linux"; target = "x86_64-linux"; };
+            servo       = { name = "servo";  local = "x86_64-linux"; target = "x86_64-linux";  };
-            desko =  { name = "desko"; local = "x86_64-linux"; target = "x86_64-linux"; };
+            desko       = { name = "desko";  local = "x86_64-linux"; target = "x86_64-linux";  };
-            lappy =  { name = "lappy"; local = "x86_64-linux"; target = "x86_64-linux"; };
+            desko-light = { name = "desko";  local = "x86_64-linux"; target = "x86_64-linux";  light = true; };
-            moby  =  { name = "moby";  local = "x86_64-linux"; target = "aarch64-linux"; };
+            lappy       = { name = "lappy";  local = "x86_64-linux"; target = "x86_64-linux";  };
-            rescue = { name = "rescue"; local = "x86_64-linux"; target = "x86_64-linux"; };
+            lappy-light = { name = "lappy";  local = "x86_64-linux"; target = "x86_64-linux";  light = true; };
+            moby        = { name = "moby";   local = "x86_64-linux"; target = "aarch64-linux"; };
+            moby-light  = { name = "moby";   local = "x86_64-linux"; target = "aarch64-linux"; light = true; };
+            rescue      = { name = "rescue"; local = "x86_64-linux"; target = "x86_64-linux";  };
           };
           # cross-compiled builds: instead of emulating the host, build using a cross-compiler.
           # - these are faster to *build* than the emulated variants (useful when tweaking packages),
           # - but fewer of their packages can be found in upstream caches.
           cross = mapAttrValues evalHost hosts;
           emulated = mapAttrValues
-            ({name, local, target}: evalHost {
+            (args: evalHost (args // { local = null; }))
-              inherit name target;
-              local = null;
-            })
             hosts;
           prefixAttrs = prefix: attrs: mapAttrs'
             (name: value: {
@@ -162,9 +169,9 @@
           (prefixAttrs "cross-" cross) //
           (prefixAttrs "emulated-" emulated) // {
             # prefer native builds for these machines:
-            inherit (emulated) servo desko lappy rescue;
+            inherit (emulated) servo desko desko-light lappy lappy-light rescue;
             # prefer cross-compiled builds for these machines:
-            inherit (cross) moby;
+            inherit (cross) moby moby-light;
           };
 
       # unofficial output
@@ -184,8 +191,9 @@
       imgs = mapAttrValues (host: host.config.system.build.img) self.nixosConfigurations;
 
       # unofficial output
-      host-pkgs = mapAttrValues (host: host.config.system.build.pkgs) self.nixosConfigurations;
+      hostConfigs = mapAttrValues (host: host.config) self.nixosConfigurations;
-      host-programs = mapAttrValues (host: mapAttrValues (p: p.package) host.config.sane.programs) self.nixosConfigurations;
+      hostPkgs = mapAttrValues (host: host.config.system.build.pkgs) self.nixosConfigurations;
+      hostPrograms = mapAttrValues (host: mapAttrValues (p: p.package) host.config.sane.programs) self.nixosConfigurations;
 
       overlays = {
         # N.B.: `nix flake check` requires every overlay to take `final: prev:` at defn site,
@@ -263,9 +271,14 @@
             # let the user handle that edge case by re-running this whole command
             nixos-rebuild --flake '.#${host}' ${action} --target-host colin@${addr} --use-remote-sudo $@
           '';
+          deployApp = host: addr: action: {
+            type = "app";
+            program = ''${deployScript host addr action}'';
+          };
 
           # pkg updating.
           # a cleaner alternative lives here: <https://discourse.nixos.org/t/how-can-i-run-the-updatescript-of-personal-packages/25274/2>
+          # mkUpdater :: [ String ] -> { type = "app"; program = path; }
           mkUpdater = attrPath: {
             type = "app";
             program = let
@@ -290,7 +303,7 @@
               } else {}
             )
             (pkgs.lib.getAttrFromPath basePath sanePkgs);
-          mkUpdaters = { ignore ? [] }@opts: basePath:
+          mkUpdaters = { ignore ? [], flakePrefix ? [] }@opts: basePath:
             let
               updaters = mkUpdatersNoAliases opts basePath;
               invokeUpdater = name: pkg:
@@ -300,7 +313,7 @@
 
                   # in case `name` has a `.` in it, we have to quote it
                   escapedPath = builtins.map (p: ''"${p}"'') fullPath;
-                  updatePath = builtins.concatStringsSep "." ([ "update" "pkgs" ] ++ escapedPath);
+                  updatePath = builtins.concatStringsSep "." (flakePrefix ++ escapedPath);
                 in pkgs.lib.optionalString doUpdateByDefault (
                   pkgs.lib.escapeShellArgs [
                     "nix" "run" ".#${updatePath}"
@@ -308,8 +321,9 @@
                 );
             in {
               type = "app";
+              # top-level app just invokes the updater of everything one layer below it
               program = builtins.toString (pkgs.writeShellScript
-                (builtins.concatStringsSep "-" (["update"] ++ basePath))
+                (builtins.concatStringsSep "-" (flakePrefix ++ basePath))
                 (builtins.concatStringsSep
                   "\n"
                   (pkgs.lib.mapAttrsToList invokeUpdater updaters)
@@ -329,9 +343,12 @@
                 - `nix run '.#update.feeds'`
                   - updates metadata for all feeds
                 - `nix run '.#init-feed' <url>`
-                - `nix run '.#deploy-{lappy,moby,moby-test,servo}' [nixos-rebuild args ...]`
+                - `nix run '.#deploy.{desko,lappy,moby,servo}[-light][.test]' [nixos-rebuild args ...]`
                 - `nix run '.#check'`
                   - make sure all systems build; NUR evaluates
+
+                specific build targets of interest:
+                - `nix build '.#imgs.rescue'`
               '';
             in builtins.toString (pkgs.writeShellScript "nixos-config-help" ''
               cat ${helpMsg}
@@ -340,29 +357,27 @@
               nix flake show --option allow-import-from-derivation true
             '');
           };
-          update.pkgs = mkUpdaters { ignore = [ ["feeds"] ]; } [];
+          # wrangle some names to get package updaters which refer back into the flake, but also conditionally ignore certain paths (e.g. sane.feeds).
-          update.feeds = mkUpdaters {} [ "feeds" ];
+          # TODO: better design
+          update = rec {
+            _impl.pkgs.sane = mkUpdaters { flakePrefix = [ "update" "_impl" "pkgs" ]; ignore = [ [ "sane" "feeds" ] ]; } [ "sane" ];
+            pkgs = _impl.pkgs.sane;
+            _impl.feeds.sane.feeds = mkUpdaters { flakePrefix = [ "update" "_impl" "feeds" ]; } [ "sane" "feeds" ];
+            feeds = _impl.feeds.sane.feeds;
+          };
 
           init-feed = {
             type = "app";
             program = "${pkgs.feeds.init-feed}";
           };
 
-          deploy-lappy = {
+          deploy = {
-            type = "app";
+            lappy       = deployApp "lappy"       "lappy" "switch";
-            program = ''${deployScript "lappy" "lappy" "switch"}'';
+            lappy-light = deployApp "lappy-light" "lappy" "switch";
-          };
+            moby        = deployApp "moby"        "moby"  "switch";
-          deploy-moby-test = {
+            moby-light  = deployApp "moby-light"  "moby"  "switch";
-            type = "app";
+            moby-test   = deployApp "moby"        "moby"  "test";
-            program = ''${deployScript "moby" "moby-hn" "test"}'';
+            servo       = deployApp "servo"       "servo" "switch";
-          };
-          deploy-moby = {
-            type = "app";
-            program = ''${deployScript "moby" "moby-hn" "switch"}'';
-          };
-          deploy-servo = {
-            type = "app";
-            program = ''${deployScript "servo" "servo" "switch"}'';
           };
 
           sync-moby = {
@@ -392,9 +407,12 @@
               RC0=$?
               nix run '.#check.host-configs'
               RC1=$?
+              nix run '.#check.rescue'
+              RC2=$?
               echo "nur: $RC0"
               echo "host-configs: $RC1"
-              exit $(($RC0 | $RC1))
+              echo "rescue: $RC2"
+              exit $(($RC0 | $RC1 | $RC2))
             '');
           };
 
@@ -418,18 +436,27 @@
           check.host-configs = {
             type = "app";
             program = let
-              checkHost = host: ''
+              checkHost = host: let
-                nix build '.#nixosConfigurations.${host}.config.system.build.toplevel' --out-link ./result-${host} -j2 $@
+                shellHost = pkgs.lib.replaceStrings [ "-" ] [ "_" ] host;
-                RC_${host}=$?
+              in ''
+                nix build -v '.#nixosConfigurations.${host}.config.system.build.toplevel' --out-link ./result-${host} -j2 $@
+                RC_${shellHost}=$?
               '';
             in builtins.toString (pkgs.writeShellScript
               "check-host-configs"
               ''
+                # build minimally-usable hosts first, then their full image.
+                # this gives me a minimal image i can deploy or copy over, early.
+                ${checkHost "desko-light"}
+                ${checkHost "moby-light"}
+                ${checkHost "lappy-light"}
+
                 ${checkHost "desko"}
                 ${checkHost "lappy"}
                 ${checkHost "servo"}
                 ${checkHost "moby"}
                 ${checkHost "rescue"}
+
                 echo "desko: $RC_desko"
                 echo "lappy: $RC_lappy"
                 echo "servo: $RC_servo"
@@ -439,6 +466,13 @@
               ''
             );
           };
+
+          check.rescue = {
+            type = "app";
+            program = builtins.toString (pkgs.writeShellScript "check-rescue" ''
+              nix build -v '.#imgs.rescue' --out-link ./result-rescue-img -j2
+            '');
+          };
         };
 
       templates = {

hosts/by-name/desko/default.nix

@@ -11,23 +11,29 @@
 
   sops.secrets.colin-passwd.neededForUsers = true;
 
+  sane.ports.openFirewall = true;  # for e.g. nix-serve
+
   sane.roles.build-machine.enable = true;
-  sane.roles.ac = true;
   sane.roles.client = true;
   sane.roles.dev-machine = true;
+  sane.roles.pc = true;
   sane.services.wg-home.enable = true;
   sane.services.wg-home.ip = config.sane.hosts.by-name."desko".wg-home.ip;
   sane.services.duplicity.enable = true;
   sane.services.nixserve.secretKeyFile = config.sops.secrets.nix_serve_privkey.path;
 
+  sane.nixcache.substituters.desko = false;
+  sane.nixcache.remote-builders.desko = false;
+
   sane.gui.sway.enable = true;
   sane.programs.iphoneUtils.enableFor.user.colin = true;
   sane.programs.steam.enableFor.user.colin = true;
 
-  sane.programs.guiApps.suggestedPrograms = [ "desktopGuiApps" ];
-  sane.programs.consoleUtils.suggestedPrograms = [ "consoleMediaUtils" "desktopConsoleUtils" ];
   # sane.programs.devPkgs.enableFor.user.colin = true;
 
+  sane.programs.signal-desktop.config.autostart = true;
+  sane.programs."gnome.geary".config.autostart = true;
+
   boot.loader.efi.canTouchEfiVariables = false;
   sane.image.extraBootFiles = [ pkgs.bootpart-uefi-x86_64 ];
 

hosts/by-name/desko/fs.nix

@@ -1,14 +1,13 @@
 { ... }:
 
 {
-  sane.persist.root-on-tmpfs = true;
   # increase /tmp space (defaults to 50% of RAM) for building large nix things.
   # a cross-compiled kernel, particularly, will easily use 30+GB of tmp
   fileSystems."/tmp".options = [ "size=64G" ];
 
   fileSystems."/nix" = {
-    # device = "/dev/disk/by-uuid/985a0a32-da52-4043-9df7-615adec2e4ff";
+    # device = "/dev/disk/by-uuid/0ab0770b-7734-4167-88d9-6e4e20bb2a56";
-    device = "/dev/disk/by-uuid/0ab0770b-7734-4167-88d9-6e4e20bb2a56";
+    device = "/dev/disk/by-uuid/845d85bf-761d-431b-a406-e6f20909154f";
     fsType = "btrfs";
     options = [
       "compress=zstd"
@@ -17,8 +16,8 @@
   };
 
   fileSystems."/boot" = {
-    # device = "/dev/disk/by-uuid/CAA7-E7D2";
+    # device = "/dev/disk/by-uuid/41B6-BAEF";
-    device = "/dev/disk/by-uuid/41B6-BAEF";
+    device = "/dev/disk/by-uuid/5049-9AFD";
     fsType = "vfat";
   };
 }

hosts/by-name/lappy/default.nix

@@ -7,6 +7,7 @@
 
   sane.roles.client = true;
   sane.roles.dev-machine = true;
+  sane.roles.pc = true;
   sane.services.wg-home.enable = true;
   sane.services.wg-home.ip = config.sane.hosts.by-name."lappy".wg-home.ip;
 
@@ -15,11 +16,7 @@
   boot.loader.efi.canTouchEfiVariables = false;
   sane.image.extraBootFiles = [ pkgs.bootpart-uefi-x86_64 ];
 
-  sane.programs.guiApps.suggestedPrograms = [
+  sane.programs.stepmania.enableFor.user.colin = true;
-    "desktopGuiApps"
-    "stepmania"
-  ];
-  sane.programs.consoleUtils.suggestedPrograms = [ "consoleMediaUtils" "desktopConsoleUtils" ];
 
   sops.secrets.colin-passwd.neededForUsers = true;
 

hosts/by-name/lappy/fs.nix

@@ -1,8 +1,6 @@
 { ... }:
 
 {
-  sane.persist.root-on-tmpfs = true;
-
   fileSystems."/nix" = {
     device = "/dev/disk/by-uuid/75230e56-2c69-4e41-b03e-68475f119980";
     fsType = "btrfs";

hosts/by-name/lappy/polyfill.nix

@@ -33,7 +33,7 @@
       #   and so it just wouldn't handle any button inputs (sxmo_hook_inputhandler.sh not on path)
       SXMO_DEVICE_NAME = "three_button_touchscreen";
     };
-    package = pkgs.sxmo-utils-latest.overrideAttrs (base: {
+    package = (pkgs.sxmo-utils.override { preferSystemd = true; }).overrideAttrs (base: {
       postPatch = (base.postPatch or "") + ''
         # after volume-button navigation mode, restore full keyboard functionality
         cp ${./xkb_mobile_normal_buttons} ./configs/xkb/xkb_mobile_normal_buttons

hosts/by-name/moby/default.nix

@@ -20,15 +20,10 @@
   ];
 
   sane.roles.client = true;
+  sane.roles.handheld = true;
   sane.zsh.showDeadlines = false;  # unlikely to act on them when in shell
   sane.services.wg-home.enable = true;
   sane.services.wg-home.ip = config.sane.hosts.by-name."moby".wg-home.ip;
-  sane.wowlan.enable = true;
-  sane.wowlan.patterns = [
-    { ipv4.destPort = 22; }  # wake on SSH
-    { ipv4.srcPort = 2587; }  # wake on `ntfy-sh` push from servo
-    { arp.queryIp = [ 10 78 79 54 ]; }  # wake when somebody is doing an ARP query against us
-  ];
 
   # XXX colin: phosh doesn't work well with passwordless login,
   # so set this more reliable default password should anything go wrong
@@ -38,25 +33,28 @@
   sops.secrets.colin-passwd.neededForUsers = true;
 
   sane.gui.sxmo.enable = true;
-  sane.programs.guiApps.suggestedPrograms = [ "handheldGuiApps" ];
   # sane.programs.consoleUtils.enableFor.user.colin = false;
   # sane.programs.guiApps.enableFor.user.colin = false;
   sane.programs.blueberry.enableFor.user.colin = false;  # bluetooth manager: doesn't cross compile!
-  sane.programs.sequoia.enableFor.user.colin = false;
+  sane.programs.dialect.enableFor.user.colin = false;  # drags in 700MB of x86 dependencies (e.g. gtk4)
-  sane.programs.tuiApps.enableFor.user.colin = false;  # visidata, others, don't compile well
+  sane.programs.mercurial.enableFor.user.colin = false;  # does not cross compile
-  # disabled for faster deploys
+  sane.programs.nvme-cli.enableFor.system = false;  # does not cross compile (libhugetlbfs)
-  sane.programs.soundconverter.enableFor.user.colin = false;
-  sane.programs.eg25-control.enableFor.user.colin = true;
 
-  sane.programs.ntfy-sh.config.autostart = true;
+  # enabled for easier debugging
+  sane.programs.eg25-control.enableFor.user.colin = true;
+  sane.programs.rtl8723cs-wowlan.enableFor.user.colin = true;
+
+  # sane.programs.ntfy-sh.config.autostart = true;
   sane.programs.dino.config.autostart = true;
+  sane.programs.signal-desktop.config.autostart = true;
+  # sane.programs."gnome.geary".config.autostart = true;
   # sane.programs.calls.config.autostart = true;
+  sane.programs.mpv.config.vo = "wlshm";  #< see hosts/common/programs/mpv.nix for details
 
   sane.programs.firefox.mime.priority = 300;  # prefer other browsers when possible
   # HACK/TODO: make `programs.P.env.VAR` behave according to `mime.priority`
   sane.programs.firefox.env = lib.mkForce {};
   sane.programs.epiphany.env.BROWSER = "epiphany";
-  sane.programs.firefox.enableFor.user.colin = false;  # use epiphany instead
 
   # note the .conf.d approach: using ~/.config/pipewire/pipewire.conf directly breaks all audio,
   # presumably because that deletes the defaults entirely whereas the .conf.d approach selectively overrides defaults
@@ -85,8 +83,6 @@
     }
   '';
 
-  # sane.programs.mpv.enableFor.user.colin = true;
-
   boot.loader.efi.canTouchEfiVariables = false;
   # /boot space is at a premium. default was 20.
   # even 10 can be too much
@@ -95,16 +91,6 @@
   # mobile.boot.stage-1.enable = false;
   # boot.initrd.systemd.enable = false;
   # boot.initrd.services.swraid.enable = false;  # attempt to fix dm_mod stuff
-  # disable proximity sensor.
-  # the filtering/calibration is bad that it causes the screen to go fully dark at times.
-  boot.blacklistedKernelModules = [ "stk3310" ];
-
-  # without this some GUI apps fail: `DRM_IOCTL_MODE_CREATE_DUMB failed: Cannot allocate memory`
-  # this is because they can't allocate enough video ram.
-  # the default CMA seems to be 32M.
-  # i was running fine with 256MB from 2022/07-ish through 2022/12-ish, but then the phone quit reliably coming back from sleep: maybe a memory leak?
-  # `cat /proc/meminfo` to see CmaTotal/CmaFree if interested in tuning this.
-  boot.kernelParams = [ "cma=512M" ];
 
   # hardware.firmware makes the referenced files visible to the kernel, for whenever a driver explicitly asks for them.
   # these files are visible from userspace by following `/sys/module/firmware_class/parameters/path`
@@ -115,7 +101,14 @@
   #   ov5640_af.bin   (camera module)
   # hardware.firmware = [ config.mobile.device.firmware ];
   # hardware.firmware = [ pkgs.rtl8723cs-firmware ];
-  hardware.firmware = [ pkgs.linux-firmware-megous ];
+  hardware.firmware = [
+    (pkgs.linux-firmware-megous.override {
+      # rtl_bt = false probably means no bluetooth connectivity.
+      # N.B.: DON'T RE-ENABLE without first confirming that wake-on-lan works during suspend (rtcwake).
+      # it seems the rtl_bt stuff ("bluetooth coexist") might make wake-on-LAN radically more flaky.
+      rtl_bt = false;
+    })
+  ];
 
   system.stateVersion = "21.11";
 
@@ -133,7 +126,12 @@
   # see pkgs/patched/alsa-ucm-conf for more info.
   environment.variables.ALSA_CONFIG_UCM2 = "/run/current-system/sw/share/alsa/ucm2";
   environment.pathsToLink = [ "/share/alsa/ucm2" ];
-  environment.systemPackages = [ pkgs.alsa-ucm-conf-sane ];
+  environment.systemPackages = [
+    (pkgs.alsa-ucm-conf-sane.override {
+      # internal speaker has a tendency to break :(
+      preferEarpiece = true;
+    })
+  ];
   systemd = let
     ucm-env = config.environment.variables.ALSA_CONFIG_UCM2;
   in {

hosts/by-name/moby/fs.nix

@@ -1,7 +1,6 @@
 { ... }:
 
 {
-  sane.persist.root-on-tmpfs = true;
   fileSystems."/nix" = {
     device = "/dev/disk/by-uuid/1f1271f8-53ce-4081-8a29-60a4a6b5d6f9";
     fsType = "btrfs";

hosts/by-name/moby/kernel.nix

@@ -66,6 +66,24 @@ in
     # target = "zImage";  # <-- confuses other parts of nixos :-(
   };
 
+  # disable proximity sensor.
+  # the filtering/calibration is bad that it causes the screen to go fully dark at times.
+  boot.blacklistedKernelModules = [ "stk3310" ];
+
+  boot.kernelParams = [
+    # without this some GUI apps fail: `DRM_IOCTL_MODE_CREATE_DUMB failed: Cannot allocate memory`
+    # this is because they can't allocate enough video ram.
+    # see related nixpkgs issue: <https://github.com/NixOS/nixpkgs/issues/260222>
+    #
+    # the default CMA seems to be 32M.
+    # i was running fine with 256MB from 2022/07-ish through 2022/12-ish, but then the phone quit reliably coming back from sleep (phosh): maybe a memory leak?
+    # `cat /proc/meminfo` to see CmaTotal/CmaFree if interested in tuning this.
+    "cma=512M"
+    # 2023/10/20: potential fix for the lima (GPU) timeout bugs:
+    # - <https://gitlab.com/postmarketOS/pmaports/-/issues/805#note_890467824>
+    "lima.sched_timeout_ms=2000"
+  ];
+
   services.xserver.displayManager.job.preStart = ensureHWReady;
   systemd.services.greetd.preStart = ensureHWReady;
 }

hosts/by-name/moby/polyfill.nix

@@ -31,92 +31,7 @@
       SXMO_LISGD_INPUT_DEVICE = "/dev/input/by-path/platform-1c2ac00.i2c-event";
       # vol and power are detected correctly by upstream
 
-
       ### preferences
-      # notable bemenu options:
-      #   - see `bemenu --help` for all
-      #   -P, --prefix          text to show before highlighted item.
-      #   --scrollbar           display scrollbar. (none (default), always, autohide)
-      #   -H, --line-height     defines the height to make each menu line (0 = default height). (wx)
-      #   -M, --margin          defines the empty space on either side of the menu. (wx)
-      #   -W, --width-factor    defines the relative width factor of the menu (from 0 to 1). (wx)
-      #   -B, --border          defines the width of the border in pixels around the menu. (wx)
-      #   -R  --border-radius   defines the radius of the border around the menu (0 = no curved borders).
-      #   --ch                  defines the height of the cursor (0 = scales with line height). (wx)
-      #   --cw                  defines the width of the cursor. (wx)
-      #   --hp                  defines the horizontal padding for the entries in single line mode. (wx)
-      #   --fn                  defines the font to be used ('name [size]'). (wx)
-      #   --tb                  defines the title background color. (wx)
-      #   --tf                  defines the title foreground color. (wx)
-      #   --fb                  defines the filter background color. (wx)
-      #   --ff                  defines the filter foreground color. (wx)
-      #   --nb                  defines the normal background color. (wx)
-      #   --nf                  defines the normal foreground color. (wx)
-      #   --hb                  defines the highlighted background color. (wx)
-      #   --hf                  defines the highlighted foreground color. (wx)
-      #   --fbb                 defines the feedback background color. (wx)
-      #   --fbf                 defines the feedback foreground color. (wx)
-      #   --sb                  defines the selected background color. (wx)
-      #   --sf                  defines the selected foreground color. (wx)
-      #   --ab                  defines the alternating background color. (wx)
-      #   --af                  defines the alternating foreground color. (wx)
-      #   --scb                 defines the scrollbar background color. (wx)
-      #   --scf                 defines the scrollbar foreground color. (wx)
-      #   --bdr                 defines the border color. (wx)
-      #
-      # colors are specified as `#RRGGBB`
-      # defaults:
-      # --ab "#222222"
-      # --af "#bbbbbb"
-      # --bdr "#005577"
-      # --border 3
-      # --cb "#222222"
-      # --center
-      # --cf "#bbbbbb"
-      # --fb "#222222"
-      # --fbb "#eeeeee"
-      # --fbf "#222222"
-      # --ff "#bbbbbb"
-      # --fixed-height
-      # --fn 'Sxmo 14'
-      # --hb "#005577"
-      # --hf "#eeeeee"
-      # --line-height 20
-      # --list 16
-      # --margin 40
-      # --nb "#222222"
-      # --nf "#bbbbbb"
-      # --no-overlap
-      # --no-spacing
-      # --sb "#323232"
-      # --scb "#005577"
-      # --scf "#eeeeee"
-      # --scrollbar autohide
-      # --tb "#005577"
-      # --tf "#eeeeee"
-      # --wrap
-      BEMENU_OPTS = let
-        bg = "#1d1721";       # slight purple
-        fg0 = "#d8d8d8";      # inactive text (light grey)
-        fg1 = "#ffffff";      # active text   (white)
-        accent0 = "#1f5e54";  # darker but saturated teal
-        accent1 = "#418379";  # teal (matches nixos-bg)
-        accent2 = "#5b938a";  # brighter but muted teal
-      in lib.concatStringsSep " " [
-        "--wrap --scrollbar autohide --fixed-height"
-        "--center --margin 45"
-        "--no-spacing"
-        # XXX: font size doesn't seem to take effect (would prefer larger)
-        "--fn 'monospace 14' --line-height 22 --border 3"
-        "--bdr '${accent0}'"                     # border
-        "--scf '${accent2}' --scb '${accent0}'"  # scrollbar
-        "--tb '${accent0}' --tf '${fg0}'"        # title
-        "--fb '${accent0}' --ff '${fg1}'"        # filter (i.e. text that's been entered)
-        "--hb '${accent1}' --hf '${fg1}'"        # selected item
-        "--nb '${bg}' --nf '${fg0}'"             # normal lines (even)
-        "--ab '${bg}' --af '${fg0}'"             # alternated lines (odd)
-        "--cf '${accent0}' --cb '${accent0}'"    # cursor (not very useful)
-      ];
       DEFAULT_COUNTRY = "US";
 
       SXMO_AUTOROTATE = "1";  # enable auto-rotation at launch. has no meaning in stock/upstream sxmo-utils
@@ -141,9 +56,22 @@
       # SXMO_ROTATION_GRAVITY = "12500";    # kinda uncomfortable when walking
       SXMO_ROTATION_GRAVITY = "12000";
       SXMO_SCREENSHOT_DIR = "/home/colin/Pictures";  # default: "$HOME"
-      # test new scales by running `swaymsg -- output DSI-1 scale x.y`
+
+      # sway/wayland scaling:
+      # - conflicting info out there on how scaling actually works
+      #   at the least, for things where it matters (mpv), it seems like scale settings have 0 effect on perf
+      # ways to enforce scaling:
+      # - <https://wiki.archlinux.org/title/HiDPI>
+      # - `swaymsg -- output DSI-1 scale 2.0`  (scales everything)
+      # - `dconf write /org/gnome/desktop/interface/text-scaling-factor 2.0`  (scales ONLY TEXT)
+      # - `GDK_DPI_SCALE=2.0`  (scales ONLY TEXT)
+      #
+      # application notes:
+      # - cozy: in landscape, playback position is not visible unless scale <= 1.7
+      #   - if in a tab, then scale 1.6 is the max
       # SXMO_SWAY_SCALE = "1.5";  # hard to press gPodder icons
-      SXMO_SWAY_SCALE = "1.8";
+      SXMO_SWAY_SCALE = "1.6";
+      # SXMO_SWAY_SCALE = "1.8";
       # SXMO_SWAY_SCALE = "2";
       SXMO_WORKSPACE_WRAPPING = "5";  # how many workspaces. default: 4
 

hosts/by-name/rescue/default.nix

@@ -4,12 +4,15 @@
     ./fs.nix
   ];
 
-  boot.loader.generic-extlinux-compatible.enable = true;
   boot.loader.efi.canTouchEfiVariables = false;
   sane.image.extraBootFiles = [ pkgs.bootpart-uefi-x86_64 ];
-  # sane.persist.enable = false;  # TODO: disable (but run `nix flake check` to ensure it works!)
+  sane.persist.enable = false;
   sane.nixcache.enable = false;  # don't want to be calling out to dead machines that we're *trying* to rescue
 
+  # auto-login at shell
+  services.getty.autologinUser = "colin";
+  # users.users.colin.initialPassword = "colin";
+
   # docs: https://nixos.org/manual/nixos/stable/options.html#opt-system.stateVersion
   system.stateVersion = "21.05";
 }

hosts/by-name/rescue/fs.nix

@@ -1,7 +1,7 @@
 { ... }:
 
 {
-  fileSystems."/" = {
+  fileSystems."/nix" = {
     device = "/dev/disk/by-uuid/44445555-6666-7777-8888-999900001111";
     fsType = "ext4";
   };

hosts/by-name/servo/default.nix

@@ -14,12 +14,11 @@
     signaldctl.enableFor.user.colin = true;
   };
 
-  sane.roles.ac = true;
   sane.roles.build-machine.enable = true;
   sane.roles.build-machine.emulation = false;
   sane.zsh.showDeadlines = false;  # ~/knowledge doesn't always exist
   sane.programs.consoleUtils.suggestedPrograms = [
-    "desktopConsoleUtils"
+    "pcConsoleUtils"
     "sane-scripts.stop-all-servo"
   ];
   sane.services.dyn-dns.enable = true;
@@ -30,6 +29,8 @@
   sane.services.wg-home.ip = config.sane.hosts.by-name."servo".wg-home.ip;
   sane.nixcache.substituters.servo = false;
   sane.nixcache.substituters.desko = false;
+  sane.nixcache.remote-builders.desko = false;
+  sane.nixcache.remote-builders.servo = false;
   # sane.services.duplicity.enable = true;  # TODO: re-enable after HW upgrade
 
   # automatically log in at the virtual consoles.

hosts/by-name/servo/fs.nix

@@ -1,7 +1,6 @@
 { ... }:
 
 {
-  sane.persist.root-on-tmpfs = true;
   # increase /tmp space (defaults to 50% of RAM) for building large nix things.
   # even the stock `nixpkgs.linux` consumes > 16 GB of tmp
   fileSystems."/tmp".options = [ "size=32G" ];
@@ -36,7 +35,7 @@
   };
   sane.fs."/mnt/persist/ext".mount = {};
 
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     # TODO: this is overly broad; only need media and share directories to be persisted
     { user = "colin"; group = "users"; path = "/var/lib/uninsane"; }
   ];
@@ -69,7 +68,7 @@
     the contents should be a subset of what's in ../media/datasets.
   '';
   # make sure large media is stored to the HDD
-  sane.persist.sys.ext = [
+  sane.persist.sys.byStore.ext = [
     {
       user = "colin";
       group = "users";

hosts/by-name/servo/net.nix

@@ -1,220 +1,233 @@
-{ config, pkgs, ... }:
+{ config, lib, pkgs, ... }:
 
+let
+  portOpts = with lib; types.submodule {
+    options = {
+      visibleTo.ovpn = mkOption {
+        type = types.bool;
+        default = false;
+      };
+    };
+  };
+in
 {
-  networking.domain = "uninsane.org";
+  options = with lib; {
-
+    sane.ports.ports = mkOption {
-  sane.ports.openFirewall = true;
+      # add the `visibleTo.ovpn` option
-  sane.ports.openUpnp = true;
+      type = types.attrsOf portOpts;
-
+    };
-  # view refused packets with: `sudo journalctl -k`
-  # networking.firewall.logRefusedPackets = true;
-
-  # The global useDHCP flag is deprecated, therefore explicitly set to false here.
-  # Per-interface useDHCP will be mandatory in the future, so this generated config
-  # replicates the default behaviour.
-  networking.useDHCP = false;
-  networking.interfaces.eth0.useDHCP = true;
-  # XXX colin: probably don't need this. wlan0 won't be populated unless i touch a value in networking.interfaces.wlan0
-  networking.wireless.enable = false;
-
-  # this is needed to forward packets from the VPN to the host
-  boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
-
-  # unless we add interface-specific settings for each VPN, we have to define nameservers globally.
-  # networking.nameservers = [
-  #   "1.1.1.1"
-  #   "9.9.9.9"
-  # ];
-
-  # use systemd's stub resolver.
-  # /etc/resolv.conf isn't sophisticated enough to use different servers per net namespace (or link).
-  # instead, running the stub resolver on a known address in the root ns lets us rewrite packets
-  # in the ovnps namespace to use the provider's DNS resolvers.
-  # a weakness is we can only query 1 NS at a time (unless we were to clone the packets?)
-  # there also seems to be some cache somewhere that's shared between the two namespaces.
-  # i think this is a libc thing. might need to leverage proper cgroups to _really_ kill it.
-  # - getent ahostsv4 www.google.com
-  # - try fix: <https://serverfault.com/questions/765989/connect-to-3rd-party-vpn-server-but-dont-use-it-as-the-default-route/766290#766290>
-  services.resolved.enable = true;
-  # without DNSSEC:
-  # - dig matrix.org => works
-  # - curl https://matrix.org => works
-  # with default DNSSEC:
-  # - dig matrix.org => works
-  # - curl https://matrix.org => fails
-  # i don't know why. this might somehow be interfering with the DNS run on this device (trust-dns)
-  services.resolved.dnssec = "false";
-  networking.nameservers = [
-    # use systemd-resolved resolver
-    # full resolver (which understands /etc/hosts) lives on 127.0.0.53
-    # stub resolver (just forwards upstream) lives on 127.0.0.54
-    "127.0.0.53"
-  ];
-
-  # nscd -- the Name Service Caching Daemon -- caches DNS query responses
-  # in a way that's unaware of my VPN routing, so routes are frequently poor against
-  # services which advertise different IPs based on geolocation.
-  # nscd claims to be usable without a cache, but in practice i can't get it to not cache!
-  # nsncd is the Name Service NON-Caching Daemon. it's a drop-in that doesn't cache;
-  # this is OK on the host -- because systemd-resolved caches. it's probably sub-optimal
-  # in the netns and we query upstream DNS more often than needed. hm.
-  # TODO: run a separate recursive resolver in each namespace.
-  services.nscd.enableNsncd = true;
-
-  # services.resolved.extraConfig = ''
-  #   # docs: `man resolved.conf`
-  #   # DNS servers to use via the `wg-ovpns` interface.
-  #   #   i hope that from the root ns, these aren't visible.
-  #   DNS=46.227.67.134%wg-ovpns 192.165.9.158%wg-ovpns
-  #   FallbackDNS=1.1.1.1 9.9.9.9
-  # '';
-
-  # OVPN CONFIG (https://www.ovpn.com):
-  # DOCS: https://nixos.wiki/wiki/WireGuard
-  # if you `systemctl restart wireguard-wg-ovpns`, make sure to also restart any other services in `NetworkNamespacePath = .../ovpns`.
-  # TODO: why not create the namespace as a seperate operation (nix config for that?)
-  networking.wireguard.enable = true;
-  networking.wireguard.interfaces.wg-ovpns = let
-    ip = "${pkgs.iproute2}/bin/ip";
-    in-ns = "${ip} netns exec ovpns";
-    iptables = "${pkgs.iptables}/bin/iptables";
-    veth-host-ip = "10.0.1.5";
-    veth-local-ip = "10.0.1.6";
-    vpn-ip = "185.157.162.178";
-    # DNS = 46.227.67.134, 192.165.9.158, 2a07:a880:4601:10f0:cd45::1, 2001:67c:750:1:cafe:cd45::1
-    vpn-dns = "46.227.67.134";
-  in {
-    privateKeyFile = config.sops.secrets.wg_ovpns_privkey.path;
-    # wg is active only in this namespace.
-    # run e.g. ip netns exec ovpns <some command like ping/curl/etc, it'll go through wg>
-    #   sudo ip netns exec ovpns ping www.google.com
-    interfaceNamespace = "ovpns";
-    ips = [
-      "185.157.162.178/32"
-    ];
-    peers = [
-      {
-        publicKey = "SkkEZDCBde22KTs/Hc7FWvDBfdOCQA4YtBEuC3n5KGs=";
-        endpoint = "185.157.162.10:9930";
-        # alternatively: use hostname, but that presents bootstrapping issues (e.g. if host net flakes)
-        # endpoint = "vpn36.prd.amsterdam.ovpn.com:9930";
-        allowedIPs = [ "0.0.0.0/0" ];
-        # nixOS says this is important for keeping NATs active
-        persistentKeepalive = 25;
-        # re-executes wg this often. docs hint that this might help wg notice DNS/hostname changes.
-        # so, maybe that helps if we specify endpoint as a domain name
-        # dynamicEndpointRefreshSeconds = 30;
-        # when refresh fails, try it again after this period instead.
-        # TODO: not avail until nixpkgs upgrade
-        # dynamicEndpointRefreshRestartSeconds = 5;
-      }
-    ];
-    preSetup = "" + ''
-      ${ip} netns add ovpns || echo "ovpns already exists"
-    '';
-    postShutdown = "" + ''
-      ${in-ns} ip link del ovpns-veth-b || echo "couldn't delete ovpns-veth-b"
-      ${ip} link del ovpns-veth-a || echo "couldn't delete ovpns-veth-a"
-      ${ip} netns delete ovpns || echo "couldn't delete ovpns"
-      # restore rules/routes
-      ${ip} rule del from ${veth-host-ip} lookup ovpns pref 50 || echo "couldn't delete init -> ovpns rule"
-      ${ip} route del default via ${veth-local-ip} dev ovpns-veth-a proto kernel src ${veth-host-ip} metric 1002 table ovpns || echo "couldn't delete init -> ovpns route"
-      ${ip} rule add from all lookup local pref 0
-      ${ip} rule del from all lookup local pref 100
-    '';
-    postSetup = "" + ''
-      # DOCS:
-      # - some of this approach is described here: <https://josephmuia.ca/2018-05-16-net-namespaces-veth-nat/>
-      # - iptables primer: <https://danielmiessler.com/study/iptables/>
-      # create veth pair
-      ${ip} link add ovpns-veth-a type veth peer name ovpns-veth-b
-      ${ip} addr add ${veth-host-ip}/24 dev ovpns-veth-a
-      ${ip} link set ovpns-veth-a up
-
-      # mv veth-b into the ovpns namespace
-      ${ip} link set ovpns-veth-b netns ovpns
-      ${in-ns} ip addr add ${veth-local-ip}/24 dev ovpns-veth-b
-      ${in-ns} ip link set ovpns-veth-b up
-
-      # make it so traffic originating from the host side of the veth
-      # is sent over the veth no matter its destination.
-      ${ip} rule add from ${veth-host-ip} lookup ovpns pref 50
-      # for traffic originating at the host veth to the WAN, use the veth as our gateway
-      # not sure if the metric 1002 matters.
-      ${ip} route add default via ${veth-local-ip} dev ovpns-veth-a proto kernel src ${veth-host-ip} metric 1002 table ovpns
-      # give the default route lower priority
-      ${ip} rule add from all lookup local pref 100
-      ${ip} rule del from all lookup local pref 0
-
-      # bridge HTTP traffic:
-      # any external port-80 request sent to the VPN addr will be forwarded to the rootns.
-      # this exists so LetsEncrypt can procure a cert for the MX over http.
-      # TODO: we could use _acme_challence.mx.uninsane.org CNAME to avoid this forwarding
-      # - <https://community.letsencrypt.org/t/where-does-letsencrypt-resolve-dns-from/37607/8>
-      ${in-ns} ${iptables} -A PREROUTING -t nat -p tcp --dport 80 -m iprange --dst-range ${vpn-ip} \
-        -j DNAT --to-destination ${veth-host-ip}:80
-
-      # we also bridge DNS traffic
-      ${in-ns} ${iptables} -A PREROUTING -t nat -p udp --dport 53 -m iprange --dst-range ${vpn-ip} \
-        -j DNAT --to-destination ${veth-host-ip}
-      ${in-ns} ${iptables} -A PREROUTING -t nat -p tcp --dport 53 -m iprange --dst-range ${vpn-ip} \
-        -j DNAT --to-destination ${veth-host-ip}
-
-      # in order to access DNS in this netns, we need to route it to the VPN's nameservers
-      # - alternatively, we could fix DNS servers like 1.1.1.1.
-      ${in-ns} ${iptables} -A OUTPUT -t nat -p udp --dport 53 -m iprange --dst-range 127.0.0.53 \
-        -j DNAT --to-destination ${vpn-dns}:53
-    '';
   };
 
-  # create a new routing table that we can use to proxy traffic out of the root namespace
+  config = {
-  # through the ovpns namespace, and to the WAN via VPN.
+    networking.domain = "uninsane.org";
-  networking.iproute2.rttablesExtraConfig = ''
+
-    5 ovpns
+    sane.ports.openFirewall = true;
-  '';
+    sane.ports.openUpnp = true;
-  networking.iproute2.enable = true;
+
+    # view refused packets with: `sudo journalctl -k`
+    # networking.firewall.logRefusedPackets = true;
+
+    # The global useDHCP flag is deprecated, therefore explicitly set to false here.
+    # Per-interface useDHCP will be mandatory in the future, so this generated config
+    # replicates the default behaviour.
+    networking.useDHCP = false;
+    networking.interfaces.eth0.useDHCP = true;
+    # XXX colin: probably don't need this. wlan0 won't be populated unless i touch a value in networking.interfaces.wlan0
+    networking.wireless.enable = false;
+
+    # this is needed to forward packets from the VPN to the host
+    boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
+
+    # unless we add interface-specific settings for each VPN, we have to define nameservers globally.
+    # networking.nameservers = [
+    #   "1.1.1.1"
+    #   "9.9.9.9"
+    # ];
+
+    # use systemd's stub resolver.
+    # /etc/resolv.conf isn't sophisticated enough to use different servers per net namespace (or link).
+    # instead, running the stub resolver on a known address in the root ns lets us rewrite packets
+    # in the ovnps namespace to use the provider's DNS resolvers.
+    # a weakness is we can only query 1 NS at a time (unless we were to clone the packets?)
+    # there also seems to be some cache somewhere that's shared between the two namespaces.
+    # i think this is a libc thing. might need to leverage proper cgroups to _really_ kill it.
+    # - getent ahostsv4 www.google.com
+    # - try fix: <https://serverfault.com/questions/765989/connect-to-3rd-party-vpn-server-but-dont-use-it-as-the-default-route/766290#766290>
+    services.resolved.enable = true;
+    # without DNSSEC:
+    # - dig matrix.org => works
+    # - curl https://matrix.org => works
+    # with default DNSSEC:
+    # - dig matrix.org => works
+    # - curl https://matrix.org => fails
+    # i don't know why. this might somehow be interfering with the DNS run on this device (trust-dns)
+    services.resolved.dnssec = "false";
+    networking.nameservers = [
+      # use systemd-resolved resolver
+      # full resolver (which understands /etc/hosts) lives on 127.0.0.53
+      # stub resolver (just forwards upstream) lives on 127.0.0.54
+      "127.0.0.53"
+    ];
+
+    # nscd -- the Name Service Caching Daemon -- caches DNS query responses
+    # in a way that's unaware of my VPN routing, so routes are frequently poor against
+    # services which advertise different IPs based on geolocation.
+    # nscd claims to be usable without a cache, but in practice i can't get it to not cache!
+    # nsncd is the Name Service NON-Caching Daemon. it's a drop-in that doesn't cache;
+    # this is OK on the host -- because systemd-resolved caches. it's probably sub-optimal
+    # in the netns and we query upstream DNS more often than needed. hm.
+    # TODO: run a separate recursive resolver in each namespace.
+    services.nscd.enableNsncd = true;
+
+    # services.resolved.extraConfig = ''
+    #   # docs: `man resolved.conf`
+    #   # DNS servers to use via the `wg-ovpns` interface.
+    #   #   i hope that from the root ns, these aren't visible.
+    #   DNS=46.227.67.134%wg-ovpns 192.165.9.158%wg-ovpns
+    #   FallbackDNS=1.1.1.1 9.9.9.9
+    # '';
+
+    # OVPN CONFIG (https://www.ovpn.com):
+    # DOCS: https://nixos.wiki/wiki/WireGuard
+    # if you `systemctl restart wireguard-wg-ovpns`, make sure to also restart any other services in `NetworkNamespacePath = .../ovpns`.
+    # TODO: why not create the namespace as a seperate operation (nix config for that?)
+    networking.wireguard.enable = true;
+    networking.wireguard.interfaces.wg-ovpns = let
+      ip = "${pkgs.iproute2}/bin/ip";
+      in-ns = "${ip} netns exec ovpns";
+      iptables = "${pkgs.iptables}/bin/iptables";
+      veth-host-ip = "10.0.1.5";
+      veth-local-ip = "10.0.1.6";
+      vpn-ip = "185.157.162.178";
+      # DNS = 46.227.67.134, 192.165.9.158, 2a07:a880:4601:10f0:cd45::1, 2001:67c:750:1:cafe:cd45::1
+      vpn-dns = "46.227.67.134";
+      bridgePort = port: proto: ''
+        ${in-ns} ${iptables} -A PREROUTING -t nat -p ${proto} --dport ${port} -m iprange --dst-range ${vpn-ip} \
+          -j DNAT --to-destination ${veth-host-ip}
+      '';
+      bridgeStatements = lib.foldlAttrs
+        (acc: port: portCfg: acc ++ (builtins.map (bridgePort port) portCfg.protocol))
+        []
+        config.sane.ports.ports;
+    in {
+      privateKeyFile = config.sops.secrets.wg_ovpns_privkey.path;
+      # wg is active only in this namespace.
+      # run e.g. ip netns exec ovpns <some command like ping/curl/etc, it'll go through wg>
+      #   sudo ip netns exec ovpns ping www.google.com
+      interfaceNamespace = "ovpns";
+      ips = [
+        "185.157.162.178/32"
+      ];
+      peers = [
+        {
+          publicKey = "SkkEZDCBde22KTs/Hc7FWvDBfdOCQA4YtBEuC3n5KGs=";
+          endpoint = "185.157.162.10:9930";
+          # alternatively: use hostname, but that presents bootstrapping issues (e.g. if host net flakes)
+          # endpoint = "vpn36.prd.amsterdam.ovpn.com:9930";
+          allowedIPs = [ "0.0.0.0/0" ];
+          # nixOS says this is important for keeping NATs active
+          persistentKeepalive = 25;
+          # re-executes wg this often. docs hint that this might help wg notice DNS/hostname changes.
+          # so, maybe that helps if we specify endpoint as a domain name
+          # dynamicEndpointRefreshSeconds = 30;
+          # when refresh fails, try it again after this period instead.
+          # TODO: not avail until nixpkgs upgrade
+          # dynamicEndpointRefreshRestartSeconds = 5;
+        }
+      ];
+      preSetup = ''
+        ${ip} netns add ovpns || echo "ovpns already exists"
+      '';
+      postShutdown = ''
+        ${in-ns} ip link del ovpns-veth-b || echo "couldn't delete ovpns-veth-b"
+        ${ip} link del ovpns-veth-a || echo "couldn't delete ovpns-veth-a"
+        ${ip} netns delete ovpns || echo "couldn't delete ovpns"
+        # restore rules/routes
+        ${ip} rule del from ${veth-host-ip} lookup ovpns pref 50 || echo "couldn't delete init -> ovpns rule"
+        ${ip} route del default via ${veth-local-ip} dev ovpns-veth-a proto kernel src ${veth-host-ip} metric 1002 table ovpns || echo "couldn't delete init -> ovpns route"
+        ${ip} rule add from all lookup local pref 0
+        ${ip} rule del from all lookup local pref 100
+      '';
+      postSetup = ''
+        # DOCS:
+        # - some of this approach is described here: <https://josephmuia.ca/2018-05-16-net-namespaces-veth-nat/>
+        # - iptables primer: <https://danielmiessler.com/study/iptables/>
+        # create veth pair
+        ${ip} link add ovpns-veth-a type veth peer name ovpns-veth-b
+        ${ip} addr add ${veth-host-ip}/24 dev ovpns-veth-a
+        ${ip} link set ovpns-veth-a up
+
+        # mv veth-b into the ovpns namespace
+        ${ip} link set ovpns-veth-b netns ovpns
+        ${in-ns} ip addr add ${veth-local-ip}/24 dev ovpns-veth-b
+        ${in-ns} ip link set ovpns-veth-b up
+
+        # make it so traffic originating from the host side of the veth
+        # is sent over the veth no matter its destination.
+        ${ip} rule add from ${veth-host-ip} lookup ovpns pref 50
+        # for traffic originating at the host veth to the WAN, use the veth as our gateway
+        # not sure if the metric 1002 matters.
+        ${ip} route add default via ${veth-local-ip} dev ovpns-veth-a proto kernel src ${veth-host-ip} metric 1002 table ovpns
+        # give the default route lower priority
+        ${ip} rule add from all lookup local pref 100
+        ${ip} rule del from all lookup local pref 0
+
+        # in order to access DNS in this netns, we need to route it to the VPN's nameservers
+        # - alternatively, we could fix DNS servers like 1.1.1.1.
+        ${in-ns} ${iptables} -A OUTPUT -t nat -p udp --dport 53 -m iprange --dst-range 127.0.0.53 \
+          -j DNAT --to-destination ${vpn-dns}:53
+      '' + (lib.concatStringsSep "\n" bridgeStatements);
+    };
+
+    # create a new routing table that we can use to proxy traffic out of the root namespace
+    # through the ovpns namespace, and to the WAN via VPN.
+    networking.iproute2.rttablesExtraConfig = ''
+      5 ovpns
+    '';
+    networking.iproute2.enable = true;
 
 
-  # HURRICANE ELECTRIC CONFIG:
+    # HURRICANE ELECTRIC CONFIG:
-  # networking.sits = {
+    # networking.sits = {
-  #   hurricane = {
+    #   hurricane = {
-  #     remote = "216.218.226.238";
+    #     remote = "216.218.226.238";
-  #     local = "192.168.0.5";
+    #     local = "192.168.0.5";
-  #     # local = "10.0.0.5";
+    #     # local = "10.0.0.5";
-  #     # remote = "10.0.0.1";
+    #     # remote = "10.0.0.1";
-  #     # local = "10.0.0.22";
+    #     # local = "10.0.0.22";
-  #     dev = "eth0";
+    #     dev = "eth0";
-  #     ttl = 255;
+    #     ttl = 255;
-  #   };
+    #   };
-  # };
+    # };
-  # networking.interfaces."hurricane".ipv6 = {
+    # networking.interfaces."hurricane".ipv6 = {
-  #   addresses = [
+    #   addresses = [
-  #     # mx.uninsane.org (publically routed /64)
+    #     # mx.uninsane.org (publically routed /64)
-  #     {
+    #     {
-  #       address = "2001:470:b:465::1";
+    #       address = "2001:470:b:465::1";
-  #       prefixLength = 128;
+    #       prefixLength = 128;
-  #     }
+    #     }
-  #     # client addr
+    #     # client addr
-  #     # {
+    #     # {
-  #     #   address = "2001:470:a:466::2";
+    #     #   address = "2001:470:a:466::2";
-  #     #   prefixLength = 64;
+    #     #   prefixLength = 64;
-  #     # }
+    #     # }
-  #   ];
+    #   ];
-  #   routes = [
+    #   routes = [
-  #     {
+    #     {
-  #       address = "::";
+    #       address = "::";
-  #       prefixLength = 0;
+    #       prefixLength = 0;
-  #       # via = "2001:470:a:466::1";
+    #       # via = "2001:470:a:466::1";
-  #     }
+    #     }
-  #   ];
+    #   ];
-  # };
+    # };
 
-  # # after configuration, we want the hurricane device to look like this:
+    # # after configuration, we want the hurricane device to look like this:
-  # # hurricane: flags=209<UP,POINTOPOINT,RUNNING,NOARP>  mtu 1480
+    # # hurricane: flags=209<UP,POINTOPOINT,RUNNING,NOARP>  mtu 1480
-  # #            inet6 2001:470:a:450::2  prefixlen 64  scopeid 0x0<global>
+    # #            inet6 2001:470:a:450::2  prefixlen 64  scopeid 0x0<global>
-  # #            inet6 fe80::c0a8:16  prefixlen 64  scopeid 0x20<link>
+    # #            inet6 fe80::c0a8:16  prefixlen 64  scopeid 0x20<link>
-  # #            sit  txqueuelen 1000  (IPv6-in-IPv4)
+    # #            sit  txqueuelen 1000  (IPv6-in-IPv4)
-  # # test with:
+    # # test with:
-  # #   curl --interface hurricane http://[2607:f8b0:400a:80b::2004]
+    # #   curl --interface hurricane http://[2607:f8b0:400a:80b::2004]
-  # #   ping 2607:f8b0:400a:80b::2004
+    # #   ping 2607:f8b0:400a:80b::2004
+  };
 }

hosts/by-name/servo/services/calibre.nix

@@ -12,7 +12,7 @@ in
 # > AttributeError: 'NoneType' object has no attribute 'query'
 lib.mkIf false
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     { inherit user group; mode = "0700"; path = svc-dir; }
   ];
 

hosts/by-name/servo/services/coturn.nix

@@ -0,0 +1,124 @@
+# TURN/STUN NAT traversal service
+# commonly used to establish realtime calls with prosody, or possibly matrix/synapse
+#
+# - <https://github.com/coturn/coturn/>
+# - `man turnserver`
+# - config docs: <https://github.com/coturn/coturn/blob/master/examples/etc/turnserver.conf>
+#
+# N.B. during operation it's NORMAL to see "error 401".
+# during session creation:
+# - client sends Allocate request
+# - server replies error 401, providing a realm and nonce
+# - client uses realm + nonce + shared secret to construct an auth key & call Allocate again
+# - server replies Allocate Success Response
+# - source: <https://stackoverflow.com/a/66643135>
+#
+# N.B. this safest implementation routes all traffic THROUGH A VPN
+# - that adds a lot of latency, but in practice turns out to be inconsequential.
+#   i guess ICE allows clients to prefer the other party's lower-latency server, in practice?
+# - still, this is the "safe" implementation because STUN works with IP addresses instead of domain names:
+#   1. client A queries the STUN server to determine its own IP address/port.
+#   2. client A tells client B which IP address/port client A is visible on.
+#   3. client B contacts that IP address/port
+#   this only works so long as the IP address/port which STUN server sees client A on is publicly routable.
+#   that is NOT the case when the STUN server and client A are on the same LAN
+#   even if client A contacts the STUN server via its WAN address with port reflection enabled.
+#   hence, there's no obvious way to put the STUN server on the same LAN as either client and expect the rest to work.
+{ lib, ... }:
+let
+  # TODO: this range could be larger, but right now that's costly because each element is its own UPnP forward
+  # TURN port range (inclusive)
+  turnPortLow = 49152;
+  turnPortHigh = 49167;
+  turnPortRange = lib.range turnPortLow turnPortHigh;
+in
+{
+  sane.ports.ports = lib.mkMerge ([
+    {
+      "3478" = {
+        # this is the "control" port.
+        # i.e. no client data is forwarded through it, but it's where clients request tunnels.
+        protocol = [ "tcp" "udp" ];
+        # visibleTo.lan = true;
+        # visibleTo.wan = true;
+        visibleTo.ovpn = true;
+        description = "colin-stun-turn";
+      };
+      "5349" = {
+        # the other port 3478 also supports TLS/DTLS, but presumably clients wanting TLS will default 5349
+        protocol = [ "tcp" ];
+        # visibleTo.lan = true;
+        # visibleTo.wan = true;
+        visibleTo.ovpn = true;
+        description = "colin-stun-turn-over-tls";
+      };
+    }
+  ] ++ (builtins.map
+    (port: {
+      "${builtins.toString port}" = let
+        count = port - turnPortLow + 1;
+        numPorts = turnPortHigh - turnPortLow + 1;
+      in {
+        protocol = [ "tcp" "udp" ];
+        # visibleTo.lan = true;
+        # visibleTo.wan = true;
+        visibleTo.ovpn = true;
+        description = "colin-turn-${builtins.toString count}-of-${builtins.toString numPorts}";
+      };
+    })
+    turnPortRange
+  ));
+
+  services.nginx.virtualHosts."turn.uninsane.org" = {
+    # allow ACME to procure a cert via nginx for this domain
+    enableACME = true;
+  };
+  sane.dns.zones."uninsane.org".inet = {
+    # CNAME."turn" = "servo.wan";
+    # CNAME."turn" = "ovpns";
+    # CNAME."turn" = "native";
+    # XXX: SRV records have to point to something with a A/AAAA record; no CNAMEs
+    A."turn" = "%AOVPNS%";
+    # A."turn" = "%AWAN%";
+
+    SRV."_stun._udp" =                        "5 50 3478 turn";
+    SRV."_stun._tcp" =                        "5 50 3478 turn";
+    SRV."_stuns._tcp" =                       "5 50 5349 turn";
+    SRV."_turn._udp" =                        "5 50 3478 turn";
+    SRV."_turn._tcp" =                        "5 50 3478 turn";
+    SRV."_turns._tcp" =                       "5 50 5349 turn";
+  };
+
+  sane.derived-secrets."/var/lib/coturn/shared_secret.bin" = {
+    encoding = "base64";
+    # TODO: make this not globally readable
+    acl.mode = "0644";
+  };
+  sane.fs."/var/lib/coturn/shared_secret.bin".wantedBeforeBy = [ "coturn.service" ];
+
+  # provide access to certs
+  users.users.turnserver.extraGroups = [ "nginx" ];
+
+  services.coturn.enable = true;
+  services.coturn.realm = "turn.uninsane.org";
+  services.coturn.cert = "/var/lib/acme/turn.uninsane.org/fullchain.pem";
+  services.coturn.pkey = "/var/lib/acme/turn.uninsane.org/key.pem";
+  services.coturn.use-auth-secret = true;
+  services.coturn.static-auth-secret-file = "/var/lib/coturn/shared_secret.bin";
+  services.coturn.lt-cred-mech = true;
+  services.coturn.min-port = turnPortLow;
+  services.coturn.max-port = turnPortHigh;
+  # services.coturn.secure-stun = true;
+  services.coturn.extraConfig = lib.concatStringsSep "\n" [
+    "verbose"
+    # "Verbose"  #< even MORE verbosity than "verbose"
+    # "no-multicast-peers"  # disables sending to IPv4 broadcast addresses (e.g. 224.0.0.0/3)
+    "listening-ip=10.0.1.5"
+    # "external-ip=185.157.162.178/10.0.1.5"
+    "external-ip=185.157.162.178"
+    # "listening-ip=10.78.79.51"  # can be specified multiple times; omit for *
+    # "external-ip=97.113.128.229/10.78.79.51"
+    # "external-ip=97.113.128.229"
+    # "mobility"  # "mobility with ICE (MICE) specs support" (?)
+  ];
+}

hosts/by-name/servo/services/default.nix

@@ -2,6 +2,7 @@
 {
   imports = [
     ./calibre.nix
+    ./coturn.nix
     ./ddns-afraid.nix
     ./ddns-he.nix
     ./email
@@ -17,14 +18,16 @@
     ./komga.nix
     ./lemmy.nix
     ./matrix
+    ./monero.nix
     ./navidrome.nix
     ./nginx.nix
     ./nixserve.nix
-    ./ntfy.nix
+    ./ntfy
     ./pict-rs.nix
     ./pleroma.nix
     ./postgres.nix
-    ./prosody.nix
+    ./prosody
+    ./slskd.nix
     ./transmission.nix
     ./trust-dns.nix
     ./wikipedia.nix

hosts/by-name/servo/services/ejabberd.nix

@@ -40,8 +40,11 @@ let
   turnPortHigh = 49167;
   turnPortRange = lib.range turnPortLow turnPortHigh;
 in
+# XXX(2023/10/15): disabled in favor of Prosody.
+# everything configured below was fine: used ejabberd for several months.
+lib.mkIf false
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     { user = "ejabberd"; group = "ejabberd"; path = "/var/lib/ejabberd"; }
   ];
   sane.ports.ports = lib.mkMerge ([
@@ -114,6 +117,9 @@ in
     turnPortRange
   ));
 
+  # this ejabberd config uses builtin STUN/TURN server, so hack to ensure no other implementation fights for ports
+  services.coturn.enable = false;
+
   # provide access to certs
   # TODO: this should just be `acme`. then we also add nginx to the `acme` group.
   #   why is /var/lib/acme/* owned by `nginx` group??

hosts/by-name/servo/services/email/default.nix

@@ -22,6 +22,13 @@
 #     - but postfix delegates authorization of that outgoing mail to dovecot, on the server side
 #
 # - local clients (i.e. sendmail) interact only with postfix
+#
+# debugging: general connectivity issues
+# - test that inbound port 25 is unblocked:
+#   - `curl https://canyouseeme.org/ --data 'port=25&IP=185.157.162.178' | grep 'see your service'`
+#     - and retry with port 465, 587
+#     - i think this API requires the queried IP match the source IP
+#   - if necessary, `systemctl stop postfix` and `sudo nc -l 185.157.162.178 25`, then try https://canyouseeme.org
 
 { ... }:
 {

hosts/by-name/servo/services/email/postfix.nix

@@ -18,7 +18,7 @@ let
   };
 in
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     # TODO: mode? could be more granular
     { user = "opendkim"; group = "opendkim"; path = "/var/lib/opendkim"; }
     { user = "root"; group = "root"; path = "/var/lib/postfix"; }
@@ -28,21 +28,25 @@ in
     # "/var/lib/dovecot"
   ];
 
-  sane.ports.ports."25" = {
+  # XXX(2023/10/20): opening these ports in the firewall has the OPPOSITE effect as intended.
-    protocol = [ "tcp" ];
+  # these ports are only routable so long as they AREN'T opened.
-    visibleTo.ovpn = true;
+  # probably some cursed interaction with network namespaces introduced after 2023/10/10.
-    description = "colin-smtp-mx.uninsane.org";
+  # sane.ports.ports."25" = {
-  };
+  #   protocol = [ "tcp" ];
-  sane.ports.ports."465" = {
+  #   # XXX visibleTo.lan effectively means "open firewall, but don't configure any NAT/forwarding"
-    protocol = [ "tcp" ];
+  #   visibleTo.lan = true;
-    visibleTo.ovpn = true;
+  #   description = "colin-smtp-mx.uninsane.org";
-    description = "colin-smtps-mx.uninsane.org";
+  # };
-  };
+  # sane.ports.ports."465" = {
-  sane.ports.ports."587" = {
+  #   protocol = [ "tcp" ];
-    protocol = [ "tcp" ];
+  #   visibleTo.lan = true;
-    visibleTo.ovpn = true;
+  #   description = "colin-smtps-mx.uninsane.org";
-    description = "colin-smtps-submission-mx.uninsane.org";
+  # };
-  };
+  # sane.ports.ports."587" = {
+  #   protocol = [ "tcp" ];
+  #   visibleTo.lan = true;
+  #   description = "colin-smtps-submission-mx.uninsane.org";
+  # };
 
   # exists only to manage certs for Postfix
   services.nginx.virtualHosts."mx.uninsane.org" = {

hosts/by-name/servo/services/export/default.nix

@@ -29,7 +29,7 @@
   # - `sudo btrfs qgroup limit 20G /mnt/persist/ext/persist/var/export/playground`
   # to query the quota/status:
   # - `sudo btrfs qgroup show -re /var/export/playground`
-  sane.persist.sys.ext = [
+  sane.persist.sys.byStore.ext = [
     { user = "root"; group = "export"; mode = "0775"; path = "/var/export/playground"; }
   ];
 

hosts/by-name/servo/services/export/sftpgo.nix

@@ -175,5 +175,10 @@ in
   systemd.services.sftpgo.serviceConfig = {
     ReadOnlyPaths = [ "/var/export" ];
     ReadWritePaths = [ "/var/export/playground" ];
+    after = [ "network-online.target" ];
+    wants = [ "network-online.target" ];
+
+    Restart = "always";
+    RestartSec = "20s";
   };
 }

hosts/by-name/servo/services/freshrss.nix

@@ -15,7 +15,7 @@
     owner = config.users.users.freshrss.name;
     mode = "0400";
   };
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     { user = "freshrss"; group = "freshrss"; path = "/var/lib/freshrss"; }
   ];
 

hosts/by-name/servo/services/gitea.nix

@@ -2,7 +2,7 @@
 { config, pkgs, lib, ... }:
 
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     # TODO: mode? could be more granular
     { user = "git"; group = "gitea"; path = "/var/lib/gitea"; }
   ];
@@ -13,6 +13,10 @@
   services.gitea.appName = "Perfectly Sane Git";
   # services.gitea.disableRegistration = true;
 
+  services.gitea.database.createDatabase = false;  #< silence warning which wants db user and name to be equal
+  # TODO: remove this after merge: <https://github.com/NixOS/nixpkgs/pull/268849>
+  services.gitea.database.socket = "/run/postgresql";  #< would have been set if createDatabase = true
+
   # gitea doesn't create the git user
   users.users.git = {
     description = "Gitea Service";

hosts/by-name/servo/services/ipfs.nix

@@ -10,7 +10,7 @@
 
 lib.mkIf false # i don't actively use ipfs anymore
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     # TODO: mode? could be more granular
     { user = "261"; group = "261"; path = "/var/lib/ipfs"; }
   ];

hosts/by-name/servo/services/jackett.nix

@@ -1,7 +1,7 @@
 { ... }:
 
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     # TODO: mode? we only need this to save Indexer creds ==> migrate to config?
     { user = "root"; group = "root"; path = "/var/lib/jackett"; }
   ];

hosts/by-name/servo/services/jellyfin.nix

@@ -40,7 +40,7 @@
     description = "colin-jellyfin-https-lan";
   };
 
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     { user = "jellyfin"; group = "jellyfin"; mode = "0700"; path = "/var/lib/jellyfin"; }
   ];
   sane.fs."/var/lib/jellyfin/config/logging.json" = {

hosts/by-name/servo/services/komga.nix

@@ -4,7 +4,7 @@ let
   inherit (svc-cfg) user group port stateDir;
 in
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     { inherit user group; mode = "0700"; path = stateDir; }
   ];
 

hosts/by-name/servo/services/matrix/default.nix

@@ -1,6 +1,16 @@
 # docs: <https://nixos.wiki/wiki/Matrix>
 # docs: <https://nixos.org/manual/nixos/stable/index.html#module-services-matrix-synapse>
 # example config: <https://github.com/matrix-org/synapse/blob/develop/docs/sample_config.yaml>
+#
+# ENABLING PUSH NOTIFICATIONS (with UnifiedPush/ntfy):
+# - Matrix "pushers" API spec: <https://spec.matrix.org/latest/client-server-api/#post_matrixclientv3pushersset>
+# - first, view notification settings:
+#   - obtain your client's auth token. e.g. Element -> profile -> help/about -> access token.
+#   - `curl --header 'Authorization: Bearer <your_access_token>' localhost:8008/_matrix/client/v3/pushers | jq .`
+# - enable a new notification destination:
+#   - `curl --header "Authorization: Bearer <your_access_token>" --data '{ "app_display_name": "<topic>", "app_id": "ntfy.uninsane.org", "data": { "url": "https://ntfy.uninsane.org/_matrix/push/v1/notify", "format": "event_id_only" }, "device_display_name": "<topic>", "kind": "http", "lang": "en-US", "profile_tag": "", "pushkey": "<topic>" }' localhost:8008/_matrix/client/v3/pushers/set`
+# - delete a notification destination by setting `kind` to `null` (otherwise, request is identical to above)
+#
 { config, lib, pkgs, ... }:
 
 {
@@ -10,7 +20,7 @@
     ./signal.nix
   ];
 
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     { user = "matrix-synapse"; group = "matrix-synapse"; path = "/var/lib/matrix-synapse"; }
   ];
   services.matrix-synapse.enable = true;
@@ -60,25 +70,23 @@
     config.sops.secrets."matrix_synapse_secrets.yaml".path
   ];
 
-  # services.matrix-synapse.extraConfigFiles = [builtins.toFile "matrix-synapse-extra-config" ''
+  systemd.services.matrix-synapse.postStart = ''
-  #   admin_contact: "admin.matrix@uninsane.org"
+    ACCESS_TOKEN=$(${pkgs.coreutils}/bin/cat ${config.sops.secrets.matrix_access_token.path})
-  #   registrations_require_3pid:
+    TOPIC=$(${pkgs.coreutils}/bin/cat ${config.sops.secrets.ntfy-sh-topic.path})
-  #     - email
+
-  #   email:
+    echo "ensuring ntfy push gateway"
-  #     smtp_host: "mx.uninsane.org"
+    ${pkgs.curl}/bin/curl \
-  #     smtp_port: 587
+      --header "Authorization: Bearer $ACCESS_TOKEN" \
-  #     smtp_user: "matrix-synapse"
+      --data "{ \"app_display_name\": \"ntfy-adapter\", \"app_id\": \"ntfy.uninsane.org\", \"data\": { \"url\": \"https://ntfy.uninsane.org/_matrix/push/v1/notify\", \"format\": \"event_id_only\" }, \"device_display_name\": \"ntfy-adapter\", \"kind\": \"http\", \"lang\": \"en-US\", \"profile_tag\": \"\", \"pushkey\": \"$TOPIC\" }" \
-  #     smtp_pass: "${secrets.matrix-synapse.smtp_pass}"
+      localhost:8008/_matrix/client/v3/pushers/set
-  #     require_transport_security: true
+
-  #     enable_tls: true
+    echo "registered push gateways:"
-  #     notif_from: "%(app)s <notify.matrix@uninsane.org>"
+    ${pkgs.curl}/bin/curl \
-  #     app_name: "Uninsane Matrix"
+      --header "Authorization: Bearer $ACCESS_TOKEN" \
-  #     enable_notifs: true
+      localhost:8008/_matrix/client/v3/pushers \
-  #     validation_token_lifetime: 96h
+      | ${pkgs.jq}/bin/jq .
-  #     invite_client_location: "https://web.matrix.uninsane.org"
+  '';
-  #     subjects:
+
-  #       email_validation: "[%(server_name)s] Validate your email"
-  # ''];
 
   # new users may be registered on the CLI:
   #   register_new_matrix_user -c /nix/store/8n6kcka37jhmi4qpd2r03aj71pkyh21s-homeserver.yaml http://localhost:8008
@@ -149,4 +157,9 @@
   sops.secrets."matrix_synapse_secrets.yaml" = {
     owner = config.users.users.matrix-synapse.name;
   };
+  sops.secrets."matrix_access_token" = {
+    owner = config.users.users.matrix-synapse.name;
+  };
+  # provide access to ntfy-sh-topic secret
+  users.users.matrix-synapse.extraGroups = [ "ntfy-sh" ];
 }

hosts/by-name/servo/services/matrix/discord-puppet.nix

@@ -5,7 +5,7 @@
 # - recommended to use mautrix-discord: <https://github.com/NixOS/nixpkgs/pull/200462>
 lib.mkIf false
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     { user = "matrix-synapse"; group = "matrix-synapse"; path = "/var/lib/mx-puppet-discord"; }
   ];
 

hosts/by-name/servo/services/matrix/irc.nix

@@ -101,7 +101,7 @@ in
     })
   ];
 
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     # TODO: mode?
     { user = "matrix-appservice-irc"; group = "matrix-appservice-irc"; path = "/var/lib/matrix-appservice-irc"; }
   ];

hosts/by-name/servo/services/matrix/signal.nix

@@ -2,7 +2,7 @@
 # - <https://github.com/mautrix/signal/blob/master/mautrix_signal/example-config.yaml>
 { config, pkgs, ... }:
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     { user = "mautrix-signal"; group = "mautrix-signal"; path = "/var/lib/mautrix-signal"; }
     { user = "signald"; group = "signald"; path = "/var/lib/signald"; }
   ];

hosts/by-name/servo/services/monero.nix

@@ -0,0 +1,37 @@
+# as of 2023/11/26: complete downloaded blockchain should be 200GiB on disk, give or take.
+{ ... }:
+{
+  sane.persist.sys.byStore.ext = [
+    # /var/lib/monero/lmdb is what consumes most of the space
+    { user = "monero"; group = "monero"; path = "/var/lib/monero"; }
+  ];
+
+  services.monero.enable = true;
+  services.monero.limits.upload = 5000;  # in kB/s
+  services.monero.extraConfig = ''
+    # see: monero doc/ANONYMITY_NETWORKS.md
+    #
+    # "If any anonymity network is enabled, transactions being broadcast that lack a valid 'context'
+    # (i.e. the transaction did not come from a P2P connection) will only be sent to peers on anonymity networks."
+    #
+    # i think this means that setting tx-proxy here ensures any transactions sent locally to my node (via RPC)
+    # will be sent over an anonymity network.
+    tx-proxy=i2p,127.0.0.1:9000
+    tx-proxy=tor,127.0.0.1:9050
+  '';
+
+  services.i2p.enable = true;
+  # tor: `tor.enable` doesn't start a relay, exit node, proxy, etc. it's minimal.
+  # tor.client.enable configures a torsocks proxy, accessible *only* to localhost.
+  services.tor.enable = true;
+  services.tor.client.enable = true;
+
+  # monero ports: <https://monero.stackexchange.com/questions/604/what-ports-does-monero-use-rpc-p2p-etc>
+  # - 18080 = "P2P" monero node <-> monero node connections
+  # - 18081 = "RPC" monero client -> monero node connections
+  sane.ports.ports."18080" = {
+    protocol = [ "tcp" ];
+    visibleTo.wan = true;
+    description = "colin-monero-p2p";
+  };
+}

hosts/by-name/servo/services/navidrome.nix

@@ -1,7 +1,7 @@
 { lib, ... }:
 
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     { user = "navidrome"; group = "navidrome"; path = "/var/lib/navidrome"; }
   ];
   services.navidrome.enable = true;

hosts/by-name/servo/services/nginx.nix

@@ -1,4 +1,5 @@
-# docs: https://nixos.wiki/wiki/Nginx
+# docs: <https://nixos.wiki/wiki/Nginx>
+# docs: <https://nginx.org/en/docs/>
 { config, lib, pkgs, ... }:
 
 let
@@ -133,7 +134,7 @@ in
   security.acme.acceptTerms = true;
   security.acme.defaults.email = "admin.acme@uninsane.org";
 
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     # TODO: mode?
     { user = "acme"; group = "acme"; path = "/var/lib/acme"; }
     { user = "colin"; group = "users"; path = "/var/www/sites"; }

hosts/by-name/servo/services/ntfy/default.nix

@@ -0,0 +1,14 @@
+# ntfy: UnifiedPush notification delivery system
+# - used to get push notifications out of Matrix and onto a Phone (iOS, Android, or a custom client)
+{ config, ... }:
+{
+  imports = [
+    ./ntfy-waiter.nix
+    ./ntfy-sh.nix
+  ];
+  sops.secrets."ntfy-sh-topic" = {
+    mode = "0440";
+    owner = config.users.users.ntfy-sh.name;
+    group = config.users.users.ntfy-sh.name;
+  };
+}

hosts/by-name/servo/services/ntfy/ntfy-sh.nix

@@ -30,7 +30,7 @@ let
   altPort = 2587;
 in
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     # not 100% necessary to persist this, but ntfy does keep a 12hr (by default) cache
     # for pushing notifications to users who become offline.
     # ACLs also live here.
@@ -61,12 +61,6 @@ in
     ${pkgs.ntfy-sh}/bin/ntfy access everyone "$topic" read-write
   '';
 
-  sops.secrets."ntfy-sh-topic" = {
-    mode = "0440";
-    owner = config.users.users.ntfy-sh.name;
-    group = config.users.users.ntfy-sh.name;
-  };
-
 
   services.nginx.virtualHosts."ntfy.uninsane.org" = {
     forceSSL = true;

hosts/by-name/servo/services/ntfy/ntfy-waiter

@@ -0,0 +1,151 @@
+#!/usr/bin/env nix-shell
+#!nix-shell -i python3 -p "python3.withPackages (ps: [  ])" -p ntfy-sh
+
+import argparse
+import logging
+import os
+import socket
+import subprocess
+import sys
+import threading
+import time
+
+logger = logging.getLogger(__name__)
+
+LISTEN_QUEUE = 3
+WAKE_MESSAGE = b'notification\n'
+
+class Client:
+    def __init__(self, sock, addr_info, live_after: float):
+        self.live_after = live_after
+        self.sock = sock
+        self.addr_info = addr_info
+
+    def __cmp__(self, other: 'Client'):
+        return cmp(self.addr_info, other.addr_info)
+
+    def try_notify(self, message: bytes) -> bool:
+        """
+        returns true if we send a packet to notify client.
+        fals otherwise (e.g. the socket is dead).
+        """
+        ttl = self.live_after - time.time()
+        if ttl > 0:
+            logger.debug(f"sleeping {ttl:.2f}s until client {self.addr_info} is ready to receive notification")
+            time.sleep(ttl)
+
+        try:
+            self.sock.sendall(message)
+        except Exception as e:
+            logger.warning(f"failed to notify client {self.addr_info} {e}")
+            return False
+        else:
+            logger.info(f"successfully notified {self.addr_info}: {message}")
+            return True
+
+class Adapter:
+    def __init__(self, host: str, port: int, silence: int, topic: str):
+        self.host = host
+        self.port = port
+        self.silence = silence
+        self.topic = topic
+        self.clients = set()
+
+    def log_clients(self):
+        clients_str = '\n'.join(f'  {c.addr_info}' for c in self.clients)
+        logger.debug(f"clients alive ({len(self.clients)}):\n{clients_str}")
+
+    def add_client(self, client: Client):
+        # it's a little bit risky to keep more than one client at the same IP address,
+        # because it's possible a notification comes in and we ring the old connection,
+        # even when the new connection says "don't ring yet".
+        for c in set(self.clients):
+            if c.addr_info[0] == client.addr_info[0]:
+                logger.info(f"purging old client before adding new one at same address: {c.addr_info} -> {client.addr_info}")
+                self.clients.remove(c)
+
+        logger.info(f"accepted client at {client.addr_info}")
+        self.clients.add(client)
+
+    def listener_loop(self):
+        logger.info(f"listening for connections on {self.host}:{self.port}")
+        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        s.bind((self.host, self.port))
+        s.listen(LISTEN_QUEUE)
+        while True:
+            conn, addr_info = s.accept()
+            self.add_client(Client(conn, addr_info, live_after = time.time() + self.silence))
+
+    def notify_clients(self, message: bytes = WAKE_MESSAGE):
+        # notify every client, and drop any which have disconnected.
+        # note that we notify based on age (oldest -> youngest)
+        # because notifying young clients might entail sleeping until they're ready.
+        clients = sorted(self.clients, key=lambda c: (c.live_after, c.addr_info))
+
+        dead_clients = [
+            c for c in clients if not c.try_notify(message)
+        ]
+        for c in dead_clients:
+            self.clients.remove(c)
+
+        self.log_clients()
+
+    def notify_loop(self):
+        logger.info("waiting for notification events")
+        ntfy_proc = subprocess.Popen(
+            [
+                "ntfy",
+                "sub",
+                f"https://ntfy.uninsane.org/{self.topic}"
+            ],
+            stdout=subprocess.PIPE
+        )
+        for line in iter(ntfy_proc.stdout.readline, b''):
+            logger.debug(f"received notification: {line}")
+            self.notify_clients()
+
+def get_topic() -> str:
+    return open('/run/secrets/ntfy-sh-topic', 'rt').read().strip()
+
+def run_forever(callable):
+    try:
+        callable()
+    except Exception as e:
+        logger.error(f"{callable} failed: {e}")
+    else:
+        logger.error(f"{callable} unexpectedly returned")
+    # sys.exit(1)
+    os._exit(1)  # sometimes `sys.exit()` doesn't actually exit...
+
+def main():
+    logging.basicConfig()
+    logging.getLogger().setLevel(logging.DEBUG)
+
+    parser = argparse.ArgumentParser(description="accept connections and notify the other end upon ntfy activity, with a guaranteed amount of silence")
+    parser.add_argument('--verbose', action='store_true')
+    parser.add_argument('--host', type=str, default='')
+    parser.add_argument('--port', type=int)
+    parser.add_argument('--silence', type=int, help="number of seconds to remain silent upon accepting a connection")
+
+    args = parser.parse_args()
+
+    if args.verbose:
+        logging.getLogger().setLevel(logging.DEBUG)
+    else:
+        logging.getLogger().setLevel(logging.INFO)
+
+    adapter = Adapter(args.host, args.port, args.silence, get_topic())
+
+    listener_loop = threading.Thread(target=run_forever, name="listener_loop", args=(adapter.listener_loop,))
+    notify_loop = threading.Thread(target=run_forever, name="notify_loop", args=(adapter.notify_loop,))
+
+    # TODO: this method of exiting seems to sometimes leave the listener behind (?)
+    # preventing anyone else from re-binding the port.
+    listener_loop.start()
+    notify_loop.start()
+    listener_loop.join()
+    notify_loop.join()
+
+
+if __name__ == '__main__':
+    main()

hosts/by-name/servo/services/ntfy/ntfy-waiter.nix

@@ -0,0 +1,72 @@
+# service which adapts ntfy-sh into something suitable specifically for the Pinephone's
+# wake-on-lan (WoL) feature.
+# notably, it provides a mechanism by which the caller can be confident of an interval in which
+# zero traffic will occur on the TCP connection, thus allowing it to enter sleep w/o fear of hitting
+# race conditions in the Pinephone WoL feature.
+{ config, lib, pkgs, ... }:
+let
+  cfg = config.sane.ntfy-waiter;
+  portLow = 5550;
+  portHigh = 5559;
+  portRange = lib.range portLow portHigh;
+  numPorts = portHigh - portLow + 1;
+  mkService = port: let
+    silence = port - portLow;
+    flags = lib.optional cfg.verbose "--verbose";
+    cli = [
+      "${cfg.package}/bin/ntfy-waiter"
+      "--port"
+      "${builtins.toString port}"
+      "--silence"
+      "${builtins.toString silence}"
+    ] ++ flags;
+  in {
+    "ntfy-waiter-${builtins.toString silence}" = {
+      # TODO: run not as root (e.g. as ntfy-sh)
+      description = "wait for notification, with ${builtins.toString silence} seconds of guaranteed silence";
+      serviceConfig = {
+        Type = "simple";
+        Restart = "always";
+        RestartSec = "5s";
+        ExecStart = lib.concatStringsSep " " cli;
+      };
+      after = [ "network.target" ];
+      wantedBy = [ "default.target" ];
+    };
+  };
+in
+{
+  options = with lib; {
+    sane.ntfy-waiter.enable = mkOption {
+      type = types.bool;
+      default = true;
+    };
+    sane.ntfy-waiter.verbose = mkOption {
+      type = types.bool;
+      default = true;
+    };
+    sane.ntfy-waiter.package = mkOption {
+      type = types.package;
+      default = pkgs.static-nix-shell.mkPython3Bin {
+        pname = "ntfy-waiter";
+        src = ./.;
+        pkgs = [ "ntfy-sh" ];
+      };
+      description = ''
+        exposed to provide an attr-path by which one may build the package for manual testing.
+      '';
+    };
+  };
+
+  config = lib.mkIf cfg.enable {
+    sane.ports.ports = lib.mkMerge (lib.forEach portRange (port: {
+      "${builtins.toString port}" = {
+        protocol = [ "tcp" ];
+        visibleTo.lan = true;
+        visibleTo.wan = true;
+        description = "colin-notification-waiter-${builtins.toString (port+1)}-of-${builtins.toString numPorts}";
+      };
+    }));
+    systemd.services = lib.mkMerge (builtins.map mkService portRange);
+  };
+}

hosts/by-name/servo/services/pict-rs.nix

@@ -5,7 +5,7 @@ let
   cfg = config.services.pict-rs;
 in
 {
-  sane.persist.sys.plaintext = lib.mkIf cfg.enable [
+  sane.persist.sys.byStore.plaintext = lib.mkIf cfg.enable [
     { user = "pict-rs"; group = "pict-rs"; path = cfg.dataDir; }
   ];
 

hosts/by-name/servo/services/pleroma.nix

@@ -14,7 +14,7 @@ let
   # logLevel = "debug";
 in
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     { user = "pleroma"; group = "pleroma"; path = "/var/lib/pleroma"; }
   ];
   services.pleroma.enable = true;

hosts/by-name/servo/services/postgres.nix

@@ -6,7 +6,7 @@ let
   KiB = n: 1024*n;
 in
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     # TODO: mode?
     { user = "postgres"; group = "postgres"; path = "/var/lib/postgresql"; }
   ];

hosts/by-name/servo/services/prosody.nix

@@ -1,81 +0,0 @@
-# example configs:
-# - <https://github.com/kittywitch/nixfiles/blob/main/services/prosody.nix>
-# create users with:
-# - `sudo -u prosody prosodyctl adduser colin@uninsane.org`
-
-{ lib, ... }:
-
-# XXX disabled: doesn't send messages to nixnet.social (only receives them).
-# nixnet runs ejabberd, so revisiting that.
-lib.mkIf false
-{
-  sane.persist.sys.plaintext = [
-    { user = "prosody"; group = "prosody"; path = "/var/lib/prosody"; }
-  ];
-  sane.ports.ports."5222" = {
-    protocol = [ "tcp" ];
-    visibleTo.lan = true;
-    visibleTo.wan = true;
-    description = "colin-xmpp-client-to-server";
-  };
-  sane.ports.ports."5269" = {
-    protocol = [ "tcp" ];
-    visibleTo.wan = true;
-    description = "colin-xmpp-server-to-server";
-  };
-  sane.ports.ports."5280" = {
-    protocol = [ "tcp" ];
-    visibleTo.lan = true;
-    visibleTo.wan = true;
-    description = "colin-xmpp-bosh";
-  };
-  sane.ports.ports."5281" = {
-    protocol = [ "tcp" ];
-    visibleTo.lan = true;
-    visibleTo.wan = true;
-    description = "colin-xmpp-prosody-https";  # necessary?
-  };
-
-  # provide access to certs
-  users.users.prosody.extraGroups = [ "nginx" ];
-
-  security.acme.certs."uninsane.org".extraDomainNames = [
-    "conference.xmpp.uninsane.org"
-    "upload.xmpp.uninsane.org"
-  ];
-
-  services.prosody = {
-    enable = true;
-    admins = [ "colin@uninsane.org" ];
-    # allowRegistration = false;
-    # extraConfig = ''
-    #   s2s_require_encryption = true
-    #   c2s_require_encryption = true
-    # '';
-
-    extraModules = [ "private" "vcard" "privacy" "compression" "component" "muc" "pep" "adhoc" "lastactivity" "admin_adhoc" "blocklist"];
-
-    ssl.cert = "/var/lib/acme/uninsane.org/fullchain.pem";
-    ssl.key = "/var/lib/acme/uninsane.org/key.pem";
-
-    muc = [
-      {
-        domain = "conference.xmpp.uninsane.org";
-      }
-    ];
-    uploadHttp.domain = "upload.xmpp.uninsane.org";
-
-    virtualHosts = {
-      localhost = {
-        domain = "localhost";
-        enabled = true;
-      };
-      "xmpp.uninsane.org" = {
-        domain = "uninsane.org";
-        enabled = true;
-        ssl.cert = "/var/lib/acme/uninsane.org/fullchain.pem";
-        ssl.key = "/var/lib/acme/uninsane.org/key.pem";
-      }; 
-    };
-  };
-}

hosts/by-name/servo/services/prosody/default.nix

@@ -0,0 +1,289 @@
+# example configs:
+# - official: <https://prosody.im/doc/example_config>
+# - nixos: <https://github.com/kittywitch/nixfiles/blob/main/services/prosody.nix>
+# config options:
+# - <https://prosody.im/doc/configure>
+#
+# modules:
+# - main: <https://prosody.im/doc/modules>
+# - community: <https://modules.prosody.im/index.html>
+#
+# debugging:
+# - logging:
+#   - enable `stanza_debug` module
+#   - enable `log.debug = "*syslog"` in extraConfig
+# - interactive:
+#   - `telnet localhost 5582` (this is equal to `prosodyctl shell` -- but doesn't hang)
+#     - `watch:stanzas(target_spec, filter)` -> to log stanzas, for version > 0.12
+#   - console docs: <https://prosody.im/doc/console>
+#   - can modify/inspect arbitrary internals (lua) by prefixing line with `> `
+#     - e.g. `> _G` to print all globals
+#
+# sanity checks:
+# - `sudo -u prosody -g prosody prosodyctl check connectivity`
+# - `sudo -u prosody -g prosody prosodyctl check turn`
+# - `sudo -u prosody -g prosody prosodyctl check turn -v --ping=stun.conversations.im`
+#   - checks that my stun/turn server is usable by clients of conversations.im (?)
+# - `sudo -u prosody -g prosody prosodyctl check`  (dns, config, certs)
+#
+#
+# create users with:
+# - `sudo -u prosody prosodyctl adduser colin@uninsane.org`
+#
+#
+# federation/support matrix:
+# - nixnet.services (runs ejabberd):
+#   - WORKS: sending and receiving PMs and calls (2023/10/15)
+#     - N.B.: it didn't originally work; was solved by disabling the lua-unbound DNS option & forcing the system/local resolver
+# - cheogram (XMPP <-> SMS gateway):
+#   - WORKS: sending and receiving PMs, images (2023/10/15)
+#   - PARTIAL: calls (xmpp -> tel works; tel -> xmpp fails)
+#     - maybe i need to setup stun/turn
+#
+# TODO:
+# - enable push notifications (mod_cloud_notify)
+# - optimize coturn (e.g. move off of the VPN!)
+# - ensure muc is working
+# - enable file uploads
+#   - "upload.xmpp.uninsane.org:http_upload: URL: <https://upload.xmpp.uninsane.org:5281/upload> - Ensure this can be reached by users"
+# - disable or fix bosh (jabber over http):
+#   - "certmanager: No certificate/key found for client_https port 0"
+
+{ lib, pkgs, ... }:
+
+let
+  # enables very verbose logging
+  enableDebug = false;
+in
+{
+  sane.persist.sys.byStore.plaintext = [
+    { user = "prosody"; group = "prosody"; path = "/var/lib/prosody"; }
+  ];
+  sane.ports.ports."5000" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-xmpp-prosody-fileshare-proxy65";
+  };
+  sane.ports.ports."5222" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-xmpp-client-to-server";
+  };
+  sane.ports.ports."5223" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-xmpps-client-to-server";  # XMPP over TLS
+  };
+  sane.ports.ports."5269" = {
+    protocol = [ "tcp" ];
+    visibleTo.wan = true;
+    description = "colin-xmpp-server-to-server";
+  };
+  sane.ports.ports."5270" = {
+    protocol = [ "tcp" ];
+    visibleTo.wan = true;
+    description = "colin-xmpps-server-to-server";  # XMPP over TLS
+  };
+  sane.ports.ports."5280" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-xmpp-bosh";
+  };
+  sane.ports.ports."5281" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-xmpp-prosody-https";  # necessary?
+  };
+
+  users.users.prosody.extraGroups = [
+    "nginx"  # provide access to certs
+    "ntfy-sh"  # access to secret ntfy topic
+  ];
+
+  security.acme.certs."uninsane.org".extraDomainNames = [
+    "xmpp.uninsane.org"
+    "conference.xmpp.uninsane.org"
+    "upload.xmpp.uninsane.org"
+  ];
+
+  # exists so the XMPP server's cert can obtain altNames for all its resources
+  services.nginx.virtualHosts."xmpp.uninsane.org" = {
+    useACMEHost = "uninsane.org";
+  };
+  services.nginx.virtualHosts."conference.xmpp.uninsane.org" = {
+    useACMEHost = "uninsane.org";
+  };
+  services.nginx.virtualHosts."upload.xmpp.uninsane.org" = {
+    useACMEHost = "uninsane.org";
+  };
+
+  sane.dns.zones."uninsane.org".inet = {
+    # XXX: SRV records have to point to something with a A/AAAA record; no CNAMEs
+    A."xmpp" =                "%ANATIVE%";
+    CNAME."conference.xmpp" = "xmpp";
+    CNAME."upload.xmpp" =     "xmpp";
+
+    # _Service._Proto.Name    TTL Class SRV    Priority Weight Port Target
+    # - <https://xmpp.org/extensions/xep-0368.html>
+    # something's requesting the SRV records for conference.xmpp, so let's include it
+    # nothing seems to request XMPP SRVs for the other records (except @)
+    # lower numerical priority field tells clients to prefer this method
+    SRV."_xmpps-client._tcp.conference.xmpp" =       "3 50 5223 xmpp";
+    SRV."_xmpps-server._tcp.conference.xmpp" =       "3 50 5270 xmpp";
+    SRV."_xmpp-client._tcp.conference.xmpp" =        "5 50 5222 xmpp";
+    SRV."_xmpp-server._tcp.conference.xmpp" =        "5 50 5269 xmpp";
+
+    SRV."_xmpps-client._tcp" =                "3 50 5223 xmpp";
+    SRV."_xmpps-server._tcp" =                "3 50 5270 xmpp";
+    SRV."_xmpp-client._tcp" =                 "5 50 5222 xmpp";
+    SRV."_xmpp-server._tcp" =                 "5 50 5269 xmpp";
+  };
+
+  # help Prosody find its certificates.
+  # pointing it to /var/lib/acme doesn't quite work because it expects the private key
+  # to be named `privkey.pem` instead of acme's `key.pem`
+  # <https://prosody.im/doc/certificates#automatic_location>
+  sane.fs."/etc/prosody/certs/uninsane.org/fullchain.pem" = {
+    symlink.target = "/var/lib/acme/uninsane.org/fullchain.pem";
+    wantedBeforeBy = [ "prosody.service" ];
+  };
+  sane.fs."/etc/prosody/certs/uninsane.org/privkey.pem" = {
+    symlink.target = "/var/lib/acme/uninsane.org/key.pem";
+    wantedBeforeBy = [ "prosody.service" ];
+  };
+
+  services.prosody = {
+    enable = true;
+    package = pkgs.prosody.override {
+      # XXX(2023/10/15): build without lua-unbound support.
+      # this forces Prosody to fall back to the default Lua DNS resolver, which seems more reliable.
+      # fixes errors like "unbound.queryXYZUV: Resolver error: out of memory"
+      # related: <https://issues.prosody.im/1737#comment-11>
+      lua.withPackages = selector: pkgs.lua.withPackages (p:
+        selector (p // { luaunbound = null; })
+      );
+      # withCommunityModules = [ "turncredentials" ];
+    };
+    admins = [ "colin@uninsane.org" ];
+    # allowRegistration = false;  # defaults to false
+
+    muc = [
+      {
+        domain = "conference.xmpp.uninsane.org";
+      }
+    ];
+    uploadHttp.domain = "upload.xmpp.uninsane.org";
+
+    virtualHosts = {
+      # "Prosody requires at least one enabled VirtualHost to function. You can
+      # safely remove or disable 'localhost' once you have added another."
+      # localhost = {
+      #   domain = "localhost";
+      #   enabled = true;
+      # };
+      "xmpp.uninsane.org" = {
+        domain = "uninsane.org";
+        enabled = true;
+      };
+    };
+
+    ## modules:
+    # these are enabled by default, via <repo:nixos/nixpkgs:/pkgs/servers/xmpp/prosody/default.nix>
+    # - cloud_notify
+    # - http_upload
+    # - vcard_muc
+    # these are enabled by the module defaults (services.prosody.modules.<foo>)
+    # - admin_adhoc
+    # - blocklist
+    # - bookmarks
+    # - carbons
+    # - cloud_notify
+    # - csi
+    # - dialback
+    # - disco
+    # - http_files
+    # - mam
+    # - pep
+    # - ping
+    # - private
+    #   - XEP-0049: let clients store arbitrary (private) data on the server
+    # - proxy65
+    #   - XEP-0065: allow server to proxy file transfers between two clients who are behind NAT
+    # - register
+    # - roster
+    # - saslauth
+    # - smacks
+    # - time
+    # - tls
+    # - uptime
+    # - vcard_legacy
+    # - version
+
+    extraPluginPaths = [ ./modules ];
+
+    extraModules = [
+      # admin_shell: allows `prosodyctl shell` to work
+      # see: <https://prosody.im/doc/modules/mod_admin_shell>
+      # see: <https://prosody.im/doc/console>
+      "admin_shell"
+      "admin_telnet"  #< needed by admin_shell
+      # lastactivity: XEP-0012: allow users to query how long another user has been idle for
+      # - not sure why i enabled this; think it was in someone's config i referenced
+      "lastactivity"
+      # allows prosody to share TURN/STUN secrets with XMPP clients to provide them access to the coturn server.
+      # see: <https://prosody.im/doc/coturn>
+      "turn_external"
+      # legacy coturn integration
+      # see: <https://modules.prosody.im/mod_turncredentials.html>
+      # "turncredentials"
+      "sane_ntfy"
+    ] ++ lib.optionals enableDebug [
+      "stanza_debug"  #< logs EVERY stanza as debug: <https://prosody.im/doc/modules/mod_stanza_debug>
+    ];
+
+    extraConfig = ''
+      local function readAll(file)
+        local f = assert(io.open(file, "rb"))
+        local content = f:read("*all")
+        f:close()
+        -- remove trailing newline
+        return string.gsub(content, "%s+", "")
+      end
+
+      -- logging docs:
+      -- - <https://prosody.im/doc/logging>
+      -- - <https://prosody.im/doc/advanced_logging>
+      -- levels: debug, info, warn, error
+      log = {
+        ${if enableDebug then "debug" else "info"} = "*syslog";
+      }
+
+      -- see: <https://prosody.im/doc/certificates#automatic_location>
+      -- try to solve: "certmanager: Error indexing certificate directory /etc/prosody/certs: cannot open /etc/prosody/certs: No such file or directory"
+      -- only, this doesn't work because prosody doesn't like acme's naming scheme
+      -- certificates = "/var/lib/acme"
+
+      c2s_direct_tls_ports = { 5223 }
+      s2s_direct_tls_ports = { 5270 }
+
+      turn_external_host = "turn.uninsane.org"
+      turn_external_secret = readAll("/var/lib/coturn/shared_secret.bin")
+      -- turn_external_user = "prosody"
+
+      -- legacy mod_turncredentials integration
+      -- turncredentials_host = "turn.uninsane.org"
+      -- turncredentials_secret = readAll("/var/lib/coturn/shared_secret.bin")
+
+      ntfy_binary = "${pkgs.ntfy-sh}/bin/ntfy"
+      ntfy_topic = readAll("/run/secrets/ntfy-sh-topic")
+
+      -- s2s_require_encryption = true
+      -- c2s_require_encryption = true
+    '';
+  };
+}

hosts/by-name/servo/services/prosody/modules/mod_sane_ntfy.lua

@@ -0,0 +1,52 @@
+-- simple proof-of-concept Prosody module
+-- module development guide: <https://prosody.im/doc/developers/modules>
+-- module API docs: <https://prosody.im/doc/developers/moduleapi>
+--
+-- much of this code is lifted from Prosody's own `mod_cloud_notify`
+
+local jid = require"util.jid";
+
+local ntfy = module:get_option_string("ntfy_binary", "ntfy");
+local ntfy_topic = module:get_option_string("ntfy_topic", "xmpp");
+
+module:log("info", "initialized");
+
+local function is_urgent(stanza)
+  if stanza.name == "message" then
+    if stanza:get_child("propose", "urn:xmpp:jingle-message:0") then
+      return true, "jingle call";
+    end
+  end
+end
+
+local function publish_ntfy(message)
+  -- message should be the message to publish
+  local ntfy_url = string.format("https://ntfy.uninsane.org/%s", ntfy_topic)
+  local cmd = string.format("%s pub %q %q", ntfy, ntfy_url, message)
+  module.log("debug", "invoking ntfy: %s", cmd)
+  local success, reason, code = os.execute(cmd)
+  if not success then
+    module:log("warn", "ntfy failed: %s => %s %d", cmd, reason, code)
+  end
+end
+
+
+local function archive_message_added(event)
+  -- event is: { origin = origin, stanza = stanza, for_user = store_user, id = id }
+  local stanza = event.stanza;
+  local to = stanza.attr.to;
+  to = to and jid.split(to) or event.origin.username;
+
+  -- only notify if the stanza destination is the mam user we store it for
+  if event.for_user == to then
+    local is_urgent_stanza, urgent_reason = is_urgent(event.stanza);
+
+    if is_urgent_stanza then
+      module:log("info", "urgent push for %s (%s)", to, urgent_reason);
+      publish_ntfy(urgent_reason)
+    end
+  end
+end
+
+
+module:hook("archive-message-added", archive_message_added);

hosts/by-name/servo/services/slskd.nix

@@ -0,0 +1,74 @@
+# Soulseek daemon (p2p file sharing with an emphasis on Music)
+# docs: <https://github.com/slskd/slskd/blob/master/docs/config.md>
+#
+# config precedence (higher precedence overrules lower precedence):
+# - Default Values < Environment Variables < YAML Configuraiton File < Command Line Arguments
+{ config, lib, ... }:
+{
+  sane.persist.sys.byStore.plaintext = [
+    { user = "slskd"; group = "slskd"; path = "/var/lib/slskd"; }
+  ];
+  sops.secrets."slskd_env" = {
+    owner = config.users.users.slskd.name;
+    mode = "0400";
+  };
+
+  users.users.slskd.extraGroups = [ "media" ];
+
+  sane.ports.ports."50000" = {
+    protocol = [ "tcp" ];
+    # not visible to WAN: i run this in a separate netns
+    visibleTo.ovpn = true;
+    description = "colin-soulseek";
+  };
+
+  sane.dns.zones."uninsane.org".inet.CNAME."soulseek" = "native";
+
+  services.nginx.virtualHosts."soulseek.uninsane.org" = {
+    forceSSL = true;
+    enableACME = true;
+    locations."/" = {
+      proxyPass = "http://10.0.1.6:5001";
+      proxyWebsockets = true;
+    };
+  };
+
+  services.slskd.enable = true;
+  # env file, for auth (SLSKD_SLSK_PASSWORD, SLSKD_SLSK_USERNAME)
+  services.slskd.environmentFile = config.sops.secrets.slskd_env.path;
+  services.slskd.settings = {
+    soulseek.diagnostic_level = "Debug";  # one of "None"|"Warning"|"Info"|"Debug"
+    shares.directories = [
+      # folders to share
+      # syntax: <https://github.com/slskd/slskd/blob/master/docs/config.md#directories>
+      # [Alias]/path/on/disk
+      # NOTE: Music library is quick to scan; videos take a solid 10min to scan.
+      # TODO: re-enable the other libraries
+      # "[Audioooks]/var/lib/uninsane/media/Books/Audiobooks"
+      # "[Books]/var/lib/uninsane/media/Books/Books"
+      # "[Manga]/var/lib/uninsane/media/Books/Visual"
+      # "[games]/var/lib/uninsane/media/games"
+      "[Music]/var/lib/uninsane/media/Music"
+      # "[Film]/var/lib/uninsane/media/Videos/Film"
+      # "[Shows]/var/lib/uninsane/media/Videos/Shows"
+    ];
+    # directories.downloads = "..." # TODO
+    # directories.incomplete = "..." # TODO
+    # what unit is this? kbps??
+    global.upload.speed_limit = 32000;
+    web.logging = true;
+    debug = true;
+    flags.no_logo = true;  # don't show logo at start
+    # flags.volatile = true;  # store searches and active transfers in RAM (completed transfers still go to disk). rec for btrfs/zfs
+  };
+
+  systemd.services.slskd = {
+    serviceConfig = {
+      # run this behind the OVPN static VPN
+      NetworkNamespacePath = "/run/netns/ovpns";
+      Restart = "on-failure";
+      RestartSec = "30s";
+      Group = "media";
+    };
+  };
+}

hosts/by-name/servo/services/transmission.nix

@@ -1,7 +1,7 @@
 { config, pkgs, ... }:
 
 {
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     # TODO: mode? we need this specifically for the stats tracking in .config/
     { user = "transmission"; group = config.users.users.transmission.group; path = "/var/lib/transmission"; }
   ];
@@ -91,5 +91,10 @@
   };
 
   sane.dns.zones."uninsane.org".inet.CNAME."bt" = "native";
+  sane.ports.ports."51413" = {
+    protocol = [ "tcp" "udp" ];
+    visibleTo.ovpn = true;
+    description = "colin-bittorrent";
+  };
 }
 

hosts/by-name/servo/services/trust-dns.nix

@@ -11,12 +11,15 @@ in lib.mkMerge [
   # don't bind to IPv6 until i explicitly test that stack
   services.trust-dns.settings.listen_addrs_ipv6 = [];
   services.trust-dns.quiet = true;
+  # FIXME(2023/11/26): services.trust-dns.debug doesn't log requests: use RUST_LOG=debug env for that.
+  # - see: <https://github.com/hickory-dns/hickory-dns/issues/2082>
   # services.trust-dns.debug = true;
 
   sane.ports.ports."53" = {
     protocol = [ "udp" "tcp" ];
     visibleTo.lan = true;
     visibleTo.wan = true;
+    visibleTo.ovpn = true;
     description = "colin-dns-hosting";
   };
 
@@ -145,11 +148,12 @@ in lib.mkMerge [
             -e s/%CNAMENATIVE%/servo.${flavor}/ \
             -e s/%ANATIVE%/${anative}/ \
             -e s/%AWAN%/$wan/ \
+            -e s/%AOVPNS%/185.157.162.178/ \
             ${zoneTemplate} > ${zoneFor flavor}
         '';
         serviceConfig = config.systemd.services.trust-dns.serviceConfig // {
           ExecStart = ''
-            ${pkgs.trust-dns}/bin/trust-dns \
+            ${pkgs.trust-dns}/bin/${pkgs.trust-dns.meta.mainProgram} \
             --port ${builtins.toString port} \
             --zonedir ${zoneDirFor flavor}/ \
             --config ${configFile} ${flagsStr}

hosts/common/default.nix

@@ -5,6 +5,7 @@
     ./fs.nix
     ./hardware
     ./home
+    ./hostnames.nix
     ./hosts.nix
     ./ids.nix
     ./machine-id.nix
@@ -21,25 +22,54 @@
   sane.nixcache.enable-trusted-keys = true;
   sane.nixcache.enable = lib.mkDefault true;
   sane.persist.enable = lib.mkDefault true;
+  sane.root-on-tmpfs = lib.mkDefault true;
   sane.programs.sysadminUtils.enableFor.system = lib.mkDefault true;
   sane.programs.consoleUtils.enableFor.user.colin = lib.mkDefault true;
 
-  nixpkgs.config.allowUnfree = true;
+  nixpkgs.config.allowUnfree = true;  # NIXPKGS_ALLOW_UNFREE=1
-  nixpkgs.config.allowBroken = true;  # NIXPKGS_ALLOW_BROKEN
+  nixpkgs.config.allowBroken = true;  # NIXPKGS_ALLOW_BROKEN=1
 
   # time.timeZone = "America/Los_Angeles";
   time.timeZone = "Etc/UTC";  # DST is too confusing for me => use a stable timezone
 
-  # allow `nix flake ...` command
-  # TODO: is this still required?
   nix.extraOptions = ''
+    # see: `man nix.conf`
+    # useful when a remote builder has a faster internet connection than me
+    builders-use-substitutes = true  # default: false
+    # maximum seconds to wait when connecting to binary substituter
+    connect-timeout = 3  # default: 0
+    # download-attempts = 5  # default: 5
+    # allow `nix flake ...` command
     experimental-features = nix-command flakes
+    # whether to build from source when binary substitution fails
+    fallback = true  # default: false
+    # whether to keep building dependencies if any other one fails
+    keep-going = true  # default: false
+    # whether to keep build-only dependencies of GC roots (e.g. C compiler) when doing GC
+    keep-outputs = true  # default: false
+    # how many lines to show from failed build
+    log-lines = 30  # default: 10
+    # narinfo-cache-negative-ttl = 3600  # default: 3600
+    # whether to use ~/.local/state/nix/profile instead of ~/.nix-profile, etc
+    use-xdg-base-directories = true  # default: false
+    # whether to warn if repository has uncommited changes
+    warn-dirty = false  # default: true
   '';
   # hardlinks identical files in the nix store to save 25-35% disk space.
   # unclear _when_ this occurs. it's not a service.
   # does the daemon continually scan the nix store?
   # does the builder use some content-addressed db to efficiently dedupe?
   nix.settings.auto-optimise-store = true;
+  # TODO: see if i can remove this?
+  nix.settings.trusted-users = [ "root" ];
+
+  services.journald.extraConfig = ''
+    # docs: `man journald.conf`
+    # merged journald config is deployed to /etc/systemd/journald.conf
+    [Journal]
+    # disable journal compression because the underlying fs is compressed
+    Compress=no
+  '';
 
   systemd.services.nix-daemon.serviceConfig = {
     # the nix-daemon manages nix builders

hosts/common/feeds.nix

@@ -57,84 +57,57 @@ let
     };
 
   podcasts = [
-    (fromDb "lexfridman.com/podcast" // rat)
+    (fromDb "acquiredlpbonussecretsecret.libsyn.com" // tech)  # ACQ2 - more "Acquired" episodes
-    ## Astral Codex Ten
-    (fromDb "sscpodcast.libsyn.com" // rat)
-    ## Less Wrong Curated
-    (fromDb "feeds.libsyn.com/421877" // rat)
-    ## Econ Talk
-    (fromDb "feeds.simplecast.com/wgl4xEgL" // rat)
-    ## Cory Doctorow -- both podcast & text entries
-    (fromDb "craphound.com" // pol)
-    ## Maggie Killjoy -- referenced by Cory Doctorow
-    (fromDb "omny.fm/shows/cool-people-who-did-cool-stuff" // pol)
-    ## also Maggie Killjoy
-    (fromDb "feeds.megaphone.fm/behindthebastards" // pol)
-    ## Jennifer Briney
-    (fromDb "congressionaldish.libsyn.com" // pol)
-    (fromDb "werenotwrong.fireside.fm" // pol)
-    (fromDb "politicalorphanage.libsyn.com" // pol)
-    # (mkPod "https://podcasts.la.utexas.edu/this-is-democracy/feed/podcast/" // pol // weekly)
-    ## Civboot -- https://anchor.fm/civboot
-    (fromDb "anchor.fm/s/34c7232c/podcast/rss" // tech)
-    ## Emerge: making sense of what's next -- <https://www.whatisemerging.com/emergepodcast>
-    (mkPod "https://anchor.fm/s/21bc734/podcast/rss" // pol // infrequent)
-    (fromDb "feeds.feedburner.com/80000HoursPodcast" // rat)
-    ## Daniel Huberman on sleep
-    (fromDb "feeds.megaphone.fm/hubermanlab" // uncat)
-    ## Multidisciplinary Association for Psychedelic Studies
-    (fromDb "mapspodcast.libsyn.com" // uncat)
     (fromDb "allinchamathjason.libsyn.com" // pol)
-    (fromDb "feeds.transistor.fm/acquired" // tech)
+    (fromDb "anchor.fm/s/34c7232c/podcast/rss" // tech)  # Civboot -- https://anchor.fm/civboot
-    ## ACQ2 - more "Acquired" episodes
-    (fromDb "acquiredlpbonussecretsecret.libsyn.com" // tech)
-    # The Intercept - Deconstructed
-    (fromDb "rss.acast.com/deconstructed")
-    # (fromDb "rss.prod.firstlook.media/deconstructed/podcast.rss" // pol)  #< possible URL rot
-    ## The Daily
-    (mkPod "https://feeds.simplecast.com/54nAGcIl" // pol // daily)
-    # The Intercept - Intercepted
-    (fromDb "rss.acast.com/intercepted-with-jeremy-scahill")
-    # (fromDb "rss.prod.firstlook.media/intercepted/podcast.rss" // pol)  #< possible URL rot
-    (fromDb "podcast.posttv.com/itunes/post-reports.xml" // pol)
-    ## Eric Weinstein
-    (fromDb "rss.art19.com/the-portal" // rat)
-    (fromDb "darknetdiaries.com" // tech)
-    ## Radiolab -- also available here, but ONLY OVER HTTP: <http://feeds.wnyc.org/radiolab>
-    (fromDb "feeds.feedburner.com/radiolab" // pol)
-    ## Sam Harris
-    (fromDb "wakingup.libsyn.com" // pol)
-    ## 99% Invisible -- also available here: <https://feeds.simplecast.com/BqbsxVfO>
-    (fromDb "feeds.99percentinvisible.org/99percentinvisible" // pol)
-    (fromDb "rss.acast.com/ft-tech-tonic" // tech)
-    (fromDb "feeds.feedburner.com/dancarlin/history" // rat)
-    (fromDb "rss.art19.com/60-minutes" // pol)
-    ## The Verge - Decoder
-    (fromDb "feeds.megaphone.fm/recodedecode" // tech)
-    ## Matrix (chat) Live
-    (fromDb "feed.podbean.com/matrixlive/feed.xml" // tech)
     (fromDb "cast.postmarketos.org" // tech)
+    (fromDb "congressionaldish.libsyn.com" // pol)  # Jennifer Briney
+    (fromDb "craphound.com" // pol)  # Cory Doctorow -- both podcast & text entries
+    (fromDb "darknetdiaries.com" // tech)
+    (fromDb "feed.podbean.com/matrixlive/feed.xml" // tech)  # Matrix (chat) Live
+    (fromDb "feeds.99percentinvisible.org/99percentinvisible" // pol)  # 99% Invisible -- also available here: <https://feeds.simplecast.com/BqbsxVfO>
+    (fromDb "feeds.feedburner.com/80000HoursPodcast" // rat)
+    (fromDb "feeds.feedburner.com/dancarlin/history" // rat)
+    (fromDb "feeds.feedburner.com/radiolab" // pol)  # Radiolab -- also available here, but ONLY OVER HTTP: <http://feeds.wnyc.org/radiolab>
+    (fromDb "feeds.libsyn.com/421877" // rat)  # Less Wrong Curated
+    (fromDb "feeds.megaphone.fm/behindthebastards" // pol)  # also Maggie Killjoy
+    (fromDb "feeds.megaphone.fm/hubermanlab" // uncat)  # Daniel Huberman on sleep
+    (fromDb "feeds.megaphone.fm/recodedecode" // tech)  # The Verge - Decoder
+    (fromDb "feeds.simplecast.com/54nAGcIl" // pol)  # The Daily
+    (fromDb "feeds.simplecast.com/82FI35Px" // pol)  # Ezra Klein Show
+    (fromDb "feeds.simplecast.com/wgl4xEgL" // rat)  # Econ Talk
+    (fromDb "feeds.simplecast.com/xKJ93w_w" // uncat)  # Atlas Obscura
+    (fromDb "feeds.simplecast.com/l2i9YnTd" // tech // pol)  # Hard Fork (NYtimes tech)
+    (fromDb "feeds.transistor.fm/acquired" // tech)
+    (fromDb "lexfridman.com/podcast" // rat)
+    (fromDb "mapspodcast.libsyn.com" // uncat)  # Multidisciplinary Association for Psychedelic Studies
+    (fromDb "omegataupodcast.net" // tech)  # 3/4 German; 1/4 eps are English
+    (fromDb "omny.fm/shows/cool-people-who-did-cool-stuff" // pol)  # Maggie Killjoy -- referenced by Cory Doctorow
+    (fromDb "podcast.posttv.com/itunes/post-reports.xml" // pol)
     (fromDb "podcast.thelinuxexp.com" // tech)
-    ## Michael Malice - Your Welcome -- also available here: <https://origin.podcastone.com/podcast?categoryID2=2232>
+    (fromDb "politicalorphanage.libsyn.com" // pol)
-    # (fromDb "rss.art19.com/your-welcome" // pol)
+    (fromDb "reverseengineering.libsyn.com/rss" // tech)  # UnNamed Reverse Engineering Podcast
+    (fromDb "rss.acast.com/deconstructed")  # The Intercept - Deconstructed
+    (fromDb "rss.acast.com/ft-tech-tonic" // tech)
+    (fromDb "rss.acast.com/intercepted-with-jeremy-scahill")  # The Intercept - Intercepted
+    (fromDb "rss.art19.com/60-minutes" // pol)
+    (fromDb "rss.art19.com/the-portal" // rat)  # Eric Weinstein
     (fromDb "seattlenice.buzzsprout.com" // pol)
-    ## Sci-Fi? has Peter Watts; author of No Moods, Ads or Cutesy Fucking Icons (rifters.com)
+    (fromDb "sharkbytes.transistor.fm" // tech)  # Wireshark Podcast o_0
-    (fromDb "talesfromthebridge.buzzsprout.com" // tech)
+    (fromDb "sscpodcast.libsyn.com" // rat)  # Astral Codex Ten
-    ## UnNamed Reverse Engineering Podcast
+    (fromDb "talesfromthebridge.buzzsprout.com" // tech)  # Sci-Fi? has Peter Watts; author of No Moods, Ads or Cutesy Fucking Icons (rifters.com)
-    (fromDb "reverseengineering.libsyn.com/rss" // tech)
+    (fromDb "techwontsave.us" // pol)  # rec by Cory Doctorow
-    ## The Witch Trials of J.K. Rowling
+    # (fromDb "trashfuturepodcast.podbean.com" // pol)  # rec by Cory Doctorow, but way rambly
-    ## - <https://www.thefp.com/witchtrials>
+    (fromDb "wakingup.libsyn.com" // pol)  # Sam Harris
-    (mkPod "https://feeds.megaphone.fm/RUNMED9919162779" // pol // infrequent)
+    (fromDb "werenotwrong.fireside.fm" // pol)
-    ## Atlas Obscura
+
-    (fromDb "feeds.simplecast.com/xKJ93w_w" // uncat)
+    # (fromDb "rss.art19.com/your-welcome" // pol)  # Michael Malice - Your Welcome -- also available here: <https://origin.podcastone.com/podcast?categoryID2=2232>
-    ## Ezra Klein Show
+    # (fromDb "rss.prod.firstlook.media/deconstructed/podcast.rss" // pol)  #< possible URL rot
-    (fromDb "feeds.simplecast.com/82FI35Px" // pol)
+    # (fromDb "rss.prod.firstlook.media/intercepted/podcast.rss" // pol)  #< possible URL rot
-    ## Wireshark Podcast o_0
+    # (mkPod "https://anchor.fm/s/21bc734/podcast/rss" // pol // infrequent)  # Emerge: making sense of what's next -- <https://www.whatisemerging.com/emergepodcast>
-    (fromDb "sharkbytes.transistor.fm" // tech)
+    # (mkPod "https://audioboom.com/channels/5097784.rss" // tech)  # Lateral with Tom Scott
-    ## 3/4 German; 1/4 eps are English
+    # (mkPod "https://feeds.megaphone.fm/RUNMED9919162779" // pol // infrequent)  # The Witch Trials of J.K. Rowling: <https://www.thefp.com/witchtrials>
-    (fromDb "omegataupodcast.net" // tech)
+    # (mkPod "https://podcasts.la.utexas.edu/this-is-democracy/feed/podcast/" // pol // weekly)
-    ## Lateral with Tom Scott
-    (mkPod "https://audioboom.com/channels/5097784.rss" // tech)
   ];
 
   texts = [
@@ -164,10 +137,12 @@ let
     # DEVELOPERS
     (fromDb "blog.jmp.chat" // tech)
     (fromDb "uninsane.org" // tech)
+    (fromDb "blog.thalheim.io" // tech)  # Mic92
     (fromDb "ascii.textfiles.com" // tech)  # Jason Scott
     (fromDb "xn--gckvb8fzb.com" // tech)
     (fromDb "amosbbatto.wordpress.com" // tech)
     (fromDb "fasterthanli.me" // tech)
+    (fromDb "jeffgeerling.com" // tech)
     (fromDb "mg.lol" // tech)
     # (fromDb "drewdevault.com" // tech)
     ## Ken Shirriff
@@ -178,6 +153,7 @@ let
     (fromDb "vitalik.ca" // tech)
     ## ian (Sanctuary)
     (fromDb "sagacioussuricata.com" // tech)
+    (fromDb "artemis.sh" // tech)
     ## Bunnie Juang
     (fromDb "bunniestudios.com" // tech)
     (fromDb "blog.danieljanus.pl" // tech)
@@ -188,6 +164,8 @@ let
     (mkText "https://anish.lakhwara.com/home.html" // tech // weekly)
     (fromDb "jefftk.com" // tech)
     (fromDb "pomeroyb.com" // tech)
+    (fromDb "harihareswara.net" // tech // pol)  # rec by Cory Doctorow
+    (fromDb "mako.cc/copyrighteous" // tech // pol)  # rec by Cory Doctorow
     # (mkText "https://til.simonwillison.net/tils/feed.atom" // tech // weekly)
 
     # TECH PROJECTS

hosts/common/hardware/default.nix

@@ -9,8 +9,20 @@
   # useful emergency utils
   boot.initrd.extraUtilsCommands = ''
     copy_bin_and_libs ${pkgs.btrfs-progs}/bin/btrfstune
+    copy_bin_and_libs ${pkgs.util-linux}/bin/{cfdisk,lsblk,lscpu}
+    copy_bin_and_libs ${pkgs.gptfdisk}/bin/{cgdisk,gdisk}
+    copy_bin_and_libs ${pkgs.smartmontools}/bin/smartctl
+    copy_bin_and_libs ${pkgs.e2fsprogs}/bin/resize2fs
+  '' + lib.optionalString pkgs.stdenv.hostPlatform.isx86_64 ''
+    copy_bin_and_libs ${pkgs.nvme-cli}/bin/nvme  # doesn't cross compile
   '';
-  boot.kernelParams = [ "boot.shell_on_fail" ];
+  boot.kernelParams = [
+    "boot.shell_on_fail"
+    #v experimental full pre-emption for hopefully better call/audio latency on moby.
+    # also toggleable at runtime via /sys/kernel/debug/sched/preempt
+    # defaults to preempt=voluntary
+    # "preempt=full"
+  ];
   # other kernelParams:
   #   "boot.trace"
   #   "systemd.log_level=debug"

hosts/common/home/keyring/default.nix

@@ -7,7 +7,7 @@ let
   };
 in
 {
-  sane.user.persist.private = [ ".local/share/keyrings" ];
+  sane.user.persist.byStore.private = [ ".local/share/keyrings" ];
 
   sane.user.fs."private/.local/share/keyrings/default" = {
     generated.command = [ "${init-keyring}/bin/init-keyring" ];

hosts/common/home/ssh.nix

@@ -13,7 +13,7 @@ let
 in
 {
   # ssh key is stored in private storage
-  sane.user.persist.private = [
+  sane.user.persist.byStore.private = [
     { type = "file"; path = ".ssh/id_ed25519"; }
   ];
   sane.user.fs.".ssh/id_ed25519.pub" = lib.mkIf (user-pubkey != null) {

hosts/common/hostnames.nix

@@ -0,0 +1,16 @@
+# TODO: move to hosts/common/
+{ config, lib, ... }:
+
+{
+  # give each host a shortname that all the other hosts know, to allow easy comms.
+  networking.hosts = lib.mkMerge (builtins.map
+    (host: let
+      cfg = config.sane.hosts.by-name."${host}";
+    in {
+      "${cfg.lan-ip}" = [ host ];
+    } // lib.optionalAttrs (cfg.wg-home.ip != null) {
+      "${cfg.wg-home.ip}" = [ "${host}-hn" ];
+    })
+    (builtins.attrNames config.sane.hosts.by-name)
+  );
+}

hosts/common/hosts.nix

@@ -36,4 +36,10 @@
     wg-home.endpoint = "uninsane.org:51820";
     lan-ip = "10.78.79.51";
   };
+
+  sane.hosts.by-name."supercap" = {
+    ssh.authorized = false;
+    ssh.host_pubkey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHf/mqqkX45EWAcquV04MC3SUljTApdclH1gjI19F+PA";
+    lan-ip = "10.78.79.232";
+  };
 }

hosts/common/ids.nix

@@ -49,6 +49,10 @@
   sane.ids.media.gid = 2414;
   sane.ids.ntfy-sh.uid = 2415;
   sane.ids.ntfy-sh.gid = 2415;
+  sane.ids.monero.uid = 2416;
+  sane.ids.monero.gid = 2416;
+  sane.ids.slskd.uid = 2417;
+  sane.ids.slskd.gid = 2417;
 
   sane.ids.colin.uid = 1000;
   sane.ids.guest.uid = 1100;
@@ -63,6 +67,8 @@
   sane.ids.systemd-oom.uid = 2005;
   sane.ids.systemd-oom.gid = 2005;
   sane.ids.wireshark.gid = 2006;
+  sane.ids.nixremote.uid = 2007;
+  sane.ids.nixremote.gid = 2007;
 
   # found on graphical hosts
   sane.ids.nm-iodine.uid = 2101;  # desko/moby/lappy

hosts/common/nix-path/default.nix

@@ -1,4 +1,4 @@
-{ pkgs, ... }:
+{ pkgs, sane-lib, ... }:
 
 {
   # allow `nix-shell` (and probably nix-index?) to locate our patched and custom packages
@@ -10,4 +10,7 @@
     # to avoid switching so much during development
     "nixpkgs-overlays=/home/colin/dev/nixos/hosts/common/nix-path/overlay"
   ];
+
+  # ensure new deployments have a source of this repo with which they can bootstrap.
+  environment.etc."nixos".source = ../../..;
 }

hosts/common/persist.nix

@@ -5,12 +5,12 @@
   # store /home/colin/a/b in /home/private/a/b instead of /home/private/home/colin/a/b
   sane.persist.stores.private.prefix = "/home/colin";
 
-  sane.persist.sys.plaintext = [
+  sane.persist.sys.byStore.plaintext = [
     # TODO: these should be private.. somehow
     "/var/log"
     "/var/backup"  # for e.g. postgres dumps
   ];
-  sane.persist.sys.cryptClearOnBoot = [
+  sane.persist.sys.byStore.cryptClearOnBoot = [
     "/var/lib/systemd/coredump"
   ];
 }

hosts/common/programs/abaddon.nix

@@ -0,0 +1,107 @@
+# discord gtk3 client
+{ config, lib, pkgs, ... }:
+let
+  cfg = config.sane.programs.abaddon;
+in
+{
+  sane.programs.abaddon = {
+    configOption = with lib; mkOption {
+      default = {};
+      type = types.submodule {
+        options.autostart = mkOption {
+          type = types.bool;
+          default = true;
+        };
+      };
+    };
+
+    package = pkgs.abaddon.overrideAttrs (upstream: {
+      patches = (upstream.patches or []) ++ [
+        (pkgs.fetchpatch {
+          url = "https://git.uninsane.org/colin/abaddon/commit/eb551f188d34679f75adcbc83cb8d5beb4d19fd6.patch";
+          name = ''"view members" default to false'';
+          hash = "sha256-9BX8iO86CU1lNrKS1G2BjDR+3IlV9bmhRNTsLrxChwQ=";
+        })
+        (pkgs.fetchpatch {
+          # this makes it so Abaddon reports its app_name in notifications.
+          # not 100% necessary; just a nice-to-have. maybe don't rely on it until it's merged upstream.
+          # upstream PR: <https://github.com/uowuo/abaddon/pull/247>
+          url = "https://git.uninsane.org/colin/abaddon/commit/18cd863fdbb5e6b1e9aaf9394dbd673d51839f30.patch";
+          name = "set glib application name";
+          hash = "sha256-IFYxf1D8hIsxgZehGd6hL3zJiBkPZfWGm+Faaa5ZFl4=";
+        })
+      ];
+    });
+
+    suggestedPrograms = [ "gnome-keyring" ];
+
+    fs.".config/abaddon/abaddon.ini".symlink.text = ''
+      # see abaddon README.md for options.
+      # at time of writing:
+      # | Setting       | Type    | Default | Description                                                                                      |
+      # |[discord]------|---------|---------|--------------------------------------------------------------------------------------------------|
+      # | `gateway`     | string  |         | override url for Discord gateway. must be json format and use zlib stream compression            |
+      # | `api_base`    | string  |         | override base url for Discord API                                                                |
+      # | `memory_db`   | boolean | false   | if true, Discord data will be kept in memory as opposed to on disk                               |
+      # | `token`       | string  |         | Discord token used to login, this can be set from the menu                                       |
+      # | `prefetch`    | boolean | false   | if true, new messages will cause the avatar and image attachments to be automatically downloaded |
+      # | `autoconnect` | boolean | false   | autoconnect to discord                                                                           |
+      # |[http]--------|--------|---------|---------------------------------------------------------------------------------------------|
+      # | `user_agent` | string |         | sets the user-agent to use in HTTP requests to the Discord API (not including media/images) |
+      # | `concurrent` | int    | 20      | how many images can be concurrently retrieved                                               |
+      # |[gui}------------------------|---------|---------|----------------------------------------------------------------------------------------------------------------------------|
+      # | `member_list_discriminator` | boolean | true    | show user discriminators in the member list                                                                                |
+      # | `stock_emojis`              | boolean | true    | allow abaddon to substitute unicode emojis with images from emojis.bin, must be false to allow GTK to render emojis itself |
+      # | `custom_emojis`             | boolean | true    | download and use custom Discord emojis                                                                                     |
+      # | `css`                       | string  |         | path to the main CSS file                                                                                                  |
+      # | `animations`                | boolean | true    | use animated images where available (e.g. server icons, emojis, avatars). false means static images will be used           |
+      # | `animated_guild_hover_only` | boolean | true    | only animate guild icons when the guild is being hovered over                                                              |
+      # | `owner_crown`               | boolean | true    | show a crown next to the owner                                                                                             |
+      # | `unreads`                   | boolean | true    | show unread indicators and mention badges                                                                                  |
+      # | `save_state`                | boolean | true    | save the state of the gui (active channels, tabs, expanded channels)                                                       |
+      # | `alt_menu`                  | boolean | false   | keep the menu hidden unless revealed with alt key                                                                          |
+      # | `hide_to_tray`              | boolean | false   | hide abaddon to the system tray on window close                                                                            |
+      # | `show_deleted_indicator`    | boolean | true    | show \[deleted\] indicator next to deleted messages instead of actually deleting the message                               |
+      # | `font_scale`                | double  |         | scale font rendering. 1 is unchanged                                                                                       |
+      # |[style]------------------|--------|-----------------------------------------------------|
+      # | `linkcolor`             | string | color to use for links in messages                  |
+      # | `expandercolor`         | string | color to use for the expander in the channel list   |
+      # | `nsfwchannelcolor`      | string | color to use for NSFW channels in the channel list  |
+      # | `channelcolor`          | string | color to use for SFW channels in the channel list   |
+      # | `mentionbadgecolor`     | string | background color for mention badges                 |
+      # | `mentionbadgetextcolor` | string | color to use for number displayed on mention badges |
+      # | `unreadcolor`           | string | color to use for the unread indicator               |
+      # |[notifications]|---------|--------------------------|-------------------------------------------------------------------------------|
+      # | `enabled`     | boolean | true (if not on Windows) | Enable desktop notifications                                                  |
+      # | `playsound`   | boolean | true                     | Enable notification sounds. Requires ENABLE_NOTIFICATION_SOUNDS=TRUE in CMake |
+      # |[voice]--|--------|------------------------------------|------------------------------------------------------------|
+      # | `vad`   | string | rnnoise if enabled, gate otherwise | Method used for voice activity detection. Changeable in UI |
+      # |[windows]|---------|---------|-------------------------|
+      # | `hideconsole` | boolean | true    | Hide console on startup |
+
+      # N.B.: abaddon writes this file itself (and even when i don't change anything internally).
+      # it prefers no spaces around the equal sign.
+      [discord]
+      autoconnect=true
+
+      [notifications]
+      # playsound: i manage sounds via swaync
+      playsound=false
+    '';
+
+    persist.byStore.private = [
+      ".cache/abaddon"
+    ];
+
+    services.abaddon = {
+      description = "unofficial Discord chat client";
+      wantedBy = lib.mkIf cfg.config.autostart [ "default.target" ];
+      serviceConfig = {
+        ExecStart = "${cfg.package}/bin/abaddon";
+        Type = "simple";
+        Restart = "always";
+        RestartSec = "20s";
+      };
+    };
+  };
+}

hosts/common/programs/animatch.nix

@@ -0,0 +1,10 @@
+{ ... }:
+{
+  sane.programs.animatch = {
+    persist.byStore.plaintext = [
+      # game progress
+      ".config/Holy Pangolin/Animatch"
+      ".local/share/Holy Pangolin/Animatch"  # i think this one might be wrong
+    ];
+  };
+}

hosts/common/programs/assorted.nix

@@ -20,6 +20,7 @@ in
       "sane-scripts.bt-show"
     ];
     "sane-scripts.dev" = declPackageSet [
+      "sane-scripts.clone"
       "sane-scripts.dev-cargo-loop"
       "sane-scripts.git-init"
     ];
@@ -41,12 +42,13 @@ in
       "sane-scripts.secrets-unlock"
       "sane-scripts.secrets-update-keys"
       "sane-scripts.shutdown"
-      "sane-scripts.ssl-dump"
       "sane-scripts.sudo-redirect"
       "sane-scripts.sync-from-servo"
       "sane-scripts.vpn"
       "sane-scripts.which"
       "sane-scripts.wipe-browser"
+      "sane-scripts.wipe-flare"
+      "sane-scripts.wipe-fractal"
     ];
     "sane-scripts.sys-utils" = declPackageSet [
       "sane-scripts.ip-port-forward"
@@ -58,8 +60,10 @@ in
       "btrfs-progs"
       "cacert.unbundled"  # some services require unbundled /etc/ssl/certs
       "cryptsetup"
+      "ddrescue"
       "dig"
       "dtc"  # device tree [de]compiler
+      "e2fsprogs"  # resize2fs
       "efibootmgr"
       "ethtool"
       "fatresize"
@@ -67,8 +71,9 @@ in
       "file"
       # "fwupd"
       "gawk"
+      "gdb"  # to debug segfaults
       "git"
-      "gptfdisk"
+      "gptfdisk"  # gdisk
       "hdparm"
       "htop"
       "iftop"
@@ -86,6 +91,7 @@ in
       "netcat"
       "nethogs"
       "nmap"
+      "nvme-cli"  # nvme
       "openssl"
       "parted"
       "pciutils"
@@ -93,15 +99,17 @@ in
       "pstree"
       "ripgrep"
       "screen"
-      "smartmontools"
+      "smartmontools"  # smartctl
       "socat"
       "strace"
       "subversion"
       "tcpdump"
       "tree"
       "usbutils"
+      "util-linux"  # lsblk, lscpu, etc
       "wget"
       "wirelesstools"  # iwlist
+      "xq"  # jq for XML
     ];
     sysadminExtraUtils = declPackageSet [
       "backblaze-b2"
@@ -117,12 +125,13 @@ in
     #   - debugging?
     consoleUtils = declPackageSet [
       "alsaUtils"  # for aplay, speaker-test
-      "binutils"  # for strings; though this brings 80MB of unrelated baggage too
+      "binutils-unwrapped"  # for strings; though this brings 80MB of unrelated baggage too
       # "cdrtools"
       "clinfo"
       "dmidecode"
       "dtrx"  # `unar` alternative, "Do The Right eXtraction"
       "efivar"
+      "eza"  # a better 'ls'
       # "flashrom"
       "git"  # needed as a user package, for config.
       # "gnupg"
@@ -134,6 +143,7 @@ in
       "lm_sensors"  # for sensors-detect. TODO: what needs this? lift into the consumer
       "lshw"
       # "memtester"
+      "mercurial"  # hg
       "neovim"  # needed as a user package, for swap persistence
       # "nettools"
       # "networkmanager"
@@ -143,7 +153,7 @@ in
       # "oathToolkit"  # for oathtool
       # "ponymix"
       "pulsemixer"
-      "python3"
+      "python3-repl"
       # "python3Packages.eyeD3"  # music tagging
       "ripgrep"  # needed as a user package so that its user-level config file can be installed
       "rsync"
@@ -158,13 +168,12 @@ in
       # "unar"
       "unzip"
       "wireguard-tools"
-      "xdg-terminal-exec"
       "xdg-utils"  # for xdg-open
       # "yarn"
       "zsh"
     ];
 
-    desktopConsoleUtils = declPackageSet [
+    pcConsoleUtils = declPackageSet [
       "gh"  # MS GitHub cli
       "nix-index"
       "nixpkgs-review"
@@ -179,11 +188,11 @@ in
       "yt-dlp"
     ];
 
-    tuiApps = declPackageSet [
+    pcTuiApps = declPackageSet [
       "aerc"  # email client
       "msmtp"  # sendmail
       "offlineimap"  # email mailbox sync
-      "sfeed"  # RSS fetcher
+      # "sfeed"  # RSS fetcher
       "visidata"  # TUI spreadsheet viewer/editor
       "w3m"  # web browser
     ];
@@ -198,7 +207,9 @@ in
     devPkgs = declPackageSet [
       "cargo"
       "clang"
+      "lua"
       "nodejs"
+      "patchelf"
       "rustc"
       "tree-sitter"
     ];
@@ -206,17 +217,19 @@ in
 
     # INDIVIDUAL PACKAGE DEFINITIONS
 
-    cargo.persist.plaintext = [ ".cargo" ];
+    cargo.persist.byStore.plaintext = [ ".cargo" ];
 
     # creds, but also 200 MB of node modules, etc
-    discord.persist.private = [ ".config/discord" ];
+    discord.persist.byStore.private = [ ".config/discord" ];
+
+    endless-sky.persist.byStore.plaintext = [ ".local/share/endless-sky" ];
 
     # `emote` will show a first-run dialog based on what's in this directory.
     # mostly, it just keeps a LRU of previously-used emotes to optimize display order.
     # TODO: package [smile](https://github.com/mijorus/smile) for probably a better mobile experience.
-    emote.persist.plaintext = [ ".local/share/Emote" ];
+    emote.persist.byStore.plaintext = [ ".local/share/Emote" ];
 
-    fluffychat-moby.persist.plaintext = [ ".local/share/chat.fluffy.fluffychat" ];
+    fluffychat-moby.persist.byStore.plaintext = [ ".local/share/chat.fluffy.fluffychat" ];
 
     font-manager.package = pkgs.font-manager.override {
       # build without the "Google Fonts" integration feature, to save closure / avoid webkitgtk_4_0
@@ -225,45 +238,47 @@ in
 
     # MS GitHub stores auth token in .config
     # TODO: we can populate gh's stuff statically; it even lets us use the same oauth across machines
-    gh.persist.private = [ ".config/gh" ];
+    gh.persist.byStore.private = [ ".config/gh" ];
 
-    "gnome.gnome-maps".persist.plaintext = [ ".cache/shumate" ];
+    gnome-2048.persist.byStore.plaintext = [ ".local/share/gnome-2048/scores" ];
-    "gnome.gnome-maps".persist.private = [ ".local/share/maps-places.json" ];
+
+    "gnome.gnome-maps".persist.byStore.plaintext = [ ".cache/shumate" ];
+    "gnome.gnome-maps".persist.byStore.private = [ ".local/share/maps-places.json" ];
 
     # actual monero blockchain (not wallet/etc; safe to delete, just slow to regenerate)
     # XXX: is it really safe to persist this? it doesn't have info that could de-anonymize if captured?
-    monero-gui.persist.plaintext = [ ".bitmonero" ];
+    monero-gui.persist.byStore.plaintext = [ ".bitmonero" ];
 
-    mumble.persist.private = [ ".local/share/Mumble" ];
+    mumble.persist.byStore.private = [ ".local/share/Mumble" ];
 
     # settings (electron app)
-    obsidian.persist.plaintext = [ ".config/obsidian" ];
+    obsidian.persist.byStore.plaintext = [ ".config/obsidian" ];
 
-    # creds, media
+    python3-repl.package = pkgs.python3.withPackages (ps: with ps; [
-    signal-desktop.persist.private = [ ".config/Signal" ];
+      requests
+    ]);
+
+    shattered-pixel-dungeon.persist.byStore.plaintext = [ ".local/share/.shatteredpixel/shattered-pixel-dungeon" ];
 
     # printer/filament settings
-    slic3r.persist.plaintext = [ ".Slic3r" ];
+    slic3r.persist.byStore.plaintext = [ ".Slic3r" ];
 
-    # creds, widevine .so download. TODO: could easily manage these statically.
+    space-cadet-pinball.persist.byStore.plaintext = [ ".local/share/SpaceCadetPinball" ];
-    spotify.persist.plaintext = [ ".config/spotify" ];
 
-    tdesktop.persist.private = [ ".local/share/TelegramDesktop" ];
+    superTux.persist.byStore.plaintext = [ ".local/share/supertux2" ];
 
-    tokodon.persist.private = [ ".cache/KDE/tokodon" ];
+    tdesktop.persist.byStore.private = [ ".local/share/TelegramDesktop" ];
 
-    # hardenedMalloc solves an "unable to connect to Tor" error when pressing the "connect" button
+    tokodon.persist.byStore.private = [ ".cache/KDE/tokodon" ];
-    # - still required as of 2023/07/14
-    tor-browser-bundle-bin.package = pkgs.tor-browser-bundle-bin.override {
-      useHardenedMalloc = false;
-    };
 
-    whalebird.persist.private = [ ".config/Whalebird" ];
+    vvvvvv.persist.byStore.plaintext = [ ".local/share/VVVVVV" ];
 
-    yarn.persist.plaintext = [ ".cache/yarn" ];
+    whalebird.persist.byStore.private = [ ".config/Whalebird" ];
+
+    yarn.persist.byStore.plaintext = [ ".cache/yarn" ];
 
     # zcash coins. safe to delete, just slow to regenerate (10-60 minutes)
-    zecwallet-lite.persist.private = [ ".zcash" ];
+    zecwallet-lite.persist.byStore.private = [ ".zcash" ];
   };
 
   programs.feedbackd = lib.mkIf config.sane.programs.feedbackd.enabled {

hosts/common/programs/bemenu.nix

@@ -0,0 +1,103 @@
+{ lib, pkgs, ... }:
+# notable bemenu options:
+#   - see `bemenu --help` for all
+#   -P, --prefix          text to show before highlighted item.
+#   --scrollbar           display scrollbar. (none (default), always, autohide)
+#   -H, --line-height     defines the height to make each menu line (0 = default height). (wx)
+#   -M, --margin          defines the empty space on either side of the menu. (wx)
+#   -W, --width-factor    defines the relative width factor of the menu (from 0 to 1). (wx)
+#   -B, --border          defines the width of the border in pixels around the menu. (wx)
+#   -R  --border-radius   defines the radius of the border around the menu (0 = no curved borders).
+#   --ch                  defines the height of the cursor (0 = scales with line height). (wx)
+#   --cw                  defines the width of the cursor. (wx)
+#   --hp                  defines the horizontal padding for the entries in single line mode. (wx)
+#   --fn                  defines the font to be used ('name [size]'). (wx)
+#   --tb                  defines the title background color. (wx)
+#   --tf                  defines the title foreground color. (wx)
+#   --fb                  defines the filter background color. (wx)
+#   --ff                  defines the filter foreground color. (wx)
+#   --nb                  defines the normal background color. (wx)
+#   --nf                  defines the normal foreground color. (wx)
+#   --hb                  defines the highlighted background color. (wx)
+#   --hf                  defines the highlighted foreground color. (wx)
+#   --fbb                 defines the feedback background color. (wx)
+#   --fbf                 defines the feedback foreground color. (wx)
+#   --sb                  defines the selected background color. (wx)
+#   --sf                  defines the selected foreground color. (wx)
+#   --ab                  defines the alternating background color. (wx)
+#   --af                  defines the alternating foreground color. (wx)
+#   --scb                 defines the scrollbar background color. (wx)
+#   --scf                 defines the scrollbar foreground color. (wx)
+#   --bdr                 defines the border color. (wx)
+#
+# colors are specified as `#RRGGBB`
+# defaults:
+# --ab "#222222"
+# --af "#bbbbbb"
+# --bdr "#005577"
+# --border 3
+# --cb "#222222"
+# --center
+# --cf "#bbbbbb"
+# --fb "#222222"
+# --fbb "#eeeeee"
+# --fbf "#222222"
+# --ff "#bbbbbb"
+# --fixed-height
+# --fn 'Sxmo 14'
+# --hb "#005577"
+# --hf "#eeeeee"
+# --line-height 20
+# --list 16
+# --margin 40
+# --nb "#222222"
+# --nf "#bbbbbb"
+# --no-overlap
+# --no-spacing
+# --sb "#323232"
+# --scb "#005577"
+# --scf "#eeeeee"
+# --scrollbar autohide
+# --tb "#005577"
+# --tf "#eeeeee"
+# --wrap
+let
+  bg = "#1d1721";       # slight purple
+  fg0 = "#d8d8d8";      # inactive text (light grey)
+  fg1 = "#ffffff";      # active text   (white)
+  accent0 = "#1f5e54";  # darker but saturated teal
+  accent1 = "#418379";  # teal (matches nixos-bg)
+  accent2 = "#5b938a";  # brighter but muted teal
+  bemenuArgs = [
+    "--wrap --scrollbar autohide --fixed-height"
+    "--center --margin 45"
+    "--no-spacing"
+    # XXX: font size doesn't seem to take effect (would prefer larger)
+    "--fn 'monospace 14' --line-height 22 --border 3"
+    "--bdr '${accent0}'"                     # border
+    "--scf '${accent2}' --scb '${accent0}'"  # scrollbar
+    "--tb '${accent0}' --tf '${fg0}'"        # title
+    "--fb '${accent0}' --ff '${fg1}'"        # filter (i.e. text that's been entered)
+    "--hb '${accent1}' --hf '${fg1}'"        # selected item
+    "--nb '${bg}' --nf '${fg0}'"             # normal lines (even)
+    "--ab '${bg}' --af '${fg0}'"             # alternated lines (odd)
+    "--cf '${accent0}' --cb '${accent0}'"    # cursor (not very useful)
+  ];
+  bemenuOpts = lib.concatStringsSep " " bemenuArgs;
+in
+{
+  sane.programs.bemenu = {
+    package = pkgs.bemenu.overrideAttrs (upstream: {
+      nativeBuildInputs = (upstream.nativeBuildInputs or []) ++ [
+        pkgs.makeWrapper
+      ];
+      # can alternatively be specified as CLI flags
+      postInstall = (upstream.postInstall or "") + ''
+        wrapProgram $out/bin/bemenu \
+          --set BEMENU_OPTS "${bemenuOpts}"
+        wrapProgram $out/bin/bemenu-run \
+          --set BEMENU_OPTS "${bemenuOpts}"
+      '';
+    });
+  };
+}

hosts/common/programs/brave.nix

@@ -0,0 +1,9 @@
+{ ... }:
+{
+  sane.programs.brave = {
+    persist.byStore.cryptClearOnBoot = [
+      ".cache/BraveSoftware"
+      ".config/BraveSoftware"
+    ];
+  };
+}

hosts/common/programs/calls.nix

@@ -30,7 +30,7 @@ in
       };
     };
 
-    persist.private = [
+    persist.byStore.private = [
       # ".cache/folks"      # contact avatars?
       # ".config/calls"
       ".local/share/calls"  # call "records"

hosts/common/programs/cantata.nix

@@ -4,7 +4,7 @@
 { ... }:
 {
   sane.programs.cantata = {
-    persist.plaintext = [
+    persist.byStore.plaintext = [
       ".cache/cantata"  # album art
       ".local/share/cantata/library"  # library index (?)
     ];

hosts/common/programs/chatty.nix

@@ -36,7 +36,7 @@ in
     # package = chattyNoOauth;
     package = chatty-latest;
     suggestedPrograms = [ "gnome-keyring" ];
-    persist.private = [
+    persist.byStore.private = [
       ".local/share/chatty"  # matrix avatars and files
       # not just XMPP; without this Chatty will regenerate its device-id every boot.
       # .purple/ contains XMPP *and* Matrix auth, logs, avatar cache, and a bit more

hosts/common/programs/conky/battery_estimate

@@ -1,14 +1,34 @@
 #!/usr/bin/env nix-shell
 #!nix-shell -i bash
 
-full=$(cat /sys/class/power_supply/axp20x-battery/charge_full_design)
-rate=$(cat /sys/class/power_supply/axp20x-battery/current_now)
-perc=$(cat /sys/class/power_supply/axp20x-battery/capacity)
-perc_left=$((100 - $perc))
 # these icons come from sxmo; they only render in nerdfonts
 bat_dis="󱊢"
 bat_chg="󱊥"
 
+try_path() {
+  # returns:
+  # - perc, perc_left  (0-100)
+  # - full, rate (pos means charging)
+  if [ -f "$1/capacity" ]; then
+    perc=$(cat "$1/capacity")
+    perc_left=$((100 - $perc))
+  fi
+
+  if [ -f "$1/charge_full_design" ] && [ -f "$1/current_now" ]; then
+    # current is positive when charging
+    full=$(cat "$1/charge_full_design")
+    rate=$(cat "$1/current_now")
+  fi
+  if [ -f "$1/energy_full" ] && [ -f "$1/energy_now" ]; then
+    # energy is positive when discharging
+    full=$(cat "$1/energy_full")
+    rate=-$(cat "$1/energy_now")
+  fi
+}
+
+try_path "/sys/class/power_supply/axp20x-battery"  # Pinephone
+try_path "/sys/class/power_supply/BAT0"  # Thinkpad
+
 fmt_minutes() {
   # args: <battery symbol> <text if ludicrous estimate> <estimated minutes to full/empty>
   if [[ $3 -gt 1440 ]]; then
@@ -27,6 +47,6 @@ if [[ $rate -lt 0 ]]; then
 elif [[ $rate -gt 0 ]]; then
   # charging
   fmt_minutes "$bat_chg" '100%' "$(($full * 60 * $perc_left / (100 * $rate)))"
-else
+elif [[ "$perc" != "" ]]; then
   echo "$bat_dis $perc%"
 fi

hosts/common/programs/cozy.nix

@@ -3,7 +3,7 @@
 {
   sane.programs.cozy = {
     # cozy uses a sqlite db for its config and exposes no CLI options other than --help and --debug
-    persist.plaintext = [
+    persist.byStore.plaintext = [
       ".local/share/cozy"  # sqlite db (config & index?)
       ".cache/cozy"  # offline cache
     ];

hosts/common/programs/default.nix

@@ -2,31 +2,39 @@
 
 {
   imports = [
+    ./abaddon.nix
     ./aerc.nix
     ./alacritty.nix
+    ./animatch.nix
     ./assorted.nix
+    ./bemenu.nix
+    ./brave.nix
     ./calls.nix
     ./cantata.nix
     ./chatty.nix
     ./conky
     ./cozy.nix
+    ./dialect.nix
     ./dino.nix
     ./element-desktop.nix
     ./epiphany.nix
     ./evince.nix
     ./feedbackd.nix
     ./firefox.nix
+    ./flare-signal.nix
     ./fontconfig.nix
     ./fractal.nix
     ./fwupd.nix
     ./g4music.nix
     ./gajim.nix
+    ./geary.nix
     ./git.nix
     ./gnome-feeds.nix
     ./gnome-keyring.nix
     ./gnome-weather.nix
     ./gpodder.nix
     ./gthumb.nix
+    ./gtkcord4.nix
     ./helix.nix
     ./imagemagick.nix
     ./jellyfin-media-player.nix
@@ -51,14 +59,22 @@
     ./rhythmbox.nix
     ./ripgrep.nix
     ./sfeed.nix
+    ./signal-desktop.nix
     ./splatmoji.nix
+    ./spot.nix
+    ./spotify.nix
     ./steam.nix
     ./stepmania.nix
     ./sublime-music.nix
+    ./supertuxkart.nix
+    ./sway-autoscaler
     ./swaynotificationcenter.nix
     ./tangram.nix
+    ./tor-browser-bundle-bin.nix
     ./tuba.nix
     ./vlc.nix
+    ./wike.nix
+    ./wine.nix
     ./wireshark.nix
     ./xarchiver.nix
     ./zeal.nix

hosts/common/programs/dialect.nix

@@ -0,0 +1,13 @@
+{ pkgs, ... }:
+{
+  sane.programs.dialect = {
+    package = pkgs.dialect.overrideAttrs (upstream: {
+      # TODO: send upstream
+      # TODO: figure out how to get audio working
+      # TODO: move to runtimeDependencies?
+      buildInputs = upstream.buildInputs ++ [
+        pkgs.glib-networking  # for TLS
+      ];
+    });
+  };
+}

hosts/common/programs/dino.nix

@@ -18,6 +18,8 @@
 #   - fix is to toggle it off -> on in the Dino UI
 # - default mic gain is WAY TOO MUCH (heavily distorted)
 # - TODO: dino should have more optimal niceness/priority to ensure it can process its buffers
+#   - possibly this is solved by enabling RealtimeKit (rtkit)
+# - TODO: see if Dino calls work better with `echo full > /sys/kernel/debug/sched/preempt`
 #
 # probably fixed:
 # - once per 1-2 minutes dino will temporarily drop mic input:
@@ -43,10 +45,10 @@ in
       };
     };
 
-    persist.private = [ ".local/share/dino" ];
+    persist.byStore.private = [ ".local/share/dino" ];
 
     services.dino = {
-      description = "auto-start and maintain dino XMPP connection";
+      description = "dino XMPP client";
       wantedBy = lib.mkIf cfg.config.autostart [ "default.target" ];
       serviceConfig = {
         ExecStart = "${cfg.package}/bin/dino";

hosts/common/programs/element-desktop.nix

@@ -4,11 +4,16 @@
 #     - <https://github.com/vector-im/element-desktop/issues/1029#issuecomment-1632688224>
 #   - `rm -rf ~/.config/Element/GPUCache`
 #     - <https://github.com/NixOS/nixpkgs/issues/244486>
-{ ... }:
+{ pkgs, ... }:
 {
   sane.programs.element-desktop = {
+    package = pkgs.element-desktop.override {
+      # use pre-build electron because otherwise it takes 4 hrs to build from source.
+      electron = pkgs.electron-bin;
+    };
+
     # creds/session keys, etc
-    persist.private = [ ".config/Element" ];
+    persist.byStore.private = [ ".config/Element" ];
 
     suggestedPrograms = [ "gnome-keyring" ];
   };

hosts/common/programs/epiphany.nix

@@ -8,7 +8,7 @@
 { pkgs, ... }:
 {
   sane.programs.epiphany = {
-    # XXX(2023/07/08): running on moby without this hack fails, with:
+    # XXX(2023/07/08): running on moby without `WEBKIT_DISABLE_SANDBOX...` fails, with:
     # - `bwrap: Can't make symlink at /var/run: File exists`
     # this could be due to:
     # - epiphany is somewhere following a symlink into /var/run instead of /run
@@ -19,6 +19,9 @@
     # - <https://gitlab.gnome.org/GNOME/gnome-builder/-/issues/1164>
     # - <https://github.com/flatpak/flatpak/issues/3477>
     # - <https://github.com/NixOS/nixpkgs/issues/197085>
+    #
+    # TODO: consider `WEBKIT_USE_SINGLE_WEB_PROCESS=1` for better perf
+    # - this runs all tabs in 1 process. which is fine, if i'm not a heavy multi-tabber
     package = pkgs.epiphany.overrideAttrs (upstream: {
       preFixup = ''
         gappsWrapperArgs+=(
@@ -26,7 +29,7 @@
         );
       '' + (upstream.preFixup or "");
     });
-    persist.private = [
+    persist.byStore.private = [
       ".cache/epiphany"
       ".local/share/epiphany"
       # also .config/epiphany, but appears empty

hosts/common/programs/firefox.nix

@@ -13,7 +13,14 @@ let
   mobile-prefs = lib.optionals false pkgs.librewolf-pmos-mobile.extraPrefsFiles;
   # allow easy switching between firefox and librewolf with `defaultSettings`, below
   librewolfSettings = {
-    browser = pkgs.librewolf-unwrapped;
+    browser = pkgs.librewolf-unwrapped.overrideAttrs (upstream: {
+      # TEMP(2023/11/21): fix eval bug in wrapFirefox
+      # see: <https://github.com/NixOS/nixpkgs/pull/244591>
+      passthru = upstream.passthru // {
+        requireSigning = false;
+        allowAddonSideload = true;
+      };
+    });
     extraPrefsFiles = pkgs.librewolf-unwrapped.extraPrefsFiles ++ mobile-prefs;
     libName = "librewolf";
     dotDir = ".librewolf";
@@ -225,6 +232,13 @@ in
           // treat it as unrevoked.
           // see: <https://librewolf.net/docs/faq/#im-getting-sec_error_ocsp_server_error-what-can-i-do>
           defaultPref("security.OCSP.require", false);
+
+          // scrollbar configuration, see: <https://artemis.sh/2023/10/12/scrollbars.html>
+          // style=4 gives rectangular scrollbars
+          // could also enable "always show scrollbars" in about:preferences -- not sure what the actual pref name for that is
+          // note that too-large scrollbars (like 50px wide) tend to obscure content (and make buttons unclickable)
+          defaultPref("widget.non-native-theme.scrollbar.size.override", 20);
+          defaultPref("widget.non-native-theme.scrollbar.style", 4);
         '';
         fs."${cfg.browser.dotDir}/default".dir = {};
         # instruct Firefox to put the profile in a predictable directory (so we can do things like persist just it).

hosts/common/programs/flare-signal.nix

@@ -0,0 +1,47 @@
+# Flare is a 3rd-party GTK4 Signal app.
+# UI is effectively a clone of Fractal.
+#
+# compatibility:
+# - desko: works fine. pairs, and exchanges contact list (but not message history) with the paired device. exchanges future messages fine.
+# - moby (cross compiled flare-signal-nixified): nope. it pairs, but can only *receive* messages and never *send* them.
+#   - even `rsync`ing the data and keyrings from desko -> moby, still fails in that same manner.
+#   - console shows error messages. quite possibly an endianness mismatch somewhere
+# - moby (partially-emulated flare-signal): works! pairs and can send/receive messages, same as desko.
+#
+# error signatures (to reset, run `sane-wipe-fractal`):
+# - upon sending a message, the other side receives it, but Signal desktop gets "A message from Colin could not be delivered" and the local CLI shows:
+#   ```
+#   ERROR libsignal_service::websocket] SignalWebSocket: Websocket error: SignalWebSocket: end of application request stream; socket closing
+#   ERROR presage::manager] Error opening envelope: ProtobufDecodeError(DecodeError { description: "invalid tag value: 0", stack: [("Content", "data_message")] }), message will be skipped!
+#   ERROR presage::manager] Error opening envelope: ProtobufDecodeError(DecodeError { description: "invalid tag value: 0", stack: [("Content", "data_message")] }), message will be skipped!
+#   ```
+# - this occurs on moby, desko, `flare-signal` and `flare-signal-nixified`
+# - the Websocket error seems to be unrelated, occurs during normal/good operation
+# - related issues: <https://github.com/whisperfish/presage/issues/152>
+# error when sending from Flare to other Flare device:
+# - ```
+#   ERROR libsignal_protocol::session_cipher] Message from <UUID>.3 failed to decrypt; sender ratchet public key <key> message counter 1
+#       No current session
+#   ERROR presage::manager] Error opening envelope: SignalProtocolError(InvalidKyberPreKeyId), message will be skipped!
+#   ```
+# - but signal iOS will still read it.
+#
+# well, seems to have unpredictable errors particularly when being used on multiple devices.
+# desktop _seems_ more reliable than on mobile, but not confident.
+
+{ pkgs, ... }:
+{
+  sane.programs.flare-signal = {
+    package = pkgs.flare-signal-nixified;
+    # package = pkgs.flare-signal;
+    persist.byStore.private = [
+      # everything: conf, state, files, all opaque
+      ".local/share/flare"
+      # also persists a secret in ~/.local/share/keyrings. reset with:
+      # - `secret-tool search --all --unlock 'xdg:schema' 'de.schmidhuberj.Flare'`
+      # - `secret-tool clear 'xdg:schema' 'de.schmidhuberj.Flare'`
+      # and it persists some dconf settings (e.g. device name). reset with:
+      # - `dconf reset -f /de/schmidhuberj/Flare/`.
+    ];
+  };
+}

hosts/common/programs/fractal.nix

@@ -1,3 +1,22 @@
+# Fractal: GTK4 instant messenger client for the Matrix protocol
+#
+# very susceptible to state corruption during hard power-cycles.
+# if it stalls while launching, especially with a brief message at bottom
+# "unable to open store"
+# then:
+# - remove ~/.local/share/stable/*
+#   - this might give I/O error, in which case remove the corresponding path under
+#     /nix/persist/home/colin/private (which can be found by correlating timestamps/sizes with that in ~/private/.local/share/stable).
+# - reboot (maybe necessary).
+# - now you can send messages, and read messages in unencrypted rooms, but not read messages from encrypted rooms.
+# to fix encrypted message receipt:
+# - start from above (fractal closed, no ~/.local/share/stable/*)
+# - in ~/.local/share/keyrings/Default_keyring.keyring:
+#   - find the entry that says "display-name=Fractal: Matrix credentials for <mxid>"
+#   - remove that entry and all associated entries (i.e. ones with same number but different :attributeN)
+#   - REBOOT. otherwise keyring stuff seems to stay cached in RAM
+#   - login to Fractal. give an hour to sync.
+#   - it'll kick you back to a page asking you to cross-sign. open FluffyChat and do the emoji compare. success!
 { config, lib, pkgs, ... }:
 let
   cfg = config.sane.programs.fractal;
@@ -18,10 +37,11 @@ in
       };
     };
 
-    persist.private = [
+    persist.byStore.private = [
       # XXX by default fractal stores its state in ~/.local/share/<build-profile>/<UUID>.
       ".local/share/hack"    # for debug-like builds
       ".local/share/stable"  # for normal releases
+      ".local/share/fractal" # for version 5+, i think?
     ];
 
     suggestedPrograms = [ "gnome-keyring" ];

hosts/common/programs/g4music.nix

@@ -8,7 +8,7 @@
 { ... }:
 {
   sane.programs.g4music = {
-    persist.plaintext = [
+    persist.byStore.plaintext = [
       # index?
       ".cache/com.github.neithern.g4music"
     ];

hosts/common/programs/gajim.nix

@@ -1,7 +1,7 @@
 { ... }:
 {
   sane.programs.gajim = {
-    persist.private = [
+    persist.byStore.private = [
       # avatars, thumbnails...
       ".cache/gajim"
       # sqlite database labeled "settings". definitely includes UI theming

hosts/common/programs/geary.nix

@@ -0,0 +1,82 @@
+# geary is a gtk3 email client.
+# outstanding issues:
+# - it uses webkitgtk_4_1, which is expensive to build.
+#   could be upgraded to webkitgtk latest if upgraded to gtk4
+#   <https://gitlab.gnome.org/GNOME/geary/-/issues/1212>
+{ config, lib, ... }:
+let
+  cfg = config.sane.programs."gnome.geary";
+in
+{
+  sane.programs."gnome.geary" = {
+    configOption = with lib; mkOption {
+      default = {};
+      type = types.submodule {
+        options.autostart = mkOption {
+          type = types.bool;
+          default = false;
+        };
+      };
+    };
+
+    slowToBuild = true;  # uses webkitgtk 4.1
+    persist.byStore.private = [
+      # attachments, and email -- contained in a sqlite db
+      ".local/share/geary"
+      # also `.cache/geary/web-resources`, which tends to stay << 1 MiB
+    ];
+    fs.".config/geary/account_01/geary.ini".symlink.text = ''
+      [Metadata]
+      version=1
+      status=enabled
+
+      [Account]
+      ordinal=2
+      label=
+      # 14 = "fetch last 14d of mail every time i connect"
+      # -1 = "fetch *all* mail"
+      prefetch_days=-1
+      save_drafts=true
+      save_sent=true
+      use_signature=false
+      signature=
+      sender_mailboxes=colin@uninsane.org;
+      service_provider=other
+
+      [Folders]
+      archive_folder=Archive;
+      drafts_folder=
+      sent_folder=
+      junk_folder=
+      trash_folder=
+
+      [Incoming]
+      login=colin
+      remember_password=true
+      host=imap.uninsane.org
+      port=993
+      transport_security=transport
+      credentials=custom
+
+      [Outgoing]
+      remember_password=true
+      host=mx.uninsane.org
+      port=465
+      transport_security=transport
+      credentials=use-incoming
+    '';
+    secrets.".config/geary/account_02/geary.ini" = ../../../secrets/common/geary_account_02.ini.bin;
+
+    services.geary = {
+      description = "Geary email client";
+      wantedBy = lib.mkIf cfg.config.autostart [ "default.target" ];
+      serviceConfig = {
+        ExecStart = "${cfg.package}/bin/geary";
+        Type = "simple";
+        Restart = "always";
+        RestartSec = "20s";
+      };
+    };
+  };
+
+}

hosts/common/programs/gnome-weather.nix

@@ -3,7 +3,7 @@
 { ... }:
 {
   sane.programs.gnome-weather = {
-    persist.plaintext = [
+    persist.byStore.plaintext = [
       ".cache/libgweather"
     ];
   };

hosts/common/programs/gpodder.nix

@@ -1,4 +1,5 @@
-# gnome feeds RSS viewer
+# help:
+# - #gpodder on irc.libera.chat
 { config, pkgs, sane-lib, ... }:
 
 let
@@ -7,12 +8,22 @@ let
   wanted-feeds = feeds.filterByFormat ["podcast"] all-feeds;
 in {
   sane.programs.gpodder = {
-    package = pkgs.gpodder-adaptive-configured;
+    package = pkgs.gpodder-adaptive-configured.overrideAttrs (base: {
+      # environment variables:
+      # - GPODDER_HOME (defaults to "~/gPodder")
+      # - GPODDER_DOWNLOAD_DIR (defaults to "$GPODDER_HOME/Downloads")
+      # - GPODDER_WRITE_LOGS ("yes" or "no")
+      # - GPODDER_EXTENSIONS
+      # - GPODDER_DISABLE_EXTENSIONS ("yes" or "no")
+      extraMakeWrapperArgs = (base.extraMakeWrapperArgs or []) ++ [
+        "--set" "GPODDER_HOME" "~/.local/share/gPodder"
+      ];
+    });
     # package = pkgs.gpodder-configured;
     fs.".config/gpodderFeeds.opml".symlink.text = feeds.feedsToOpml wanted-feeds;
 
     # XXX: we preserve the whole thing because if we only preserve gPodder/Downloads
     #   then startup is SLOW during feed import, and we might end up with zombie eps in the dl dir.
-    persist.plaintext = [ "gPodder" ];
+    persist.byStore.plaintext = [ ".local/share/gPodder" ];
   };
 }

hosts/common/programs/gtkcord4.nix

@@ -0,0 +1,33 @@
+{ config, lib, pkgs, ... }:
+let
+  cfg = config.sane.programs.gtkcord4;
+in
+{
+  sane.programs.gtkcord4 = {
+    configOption = with lib; mkOption {
+      default = {};
+      type = types.submodule {
+        options.autostart = mkOption {
+          type = types.bool;
+          default = false;
+        };
+      };
+    };
+
+    persist.byStore.private = [
+      ".cache/gtkcord4"
+      ".config/gtkcord4"  # empty?
+    ];
+
+    services.gtkcord4 = {
+      description = "unofficial Discord chat client";
+      wantedBy = lib.mkIf cfg.config.autostart [ "default.target" ];
+      serviceConfig = {
+        ExecStart = "${cfg.package}/bin/gtkcord4";
+        Type = "simple";
+        Restart = "always";
+        RestartSec = "20s";
+      };
+    };
+  };
+}

hosts/common/programs/helix.nix

@@ -7,7 +7,7 @@
     # grammars need to be persisted when developing them
     # - `hx --grammar fetch` and `hx --grammar build`
     # but otherwise, they ship as part of HELIX_RUNTIME, in the nix store
-    # persist.plaintext = [ ".config/helix/runtime/grammars" ];
+    # persist.byStore.plaintext = [ ".config/helix/runtime/grammars" ];
     fs.".config/helix/config.toml".symlink.text = ''
       # docs: <https://docs.helix-editor.com/configuration.html>
       [editor.soft-wrap]

hosts/common/programs/jellyfin-media-player.nix

@@ -10,6 +10,6 @@
     # jellyfin stores things in a bunch of directories: this one persists auth info.
     # it *might* be possible to populate this externally (it's Qt stuff), but likely to
     # be fragile and take an hour+ to figure out.
-    persist.plaintext = [ ".local/share/Jellyfin Media Player" ];
+    persist.byStore.plaintext = [ ".local/share/Jellyfin Media Player" ];
   };
 }

hosts/common/programs/komikku.nix

@@ -3,6 +3,6 @@
   sane.programs.komikku = {
     secrets.".local/share/komikku/keyrings/plaintext.keyring" = ../../../secrets/common/komikku_accounts.json.bin;
     # downloads end up here, and without the toplevel database komikku doesn't know they exist.
-    persist.plaintext = [ ".local/share/komikku" ];
+    persist.byStore.plaintext = [ ".local/share/komikku" ];
   };
 }

hosts/common/programs/koreader/default.nix

@@ -26,9 +26,8 @@ in {
     package = pkgs.koreader-from-src;
     # koreader applies these lua "patches" at boot:
     # - <https://github.com/koreader/koreader/wiki/User-patches>
-    # - TODO: upstream this patch to koreader
+    # - 2023/10/29: koreader code hasn't changed, but somehow FTP browser seems usable even without the isConnected patch now.
-    # fs.".config/koreader/patches".symlink.target = "${./.}";
+    # fs.".config/koreader/patches/2-colin-NetworkManager-isConnected.lua".symlink.target = "${./2-colin-NetworkManager-isConnected.lua}";
-    fs.".config/koreader/patches/2-colin-NetworkManager-isConnected.lua".symlink.target = "${./2-colin-NetworkManager-isConnected.lua}";
 
     # koreader news plugin, enabled by default. file format described here:
     # - <repo:koreader/koreader:plugins/newsdownloader.koplugin/feed_config.lua>
@@ -37,12 +36,14 @@ in {
         ${lib.concatStringsSep ",\n  " koreaderRssEntries}
       }--do NOT change this line
     '';
+    # easier to navigate via filebrowser than finding the news menu entry
+    fs."Books/rss-koreader".symlink.target = "../.config/koreader/news";
 
     # koreader on aarch64 errors if there's no fonts directory (sandboxing thing, i guess)
     fs.".local/share/fonts".dir = {};
 
     # history, cache, dictionaries...
     # could be more explicit if i symlinked the history.lua file to somewhere it can persist better.
-    persist.plaintext = [ ".config/koreader" ];
+    persist.byStore.plaintext = [ ".config/koreader" ];
   };
 }

hosts/common/programs/lemoa.nix

@@ -2,6 +2,6 @@
 {
   sane.programs.lemoa = {
     # creds
-    persist.private = [ ".local/share/io.github.lemmygtk.lemoa" ];
+    persist.byStore.private = [ ".local/share/io.github.lemmygtk.lemoa" ];
   };
 }

hosts/common/programs/libreoffice.nix

@@ -6,6 +6,8 @@
     # package = pkgs.libreoffice-still;
     package = pkgs.libreoffice-fresh;
 
+    slowToBuild = true;
+
     # disable first-run stuff
     fs.".config/libreoffice/4/user/registrymodifications.xcu".symlink.text = ''
       <?xml version="1.0" encoding="UTF-8"?>

hosts/common/programs/mepo.nix

@@ -4,9 +4,9 @@
 
 {
   sane.programs.mepo = {
-    persist.plaintext = [ ".cache/mepo/tiles" ];
+    persist.byStore.plaintext = [ ".cache/mepo/tiles" ];
     # ~/.cache/mepo/savestate has precise coordinates and pins: keep those private
-    persist.private = [
+    persist.byStore.private = [
       { type = "file"; path = ".cache/mepo/savestate"; }
     ];
 

hosts/common/programs/mopidy.nix

@@ -42,10 +42,10 @@ in
       # mopidy-moped: <https://github.com/martijnboland/moped>
       # mopidy-muse: <https://github.com/cristianpb/muse>
     ]);
-    persist.plaintext = [
+    persist.byStore.plaintext = [
       ".local/share/mopidy/local"  # thumbs, library db
     ];
-    persist.private = [
+    persist.byStore.private = [
       ".local/share/mopidy/http"  # cookie
     ];
     secrets.".config/mopidy/mopidy.conf" = ../../../secrets/common/mopidy.conf.bin;

hosts/common/programs/mpv.nix

@@ -3,41 +3,57 @@
 # - <https://github.com/mpv-player/mpv/wiki>
 # curated mpv mods/scripts/users:
 # - <https://github.com/stax76/awesome-mpv>
-{ pkgs, ... }:
+{ config, lib, pkgs, ... }:
 
+let
+  cfg = config.sane.programs.mpv;
+in
 {
   sane.programs.mpv = {
+    configOption = with lib; mkOption {
+      default = {};
+      type = types.submodule {
+        options.vo = mkOption {
+          type = types.nullOr types.str;
+          default = null;
+          description = "--vo=FOO flag to pass to mpv";
+        };
+      };
+    };
     package = pkgs.wrapMpv pkgs.mpv-unwrapped {
       scripts = with pkgs.mpvScripts; [
         mpris
         # uosc
         pkgs.mpv-uosc-latest
       ];
-      extraMakeWrapperArgs = [
+      extraMakeWrapperArgs = lib.optionals (cfg.config.vo != null) [
         # 2023/08/29: fixes an error where mpv on moby launches with the message
         #   "DRM_IOCTL_MODE_CREATE_DUMB failed: Cannot allocate memory"
         #   audio still works, and controls, screenshotting, etc -- just not the actual rendering
         # this is likely a regression for mpv 0.36.0.
         # the actual error message *appears* to come from the mesa library, but it's tough to trace.
+        #
+        # backend compatibility (2023/10/22):
         # run with `--vo=help` to see a list of all output options.
-        # non-exhaustive (F=fails, W=works)
+        # non-exhaustive (W=works, F=fails, A=audio-only, U=audio+ui only (no video))
-        # ? libmpv           render API for libmpv
-        # ? gpu              Shader-based GPU Renderer
-        # ? gpu-next         Video output based on libplacebo
-        # ? vdpau            VDPAU with X11
-        # ? wlshm            Wayland SHM video output (software scaling)
-        # ? xv               X11/Xv
-        # W sdl              SDL 2.0 Renderer
-        # F dmabuf-wayland   Wayland dmabuf video output
-        # ? vaapi            VA API with X11
-        # ? x11              X11 (software scaling)
         # ? null             Null video output
-        # ? caca             libcaca
+        # A (default)
+        # A dmabuf-wayland   Wayland dmabuf video output
+        # A libmpv           render API for libmpv  (mpv plays the audio, but doesn't even render a window)
+        # A vdpau            VDPAU with X11
         # F drm              Direct Rendering Manager (software scaling)
-        "--add-flags" "--vo=sdl"
+        # F gpu-next         Video output based on libplacebo
+        # F vaapi            VA API with X11
+        # F x11              X11 (software scaling)
+        # F xv               X11/Xv
+        # U gpu              Shader-based GPU Renderer
+        # W caca             libcaca  (terminal rendering)
+        # W sdl              SDL 2.0 Renderer
+        # W wlshm            Wayland SHM video output (software scaling)
+        "--add-flags" "--vo=${cfg.config.vo}"
       ];
     };
-    persist.plaintext = [ ".local/state/mpv/watch_later" ];
+    persist.byStore.plaintext = [ ".local/state/mpv/watch_later" ];
     fs.".config/mpv/input.conf".symlink.text = ''
       # let volume/power keys be interpreted by the system.
       # this is important for sxmo.
@@ -101,6 +117,7 @@
     mime.priority = 50;  # default = 100; 50 in order to take precedence over vlc.
     mime.associations."audio/flac" = "mpv.desktop";
     mime.associations."audio/mpeg" = "mpv.desktop";
+    mime.associations."audio/x-opus+ogg" = "mpv.desktop";
     mime.associations."audio/x-vorbis+ogg" = "mpv.desktop";
     mime.associations."video/mp4" = "mpv.desktop";
     mime.associations."video/quicktime" = "mpv.desktop";

hosts/common/programs/neovim.nix

@@ -85,27 +85,13 @@ let
   plugin-config-lua = concatMapStrings (p: optionalString (p.type or "" == "lua") p.config) plugins;
 in
 {
-  # private because there could be sensitive things in the swap
   sane.programs.neovim = {
-    persist.private = [ ".cache/vim-swap" ];
+    # package = config.programs.neovim.finalPackage;
-    env.EDITOR = "vim";
+    package = pkgs.wrapNeovimUnstable pkgs.neovim-unwrapped (pkgs.neovimUtils.makeNeovimConfig {
-    # git claims it should use EDITOR, but it doesn't!
+      withRuby = false;  #< doesn't cross-compile w/o binfmt
-    env.GIT_EDITOR = "vim";
+      viAlias = true;
-    mime.priority = 200;  # default=100 => yield to other, more specialized applications
+      vimAlias = true;
-    mime.associations."application/schema+json" = "nvim.desktop";
+      plugins = plugin-packages;
-    mime.associations."plain/text" = "nvim.desktop";
-    mime.associations."text/markdown" = "nvim.desktop";
-  };
-
-  programs.neovim = mkIf config.sane.programs.neovim.enabled {
-    # neovim: https://github.com/neovim/neovim
-    enable = true;
-    viAlias = true;
-    vimAlias = true;
-    configure = {
-      packages.plugins = {
-        start = plugin-packages;
-      };
       customRC = ''
         " let the terminal handle mouse events, that way i get OS-level ctrl+shift+c/etc
         " this used to be default, until <https://github.com/neovim/neovim/pull/19290>
@@ -147,6 +133,16 @@ in
         ${plugin-config-lua}
         EOF
       '';
-    };
+    });
+
+    # private because there could be sensitive things in the swap
+    persist.byStore.private = [ ".cache/vim-swap" ];
+    env.EDITOR = "vim";
+    # git claims it should use EDITOR, but it doesn't!
+    env.GIT_EDITOR = "vim";
+    mime.priority = 200;  # default=100 => yield to other, more specialized applications
+    mime.associations."application/schema+json" = "nvim.desktop";
+    mime.associations."plain/text" = "nvim.desktop";
+    mime.associations."text/markdown" = "nvim.desktop";
   };
 }

hosts/common/programs/newsflash.nix

@@ -7,7 +7,7 @@ let
   wanted-feeds = feeds.filterByFormat ["text" "image"] all-feeds;
 in {
   sane.programs.newsflash = {
-    persist.plaintext = [ ".local/share/news-flash" ];
+    persist.byStore.plaintext = [ ".local/share/news-flash" ];
     fs.".config/newsflashFeeds.opml".symlink.text =
       feeds.feedsToOpml wanted-feeds
     ;

hosts/common/programs/nheko.nix

@@ -1,9 +1,11 @@
 { ... }:
 {
-  # not strictly necessary, but allows caching articles; offline use, etc.
+  sane.programs.nheko = {
-  sane.programs.nheko.persist.private = [
+    # not strictly necessary, but allows caching articles; offline use, etc.
-    ".config/nheko"  # config file (including client token)
+    persist.byStore.private = [
-    ".cache/nheko"  # media cache
+      ".config/nheko"  # config file (including client token)
-    ".local/share/nheko"  # per-account state database
+      ".cache/nheko"  # media cache
-  ];
+      ".local/share/nheko"  # per-account state database
+    ];
+  };
 }

hosts/common/programs/nix-index.nix

@@ -2,6 +2,6 @@
 {
   # provides `nix-locate`, backed by the manually run `nix-index`
   sane.programs.nix-index = {
-    persist.plaintext = [ ".cache/nix-index" ];
+    persist.byStore.plaintext = [ ".cache/nix-index" ];
   };
 }