sxmo: have sway launch sxmo -- not the other way around

this lets me treat sxmo as just some nice scripts which run atop an existing DE (sway), rather than the opposite

can share more code with my desktop/laptop

.sops.yaml

@@ -8,7 +8,7 @@ keys:
   - &host_servo age1tzlyex2z6t88tg9h82943e39shxhmqeyr7ywhlwpdjmyqsndv3qq27x0rf
   - &host_moby age18vq5ktwgeaysucvw9t67drqmg5zd5c5k3le34yqxckkfj7wqdqgsd4ejmt
 creation_rules:
-  - path_regex: secrets/universal*
+  - path_regex: secrets/common*
     key_groups:
     - age:
       - *user_desko_colin
@@ -26,19 +26,19 @@ creation_rules:
       - *user_lappy_colin
       - *user_servo_colin
       - *host_servo
-  - path_regex: secrets/desko.yaml$
+  - path_regex: secrets/desko*
     key_groups:
     - age:
       - *user_desko_colin
       - *user_lappy_colin
       - *host_desko
-  - path_regex: secrets/lappy.yaml$
+  - path_regex: secrets/lappy*
     key_groups:
     - age:
       - *user_lappy_colin
       - *user_desko_colin
       - *host_lappy
-  - path_regex: secrets/moby.yaml$
+  - path_regex: secrets/moby*
     key_groups:
     - age:
       - *user_desko_colin

README.md

@@ -0,0 +1,111 @@
+## What's Here
+
+this is the top-level repo from which i configure/deploy all my NixOS machines:
+- desktop
+- laptop
+- server
+- mobile phone
+
+i enjoy a monorepo approach. this repo references [nixpkgs][nixpkgs], a couple 3rd party
+nix modules like [sops][sops], the sources for [uninsane.org][uninsane-org], and that's
+about it. custom derivations and modules (some of which i try to upstream) live
+directly here; even the sources for those packages is often kept here too.
+
+[nixpkgs]: https://github.com/NixOS/nixpkgs
+[sops]: https://github.com/Mic92/sops-nix
+[uninsane-org]: https://uninsane.org
+
+## Layout
+- `doc/`
+    - instructions for tasks i find myself doing semi-occasionally in this repo.
+- `hosts/`
+    - the bulk of config which isn't factored with external use in mind.
+    - that is, if you were to add this repo to a flake.nix for your own use,
+      you won't likely be depending on anything in this directory.
+- `integrations/`
+    - code intended for consumption by external tools (e.g. the Nix User Repos)
+- `modules/`
+    - config which is gated behind `enable` flags, in similar style to nixpkgs'
+      `nixos/` directory.
+    - if you depend on this repo, it's most likely for something in this directory.
+- `nixpatches/`
+    - literally, diffs i apply atop upstream nixpkgs before performing further eval.
+- `overlays/`
+    - exposed via the `overlays` output in `flake.nix`.
+    - predominantly a list of `callPackage` directives.
+- `pkgs/`
+    - derivations for things not yet packaged in nixpkgs.
+    - derivations for things from nixpkgs which i need to `override` for some reason.
+    - inline code for wholly custom packages (e.g. `pkgs/additional/sane-scripts/` for CLI tools
+      that are highly specific to my setup).
+- `scripts/`
+    - scripts which aren't reachable on a deployed system, but may aid manual deployments
+- `secrets/`
+    - encrypted keys, API tokens, anything which one or more of my machines needs
+      read access to but shouldn't be world-readable.
+    - not much to see here
+- `templates/`
+    - exposed via the `templates` output in `flake.nix`.
+    - used to instantiate short-lived environments.
+    - used to auto-fill the boiler-plate portions of new packages.
+
+
+## Key Points of Interest
+
+i.e. you might find value in using these in your own config:
+
+- `modules/fs/`
+    - use this to statically define leafs and nodes anywhere in the filesystem,
+      not just inside `/nix/store`.
+    - e.g. specify that `/var/www` should be:
+        - owned by a specific user/group
+        - set to a specific mode
+        - symlinked to some other path
+        - populated with some statically-defined data
+        - populated according to some script
+        - created as a dependency of some service (e.g. `nginx`)
+    - values defined here are applied neither at evaluation time _nor_ at activation time.
+        - rather, they become systemd services.
+        - systemd manages dependencies
+        - e.g. link `/var/www -> /mnt/my-drive/www` only _after_ `/mnt/my-drive/www` appears)
+    - this is akin to using [Home Manager's][home-manager] file API -- the part which lets you
+      statically define `~/.config` files -- just with a different philosophy.
+- `modules/persist/`
+    - my alternative to the Impermanence module.
+    - this builds atop `modules/fs/` to achieve things stock impermanence can't:
+        - persist things to encrypted storage which is unlocked at login time (pam_mount).
+        - "persist" cache directories -- to free up RAM -- but auto-wipe them on mount
+          and encrypt them to ephemeral keys so they're unreadable post shutdown/unmount.
+- `modules/programs.nix`
+    - like nixpkgs' `programs` options, but allows both system-wide or per-user deployment.
+    - allows `fs` and `persist` config values to be gated behind program deployment:
+        - e.g. `/home/<user>/.mozilla/firefox` is persisted only for users who
+          `sane.programs.firefox.enableFor.user."<user>" = true;`
+- `modules/users.nix`
+    - convenience layer atop the above modules so that you can just write
+      `fs.".config/git"` instead of `fs."/home/colin/.config/git"`
+
+some things in here could easily find broader use. if you would find benefit in
+them being factored out of my config, message me and we could work to make that happen.
+
+[home-manager]: https://github.com/nix-community/home-manager
+
+## Using This Repo In Your Own Config
+
+this should be a pretty "standard" flake. just reference it, and import either
+- `nixosModules.sane` (for the modules)
+- `overlays.pkgs` (for the packages)
+
+## Mirrors
+
+this repo exists in a few known locations:
+- primary: <https://git.uninsane.org/colin/nix-files>
+- mirror: <https://github.com/nix-community/nur-combined/tree/master/repos/colinsane>
+
+## Contact
+
+if you want to contact me for questions, or collaborate to split something useful into a shared repo, etc,
+you can reach me via any method listed [here](https://uninsane.org/about).
+patches, for this repo or any other i host, will be warmly welcomed in any manner you see fit:
+`git send-email`, DM'ing the patch over Matrix/Lemmy/ActivityPub/etc, even a literal PR where you
+link me to your own clone.

TODO.md

@@ -0,0 +1,103 @@
+## BUGS
+- mpv UI is sometimes blank for audio/podcasts?
+  - i think it's when the audio file has no thumbnail?
+- why i need to manually restart `wireguard-wg-ovpns` on servo periodically
+  - else DNS fails
+- fix epiphany URL bar input on moby
+- sxmo: wvkbd: missing font for icons on the 3rd page
+
+## REFACTORING:
+
+### sops/secrets
+- attach secrets to the thing they're used by (sane.programs)
+- rework secrets to leverage `sane.fs`
+- remove sops activation script as it's covered by my systemd sane.fs impl
+
+### roles
+- allow any host to take the role of `uninsane.org`
+  - will make it easier to test new services?
+
+### upstreaming
+- split out a sxmo module usable by NUR consumers
+- bump nodejs version in lemmy-ui
+- add updateScripts to all my packages in nixpkgs
+- fix lightdm-mobile-greeter for newer libhandy
+- port zecwallet-lite to a from-source build
+- REVIEW/integrate jellyfin dataDir config: <https://github.com/NixOS/nixpkgs/pull/233617>
+- remove `libsForQt5.callPackage` broadly: <https://github.com/NixOS/nixpkgs/issues/180841>
+
+#### upstreaming to non-nixpkgs repos
+- gtk: build schemas even on cross compilation: <https://github.com/NixOS/nixpkgs/pull/247844>
+- sxmo: add new app entries
+
+
+## IMPROVEMENTS:
+### security/resilience
+- validate duplicity backups!
+- encrypt more ~ dirs (~/archives, ~/records, ..?)
+  - best to do this after i know for sure i have good backups
+- have `sane.programs` be wrapped such that they run in a cgroup?
+  - at least, only give them access to the portion of the fs they *need*.
+  - Android takes approach of giving each app its own user: could hack that in here.
+  - **systemd-run** takes a command and runs it in a temporary scope (cgroup)
+    - presumably uses the same options as systemd services
+    - see e.g. <https://github.com/NixOS/nixpkgs/issues/113903#issuecomment-857296349>
+  - flatpak does this, somehow
+  - apparmor?  SElinux?  (desktop) "portals"?
+  - see Spectrum OS; Alyssa Ross; etc
+  - bubblewrap-based sandboxing: <https://github.com/nixpak/nixpak>
+- canaries for important services
+  - e.g. daily email checks; daily backup checks
+  - integrate `nix check` into Gitea actions?
+
+### user experience
+- moby: sxmo: fix youtube scripts (package youtube-cli)
+- moby: tune GPS
+  - run only geoclue, and not gpsd, to save power?
+  - tune QGPS setting in eg25-control, for less jitter?
+  - direct mepo to prefer gpsd, with fallback to geoclue, for better accuracy?
+  - configure geoclue to do some smoothing?
+  - manually do smoothing, as some layer between mepo and geoclue/gpsd?
+- neovim: set up language server (lsp; rnix-lsp; nvim-lspconfig)
+- Helix: make copy-to-system clipboard be the default
+- firefox/librewolf: persist history
+  - just not cookies or tabs
+- moby: improve gPodder launch time
+- moby: theme GTK apps (i.e. non-adwaita styles)
+  - especially, make the menubar collapsible
+  - try Gradience tool specifically for theming adwaita? <https://linuxphoneapps.org/apps/com.github.gradienceteam.gradience/>
+- package Nix/NixOS docs for Zeal
+  - install [doc-browser](https://github.com/qwfy/doc-browser)
+  - this supports both dash (zeal) *and* the datasets from <https://devdocs.io> (which includes nix!)
+  - install [devhelp](https://wiki.gnome.org/Apps/Devhelp)  (gnome)
+- have xdg-open parse `<repo:...> URIs (or adjust them so that it _can_ parse)
+- sane-bt-search: show details like 5.1 vs stereo, h264 vs h265
+- uninsane.org: make URLs relative to allow local use (and as offline homepage)
+- email: fix so that local mail doesn't go to junk
+  - git sendmail flow adds the DKIM signatures, but gets delivered locally w/o having the sig checked, so goes into Junk
+  - could change junk filter from "no DKIM success" to explicit "DKIM failed"
+- sxmo: don't put all deps on PATH
+  - maybe: use resholve to hard-code them
+    - this is the most "correct", but least patchable
+  - maybe: express each invocation as a function in sxmo_common.sh
+    - this will require some patching to handle `exec <foo>` style
+  - maybe: save original PATH and reset it before invoking user files
+
+### perf
+- add `pkgs.impure-cached.<foo>` package set to build things with ccache enabled
+  - every package here can be auto-generated, and marked with some env var so that it doesn't pollute the pure package set
+  - would be super handy for package prototyping!
+- why does nixos-rebuild switch take 5 minutes when net is flakey?
+  - trying to auto-mount servo?
+  - something to do with systemd services restarting/stalling
+  - maybe wireguard & its refresh operation, specifically?
+- get moby to build without binfmt emulation (i.e. make all emulation explicit)
+  - then i can distribute builds across servo + desko, and also allow servo to pull packages from desko w/o worrying about purity
+
+
+## NEW FEATURES:
+- migrate MAME cabinet to nix
+  - boot it from PXE from servo?
+- deploy to new server, and use it as a remote builder
+- enable IPv6
+- package lemonade lemmy app: <https://linuxphoneapps.org/apps/ml.mdwalters.lemonade/>

doc/adding-a-program.md

@@ -0,0 +1,13 @@
+to ship `pkgs.foo` on some host, either:
+- add it as an entry in `suggestedPrograms` to the appropriate category in `hosts/common/programs/assorted.nix`, or
+- `sane.programs.foo.enableFor.user.colin = true` in `hosts/by-name/myhost/default.nix`
+
+if the program needs customization (persistence, configs, secrets):
+- add a file for it at `hosts/common/programs/<foo>.nix`
+- set the options, `sane.programs.foo.{fs,persist}`
+
+if it's unclear what fs paths a program uses:
+- run one of these commands, launch the program, run it again, and `diff`:
+  - `du -x --apparent-size ~`
+  - `find ~ -xdev`
+- or, inspect the whole tmpfs root with `ncdu -x /`

flake.lock

@@ -1,12 +1,15 @@
 {
   "nodes": {
     "flake-utils": {
+      "inputs": {
+        "systems": "systems"
+      },
       "locked": {
-        "lastModified": 1659877975,
-        "narHash": "sha256-zllb8aq3YO3h8B/U0/J1WBgAL8EX5yWf5pMj3G0NAmc=",
+        "lastModified": 1687709756,
+        "narHash": "sha256-Y5wKlQSkgEK2weWdOu4J3riRd+kV/VCgHsqLNTTWQ/0=",
         "owner": "numtide",
         "repo": "flake-utils",
-        "rev": "c0e246b9b83f637f4681389ecabcb2681b4f3af0",
+        "rev": "dbabf0ca0c0c4bce6ea5eaf65af5cb694d2082c7",
         "type": "github"
       },
       "original": {
@@ -18,11 +21,11 @@
     "mobile-nixos": {
       "flake": false,
       "locked": {
-        "lastModified": 1674880620,
-        "narHash": "sha256-JMALuC7xcoH/T66sKTVLuItHfOJBCWsNKpE49Qrvs80=",
+        "lastModified": 1690059310,
+        "narHash": "sha256-4zcoDp8wwZVfGSzXltC5x+eH4kDWC/eJpyQNgr7shAA=",
         "owner": "nixos",
         "repo": "mobile-nixos",
-        "rev": "7478a9ffad737486951186b66f6c5535dc5802e2",
+        "rev": "56fc9f9619f305f0865354975a98d22410eed127",
         "type": "github"
       },
       "original": {
@@ -31,46 +34,62 @@
         "type": "github"
       }
     },
+    "nix-serve": {
+      "inputs": {
+        "nixpkgs": "nixpkgs"
+      },
+      "locked": {
+        "lastModified": 1687251388,
+        "narHash": "sha256-E9cVlgeCvzPbA/G3mCDCzz8TdRwXyGYzIjmwcvIfghg=",
+        "owner": "edolstra",
+        "repo": "nix-serve",
+        "rev": "d6df5bd8584f37e22cff627db2fc4058a4aab5ee",
+        "type": "github"
+      },
+      "original": {
+        "owner": "edolstra",
+        "repo": "nix-serve",
+        "type": "github"
+      }
+    },
     "nixpkgs": {
-      "inputs": {
-        "nixpkgs": [
-          "nixpkgs-unpatched"
-        ]
-      },
       "locked": {
-        "lastModified": 1,
-        "narHash": "sha256-FTUAvxSeQToawyfVP9/S2143D5EgCbk88qI2PePLQQ8=",
-        "path": "/nix/store/s9v0l913m4drrddglbjqa384nxxwhxca-source/nixpatches",
-        "type": "path"
+        "lastModified": 1606086654,
+        "narHash": "sha256-VFl+3eGIMqNp7cyOMJ6TjM/+UcsLKtodKoYexrlTJMI=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "19db3e5ea2777daa874563b5986288151f502e27",
+        "type": "github"
       },
       "original": {
-        "path": "/nix/store/s9v0l913m4drrddglbjqa384nxxwhxca-source/nixpatches",
-        "type": "path"
+        "id": "nixpkgs",
+        "ref": "nixos-20.09",
+        "type": "indirect"
       }
     },
     "nixpkgs-stable": {
       "locked": {
-        "lastModified": 1675556398,
-        "narHash": "sha256-5Gf5KlmFXfIGVQb2hmiiE7FQHoLd4UtEhIolLQvNB/A=",
+        "lastModified": 1693097136,
+        "narHash": "sha256-fBZSMdBaoZ0INFbyZ5s0DOF7zDNcLsLxgkwdDh3l9Pc=",
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "e32c33811815ca4a535a16faf1c83eeb4493145b",
+        "rev": "9117c4e9dc117a6cd0319cca40f2349ed333669d",
         "type": "github"
       },
       "original": {
         "owner": "NixOS",
-        "ref": "release-22.11",
+        "ref": "release-23.05",
         "repo": "nixpkgs",
         "type": "github"
       }
     },
     "nixpkgs-unpatched": {
       "locked": {
-        "lastModified": 1675942811,
-        "narHash": "sha256-/v4Z9mJmADTpXrdIlAjFa1e+gkpIIROR670UVDQFwIw=",
+        "lastModified": 1693663421,
+        "narHash": "sha256-ImMIlWE/idjcZAfxKK8sQA7A1Gi/O58u5/CJA+mxvl8=",
         "owner": "nixos",
         "repo": "nixpkgs",
-        "rev": "724bfc0892363087709bd3a5a1666296759154b1",
+        "rev": "e56990880811a451abd32515698c712788be5720",
         "type": "github"
       },
       "original": {
@@ -83,7 +102,7 @@
     "root": {
       "inputs": {
         "mobile-nixos": "mobile-nixos",
-        "nixpkgs": "nixpkgs",
+        "nix-serve": "nix-serve",
         "nixpkgs-unpatched": "nixpkgs-unpatched",
         "sops-nix": "sops-nix",
         "uninsane-dot-org": "uninsane-dot-org"
@@ -92,16 +111,16 @@
     "sops-nix": {
       "inputs": {
         "nixpkgs": [
-          "nixpkgs"
+          "nixpkgs-unpatched"
         ],
         "nixpkgs-stable": "nixpkgs-stable"
       },
       "locked": {
-        "lastModified": 1675872570,
-        "narHash": "sha256-RPH3CeTv7ixC2WcYiKyhmIgoH/9tur4Kr+3Vg/pleQk=",
+        "lastModified": 1693404499,
+        "narHash": "sha256-cx/7yvM/AP+o/3wPJmA9W9F+WHemJk5t+Xcr+Qwkqhg=",
         "owner": "Mic92",
         "repo": "sops-nix",
-        "rev": "8fec29b009c19538e68d5d814ec74e04f662fbd1",
+        "rev": "d9c5dc41c4b1f74c77f0dbffd0f3a4ebde447b7a",
         "type": "github"
       },
       "original": {
@@ -110,19 +129,34 @@
         "type": "github"
       }
     },
+    "systems": {
+      "locked": {
+        "lastModified": 1681028828,
+        "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
+        "owner": "nix-systems",
+        "repo": "default",
+        "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-systems",
+        "repo": "default",
+        "type": "github"
+      }
+    },
     "uninsane-dot-org": {
       "inputs": {
         "flake-utils": "flake-utils",
         "nixpkgs": [
-          "nixpkgs"
+          "nixpkgs-unpatched"
         ]
       },
       "locked": {
-        "lastModified": 1675131883,
-        "narHash": "sha256-yBgJDG72YqIr1bltasqHD1E/kHc9uRFgDjxDmy6kI8M=",
+        "lastModified": 1691106178,
+        "narHash": "sha256-3mZ9gTvMpbZA9ea9ovoQpn2wKuQY0QZ7MDdEjArYdAQ=",
         "ref": "refs/heads/master",
-        "rev": "b099c24091cc192abf3997b94342d4b31cc5757b",
-        "revCount": 170,
+        "rev": "f4d91aa201b6e49af690f250d4786bd1d8b4dcfd",
+        "revCount": 205,
         "type": "git",
         "url": "https://git.uninsane.org/colin/uninsane"
       },

flake.nix

@@ -12,21 +12,38 @@
 # - Flake RFC: <https://github.com/tweag/rfcs/blob/flakes/rfcs/0049-flakes.md>
 #   - Discussion: <https://github.com/NixOS/rfcs/pull/49>
 # - <https://serokell.io/blog/practical-nix-flakes>
+#
+#
+# COMMON OPERATIONS:
+# - update a specific flake input:
+#   - `nix flake lock --update-input nixpkgs`
 
 {
   # XXX: use the `github:` scheme instead of the more readable git+https: because it's *way* more efficient
   # preferably, i would rewrite the human-readable https URLs to nix-specific github: URLs with a helper,
   # but `inputs` is required to be a strict attrset: not an expression.
   inputs = {
-    # <https://github.com/nixos/nixpkgs/tree/nixos-22.11>
-    # nixpkgs-stable.url = "github:nixos/nixpkgs?ref=nixos-22.11";
-
+    # branch workflow:
+    # - daily:
+    #   - nixos-unstable cut from master after enough packages have been built in caches.
+    # - every 6 hours:
+    #   - master auto-merged into staging.
+    #   - staging-next auto-merged into staging.
+    # - manually, approximately once per month:
+    #   - staging-next is cut from staging.
+    #   - staging-next merged into master.
+    #
+    # which branch to source from?
+    # - for everyday development, prefer `nixos-unstable` branch, as it provides good caching.
+    # - if need to test bleeding updates (e.g. if submitting code into staging):
+    #   - use `staging-next` if it's been cut (i.e. if there's an active staging-next -> master PR)
+    #   - use `staging` if no staging-next branch has been cut.
+    #
     # <https://github.com/nixos/nixpkgs/tree/nixos-unstable>
     nixpkgs-unpatched.url = "github:nixos/nixpkgs?ref=nixos-unstable";
-    nixpkgs = {
-      url = "./nixpatches";
-      inputs.nixpkgs.follows = "nixpkgs-unpatched";
-    };
+    # nixpkgs-unpatched.url = "github:nixos/nixpkgs?ref=staging-next";
+    # nixpkgs-unpatched.url = "github:nixos/nixpkgs?ref=staging";
+
     mobile-nixos = {
       # <https://github.com/nixos/mobile-nixos>
       url = "github:nixos/mobile-nixos";
@@ -35,66 +52,104 @@
     sops-nix = {
       # <https://github.com/Mic92/sops-nix>
       url = "github:Mic92/sops-nix";
-      inputs.nixpkgs.follows = "nixpkgs";
+      # inputs.nixpkgs.follows = "nixpkgs";
+      inputs.nixpkgs.follows = "nixpkgs-unpatched";
     };
     uninsane-dot-org = {
       url = "git+https://git.uninsane.org/colin/uninsane";
-      inputs.nixpkgs.follows = "nixpkgs";
+      # inputs.nixpkgs.follows = "nixpkgs";
+      inputs.nixpkgs.follows = "nixpkgs-unpatched";
+    };
+    nix-serve = {
+      # <https://github.com/edolstra/nix-serve>
+      url = "github:edolstra/nix-serve";
     };
   };
 
   outputs = {
     self,
-    nixpkgs,
     nixpkgs-unpatched,
     mobile-nixos,
     sops-nix,
     uninsane-dot-org,
+    nix-serve,
     ...
   }@inputs:
     let
-      nixpkgsCompiledBy = local: nixpkgs.legacyPackages."${local}";
-
-      evalHost = { name, local, target }:
-        let
-          # XXX: we'd prefer to use `nixosSystem = (nixpkgsCompiledBy target).nixos`
-          # but it doesn't propagate config to the underlying pkgs, meaning it doesn't let you use
-          # non-free packages even after setting nixpkgs.allowUnfree.
-          # XXX: patch using the target -- not local -- otherwise the target will
-          # need to emulate the host in order to rebuild!
-          nixosSystem = import ((nixpkgsCompiledBy target).path + "/nixos/lib/eval-config.nix");
-        in
-          (nixosSystem {
-            # we use pkgs built for and *by* the target, i.e. emulation, by default.
-            # cross compilation only happens on explicit access to `pkgs.cross`
-            system = target;
-            modules = [
-              (import ./hosts/instantiate.nix { localSystem = local; hostName = name; })
-              self.nixosModules.default
-              self.nixosModules.passthru
-              {
-                nixpkgs.overlays = [
-                  self.overlays.default
-                  self.overlays.passthru
-                  self.overlays.pins
-                ];
-              }
-            ];
-          });
-    in {
-      nixosConfigurations = {
-        servo = evalHost { name = "servo"; local = "x86_64-linux"; target = "x86_64-linux"; };
-        desko = evalHost { name = "desko"; local = "x86_64-linux"; target = "x86_64-linux"; };
-        lappy = evalHost { name = "lappy"; local = "x86_64-linux"; target = "x86_64-linux"; };
-        moby = evalHost { name = "moby"; local = "aarch64-linux"; target = "aarch64-linux"; };
-        # special cross-compiled variant, to speed up deploys from an x86 box to the arm target
-        # note that these *do* produce different store paths, because the closure for the tools used to cross compile
-        # v.s. emulate differ.
-        # so deploying foo-cross and then foo incurs some rebuilding.
-        moby-cross = evalHost { name = "moby"; local = "x86_64-linux"; target = "aarch64-linux"; };
-        rescue = evalHost { name = "rescue"; local = "x86_64-linux"; target = "x86_64-linux"; };
+      inherit (builtins) attrNames elem listToAttrs map mapAttrs;
+      mapAttrs' = f: set:
+        listToAttrs (map (attr: f attr set.${attr}) (attrNames set));
+      # mapAttrs but without the `name` argument
+      mapAttrValues = f: mapAttrs (_: f);
+      # rather than apply our nixpkgs patches as a flake input, do that here instead.
+      # this (temporarily?) resolves the bad UX wherein a subflake residing in the same git
+      # repo as the main flake causes the main flake to have an unstable hash.
+      nixpkgs = (import ./nixpatches/flake.nix).outputs {
+        self = nixpkgs;
+        nixpkgs = nixpkgs-unpatched;
       };
 
+      nixpkgsCompiledBy = system: nixpkgs.legacyPackages."${system}";
+
+      evalHost = { name, local, target }: nixpkgs.lib.nixosSystem {
+        system = target;
+        modules = [
+          {
+            nixpkgs = (if (local != null) then {
+              buildPlatform = local;
+            } else {}) // {
+              # TODO: does the earlier `system` arg to nixosSystem make its way here?
+              hostPlatform.system = target;
+            };
+            # nixpkgs.buildPlatform = local;  # set by instantiate.nix instead
+            # nixpkgs.config.replaceStdenv = { pkgs }: pkgs.ccacheStdenv;
+          }
+          (import ./hosts/instantiate.nix { hostName = name; })
+          self.nixosModules.default
+          self.nixosModules.passthru
+          {
+            nixpkgs.overlays = [
+              self.overlays.passthru
+              self.overlays.sane-all
+            ];
+          }
+        ];
+      };
+    in {
+      nixosConfigurations =
+        let
+          hosts = {
+            servo =  { name = "servo"; local = "x86_64-linux"; target = "x86_64-linux"; };
+            desko =  { name = "desko"; local = "x86_64-linux"; target = "x86_64-linux"; };
+            lappy =  { name = "lappy"; local = "x86_64-linux"; target = "x86_64-linux"; };
+            moby  =  { name = "moby";  local = "x86_64-linux"; target = "aarch64-linux"; };
+            rescue = { name = "rescue"; local = "x86_64-linux"; target = "x86_64-linux"; };
+          };
+          # cross-compiled builds: instead of emulating the host, build using a cross-compiler.
+          # - these are faster to *build* than the emulated variants (useful when tweaking packages),
+          # - but fewer of their packages can be found in upstream caches.
+          cross = mapAttrValues evalHost hosts;
+          emulated = mapAttrValues
+            ({name, local, target}: evalHost {
+              inherit name target;
+              local = null;
+            })
+            hosts;
+          prefixAttrs = prefix: attrs: mapAttrs'
+            (name: value: {
+              name = prefix + name;
+              inherit value;
+            })
+            attrs;
+        in
+          (prefixAttrs "cross-" cross) //
+          (prefixAttrs "emulated-" emulated) // {
+            # prefer native builds for these machines:
+            inherit (emulated) servo desko lappy rescue;
+            # prefer cross-compiled builds for these machines:
+            inherit (cross) moby;
+          };
+
       # unofficial output
       # this produces a EFI-bootable .img file (GPT with a /boot partition and a system (/ or /nix) partition).
       # after building this:
@@ -109,27 +164,38 @@
       #   - if fs wasn't resized automatically, then `sudo btrfs filesystem resize max /`
       #   - checkout this flake into /etc/nixos AND UPDATE THE FS UUIDS.
       #   - `nixos-rebuild --flake './#<host>' switch`
-      imgs = builtins.mapAttrs (_: host-dfn: host-dfn.config.system.build.img) self.nixosConfigurations;
+      imgs = mapAttrValues (host: host.config.system.build.img) self.nixosConfigurations;
 
-      host-pkgs = builtins.mapAttrs (_: host-dfn: host-dfn.config.system.build.pkgs) self.nixosConfigurations;
+      # unofficial output
+      host-pkgs = mapAttrValues (host: host.config.system.build.pkgs) self.nixosConfigurations;
+      host-programs = mapAttrValues (host: mapAttrValues (p: p.package) host.config.sane.programs) self.nixosConfigurations;
 
-      overlays = rec {
-        default = pkgs;
-        pkgs = import ./overlays/pkgs.nix;
-        pins = import ./overlays/pins.nix;  # TODO: move to `nixpatches/` input
-        passthru =
+      overlays = {
+        # N.B.: `nix flake check` requires every overlay to take `final: prev:` at defn site,
+        #   hence the weird redundancy.
+        default = final: prev: self.overlays.pkgs final prev;
+        sane-all = final: prev: import ./overlays/all.nix final prev;
+        disable-flakey-tests = final: prev: import ./overlays/disable-flakey-tests.nix final prev;
+        pkgs = final: prev: import ./overlays/pkgs.nix final prev;
+        pins = final: prev: import ./overlays/pins.nix final prev;
+        preferences = final: prev: import ./overlays/preferences.nix final prev;
+        optimizations = final: prev: import ./overlays/optimizations.nix final prev;
+        passthru = final: prev:
           let
-            stable =
-              if inputs ? "nixpkgs-stable" then (
-                next: prev: {
-                  stable = inputs.nixpkgs-stable.legacyPackages."${prev.stdenv.hostPlatform.system}";
-                }
-              ) else (next: prev: {});
             mobile = (import "${mobile-nixos}/overlay/overlay.nix");
             uninsane = uninsane-dot-org.overlay;
+            # nix-serve' = nix-serve.overlay;
+            nix-serve' = next: prev: {
+              # XXX(2023/03/02): upstream isn't compatible with modern `nix`. probably the perl bindings.
+              # - we use the package built against `nixpkgs` specified in its flake rather than use its overlay,
+              #   to get around this.
+              inherit (nix-serve.packages."${next.system}") nix-serve;
+            };
           in
-            next: prev:
-              (stable next prev) // (mobile next prev) // (uninsane next prev);
+            (mobile final prev)
+            // (uninsane final prev)
+            // (nix-serve' final prev)
+          ;
       };
 
       nixosModules = rec {
@@ -153,36 +219,162 @@
           aarch64-linux = allPkgsFor "aarch64-linux";
         };
 
-      # extract only our own packages from the full set
-      packages = builtins.mapAttrs
-        (_: full: full.sane // { inherit (full) sane uninsane-dot-org; })
-        self.legacyPackages;
+      # extract only our own packages from the full set.
+      # because of `nix flake check`, we flatten the package set and only surface x86_64-linux packages.
+      packages = mapAttrs
+        (system: allPkgs:
+          allPkgs.lib.filterAttrs (name: pkg:
+            # keep only packages which will pass `nix flake check`, i.e. keep only:
+            # - derivations (not package sets)
+            # - packages that build for the given platform
+            (! elem name [ "feeds" "pythonPackagesExtensions" ])
+            && (allPkgs.lib.meta.availableOn allPkgs.stdenv.hostPlatform pkg)
+          )
+          (
+            # expose sane packages and chosen inputs (uninsane.org)
+            (import ./pkgs { pkgs = allPkgs; }) // {
+              inherit (allPkgs) uninsane-dot-org;
+            }
+          )
+        )
+        # self.legacyPackages;
+        { inherit (self.legacyPackages) x86_64-linux; }
+      ;
 
       apps."x86_64-linux" =
         let
           pkgs = self.legacyPackages."x86_64-linux";
+          deployScript = host: addr: action: pkgs.writeShellScript "deploy-${host}" ''
+            nix build '.#nixosConfigurations.${host}.config.system.build.toplevel' --out-link ./result-${host} $@
+            sudo nix sign-paths -r -k /run/secrets/nix_serve_privkey $(readlink ./result-${host})
+
+            # XXX: this triggers another config eval & (potentially) build.
+            # if the config changed between these invocations, the above signatures might not apply to the deployed config.
+            # let the user handle that edge case by re-running this whole command
+            nixos-rebuild --flake '.#${host}' ${action} --target-host colin@${addr} --use-remote-sudo $@
+          '';
         in {
+          help = {
+            type = "app";
+            program = let
+              helpMsg = builtins.toFile "nixos-config-help-message" ''
+                commands:
+                - `nix run '.#help'`
+                  - show this message
+                - `nix run '.#update-feeds'`
+                  - updates metadata for all feeds
+                - `nix run '.#init-feed' <url>`
+                - `nix run '.#deploy-{lappy,moby,moby-test,servo}' [nixos-rebuild args ...]`
+                - `nix run '.#check-nur'`
+              '';
+            in builtins.toString (pkgs.writeShellScript "nixos-config-help" ''
+              cat ${helpMsg}
+            '');
+          };
           update-feeds = {
             type = "app";
-            program = "${pkgs.feeds.passthru.updateScript}";
+            program = "${pkgs.feeds.updateScript}";
           };
 
           init-feed = {
-            # use like `nix run '.#init-feed' uninsane.org`
             type = "app";
-            program = "${pkgs.feeds.passthru.initFeedScript}";
+            program = "${pkgs.feeds.initFeedScript}";
+          };
+
+          deploy-lappy = {
+            type = "app";
+            program = ''${deployScript "lappy" "lappy" "switch"}'';
+          };
+          deploy-moby-test = {
+            type = "app";
+            program = ''${deployScript "moby" "moby-hn" "test"}'';
+          };
+          deploy-moby = {
+            type = "app";
+            program = ''${deployScript "moby" "moby-hn" "switch"}'';
+          };
+          deploy-servo = {
+            type = "app";
+            program = ''${deployScript "servo" "servo" "switch"}'';
+          };
+
+          sync-moby = {
+            # copy music from the current device to moby
+            # TODO: should i actually sync from /mnt/servo-media/Music instead of the local drive?
+            type = "app";
+            program = builtins.toString (pkgs.writeShellScript "sync-to-moby" ''
+              sudo mount /mnt/moby-home
+              ${pkgs.sane-scripts.sync-music}/bin/sane-sync-music ~/Music /mnt/moby-home/Music
+            '');
+          };
+
+          sync-lappy = {
+            # copy music from servo to lappy
+            # can run this from any device that has ssh access to lappy
+            type = "app";
+            program = builtins.toString (pkgs.writeShellScript "sync-to-lappy" ''
+              sudo mount /mnt/lappy-home
+              ${pkgs.sane-scripts.sync-music}/bin/sane-sync-music /mnt/servo-media/Music /mnt/lappy-home/Music
+            '');
+          };
+
+          check-nur = {
+            # `nix run '.#check-nur'`
+            # validates that my repo can be included in the Nix User Repository
+            type = "app";
+            program = builtins.toString (pkgs.writeShellScript "check-nur" ''
+              cd ${./.}/integrations/nur
+              NIX_PATH= NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1 nix-env -f . -qa \* --meta --xml \
+                --allowed-uris https://static.rust-lang.org \
+                --option restrict-eval true \
+                --option allow-import-from-derivation true \
+                --drv-path --show-trace \
+                -I nixpkgs=$(nix-instantiate --find-file nixpkgs) \
+                -I ../../
+            '');
+          };
+
+          check-host-configs = {
+            type = "app";
+            program = builtins.toString (pkgs.writeShellScript
+              "check-host-configs"
+              (builtins.concatStringsSep "\n" (builtins.map
+                (host: "nix build '.#nixosConfigurations.${host}.config.system.build.toplevel' --out-link ./result-${host} -j1 $@ &")
+                [ "desko" "lappy" "servo" "moby" "rescue" ]
+                # not part of the `map`. wait for all builds to complete
+                ++ [ "wait" ]
+              ))
+            );
           };
         };
 
       templates = {
-        python-data = {
+        env.python-data = {
           # initialize with:
-          # - `nix flake init -t '/home/colin/dev/nixos/#python-data'`
+          # - `nix flake init -t '/home/colin/dev/nixos/#env.python-data'`
           # then enter with:
           # - `nix develop`
-          path = ./templates/python-data;
+          path = ./templates/env/python-data;
           description = "python environment for data processing";
         };
+        pkgs.rust-inline = {
+          # initialize with:
+          # - `nix flake init -t '/home/colin/dev/nixos/#pkgs.rust-inline'`
+          path = ./templates/pkgs/rust-inline;
+          description = "rust package and development environment (inline rust sources)";
+        };
+        pkgs.rust = {
+          # initialize with:
+          # - `nix flake init -t '/home/colin/dev/nixos/#pkgs.rust'`
+          path = ./templates/pkgs/rust;
+          description = "rust package fit to ship in nixpkgs";
+        };
+        pkgs.make = {
+          # initialize with:
+          # - `nix flake init -t '/home/colin/dev/nixos/#pkgs.make'`
+          path = ./templates/pkgs/make;
+          description = "default Makefile-based derivation";
+        };
       };
     };
 }

hosts/README.md

@@ -0,0 +1,7 @@
+## directory structure
+- by-name/<hostname>: configuration which is evaluated _only_ for the given hostname
+- common/: configuration which applies to all hosts
+- modules/: nixpkgs-style modules which may be used by multiple hosts, but configured separately per host.
+    - ideally no module here has effect unless `enable`d
+        - however, `enable` may default to true
+        - and in practice some of these modules surely aren't fully "disableable"

hosts/by-name/desko/default.nix

@@ -4,15 +4,29 @@
     ./fs.nix
   ];
 
+  # sane.guest.enable = true;
+
+  # services.distccd.enable = true;
+  # sane.programs.distcc.enableFor.user.guest = true;
+
+  sops.secrets.colin-passwd.neededForUsers = true;
+
+  sane.roles.build-machine.enable = true;
+  sane.roles.ac = true;
   sane.roles.client = true;
+  sane.roles.dev-machine = true;
   sane.services.wg-home.enable = true;
   sane.services.wg-home.ip = config.sane.hosts.by-name."desko".wg-home.ip;
   sane.services.duplicity.enable = true;
-  sane.services.nixserve.enable = true;
-  sane.services.nixserve.sopsFile = ../../../secrets/desko.yaml;
-  sane.persist.enable = true;
+  sane.services.nixserve.secretKeyFile = config.sops.secrets.nix_serve_privkey.path;
 
   sane.gui.sway.enable = true;
+  sane.programs.iphoneUtils.enableFor.user.colin = true;
+  sane.programs.steam.enableFor.user.colin = true;
+
+  sane.programs.guiApps.suggestedPrograms = [ "desktopGuiApps" ];
+  sane.programs.consoleUtils.suggestedPrograms = [ "consoleMediaUtils" "desktopConsoleUtils" ];
+  # sane.programs.devPkgs.enableFor.user.colin = true;
 
   boot.loader.efi.canTouchEfiVariables = false;
   sane.image.extraBootFiles = [ pkgs.bootpart-uefi-x86_64 ];
@@ -20,13 +34,9 @@
   # needed to use libimobiledevice/ifuse, for iphone sync
   services.usbmuxd.enable = true;
 
-  sops.secrets.colin-passwd = {
-    sopsFile = ../../../secrets/desko.yaml;
-    neededForUsers = true;
-  };
-
   # don't enable wifi by default: it messes with connectivity.
   systemd.services.iwd.enable = false;
+  systemd.services.wpa_supplicant.enable = false;
 
   # default config: https://man.archlinux.org/man/snapper-configs.5
   # defaults to something like:
@@ -35,28 +45,11 @@
   services.snapper.configs.nix = {
     # TODO: for the impermanent setup, we'd prefer to just do /nix/persist,
     # but that also requires setting up the persist dir as a subvol
-    subvolume = "/nix";
+    SUBVOLUME = "/nix";
     # TODO: ALLOW_USERS doesn't seem to work. still need `sudo snapper -c nix list`
-    extraConfig = ''
-      ALLOW_USERS = "colin";
-    '';
+    ALLOW_USERS = [ "colin" ];
   };
 
-  sops.secrets.duplicity_passphrase = {
-    sopsFile = ../../../secrets/desko.yaml;
-  };
-
-  programs.steam = {
-    enable = true;
-    # not sure if needed: stole this whole snippet from the wiki
-    remotePlay.openFirewall = true; # Open ports in the firewall for Steam Remote Play
-    dedicatedServer.openFirewall = true; # Open ports in the firewall for Source Dedicated Server
-  };
-  sane.user.persist.plaintext = [
-    ".steam"
-    ".local/share/Steam"
-  ];
-
   # docs: https://nixos.org/manual/nixos/stable/options.html#opt-system.stateVersion
   system.stateVersion = "21.05";
 }

hosts/by-name/desko/fs.nix

@@ -2,17 +2,10 @@
 
 {
   sane.persist.root-on-tmpfs = true;
-  # we need a /tmp for building large nix things.
+  # increase /tmp space (defaults to 50% of RAM) for building large nix things.
   # a cross-compiled kernel, particularly, will easily use 30+GB of tmp
-  fileSystems."/tmp" = {
-    device = "none";
-    fsType = "tmpfs";
-    options = [
-      "mode=777"
-      "size=64G"
-      "defaults"
-    ];
-  };
+  fileSystems."/tmp".options = [ "size=64G" ];
+
   fileSystems."/nix" = {
     # device = "/dev/disk/by-uuid/985a0a32-da52-4043-9df7-615adec2e4ff";
     device = "/dev/disk/by-uuid/0ab0770b-7734-4167-88d9-6e4e20bb2a56";

hosts/by-name/lappy/default.nix

@@ -2,23 +2,26 @@
 {
   imports = [
     ./fs.nix
+    ./polyfill.nix
   ];
 
   sane.roles.client = true;
+  sane.roles.dev-machine = true;
   sane.services.wg-home.enable = true;
   sane.services.wg-home.ip = config.sane.hosts.by-name."lappy".wg-home.ip;
 
   # sane.guest.enable = true;
-  sane.gui.sway.enable = true;
-  sane.persist.enable = true;
-  sane.nixcache.enable = true;
+  sane.gui.sxmo.enable = true;
   boot.loader.efi.canTouchEfiVariables = false;
   sane.image.extraBootFiles = [ pkgs.bootpart-uefi-x86_64 ];
 
-  sops.secrets.colin-passwd = {
-    sopsFile = ../../../secrets/lappy.yaml;
-    neededForUsers = true;
-  };
+  sane.programs.guiApps.suggestedPrograms = [
+    "desktopGuiApps"
+    "stepmania"
+  ];
+  sane.programs.consoleUtils.suggestedPrograms = [ "consoleMediaUtils" "desktopConsoleUtils" ];
+
+  sops.secrets.colin-passwd.neededForUsers = true;
 
   # default config: https://man.archlinux.org/man/snapper-configs.5
   # defaults to something like:
@@ -27,12 +30,10 @@
   services.snapper.configs.nix = {
     # TODO: for the impermanent setup, we'd prefer to just do /nix/persist,
     # but that also requires setting up the persist dir as a subvol
-    subvolume = "/nix";
+    SUBVOLUME = "/nix";
+    ALLOW_USERS = [ "colin" ];
   };
 
-  # TODO: only here for debugging
-  # services.ipfs.enable = true;
-
   # docs: https://nixos.org/manual/nixos/stable/options.html#opt-system.stateVersion
   system.stateVersion = "21.05";
 }

hosts/by-name/lappy/fs.nix

@@ -2,15 +2,6 @@
 
 {
   sane.persist.root-on-tmpfs = true;
-  # we need a /tmp of default size (half RAM) for building large nix things
-  fileSystems."/tmp" = {
-    device = "none";
-    fsType = "tmpfs";
-    options = [
-      "mode=777"
-      "defaults"
-    ];
-  };
 
   fileSystems."/nix" = {
     device = "/dev/disk/by-uuid/75230e56-2c69-4e41-b03e-68475f119980";

hosts/by-name/lappy/polyfill.nix

@@ -0,0 +1,41 @@
+# doesn't actually *enable* anything,
+# but sets up any modules such that if they *were* enabled, they'll act as expected.
+{ pkgs, ... }:
+{
+  sane.gui.sxmo = {
+    greeter = "greetd-sway-gtkgreet";
+    settings = {
+      # XXX: make sure the user is part of the `input` group!
+      SXMO_LISGD_INPUT_DEVICE = "/dev/input/by-id/usb-Wacom_Co._Ltd._Pen_and_multitouch_sensor-event-if00";
+      # these identifiers are from `swaymsg -t get_inputs`
+      SXMO_VOLUME_BUTTON = "1:1:AT_Translated_Set_2_keyboard";
+      # SXMO_VOLUME_BUTTON = "none";
+      SXMO_POWER_BUTTON = "0:1:Power_Button";
+      # SXMO_POWER_BUTTON = "none";
+      SXMO_DISABLE_LEDS = "1";
+      SXMO_UNLOCK_IDLE_TIME = "120";  # default
+      # sxmo tries to determine device type from /proc/device-tree/compatible,
+      # but that doesn't seem to exist on NixOS?  (or maybe it just doesn't exist
+      # on non-aarch64 builds).
+      # the device type informs (at least):
+      # - SXMO_WIFI_MODULE
+      # - SXMO_RTW_SCAN_INTERVAL
+      # - SXMO_SYS_FILES
+      # - SXMO_TOUCHSCREEN_ID
+      # - SXMO_MONITOR
+      # - SXMO_ALSA_CONTROL_NAME
+      # - SXMO_SWAY_SCALE
+      # see <repo:mil/sxmo-utils:scripts/deviceprofiles>
+      # SXMO_DEVICE_NAME = "pine64,pinephone-1.2";
+      # if sxmo doesn't know the device, it can't decide whether to use one_button or three_button mode
+      #   and so it just wouldn't handle any button inputs (sxmo_hook_inputhandler.sh not on path)
+      SXMO_DEVICE_NAME = "three_button_touchscreen";
+    };
+    package = pkgs.sxmo-utils-latest.overrideAttrs (base: {
+      postPatch = (base.postPatch or "") + ''
+        # after volume-button navigation mode, restore full keyboard functionality
+        cp ${./xkb_mobile_normal_buttons} ./configs/xkb/xkb_mobile_normal_buttons
+      '';
+    });
+  };
+}

hosts/by-name/lappy/xkb_mobile_normal_buttons

@@ -0,0 +1,7 @@
+xkb_keymap {
+	xkb_keycodes  { include "evdev+aliases(qwerty)"	};
+	xkb_types     { include "complete"	};
+	xkb_compat    { include "complete"	};
+	xkb_symbols   { include "pc+us+inet(evdev)"	};
+	xkb_geometry  { include "pc(pc105)"	};
+};

hosts/by-name/moby/default.nix

@@ -1,54 +1,62 @@
+# Pinephone
+# other setups to reference:
+# - <https://hamblingreen.gitlab.io/2022/03/02/my-pinephone-setup.html>
+#   - sxmo Arch user. lots of app recommendations
+#
+# wikis, resources, ...:
+# - Linux Phone Apps: <https://linuxphoneapps.org/>
+#   - massive mobile-friendly app database
+# - Mobian wiki: <https://wiki.mobian-project.org/doku.php?id=start>
+#   - recommended apps, chatrooms
+
 { config, pkgs, lib, ... }:
 {
   imports = [
-    ./firmware.nix
+    ./bootloader.nix
     ./fs.nix
+    ./gps.nix
     ./kernel.nix
+    ./polyfill.nix
   ];
 
   sane.roles.client = true;
+  sane.zsh.showDeadlines = false;  # unlikely to act on them when in shell
   sane.services.wg-home.enable = true;
   sane.services.wg-home.ip = config.sane.hosts.by-name."moby".wg-home.ip;
 
-  # cross-compiled documentation is *slow*.
-  # no obvious way to natively compile docs (2022/09/29).
-  # entrypoint is nixos/modules/misc/documentation.nix
-  # doc building happens in nixos/doc/manual/default.nix
-  # TODO: we could *maybe* inject pkgs.buildPackages.xyz = cross.buildPackages.xyz?
-  documentation.nixos.enable = false;
-
   # XXX colin: phosh doesn't work well with passwordless login,
   # so set this more reliable default password should anything go wrong
   users.users.colin.initialPassword = "147147";
   services.getty.autologinUser = "root";  # allows for emergency maintenance?
 
-  sops.secrets.colin-passwd = {
-    sopsFile = ../../../secrets/moby.yaml;
-    neededForUsers = true;
-  };
-
-  sane.web-browser = {
-    # compromise impermanence for the sake of usability
-    persistCache = "private";
-    persistData = "private";
-
-    # i don't do crypto stuff on moby
-    addons.ether-metamask.enable = false;
-    # addons.sideberry.enable = false;
-  };
+  sops.secrets.colin-passwd.neededForUsers = true;
 
   sane.user.persist.plaintext = [
+    # TODO: make this just generally conditional upon pulse being enabled?
     ".config/pulse"  # persist pulseaudio volume
   ];
 
-  sane.nixcache.enable = true;
-  sane.persist.enable = true;
-  sane.gui.phosh.enable = true;
+  sane.gui.sxmo.enable = true;
+  sane.programs.guiApps.suggestedPrograms = [ "handheldGuiApps" ];
+  # sane.programs.consoleUtils.enableFor.user.colin = false;
+  # sane.programs.guiApps.enableFor.user.colin = false;
+  sane.programs.sequoia.enableFor.user.colin = false;
+  sane.programs.tuiApps.enableFor.user.colin = false;  # visidata, others, don't compile well
+  # disabled for faster deploys
+  sane.programs.soundconverter.enableFor.user.colin = false;
+  sane.programs.eg25-control.enableFor.user.colin = true;
+
+  # sane.programs.firefox.mime.priority = 300;  # prefer other browsers when possible
+  # HACK/TODO: make `programs.P.env.VAR` behave according to `mime.priority`
+  # sane.programs.firefox.env = lib.mkForce {};
+  # sane.programs.epiphany.env.BROWSER = "epiphany";
+  # sane.programs.firefox.enableFor.user.colin = false;  # use epiphany instead
+
+  # sane.programs.mpv.enableFor.user.colin = true;
 
   boot.loader.efi.canTouchEfiVariables = false;
   # /boot space is at a premium. default was 20.
   # even 10 can be too much
-  # TODO: compress moby kernels!
   boot.loader.generic-extlinux-compatible.configurationLimit = 8;
   # mobile.bootloader.enable = false;
   # mobile.boot.stage-1.enable = false;
@@ -83,14 +91,50 @@
   # enable rotation sensor
   hardware.sensor.iio.enable = true;
 
-  # from https://gitlab.manjaro.org/manjaro-arm/packages/community/phosh/alsa-ucm-pinephone
-  # mobile-nixos does this same thing, with *slightly different settings*.
-  # i trust manjaro more because the guy maintaining that is actively trying to upstream into alsa-ucm-conf.
-  # an alternative may be to build a custom alsa with the PinePhone config patch applied:
-  # - <https://github.com/alsa-project/alsa-ucm-conf/pull/134>
-  # that would make this be not device-specific
-  environment.variables.ALSA_CONFIG_UCM2 = "${./ucm2}";
-  systemd.services.pulseaudio.environment.ALSA_CONFIG_UCM2 = "${./ucm2}";
+  # inject specialized alsa configs via the environment.
+  # specifically, this gets the pinephone headphones & internal earpiece working.
+  # see pkgs/patched/alsa-ucm-conf for more info.
+  environment.variables.ALSA_CONFIG_UCM2 = "/run/current-system/sw/share/alsa/ucm2";
+  environment.pathsToLink = [ "/share/alsa/ucm2" ];
+  environment.systemPackages = [ pkgs.alsa-ucm-conf-sane ];
+  systemd =
+    let ucm-env = config.environment.variables.ALSA_CONFIG_UCM2;
+    in {
+    # cribbed from <repo:nixos/mobile-nixos:modules/quirks/audio.nix>
+
+    # pulseaudio
+    user.services.pulseaudio.environment.ALSA_CONFIG_UCM2 = ucm-env;
+    services.pulseaudio.environment.ALSA_CONFIG_UCM2      = ucm-env;
+
+    # pipewire
+    user.services.pipewire.environment.ALSA_CONFIG_UCM2       = ucm-env;
+    user.services.pipewire-pulse.environment.ALSA_CONFIG_UCM2 = ucm-env;
+    user.services.wireplumber.environment.ALSA_CONFIG_UCM2    = ucm-env;
+    services.pipewire.environment.ALSA_CONFIG_UCM2            = ucm-env;
+    services.pipewire-pulse.environment.ALSA_CONFIG_UCM2      = ucm-env;
+    services.wireplumber.environment.ALSA_CONFIG_UCM2         = ucm-env;
+
+
+    # TODO: move elsewhere...
+    services.ModemManager.serviceConfig = {
+      # N.B.: the extra "" in ExecStart serves to force upstream ExecStart to be ignored
+      ExecStart = [ "" "${pkgs.modemmanager}/bin/ModemManager --debug" ];
+      # --debug sets DEBUG level logging: so reset
+      ExecStartPost = [ "${pkgs.modemmanager}/bin/mmcli --set-logging=INFO" ];
+    };
+  };
+
+  services.udev.extraRules = let
+    chmod = "${pkgs.coreutils}/bin/chmod";
+    chown = "${pkgs.coreutils}/bin/chown";
+  in ''
+    # make Pinephone flashlight writable by user.
+    # taken from postmarketOS: <repo:postmarketOS/pmaports:device/main/device-pine64-pinephone/60-flashlight.rules>
+    SUBSYSTEM=="leds", DEVPATH=="*/*:flash", RUN+="${chmod} g+w /sys%p/brightness /sys%p/flash_strobe", RUN+="${chown} :video /sys%p/brightness /sys%p/flash_strobe"
+
+    # make Pinephone front LEDs writable by user.
+    SUBSYSTEM=="leds", DEVPATH=="*/*:indicator", RUN+="${chmod} g+w /sys%p/brightness", RUN+="${chown} :video /sys%p/brightness"
+  '';
 
   hardware.opengl.driSupport = true;
 }

hosts/by-name/moby/gps.nix

@@ -0,0 +1,42 @@
+# pinephone GPS happens in EG25 modem
+# serial control interface to modem is /dev/ttyUSB2
+# after enabling GPS, readout is /dev/ttyUSB1
+#
+# minimal process to enable modem and GPS:
+# - `echo 1 > /sys/class/modem-power/modem-power/device/powered`
+# - `screen /dev/ttyUSB2 115200`
+#   - `AT+QGPSCFG="nmeasrc",1`
+#   - `AT+QGPS=1`
+#
+# now, something like `gpsd` can directly read from /dev/ttyUSB1.
+#
+# initial GPS fix can take 15+ minutes.
+# meanwhile, services like eg25-manager can speed this up by uploading assisted GPS data to the modem.
+#
+# geoclue somehow fits in here as a geospatial provider that leverages GPS and also other sources like radio towers
+
+{ lib, ... }:
+{
+  # test gpsd with `gpspipe -w -n 10 2> /dev/null | grep -m 1 TPV | jq '.lat, .lon' | tr '\n' ' '`
+  # ^ should return <lat> <long>
+  services.gpsd.enable = true;
+  services.gpsd.devices = [ "/dev/ttyUSB1" ];
+
+  # test geoclue2 by building `geoclue2-with-demo-agent`
+  # and running "${geoclue2-with-demo-agent}/libexec/geoclue-2/demos/where-am-i"
+  services.geoclue2.enable = true;
+  services.geoclue2.appConfig.where-am-i = {
+    # this is the default "agent", shipped by geoclue package: allow it to use location
+    isAllowed = true;
+    isSystem = false;
+    # XXX: setting users != [] might be causing `where-am-i` to time out
+    # users = [
+    #   # restrict to only one set of users. empty array (default) means "allow any user to access geolocation".
+    #   (builtins.toString config.users.users.colin.uid)
+    # ];
+  };
+  systemd.services.geoclue.after = lib.mkForce [];  #< defaults to network-online, but not all my sources require network
+
+
+  sane.services.eg25-control.enable = true;
+}

hosts/by-name/moby/kernel.nix

@@ -1,146 +1,71 @@
-{ lib, pkgs, ... }:
+{ pkgs, ... }:
 let
-  # use the last commit on the 5.18 branch (5.18.14)
-  # manjaro's changes between kernel patch versions tend to be minimal if any.
-  manjaroBase = "https://gitlab.manjaro.org/manjaro-arm/packages/core/linux/-/raw/25bd828cd47b1c6e09fcbcf394a649b89d2876dd";
-  manjaroPatch = name: sha256: {
-    inherit name;
-    patch = pkgs.fetchpatch {
-      inherit name;
-      url = "${manjaroBase}/${name}?inline=false";
-      inherit sha256;
-    };
-  };
+  dmesg = "${pkgs.util-linux}/bin/dmesg";
+  grep = "${pkgs.gnugrep}/bin/grep";
+  modprobe = "${pkgs.kmod}/bin/modprobe";
+  ensureHWReady = ''
+    # common boot failure:
+    # blank screen (no backlight even), with the following log:
+    # ```syslog
+    # sun8i-dw-hdmi 1ee0000.hdmi: Couldn't get the HDMI PHY
+    # ...
+    # sun4i-drm display-engine: Couldn't bind all pipelines components
+    # ...
+    # sun8i-dw-hdmi: probe of 1ee0000.hdmi failed with error -17
+    # ```
+    #
+    # in particular, that `probe ... failed` occurs *only* on failed boots
+    # (the other messages might sometimes occur even on successful runs?)
+    #
+    # reloading the sun8i hdmi driver usually gets the screen on, showing boot text.
+    # then restarting display-manager.service gets us to the login.
+    #
+    # NB: the above log is default level. though less specific, there's a `err` level message that also signals this:
+    # sun4i-drm display-engine: failed to bind 1ee0000.hdmi (ops sun8i_dw_hdmi_ops [sun8i_drm_hdmi]): -17
+    # NB: this is the most common, but not the only, failure mode for `display-manager`.
+    # another error seems characterized by these dmesg logs, in which reprobing sun8i_drm_hdmi does not fix:
+    # ```syslog
+    # sun6i-mipi-dsi 1ca0000.dsi: Couldn't get the MIPI D-PHY
+    # sun4i-drm display-engine: Couldn't bind all pipelines components
+    # sun6i-mipi-dsi 1ca0000.dsi: Couldn't register our component
+    # ```
 
-  # the idea for patching off Manjaro's kernel comes from jakewaksbaum:
-  # - https://git.sr.ht/~jakewaksbaum/pi/tree/af20aae5653545d6e67a459b59ee3e1ca8a680b0/item/kernel/default.nix
-  # - he later abandoned this, i think because he's using the Pinephone Pro which received mainline support.
-  manjaroPatches = [
-    (manjaroPatch
-      "1001-arm64-dts-allwinner-add-hdmi-sound-to-pine-devices.patch"
-      "sha256-DApd791A+AxB28Ven/MVAyuyVphdo8KQDx8O7oxVPnc="
-    )
-    # these patches below are critical to enable wifi (RTL8723CS)
-    # - the alternative is a wholly forked kernel by megi/megous:
-    #   - https://xnux.eu/howtos/build-pinephone-kernel.html#toc-how-to-build-megi-s-pinehpone-kernel
-    # - i don't know if these patches are based on megi's or original
-    (manjaroPatch
-      "2001-Bluetooth-Add-new-quirk-for-broken-local-ext-features.patch"
-      "sha256-CExhJuUWivegxPdnzKINEsKrMFx/m/1kOZFmlZ2SEOc="
-    )
-    (manjaroPatch
-      "2002-Bluetooth-btrtl-add-support-for-the-RTL8723CS.patch"
-      "sha256-dDdvOphTcP/Aog93HyH+L9m55laTgtjndPSE4/rnzUA="
-    )
-    (manjaroPatch
-      "2004-arm64-dts-allwinner-enable-bluetooth-pinetab-pinepho.patch"
-      "sha256-o43P3WzXyHK1PF+Kdter4asuyGAEKO6wf5ixcco2kCQ="
-    )
-    # XXX: this one has a Makefile, which hardcodes /sbin/depmod:
-    # - drivers/staging/rtl8723cs/Makefile
-    # - not sure if this is problematic?
-    (manjaroPatch
-      "2005-staging-add-rtl8723cs-driver.patch"
-      "sha256-6ywm3dQQ5JYl60CLKarxlSUukwi4QzqctCj3tVgzFbo="
-    )
-  ];
-
-  # pinephone uses the linux dtb at arch/arm64/boot/dts/allwinner/sun50i-a64-pinephone.dtsi
-  # - this includes sun50i-a64.dtsi
-  # - and sun50i-a64-cpu-opp.dtsi
-  # - no need to touch the allwinner-h6 stuff: that's the SBC pine product
-  # - i think it's safe to ignore sun9i stuff, but i don't know what it is
-  kernelConfig = with lib.kernel; {
-    # NB: nix adds the CONFIG_ prefix to each of these.
-    # if you add the prefix yourself nix will IGNORE YOUR CONFIG.
-    RTL8723CS = module;
-    BT_HCIUART_3WIRE = yes;
-    BT_HCIUART_RTL = yes;
-    RTL8XXXU_UNTESTED = yes;
-    BT_BNEP_MC_FILTER = yes;
-    BT_BNEP_PROTO_FILTER = yes;
-    BT_HS = yes;
-    BT_LE = yes;
-    # relevant configs inherited from nixos defaults (or above additions):
-    # CONFIG_BT=m
-    # CONFIG_BT_BREDR=y
-    # CONFIG_BT_RFCOMM=m
-    # CONFIG_BT_RFCOMM_TTY=y
-    # CONFIG_BT_BNEP=m
-    # CONFIG_BT_HIDP=m
-    # CONFIG_BT_RTL=m
-    # CONFIG_BT_HCIBTUSB=m
-    # CONFIG_BT_HCIBTUSB_BCM=y
-    # CONFIG_BT_HCIBTUSB_RTL=y
-    # CONFIG_BT_HCIUART=m
-    # CONFIG_BT_HCIUART_SERDEV=y
-    # CONFIG_BT_HCIUART_H4=y
-    # CONFIG_BT_HCIUART_LL=y
-    # CONFIG_RTL_CARDS=m
-    # CONFIG_RTLWIFI=m
-    # CONFIG_RTLWIFI_PCI=m
-    # CONFIG_RTLWIFI_USB=m
-    # CONFIG_RTLWIFI_DEBUG=y
-    # CONFIG_RTL8723_COMMON=m
-    # CONFIG_RTLBTCOEXIST=m
-    # CONFIG_RTL8XXXU=m
-    # CONFIG_RTLLIB=m
-    # consider adding (from mobile-nixos):
-    # maybe: CONFIG_BT_HCIUART_3WIRE=y
-    # maybe: CONFIG_BT_HCIUART_RTL=y
-    # maybe: CONFIG_RTL8XXXU_UNTESTED=y
-    # consider adding (from manjaro):
-    # CONFIG_BT_6LOWPAN=m  (not listed as option in nixos kernel)
-    # these are referenced in the rtl8723 source, but not known to config (and not in mobile-nixos config
-    # maybe: CONFIG_RTL_ODM_WLAN_DRIVER
-    # maybe: CONFIG_RTL_TRIBAND_SUPPORT
-    # maybe: CONFIG_SDIO_HCI
-    # maybe: CONFIG_USB_HCI
-  };
-
-  # create a kernelPatch which overrides nixos' defconfig with extra options
-  patchDefconfig = config: {
-    # defconfig options. this method comes from here:
-    # - https://discourse.nixos.org/t/the-correct-way-to-override-the-latest-kernel-config/533/9
-    name = "sane-moby-defconfig";
-    patch = null;
-    extraStructuredConfig = config;
-  };
+    if (${dmesg} --kernel --level err --color=never --notime | ${grep} -q 'sun4i-drm display-engine: failed to bind 1ee0000.hdmi')
+    then
+      echo "reprobing sun8i_drm_hdmi"
+      # if a command here fails it errors the whole service, so prefer to log instead
+      ${modprobe} -r sun8i_drm_hdmi || echo "failed to unload sun8i_drm_hdmi"
+      ${modprobe} sun8i_drm_hdmi || echo "failed to load sub8i_drm_hdmi"
+    fi
+  '';
 in
 {
-  # use Megi's kernel:
-  # even with the Manjaro patches, stock 5.18 has a few issues on Pinephone:
-  # - no battery charging
-  # - phone rotation sensor is off by 90 degrees
-  # - ambient light sensor causes screen brightness to be shakey
-  # - phosh greeter may not appear after wake from sleep
-  boot.kernelPackages = pkgs.cross.linuxPackagesFor pkgs.cross.linux-megous;
-
-  boot.kernelPatches = [
-    (patchDefconfig (kernelConfig //
-      (with lib.kernel; {
-        # disabling the sun5i_eink driver avoids this compilation error:
-        # CC [M]  drivers/video/fbdev/sun5i-eink-neon.o
-        # aarch64-unknown-linux-gnu-gcc: error: unrecognized command line option '-mfloat-abi=softfp'
-        # aarch64-unknown-linux-gnu-gcc: error: unrecognized command line option '-mfpu=neon'
-        # make[3]: *** [../scripts/Makefile.build:289: drivers/video/fbdev/sun5i-eink-neon.o] Error 1
-        FB_SUN5I_EINK = no;
-        # used by the pinephone pro, but fails to compile with:
-        # ../drivers/media/i2c/ov8858.c:1834:27: error: implicit declaration of function 'compat_ptr'
-        VIDEO_OV8858 = no;
-      })
-    ))
-  ];
-
-  # alternatively, use nixos' kernel and add the stuff we want:
-  # # cross-compilation optimization:
-  # boot.kernelPackages =
-  #   let p = (import nixpkgs { localSystem = "x86_64-linux"; });
-  #   in p.pkgsCross.aarch64-multiplatform.linuxPackages_5_18;
-  # # non-cross:
-  # # boot.kernelPackages = pkgs.linuxPackages_5_18;
+  boot.kernelPackages = pkgs.linuxPackagesFor pkgs.linux-megous;
+  # boot.kernelPackages = pkgs.linuxPackagesFor pkgs.linux-manjaro;
+  # boot.kernelPackages = pkgs.linuxPackagesFor pkgs.linux_latest;
 
+  # alternatively, apply patches directly to stock nixos kernel:
   # boot.kernelPatches = manjaroPatches ++ [
   #   (patchDefconfig kernelConfig)
   # ];
+
+  # configure nixos to build a compressed kernel image, since it doesn't usually do that for aarch64 target.
+  # without this i run out of /boot space in < 10 generations
+  nixpkgs.hostPlatform.linux-kernel = {
+    # defaults:
+    name = "aarch64-multiplatform";
+    baseConfig = "defconfig";
+    DTB = true;
+    autoModules = true;
+    preferBuiltin = true;
+    # extraConfig = ...
+    # ^-- raspberry pi stuff: we don't need it.
+
+    # target = "Image";  # <-- default
+    target = "Image.gz";  # <-- compress the kernel image
+    # target = "zImage";  # <-- confuses other parts of nixos :-(
+  };
+
+  services.xserver.displayManager.job.preStart = ensureHWReady;
+  systemd.services.greetd.preStart = ensureHWReady;
 }

hosts/by-name/moby/polyfill.nix

@@ -0,0 +1,179 @@
+# this file configures preferences per program, without actually enabling any programs.
+# the goal is to separate the place where we decide *what* to use (i.e. `sane.programs.firefox.enable = true` -- at the toplevel)
+# from where we specific how that thing should behave *if* it's in use.
+#
+# NixOS backgrounds:
+# - <https://github.com/NixOS/nixos-artwork>
+#   - <https://github.com/NixOS/nixos-artwork/issues/50>  (colorful; unmerged)
+#   - <https://github.com/NixOS/nixos-artwork/pull/60/files>  (desktop-oriented; clean; unmerged)
+# - <https://itsfoss.com/content/images/2023/04/nixos-tutorials.png>
+
+{ lib, pkgs, sane-lib, ... }:
+let
+  # TODO: generate this from the .svg
+  # bg = ./nixos-bg-02.png;
+  bg = pkgs.runCommand "nixos-bg.png" { nativeBuildInputs = [ pkgs.inkscape ]; } ''
+    inkscape ${./nixos-bg-02.svg} -o $out
+  '';
+in
+{
+  sane.programs.firefox.config = {
+    # compromise impermanence for the sake of usability
+    persistCache = "private";
+    persistData = "private";
+
+    # i don't do crypto stuff on moby
+    addons.ether-metamask.enable = false;
+    # sidebery UX doesn't make sense on small screen
+    addons.sidebery.enable = false;
+  };
+
+  sane.gui.sxmo = {
+    nogesture = true;
+    settings = {
+      ### hardware: touch screen
+      SXMO_LISGD_INPUT_DEVICE = "/dev/input/by-path/platform-1c2ac00.i2c-event";
+      # vol and power are detected correctly by upstream
+
+
+      ### preferences
+      # notable bemenu options:
+      #   - see `bemenu --help` for all
+      #   -P, --prefix          text to show before highlighted item.
+      #   --scrollbar           display scrollbar. (none (default), always, autohide)
+      #   -H, --line-height     defines the height to make each menu line (0 = default height). (wx)
+      #   -M, --margin          defines the empty space on either side of the menu. (wx)
+      #   -W, --width-factor    defines the relative width factor of the menu (from 0 to 1). (wx)
+      #   -B, --border          defines the width of the border in pixels around the menu. (wx)
+      #   -R  --border-radius   defines the radius of the border around the menu (0 = no curved borders).
+      #   --ch                  defines the height of the cursor (0 = scales with line height). (wx)
+      #   --cw                  defines the width of the cursor. (wx)
+      #   --hp                  defines the horizontal padding for the entries in single line mode. (wx)
+      #   --fn                  defines the font to be used ('name [size]'). (wx)
+      #   --tb                  defines the title background color. (wx)
+      #   --tf                  defines the title foreground color. (wx)
+      #   --fb                  defines the filter background color. (wx)
+      #   --ff                  defines the filter foreground color. (wx)
+      #   --nb                  defines the normal background color. (wx)
+      #   --nf                  defines the normal foreground color. (wx)
+      #   --hb                  defines the highlighted background color. (wx)
+      #   --hf                  defines the highlighted foreground color. (wx)
+      #   --fbb                 defines the feedback background color. (wx)
+      #   --fbf                 defines the feedback foreground color. (wx)
+      #   --sb                  defines the selected background color. (wx)
+      #   --sf                  defines the selected foreground color. (wx)
+      #   --ab                  defines the alternating background color. (wx)
+      #   --af                  defines the alternating foreground color. (wx)
+      #   --scb                 defines the scrollbar background color. (wx)
+      #   --scf                 defines the scrollbar foreground color. (wx)
+      #   --bdr                 defines the border color. (wx)
+      #
+      # colors are specified as `#RRGGBB`
+      # defaults:
+      # --ab "#222222"
+      # --af "#bbbbbb"
+      # --bdr "#005577"
+      # --border 3
+      # --cb "#222222"
+      # --center
+      # --cf "#bbbbbb"
+      # --fb "#222222"
+      # --fbb "#eeeeee"
+      # --fbf "#222222"
+      # --ff "#bbbbbb"
+      # --fixed-height
+      # --fn 'Sxmo 14'
+      # --hb "#005577"
+      # --hf "#eeeeee"
+      # --line-height 20
+      # --list 16
+      # --margin 40
+      # --nb "#222222"
+      # --nf "#bbbbbb"
+      # --no-overlap
+      # --no-spacing
+      # --sb "#323232"
+      # --scb "#005577"
+      # --scf "#eeeeee"
+      # --scrollbar autohide
+      # --tb "#005577"
+      # --tf "#eeeeee"
+      # --wrap
+      BEMENU_OPTS = let
+        bg = "#1d1721";       # slight purple
+        fg0 = "#d8d8d8";      # inactive text (light grey)
+        fg1 = "#ffffff";      # active text   (white)
+        accent0 = "#1f5e54";  # darker but saturated teal
+        accent1 = "#418379";  # teal (matches nixos-bg)
+        accent2 = "#5b938a";  # brighter but muted teal
+      in lib.concatStringsSep " " [
+        "--wrap --scrollbar autohide --fixed-height"
+        "--center --margin 45"
+        "--no-spacing"
+        # XXX: font size doesn't seem to take effect (would prefer larger)
+        "--fn 'Sxmo 14' --line-height 22 --border 3"
+        "--bdr '${accent0}'"                     # border
+        "--scf '${accent2}' --scb '${accent0}'"  # scrollbar
+        "--tb '${accent0}' --tf '${fg0}'"        # title
+        "--fb '${accent0}' --ff '${fg1}'"        # filter (i.e. text that's been entered)
+        "--hb '${accent1}' --hf '${fg1}'"        # selected item
+        "--nb '${bg}' --nf '${fg0}'"             # normal lines (even)
+        "--ab '${bg}' --af '${fg0}'"             # alternated lines (odd)
+        "--cf '${accent0}' --cb '${accent0}'"    # cursor (not very useful)
+      ];
+      DEFAULT_COUNTRY = "US";
+
+      # BEMENU lines (wayland DMENU):
+      # - camera is 9th entry
+      # - flashlight is 10th entry
+      # - config is 14th entry. inside that:
+      #   - autorotate is 11th entry
+      #   - system menu is 19th entry
+      #   - close is 20th entry
+      # - power is 15th entry
+      # - close is 16th entry
+      SXMO_BEMENU_LANDSCAPE_LINES = "11";  # default 8
+      SXMO_BEMENU_PORTRAIT_LINES = "16";  # default 16
+      SXMO_BG_IMG = "${bg}";
+      SXMO_LOCK_IDLE_TIME = "15";  # how long between screenoff -> lock -> back to screenoff (default: 8)
+      # gravity: how far to tilt the device before the screen rotates
+      # for a given setting, normal <-> invert requires more movement then left <-> right
+      # i.e. the settingd doesn't feel completely symmetric
+      # SXMO_ROTATION_GRAVITY default is 16374
+      # SXMO_ROTATION_GRAVITY = "12800";  # uncomfortably high
+      # SXMO_ROTATION_GRAVITY = "12500";    # kinda uncomfortable when walking
+      SXMO_ROTATION_GRAVITY = "12000";
+      SXMO_SCREENSHOT_DIR = "/home/colin/Pictures";  # default: "$HOME"
+      # test new scales by running `swaymsg -- output DSI-1 scale x.y`
+      # SXMO_SWAY_SCALE = "1.5";  # hard to press gPodder icons
+      SXMO_SWAY_SCALE = "1.8";
+      # SXMO_SWAY_SCALE = "2";
+      SXMO_WORKSPACE_WRAPPING = "5";  # how many workspaces. default: 4
+
+      # wvkbd layers:
+      # - full
+      # - landscape
+      # - special  (e.g. coding symbols like ~)
+      # - emoji
+      # - nav
+      # - simple  (like landscape, but no parens/tab/etc; even fewer chars)
+      # - simplegrid  (simple, but grid layout)
+      # - dialer  (digits)
+      # - cyrillic
+      # - arabic
+      # - persian
+      # - greek
+      # - georgian
+      WVKBD_LANDSCAPE_LAYERS = "landscape,special,emoji";
+      WVKBD_LAYERS = "full,special,emoji";
+    };
+    package = pkgs.sxmo-utils-latest.overrideAttrs (base: {
+      postPatch = (base.postPatch or "") + ''
+        cat <<EOF >> ./configs/default_hooks/sxmo_hook_start.sh
+        # rotate UI based on physical display angle by default
+        sxmo_daemons.sh start autorotate sxmo_autorotate.sh
+        EOF
+      '';
+    });
+  };
+}

hosts/by-name/rescue/default.nix

@@ -1,4 +1,4 @@
-{ config, pkgs, ... }:
+{ pkgs, ... }:
 {
   imports = [
     ./fs.nix
@@ -7,6 +7,8 @@
   boot.loader.generic-extlinux-compatible.enable = true;
   boot.loader.efi.canTouchEfiVariables = false;
   sane.image.extraBootFiles = [ pkgs.bootpart-uefi-x86_64 ];
+  # sane.persist.enable = false;  # TODO: disable (but run `nix flake check` to ensure it works!)
+  sane.nixcache.enable = false;  # don't want to be calling out to dead machines that we're *trying* to rescue
 
   # docs: https://nixos.org/manual/nixos/stable/options.html#opt-system.stateVersion
   system.stateVersion = "21.05";

hosts/by-name/servo/default.nix

@@ -4,7 +4,6 @@
   imports = [
     ./fs.nix
     ./net.nix
-    ./secrets.nix
     ./services
   ];
 
@@ -15,10 +14,20 @@
     signaldctl.enableFor.user.colin = true;
   };
 
-  sane.persist.enable = true;
+  sane.roles.ac = true;
+  sane.roles.build-machine.enable = true;
+  sane.roles.build-machine.emulation = false;
+  sane.zsh.showDeadlines = false;  # ~/knowledge doesn't always exist
+  sane.programs.consoleUtils.suggestedPrograms = [
+    "desktopConsoleUtils"
+    "sane-scripts.stop-all-servo"
+  ];
   sane.services.dyn-dns.enable = true;
   sane.services.wg-home.enable = true;
+  sane.services.wg-home.enableWan = true;
   sane.services.wg-home.ip = config.sane.hosts.by-name."servo".wg-home.ip;
+  sane.nixcache.substituters.servo = false;
+  sane.nixcache.substituters.desko = false;
   # sane.services.duplicity.enable = true;  # TODO: re-enable after HW upgrade
 
   # automatically log in at the virtual consoles.

hosts/by-name/servo/fs.nix

@@ -2,15 +2,9 @@
 
 {
   sane.persist.root-on-tmpfs = true;
-  # we need a /tmp for building large nix things
-  fileSystems."/tmp" = {
-    device = "none";
-    fsType = "tmpfs";
-    options = [
-      "mode=777"
-      "defaults"
-    ];
-  };
+  # increase /tmp space (defaults to 50% of RAM) for building large nix things.
+  # even the stock `nixpkgs.linux` consumes > 16 GB of tmp
+  fileSystems."/tmp".options = [ "size=32G" ];
 
   fileSystems."/nix" = {
     device = "/dev/disk/by-uuid/cc81cca0-3cc7-4d82-a00c-6243af3e7776";
@@ -44,7 +38,7 @@
 
   sane.persist.sys.plaintext = [
     # TODO: this is overly broad; only need media and share directories to be persisted
-    { user = "colin"; group = "users"; directory = "/var/lib/uninsane"; }
+    { user = "colin"; group = "users"; path = "/var/lib/uninsane"; }
   ];
   # make sure large media is stored to the HDD
   sane.persist.sys.ext = [
@@ -52,21 +46,22 @@
       user = "colin";
       group = "users";
       mode = "0777";
-      directory = "/var/lib/uninsane/media/Videos";
+      path = "/var/lib/uninsane/media/Videos";
     }
     {
       user = "colin";
       group = "users";
       mode = "0777";
-      directory = "/var/lib/uninsane/media/freeleech";
+      path = "/var/lib/uninsane/media/freeleech";
+    }
+    {
+      user = "colin";
+      group = "users";
+      mode = "0777";
+      path = "/var/lib/uninsane/media/datasets";
     }
   ];
 
-  # in-memory compressed RAM (seems to be dynamically sized)
-  # zramSwap = {
-  #   enable = true;
-  # };
-
   # btrfs doesn't easily support swapfiles
   # swapDevices = [
   #   { device = "/nix/persist/swapfile"; size = 4096; }

hosts/by-name/servo/net.nix

@@ -3,6 +3,12 @@
 {
   networking.domain = "uninsane.org";
 
+  sane.ports.openFirewall = true;
+  sane.ports.openUpnp = true;
+
+  # view refused packets with: `sudo journalctl -k`
+  # networking.firewall.logRefusedPackets = true;
+
   # The global useDHCP flag is deprecated, therefore explicitly set to false here.
   # Per-interface useDHCP will be mandatory in the future, so this generated config
   # replicates the default behaviour.
@@ -11,9 +17,6 @@
   # XXX colin: probably don't need this. wlan0 won't be populated unless i touch a value in networking.interfaces.wlan0
   networking.wireless.enable = false;
 
-  # networking.firewall.enable = false;
-  networking.firewall.enable = true;
-
   # this is needed to forward packets from the VPN to the host
   boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
 
@@ -33,6 +36,14 @@
   # - getent ahostsv4 www.google.com
   # - try fix: <https://serverfault.com/questions/765989/connect-to-3rd-party-vpn-server-but-dont-use-it-as-the-default-route/766290#766290>
   services.resolved.enable = true;
+  # without DNSSEC:
+  # - dig matrix.org => works
+  # - curl https://matrix.org => works
+  # with default DNSSEC:
+  # - dig matrix.org => works
+  # - curl https://matrix.org => fails
+  # i don't know why. this might somehow be interfering with the DNS run on this device (trust-dns)
+  services.resolved.dnssec = "false";
   networking.nameservers = [
     # use systemd-resolved resolver
     # full resolver (which understands /etc/hosts) lives on 127.0.0.53
@@ -145,9 +156,9 @@
 
       # we also bridge DNS traffic
       ${in-ns} ${iptables} -A PREROUTING -t nat -p udp --dport 53 -m iprange --dst-range ${vpn-ip} \
-        -j DNAT --to-destination ${veth-host-ip}:53
+        -j DNAT --to-destination ${veth-host-ip}
       ${in-ns} ${iptables} -A PREROUTING -t nat -p tcp --dport 53 -m iprange --dst-range ${vpn-ip} \
-        -j DNAT --to-destination ${veth-host-ip}:53
+        -j DNAT --to-destination ${veth-host-ip}
 
       # in order to access DNS in this netns, we need to route it to the VPN's nameservers
       # - alternatively, we could fix DNS servers like 1.1.1.1.

hosts/by-name/servo/secrets.nix

@@ -1,41 +0,0 @@
-{ ... }:
-
-{
-  sops.secrets."ddns_afraid" = {
-    sopsFile = ../../../secrets/servo.yaml;
-  };
-  sops.secrets."ddns_he" = {
-    sopsFile = ../../../secrets/servo.yaml;
-  };
-
-  sops.secrets."dovecot_passwd" = {
-    sopsFile = ../../../secrets/servo.yaml;
-  };
-
-  sops.secrets."duplicity_passphrase" = {
-    sopsFile = ../../../secrets/servo.yaml;
-  };
-
-  sops.secrets."freshrss_passwd" = {
-    sopsFile = ../../../secrets/servo.yaml;
-  };
-
-  sops.secrets."matrix_synapse_secrets" = {
-    sopsFile = ../../../secrets/servo.yaml;
-  };
-  sops.secrets."mautrix_signal_env" = {
-    sopsFile = ../../../secrets/servo/mautrix_signal_env.bin;
-  };
-
-  sops.secrets."mediawiki_pw" = {
-    sopsFile = ../../../secrets/servo.yaml;
-  };
-
-  sops.secrets."pleroma_secrets" = {
-    sopsFile = ../../../secrets/servo.yaml;
-  };
-
-  sops.secrets."wg_ovpns_privkey" = {
-    sopsFile = ../../../secrets/servo.yaml;
-  };
-}

hosts/by-name/servo/services/calibre.nix

@@ -0,0 +1,34 @@
+{ config, lib, ... }:
+
+let
+  cweb-cfg = config.services.calibre-web;
+  inherit (cweb-cfg) user group;
+  inherit (cweb-cfg.listen) ip port;
+  svc-dir = "/var/lib/${cweb-cfg.dataDir}";
+in
+# XXX: disabled because of runtime errors like:
+# > File "/nix/store/c7jqvx980nlg9xhxi065cba61r2ain9y-calibre-web-0.6.19/lib/python3.10/site-packages/calibreweb/cps/db.py", line 926, in speaking_language
+# > languages = self.session.query(Languages) \
+# > AttributeError: 'NoneType' object has no attribute 'query'
+lib.mkIf false
+{
+  sane.persist.sys.plaintext = [
+    { inherit user group; mode = "0700"; path = svc-dir; }
+  ];
+
+  services.calibre-web.enable = true;
+  services.calibre-web.listen.ip = "127.0.0.1";
+  # XXX: externally populate `${svc-dir}/metadata.db` (once) from
+  #   <https://github.com/janeczku/calibre-web/blob/master/library/metadata.db>
+  # i don't know why you have to do this??
+  # services.calibre-web.options.calibreLibrary = svc-dir;
+
+  services.nginx.virtualHosts."calibre.uninsane.org" = {
+    addSSL = true;
+    enableACME = true;
+    locations."/" = {
+      proxyPass = "http://${ip}:${builtins.toString port}";
+    };
+  };
+  sane.dns.zones."uninsane.org".inet.CNAME."calibre" = "native";
+}

hosts/by-name/servo/services/ddns-afraid.nix

@@ -6,7 +6,7 @@ lib.mkIf false
   systemd.services.ddns-afraid = {
     description = "update dynamic DNS entries for freedns.afraid.org";
     serviceConfig = {
-      EnvironmentFile = config.sops.secrets.ddns_afraid.path;
+      EnvironmentFile = config.sops.secrets."ddns_afraid.env".path;
       # TODO: ProtectSystem = "strict";
       # TODO: ProtectHome = "full";
       # TODO: PrivateTmp = true;

hosts/by-name/servo/services/ddns-he.nix

@@ -6,7 +6,7 @@ lib.mkIf false
   systemd.services.ddns-he = {
     description = "update dynamic DNS entries for HurricaneElectric";
     serviceConfig = {
-      EnvironmentFile = config.sops.secrets.ddns_he.path;
+      EnvironmentFile = config.sops.secrets."ddns_he.env".path;
       # TODO: ProtectSystem = "strict";
       # TODO: ProtectHome = "full";
       # TODO: PrivateTmp = true;

hosts/by-name/servo/services/default.nix

@@ -1,22 +1,27 @@
 { ... }:
 {
   imports = [
+    ./calibre.nix
     ./ddns-afraid.nix
     ./ddns-he.nix
+    ./email
     ./ejabberd.nix
     ./freshrss.nix
+    ./export
     ./gitea.nix
     ./goaccess.nix
     ./ipfs.nix
     ./jackett.nix
     ./jellyfin.nix
     ./kiwix-serve.nix
+    ./komga.nix
+    ./lemmy.nix
     ./matrix
     ./navidrome.nix
     ./nixserve.nix
     ./nginx.nix
+    ./pict-rs.nix
     ./pleroma.nix
-    ./postfix.nix
     ./postgres.nix
     ./prosody.nix
     ./transmission.nix

hosts/by-name/servo/services/ejabberd.nix

@@ -14,36 +14,105 @@
 #
 # compliance tests:
 # - <https://compliance.conversations.im/server/uninsane.org/#xep0352>
+#
+# administration:
+# - `sudo -u ejabberd ejabberdctl help`
+#
+# federation/support matrix:
+# - avatars
+#   - nixnet.services + dino: works in MUCs but not DMs  (as of 2023 H1)
+#   - movim.eu + dino: works in DMs, MUCs untested  (as of 2023/08/29)
+# - calls
+#   - local + dino: audio, video, works in DMs  (as of 2023/08/29)
+#   - movim.eu + dino: audio, video, works in DMs, no matter which side initiates  (as of 2023/08/30)
+#   - +native-cell-number@cheogram.com + dino: audio works in DMs, no matter which side initiates  (as of 2023/09/01)
+#     - can receive calls even if sender isn't in my roster
+#     - this is presumably using JMP.chat's SIP servers, which then convert it to XMPP call
+#
+# bugs:
+# - 2023/09/01: will randomly stop federating. `systemctl restart ejabberd` fixes, but takes 10 minutes.
 { config, lib, pkgs, ... }:
 
-# XXX: avatar support works in MUCs but not DMs
-# lib.mkIf false
+let
+  # TODO: this range could be larger, but right now that's costly because each element is its own UPnP forward
+  # TURN port range (inclusive)
+  turnPortLow = 49152;
+  turnPortHigh = 49167;
+  turnPortRange = lib.range turnPortLow turnPortHigh;
+in
 {
   sane.persist.sys.plaintext = [
-    { user = "ejabberd"; group = "ejabberd"; directory = "/var/lib/ejabberd"; }
+    { user = "ejabberd"; group = "ejabberd"; path = "/var/lib/ejabberd"; }
   ];
-  networking.firewall.allowedTCPPorts = [
-    3478  # STUN/TURN
-    5222  # XMPP  client -> server
-    5223  # XMPPS client -> server (XMPP over TLS)
-    5269  # XMPP  server -> server
-    5270  # XMPPS server -> server (XMPP over TLS)
-    5280  # bosh
-    5281  # bosh (https) ??
-    5349  # STUN/TURN (TLS)
-    5443  # web services (file uploads, websockets, admin)
-  ];
-  networking.firewall.allowedUDPPorts = [
-    3478  # STUN/TURN
-  ];
-  networking.firewall.allowedTCPPortRanges = [{
-    from = 49152;  # TURN
-    to = 65535;
-  }];
-  networking.firewall.allowedUDPPortRanges = [{
-    from = 49152;  # TURN
-    to = 65535;
-  }];
+  sane.ports.ports = lib.mkMerge ([
+    {
+      "3478" = {
+        protocol = [ "tcp" "udp" ];
+        visibleTo.lan = true;
+        visibleTo.wan = true;
+        description = "colin-xmpp-stun-turn";
+      };
+      "5222" = {
+        protocol = [ "tcp" ];
+        visibleTo.lan = true;
+        visibleTo.wan = true;
+        description = "colin-xmpp-client-to-server";
+      };
+      "5223" = {
+        protocol = [ "tcp" ];
+        visibleTo.lan = true;
+        visibleTo.wan = true;
+        description = "colin-xmpps-client-to-server";  # XMPP over TLS
+      };
+      "5269" = {
+        protocol = [ "tcp" ];
+        visibleTo.wan = true;
+        description = "colin-xmpp-server-to-server";
+      };
+      "5270" = {
+        protocol = [ "tcp" ];
+        visibleTo.wan = true;
+        description = "colin-xmpps-server-to-server";  # XMPP over TLS
+      };
+      "5280" = {
+        protocol = [ "tcp" ];
+        visibleTo.lan = true;
+        visibleTo.wan = true;
+        description = "colin-xmpp-bosh";
+      };
+      "5281" = {
+        protocol = [ "tcp" ];
+        visibleTo.lan = true;
+        visibleTo.wan = true;
+        description = "colin-xmpp-bosh-https";
+      };
+      "5349" = {
+        protocol = [ "tcp" ];
+        visibleTo.lan = true;
+        visibleTo.wan = true;
+        description = "colin-xmpp-stun-turn-over-tls";
+      };
+      "5443" = {
+        protocol = [ "tcp" ];
+        visibleTo.lan = true;
+        visibleTo.wan = true;
+        description = "colin-xmpp-web-services";  # file uploads, websockets, admin
+      };
+    }
+  ] ++ (builtins.map
+    (port: {
+      "${builtins.toString port}" = let
+        count = port - turnPortLow + 1;
+        numPorts = turnPortHigh - turnPortLow + 1;
+      in {
+        protocol = [ "tcp" "udp" ];
+        visibleTo.lan = true;
+        visibleTo.wan = true;
+        description = "colin-xmpp-turn-${builtins.toString count}-of-${builtins.toString numPorts}";
+      };
+    })
+    turnPortRange
+  ));
 
   # provide access to certs
   # TODO: this should just be `acme`. then we also add nginx to the `acme` group.
@@ -75,9 +144,9 @@
     useACMEHost = "uninsane.org";
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet = {
+  sane.dns.zones."uninsane.org".inet = {
     # XXX: SRV records have to point to something with a A/AAAA record; no CNAMEs
-    A."xmpp" =                "%NATIVE%";
+    A."xmpp" =                "%ANATIVE%";
     CNAME."muc.xmpp" =        "xmpp";
     CNAME."pubsub.xmpp" =     "xmpp";
     CNAME."upload.xmpp" =     "xmpp";
@@ -232,18 +301,18 @@
             module: ejabberd_stun
             transport: tcp
             use_turn: true
-            turn_min_port: 49152
-            turn_max_port: 65535
-            turn_ipv4_address: %NATIVE%
+            turn_min_port: ${builtins.toString turnPortLow}
+            turn_max_port: ${builtins.toString turnPortHigh}
+            turn_ipv4_address: %ANATIVE%
           -
             # STUN+TURN UDP
             port: 3478
             module: ejabberd_stun
             transport: udp
             use_turn: true
-            turn_min_port: 49152
-            turn_max_port: 65535
-            turn_ipv4_address: %NATIVE%
+            turn_min_port: ${builtins.toString turnPortLow}
+            turn_max_port: ${builtins.toString turnPortHigh}
+            turn_ipv4_address: %ANATIVE%
           -
             # STUN+TURN TLS over TCP
             port: 5349
@@ -252,9 +321,9 @@
             tls: true
             certfile: /var/lib/acme/uninsane.org/full.pem
             use_turn: true
-            turn_min_port: 49152
-            turn_max_port: 65535
-            turn_ipv4_address: %NATIVE%
+            turn_min_port: ${builtins.toString turnPortLow}
+            turn_max_port: ${builtins.toString turnPortHigh}
+            turn_ipv4_address: %ANATIVE%
 
         # TODO: enable mod_fail2ban
         # TODO(low): look into mod_http_fileserver for serving macros?
@@ -387,7 +456,7 @@
     # config is 444 (not 644), so we want to write out-of-place and then atomically move
     # TODO: factor this out into `sane-woop` helper?
     rm -f /var/lib/ejabberd/ejabberd.yaml.new
-    ${sed} "s/%NATIVE%/$ip/" ${config-in} > /var/lib/ejabberd/ejabberd.yaml.new
+    ${sed} "s/%ANATIVE%/$ip/" ${config-in} > /var/lib/ejabberd/ejabberd.yaml.new
     mv /var/lib/ejabberd/ejabberd.yaml{.new,}
   '';
 

hosts/by-name/servo/services/email/default.nix

@@ -0,0 +1,37 @@
+# nix configs to reference:
+# - <https://gitlab.com/simple-nixos-mailserver/nixos-mailserver>
+# - <https://github.com/nix-community/nur-combined/-/tree/master/repos/eh5/machines/srv-m/mail-rspamd.nix>
+#   - postfix / dovecot / rspamd / stalwart-jmap / sogo
+#
+# rspamd:
+# - nixos: <https://nixos.wiki/wiki/Rspamd>
+# - guide: <https://rspamd.com/doc/quickstart.html>
+# - non-nixos example: <https://dataswamp.org/~solene/2021-07-13-smtpd-rspamd.html>
+#
+#
+# my rough understanding of the pieces:
+# - postfix handles SMTP protocol with the rest of the world.
+# - dovecot implements IMAP protocol.
+#   - client auth (i.e. validate that user@uninsane.org is who they claim)
+#   - "folders" (INBOX, JUNK) are internal to dovecot?
+#     or where do folders live, on-disk?
+#
+# - non-local clients (i.e. me) interact with BOTH postfix and dovecot, but primarily dovecot:
+#   - mail reading is done via IMAP (so, dovecot)
+#   - mail sending is done via SMTP/submission port (so, postfix)
+#     - but postfix delegates authorization of that outgoing mail to dovecot, on the server side
+#
+# - local clients (i.e. sendmail) interact only with postfix
+
+{ ... }:
+{
+  imports = [
+    ./dovecot.nix
+    ./postfix.nix
+  ];
+
+
+  #### SPAM FILTERING
+  # services.rspamd.enable = true;
+  # services.rspamd.postfix.enable = true;
+}

hosts/by-name/servo/services/email/dovecot.nix

@@ -0,0 +1,142 @@
+# dovecot config options: <https://doc.dovecot.org/configuration_manual/>
+#
+# sieve docs:
+# - sieve language examples: <https://doc.dovecot.org/configuration_manual/sieve/examples/>
+# - sieve protocol/language: <https://proton.me/support/sieve-advanced-custom-filters>
+
+{ config, lib, pkgs, ... }:
+{
+  sane.ports.ports."143" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-imap-imap.uninsane.org";
+  };
+  sane.ports.ports."993" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-imaps-imap.uninsane.org";
+  };
+
+  # exists only to manage certs for dovecot
+  services.nginx.virtualHosts."imap.uninsane.org" = {
+    enableACME = true;
+  };
+
+  sane.dns.zones."uninsane.org".inet = {
+    CNAME."imap" = "native";
+  };
+
+  sops.secrets."dovecot_passwd" = {
+    owner = config.users.users.dovecot2.name;
+    # TODO: debug why mail can't be sent without this being world-readable
+    mode = "0444";
+  };
+
+  # inspired by https://gitlab.com/simple-nixos-mailserver/nixos-mailserver/
+  services.dovecot2.enable = true;
+  # services.dovecot2.enableLmtp = true;
+  services.dovecot2.sslServerCert = "/var/lib/acme/imap.uninsane.org/fullchain.pem";
+  services.dovecot2.sslServerKey = "/var/lib/acme/imap.uninsane.org/key.pem";
+  services.dovecot2.enablePAM = false;
+
+  # sieve scripts require me to set a user for... idk why?
+  services.dovecot2.mailUser = "colin";
+  services.dovecot2.mailGroup = "users";
+  users.users.colin.isSystemUser = lib.mkForce false;
+
+  services.dovecot2.extraConfig =
+  let
+    passwdFile = config.sops.secrets.dovecot_passwd.path;
+  in
+    ''
+    passdb {
+      driver = passwd-file
+      args = ${passwdFile}
+    }
+    userdb {
+      driver = passwd-file
+      args = ${passwdFile}
+    }
+
+    # allow postfix to query our auth db
+    service auth {
+      unix_listener auth {
+        mode = 0660
+        user = postfix
+        group = postfix
+      }
+    }
+    auth_mechanisms = plain login
+
+    # accept incoming messaging from postfix
+    # service lmtp {
+    #   unix_listener dovecot-lmtp {
+    #     mode = 0600
+    #     user = postfix
+    #     group = postfix
+    #   }
+    # }
+
+    # plugin {
+    #   sieve_plugins = sieve_imapsieve
+    # }
+
+    mail_debug = yes
+    auth_debug = yes
+    # verbose_ssl = yes
+  '';
+
+  services.dovecot2.mailboxes = {
+    # special-purpose mailboxes: "All" "Archive" "Drafts" "Flagged" "Junk" "Sent" "Trash"
+    # RFC6154 describes these special mailboxes: https://www.ietf.org/rfc/rfc6154.html
+    # how these boxes are treated is 100% up to the client and server to decide.
+    # client behavior:
+    # iOS
+    #   - Drafts: ?
+    #   - Sent: works
+    #   - Trash: works
+    #   - Junk: works ("mark" -> "move to Junk")
+    # aerc
+    #   - Drafts: works
+    #   - Sent: works
+    #   - Trash: no; deleted messages are actually deleted
+    #       use `:move trash` instead
+    #   - Junk: ?
+    # Sent mailbox: all sent messages are copied to it. unclear if this happens server-side or client-side.
+    Drafts = { specialUse = "Drafts"; auto = "create"; };
+    Sent = { specialUse = "Sent"; auto = "create"; };
+    Trash = { specialUse = "Trash"; auto = "create"; };
+    Junk = { specialUse = "Junk"; auto = "create"; };
+  };
+
+  services.dovecot2.mailPlugins = {
+    perProtocol = {
+      # imap.enable = [
+      #   "imap_sieve"
+      # ];
+      lda.enable = [
+        "sieve"
+      ];
+      # lmtp.enable = [
+      #   "sieve"
+      # ];
+    };
+  };
+  services.dovecot2.modules = [
+    pkgs.dovecot_pigeonhole  # enables sieve execution (?)
+  ];
+  services.dovecot2.sieveScripts = {
+    # if any messages fail to pass (or lack) DKIM, move them to Junk
+    # XXX the key name ("after") is only used to order sieve execution/ordering
+    after = builtins.toFile "ensuredkim.sieve" ''
+      require "fileinto";
+
+      if not header :contains "Authentication-Results" "dkim=pass" {
+        fileinto "Junk";
+        stop;
+      }
+    '';
+  };
+}

hosts/by-name/servo/services/email/postfix.nix

@@ -1,7 +1,6 @@
-# DOCS:
-# - dovecot config: <https://doc.dovecot.org/configuration_manual/>
+# postfix config options: <https://www.postfix.org/postconf.5.html>
 
-{ config, lib, ... }:
+{ lib, pkgs, ... }:
 
 let
   submissionOptions = {
@@ -21,37 +20,40 @@ in
 {
   sane.persist.sys.plaintext = [
     # TODO: mode? could be more granular
-    { user = "opendkim"; group = "opendkim"; directory = "/var/lib/opendkim"; }
-    { user = "root"; group = "root"; directory = "/var/lib/postfix"; }
-    { user = "root"; group = "root"; directory = "/var/spool/mail"; }
+    { user = "opendkim"; group = "opendkim"; path = "/var/lib/opendkim"; }
+    { user = "root"; group = "root"; path = "/var/lib/postfix"; }
+    { user = "root"; group = "root"; path = "/var/spool/mail"; }
     # *probably* don't need these dirs:
     # "/var/lib/dhparams"          # https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/security/dhparams.nix
     # "/var/lib/dovecot"
   ];
 
-  networking.firewall.allowedTCPPorts = [
-    25   # SMTP
-    143  # IMAP
-    465  # SMTPS
-    587  # SMTPS/submission
-    993  # IMAPS
-  ];
-
-  # exists only to manage certs for dovecot
-  services.nginx.virtualHosts."imap.uninsane.org" = {
-    enableACME = true;
+  sane.ports.ports."25" = {
+    protocol = [ "tcp" ];
+    visibleTo.ovpn = true;
+    description = "colin-smtp-mx.uninsane.org";
   };
+  sane.ports.ports."465" = {
+    protocol = [ "tcp" ];
+    visibleTo.ovpn = true;
+    description = "colin-smtps-mx.uninsane.org";
+  };
+  sane.ports.ports."587" = {
+    protocol = [ "tcp" ];
+    visibleTo.ovpn = true;
+    description = "colin-smtps-submission-mx.uninsane.org";
+  };
+
   # exists only to manage certs for Postfix
   services.nginx.virtualHosts."mx.uninsane.org" = {
     enableACME = true;
   };
 
 
-  sane.services.trust-dns.zones."uninsane.org".inet = {
+  sane.dns.zones."uninsane.org".inet = {
     MX."@" = "10 mx.uninsane.org.";
     # XXX: RFC's specify that the MX record CANNOT BE A CNAME
     A."mx" = "185.157.162.178";
-    CNAME."imap" = "native";
 
     # Sender Policy Framework:
     #   +mx     => mail passes if it originated from the MX
@@ -62,7 +64,7 @@ in
 
     # DKIM public key:
     TXT."mx._domainkey" =
-      "v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkSyMufc2KrRx3j17e/LyB+3eYSBRuEFT8PUka8EDX04QzCwDPdkwgnj3GNDvnB5Ktb05Cf2SJ/S1OLqNsINxJRWtkVfZd/C339KNh9wrukMKRKNELL9HLUw0bczOI4gKKFqyrRE9qm+4csCMAR79Te9FCjGV/jVnrkLdPT0GtFwIDAQAB"
+      "v=DKIM1;k=rsa;p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkSyMufc2KrRx3j17e/LyB+3eYSBRuEFT8PUka8EDX04QzCwDPdkwgnj3GNDvnB5Ktb05Cf2SJ/S1OLqNsINxJRWtkVfZd/C339KNh9wrukMKRKNELL9HLUw0bczOI4gKKFqyrRE9qm+4csCMAR79Te9FCjGV/jVnrkLdPT0GtFwIDAQAB"
     ;
 
     # DMARC fields <https://datatracker.ietf.org/doc/html/rfc7489>:
@@ -95,18 +97,40 @@ in
     @uninsane.org colin
   '';
 
-  services.postfix.extraConfig = ''
+  services.postfix.config = {
     # smtpd_milters = local:/run/opendkim/opendkim.sock
     # milter docs: http://www.postfix.org/MILTER_README.html
-    # mail filters for receiving email and authorized SMTP clients
+    # mail filters for receiving email and from authorized SMTP clients (i.e. via submission)
     # smtpd_milters = inet:185.157.162.190:8891
-    smtpd_milters = unix:/run/opendkim/opendkim.sock
+    # opendkim.sock will add a Authentication-Results header, with `dkim=pass|fail|...` value to received messages
+    smtpd_milters = "unix:/run/opendkim/opendkim.sock";
     # mail filters for sendmail
-    non_smtpd_milters = $smtpd_milters
-    milter_default_action = accept
-    inet_protocols = ipv4
-    smtp_tls_security_level = may
-  '';
+    non_smtpd_milters = "$smtpd_milters";
+
+    # what to do when a milter exits unexpectedly:
+    milter_default_action = "accept";
+
+    inet_protocols = "ipv4";
+    smtp_tls_security_level = "may";
+
+    # hand received mail over to dovecot so that it can run sieves & such
+    mailbox_command = ''${pkgs.dovecot}/libexec/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"'';
+
+    # hand received mail over to dovecot
+    # virtual_alias_maps = [
+    #   "hash:/etc/postfix/virtual"
+    # ];
+    # mydestination = "";
+    # virtual_mailbox_domains = [ "localhost" "uninsane.org" ];
+    # # virtual_mailbox_maps = "hash:/etc/postfix/virtual";
+    # virtual_transport = "lmtp:unix:/run/dovecot2/dovecot-lmtp";
+
+    # anti-spam options: <https://www.postfix.org/SMTPD_ACCESS_README.html>
+    # reject_unknown_sender_domain: causes postfix to `dig <sender> MX` and make sure that exists.
+    # but may cause problems receiving mail from google & others who load-balance?
+    # - <https://unix.stackexchange.com/questions/592131/how-to-reject-email-from-unknown-domains-with-postfix-on-centos>
+    # smtpd_sender_restrictions = reject_unknown_sender_domain
+  };
 
   services.postfix.enableSubmission = true;
   services.postfix.submissionOptions = submissionOptions;
@@ -121,6 +145,8 @@ in
   };
 
 
+  #### OPENDKIM
+
   services.opendkim.enable = true;
   # services.opendkim.domains = "csl:uninsane.org";
   services.opendkim.domains = "uninsane.org";
@@ -144,59 +170,6 @@ in
     UMask = lib.mkForce "0011";
   };
 
-  # inspired by https://gitlab.com/simple-nixos-mailserver/nixos-mailserver/
-  services.dovecot2.enable = true;
-  services.dovecot2.mailboxes = {
-    # special-purpose mailboxes: "All" "Archive" "Drafts" "Flagged" "Junk" "Sent" "Trash"
-    # RFC6154 describes these special mailboxes: https://www.ietf.org/rfc/rfc6154.html
-    # how these boxes are treated is 100% up to the client and server to decide.
-    # client behavior:
-    # iOS
-    #   - Drafts: ?
-    #   - Sent: works
-    #   - Trash: works
-    # aerc
-    #   - Drafts: works
-    #   - Sent: works
-    #   - Trash: no; deleted messages are actually deleted
-    #       use `:move trash` instead
-    # Sent mailbox: all sent messages are copied to it. unclear if this happens server-side or client-side.
-    Drafts = { specialUse = "Drafts"; auto = "create"; };
-    Sent = { specialUse = "Sent"; auto = "create"; };
-    Trash = { specialUse = "Trash"; auto = "create"; };
-  };
-  services.dovecot2.sslServerCert = "/var/lib/acme/imap.uninsane.org/fullchain.pem";
-  services.dovecot2.sslServerKey = "/var/lib/acme/imap.uninsane.org/key.pem";
-  services.dovecot2.enablePAM = false;
-  services.dovecot2.extraConfig =
-  let
-    passwdFile = config.sops.secrets.dovecot_passwd.path;
-  in
-    ''
-    passdb {
-      driver = passwd-file
-      args = ${passwdFile}
-    }
-    userdb {
-      driver = passwd-file
-      args = ${passwdFile}
-    }
-
-    # allow postfix to query our auth db
-    service auth {
-      unix_listener auth {
-        mode = 0660
-        user = postfix
-        group = postfix
-      }
-    }
-    auth_mechanisms = plain login
-
-
-    mail_debug = yes
-    auth_debug = yes
-    # verbose_ssl = yes
-  '';
 
   #### OUTGOING MESSAGE REWRITING:
   services.postfix.enableHeaderChecks = true;
@@ -218,10 +191,4 @@ in
     #   pattern = "/^Subject:.*activate your account/";
     # }
   ];
-
-  sops.secrets."dovecot_passwd" = {
-    owner = config.users.users.dovecot2.name;
-    # TODO: debug why mail can't be sent without this being world-readable
-    mode = "0444";
-  };
 }

hosts/by-name/servo/services/export/default.nix

@@ -0,0 +1,53 @@
+{ config, ... }:
+{
+  imports = [
+    ./nfs.nix
+    ./sftpgo.nix
+  ];
+
+  users.groups.export = {};
+
+  fileSystems."/var/export/media" = {
+    # everything in here could be considered publicly readable (based on the viewer's legal jurisdiction)
+    device = "/var/lib/uninsane/media";
+    options = [ "rbind" ];
+  };
+  # fileSystems."/var/export/playground" = {
+  #   device = config.fileSystems."/mnt/persist/ext".device;
+  #   fsType = "btrfs";
+  #   options = [
+  #     "subvol=export-playground"
+  #     "compress=zstd"
+  #     "defaults"
+  #   ];
+  # };
+  # N.B.: the backing directory should be manually created here **as a btrfs subvolume** and with a quota.
+  # - `sudo btrfs subvolume create /mnt/persist/ext/persist/var/export/playground`
+  # - `sudo btrfs quota enable /mnt/persist/ext/persist/var/export/playground`
+  # - `sudo btrfs quota rescan -sw /mnt/persist/ext/persist/var/export/playground`
+  # to adjust the limits (which apply at the block layer, i.e. post-compression):
+  # - `sudo btrfs qgroup limit 20G /mnt/persist/ext/persist/var/export/playground`
+  # to query the quota/status:
+  # - `sudo btrfs qgroup show -re /var/export/playground`
+  sane.persist.sys.ext = [
+    { user = "root"; group = "export"; mode = "0775"; path = "/var/export/playground"; }
+  ];
+
+  sane.fs."/var/export/README.md" = {
+    wantedBy = [ "nfs.service" "sftpgo.service" ];
+    file.text = ''
+      - media/         read-only:  Videos, Music, Books, etc
+      - playground/    read-write: use it to share files with other users of this server
+    '';
+  };
+
+  sane.fs."/var/export/playground/README.md" = {
+    wantedBy = [ "nfs.service" "sftpgo.service" ];
+    file.text = ''
+      this directory is intentionally read+write by anyone with access (i.e. on the LAN).
+      - share files
+      - write poetry
+      - be a friendly troll
+    '';
+  };
+}

hosts/by-name/servo/services/export/nfs.nix

@@ -0,0 +1,110 @@
+# docs:
+# - <https://nixos.wiki/wiki/NFS>
+# - <https://wiki.gentoo.org/wiki/Nfs-utils>
+# system files:
+# - /etc/exports
+# system services:
+# - nfs-server.service
+# - nfs-idmapd.service
+# - nfs-mountd.service
+# - nfsdcld.service
+# - rpc-statd.service
+# - rpcbind.service
+#
+# TODO: force files to be 755, or 750.
+# - could maybe be done with some mount option?
+
+{ config, lib, ... }:
+{
+  services.nfs.server.enable = true;
+
+  # see which ports NFS uses with:
+  # - `rpcinfo -p`
+  sane.ports.ports."111" = {
+    protocol = [ "tcp" "udp" ];
+    visibleTo.lan = true;
+    description = "NFS server portmapper";
+  };
+  sane.ports.ports."2049" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    description = "NFS server";
+  };
+  sane.ports.ports."4000" = {
+    protocol = [ "udp" ];
+    visibleTo.lan = true;
+    description = "NFS server status daemon";
+  };
+  sane.ports.ports."4001" = {
+    protocol = [ "tcp" "udp" ];
+    visibleTo.lan = true;
+    description = "NFS server lock daemon";
+  };
+  sane.ports.ports."4002" = {
+    protocol = [ "tcp" "udp" ];
+    visibleTo.lan = true;
+    description = "NFS server mount daemon";
+  };
+
+  # NFS4 allows these to float, but NFS3 mandates specific ports, so fix them for backwards compat.
+  services.nfs.server.lockdPort = 4001;
+  services.nfs.server.mountdPort = 4002;
+  services.nfs.server.statdPort = 4000;
+
+  # format:
+  #   fspoint	visibility(options)
+  # options:
+  # - see: <https://wiki.gentoo.org/wiki/Nfs-utils#Exports>
+  # - see [man 5 exports](https://linux.die.net/man/5/exports)
+  # - insecure:  require clients use src port > 1024
+  # - rw, ro (default)
+  # - async, sync (default)
+  # - no_subtree_check (default), subtree_check: verify not just that files requested by the client live
+  #     in the expected fs, but also that they live under whatever subdirectory of that fs is exported.
+  # - no_root_squash, root_squash (default): map requests from uid 0 to user `nobody`.
+  # - crossmnt:  reveal filesystems that are mounted under this endpoint
+  # - fsid:  must be zero for the root export
+  #   - fsid=root  is alias for fsid=0
+  # - mountpoint[=/path]:  only export the directory if it's a mountpoint. used to avoid exporting failed mounts.
+  # - all_squash: rewrite all client requests such that they come from anonuid/anongid
+  #   - any files a user creates are owned by local anonuid/anongid.
+  #   - users can read any local file which anonuid/anongid would be able to read.
+  #   - users can't chown to/away from anonuid/anongid.
+  #   - users can chmod files they own, to anything (making them unreadable to non-`nfsuser` export users, like FTP).
+  #   - `stat` remains unchanged, returning the real UIDs/GIDs to the client.
+  #     - thus programs which check `uid` or `gid` before trying an operation may incorrectly conclude they can't perform some op.
+  #
+  # 10.0.0.0/8 to export both to LAN (readonly, unencrypted) and wg vpn (read-write, encrypted)
+  services.nfs.server.exports =
+  let
+    fmtExport = { export, baseOpts, extraLanOpts ? [], extraVpnOpts ? [] }:
+      let
+        always = [ "subtree_check" ];
+        lanOpts = always ++ baseOpts ++ extraLanOpts;
+        vpnOpts = always ++ baseOpts ++ extraVpnOpts;
+      in "${export} 10.78.79.0/22(${lib.concatStringsSep "," lanOpts}) 10.0.10.0/24(${lib.concatStringsSep "," vpnOpts})";
+  in lib.concatStringsSep "\n" [
+    (fmtExport {
+      export = "/var/export";
+      baseOpts = [ "crossmnt" "fsid=root" ];
+      extraLanOpts = [ "ro" ];
+      extraVpnOpts = [ "rw" "no_root_squash" ];
+    })
+    (fmtExport {
+      export = "/var/export/playground";
+      baseOpts = [
+        "mountpoint"
+        "all_squash"
+        "rw"
+        "anonuid=${builtins.toString config.users.users.nfsuser.uid}"
+        "anongid=${builtins.toString config.users.groups.export.gid}"
+      ];
+    })
+  ];
+
+  users.users.nfsuser = {
+    description = "virtual user for anonymous NFS operations";
+    group = "export";
+    isSystemUser = true;
+  };
+}

hosts/by-name/servo/services/export/sftpgo.nix

@@ -0,0 +1,179 @@
+# docs:
+# - <https://github.com/drakkan/sftpgo>
+# - config options: <https://github.com/drakkan/sftpgo/blob/main/docs/full-configuration.md>
+# - config defaults: <https://github.com/drakkan/sftpgo/blob/main/sftpgo.json>
+# - nixos options: <repo:nixos/nixpkgs:nixos/modules/services/web-apps/sftpgo.nix>
+# - nixos example: <repo:nixos/nixpkgs:nixos/tests/sftpgo.nix>
+#
+# sftpgo is a FTP server that also supports WebDAV, SFTP, and web clients.
+#
+# TODO: change umask so sftpgo-created files default to 644.
+# - it does indeed appear that the 600 is not something sftpgo is explicitly doing.
+
+
+{ config, lib, pkgs, sane-lib, ... }:
+let
+  # user permissions:
+  # - see <repo:drakkan/sftpgo:internal/dataprovider/user.go>
+  # - "*" = grant all permissions
+  # - read-only perms:
+  #   - "list" = list files and directories
+  #   - "download"
+  # - rw perms:
+  #   - "upload"
+  #   - "overwrite" = allow uploads to replace existing files
+  #   - "delete" = delete files and directories
+  #     - "delete_files"
+  #     - "delete_dirs"
+  #   - "rename" = rename files and directories
+  #     - "rename_files"
+  #     - "rename_dirs"
+  #   - "create_dirs"
+  #   - "create_symlinks"
+  #   - "chmod"
+  #   - "chown"
+  #   - "chtimes" = change atime/mtime (access and modification times)
+  #
+  # home_dir:
+  # - it seems (empirically) that a user can't cd above their home directory.
+  #   though i don't have a reference for that in the docs.
+  authResponseSuccess = {
+    status = 1;
+    username = "anonymous";
+    expiration_date = 0;
+    home_dir = "/var/export";
+    # uid/gid 0 means to inherit sftpgo uid.
+    # - i.e. users can't read files which Linux user `sftpgo` can't read
+    # - uploaded files belong to Linux user `sftpgo`
+    # other uid/gid values aren't possible for localfs backend, unless i let sftpgo use `sudo`.
+    uid = 0;
+    gid = 0;
+    # uid = 65534;
+    # gid = 65534;
+    max_sessions = 0;
+    # quota_*: 0 means to not use SFTP's quota system
+    quota_size = 0;
+    quota_files = 0;
+    permissions = {
+      "/" = [ "list" "download" ];
+      "/playground" = [
+        # read-only:
+        "list"
+        "download"
+        # write:
+        "upload"
+        "overwrite"
+        "delete"
+        "rename"
+        "create_dirs"
+        "create_symlinks"
+        # intentionally omitted:
+        # "chmod"
+        # "chown"
+        # "chtimes"
+      ];
+    };
+    upload_bandwidth = 0;
+    download_bandwidth = 0;
+    filters = {
+      allowed_ip = [];
+      denied_ip = [];
+    };
+    public_keys = [];
+    # other fields:
+    # ? groups
+    # ? virtual_folders
+  };
+  authResponseFail = {
+    username = "";
+  };
+  authSuccessJson = pkgs.writeText "sftp-auth-success.json" (builtins.toJSON authResponseSuccess);
+  authFailJson = pkgs.writeText "sftp-auth-fail.json" (builtins.toJSON authResponseFail);
+  unwrappedAuthProgram = pkgs.static-nix-shell.mkBash {
+    pname = "sftpgo_external_auth_hook";
+    src = ./.;
+    pkgs = [ "coreutils" ];
+  };
+  authProgram = pkgs.writeShellScript "sftpgo-auth-hook" ''
+    ${unwrappedAuthProgram}/bin/sftpgo_external_auth_hook ${authFailJson} ${authSuccessJson}
+  '';
+in
+{
+  # Client initiates a FTP "control connection" on port 21.
+  # - this handles the client -> server commands, and the server -> client status, but not the actual data
+  # - file data, directory listings, etc need to be transferred on an ephemeral "data port".
+  # - 50000-50100 is a common port range for this.
+  sane.ports.ports = {
+    "21" = {
+      protocol = [ "tcp" ];
+      visibleTo.lan = true;
+      description = "colin-FTP server";
+    };
+  } // (sane-lib.mapToAttrs
+    (port: {
+      name = builtins.toString port;
+      value = {
+        protocol = [ "tcp" ];
+        visibleTo.lan = true;
+        description = "colin-FTP server data port range";
+      };
+    })
+    (lib.range 50000 50100)
+  );
+
+  services.sftpgo = {
+    enable = true;
+    group = "export";
+    settings = {
+      ftpd = {
+        bindings = [
+          {
+            # binding this means any wireguard client can connect
+            address = "10.0.10.5";
+            port = 21;
+            debug = true;
+          }
+          {
+            # binding this means any LAN client can connect
+            address = "10.78.79.51";
+            port = 21;
+            debug = true;
+          }
+        ];
+
+        # active mode is susceptible to "bounce attacks", without much benefit over passive mode
+        disable_active_mode = true;
+        hash_support = true;
+        passive_port_range = {
+          start = 50000;
+          end = 50100;
+        };
+
+        banner = ''
+          Welcome, friends, to Colin's read-only FTP server! Also available via NFS on the same host.
+          Username: "anonymous"
+          Password: "anonymous"
+          CONFIGURE YOUR CLIENT FOR "PASSIVE" mode, e.g. `ftp --passive uninsane.org`
+          Please let me know if anything's broken or not as it should be. Otherwise, browse and DL freely :)
+        '';
+
+      };
+      data_provider = {
+        driver = "memory";
+        external_auth_hook = "${authProgram}";
+        # track_quota:
+        # - 0: disable quota tracking
+        # - 1: quota is updated on every upload/delete, even if user has no quota restriction
+        # - 2: quota is updated on every upload/delete, but only if user/folder has a quota restriction  (default, i think)
+        # track_quota = 2;
+      };
+    };
+  };
+
+  users.users.sftpgo.extraGroups = [ "export" ];
+
+  systemd.services.sftpgo.serviceConfig = {
+    ReadOnlyPaths = [ "/var/export" ];
+    ReadWritePaths = [ "/var/export/playground" ];
+  };
+}

hosts/by-name/servo/services/export/sftpgo_external_auth_hook

@@ -0,0 +1,23 @@
+#!/usr/bin/env nix-shell
+#!nix-shell -i bash -p coreutils
+# vim: set filetype=bash :
+#
+# available environment variables:
+# - SFTPGO_AUTHD_USERNAME
+# - SFTPGO_AUTHD_USER
+# - SFTPGO_AUTHD_IP
+# - SFTPGO_AUTHD_PROTOCOL  = { "DAV", "FTP", "HTTP", "SSH" }
+# - SFTPGO_AUTHD_PASSWORD
+# - SFTPGO_AUTHD_PUBLIC_KEY
+# - SFTPGO_AUTHD_KEYBOARD_INTERACTIVE
+# - SFTPGO_AUTHD_TLS_CERT
+#
+#
+# call with <script_name> /path/to/fail/response.json /path/to/success/response.json
+
+
+if [ "$SFTPGO_AUTHD_USERNAME" = "anonymous" ]; then
+  cat "$2"
+else
+  cat "$1"
+fi

hosts/by-name/servo/services/freshrss.nix

@@ -16,7 +16,7 @@
     mode = "0400";
   };
   sane.persist.sys.plaintext = [
-    { user = "freshrss"; group = "freshrss"; directory = "/var/lib/freshrss"; }
+    { user = "freshrss"; group = "freshrss"; path = "/var/lib/freshrss"; }
   ];
 
   services.freshrss.enable = true;
@@ -59,5 +59,5 @@
     # the routing is handled by services.freshrss.virtualHost
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet.CNAME."rss" = "native";
+  sane.dns.zones."uninsane.org".inet.CNAME."rss" = "native";
 }

hosts/by-name/servo/services/gitea.nix

@@ -1,18 +1,16 @@
+# config options: <https://docs.gitea.io/en-us/administration/config-cheat-sheet/>
 { config, pkgs, lib, ... }:
 
 {
   sane.persist.sys.plaintext = [
     # TODO: mode? could be more granular
-    { user = "git"; group = "gitea"; directory = "/var/lib/gitea"; }
+    { user = "git"; group = "gitea"; path = "/var/lib/gitea"; }
   ];
   services.gitea.enable = true;
   services.gitea.user = "git";  # default is 'gitea'
   services.gitea.database.type = "postgres";
   services.gitea.database.user = "git";
   services.gitea.appName = "Perfectly Sane Git";
-  services.gitea.domain = "git.uninsane.org";
-  services.gitea.rootUrl = "https://git.uninsane.org/";
-  services.gitea.settings.session.COOKIE_SECURE = true;
   # services.gitea.disableRegistration = true;
 
   # gitea doesn't create the git user
@@ -27,9 +25,13 @@
   };
 
   services.gitea.settings = {
+    # options: "Trace", "Debug", "Info", "Warn", "Error", "Critical"
+    log.LEVEL = "Warn";
     server = {
       # options: "home", "explore", "organizations", "login" or URL fragment (or full URL)
       LANDING_PAGE = "explore";
+      DOMAIN = "git.uninsane.org";
+      ROOT_URL = "https://git.uninsane.org/";
     };
     service = {
       # timeout for email approval. 5760 = 4 days
@@ -44,6 +46,7 @@
       ENABLE_CAPTCHA = true;
       NOREPLY_ADDRESS = "noreply.anonymous.git@uninsane.org";
     };
+    session.COOKIE_SECURE = true;
     repository = {
       DEFAULT_BRANCH = "master";
     };
@@ -58,6 +61,8 @@
     };
     #"ui.meta" = ... to customize html author/description/etc
     mailer = {
+      # alternative is to use nixos-level config:
+      # services.gitea.mailerPasswordFile = ...
       ENABLED = true;
       MAILER_TYPE = "sendmail";
       FROM = "notify.git@uninsane.org";
@@ -69,8 +74,6 @@
       FORMAT = "RFC3339";
     };
   };
-  # options: "Trace", "Debug", "Info", "Warn", "Error", "Critical"
-  services.gitea.settings.log.LEVEL = "Warn";
 
   systemd.services.gitea.serviceConfig = {
     # nix default is AF_UNIX AF_INET AF_INET6.
@@ -95,5 +98,12 @@
     };
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet.CNAME."git" = "native";
+  sane.dns.zones."uninsane.org".inet.CNAME."git" = "native";
+
+  sane.ports.ports."22" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-git@git.uninsane.org";
+  };
 }

hosts/by-name/servo/services/goaccess.nix

@@ -64,5 +64,5 @@
     };
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet.CNAME."sink" = "native";
+  sane.dns.zones."uninsane.org".inet.CNAME."sink" = "native";
 }

hosts/by-name/servo/services/ipfs.nix

@@ -12,7 +12,7 @@ lib.mkIf false # i don't actively use ipfs anymore
 {
   sane.persist.sys.plaintext = [
     # TODO: mode? could be more granular
-    { user = "261"; group = "261"; directory = "/var/lib/ipfs"; }
+    { user = "261"; group = "261"; path = "/var/lib/ipfs"; }
   ];
 
   networking.firewall.allowedTCPPorts = [ 4001 ];
@@ -34,7 +34,7 @@ lib.mkIf false # i don't actively use ipfs anymore
     };
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet.CNAME."ipfs" = "native";
+  sane.dns.zones."uninsane.org".inet.CNAME."ipfs" = "native";
 
   # services.ipfs.enable = true;
   services.kubo.localDiscovery = true;

hosts/by-name/servo/services/jackett.nix

@@ -3,7 +3,7 @@
 {
   sane.persist.sys.plaintext = [
     # TODO: mode? we only need this to save Indexer creds ==> migrate to config?
-    { user = "root"; group = "root"; directory = "/var/lib/jackett"; }
+    { user = "root"; group = "root"; path = "/var/lib/jackett"; }
   ];
   services.jackett.enable = true;
 
@@ -24,9 +24,10 @@
     locations."/" = {
       # proxyPass = "http://ovpns.uninsane.org:9117";
       proxyPass = "http://10.0.1.6:9117";
+      recommendedProxySettings = true;
     };
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet.CNAME."jackett" = "native";
+  sane.dns.zones."uninsane.org".inet.CNAME."jackett" = "native";
 }
 

hosts/by-name/servo/services/jellyfin.nix

@@ -1,16 +1,76 @@
+# configuration options (today i don't store my config in nix):
+#
+# - jellyfin-web can be statically configured (result/share/jellyfin-web/config.json)
+#   - <https://jellyfin.org/docs/general/clients/web-config>
+#   - configure server list, plugins, "menuLinks", colors
+#
+# - jellfyin server is configured in /var/lib/jellfin/
+#   - root/default/<LibraryType>/
+#     - <LibraryName>.mblink: contains the directory name where this library lives
+#     - options.xml: contains preferences which were defined in the web UI during import
+#       - e.g. `EnablePhotos`, `EnableChapterImageExtraction`, etc.
+#   - config/encoding.xml: transcoder settings
+#   - config/system.xml: misc preferences like log file duration, audiobook resume settings, etc.
+#   - data/jellyfin.db: maybe account definitions? internal state?
+
 { config, lib, ... }:
 
-# TODO: re-enable after migrating media dir to /var/lib/uninsane/media
-# else it's too spammy
-lib.mkIf false
 {
-  networking.firewall.allowedUDPPorts = [
-    1900 7359 # DLNA: https://jellyfin.org/docs/general/networking/index.html
-  ];
+  # https://jellyfin.org/docs/general/networking/index.html
+  sane.ports.ports."1900" = {
+    protocol = [ "udp" ];
+    visibleTo.lan = true;
+    description = "colin-upnp-for-jellyfin";
+  };
+  sane.ports.ports."7359" = {
+    protocol = [ "udp" ];
+    visibleTo.lan = true;
+    description = "colin-jellyfin-specific-client-discovery";
+    # ^ not sure if this is necessary: copied this port from nixos jellyfin.openFirewall
+  };
+  # not sure if 8096/8920 get used either:
+  sane.ports.ports."8096" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    description = "colin-jellyfin-http-lan";
+  };
+  sane.ports.ports."8920" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    description = "colin-jellyfin-https-lan";
+  };
+
   sane.persist.sys.plaintext = [
-    # TODO: mode? could be more granular
-    { user = "jellyfin"; group = "jellyfin"; directory = "/var/lib/jellyfin"; }
+    { user = "jellyfin"; group = "jellyfin"; mode = "0700"; path = "/var/lib/jellyfin"; }
   ];
+  sane.fs."/var/lib/jellyfin/config/logging.json" = {
+    # "Emby.Dlna" logging: <https://jellyfin.org/docs/general/networking/dlna>
+    symlink.text = ''
+      {
+        "Serilog": {
+          "MinimumLevel": {
+            "Default": "Information",
+            "Override": {
+              "Microsoft": "Warning",
+              "System": "Warning",
+              "Emby.Dlna": "Debug",
+              "Emby.Dlna.Eventing": "Debug"
+            }
+          },
+          "WriteTo": [
+            {
+              "Name": "Console",
+              "Args": {
+                "outputTemplate": "[{Timestamp:HH:mm:ss}] [{Level:u3}] [{ThreadId}] {SourceContext}: {Message:lj}{NewLine}{Exception}"
+              }
+            }
+          ],
+          "Enrich": [ "FromLogContext", "WithThreadId" ]
+        }
+      }
+    '';
+    wantedBeforeBy = [ "jellyfin.service" ];
+  };
 
   # Jellyfin multimedia server
   # this is mostly taken from the official jellfin.org docs
@@ -61,7 +121,7 @@ lib.mkIf false
     };
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet.CNAME."jelly" = "native";
+  sane.dns.zones."uninsane.org".inet.CNAME."jelly" = "native";
 
   services.jellyfin.enable = true;
 }

hosts/by-name/servo/services/kiwix-serve.nix

@@ -13,5 +13,5 @@
     locations."/".proxyPass = "http://127.0.0.1:8013";
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet.CNAME."w" = "native";
+  sane.dns.zones."uninsane.org".inet.CNAME."w" = "native";
 }

hosts/by-name/servo/services/komga.nix

@@ -0,0 +1,22 @@
+{ config, ... }:
+let
+  svc-cfg = config.services.komga;
+  inherit (svc-cfg) user group port stateDir;
+in
+{
+  sane.persist.sys.plaintext = [
+    { inherit user group; mode = "0700"; path = stateDir; }
+  ];
+
+  services.komga.enable = true;
+  services.komga.port = 11319;  # chosen at random
+
+  services.nginx.virtualHosts."komga.uninsane.org" = {
+    addSSL = true;
+    enableACME = true;
+    locations."/" = {
+      proxyPass = "http://127.0.0.1:${builtins.toString port}";
+    };
+  };
+  sane.dns.zones."uninsane.org".inet.CNAME."komga" = "native";
+}

hosts/by-name/servo/services/lemmy.nix

@@ -0,0 +1,85 @@
+# docs:
+# - <repo:LemmyNet/lemmy:docker/federation/nginx.conf>
+# - <repo:LemmyNet/lemmy:docker/nginx.conf>
+# - <repo:LemmyNet/lemmy-ansible:templates/nginx.conf>
+
+{ config, lib, pkgs, ... }:
+let
+  inherit (builtins) toString;
+  inherit (lib) mkForce;
+  uiPort = 1234;  # default ui port is 1234
+  backendPort = 8536; # default backend port is 8536
+  #^ i guess the "backend" port is used for federation?
+  pict-rs = pkgs.pict-rs.overrideAttrs (upstream: {
+    # as of v 0.4.2, all non-GIF video is forcibly transcoded.
+    # that breaks lemmy, because of the request latency.
+    # and it eats up hella CPU.
+    # pict-rs is iffy around video altogether: mp4 seems the best supported.
+    postPatch = (upstream.postPatch or "") + ''
+      substituteInPlace src/validate.rs \
+        --replace 'if transcode_options.needs_reencode() {' 'if false {'
+    '';
+  });
+in {
+  services.lemmy = {
+    enable = true;
+    settings.hostname = "lemmy.uninsane.org";
+    # federation.debug forces outbound federation queries to be run synchronously
+    # N.B.: this option might not be read for 0.17.0+? <https://github.com/LemmyNet/lemmy/blob/c32585b03429f0f76d1e4ff738786321a0a9df98/RELEASES.md#upgrade-instructions>
+    # settings.federation.debug = true;
+    settings.port = backendPort;
+    ui.port = uiPort;
+    database.createLocally = true;
+    nginx.enable = true;
+  };
+
+  systemd.services.lemmy.serviceConfig = {
+    # fix to use a normal user so we can configure perms correctly
+    DynamicUser = mkForce false;
+    User = "lemmy";
+    Group = "lemmy";
+  };
+  systemd.services.lemmy.environment = {
+    RUST_BACKTRACE = "full";
+    # RUST_LOG = "debug";
+    # RUST_LOG = "trace";
+    # upstream defaults LEMMY_DATABASE_URL = "postgres:///lemmy?host=/run/postgresql";
+    # - Postgres complains that we didn't specify a user
+    # lemmy formats the url as:
+    # - postgres://{user}:{password}@{host}:{port}/{database}
+    # SO suggests (https://stackoverflow.com/questions/3582552/what-is-the-format-for-the-postgresql-connection-string-url):
+    # - postgresql://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]
+    # LEMMY_DATABASE_URL = "postgres://lemmy@/run/postgresql";  # connection to server on socket "/run/postgresql/.s.PGSQL.5432" failed: FATAL:  database "run/postgresql" does not exist
+    # LEMMY_DATABASE_URL = "postgres://lemmy?host=/run/postgresql";  # no PostgreSQL user name specified in startup packet
+    # LEMMY_DATABASE_URL = mkForce "postgres://lemmy@?host=/run/postgresql";  # WORKS
+    LEMMY_DATABASE_URL = mkForce "postgres://lemmy@/lemmy?host=/run/postgresql";
+  };
+  users.groups.lemmy = {};
+  users.users.lemmy = {
+    group = "lemmy";
+    isSystemUser = true;
+  };
+
+  services.nginx.virtualHosts."lemmy.uninsane.org" = {
+    forceSSL = true;
+    enableACME = true;
+  };
+
+  sane.dns.zones."uninsane.org".inet.CNAME."lemmy" = "native";
+
+  #v DO NOT REMOVE: defaults to 0.3, instead of latest, so always need to explicitly set this.
+  services.pict-rs.package = pict-rs;
+
+  # pict-rs configuration is applied in this order:
+  # - via toml
+  # - via env vars (overrides everything above)
+  # - via CLI flags (overrides everything above)
+  # some of the CLI flags have defaults, making it the only actual way to configure certain things even when docs claim otherwise.
+  # CLI args: <https://git.asonix.dog/asonix/pict-rs#user-content-running>
+  systemd.services.pict-rs.serviceConfig.ExecStart = lib.mkForce (lib.concatStringsSep " " [
+    "${lib.getBin pict-rs}/bin/pict-rs run"
+    "--media-max-frame-count" (builtins.toString (30*60*60))
+    "--media-process-timeout 120"
+    "--media-enable-full-video true"  # allow audio
+  ]);
+}

hosts/by-name/servo/services/matrix/default.nix

@@ -1,19 +1,17 @@
-# docs: https://nixos.wiki/wiki/Matrix
-# docs: https://nixos.org/manual/nixos/stable/index.html#module-services-matrix-synapse
+# docs: <https://nixos.wiki/wiki/Matrix>
+# docs: <https://nixos.org/manual/nixos/stable/index.html#module-services-matrix-synapse>
+# example config: <https://github.com/matrix-org/synapse/blob/develop/docs/sample_config.yaml>
 { config, lib, pkgs, ... }:
 
 {
   imports = [
     ./discord-puppet.nix
-    # ./irc.nix
+    ./irc.nix
     ./signal.nix
   ];
 
-  # allow synapse to read the registration files of its appservices
-  users.users.matrix-synapse.extraGroups = [ "mautrix-signal" ];
-
   sane.persist.sys.plaintext = [
-    { user = "matrix-synapse"; group = "matrix-synapse"; directory = "/var/lib/matrix-synapse"; }
+    { user = "matrix-synapse"; group = "matrix-synapse"; path = "/var/lib/matrix-synapse"; }
   ];
   services.matrix-synapse.enable = true;
   # this changes the default log level from INFO to WARN.
@@ -44,11 +42,14 @@
     }
   ];
 
+  services.matrix-synapse.settings.x_forwarded = true;  # because we proxy matrix behind nginx
+  services.matrix-synapse.settings.max_upload_size = "100M";  # default is "50M"
+
   services.matrix-synapse.settings.admin_contact = "admin.matrix@uninsane.org";
   services.matrix-synapse.settings.registrations_require_3pid = [ "email" ];
 
   services.matrix-synapse.extraConfigFiles = [
-    config.sops.secrets.matrix_synapse_secrets.path
+    config.sops.secrets."matrix_synapse_secrets.yaml".path
   ];
 
   # services.matrix-synapse.extraConfigFiles = [builtins.toFile "matrix-synapse-extra-config" ''
@@ -97,6 +98,10 @@
 
     locations."/" = {
       proxyPass = "http://127.0.0.1:8008";
+      extraConfig = ''
+        # allow uploading large files (matrix enforces a separate limit, downstream)
+        client_max_body_size  512m;
+      '';
     };
     # redirect browsers to the web client.
     # i don't think native matrix clients ever fetch the root.
@@ -127,13 +132,13 @@
     };
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet = {
+  sane.dns.zones."uninsane.org".inet = {
     CNAME."matrix" = "native";
     CNAME."web.matrix" = "native";
   };
 
 
-  sops.secrets."matrix_synapse_secrets" = {
+  sops.secrets."matrix_synapse_secrets.yaml" = {
     owner = config.users.users.matrix-synapse.name;
   };
 }

hosts/by-name/servo/services/matrix/discord-puppet.nix

@@ -1,7 +1,12 @@
 { lib, ... }:
+
+# XXX mx-discord-puppet uses nodejs_14 which is EOL
+# - mx-discord-puppet is abandoned upstream _and_ in nixpkgs
+# - recommended to use mautrix-discord: <https://github.com/NixOS/nixpkgs/pull/200462>
+lib.mkIf false
 {
   sane.persist.sys.plaintext = [
-    { user = "matrix-synapse"; group = "matrix-synapse"; directory = "/var/lib/mx-puppet-discord"; }
+    { user = "matrix-synapse"; group = "matrix-synapse"; path = "/var/lib/mx-puppet-discord"; }
   ];
 
   services.matrix-synapse.settings.app_service_config_files = [

hosts/by-name/servo/services/matrix/irc-no-reveal-bridge.patch

@@ -0,0 +1,13 @@
+diff --git a/src/irc/ConnectionInstance.ts b/src/irc/ConnectionInstance.ts
+index 688036ca..3373fa27 100644
+--- a/src/irc/ConnectionInstance.ts
++++ b/src/irc/ConnectionInstance.ts
+@@ -149,7 +149,7 @@ export class ConnectionInstance {
+         if (this.dead) {
+             return Promise.resolve();
+         }
+-        ircReason = ircReason || reason;
++        ircReason = "bye"; // don't reveal through the IRC quit message that we're a bridge
+         log.info(
+             "disconnect()ing %s@%s - %s", this.nick, this.domain, reason
+         );

hosts/by-name/servo/services/matrix/irc-no-reveal-mxid.patch

@@ -0,0 +1,50 @@
+diff --git a/config.schema.yml b/config.schema.yml
+index 2e71c8d6..42ba8ba1 100644
+--- a/config.schema.yml
++++ b/config.schema.yml
+@@ -433,7 +433,7 @@ properties:
+                                     type: "boolean"
+                                 realnameFormat:
+                                     type: "string"
+-                                    enum: ["mxid","reverse-mxid"]
++                                    enum: ["mxid","reverse-mxid","localpart"]
+                                 ipv6:
+                                     type: "object"
+                                     properties:
+diff --git a/src/irc/IdentGenerator.ts b/src/irc/IdentGenerator.ts
+index 7a2b5cf1..50f7815a 100644
+--- a/src/irc/IdentGenerator.ts
++++ b/src/irc/IdentGenerator.ts
+@@ -74,6 +74,9 @@ export class IdentGenerator {
+         else if (server.getRealNameFormat() === "reverse-mxid") {
+             realname = IdentGenerator.sanitiseRealname(IdentGenerator.switchAroundMxid(matrixUser));
+         }
++        else if (server.getRealNameFormat() == "localpart") {
++            realname = IdentGenerator.sanitiseRealname(matrixUser.localpart);
++        }
+         else {
+             throw Error('Invalid value for realNameFormat');
+         }
+diff --git a/src/irc/IrcServer.ts b/src/irc/IrcServer.ts
+index 2af73ab4..895b9783 100644
+--- a/src/irc/IrcServer.ts
++++ b/src/irc/IrcServer.ts
+@@ -101,7 +101,7 @@ export interface IrcServerConfig {
+         };
+         lineLimit: number;
+         userModes?: string;
+-        realnameFormat?: "mxid"|"reverse-mxid";
++        realnameFormat?: "mxid"|"reverse-mxid"|"localpart";
+         pingTimeoutMs: number;
+         pingRateMs: number;
+         kickOn: {
+@@ -289,7 +289,7 @@ export class IrcServer {
+         return this.config.ircClients.userModes || "";
+     }
+ 
+-    public getRealNameFormat(): "mxid"|"reverse-mxid" {
++    public getRealNameFormat(): "mxid"|"reverse-mxid"|"localpart" {
+         return this.config.ircClients.realnameFormat || "mxid";
+     }
+ 
+

hosts/by-name/servo/services/matrix/irc.nix

@@ -1,21 +1,123 @@
+# config docs:
+# - <https://github.com/matrix-org/matrix-appservice-irc/blob/develop/config.sample.yaml>
+#       probably want to remove that.
 { config, lib, ... }:
 
+let
+  ircServer = { name, additionalAddresses ? [], sasl ? true, port ? 6697 }: let
+    lowerName = lib.toLower name;
+  in {
+    # XXX sasl: appservice doesn't support NickServ identification (only SASL, or PASS if sasl = false)
+    inherit name additionalAddresses sasl port;
+    ssl = true;
+    botConfig = {
+      # bot has no presence in IRC channel; only real Matrix users
+      enabled = false;
+      # this is the IRC username/nickname *of the bot* (not visible in channels): not of the end-user.
+      # the irc username/nick of a mapped Matrix user is determined further down in `ircClients` section.
+      # if `enabled` is false, then this name probably never shows up on the IRC side (?)
+      nick = "uninsane";
+      username = "uninsane";
+      joinChannelsIfNoUsers = false;
+    };
+    dynamicChannels = {
+      enabled = true;
+      aliasTemplate = "#irc_${lowerName}_$CHANNEL";
+      published = false;  # false => irc rooms aren't listed in homeserver public rooms list
+      federate = false;  # false => Matrix users from other homeservers can't join IRC channels
+    };
+    ircClients = {
+      nickTemplate = "$LOCALPARTsane";  # @colin:uninsane.org (Matrix) -> colinsane (IRC)
+      realnameFormat = "reverse-mxid";  # @colin:uninsane.org (Matrix) -> org.uninsane:colin (IRC)
+      # realnameFormat = "localpart";  # @colin:uninsane.org (Matrix) -> colin (IRC)  -- but requires the mxid patch below
+      # by default, Matrix will convert messages greater than (3) lines into a pastebin-like URL to send to IRC.
+      lineLimit = 20;
+      # Rizon in particular allows only 4 connections from one IP before a 30min ban.
+      # that's effectively reduced to 2 during a netsplit, or maybe during a restart.
+      #   - https://wiki.rizon.net/index.php?title=Connection/Session_Limit_Exemptions
+      # especially, misconfigurations elsewhere in this config may cause hundreds of connections
+      # so this is a safeguard.
+      maxClients = 2;
+      # don't have the bridge disconnect me from IRC when idle.
+      idleTimeout = 0;
+      concurrentReconnectLimit = 2;
+      reconnectIntervalMs = 60000;
+      kickOn = {
+        # remove Matrix user from room when...
+        channelJoinFailure = false;
+        ircConnectionFailure = false;
+        userQuit = true;
+      };
+    };
+    matrixClients = {
+      userTemplate = "@irc_${lowerName}_$NICK";  # the :uninsane.org part is appended automatically
+    };
+
+    # this will let this user message the appservice with `!join #<IRCChannel>` and the rest "Just Works"
+    "@colin:uninsane.org" = "admin";
+
+    membershipLists = {
+      enabled = true;
+      global = {
+        ircToMatrix = {
+          initial = true;
+          incremental = true;
+          requireMatrixJoined = false;
+        };
+        matrixToIrc = {
+          initial = true;
+          incremental = true;
+        };
+      };
+      ignoreIdleUsersOnStartup = {
+        enabled = false;  # false => always bridge users, even if idle
+      };
+    };
+    # sync room description?
+    bridgeInfoState = {
+      enabled = true;
+      initial = true;
+    };
+
+    # for per-user IRC password:
+    # - invite @irc_${lowerName}_NickServ:uninsane.org to a DM and type `help`  => register
+    # - invite the matrix-appservice-irc user to a DM and type `!help`   => add PW to database
+    # to validate that i'm authenticated on the IRC network, DM @irc_${lowerName}_NickServ:uninsane.org:
+    # - send: `STATUS colinsane`
+    # - response should be `3`: "user recognized as owner via password identification"
+    # passwordEncryptionKeyPath = "/path/to/privkey";  # appservice will generate its own if unspecified
+  };
+in
 {
+
+  nixpkgs.overlays = [
+    (next: prev: {
+      matrix-appservice-irc = prev.matrix-appservice-irc.overrideAttrs (super: {
+        patches = super.patches or [] ++ [
+          ./irc-no-reveal-bridge.patch
+          # ./irc-no-reveal-mxid.patch
+        ];
+      });
+    })
+  ];
+
   sane.persist.sys.plaintext = [
     # TODO: mode?
-    # user and group are both "matrix-appservice-irc"
-    { user = "993"; group = "992"; directory = "/var/lib/matrix-appservice-irc"; }
+    { user = "matrix-appservice-irc"; group = "matrix-appservice-irc"; path = "/var/lib/matrix-appservice-irc"; }
   ];
 
+  # XXX: matrix-appservice-irc PreStart tries to chgrp the registration.yml to matrix-synapse,
+  # which requires matrix-appservice-irc to be of that group
+  users.users.matrix-appservice-irc.extraGroups = [ "matrix-synapse" ];
+  # weird race conditions around registration.yml mean we want matrix-synapse to be of matrix-appservice-irc group too.
+  users.users.matrix-synapse.extraGroups = [ "matrix-appservice-irc" ];
+
   services.matrix-synapse.settings.app_service_config_files = [
     "/var/lib/matrix-appservice-irc/registration.yml"  # auto-created by irc appservice
   ];
 
-  # note: Rizon allows only FOUR simultaneous IRC connections per IP: https://wiki.rizon.net/index.php?title=Connection/Session_Limit_Exemptions
-  # Rizon supports CertFP for auth: https://wiki.rizon.net/index.php?title=CertFP
   services.matrix-appservice-irc.enable = true;
   services.matrix-appservice-irc.registrationUrl = "http://127.0.0.1:8009";
-  # settings documented here: https://github.com/matrix-org/matrix-appservice-irc/blob/develop/config.sample.yaml
   services.matrix-appservice-irc.settings = {
     homeserver = {
       url = "http://127.0.0.1:8008";
@@ -28,70 +130,38 @@
 
     ircService = {
       servers = {
-        "irc.rizon.net" = {
-          name = "Rizon";
-          port = 6697;  # SSL port
-          ssl = true;
-          sasl = true;  # appservice doesn't support NickServ identification
-          botConfig = {
-            # bot has no presence in IRC channel; only real Matrix users
-            enabled = false;
-            # nick = "UninsaneDotOrg";
-            nick = "uninsane";
-            username = "uninsane";
-          };
-          dynamicChannels = {
-            enabled = true;
-            aliasTemplate = "#irc_rizon_$CHANNEL";
-          };
-          ircClients = {
-            nickTemplate = "$LOCALPARTsane";
-            # by default, Matrix will convert messages greater than (3) lines into a pastebin-like URL to send to IRC.
-            lineLimit = 20;
-          };
-          matrixClients = {
-            userTemplate = "@irc_rizon_$NICK";  # the :uninsane.org part is appended automatically
-          };
-
-          # this will let this user message the appservice with `!join #<IRCChannel>` and the rest "Just Works"
-          "@colin:uninsane.org" = "admin";
-
-          membershipLists = {
-            enabled = true;
-            global = {
-              ircToMatrix = {
-                initial = true;
-                incremental = true;
-                requireMatrixJoined = false;
-              };
-              matrixToIrc = {
-                initial = true;
-                incremental = true;
-              };
-            };
-          };
-          # sync room description?
-          bridgeInfoState = {
-            enabled = true;
-            initial = true;
-          };
-
-          # hardcoded mappings, for when dynamicChannels fails us. TODO: probably safe to remove these.
-          # mappings = {
-          #   "#chat" = {
-          #     roomIds = [ "!GXJSOTdbtxRboGtDep:uninsane.org" ];
-          #   };
-          #   # BakaBT requires account registration, which i think means my user needs to be added before the appservice user
-          #   "#BakaBT" = {
-          #     roomIds = [ "!feZKttuYuHilqPFSkD:uninsane.org" ];
-          #   };
-          # };
-          # for per-user IRC password:
-          #   invite @irc_rizon_NickServ:uninsane.org to a DM and type `help`  => register
-          #   invite the matrix-appservice-irc user to a DM and type `!help`   => add PW to database
-          # passwordEncryptionKeyPath = "/path/to/privkey";  # appservice will generate its own if unspecified
+        "irc.esper.net" = ircServer {
+          name = "esper";
+          sasl = false;
+          # notable channels:
+          # - #merveilles
         };
+        "irc.libera.chat" = ircServer {
+          name = "libera";
+          sasl = false;
+          # notable channels:
+          # - #hare
+        };
+        "irc.myanonamouse.net" = ircServer {
+          name = "MyAnonamouse";
+          additionalAddresses = [ "irc2.myanonamouse.net" ];
+          sasl = false;
+        };
+        "irc.oftc.net" = ircServer {
+          name = "oftc";
+          sasl = false;
+          # notable channels:
+          # - #sxmo
+          # - #sxmo-offtopic
+        };
+        "irc.rizon.net" = ircServer { name = "Rizon"; };
       };
     };
   };
+
+  systemd.services.matrix-appservice-irc.serviceConfig = {
+    # XXX 2023/06/20: nixos specifies this + @aio and @memlock as forbidden
+    # the service actively uses at least one of these, and both of them are fairly innocuous
+    SystemCallFilter = lib.mkForce "~@clock @cpu-emulation @debug @keyring @module @mount @obsolete @raw-io @setuid @swap";
+  };
 }

hosts/by-name/servo/services/matrix/signal.nix

@@ -3,10 +3,13 @@
 { config, pkgs, ... }:
 {
   sane.persist.sys.plaintext = [
-    { user = "mautrix-signal"; group = "mautrix-signal"; directory = "/var/lib/mautrix-signal"; }
-    { user = "signald"; group = "signald"; directory = "/var/lib/signald"; }
+    { user = "mautrix-signal"; group = "mautrix-signal"; path = "/var/lib/mautrix-signal"; }
+    { user = "signald"; group = "signald"; path = "/var/lib/signald"; }
   ];
 
+  # allow synapse to read the registration file
+  users.users.matrix-synapse.extraGroups = [ "mautrix-signal" ];
+
   services.signald.enable = true;
   services.mautrix-signal.enable = true;
   services.mautrix-signal.environmentFile =
@@ -27,7 +30,6 @@
   };
 
   sops.secrets."mautrix_signal_env" = {
-    format = "binary";
     mode = "0440";
     owner = config.users.users.mautrix-signal.name;
     group = config.users.users.matrix-synapse.name;

hosts/by-name/servo/services/navidrome.nix

@@ -2,7 +2,7 @@
 
 {
   sane.persist.sys.plaintext = [
-    { user = "navidrome"; group = "navidrome"; directory = "/var/lib/navidrome"; }
+    { user = "navidrome"; group = "navidrome"; path = "/var/lib/navidrome"; }
   ];
   services.navidrome.enable = true;
   services.navidrome.settings = {
@@ -36,5 +36,5 @@
     locations."/".proxyPass = "http://127.0.0.1:4533";
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet.CNAME."music" = "native";
+  sane.dns.zones."uninsane.org".inet.CNAME."music" = "native";
 }

hosts/by-name/servo/services/nginx.nix

@@ -13,7 +13,19 @@ let
 in
 {
 
-  networking.firewall.allowedTCPPorts = [ 80 443 ];
+  sane.ports.ports."80" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    visibleTo.ovpn = true;  # so that letsencrypt can procure a cert for the mx record
+    description = "colin-http-uninsane.org";
+  };
+  sane.ports.ports."443" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-https-uninsane.org";
+  };
 
   services.nginx.enable = true;
   services.nginx.appendConfig = ''
@@ -89,7 +101,8 @@ in
     };
 
     # allow ActivityPub clients to discover how to reach @user@uninsane.org
-    # TODO: waiting on https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3361/
+    # see: https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3361/
+    # not sure this makes sense while i run multiple AP services (pleroma, lemmy)
     # locations."/.well-known/nodeinfo" = {
     #   proxyPass = "http://127.0.0.1:4000";
     #   extraConfig = pleromaExtraConfig;
@@ -122,8 +135,8 @@ in
 
   sane.persist.sys.plaintext = [
     # TODO: mode?
-    { user = "acme"; group = "acme"; directory = "/var/lib/acme"; }
-    { user = "colin"; group = "users"; directory = "/var/www/sites"; }
+    { user = "acme"; group = "acme"; path = "/var/lib/acme"; }
+    { user = "colin"; group = "users"; path = "/var/www/sites"; }
   ];
 
   # let's encrypt default chain looks like:

hosts/by-name/servo/services/nixserve.nix

@@ -14,8 +14,8 @@
     '';
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet.CNAME."nixcache" = "native";
+  sane.dns.zones."uninsane.org".inet.CNAME."nixcache" = "native";
 
   sane.services.nixserve.enable = true;
-  sane.services.nixserve.sopsFile = ../../../../secrets/servo.yaml;
+  sane.services.nixserve.secretKeyFile = config.sops.secrets.nix_serve_privkey.path;
 }

hosts/by-name/servo/services/pict-rs.nix

@@ -0,0 +1,23 @@
+# pict-rs is an image database/store used by Lemmy.
+# i don't explicitly activate it here -- just adjust its defaults to be a bit friendlier
+{ config, lib, ... }:
+let
+  cfg = config.services.pict-rs;
+in
+{
+  sane.persist.sys.plaintext = lib.mkIf cfg.enable [
+    { user = "pict-rs"; group = "pict-rs"; path = cfg.dataDir; }
+  ];
+
+  systemd.services.pict-rs.serviceConfig = {
+    # fix to use a normal user so we can configure perms correctly
+    DynamicUser = lib.mkForce false;
+    User = "pict-rs";
+    Group = "pict-rs";
+  };
+  users.groups.pict-rs = {};
+  users.users.pict-rs = {
+    group = "pict-rs";
+    isSystemUser = true;
+  };
+}

hosts/by-name/servo/services/pleroma.nix

@@ -1,14 +1,21 @@
 # docs:
-# - https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/networking/pleroma.nix
-# - https://docs.pleroma.social/backend/configuration/cheatsheet/
+# - <https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/networking/pleroma.nix>
+# - <https://docs.pleroma.social/backend/configuration/cheatsheet/>
+# example config:
+# - <https://git.pleroma.social/pleroma/pleroma/-/blob/develop/config/config.exs>
 #
-# to run it in a oci-container: https://github.com/barrucadu/nixfiles/blob/master/services/pleroma.nix
+# to run it in a oci-container: <https://github.com/barrucadu/nixfiles/blob/master/services/pleroma.nix>
+#
+# admin frontend: <https://fed.uninsane.org/pleroma/admin>
 { config, pkgs, ... }:
 
+let
+  logLevel = "warn";
+  # logLevel = "debug";
+in
 {
   sane.persist.sys.plaintext = [
-    # TODO: mode? could be more granular
-    { user = "pleroma"; group = "pleroma"; directory = "/var/lib/pleroma"; }
+    { user = "pleroma"; group = "pleroma"; path = "/var/lib/pleroma"; }
   ];
   services.pleroma.enable = true;
   services.pleroma.secretConfigFile = config.sops.secrets.pleroma_secrets.path;
@@ -56,6 +63,7 @@
       database: "pleroma",
       hostname: "localhost",
       pool_size: 10,
+      prepare: :named,
       parameters: [
           plan_cache_mode: "force_custom_plan"
       ]
@@ -96,10 +104,22 @@
       backends: [{ExSyslogger, :ex_syslogger}]
 
     config :logger, :ex_syslogger,
-      level: :warn
-    #  level: :debug
+      level: :${logLevel}
+
+    # policies => list of message rewriting facilities to be enabled
+    # transparence => whether to publish these rules in node_info (and /about)
+    config :pleroma, :mrf,
+      policies: [Pleroma.Web.ActivityPub.MRF.SimplePolicy],
+      transparency: true
+
+    # reject => { host, reason }
+    config :pleroma, :mrf_simple,
+      reject: [ {"threads.net", "megacorp"}, {"*.threads.net", "megacorp"} ]
+      # reject: [ [host: "threads.net", reason: "megacorp"], [host: "*.threads.net", reason: "megacorp"] ]
 
     # XXX colin: not sure if this actually _does_ anything
+    # better to steal emoji from other instances?
+    # - <https://docs.pleroma.social/backend/configuration/cheatsheet/#mrf_steal_emoji>
     config :pleroma, :emoji,
       shortcode_globs: ["/emoji/**/*.png"],
       groups: [
@@ -148,6 +168,7 @@
     # inherit kTLS;
     locations."/" = {
       proxyPass = "http://127.0.0.1:4000";
+      recommendedProxySettings = true;
       # documented: https://git.pleroma.social/pleroma/pleroma/-/blob/develop/installation/pleroma.nginx
       extraConfig = ''
         # XXX colin: this block is in the nixos examples: i don't understand all of it
@@ -166,23 +187,24 @@
         add_header Referrer-Policy same-origin;
         add_header X-Download-Options noopen;
 
-        proxy_http_version 1.1;
-        proxy_set_header Upgrade $http_upgrade;
-        proxy_set_header Connection "upgrade";
-        # proxy_set_header Host $http_host;
-        proxy_set_header Host $host;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        # proxy_http_version 1.1;
+        # proxy_set_header Upgrade $http_upgrade;
+        # proxy_set_header Connection "upgrade";
+        # # proxy_set_header Host $http_host;
+        # proxy_set_header Host $host;
+        # proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 
         # colin: added this due to Pleroma complaining in its logs
         # proxy_set_header X-Real-IP $remote_addr;
         # proxy_set_header X-Forwarded-Proto $scheme;
 
+        # NB: this defines the maximum upload size
         client_max_body_size 16m;
       '';
     };
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet.CNAME."fed" = "native";
+  sane.dns.zones."uninsane.org".inet.CNAME."fed" = "native";
 
   sops.secrets."pleroma_secrets" = {
     owner = config.users.users.pleroma.name;

hosts/by-name/servo/services/postgres.nix

@@ -1,12 +1,39 @@
-{ ... }:
+{ pkgs, ... }:
 
+let
+  GiB = n: MiB 1024*n;
+  MiB = n: KiB 1024*n;
+  KiB = n: 1024*n;
+in
 {
   sane.persist.sys.plaintext = [
     # TODO: mode?
-    { user = "postgres"; group = "postgres"; directory = "/var/lib/postgresql"; }
+    { user = "postgres"; group = "postgres"; path = "/var/lib/postgresql"; }
   ];
   services.postgresql.enable = true;
-  # services.postgresql.dataDir = "/opt/postgresql/13";
+
+  # HOW TO UPDATE:
+  # postgres version updates are manual and require intervention.
+  # - `sane-stop-all-servo`
+  # - `systemctl start postgresql`
+  # - as `sudo su postgres`:
+  #   - `cd /var/log/postgresql`
+  #   - `pg_dumpall > state.sql`
+  #   - `echo placeholder > <new_version>`  # to prevent state from being created earlier than we want
+  # - then, atomically:
+  #   - update the `services.postgresql.package` here
+  #   - `dataDir` is atomically updated to match package; don't touch
+  #   - `nixos-rebuild --flake . switch ; sane-stop-all-servo`
+  # - `sudo rm -rf /var/lib/postgresql/<new_version>`
+  # - `systemctl start postgresql`
+  # - as `sudo su postgres`:
+  #   - `cd /var/lib/postgreql`
+  #   - `psql -f state.sql`
+  # - restart dependent services (maybe test one at a time)
+
+  services.postgresql.package = pkgs.postgresql_15;
+
+
   # XXX colin: for a proper deploy, we'd want to include something for Pleroma here too.
   # services.postgresql.initialScript = pkgs.writeText "synapse-init.sql" ''
   #   CREATE ROLE "matrix-synapse" WITH LOGIN PASSWORD '<password goes here>';
@@ -17,10 +44,33 @@
   #     LC_CTYPE = "C";
   # '';
 
-  # TODO: perf tuning
+  # perf tuning
   # - for recommended values see: <https://pgtune.leopard.in.ua/>
   # - for official docs (sparse), see: <https://www.postgresql.org/docs/11/config-setting.html#CONFIG-SETTING-CONFIGURATION-FILE>
-  # services.postgresql.settings = { ... }
+  services.postgresql.settings = {
+    # DB Version: 15
+    # OS Type: linux
+    # DB Type: web
+    # Total Memory (RAM): 32 GB
+    # CPUs num: 12
+    # Data Storage: ssd
+    max_connections = 200;
+    shared_buffers = "8GB";
+    effective_cache_size = "24GB";
+    maintenance_work_mem = "2GB";
+    checkpoint_completion_target = 0.9;
+    wal_buffers = "16MB";
+    default_statistics_target = 100;
+    random_page_cost = 1.1;
+    effective_io_concurrency = 200;
+    work_mem = "10485kB";
+    min_wal_size = "1GB";
+    max_wal_size = "4GB";
+    max_worker_processes = 12;
+    max_parallel_workers_per_gather = 4;
+    max_parallel_workers = 12;
+    max_parallel_maintenance_workers = 4;
+  };
 
   # daily backups to /var/backup
   services.postgresqlBackup.enable = true;

hosts/by-name/servo/services/prosody.nix

@@ -10,14 +10,31 @@
 lib.mkIf false
 {
   sane.persist.sys.plaintext = [
-    { user = "prosody"; group = "prosody"; directory = "/var/lib/prosody"; }
-  ];
-  networking.firewall.allowedTCPPorts = [
-    5222  # XMPP client -> server
-    5269  # XMPP server -> server
-    5280  # bosh
-    5281  # Prosody HTTPS port  (necessary?)
+    { user = "prosody"; group = "prosody"; path = "/var/lib/prosody"; }
   ];
+  sane.ports.ports."5222" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-xmpp-client-to-server";
+  };
+  sane.ports.ports."5269" = {
+    protocol = [ "tcp" ];
+    visibleTo.wan = true;
+    description = "colin-xmpp-server-to-server";
+  };
+  sane.ports.ports."5280" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-xmpp-bosh";
+  };
+  sane.ports.ports."5281" = {
+    protocol = [ "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-xmpp-prosody-https";  # necessary?
+  };
 
   # provide access to certs
   users.users.prosody.extraGroups = [ "nginx" ];

hosts/by-name/servo/services/transmission.nix

@@ -3,7 +3,7 @@
 {
   sane.persist.sys.plaintext = [
     # TODO: mode? we need this specifically for the stats tracking in .config/
-    { user = "transmission"; group = "transmission"; directory = "/var/lib/transmission"; }
+    { user = "transmission"; group = "transmission"; path = "/var/lib/transmission"; }
   ];
   services.transmission.enable = true;
   services.transmission.settings = {
@@ -27,7 +27,7 @@
     # units in kBps
     speed-limit-down = 3000;
     speed-limit-down-enabled = true;
-    speed-limit-up = 300;
+    speed-limit-up = 600;
     speed-limit-up-enabled = true;
 
     # see: https://git.zknt.org/mirror/transmission/commit/cfce6e2e3a9b9d31a9dafedd0bdc8bf2cdb6e876?lang=bg-BG
@@ -75,6 +75,6 @@
     };
   };
 
-  sane.services.trust-dns.zones."uninsane.org".inet.CNAME."bt" = "native";
+  sane.dns.zones."uninsane.org".inet.CNAME."bt" = "native";
 }
 

hosts/by-name/servo/services/trust-dns.nix

@@ -1,17 +1,27 @@
-{ config, pkgs, ... }:
+{ config, lib, pkgs, ... }:
 
 {
-  sane.services.trust-dns.enable = true;
+  services.trust-dns.enable = true;
 
-  sane.services.trust-dns.listenAddrsIPv4 = [
+  services.trust-dns.settings.listen_addrs_ipv4 = [
     # specify each address explicitly, instead of using "*".
     # this ensures responses are sent from the address at which the request was received.
-    "192.168.0.5"
+    config.sane.hosts.by-name."servo".lan-ip
     "10.0.1.5"
   ];
-  sane.services.trust-dns.quiet = true;
+  # don't bind to IPv6 until i explicitly test that stack
+  services.trust-dns.settings.listen_addrs_ipv6 = [];
+  services.trust-dns.quiet = true;
+  # services.trust-dns.debug = true;
 
-  sane.services.trust-dns.zones."uninsane.org".TTL = 900;
+  sane.ports.ports."53" = {
+    protocol = [ "udp" "tcp" ];
+    visibleTo.lan = true;
+    visibleTo.wan = true;
+    description = "colin-dns-hosting";
+  };
+
+  sane.dns.zones."uninsane.org".TTL = 900;
 
   # SOA record structure: <https://en.wikipedia.org/wiki/SOA_record#Structure>
   # SOA MNAME RNAME (... rest)
@@ -21,7 +31,7 @@
   # Refresh = how frequently secondary NS should query master
   # Retry = how long secondary NS should wait until re-querying master after a failure (must be < Refresh)
   # Expire = how long secondary NS should continue to reply to queries after master fails (> Refresh + Retry)
-  sane.services.trust-dns.zones."uninsane.org".inet = {
+  sane.dns.zones."uninsane.org".inet = {
     SOA."@" = ''
       ns1.uninsane.org. admin-dns.uninsane.org. (
                                       2022122101 ; Serial
@@ -30,17 +40,20 @@
                                       7d         ; Expire
                                       5m)        ; Negative response TTL
     '';
-    TXT."rev" = "2022122101";
+    TXT."rev" = "2023052901";
+
+    CNAME."native" = "%CNAMENATIVE%";
+    A."@" =      "%ANATIVE%";
+    A."wan" = "%AWAN%";
+    A."servo.lan" = config.sane.hosts.by-name."servo".lan-ip;
 
     # XXX NS records must also not be CNAME
     # it's best that we keep this identical, or a superset of, what org. lists as our NS.
     # so, org. can specify ns2/ns3 as being to the VPN, with no mention of ns1. we provide ns1 here.
-    A."ns1" =    "%NATIVE%";
+    A."ns1" =    "%ANATIVE%";
     A."ns2" =    "185.157.162.178";
     A."ns3" =    "185.157.162.178";
     A."ovpns" =  "185.157.162.178";
-    A."native" = "%NATIVE%";
-    A."@" =      "%NATIVE%";
     NS."@" = [
       "ns1.uninsane.org."
       "ns2.uninsane.org."
@@ -48,20 +61,78 @@
     ];
   };
 
-  sane.services.trust-dns.zones."uninsane.org".file =
-    "/var/lib/trust-dns/uninsane.org.zone";
+  services.trust-dns.settings.zones = [ "uninsane.org" ];
 
-  systemd.services.trust-dns.preStart = let
-    sed = "${pkgs.gnused}/bin/sed";
-    zone-dir = "/var/lib/trust-dns";
-    zone-out = "${zone-dir}/uninsane.org.zone";
-    zone-template = pkgs.writeText "uninsane.org.zone.in" config.sane.services.trust-dns.generatedZones."uninsane.org";
-  in ''
-    # make WAN records available to trust-dns
-    mkdir -p ${zone-dir}
-    ip=$(cat '${config.sane.services.dyn-dns.ipPath}')
-    ${sed} s/%NATIVE%/$ip/ ${zone-template} > ${zone-out}
-  '';
+  services.trust-dns.package =
+    let
+      sed = "${pkgs.gnused}/bin/sed";
+      zone-dir = "/var/lib/trust-dns";
+      zone-wan = "${zone-dir}/wan/uninsane.org.zone";
+      zone-lan = "${zone-dir}/lan/uninsane.org.zone";
+      zone-template = pkgs.writeText "uninsane.org.zone.in" config.sane.dns.zones."uninsane.org".rendered;
+    in pkgs.writeShellScriptBin "named" ''
+      # compute wan/lan values
+      mkdir -p ${zone-dir}/{ovpn,wan,lan}
+      wan=$(cat '${config.sane.services.dyn-dns.ipPath}')
+      lan=${config.sane.hosts.by-name."servo".lan-ip}
+
+      # create specializations that resolve native.uninsane.org to different CNAMEs
+      ${sed} s/%AWAN%/$wan/ ${zone-template} \
+        | ${sed} s/%CNAMENATIVE%/wan/ \
+        | ${sed} s/%ANATIVE%/$wan/ \
+        > ${zone-wan}
+      ${sed} s/%AWAN%/$wan/ ${zone-template} \
+        | ${sed} s/%CNAMENATIVE%/servo.lan/ \
+        | ${sed} s/%ANATIVE%/$lan/ \
+        > ${zone-lan}
+
+      # launch the different interfaces, separately
+      ${pkgs.trust-dns}/bin/named --port 53 --zonedir ${zone-dir}/wan/ $@ &
+      WANPID=$!
+      ${pkgs.trust-dns}/bin/named --port 1053 --zonedir ${zone-dir}/lan/ $@ &
+      LANPID=$!
+
+      # wait until any of the processes exits, then kill them all and exit error
+      while kill -0 $WANPID $LANPID ; do
+        sleep 5
+      done
+      kill $WANPID $LANPID
+      exit 1
+    '';
+
+  systemd.services.trust-dns.serviceConfig = {
+    DynamicUser = lib.mkForce false;
+    User = "trust-dns";
+    Group = "trust-dns";
+  };
+  users.groups.trust-dns = {};
+  users.users.trust-dns = {
+    group = "trust-dns";
+    isSystemUser = true;
+  };
 
   sane.services.dyn-dns.restartOnChange = [ "trust-dns.service" ];
+
+  networking.nat.enable = true;
+  networking.nat.extraCommands = ''
+    # redirect incoming DNS requests from LAN addresses
+    #   to the LAN-specialized DNS service
+    # N.B.: use the `nixos-*` chains instead of e.g. PREROUTING
+    #   because they get cleanly reset across activations or `systemctl restart firewall`
+    #   instead of accumulating cruft
+    iptables -t nat -A nixos-nat-pre -p udp --dport 53 \
+      -m iprange --src-range 10.78.76.0-10.78.79.255 \
+      -j DNAT --to-destination :1053
+    iptables -t nat -A nixos-nat-pre -p tcp --dport 53 \
+      -m iprange --src-range 10.78.76.0-10.78.79.255 \
+      -j DNAT --to-destination :1053
+  '';
+
+  sane.ports.ports."1053" = {
+    # because the NAT above redirects in nixos-nat-pre, LAN requests behave as though they arrived on the external interface at the redirected port.
+    # TODO: try nixos-nat-post instead?
+    protocol = [ "udp" "tcp" ];
+    visibleTo.lan = true;
+    description = "colin-redirected-dns-for-lan-namespace";
+  };
 }

hosts/common/cross.nix

@@ -1,22 +0,0 @@
-{ config, ... }:
-
-let
-  mkCrossFrom = localSystem: pkgs: import pkgs.path {
-    inherit localSystem;
-    crossSystem = pkgs.stdenv.hostPlatform.system;
-    inherit (config.nixpkgs) config overlays;
-  };
-in
-{
-  # the configuration of which specific package set `pkgs.cross` refers to happens elsewhere;
-  # here we just define them all.
-  nixpkgs.overlays = [
-    (next: prev: {
-      # non-emulated packages build *from* local *for* target.
-      # for large packages like the linux kernel which are expensive to build under emulation,
-      # the config can explicitly pull such packages from `pkgs.cross` to do more efficient cross-compilation.
-      crossFrom."x86_64-linux" = mkCrossFrom "x86_64-linux" next;
-      crossFrom."aarch64-linux" = mkCrossFrom "aarch64-linux" next;
-    })
-  ];
-}

hosts/common/default.nix

@@ -1,74 +1,78 @@
-{ pkgs, ... }:
+{ lib, pkgs, ... }:
 {
   imports = [
-    ./cross.nix
     ./feeds.nix
     ./fs.nix
     ./hardware.nix
     ./home
-    ./i2p.nix
     ./ids.nix
     ./machine-id.nix
     ./net.nix
+    ./nix-path
     ./persist.nix
-    ./programs.nix
+    ./programs
     ./secrets.nix
     ./ssh.nix
-    ./users.nix
+    ./users
     ./vpn.nix
   ];
 
   sane.nixcache.enable-trusted-keys = true;
-  sane.programs.sysadminUtils.enableFor.system = true;
-  sane.programs.consoleUtils.enableFor.user.colin = true;
-
-  # some services which use private directories error if the parent (/var/lib/private) isn't 700.
-  sane.fs."/var/lib/private".dir.acl.mode = "0700";
+  sane.nixcache.enable = lib.mkDefault true;
+  sane.persist.enable = lib.mkDefault true;
+  sane.programs.sysadminUtils.enableFor.system = lib.mkDefault true;
+  sane.programs.consoleUtils.enableFor.user.colin = lib.mkDefault true;
 
   nixpkgs.config.allowUnfree = true;
+  nixpkgs.config.allowBroken = true;  # NIXPKGS_ALLOW_BROKEN
 
   # time.timeZone = "America/Los_Angeles";
   time.timeZone = "Etc/UTC";  # DST is too confusing for me => use a stable timezone
 
   # allow `nix flake ...` command
+  # TODO: is this still required?
   nix.extraOptions = ''
     experimental-features = nix-command flakes
   '';
-  # allow `nix-shell` (and probably nix-index?) to locate our patched and custom packages
-  nix.nixPath = [
-    "nixpkgs=${pkgs.path}"
-    "nixpkgs-overlays=${../..}/overlays"
-  ];
+  # hardlinks identical files in the nix store to save 25-35% disk space.
+  # unclear _when_ this occurs. it's not a service.
+  # does the daemon continually scan the nix store?
+  # does the builder use some content-addressed db to efficiently dedupe?
+  nix.settings.auto-optimise-store = true;
 
-  fonts = {
-    enableDefaultFonts = true;
-    fonts = with pkgs; [ font-awesome twitter-color-emoji hack-font ];
-    fontconfig.enable = true;
-    fontconfig.defaultFonts = {
-      emoji = [ "Font Awesome 6 Free" "Twitter Color Emoji" ];
-      monospace = [ "Hack" ];
-      serif = [ "DejaVu Serif" ];
-      sansSerif = [ "DejaVu Sans" ];
-    };
+  systemd.services.nix-daemon.serviceConfig = {
+    # the nix-daemon manages nix builders
+    # kill nix-daemon subprocesses when systemd-oomd detects an out-of-memory condition
+    # see:
+    # - nixos PR that enabled systemd-oomd: <https://github.com/NixOS/nixpkgs/pull/169613>
+    # - systemd's docs on these properties: <https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html#ManagedOOMSwap=auto%7Ckill>
+    #
+    # systemd's docs warn that without swap, systemd-oomd might not be able to react quick enough to save the system.
+    # see `man oomd.conf` for further tunables that may help.
+    #
+    # alternatively, apply this more broadly with `systemd.oomd.enableSystemSlice = true` or `enableRootSlice`
+    # TODO: also apply this to the guest user's slice (user-1100.slice)
+    # TODO: also apply this to distccd
+    ManagedOOMMemoryPressure = "kill";
+    ManagedOOMSwap = "kill";
+  };
+
+
+  system.activationScripts.nixClosureDiff = {
+    supportsDryActivation = true;
+    text = ''
+      # show which packages changed versions or are new/removed in this upgrade
+      # source: <https://github.com/luishfonseca/dotfiles/blob/32c10e775d9ec7cc55e44592a060c1c9aadf113e/modules/upgrade-diff.nix>
+      ${pkgs.nvd}/bin/nvd --nix-bin-dir=${pkgs.nix}/bin diff /run/current-system "$systemConfig"
+    '';
   };
 
   # disable non-required packages like nano, perl, rsync, strace
   environment.defaultPackages = [];
 
-  # programs.vim.defaultEditor = true;
-  environment.variables = {
-    EDITOR = "vim";
-    # git claims it should use EDITOR, but it doesn't!
-    GIT_EDITOR = "vim";
-    # TODO: these should be moved to `home.sessionVariables` (home-manager)
-    # Electron apps should use native wayland backend:
-    #   https://nixos.wiki/wiki/Slack#Wayland
-    # Discord under sway crashes with this.
-    # NIXOS_OZONE_WL = "1";
-    # LIBGL_ALWAYS_SOFTWARE = "1";
-  };
-
   # dconf docs: <https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/desktop_migration_and_administration_guide/profiles>
+  # this lets programs temporarily write user-level dconf settings (aka gsettings).
+  # they're written to ~/.config/dconf/user, unless `DCONF_PROFILE` is set to something other than the default of /etc/dconf/profile/user
   # find keys/values with `dconf dump /`
   programs.dconf.enable = true;
   programs.dconf.packages = [
@@ -81,6 +85,7 @@
       '';
     })
   ];
+  # sane.programs.glib.enableFor.user.colin = true;  # for `gsettings`
 
   # link debug symbols into /run/current-system/sw/lib/debug
   # hopefully picked up by gdb automatically?

hosts/common/feeds.nix

@@ -1,3 +1,13 @@
+# where to find good stuff?
+# - podcasts w/ a community: <https://lemmyverse.net/communities?query=podcast>
+# - podcast rec thread: <https://lemmy.ml/post/1565858>
+#
+# candidates:
+# - The Nonlinear Library (podcast): <https://forum.effectivealtruism.org/posts/JTZTBienqWEAjGDRv/listen-to-more-ea-content-with-the-nonlinear-library>
+#   - has ~10 posts per day, text-to-speech; i would need better tagging before adding this
+# - <https://www.metaculus.com/questions/11102/introducing-the-metaculus-journal-podcast/>
+#   - dead since 2022/10 - 2023/03
+
 { lib, sane-data, ... }:
 let
   hourly = { freq = "hourly"; };
@@ -50,24 +60,42 @@ let
     (fromDb "lexfridman.com/podcast" // rat)
     ## Astral Codex Ten
     (fromDb "sscpodcast.libsyn.com" // rat)
+    ## Less Wrong Curated
+    (fromDb "feeds.libsyn.com/421877" // rat)
     ## Econ Talk
     (fromDb "feeds.simplecast.com/wgl4xEgL" // rat)
     ## Cory Doctorow -- both podcast & text entries
     (fromDb "craphound.com" // pol)
+    ## Maggie Killjoy -- referenced by Cory Doctorow
+    (fromDb "omny.fm/shows/cool-people-who-did-cool-stuff" // pol)
+    ## also Maggie Killjoy
+    (fromDb "feeds.megaphone.fm/behindthebastards" // pol)
+    ## Jennifer Briney
     (fromDb "congressionaldish.libsyn.com" // pol)
+    (fromDb "werenotwrong.fireside.fm" // pol)
+    (fromDb "usefulidiots.substack.com" // pol)
+    # (mkPod "https://podcasts.la.utexas.edu/this-is-democracy/feed/podcast/" // pol // weekly)
     ## Civboot -- https://anchor.fm/civboot
     (fromDb "anchor.fm/s/34c7232c/podcast/rss" // tech)
     ## Emerge: making sense of what's next -- <https://www.whatisemerging.com/emergepodcast>
     (mkPod "https://anchor.fm/s/21bc734/podcast/rss" // pol // infrequent)
     (fromDb "feeds.feedburner.com/80000HoursPodcast" // rat)
+    ## Daniel Huberman on sleep
+    (fromDb "feeds.megaphone.fm/hubermanlab" // uncat)
+    ## Multidisciplinary Association for Psychedelic Studies
+    (fromDb "mapspodcast.libsyn.com" // uncat)
     (fromDb "allinchamathjason.libsyn.com" // pol)
-    (fromDb "acquired.libsyn.com" // tech)
-    # The Intercept - Deconstructed; also available: <rss.acast.com/deconstructed>
-    (fromDb "rss.prod.firstlook.media/deconstructed/podcast.rss" // pol)
+    (fromDb "feeds.transistor.fm/acquired" // tech)
+    ## ACQ2 - more "Acquired" episodes
+    (fromDb "acquiredlpbonussecretsecret.libsyn.com" // tech)
+    # The Intercept - Deconstructed
+    (fromDb "rss.acast.com/deconstructed")
+    # (fromDb "rss.prod.firstlook.media/deconstructed/podcast.rss" // pol)  #< possible URL rot
     ## The Daily
     (mkPod "https://feeds.simplecast.com/54nAGcIl" // pol // daily)
-    # The Intercept - Intercepted; also available: <https://rss.acast.com/intercepted-with-jeremy-scahill>
-    (fromDb "rss.prod.firstlook.media/intercepted/podcast.rss" // pol)
+    # The Intercept - Intercepted
+    (fromDb "rss.acast.com/intercepted-with-jeremy-scahill")
+    # (fromDb "rss.prod.firstlook.media/intercepted/podcast.rss" // pol)  #< possible URL rot
     (fromDb "podcast.posttv.com/itunes/post-reports.xml" // pol)
     ## Eric Weinstein
     (fromDb "rss.art19.com/the-portal" // rat)
@@ -85,25 +113,48 @@ let
     (fromDb "feeds.megaphone.fm/recodedecode" // tech)
     ## Matrix (chat) Live
     (fromDb "feed.podbean.com/matrixlive/feed.xml" // tech)
+    (fromDb "cast.postmarketos.org" // tech)
+    (fromDb "podcast.thelinuxexp.com" // tech)
     ## Michael Malice - Your Welcome -- also available here: <https://origin.podcastone.com/podcast?categoryID2=2232>
     (fromDb "rss.art19.com/your-welcome" // pol)
     (fromDb "seattlenice.buzzsprout.com" // pol)
     ## Sci-Fi? has Peter Watts; author of No Moods, Ads or Cutesy Fucking Icons (rifters.com)
     (fromDb "talesfromthebridge.buzzsprout.com" // tech)
+    ## UnNamed Reverse Engineering Podcast
+    (fromDb "reverseengineering.libsyn.com/rss" // tech)
+    ## The Witch Trials of J.K. Rowling
+    ## - <https://www.thefp.com/witchtrials>
+    (mkPod "https://feeds.megaphone.fm/RUNMED9919162779" // pol // infrequent)
+    ## Atlas Obscura
+    (fromDb "feeds.simplecast.com/xKJ93w_w" // uncat)
+    ## Ezra Klein Show
+    (fromDb "feeds.simplecast.com/82FI35Px" // pol)
+    ## Wireshark Podcast o_0
+    (fromDb "sharkbytes.transistor.fm" // tech)
+    ## 3/4 German; 1/4 eps are English
+    (fromDb "omegataupodcast.net" // tech)
+    ## Lateral with Tom Scott
+    (mkPod "https://audioboom.com/channels/5097784.rss" // tech)
   ];
 
   texts = [
     # AGGREGATORS (> 1 post/day)
     (fromDb "lwn.net" // tech)
-    (fromDb "lesswrong.com" // rat)
-    (fromDb "econlib.org" // pol)
+    # (fromDb "lesswrong.com" // rat)
+    # (fromDb "econlib.org" // pol)
 
     # AGGREGATORS (< 1 post/day)
     (fromDb "palladiummag.com" // uncat)
     (fromDb "profectusmag.com" // uncat)
     (fromDb "semiaccurate.com" // tech)
     (mkText "https://linuxphoneapps.org/blog/atom.xml" // tech // infrequent)
+    (fromDb "tuxphones.com" // tech)
     (fromDb "spectrum.ieee.org" // tech)
+    (fromDb "theregister.com" // tech)
+    (fromDb "thisweek.gnome.org" // tech)
+    # more nixos stuff here, but unclear how to subscribe: <https://nixos.org/blog/categories.html>
+    (mkText "https://nixos.org/blog/announcements-rss.xml" // tech // infrequent)
+    (mkText "https://nixos.org/blog/stories-rss.xml" // tech // weekly)
     ## n.b.: quality RSS list here: <https://forum.merveilles.town/thread/57/share-your-rss-feeds%21-6/>
     (mkText "https://forum.merveilles.town/rss.xml" // pol // infrequent)
 
@@ -111,9 +162,12 @@ let
     (fromDb "rifters.com/crawl" // uncat)
 
     # DEVELOPERS
+    (fromDb "blog.jmp.chat" // tech)
     (fromDb "uninsane.org" // tech)
+    (fromDb "ascii.textfiles.com" // tech)  # Jason Scott
+    (fromDb "xn--gckvb8fzb.com" // tech)
     (fromDb "mg.lol" // tech)
-    (fromDb "drewdevault.com" // tech)
+    # (fromDb "drewdevault.com" // tech)
     ## Ken Shirriff
     (fromDb "righto.com" // tech)
     ## shared blog by a few NixOS devs, notably onny
@@ -128,9 +182,14 @@ let
     (fromDb "ianthehenry.com" // tech)
     (fromDb "bitbashing.io" // tech)
     (fromDb "idiomdrottning.org" // uncat)
+    (mkText "http://boginjr.com/feed" // tech // infrequent)
     (mkText "https://anish.lakhwara.com/home.html" // tech // weekly)
     (fromDb "jefftk.com" // tech)
     (fromDb "pomeroyb.com" // tech)
+    # (mkText "https://til.simonwillison.net/tils/feed.atom" // tech // weekly)
+
+    # TECH PROJECTS
+    (fromDb "blog.rust-lang.org" // tech)
 
     # (TECH; POL) COMMENTATORS
     ## Matt Webb -- engineering-ish, but dreamy
@@ -147,7 +206,8 @@ let
     (fromDb "lynalden.com" // pol)
     (fromDb "austinvernon.site" // tech)
     (mkSubstack "oversharing" // pol // daily)
-    (mkSubstack "doomberg" // tech // weekly)
+    (mkSubstack "byrnehobart" // pol // infrequent)
+    # (mkSubstack "doomberg" // tech // weekly)  # articles are all pay-walled
     ## David Rosenthal
     (fromDb "blog.dshr.org" // pol)
     ## Matt Levine
@@ -155,6 +215,7 @@ let
     (fromDb "stpeter.im/atom.xml" // pol)
     ## Peter Saint-Andre -- side project of stpeter.im
     (fromDb "philosopher.coach" // rat)
+    (fromDb "morningbrew.com/feed" // pol)
 
     # RATIONALITY/PHILOSOPHY/ETC
     (mkSubstack "samkriss" // humor // infrequent)
@@ -173,10 +234,16 @@ let
     (fromDb "sideways-view.com" // rat)
     ## Sean Carroll
     (fromDb "preposterousuniverse.com" // rat)
+    (mkSubstack "eliqian" // rat // weekly)
+    (mkText "https://acoup.blog/feed" // rat // weekly)
+    (fromDb "mindingourway.com" // rat)
 
     ## mostly dating topics. not advice, or humor, but looking through a social lens
     (fromDb "putanumonit.com" // rat)
 
+    # LOCAL
+    (fromDb "capitolhillseattle.com" // pol)
+
     # CODE
     # (mkText "https://github.com/Kaiteki-Fedi/Kaiteki/commits/master.atom" // tech // infrequent)
   ];
@@ -186,6 +253,7 @@ let
     (fromDb "xkcd.com" // img // humor)
     (fromDb "pbfcomics.com" // img // humor)
     # (mkImg "http://dilbert.com/feed" // humor // daily)
+    (fromDb "poorlydrawnlines.com/feed" // img // humor)
 
     # ART
     (fromDb "miniature-calendar.com" // img // art // daily)

hosts/common/fs.nix

@@ -1,74 +1,143 @@
-{ pkgs, ... }:
+# docs
+# - x-systemd options: <https://www.freedesktop.org/software/systemd/man/systemd.mount.html>
 
-let sshOpts = rec {
-  fsType = "fuse.sshfs";
-  optionsBase = [
-    "x-systemd.automount"
-    "_netdev"
-    "user"
-    "identityfile=/home/colin/.ssh/id_ed25519"
-    "allow_other"
-    "default_permissions"
-  ];
-  optionsColin = optionsBase ++ [
-    "transform_symlinks"
-    "idmap=user"
-    "uid=1000"
-    "gid=100"
-  ];
+{ lib, pkgs, sane-lib, ... }:
 
-  optionsRoot = optionsBase ++ [
-    # we don't transform_symlinks because that breaks the validity of remote /nix stores
-    "sftp_server=/run/wrappers/bin/sudo\\040/run/current-system/sw/libexec/sftp-server"
-  ];
-};
+let
+  fsOpts = rec {
+    common = [
+      "_netdev"
+      "noatime"
+      "user"  # allow any user with access to the device to mount the fs
+      "x-systemd.requires=network-online.target"
+      "x-systemd.after=network-online.target"
+      "x-systemd.mount-timeout=10s"  # how long to wait for mount **and** how long to wait for unmount
+    ];
+    auto = [ "x-systemd.automount" ];
+    noauto = [ "noauto" ];  # don't mount as part of remote-fs.target
+    wg = [
+      "x-systemd.requires=wireguard-wg-home.service"
+      "x-systemd.after=wireguard-wg-home.service"
+    ];
+
+    ssh = common ++ [
+      "identityfile=/home/colin/.ssh/id_ed25519"
+      "allow_other"
+      "default_permissions"
+    ];
+    sshColin = ssh ++ [
+      "transform_symlinks"
+      "idmap=user"
+      "uid=1000"
+      "gid=100"
+    ];
+    sshRoot = ssh ++ [
+      # we don't transform_symlinks because that breaks the validity of remote /nix stores
+      "sftp_server=/run/wrappers/bin/sudo\\040/run/current-system/sw/libexec/sftp-server"
+    ];
+    # in the event of hunt NFS mounts, consider:
+    # - <https://unix.stackexchange.com/questions/31979/stop-broken-nfs-mounts-from-locking-a-directory>
+
+    # NFS options: <https://linux.die.net/man/5/nfs>
+    # actimeo=n  = how long (in seconds) to cache file/dir attributes (default: 3-60s)
+    # bg         = retry failed mounts in the background
+    # retry=n    = for how many minutes `mount` will retry NFS mount operation
+    # soft       = on "major timeout", report I/O error to userspace
+    # retrans=n  = how many times to retry a NFS request before giving userspace a "server not responding" error (default: 3)
+    # timeo=n    = number of *deciseconds* to wait for a response before retrying it (default: 600)
+    #              note: client uses a linear backup, so the second request will have double this timeout, then triple, etc.
+    nfs = common ++ [
+      # "actimeo=10"
+      "bg"
+      "retrans=4"
+      "retry=0"
+      "soft"
+      "timeo=15"
+      "nofail"  # don't fail remote-fs.target when this mount fails  (not an option for sshfs else would be common)
+    ];
+  };
+  remoteHome = host: {
+    fileSystems."/mnt/${host}-home" = {
+      device = "colin@${host}:/home/colin";
+      fsType = "fuse.sshfs";
+      options = fsOpts.sshColin ++ fsOpts.noauto;
+      noCheck = true;
+    };
+    sane.fs."/mnt/${host}-home" = sane-lib.fs.wantedDir;
+  };
 in
-{
-  environment.pathsToLink = [
-    # needed to achieve superuser access for user-mounted filesystems (see optionsRoot above)
-    # we can only link whole directories here, even though we're only interested in pkgs.openssh
-    "/libexec"
-  ];
+lib.mkMerge [
+  {
+    # some services which use private directories error if the parent (/var/lib/private) isn't 700.
+    sane.fs."/var/lib/private".dir.acl.mode = "0700";
 
-  fileSystems."/mnt/servo-media-wan" = {
-    device = "colin@uninsane.org:/var/lib/uninsane/media";
-    inherit (sshOpts) fsType;
-    options = sshOpts.optionsColin;
-    noCheck = true;
-  };
-  fileSystems."/mnt/servo-media-lan" = {
-    device = "colin@servo:/var/lib/uninsane/media";
-    inherit (sshOpts) fsType;
-    options = sshOpts.optionsColin;
-    noCheck = true;
-  };
-  fileSystems."/mnt/servo-root-wan" = {
-    device = "colin@uninsane.org:/";
-    inherit (sshOpts) fsType;
-    options = sshOpts.optionsRoot;
-    noCheck = true;
-  };
-  fileSystems."/mnt/servo-root-lan" = {
-    device = "colin@servo:/";
-    inherit (sshOpts) fsType;
-    options = sshOpts.optionsRoot;
-    noCheck = true;
-  };
-  fileSystems."/mnt/desko-home" = {
-    device = "colin@desko:/home/colin";
-    inherit (sshOpts) fsType;
-    options = sshOpts.optionsColin;
-    noCheck = true;
-  };
-  fileSystems."/mnt/desko-root" = {
-    device = "colin@desko:/";
-    inherit (sshOpts) fsType;
-    options = sshOpts.optionsRoot;
-    noCheck = true;
-  };
+    # in-memory compressed RAM
+    # defaults to compressing at most 50% size of RAM
+    # claimed compression ratio is about 2:1
+    # - but on moby w/ zstd default i see 4-7:1  (ratio lowers as it fills)
+    # note that idle overhead is about 0.05% of capacity (e.g. 2B per 4kB page)
+    # docs: <https://www.kernel.org/doc/Documentation/blockdev/zram.txt>
+    #
+    # to query effectiveness:
+    # `cat /sys/block/zram0/mm_stat`. whitespace separated fields:
+    # - *orig_data_size*  (bytes)
+    # - *compr_data_size* (bytes)
+    # - mem_used_total    (bytes)
+    # - mem_limit         (bytes)
+    # - mem_used_max      (bytes)
+    # - *same_pages*  (pages which are e.g. all zeros (consumes no additional mem))
+    # - *pages_compacted*  (pages which have been freed thanks to compression)
+    # - huge_pages  (incompressible)
+    #
+    # see also:
+    # - `man zramctl`
+    zramSwap.enable = true;
+    # how much ram can be swapped into the zram device.
+    # this shouldn't be higher than the observed compression ratio.
+    # the default is 50% (why?)
+    # 100% should be "guaranteed" safe so long as the data is even *slightly* compressible.
+    # but it decreases working memory under the heaviest of loads by however much space the compressed memory occupies (e.g. 50% if 2:1; 25% if 4:1)
+    zramSwap.memoryPercent = 100;
 
-  environment.systemPackages = [
-    pkgs.sshfs-fuse
-  ];
-}
+    # fileSystems."/mnt/servo-nfs" = {
+    #   device = "servo-hn:/";
+    #   noCheck = true;
+    #   fsType = "nfs";
+    #   options = fsOpts.nfs ++ fsOpts.auto ++ fsOpts.wg;
+    # };
+    fileSystems."/mnt/servo-nfs/media" = {
+      device = "servo-hn:/media";
+      noCheck = true;
+      fsType = "nfs";
+      options = fsOpts.nfs ++ fsOpts.auto ++ fsOpts.wg;
+    };
+    fileSystems."/mnt/servo-nfs/playground" = {
+      device = "servo-hn:/playground";
+      noCheck = true;
+      fsType = "nfs";
+      options = fsOpts.nfs ++ fsOpts.auto ++ fsOpts.wg;
+    };
+    # fileSystems."/mnt/servo-media-nfs" = {
+    #   device = "servo-hn:/media";
+    #   noCheck = true;
+    #   fsType = "nfs";
+    #   options = fsOpts.common ++ fsOpts.auto;
+    # };
+    sane.fs."/mnt/servo-media" = sane-lib.fs.wantedSymlinkTo "/mnt/servo-nfs/media";
+
+    environment.pathsToLink = [
+      # needed to achieve superuser access for user-mounted filesystems (see optionsRoot above)
+      # we can only link whole directories here, even though we're only interested in pkgs.openssh
+      "/libexec"
+    ];
+
+    environment.systemPackages = [
+      pkgs.sshfs-fuse
+    ];
+  }
+
+  (remoteHome "desko")
+  (remoteHome "lappy")
+  (remoteHome "moby")
+]
 

hosts/common/hardware.nix

@@ -23,11 +23,15 @@
 
   # non-free firmware
   hardware.enableRedistributableFirmware = true;
-  services.fwupd.enable = true;
 
   # powertop will default to putting USB devices -- including HID -- to sleep after TWO SECONDS
   powerManagement.powertop.enable = false;
 
+  services.logind.extraConfig = ''
+    # don’t shutdown when power button is short-pressed
+    HandlePowerKey=ignore
+  '';
+
   # services.snapper.configs = {
   #   root = {
   #     subvolume = "/";

hosts/common/home/aerc.nix

@@ -1,11 +0,0 @@
-# Terminal UI mail client
-{ config, sane-lib, ... }:
-
-{
-  sops.secrets."aerc_accounts" = {
-    owner = config.users.users.colin.name;
-    sopsFile = ../../../secrets/universal/aerc_accounts.conf;
-    format = "binary";
-  };
-  sane.user.fs.".config/aerc/accounts.conf" = sane-lib.fs.wantedSymlinkTo config.sops.secrets.aerc_accounts.path;
-}

hosts/common/home/default.nix

@@ -1,23 +1,9 @@
 { ... }:
 {
   imports = [
-    ./aerc.nix
-    ./firefox.nix
-    ./gfeeds.nix
-    ./git.nix
-    ./gpodder.nix
-    ./keyring.nix
-    ./kitty.nix
-    ./libreoffice.nix
+    ./keyring
     ./mime.nix
-    ./mpv.nix
-    ./neovim.nix
-    ./newsflash.nix
-    ./splatmoji.nix
     ./ssh.nix
-    ./sublime-music.nix
-    ./vlc.nix
     ./xdg-dirs.nix
-    ./zsh
   ];
 }

hosts/common/home/firefox.nix

@@ -1,203 +0,0 @@
-# common settings to toggle (at runtime, in about:config):
-#   > security.ssl.require_safe_negotiation
-
-# librewolf is a forked firefox which patches firefox to allow more things
-# (like default search engines) to be configurable at runtime.
-# many of the settings below won't have effect without those patches.
-# see: https://gitlab.com/librewolf-community/settings/-/blob/master/distribution/policies.json
-
-{ config, lib, pkgs, sane-lib, ...}:
-with lib;
-let
-  cfg = config.sane.web-browser;
-  # allow easy switching between firefox and librewolf with `defaultSettings`, below
-  librewolfSettings = {
-    browser = pkgs.librewolf-unwrapped;
-    # browser = pkgs.librewolf-unwrapped.overrideAttrs (drv: {
-    #   # this allows side-loading unsigned addons
-    #   MOZ_REQUIRE_SIGNING = false;
-    # });
-    libName = "librewolf";
-    dotDir = ".librewolf";
-    cacheDir = ".cache/librewolf";  # TODO: is it?
-    desktop = "librewolf.desktop";
-  };
-  firefoxSettings = {
-    browser = pkgs.firefox-esr-unwrapped;
-    libName = "firefox";
-    dotDir = ".mozilla/firefox";
-    cacheDir = ".cache/mozilla";
-    desktop = "firefox.desktop";
-  };
-  defaultSettings = firefoxSettings;
-  # defaultSettings = librewolfSettings;
-
-  addon = name: extid: hash: pkgs.fetchFirefoxAddon {
-    inherit name hash;
-    url = "https://addons.mozilla.org/firefox/downloads/latest/${name}/latest.xpi";
-    # extid can be found by unar'ing the above xpi, and copying browser_specific_settings.gecko.id field
-    fixedExtid = extid;
-  };
-  localAddon = pkg: pkgs.fetchFirefoxAddon {
-    inherit (pkg) name;
-    src = "${pkg}/share/mozilla/extensions/\\{ec8030f7-c20a-464f-9b0e-13a3a9e97384\\}/${pkg.extid}.xpi";
-    fixedExtid = pkg.extid;
-  };
-
-  package = pkgs.wrapFirefox cfg.browser.browser {
-    # inherit the default librewolf.cfg
-    # it can be further customized via ~/.librewolf/librewolf.overrides.cfg
-    inherit (pkgs.librewolf-unwrapped) extraPrefsFiles;
-    inherit (cfg.browser) libName;
-
-    extraNativeMessagingHosts = [ pkgs.browserpass ];
-    # extraNativeMessagingHosts = [ pkgs.gopass-native-messaging-host ];
-
-    nixExtensions = concatMap (ext: optional ext.enable ext.package) (attrValues cfg.addons);
-
-    extraPolicies = {
-      NoDefaultBookmarks = true;
-      SearchEngines = {
-        Default = "DuckDuckGo";
-      };
-      AppUpdateURL = "https://localhost";
-      DisableAppUpdate = true;
-      OverrideFirstRunPage = "";
-      OverridePostUpdatePage = "";
-      DisableSystemAddonUpdate = true;
-      DisableFirefoxStudies = true;
-      DisableTelemetry = true;
-      DisableFeedbackCommands = true;
-      DisablePocket = true;
-      DisableSetDesktopBackground = false;
-
-      # remove many default search providers
-      # XXX this seems to prevent the `nixExtensions` from taking effect
-      # Extensions.Uninstall = [
-      #   "google@search.mozilla.org"
-      #   "bing@search.mozilla.org"
-      #   "amazondotcom@search.mozilla.org"
-      #   "ebay@search.mozilla.org"
-      #   "twitter@search.mozilla.org"
-      # ];
-      # XXX doesn't seem to have any effect...
-      # docs: https://github.com/mozilla/policy-templates#homepage
-      # Homepage = {
-      #   HomepageURL = "https://uninsane.org/";
-      #   StartPage = "homepage";
-      # };
-      # NewTabPage = true;
-    };
-  };
-
-  addonOpts = types.submodule {
-    options = {
-      package = mkOption {
-        type = types.package;
-      };
-      enable = mkOption {
-        type = types.bool;
-      };
-    };
-  };
-in
-{
-  options = {
-    sane.web-browser.browser = mkOption {
-      default = defaultSettings;
-      type = types.attrs;
-    };
-    sane.web-browser.persistData = mkOption {
-      description = "optional store name to which persist browsing data (like history)";
-      type = types.nullOr types.str;
-      default = null;
-    };
-    sane.web-browser.persistCache = mkOption {
-      description = "optional store name to which persist browser cache";
-      type = types.nullOr types.str;
-      default = "cryptClearOnBoot";
-    };
-    sane.web-browser.addons = mkOption {
-      type = types.attrsOf addonOpts;
-      default = {
-        # get names from:
-        # - ~/ref/nix-community/nur-combined/repos/rycee/pkgs/firefox-addons/generated-firefox-addons.nix
-        # `wget ...xpi`; `unar ...xpi`; `cat */manifest.json | jq '.browser_specific_settings.gecko.id'`
-        # browserpass-ce.package = addon "browserpass-ce" "browserpass@maximbaz.com" "sha256-sXgUBbRvMnRpeIW1MTkmTcoqtW/8RDXAkxAq1evFkpc=";
-        browserpass-extension.package = localAddon pkgs.browserpass-extension;
-        # TODO: build bypass-paywalls from source? it's mysteriously disappeared from the Mozilla store.
-        # bypass-paywalls-clean.package = addon "bypass-paywalls-clean" "{d133e097-46d9-4ecc-9903-fa6a722a6e0e}" "sha256-oUwdqdAwV3DezaTtOMx7A/s4lzIws+t2f08mwk+324k=";
-        ether-metamask.package = addon "ether-metamask" "webextension@metamask.io" "sha256-G+MwJDOcsaxYSUXjahHJmkWnjLeQ0Wven8DU/lGeMzA=";
-        i2p-in-private-browsing.package = addon "i2p-in-private-browsing" "i2ppb@eyedeekay.github.io" "sha256-dJcJ3jxeAeAkRvhODeIVrCflvX+S4E0wT/PyYzQBQWs=";
-        sidebery.package = addon "sidebery" "{3c078156-979c-498b-8990-85f7987dd929}" "sha256-YONfK/rIjlsrTgRHIt3km07Q7KnpIW89Z9r92ZSCc6w=";
-        sponsorblock.package = addon "sponsorblock" "sponsorBlocker@ajay.app" "sha256-hRsvLaAsVm3dALsTrJqHTNgRFAQcU7XSaGhr5G6+mFs=";
-        ublacklist.package = addon "ublacklist" "@ublacklist" "sha256-RqY5iHzbL2qizth7aguyOKWPyINXmrwOlf/OsfqAS48=";
-        ublock-origin.package = addon "ublock-origin" "uBlock0@raymondhill.net" "sha256-a/ivUmY1P6teq9x0dt4CbgHt+3kBsEMMXlOfZ5Hx7cg=";
-
-        browserpass-extension.enable = lib.mkDefault true;
-        # bypass-paywalls-clean.enable = lib.mkDefault true;
-        ether-metamask.enable = lib.mkDefault true;
-        i2p-in-private-browsing.enable = lib.mkDefault config.services.i2p.enable;
-        sidebery.enable = lib.mkDefault true;
-        sponsorblock.enable = lib.mkDefault true;
-        ublacklist.enable = lib.mkDefault true;
-        ublock-origin.enable = lib.mkDefault true;
-      };
-    };
-  };
-
-  config = {
-    sane.programs.web-browser = {
-      inherit package;
-      # TODO: define the persistence & fs config here
-    };
-    sane.programs.guiApps.suggestedPrograms = [ "web-browser" ];
-
-    # uBlock filter list configuration.
-    # specifically, enable the GDPR cookie prompt blocker.
-    # data.toOverwrite.filterLists is additive (i.e. it supplements the default filters)
-    # this configuration method is documented here:
-    # - <https://github.com/gorhill/uBlock/issues/2986#issuecomment-364035002>
-    # the specific attribute path is found via scraping ublock code here:
-    # - <https://github.com/gorhill/uBlock/blob/master/src/js/storage.js>
-    # - <https://github.com/gorhill/uBlock/blob/master/assets/assets.json>
-    sane.user.fs."${cfg.browser.dotDir}/managed-storage/uBlock0@raymondhill.net.json" = sane-lib.fs.wantedText ''
-      {
-       "name": "uBlock0@raymondhill.net",
-       "description": "ignored",
-       "type": "storage",
-       "data": {
-          "toOverwrite": "{\"filterLists\": [\"fanboy-cookiemonster\"]}"
-       }
-      }
-    '';
-    sane.user.fs."${cfg.browser.dotDir}/${cfg.browser.libName}.overrides.cfg" = sane-lib.fs.wantedText ''
-      // if we can't query the revocation status of a SSL cert because the issuer is offline,
-      // treat it as unrevoked.
-      // see: <https://librewolf.net/docs/faq/#im-getting-sec_error_ocsp_server_error-what-can-i-do>
-      defaultPref("security.OCSP.require", false);
-    '';
-    # flush the cache to disk to avoid it taking up too much tmp
-    sane.user.persist.byPath."${cfg.browser.cacheDir}" = lib.mkIf (cfg.persistCache != null) {
-      store = cfg.persistCache;
-    };
-
-    sane.user.persist.byPath."${cfg.browser.dotDir}/default" = lib.mkIf (cfg.persistData != null) {
-      store = cfg.persistData;
-    };
-    sane.user.fs."${cfg.browser.dotDir}/default" = sane-lib.fs.wantedDir;
-    # instruct Firefox to put the profile in a predictable directory (so we can do things like persist just it).
-    # XXX: the directory *must* exist, even if empty; Firefox will not create the directory itself.
-    sane.user.fs."${cfg.browser.dotDir}/profiles.ini" = sane-lib.fs.wantedText ''
-      [Profile0]
-      Name=default
-      IsRelative=1
-      Path=default
-      Default=1
-
-      [General]
-      StartWithLastProfile=1
-    '';
-
-  };
-}

hosts/common/home/gfeeds.nix

@@ -1,42 +0,0 @@
-# gnome feeds RSS viewer
-{ config, lib, sane-lib, ... }:
-
-let
-  feeds = sane-lib.feeds;
-  all-feeds = config.sane.feeds;
-  wanted-feeds = feeds.filterByFormat ["text" "image"] all-feeds;
-in {
-  sane.user.fs.".config/org.gabmus.gfeeds.json" = sane-lib.fs.wantedText (
-    builtins.toJSON {
-      # feed format is a map from URL to a dict,
-      #   with dict["tags"] a list of string tags.
-      feeds = sane-lib.mapToAttrs (feed: {
-        name = feed.url;
-        value.tags = [ feed.cat feed.freq ];
-      }) wanted-feeds;
-      dark_reader = false;
-      new_first = true;
-      # windowsize = {
-      #   width = 350;
-      #   height = 650;
-      # };
-      max_article_age_days = 90;
-      enable_js = false;
-      max_refresh_threads = 3;
-      # saved_items = {};
-      # read_items = [];
-      show_read_items = true;
-      full_article_title = true;
-      # views: "webview", "reader", "rsscont"
-      default_view = "rsscont";
-      open_links_externally = true;
-      full_feed_name = false;
-      refresh_on_startup = true;
-      tags = lib.unique (
-        (builtins.catAttrs "cat" wanted-feeds) ++ (builtins.catAttrs "freq" wanted-feeds)
-      );
-      open_youtube_externally = false;
-      media_player = "vlc";  # default: mpv
-    }
-  );
-}

hosts/common/home/git.nix

@@ -1,18 +0,0 @@
-{ lib, pkgs, sane-lib, ... }:
-
-let
-  mkCfg = lib.generators.toINI { };
-in
-{
-  sane.user.fs.".config/git/config" = sane-lib.fs.wantedText (mkCfg {
-    user.name = "Colin";
-    user.email = "colin@uninsane.org";
-    alias.co = "checkout";
-    # difftastic docs:
-    # - <https://difftastic.wilfred.me.uk/git.html>
-    diff.tool = "difftastic";
-    difftool.prompt = false;
-    "difftool \"difftastic\"".cmd = ''${pkgs.difftastic}/bin/difft "$LOCAL" "$REMOTE"'';
-    # now run `git difftool` to use difftastic git
-  });
-}

hosts/common/home/gpodder.nix

@@ -1,12 +0,0 @@
-# gnome feeds RSS viewer
-{ config, sane-lib, ... }:
-
-let
-  feeds = sane-lib.feeds;
-  all-feeds = config.sane.feeds;
-  wanted-feeds = feeds.filterByFormat ["podcast"] all-feeds;
-in {
-  sane.user.fs.".config/gpodderFeeds.opml" = sane-lib.fs.wantedText (
-    feeds.feedsToOpml wanted-feeds
-  );
-}

hosts/common/home/keyring.nix

@@ -1,11 +0,0 @@
-{ config, sane-lib, ... }:
-
-{
-  sane.user.persist.private = [ ".local/share/keyrings" ];
-
-  sane.user.fs."private/.local/share/keyrings/default" = {
-    generated.script.script = builtins.readFile ../../../scripts/init-keyring;
-    # TODO: is this `wantedBy` needed? can we inherit it?
-    wantedBy = [ config.sane.fs."/home/colin/private".unit ];
-  };
-}

hosts/common/home/keyring/default.nix

@@ -0,0 +1,17 @@
+{ config, pkgs, sane-lib, ... }:
+
+let
+  init-keyring = pkgs.static-nix-shell.mkBash {
+    pname = "init-keyring";
+    src = ./.;
+  };
+in
+{
+  sane.user.persist.private = [ ".local/share/keyrings" ];
+
+  sane.user.fs."private/.local/share/keyrings/default" = {
+    generated.command = [ "${init-keyring}/bin/init-keyring" ];
+    wantedBy = [ config.sane.fs."/home/colin/private".unit ];
+    wantedBeforeBy = [ ];  # don't created this as part of `multi-user.target`
+  };
+}

hosts/common/home/keyring/init-keyring

@@ -1,4 +1,5 @@
-#!/bin/sh
+#!/usr/bin/env nix-shell
+#!nix-shell -i bash
 # initializes the default libsecret keyring (used by gnome-keyring) if not already initialized.
 # this initializes it to be plaintext/unencrypted.
 

hosts/common/home/libreoffice.nix

@@ -1,14 +0,0 @@
-{ sane-lib, ... }:
-
-{
-  # libreoffice: disable first-run stuff
-  sane.user.fs.".config/libreoffice/4/user/registrymodifications.xcu" = sane-lib.fs.wantedText ''
-    <?xml version="1.0" encoding="UTF-8"?>
-    <oor:items xmlns:oor="http://openoffice.org/2001/registry" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
-      <item oor:path="/org.openoffice.Office.Common/Misc"><prop oor:name="FirstRun" oor:op="fuse"><value>false</value></prop></item>
-      <item oor:path="/org.openoffice.Office.Common/Misc"><prop oor:name="ShowTipOfTheDay" oor:op="fuse"><value>false</value></prop></item>
-    </oor:items>
-  '';
-  # <item oor:path="/org.openoffice.Setup/Product"><prop oor:name="LastTimeDonateShown" oor:op="fuse"><value>1667693880</value></prop></item>
-  # <item oor:path="/org.openoffice.Setup/Product"><prop oor:name="LastTimeGetInvolvedShown" oor:op="fuse"><value>1667693880</value></prop></item>
-}

hosts/common/home/mime.nix

@@ -1,42 +1,28 @@
-{ config, sane-lib, ...}:
+{ config, lib, ...}:
 
 let
-  www = config.sane.web-browser.browser.desktop;
-  pdf = "org.gnome.Evince.desktop";
-  md = "obsidian.desktop";
-  thumb = "org.gnome.gThumb.desktop";
-  video = "vlc.desktop";
-  # audio = "mpv.desktop";
-  audio = "vlc.desktop";
+  # ProgramConfig -> { "<mime-type>" = { priority, desktop }; }
+  weightedMimes = prog: builtins.mapAttrs (_key: desktop: { priority = prog.mime.priority; desktop = desktop; }) prog.mime.associations;
+  # [ { "<mime-type>" = { priority, desktop } ]; } ] -> { "<mime-type>" = [ { priority, desktop } ... ]; }
+  mergeMimes = mimes: lib.foldAttrs (item: acc: [item] ++ acc) [] mimes;
+  # [ { priority, desktop } ... ] -> Self
+  sortOneMimeType = associations: builtins.sort (l: r: assert l.priority != r.priority; l.priority < r.priority) associations;
+  sortMimes = mimes: builtins.mapAttrs (_k: sortOneMimeType) mimes;
+  removePriorities = mimes: builtins.mapAttrs (_k: associations: builtins.map (a: a.desktop) associations) mimes;
+
+  # [ ProgramConfig ]
+  enabledPrograms = builtins.filter (p: p.enabled) (builtins.attrValues config.sane.programs);
+  # [ { "<mime-type>" = { prority, desktop } ]
+  enabledWeightedMimes = builtins.map weightedMimes enabledPrograms;
 in
 {
-
   # the xdg mime type for a file can be found with:
   # - `xdg-mime query filetype path/to/thing.ext`
+  # the default handler for a mime type can be found with:
+  # - `xdg-mime query default <mimetype>`  (e.g. x-scheme-handler/http)
+  #
   # we can have single associations or a list of associations.
   # there's also options to *remove* [non-default] associations from specific apps
   xdg.mime.enable = true;
-  xdg.mime.defaultApplications = {
-    # AUDIO
-    "audio/flac" = audio;
-    "audio/mpeg" = audio;
-    "audio/x-vorbis+ogg" = audio;
-    # IMAGES
-    "image/heif" = thumb;  # apple codec
-    "image/png" = thumb;
-    "image/jpeg" = thumb;
-    # VIDEO
-    "video/mp4" = video;
-    "video/quicktime" = video;
-    "video/x-matroska" = video;
-    # HTML
-    "text/html" = www;
-    "x-scheme-handler/http" = www;
-    "x-scheme-handler/https" = www;
-    "x-scheme-handler/about" = www;
-    "x-scheme-handler/unknown" = www;
-    # RICH-TEXT DOCUMENTS
-    "application/pdf" = pdf;
-    "text/markdown" = md;
-  };
+  xdg.mime.defaultApplications = removePriorities (sortMimes (mergeMimes enabledWeightedMimes));
 }

hosts/common/home/mpv.nix

@@ -1,10 +0,0 @@
-{ sane-lib, ... }:
-
-{
-  # format is <key>=%<length>%<value>
-  sane.user.fs.".config/mpv/mpv.conf" = sane-lib.fs.wantedText ''
-    save-position-on-quit=%3%yes
-    keep-open=%3%yes
-  '';
-}
-

hosts/common/home/splatmoji.nix

@@ -1,19 +0,0 @@
-# borrows from:
-# - default config: <https://github.com/cspeterson/splatmoji/blob/master/splatmoji.config>
-# - wayland: <https://github.com/cspeterson/splatmoji/issues/32#issuecomment-830862566>
-{ pkgs, sane-lib, ... }:
-
-{
-  sane.user.persist.plaintext = [ ".local/state/splatmoji" ];
-  sane.user.fs.".config/splatmoji/splatmoji.config" = sane-lib.fs.wantedText ''
-    history_file=~/.local/state/splatmoji/history
-    history_length=5
-    # TODO: wayland equiv
-    paste_command=xdotool key ctrl+v
-    # rofi_command=${pkgs.wofi}/bin/wofi --dmenu --insensitive --cache-file /dev/null
-    rofi_command=${pkgs.fuzzel}/bin/fuzzel -d -i -w 60
-    xdotool_command=${pkgs.wtype}/bin/wtype
-    # TODO: wayland equiv
-    xsel_command=xsel -b -i
-  '';
-}

hosts/common/home/ssh.nix

@@ -1,24 +1,29 @@
-{ config, lib, sane-lib, ... }:
+# TODO: this should be moved to users/colin.nix
+{ config, lib, ... }:
 
-with lib;
 let
   host = config.networking.hostName;
-  user-pubkey = config.sane.ssh.pubkeys."colin@${host}".asUserKey;
-  host-keys = filter (k: k.user == "root") (attrValues config.sane.ssh.pubkeys);
-  known-hosts-text = concatStringsSep
+  user-pubkey-full = config.sane.ssh.pubkeys."colin@${host}" or {};
+  user-pubkey = user-pubkey-full.asUserKey or null;
+  host-keys = lib.filter (k: k.user == "root") (lib.attrValues config.sane.ssh.pubkeys);
+  known-hosts-text = lib.concatStringsSep
     "\n"
-    (map (k: k.asHostKey) host-keys)
+    (builtins.map (k: k.asHostKey) host-keys)
   ;
 in
 {
   # ssh key is stored in private storage
-  sane.user.persist.private = [ ".ssh/id_ed25519" ];
-  sane.user.fs.".ssh/id_ed25519.pub" = sane-lib.fs.wantedText user-pubkey;
-  sane.user.fs.".ssh/known_hosts" = sane-lib.fs.wantedText known-hosts-text;
+  sane.user.persist.private = [
+    { type = "file"; path = ".ssh/id_ed25519"; }
+  ];
+  sane.user.fs.".ssh/id_ed25519.pub" = lib.mkIf (user-pubkey != null) {
+    symlink.text = user-pubkey;
+  };
+  sane.user.fs.".ssh/known_hosts".symlink.text = known-hosts-text;
 
   users.users.colin.openssh.authorizedKeys.keys =
   let
-    user-keys = filter (k: k.user == "colin") (attrValues config.sane.ssh.pubkeys);
+    user-keys = lib.filter (k: k.user == "colin") (lib.attrValues config.sane.ssh.pubkeys);
   in
-    map (k: k.asUserKey) user-keys;
+    builtins.map (k: k.asUserKey) user-keys;
 }

hosts/common/home/sublime-music.nix

@@ -1,11 +0,0 @@
-{ config, sane-lib, ... }:
-
-{
-  # TODO: this should only be shipped on gui platforms
-  sops.secrets."sublime_music_config" = {
-    owner = config.users.users.colin.name;
-    sopsFile = ../../../secrets/universal/sublime_music_config.json.bin;
-    format = "binary";
-  };
-  sane.user.fs.".config/sublime-music/config.json" = sane-lib.fs.wantedSymlinkTo config.sops.secrets.sublime_music_config.path;
-}

hosts/common/home/vlc.nix

@@ -1,20 +0,0 @@
-{ config, lib, sane-lib, ... }:
-
-let
-  feeds = sane-lib.feeds;
-  all-feeds = config.sane.feeds;
-  wanted-feeds = feeds.filterByFormat ["podcast"] all-feeds;
-  podcast-urls = lib.concatStringsSep "|" (
-    builtins.map (feed: feed.url) wanted-feeds
-  );
-in
-{
-  sane.user.fs.".config/vlc/vlcrc" = sane-lib.fs.wantedText ''
-    [podcast]
-    podcast-urls=${podcast-urls}
-    [core]
-    metadata-network-access=0
-    [qt]
-    qt-privacy-ask=0
-  '';
-}

hosts/common/home/xdg-dirs.nix

@@ -1,9 +1,9 @@
-{ lib, sane-lib, ...}:
+{ ... }:
 
 {
   # XDG defines things like ~/Desktop, ~/Downloads, etc.
   # these clutter the home, so i mostly don't use them.
-  sane.user.fs.".config/user-dirs.dirs" = sane-lib.fs.wantedText ''
+  sane.user.fs.".config/user-dirs.dirs".symlink.text = ''
     XDG_DESKTOP_DIR="$HOME/.xdg/Desktop"
     XDG_DOCUMENTS_DIR="$HOME/dev"
     XDG_DOWNLOAD_DIR="$HOME/tmp"
@@ -16,5 +16,5 @@
 
   # prevent `xdg-user-dirs-update` from overriding/updating our config
   # see <https://manpages.ubuntu.com/manpages/bionic/man5/user-dirs.conf.5.html>
-  sane.user.fs.".config/user-dirs.conf" = sane-lib.fs.wantedText "enabled=False";
+  sane.user.fs.".config/user-dirs.conf".symlink.text = "enabled=False";
 }

hosts/common/home/zsh/default.nix

@@ -1,143 +0,0 @@
-{ pkgs, sane-lib, ... }:
-
-let
-  # powerlevel10k prompt config
-  # p10k.zsh is the auto-generated config, and i overwrite those defaults here, below.
-  p10k-overrides = ''
-    # powerlevel10k launches a gitstatusd daemon to accelerate git prompt queries.
-    # this keeps open file handles for any git repo i touch for 60 minutes (by default).
-    # that prevents unmounting whatever device the git repo is on -- particularly problematic for ~/private.
-    # i can disable gitstatusd and get slower fallback git queries:
-    # - either universally
-    # - or selectively by path
-    # see: <https://github.com/romkatv/powerlevel10k/issues/246>
-    typeset -g POWERLEVEL9K_VCS_DISABLED_DIR_PATTERN='(/home/colin/private/*|/home/colin/knowledge/*)'
-    # typeset -g POWERLEVEL9K_DISABLE_GITSTATUS=true
-
-    # show user@host also when logged into the current machine.
-    # default behavior is to show it only over ssh.
-    typeset -g POWERLEVEL9K_CONTEXT_{DEFAULT,SUDO}_CONTENT_EXPANSION='$P9K_CONTENT'
-  '';
-
-  prezto-init = ''
-    source ${pkgs.zsh-autosuggestions}/share/zsh-autosuggestions/zsh-autosuggestions.zsh
-    source ${pkgs.zsh-syntax-highlighting}/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
-    source ${pkgs.zsh-prezto}/share/zsh-prezto/init.zsh
-  '';
-in
-{
-  sane.user.persist.plaintext = [
-    # we don't need to full zsh dir -- just the history file --
-    # but zsh will sometimes backup the history file and we get fewer errors if we do proper mounts instead of symlinks.
-    # TODO: should be private?
-    ".local/share/zsh"
-    # cache gitstatus otherwise p10k fetched it from the net EVERY BOOT
-    ".cache/gitstatus"
-  ];
-
-  # zsh/prezto complains if zshrc doesn't exist; but it does allow an "empty" file.
-  sane.user.fs.".config/zsh/.zshrc" = sane-lib.fs.wantedText "# ";
-
-  # enable zsh completions
-  environment.pathsToLink = [ "/share/zsh" ];
-
-  programs.zsh = {
-    enable = true;
-    histFile = "$HOME/.local/share/zsh/history";
-    shellAliases = {
-      ":q" = "exit";
-      # common typos
-      "cd.." = "cd ..";
-      "cd../" = "cd ../";
-    };
-    setOptions = [
-      # defaults:
-      "HIST_IGNORE_DUPS"
-      "SHARE_HISTORY"
-      "HIST_FCNTL_LOCK"
-      # disable `rm *` confirmations
-      "rmstarsilent"
-    ];
-
-    # .zshenv config:
-    shellInit = ''
-      ZDOTDIR=$HOME/.config/zsh
-    '';
-
-    # .zshrc config:
-    interactiveShellInit =
-      (builtins.readFile ./p10k.zsh)
-    + p10k-overrides
-    + prezto-init
-    + ''
-      # zmv is a way to do rich moves/renames, with pattern matching/substitution.
-      # see for an example: <https://filipe.kiss.ink/zmv-zsh-rename/>
-      autoload -Uz zmv
-
-      HISTORY_IGNORE='(sane-shutdown *|sane-reboot *|rm *)'
-
-      # extra aliases
-      # TODO: move to `shellAliases` config?
-      function nd() {
-        mkdir -p "$1";
-        pushd "$1";
-      }
-
-      # auto-cd into any of these dirs by typing them and pressing 'enter':
-      hash -d 3rd="/home/colin/dev/3rd"
-      hash -d dev="/home/colin/dev"
-      hash -d knowledge="/home/colin/knowledge"
-      hash -d nixos="/home/colin/nixos"
-      hash -d nixpkgs="/home/colin/dev/3rd/nixpkgs"
-      hash -d ref="/home/colin/ref"
-      hash -d secrets="/home/colin/knowledge/secrets"
-      hash -d tmp="/home/colin/tmp"
-      hash -d uninsane="/home/colin/dev/uninsane"
-      hash -d Videos="/home/colin/Videos"
-    '';
-
-    syntaxHighlighting.enable = true;
-    vteIntegration = true;
-  };
-
-  # enable a command-not-found hook to show nix packages that might provide the binary typed.
-  programs.nix-index.enable = true;
-  programs.command-not-found.enable = false;  #< mutually exclusive with nix-index
-
-  # prezto = oh-my-zsh fork; controls prompt, auto-completion, etc.
-  # see: https://github.com/sorin-ionescu/prezto
-  # i believe this file is auto-sourced by the prezto init.zsh script.
-  sane.user.fs.".config/zsh/.zpreztorc" = sane-lib.fs.wantedText ''
-    zstyle ':prezto:*:*' color 'yes'
-
-    # modules (they ship with prezto):
-    # ENVIRONMENT: configures jobs to persist after shell exit; other basic niceties
-    # TERMINAL: auto-titles terminal (e.g. based on cwd)
-    # EDITOR: configures shortcuts like Ctrl+U=undo, Ctrl+L=clear
-    # HISTORY: `history-stat` alias, setopts for good history defaults
-    # DIRECTORY: sets AUTO_CD, adds `d` alias to list directory stack, and `1`-`9` to cd that far back the stack
-    # SPECTRUM: helpers for term colors and styling. used by prompts? might be unnecessary
-    # UTILITY: configures aliases like `ll`, `la`, disables globbing for things like rsync
-    #   adds aliases like `get` to fetch a file. also adds `http-serve` alias??
-    # COMPLETION: tab completion. requires `utility` module prior to loading
-    # TODO: enable AUTO_PARAM_SLASH
-    zstyle ':prezto:load' pmodule \
-      'environment' \
-      'terminal' \
-      'editor' \
-      'history' \
-      'directory' \
-      'spectrum' \
-      'utility' \
-      'completion' \
-      'prompt'
-
-    # default keymap. try also `vicmd` (vim normal mode, AKA "cmd mode") or `vi`.
-    zstyle ':prezto:module:editor' key-bindings 'emacs'
-
-    zstyle ':prezto:module:prompt' theme 'powerlevel10k'
-
-    # disable `mv` confirmation (and `rm`, too, unfortunately)
-    zstyle ':prezto:module:utility' safe-ops 'no'
-  '';
-}

hosts/common/i2p.nix

@@ -1,4 +0,0 @@
-{ ... }:
-{
-  # services.i2p.enable = true;
-}

hosts/common/ids.nix

@@ -1,4 +1,6 @@
 # TODO: migrate to nixpkgs `config.ids.uids`
+# - note that nixpkgs' `config.ids.uids` is strictly a database: it doesn't set anything by default
+#   whereas our impl sets the gid/uid of the user/group specified if they exist.
 { ... }:
 
 {
@@ -13,6 +15,8 @@
   sane.ids.acme.gid = 996;
   sane.ids.pleroma.uid = 997;
   sane.ids.acme.uid = 998;
+  sane.ids.matrix-appservice-irc.uid = 993;
+  sane.ids.matrix-appservice-irc.gid = 992;
 
   # greetd (used by sway)
   sane.ids.greeter.uid = 999;
@@ -28,6 +32,20 @@
   sane.ids.mautrix-signal.gid = 2404;
   sane.ids.navidrome.uid = 2405;
   sane.ids.navidrome.gid = 2405;
+  sane.ids.calibre-web.uid = 2406;
+  sane.ids.calibre-web.gid = 2406;
+  sane.ids.komga.uid = 2407;
+  sane.ids.komga.gid = 2407;
+  sane.ids.lemmy.uid = 2408;
+  sane.ids.lemmy.gid = 2408;
+  sane.ids.pict-rs.uid = 2409;
+  sane.ids.pict-rs.gid = 2409;
+  sane.ids.sftpgo.uid = 2410;
+  sane.ids.sftpgo.gid = 2410;
+  sane.ids.trust-dns.uid = 2411;
+  sane.ids.trust-dns.gid = 2411;
+  sane.ids.export.gid = 2412;
+  sane.ids.nfsuser.uid = 2413;
 
   sane.ids.colin.uid = 1000;
   sane.ids.guest.uid = 1100;
@@ -36,11 +54,12 @@
   sane.ids.sshd.uid = 2001;  # 997
   sane.ids.sshd.gid = 2001;  # 997
   sane.ids.polkituser.gid = 2002;  # 998
-  # sane.ids.systemd-coredump.gid = 2003;  # 996  # 2023/02/12: upstream now specifies this as 151
+  sane.ids.systemd-coredump.gid = 2003;  # 996  # 2023/02/12-2023/02/28: upstream temporarily specified this as 151
   sane.ids.nscd.uid = 2004;
   sane.ids.nscd.gid = 2004;
   sane.ids.systemd-oom.uid = 2005;
   sane.ids.systemd-oom.gid = 2005;
+  sane.ids.wireshark.gid = 2006;
 
   # found on graphical hosts
   sane.ids.nm-iodine.uid = 2101;  # desko/moby/lappy

hosts/common/net.nix

@@ -1,4 +1,4 @@
-{ config, lib, pkgs, ... }:
+{ lib, ... }:
 
 {
   # the default backend is "wpa_supplicant".
@@ -11,13 +11,37 @@
   # - `man iwd.config`  for global config
   # - `man iwd.network` for per-SSID config
   # use `iwctl` to control
-  networking.networkmanager.wifi.backend = "iwd";
-  networking.wireless.iwd.enable = true;
-  networking.wireless.iwd.settings = {
-    # auto-connect to a stronger network if signal drops below this value
-    # bedroom -> bedroom connection is -35 to -40 dBm
-    # bedroom -> living room connection is -60 dBm
-    General.RoamThreshold = "-52";  # default -70
-    General.RoamThreshold5G = "-52";  # default -76
-  };
+  # networking.networkmanager.wifi.backend = "iwd";
+  # networking.wireless.iwd.enable = true;
+  # networking.wireless.iwd.settings = {
+  #   # auto-connect to a stronger network if signal drops below this value
+  #   # bedroom -> bedroom connection is -35 to -40 dBm
+  #   # bedroom -> living room connection is -60 dBm
+  #   General.RoamThreshold = "-52";  # default -70
+  #   General.RoamThreshold5G = "-52";  # default -76
+  # };
+
+  # plugins mostly add support for establishing different VPN connections.
+  # the default plugin set includes mostly proprietary VPNs:
+  # - fortisslvpn (Fortinet)
+  # - iodine (DNS tunnels)
+  # - l2tp
+  # - openconnect (Cisco Anyconnect / Juniper / ocserv)
+  # - openvpn
+  # - vpnc (Cisco VPN)
+  # - sstp
+  #
+  # i don't use these, and notably they drag in huge dependency sets and don't cross compile well.
+  # e.g. openconnect drags in webkitgtk (for SSO)!
+  networking.networkmanager.plugins = lib.mkForce [];
+
+  networking.firewall.allowedUDPPorts = [
+    1900  # to received UPnP advertisements. required by sane-ip-check-upnp
+  ];
+
+  # keyfile.path = where networkmanager should look for connection credentials
+  networking.networkmanager.extraConfig = ''
+    [keyfile]
+    path=/var/lib/NetworkManager/system-connections
+  '';
 }

hosts/common/nix-path/default.nix

@@ -0,0 +1,13 @@
+{ pkgs, ... }:
+
+{
+  # allow `nix-shell` (and probably nix-index?) to locate our patched and custom packages
+  nix.nixPath = [
+    "nixpkgs=${pkgs.path}"
+    # note the import starts at repo root: this allows `./overlay/default.nix` to access the stuff at the root
+    # "nixpkgs-overlays=${../../..}/hosts/common/nix-path/overlay"
+    # as long as my system itself doesn't rely on NIXPKGS at runtime, we can point the overlays to git
+    # to avoid switching so much during development
+    "nixpkgs-overlays=/home/colin/dev/nixos/hosts/common/nix-path/overlay"
+  ];
+}

hosts/common/nix-path/overlay/default.nix

@@ -0,0 +1,4 @@
+# XXX: NIX_PATH=...:nixpkgs-overlays=... will import every overlay in the directory
+# so we prefer to give it a directory with just this *one* overlay, otherwise it imports conflicting overlays
+# and gets stuck in a loop until it OOMs
+import ../../../../overlays/all.nix

hosts/common/persist.nix

@@ -6,13 +6,11 @@
   sane.persist.stores.private.prefix = "/home/colin";
 
   sane.persist.sys.plaintext = [
+    # TODO: these should be private.. somehow
     "/var/log"
     "/var/backup"  # for e.g. postgres dumps
-    # TODO: move elsewhere
-    "/var/lib/alsa"                # preserve output levels, default devices
-    "/var/lib/colord"              # preserve color calibrations (?)
-    "/var/lib/machines"            # maybe not needed, but would be painful to add a VM and forget.
-    "/var/lib/systemd/backlight"   # backlight brightness
+  ];
+  sane.persist.sys.cryptClearOnBoot = [
     "/var/lib/systemd/coredump"
   ];
 }

hosts/common/programs.nix

@@ -1,329 +0,0 @@
-{ lib, pkgs, ... }:
-
-let
-  inherit (builtins) attrNames concatLists;
-  inherit (lib) mapAttrs mapAttrsToList mkDefault mkMerge optional;
-
-  sysadminPkgs = {
-    inherit (pkgs // {
-      # XXX can't `inherit` a nested attr, so we move them to the toplevel
-      "cacert.unbundled" = pkgs.cacert.unbundled;
-    })
-      btrfs-progs
-      "cacert.unbundled"  # some services require unbundled /etc/ssl/certs
-      cryptsetup
-      dig
-      efibootmgr
-      fatresize
-      fd
-      file
-      gawk
-      git
-      gptfdisk
-      hdparm
-      htop
-      iftop
-      inetutils  # for telnet
-      iotop
-      iptables
-      jq
-      killall
-      lsof
-      nano
-      netcat
-      nethogs
-      nmap
-      openssl
-      parted
-      pciutils
-      powertop
-      pstree
-      ripgrep
-      screen
-      smartmontools
-      socat
-      strace
-      tcpdump
-      tree
-      usbutils
-      wget
-    ;
-  };
-
-  consolePkgs = {
-    inherit (pkgs)
-      backblaze-b2
-      cdrtools
-      dmidecode
-      duplicity
-      efivar
-      flashrom
-      fwupd
-      ghostscript  # TODO: imagemagick wrapper should add gs to PATH
-      gnupg
-      gocryptfs
-      gopass
-      gopass-jsonapi
-      ifuse
-      imagemagick
-      ipfs
-      kitty  # TODO: move to GUI, but `ssh servo` from kitty sets `TERM=xterm-kitty` in the remove and breaks things
-      libimobiledevice
-      libsecret  # for managing user keyrings
-      lm_sensors  # for sensors-detect
-      lshw
-      ffmpeg
-      memtester
-      networkmanager
-      nixpkgs-review
-      # nixos-generators
-      # nettools
-      nmon
-      oathToolkit  # for oathtool
-      # ponymix
-      pulsemixer
-      python3
-      rsync
-      # python3Packages.eyeD3  # music tagging
-      sane-scripts
-      sequoia
-      snapper
-      sops
-      sox
-      speedtest-cli
-      sqlite  # to debug sqlite3 databases
-      ssh-to-age
-      sudo
-      # tageditor  # music tagging
-      unar
-      visidata
-      w3m
-      wireguard-tools
-      # youtube-dl
-      yt-dlp
-    ;
-  };
-
-  guiPkgs = {
-    inherit (pkgs // (with pkgs; {
-      # XXX can't `inherit` a nested attr, so we move them to the toplevel
-      # TODO: could use some "flatten attrs" helper instead
-      "gnome.cheese" = gnome.cheese;
-      "gnome.dconf-editor" = gnome.dconf-editor;
-      "gnome.file-roller" = gnome.file-roller;
-      "gnome.gnome-disk-utility" = gnome.gnome-disk-utility;
-      "gnome.gnome-maps" = gnome.gnome-maps;
-      "gnome.nautilus" = gnome.nautilus;
-      "gnome.gnome-system-monitor" = gnome.gnome-system-monitor;
-      "gnome.gnome-terminal" = gnome.gnome-terminal;
-      "gnome.gnome-weather" = gnome.gnome-weather;
-      "libsForQt5.plasmatube" = libsForQt5.plasmatube;
-    }))
-      aerc  # email client
-      audacity
-      celluloid  # mpv frontend
-      chromium
-      clinfo
-      dino
-      electrum
-      element-desktop
-      emote
-      evince  # works on phosh
-
-      # { pkg = fluffychat-moby; dir = [ ".local/share/chat.fluffy.fluffychat" ]; }  # TODO: ship normal fluffychat on non-moby?
-
-      foliate  # e-book reader
-      font-manager
-
-      # XXX by default fractal stores its state in ~/.local/share/<UUID>.
-      # after logging in, manually change ~/.local/share/keyrings/... to point it to some predictable subdir.
-      # then reboot (so that libsecret daemon re-loads the keyring...?)
-      # { pkg = fractal-latest; private = [ ".local/share/fractal" ]; }
-      # { pkg = fractal-next; private = [ ".local/share/fractal" ]; }
-
-      gajim  # XMPP client
-      gimp  # broken on phosh
-      "gnome.cheese"
-      "gnome.dconf-editor"
-      gnome-feeds  # RSS reader (with claimed mobile support)
-      "gnome.file-roller"
-      "gnome.gnome-disk-utility"
-      "gnome.gnome-maps"  # works on phosh
-      "gnome.nautilus"
-      # gnome-podcasts
-      "gnome.gnome-system-monitor"
-      "gnome.gnome-terminal"  # works on phosh
-      "gnome.gnome-weather"
-      gpodder-configured
-      gthumb
-      inkscape
-      kdenlive
-      kid3  # audio tagging
-      krita
-      libreoffice-fresh  # XXX colin: maybe don't want this on mobile
-      lollypop
-      mpv
-      networkmanagerapplet
-      newsflash
-      nheko
-      obsidian
-      pavucontrol
-      # picard  # music tagging
-      playerctl
-      "libsForQt5.plasmatube"  # Youtube player
-      soundconverter
-      # sublime music persists any downloaded albums here.
-      # it doesn't obey a conventional ~/Music/{Artist}/{Album}/{Track} notation, so no symlinking
-      # config (e.g. server connection details) is persisted in ~/.config/sublime-music/config.json
-      #   possible to pass config as a CLI arg (sublime-music -c config.json)
-      # { pkg = sublime-music; dir = [ ".local/share/sublime-music" ]; }
-      sublime-music-mobile
-      tdesktop  # broken on phosh
-      tokodon
-      vlc
-      # pleroma client (Electron). input is broken on phosh. TODO(2023/02/02): fix electron19 input (insecure)
-      # whalebird
-      xdg-utils  # for xdg-open
-      xterm  # broken on phosh
-    ;
-  };
-  x86GuiPkgs = {
-    inherit (pkgs)
-      discord
-
-      # kaiteki  # Pleroma client
-      # gnome.zenity # for kaiteki (it will use qarma, kdialog, or zenity)
-      # gpt2tc  # XXX: unreliable mirror
-
-      # TODO(unpin): handbrake is broken on aarch64-linux 2023/01/29
-      handbrake
-
-      logseq
-      losslesscut-bin
-      makemkv
-      monero-gui
-      signal-desktop
-      spotify
-      tor-browser-bundle-bin
-      zecwallet-lite
-    ;
-  };
-
-  # define -- but don't enable -- the packages in some attrset.
-  # use `mkDefault` for the package here so we can customize some of them further down this file
-  declarePkgs = pkgsAsAttrs: mapAttrs (_n: p: {
-    package = mkDefault p;
-  }) pkgsAsAttrs;
-in
-{
-  config = {
-    sane.programs = mkMerge [
-      (declarePkgs sysadminPkgs)
-      (declarePkgs consolePkgs)
-      (declarePkgs guiPkgs)
-      (declarePkgs x86GuiPkgs)
-      {
-        # link the various package sets into their own meta packages
-        sysadminUtils = {
-          package = null;
-          suggestedPrograms = attrNames sysadminPkgs;
-        };
-        consoleUtils = {
-          package = null;
-          suggestedPrograms = attrNames consolePkgs;
-        };
-        guiApps = {
-          package = null;
-          suggestedPrograms = (attrNames guiPkgs)
-            ++ optional (pkgs.system == "x86_64-linux") "x86GuiApps";
-        };
-        x86GuiApps = {
-          package = null;
-          suggestedPrograms = attrNames x86GuiPkgs;
-        };
-      }
-      {
-        # nontrivial package definitions
-        imagemagick.package = pkgs.imagemagick.override {
-          ghostscriptSupport = true;
-        };
-
-        dino.private = [ ".local/share/dino" ];
-
-        # creds, but also 200 MB of node modules, etc
-        discord = {
-          package = pkgs.discord.override {
-            # XXX 2022-07-31: fix to allow links to open in default web-browser:
-            #   https://github.com/NixOS/nixpkgs/issues/78961
-            nss = pkgs.nss_latest;
-          };
-          private = [ ".config/discord" ];
-        };
-
-        # creds/session keys, etc
-        element-desktop.private = [ ".config/Element" ];
-
-        # `emote` will show a first-run dialog based on what's in this directory.
-        # mostly, it just keeps a LRU of previously-used emotes to optimize display order.
-        # TODO: package [smile](https://github.com/mijorus/smile) for probably a better mobile experience.
-        emote.dir = [ ".local/share/Emote" ];
-
-        # XXX: we preserve the whole thing because if we only preserve gPodder/Downloads
-        #   then startup is SLOW during feed import, and we might end up with zombie eps in the dl dir.
-        gpodder-configured.dir = [ "gPodder" ];
-
-        # actual monero blockchain (not wallet/etc; safe to delete, just slow to regenerate)
-        # XXX: is it really safe to persist this? it doesn't have info that could de-anonymize if captured?
-        monero-gui.dir = [ ".bitmonero" ];
-
-        mpv.dir = [ ".config/mpv/watch_later" ];
-
-        # not strictly necessary, but allows caching articles; offline use, etc.
-        newsflash.dir = [ ".local/share/news-flash" ];
-        nheko.private = [
-          ".config/nheko"  # config file (including client token)
-          ".cache/nheko"  # media cache
-          ".local/share/nheko"  # per-account state database
-        ];
-
-        # settings (electron app)
-        obsidian.dir = [ ".config/obsidian" ];
-
-        # creds, media
-        signal-desktop.private = [ ".config/Signal" ];
-
-
-        # creds, widevine .so download. TODO: could easily manage these statically.
-        spotify.dir = [ ".config/spotify" ];
-
-        # sublime music persists any downloaded albums here.
-        # it doesn't obey a conventional ~/Music/{Artist}/{Album}/{Track} notation, so no symlinking
-        # config (e.g. server connection details) is persisted in ~/.config/sublime-music/config.json
-        #   possible to pass config as a CLI arg (sublime-music -c config.json)
-        # { pkg = sublime-music; dir = [ ".local/share/sublime-music" ]; }
-        sublime-music-mobile.dir = [ ".local/share/sublime-music" ];
-
-        tdesktop.private = [ ".local/share/TelegramDesktop" ];
-
-        tokodon.private = [ ".cache/KDE/tokodon" ];
-
-        # hardenedMalloc solves a crash at startup
-        # TODO 2023/02/02: is this safe to remove yet?
-        tor-browser-bundle-bin.package = pkgs.tor-browser-bundle-bin.override {
-          useHardenedMalloc = false;
-        };
-
-        # vlc remembers play position in ~/.config/vlc/vlc-qt-interface.conf
-        vlc.dir = [ ".config/vlc" ];
-
-        whalebird.private = [ ".config/Whalebird" ];
-
-        # zcash coins. safe to delete, just slow to regenerate (10-60 minutes)
-        zecwallet-lite.private = [ ".zcash" ];
-      }
-    ];
-
-    # XXX: this might not be necessary. try removing this and cacert.unbundled (servo)?
-    environment.etc."ssl/certs".source = "${pkgs.cacert.unbundled}/etc/ssl/certs/*";
-  };
-}

hosts/common/programs/aerc.nix

@@ -0,0 +1,9 @@
+# Terminal UI mail client
+{ ... }:
+
+{
+  sane.programs.aerc = {
+    secrets.".config/aerc/accounts.conf" = ../../../secrets/common/aerc_accounts.conf.bin;
+    mime.associations."x-scheme-handler/mailto" = "aerc.desktop";
+  };
+}

hosts/common/programs/alacritty.nix

@@ -0,0 +1,24 @@
+# alacritty terminal emulator
+# - config options: <https://github.com/alacritty/alacritty/blob/master/extra/man/alacritty.5.scd>
+#   - `man 5 alacritty`
+#   - defaults: <https://github.com/alacritty/alacritty/releases>  -> alacritty.yml
+# - irc: #alacritty on libera.chat
+{ lib, ... }:
+{
+  sane.programs.alacritty = {
+    env.TERMINAL = lib.mkDefault "alacritty";
+    # note: alacritty will switch to .toml config in 13.0 release
+    # - run `alacritty migrate` to convert the yaml to toml
+    fs.".config/alacritty/alacritty.yml".symlink.text = ''
+      font:
+        size: 14
+
+      key_bindings:
+        - { key: N,         mods: Control,        action: CreateNewWindow }
+        - { key: PageUp,    mods: Control,        action: ScrollPageUp }
+        - { key: PageDown,  mods: Control,        action: ScrollPageDown }
+        - { key: PageUp,    mods: Control|Shift,  action: ScrollPageUp }
+        - { key: PageDown,  mods: Control|Shift,  action: ScrollPageDown }
+    '';
+  };
+}

hosts/common/programs/assorted.nix

@@ -0,0 +1,253 @@
+{ pkgs, ... }:
+
+let
+  declPackageSet = pkgs: {
+    package = null;
+    suggestedPrograms = pkgs;
+  };
+in
+{
+  sane.programs = {
+    # PACKAGE SETS
+    "sane-scripts.backup" = declPackageSet [
+      "sane-scripts.backup-ls"
+      "sane-scripts.backup-restore"
+    ];
+    "sane-scripts.bittorrent" = declPackageSet [
+      "sane-scripts.bt-add"
+      "sane-scripts.bt-rm"
+      "sane-scripts.bt-search"
+      "sane-scripts.bt-show"
+    ];
+    "sane-scripts.dev" = declPackageSet [
+      "sane-scripts.dev-cargo-loop"
+      "sane-scripts.git-init"
+    ];
+    "sane-scripts.cli" = declPackageSet [
+      "sane-scripts.deadlines"
+      "sane-scripts.find-dotfiles"
+      "sane-scripts.ip-check"
+      "sane-scripts.ip-reconnect"
+      "sane-scripts.private-change-passwd"
+      "sane-scripts.private-do"
+      "sane-scripts.private-init"
+      "sane-scripts.private-lock"
+      "sane-scripts.private-unlock"
+      "sane-scripts.rcp"
+      "sane-scripts.reboot"
+      "sane-scripts.reclaim-boot-space"
+      "sane-scripts.reclaim-disk-space"
+      "sane-scripts.secrets-dump"
+      "sane-scripts.secrets-unlock"
+      "sane-scripts.secrets-update-keys"
+      "sane-scripts.shutdown"
+      "sane-scripts.ssl-dump"
+      "sane-scripts.sudo-redirect"
+      "sane-scripts.sync-from-servo"
+      "sane-scripts.vpn-down"
+      "sane-scripts.vpn-up"
+      "sane-scripts.which"
+      "sane-scripts.wipe-browser"
+    ];
+    "sane-scripts.sys-utils" = declPackageSet [
+      "sane-scripts.ip-port-forward"
+      "sane-scripts.sync-music"
+    ];
+
+
+    sysadminUtils = declPackageSet [
+      "btrfs-progs"
+      "cacert.unbundled"  # some services require unbundled /etc/ssl/certs
+      "cryptsetup"
+      "dig"
+      "efibootmgr"
+      "fatresize"
+      "fd"
+      "file"
+      # "fwupd"
+      "gawk"
+      "git"
+      "gptfdisk"
+      "hdparm"
+      "htop"
+      "iftop"
+      "inetutils"  # for telnet
+      "iotop"
+      "iptables"
+      "jq"
+      "killall"
+      "lsof"
+      "miniupnpc"
+      "nano"
+      #  "ncdu"  # ncurses disk usage. doesn't cross compile (zig)
+      "neovim"
+      "netcat"
+      "nethogs"
+      "nmap"
+      "openssl"
+      "parted"
+      "pciutils"
+      "powertop"
+      "pstree"
+      "ripgrep"
+      "screen"
+      "smartmontools"
+      "socat"
+      "strace"
+      "subversion"
+      "tcpdump"
+      "tree"
+      "usbutils"
+      "wget"
+      "wirelesstools"  # iwlist
+    ];
+    sysadminExtraUtils = declPackageSet [
+      "backblaze-b2"
+      "duplicity"
+      "sane-scripts.backup"
+      "sqlite"  # to debug sqlite3 databases
+    ];
+
+    # TODO: split these into smaller groups.
+    # - moby doesn't want a lot of these.
+    # - categories like
+    #   - dev?
+    #   - debugging?
+    consoleUtils = declPackageSet [
+      "alsaUtils"  # for aplay, speaker-test
+      # "cdrtools"
+      "clinfo"
+      "dmidecode"
+      "dtrx"  # `unar` alternative, "Do The Right eXtraction"
+      "efivar"
+      # "flashrom"
+      "git"  # needed as a user package, for config.
+      # "gnupg"
+      # "gocryptfs"
+      # "gopass"
+      # "gopass-jsonapi"
+      "helix"  # text editor
+      # "kitty"  # XXX needs to be in consolueUtils because `ssh servo` from kitty sets `TERM=xterm-kitty` in the remote and breaks things
+      "libsecret"  # for managing user keyrings. TODO: what needs this? lift into the consumer
+      "lm_sensors"  # for sensors-detect. TODO: what needs this? lift into the consumer
+      "lshw"
+      # "memtester"
+      "neovim"  # needed as a user package, for swap persistence
+      # "nettools"
+      # "networkmanager"
+      # "nixos-generators"
+      "nmon"
+      # "node2nix"
+      # "oathToolkit"  # for oathtool
+      # "ponymix"
+      "pulsemixer"
+      "python3"
+      # "python3Packages.eyeD3"  # music tagging
+      "ripgrep"  # needed as a user package so that its user-level config file can be installed
+      "rsync"
+      "sane-scripts.bittorrent"
+      "sane-scripts.cli"
+      "snapper"
+      "sops"
+      "speedtest-cli"
+      # "ssh-to-age"
+      "sudo"
+      # "tageditor"  # music tagging
+      # "unar"
+      "wireguard-tools"
+      "xdg-terminal-exec"
+      "xdg-utils"  # for xdg-open
+      # "yarn"
+      "zsh"
+    ];
+
+    desktopConsoleUtils = declPackageSet [
+      "gh"  # MS GitHub cli
+      "nix-index"
+      "nixpkgs-review"
+      "sane-scripts.dev"
+      "sequoia"
+    ];
+
+    consoleMediaUtils = declPackageSet [
+      "ffmpeg"
+      "imagemagick"
+      "sox"
+      "yt-dlp"
+    ];
+
+    tuiApps = declPackageSet [
+      "aerc"  # email client
+      "msmtp"  # sendmail
+      "offlineimap"  # email mailbox sync
+      "sfeed"  # RSS fetcher
+      "visidata"  # TUI spreadsheet viewer/editor
+      "w3m"  # web browser
+    ];
+
+    iphoneUtils = declPackageSet [
+      "ifuse"
+      "ipfs"
+      "libimobiledevice"
+      "sane-scripts.sync-from-iphone"
+    ];
+
+    devPkgs = declPackageSet [
+      "clang"
+      "nodejs"
+      "tree-sitter"
+    ];
+
+
+    # INDIVIDUAL PACKAGE DEFINITIONS
+
+    # creds, but also 200 MB of node modules, etc
+    discord.persist.private = [ ".config/discord" ];
+
+    # `emote` will show a first-run dialog based on what's in this directory.
+    # mostly, it just keeps a LRU of previously-used emotes to optimize display order.
+    # TODO: package [smile](https://github.com/mijorus/smile) for probably a better mobile experience.
+    emote.persist.plaintext = [ ".local/share/Emote" ];
+
+    fluffychat-moby.persist.plaintext = [ ".local/share/chat.fluffy.fluffychat" ];
+
+    # MS GitHub stores auth token in .config
+    # TODO: we can populate gh's stuff statically; it even lets us use the same oauth across machines
+    gh.persist.private = [ ".config/gh" ];
+
+    # actual monero blockchain (not wallet/etc; safe to delete, just slow to regenerate)
+    # XXX: is it really safe to persist this? it doesn't have info that could de-anonymize if captured?
+    monero-gui.persist.plaintext = [ ".bitmonero" ];
+
+    mumble.persist.private = [ ".local/share/Mumble" ];
+
+    # settings (electron app)
+    obsidian.persist.plaintext = [ ".config/obsidian" ];
+
+    # creds, media
+    signal-desktop.persist.private = [ ".config/Signal" ];
+
+    # printer/filament settings
+    slic3r.persist.plaintext = [ ".Slic3r" ];
+
+    # creds, widevine .so download. TODO: could easily manage these statically.
+    spotify.persist.plaintext = [ ".config/spotify" ];
+
+    tdesktop.persist.private = [ ".local/share/TelegramDesktop" ];
+
+    tokodon.persist.private = [ ".cache/KDE/tokodon" ];
+
+    # hardenedMalloc solves an "unable to connect to Tor" error when pressing the "connect" button
+    # - still required as of 2023/07/14
+    tor-browser-bundle-bin.package = pkgs.tor-browser-bundle-bin.override {
+      useHardenedMalloc = false;
+    };
+
+    whalebird.persist.private = [ ".config/Whalebird" ];
+
+    yarn.persist.plaintext = [ ".cache/yarn" ];
+
+    # zcash coins. safe to delete, just slow to regenerate (10-60 minutes)
+    zecwallet-lite.persist.private = [ ".zcash" ];
+  };
+}

hosts/common/programs/cantata.nix

@@ -0,0 +1,36 @@
+# cantata is a mpd frontend.
+# before launching it, run `mopidy` in some tab
+# TODO: auto-launch mopidy when cantata launches?
+{ ... }:
+{
+  sane.programs.cantata = {
+    persist.plaintext = [
+      ".cache/cantata"  # album art
+      ".local/share/cantata/library"  # library index (?)
+    ];
+    fs.".config/cantata/cantata.conf".symlink.text = ''
+      [General]
+      fetchCovers=true
+      storeCoversInMpdDir=false
+      version=2.5.0
+
+      [Connection]
+      allowLocalStreaming=true
+      applyReplayGain=true
+      autoUpdate=false
+      dir=~/Music
+      host=localhost
+      partition=
+      passwd=
+      port=6600
+      replayGain=off
+      streamUrl=
+
+      [LibraryPage]
+      artist\gridZoom=100
+      artist\searchActive=false
+      artist\viewMode=detailedtree
+    '';
+    suggestedPrograms = [ "mopidy" ];
+  };
+}

hosts/common/programs/chatty.nix

@@ -0,0 +1,46 @@
+{ pkgs, ... }:
+let
+  chattyNoOauth = pkgs.chatty.override {
+    # the OAuth feature (presumably used for web-based logins) pulls a full webkitgtk.
+    # especially when using the gtk3 version of evolution-data-server, it's an ancient webkitgtk_4_1.
+    # disable OAuth for a faster build & smaller closure
+    evolution-data-server = pkgs.evolution-data-server.override {
+      enableOAuth2 = false;
+      gnome-online-accounts = pkgs.gnome-online-accounts.override {
+        # disables the upstream "goabackend" feature -- presumably "Gnome Online Accounts Backend"
+        # frees us from webkit_4_1, in turn.
+        enableBackend = false;
+        gvfs = pkgs.gvfs.override {
+          # saves 20 minutes of build time, for unused feature
+          samba = null;
+        };
+      };
+    };
+  };
+  chatty-latest = pkgs.chatty-latest.override {
+    evolution-data-server-gtk4 = pkgs.evolution-data-server-gtk4.override {
+      gnome-online-accounts = pkgs.gnome-online-accounts.override {
+        # disables the upstream "goabackend" feature -- presumably "Gnome Online Accounts Backend"
+        # frees us from webkit_4_1, in turn.
+        enableBackend = false;
+        gvfs = pkgs.gvfs.override {
+          # saves 20 minutes of build time, for unused feature
+          samba = null;
+        };
+      };
+    };
+  };
+in
+{
+  sane.programs.chatty = {
+    # package = chattyNoOauth;
+    package = chatty-latest;
+    suggestedPrograms = [ "gnome-keyring" ];
+    persist.private = [
+      ".local/share/chatty"  # matrix avatars and files
+      # not just XMPP; without this Chatty will regenerate its device-id every boot.
+      # .purple/ contains XMPP *and* Matrix auth, logs, avatar cache, and a bit more
+      ".purple"
+    ];
+  };
+}

hosts/common/programs/cozy.nix

@@ -0,0 +1,11 @@
+{ ... }:
+
+{
+  sane.programs.cozy = {
+    # cozy uses a sqlite db for its config and exposes no CLI options other than --help and --debug
+    persist.plaintext = [
+      ".local/share/cozy"  # sqlite db (config & index?)
+      ".cache/cozy"  # offline cache
+    ];
+  };
+}

hosts/common/programs/default.nix

@@ -0,0 +1,66 @@
+{ pkgs, ... }:
+
+{
+  imports = [
+    ./aerc.nix
+    ./alacritty.nix
+    ./assorted.nix
+    ./cantata.nix
+    ./chatty.nix
+    ./cozy.nix
+    ./dino.nix
+    ./element-desktop.nix
+    ./epiphany.nix
+    ./evince.nix
+    ./firefox.nix
+    ./fontconfig.nix
+    ./fractal.nix
+    ./fwupd.nix
+    ./g4music.nix
+    ./gajim.nix
+    ./git.nix
+    ./gnome-feeds.nix
+    ./gnome-keyring.nix
+    ./gnome-weather.nix
+    ./gpodder.nix
+    ./gthumb.nix
+    ./helix.nix
+    ./imagemagick.nix
+    ./jellyfin-media-player.nix
+    ./kitty
+    ./komikku.nix
+    ./koreader
+    ./libreoffice.nix
+    ./lemoa.nix
+    ./megapixels.nix
+    ./mepo.nix
+    ./mopidy.nix
+    ./mpv.nix
+    ./msmtp.nix
+    ./neovim.nix
+    ./newsflash.nix
+    ./nheko.nix
+    ./nix-index.nix
+    ./obsidian.nix
+    ./offlineimap.nix
+    ./rhythmbox.nix
+    ./ripgrep.nix
+    ./sfeed.nix
+    ./splatmoji.nix
+    ./steam.nix
+    ./sublime-music.nix
+    ./tangram.nix
+    ./tuba.nix
+    ./vlc.nix
+    ./wireshark.nix
+    ./xarchiver.nix
+    ./zeal.nix
+    ./zsh
+  ];
+
+  config = {
+    # XXX: this might not be necessary. try removing this and cacert.unbundled (servo)?
+    environment.etc."ssl/certs".source = "${pkgs.cacert.unbundled}/etc/ssl/certs/*";
+
+  };
+}

hosts/common/programs/dino.nix

@@ -0,0 +1,15 @@
+# usage:
+# - start a DM with a rando via
+#   - '+' -> 'start conversation'
+# - add a user to your roster via
+#   - '+' -> 'start conversation' -> '+'  (opens the "add contact" dialog)
+#   - this triggers a popup on the remote side asking them for confirmation
+#   - after the remote's confirmation there will be a local popup for you to allow them to add you to their roster
+# - to make a call:
+#   - ensure the other party is in your roster
+#   - open a DM with the party
+#   - click the phone icon at top (only visible if other party is in your roster)
+{ ... }:
+{
+  sane.programs.dino.persist.private = [ ".local/share/dino" ];
+}

hosts/common/programs/element-desktop.nix

@@ -0,0 +1,9 @@
+{ ... }:
+{
+  sane.programs.element-desktop = {
+    # creds/session keys, etc
+    persist.private = [ ".config/Element" ];
+
+    suggestedPrograms = [ "gnome-keyring" ];
+  };
+}

hosts/common/programs/epiphany.nix

@@ -0,0 +1,45 @@
+# epiphany web browser
+# - GTK4/webkitgtk
+#
+# usability notes:
+# - touch-based scroll works well (for moby)
+# - URL bar constantly resets cursor to the start of the line as i type
+#   - maybe due to the URLbar suggestions getting in the way
+{ pkgs, ... }:
+{
+  sane.programs.epiphany = {
+    # XXX(2023/07/08): running on moby without this hack fails, with:
+    # - `bwrap: Can't make symlink at /var/run: File exists`
+    # this could be due to:
+    # - epiphany is somewhere following a symlink into /var/run instead of /run
+    #   - (nothing in `env` or in this repo touches /var/run)
+    # - no xdg-desktop-portal is installed  (unlikely)
+    #
+    # a few other users have hit this, in different contexts:
+    # - <https://gitlab.gnome.org/GNOME/gnome-builder/-/issues/1164>
+    # - <https://github.com/flatpak/flatpak/issues/3477>
+    # - <https://github.com/NixOS/nixpkgs/issues/197085>
+    package = pkgs.epiphany.overrideAttrs (upstream: {
+      preFixup = ''
+        gappsWrapperArgs+=(
+          --set WEBKIT_DISABLE_SANDBOX_THIS_IS_DANGEROUS "1"
+        );
+      '' + (upstream.preFixup or "");
+    });
+    persist.private = [
+      ".cache/epiphany"
+      ".local/share/epiphany"
+      # also .config/epiphany, but appears empty
+    ];
+    mime.priority = 200;  # default priority is 100: install epiphany only as a fallback
+    mime.associations = let
+      desktop = "org.gnome.Epiphany.desktop";
+    in {
+      "text/html" = desktop;
+      "x-scheme-handler/http" = desktop;
+      "x-scheme-handler/https" = desktop;
+      "x-scheme-handler/about" = desktop;
+      "x-scheme-handler/unknown" = desktop;
+    };
+  };
+}