linux push notifs: fill in the bit about notification servers

content/blog/DRAFT-2023-11-06-mobile-linux-push-notifications/index.md

@@ -66,3 +66,91 @@ figuring out the byte pattern which achieves what you want is painful, so i've g
 enter... notification servers!
 
 ## Notification Servers
+
+when an application wants to sound a notification on iOS or Android when it's not in focus, it's not the application running on the phone which does that, but actually some other server (operated by or for the developer) which tells _Apple's_ server (or Google's) about the notification, and then it reaches your phone. the justification for that is precisely to allow the type of power-saving we're aiming for here, and because that approach is such a privacy nightmare passionate people have created open source alternatives to the official MITM servers.
+
+[UnifiedPush](https://unifiedpush.org/) defines the standards for each confusingly-named portion of this data flow,
+and [ntfy](https://ntfy.sh/) provides hosted and self-hosted implementations for all the major components. most mature applications provide a way for you to integrate all this: i'll show how to do this for Matrix-synapse and Prosody (XMPP). both can be configured server-side if you have access to that, or client-side *if your client exposes an option for it*.
+
+but first, let's prove this push notification system in an application-agnostic CLI workflow.
+
+### Minimal ntfy Workflow
+
+ntfy operates a free-to-use server for the MITM component, so install the CLI package for the publish/subscribe portions and we can test a wake-on-notification setup trivially.
+
+on your phone:
+```sh
+$ ntfy sub TEST_WOWLAN_TOPIC
+```
+
+then determine the local port that ntfy process connected from with `lsof -i4 -P`, and create a wake condition for it:
+
+```sh
+# clear out previous rules
+$ rtl8723cs-wowlan enable-clean
+$ rtl8723cs-wowlan tcp --dest-port 1234
+#^ replace 1234 with the port from `lsof`
+```
+
+then sleep the phone with `rtcwake -m mem`, and from another computer (or the [ntfy web interface](https://ntfy.sh/app)) send a notification:
+
+```sh
+$ ntfy pub TEST_WOWLAN_TOPIC "hello phone"
+```
+
+if all is well, your phone should awake and the `ntfy sub` command from earlier should have printed "hello phone"!
+
+
+### Hosting a ntfy instance
+
+this part is **optional**. most applications which speak UnifiedPush provide an option to only send a summary (like "new Matrix message") instead of the actual contents, so you only really need to concern yourself with this if you want the control of self-hosting.
+
+NixOS ships [options](https://search.nixos.org/options?show=services.ntfy-sh) for hosting your own ntfy. use it like:
+
+```nix
+services.ntfy-sh.enable = true;
+```
+
+ntfy listens on port 2586 by default, so you can subscribe like `ntfy sub MY_INSTANCE_ADDRESS:2586/TEST_WOWLAN_TOPIC` and you can use a wowlan rule that matches the source port instead of checking `lsof` manually: `rtl8723cs-wowlan tcp --source-port 2586`.
+
+however, running it this way allows anyone to use your server. the easy way to overcome this is to treat the topic as a shared secret, and forbid use of any topic except your secret topic. naturally, that only works if you secure the connection so do all this behind a TLS-capable reverse proxy like nginx:
+
+```nix
+services.ntfy-sh.enable = true;
+
+services.nginx.virtualHosts."MY.HOST" = {
+  forceSSL = true;
+  listen = [
+    { addr = "0.0.0.0"; port = 2587; ssl = true; }
+    { addr = "0.0.0.0"; port = 443;  ssl = true; }
+  ];
+  locations."/" = {
+    proxyPass = "http://127.0.0.1:2586";
+    proxyWebsockets = true;  #< support websocket upgrades. without that, `ntfy sub` hangs silently
+    recommendedProxySettings = true; #< adds headers so ntfy logs include the real IP
+    extraConfig = ''
+      # absurdly long timeout (86400s=24h) so that we never hang up on clients.
+      proxy_read_timeout 86400s;
+    '';
+  };
+};
+
+services.ntfy-sh.settings = {
+  base-url = "https://MY.HOST";
+  behind-proxy = true;
+  auth-default-access = "deny-all";
+};
+systemd.services.ntfy-sh.preStart = ''
+  # note that this will fail upon first run, i.e. before ntfy has created its db.
+  # just restart the service.
+  ${pkgs.ntfy-sh}/bin/ntfy access everyone TEST_WOWLAN_TOPIC read-write
+'';
+```
+
+deploy that and subscribe to the https url this time. note the port change to 2587: you still want a unique port against which you can write a wowlan rule, and it's easier to put nginx on a new port than ntfy's default 2586.
+
+if you're thorough, you might notice some spurious wakeups with this setup. ntfy sends keep-alive packets every 45 seconds, but that's no good for us! best is to disable those keep-alives, and only put the phone to sleep for shortish durations (e.g. 10 minutes): i'll revisit the nuances around this near the end.
+
+```nix
+services.ntfy-sh.settings.keepalive-interval = "30m";
+```