From b6296e4426c4466646cd073631645427bb443ce4 Mon Sep 17 00:00:00 2001 From: Colin Date: Fri, 15 Nov 2024 12:02:52 +0000 Subject: [PATCH] blog: WIP: draft the nixos-kernel-hacking article --- .../DRAFT-2024-11-15-nixos-kernel-hacking.md | 245 ++++++++++++++++++ 1 file changed, 245 insertions(+) create mode 100644 content/blog/DRAFT-2024-11-15-nixos-kernel-hacking.md diff --git a/content/blog/DRAFT-2024-11-15-nixos-kernel-hacking.md b/content/blog/DRAFT-2024-11-15-nixos-kernel-hacking.md new file mode 100644 index 0000000..8e58510 --- /dev/null +++ b/content/blog/DRAFT-2024-11-15-nixos-kernel-hacking.md @@ -0,0 +1,245 @@ ++++ +title = "Painless kernel hacking + device bringup on NixOS" +description = "patching kernels and tweaking device trees *without* lengthy rebuilds" +# date = 2024-11-15 +# to publish: uncomment the `date` field and remove the `DRAFT` prefix ++++ + +one of my NixOS devices is a [PinePhone Pro]. +at time of writing, it does boot to a working display/touchscreen and even WiFi on stock NixOS, +however other basic features like audio and battery readouts aren't available. +actually, these features *are* available on distros like [postmarketOS] which ship kernel forks +specifically for this. but maintaining a fork is at least a bit of work, and NixOS gives you lots +_better_ ways to tweak your kernel than simply building the whole thing from a different source tree. + +## Kernel Pain Points + +skip ahead if you're already familiar with the NixOS kernel options. +the straightforward way to ship custom kernels on NixOS is something like: + +```nix +{ pkgs, ... }: +{ + boot.kernelPackages = pkgs.linuxPackagesFor (pkgs.buildLinux { + src = fetchgit { + # ... + }; + # ... + }); +} +``` +or if your kernel patches are simple enough, then track the NixOS kernel and patch it instead: +```nix +{ ... }: +{ + boot.kernelPatches = [ + { + name = "my-kernel-patch"; + patch = ./my-kernel-patch.patch; + } + ]; +} +``` + +well the huge downside to this is that tweaking just a single line in `my-kernel-patch.patch` forces +a complete rebuild of your kernel: your iteration cycle time might well be 30 minutes with this approach. + + +## Faster Kernel Iterations + +until we get something like [dynamic derivations], the path out of costly iteration is to ship whichever tweaks you're pursuing via some other mechanism than as part of the top-level kernel derivation. +the kernel is fairly pluggable; consider: + +1. device tree files can be loaded at runtime. +2. kernel modules can be loaded at runtime. + +with the right config, either of these things can be freely modified without forcing a kernel build. + +## Shipping Device Tree Patches + +Linux uses [device tree] files to determine which peripherals are available on your +device and how to use them. it's more relevant for embedded devices than for traditional x86 PCs. + +the typical boot flow for an embedded device is that the bootloader (U-Boot) knows the name of +the device it's running on (perhaps because it was built _specifically_ for that device) and communicates +this to the kernel when it hands over control of the device. the name of this device tree might look something like "pine64,pinephone-pro". + +device trees are composable by design; the spec already defines the concept of an "overlay", +such that the actual device tree to apply to a device should be the union of whatever's +defined in the kernel source tree plus any overlays that apply to the same device. +NixOS makes this feature available via the [`hardware.deviceTree.overlays`] option. + +one issue with the PinePhone Pro on stock Linux is that the volume-down button doesn't work. +but it works in the pine64 kernel, because they patched the device tree file like so: +```diff +--- arch/arm64/boot/dts/rockchip/rk3399-pinephone-pro.dts ++++ arch/arm64/boot/dts/rockchip/rk3399-pinephone-pro.dts +@@ -48,11 +48,11 @@ / { + adc-keys { + compatible = "adc-keys"; + io-channels = <&saradc 1>; + io-channel-names = "buttons"; + keyup-threshold-microvolt = <1600000>; + poll-interval = <100>; + button-down { + label = "Volume Down"; + linux,code = ; +- press-threshold-microvolt = <600000>; ++ press-threshold-microvolt = <400000>; + }; +``` + +this patch can alternately be represented as a [Device Tree Overlay] (DTO): + +```dts +// file: rk3399-pinephone-pro-lradc-fix.dtso + +// metadata for the device tree compiler +/dts-v1/; +/plugin/; + +// instruct the system to only apply this overlay +// to PinePhone Pro, and not any other devices. +/ { + compatible = "pine64,pinephone-pro"; +}; + +// address the parent node we want to patch, +// and then inject a new property value. +// this overrides `press-threshold-microvolt` +// while leaving all other properties unchanged. +&{/adc-keys/button-down} { + press-threshold-microvolt = <400000>; +}; +``` + +and then we can ship it in NixOS like so: + +```nix +{ ... }: +{ + hardware.deviceTree.overlays = [ + { + name = "rk3399-pinephone-pro-lradc-fix"; + dtsFile = ./rk3399-pinephone-pro-lradc-fix.dtso; + } + ]; +} +``` + +after adding the above to a stock NixOS config, and deploying +to a PinePhone Pro, the volume down button should now be fixed! + +## Shipping Custom Kernel Modules + +one of the less trivial patches yet to be mainlined is support for +battery monitoring on the PinePhone Pro. this feature is supported +in downstream kernels by shipping two new kernel modules: +`rk818_battery` and `rk818_charger`. + +these modules are device drivers: they detect a `rk818-battery` +device somewhere on the system, and then run code in response +to that (namely, enable some voltage regulators, and create sysfs +nodes to let userspace interact with the device). + +code wise, the typical driver is a single .c file with few if any +direct dependencies on other drivers. most driver modules are +standalone in the same sense that each nix package is standalone. + +then, we can ship new kernel modules by: +- defining a nix derivation that compiles the kernel module. +- telling nix to deploy that on the kernel's module search path. +- instructing the kernel to load our module. +- shipping a device tree overlay to associate the relevant devices + with the driver our module provides. + +### Building a Kernel Module + +TODO + +### Deploying a Kernel Module + +TODO + +### Device Tree + Module Integration + +TODO + +## Patching an Upstream Kernel Module + +this all works, however the PinePhone Pro battery integration +requires not just a new kernel module, but also to patch an existing +kernel module (`rk8xx-i2c`). since the kernel module is defined in-tree, the natural +way to do that would force a kernel rebuild every time we adjust the patches. + +we can build our patched module out-of-tree using the same method +as we build a wholly new module out-of-tree (writing the derivation +as some `patches` atop `src = linux-latest`, or just copying the entire +module source into our repo instead of tracking patches). then we just +configure the kernel to prefer our module over its in-tree module. + +a first approach might be to configure the kernel with `CONFIG_MFD_RK8XX_I2C=n`, +then ship our module as above. this works: + +```nix +{ lib, ... }: +{ + boot.kernelPatches = [ + { + name = "rk8xx-i2c-out-of-tree"; + patch = null; + extraStructuredConfig = with lib.kernel; { + MFD_RK8XX_I2C = no; + }; + } + ]; + +} +``` + +this will result in _one_ kernel rebuild, and then you can freely +edit your out-of-tree `rk8xx-i2c` module without costly rebuilds. +if you find yourself patching a lot of modules, then it may be +preferable to simply build _all_ in-tree modules as dynamic modules, +and configure out-of-tree modules to take precedence over the in-tree ones. +remove the `MFD_RK8XX_I2C = no` patch and replace it with this: + +```nix +{ ... }: +{ + nixpkgs.config.hostPlatform.linux-kernel = { + # build every module known to the mainline kernel + autoModules = true; + # and build those as dynamically loaded modules (`=m`), instead of builtins (`=y`). + preferBuiltin = false; + }; + + # default nixos behavior is to error if a kernel module is provided by more + # than one package. but we're doing that intentionally, so patch the logic + # from (AKA pkgs.aggregateModules) + # to not complain. + system.modulesTree = lib.mkForce [( + (pkgs.aggregateModules + ( config.boot.extraModulePackages ++ [ config.boot.kernelPackages.kernel ]) + ).overrideAttrs { + # when collisions are ignored, earlier items override the contents of later items + ignoreCollisions = true; + } + )]; +} +``` + + +## Conclusion + +and there you have it! if you found anything here useful, then my request to you +is to upstream your kernel work so that the next reader of this blog doesn't have +to go through the same pain 😛 + + +[PinePhone Pro]: https://en.wikipedia.org/wiki/PinePhone_Pro +[postmarketOS]: https://postmarketos.org/ +[dynamic-derivations]: https://github.com/NixOS/rfcs/pull/92 +[device tree]: https://docs.kernel.org/devicetree/usage-model.html +[`hardware.deviceTree.overlays`]: https://search.nixos.org/options?channel=unstable&show=hardware.deviceTree.overlays&from=0&size=50&sort=relevance&type=packages&query=hardware.deviceTree.overlays +[Device Tree Overlay]: TODO