colin
/
uninsane
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

nixos upstreaming: editing pass/address TODOs

This commit is contained in:
colin 2022-10-13 20:04:13 -07:00
parent 25df079540
commit ba20dbc164
1 changed files with 16 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
content/blog/DRAFT-2022-07-09-nixos-upstreaming.md
View File

@ -1,6 +1,6 @@
+++
title = "How To Casually Contribute to NixOS"
description = "remove DRAFT prefix and add a date field to link this from the index and feeds"
title = "How To Casually Contribute to NixOS/Nixpkgs"
description = "DRY flake-based workflow for upstreaming simple package changes"
+++
as a [nix/nixOS](https://nixos.org) user, you'll inherently find yourself writing code, and some of this code might be valuable to others: be it new packages, improvements upon existing packages, or new services/options. part of what makes nix so successful (IMO) is the ease of sharing work and upstreaming, so i'm setting out here to show some easy workflows for doing so.
@ -8,9 +8,9 @@ as a [nix/nixOS](https://nixos.org) user, you'll inherently find yourself writin
the number one thing you can do -- if you're not already -- is host your nix config publicly. for example, mine is [here](https://git.uninsane.org/colin/nix-files), and even without advertising it i occasionally hear from people in my orbit that they've copied some approach. as a beginner i had some public configs i found on google or [wiby](https://wiby.me) which i routinely consulted, even though many of those authors probably never heard a word from the people like me who found use in their work.
## Authoring New Packages
let's say you find a package you want but isn't yet packaged. first, confirm it doesn't exist by searching the package name (and likely variants) on github. remove all filters, so that you're searching both PRs and issues: maybe somebody opened a [package request] (TODO: link docs) and there's already something to work off of. for [example](https://github.com/NixOS/nixpkgs/issues?q=zecwallet-lite).
let's say you find a package you want but isn't yet packaged. first, confirm it doesn't exist by searching the package name (and likely variants) on github. remove all filters, so that you're searching both PRs and issues: maybe somebody opened a packaging request and there's already something to work off of. for [example](https://github.com/NixOS/nixpkgs/issues?q=zecwallet-lite).
having confirmed that, let's add a new package. anywhere in your config, add an [overlay](TODO: link docs):
having confirmed that, let's add a new package. anywhere in your config, add an [overlay](https://nixos.wiki/wiki/Overlays):
```nix
{ ... }:
{
@ -72,20 +72,20 @@ appimageTools.wrapType2 rec {
}
```
so copy that to `zecwallet-lite/default.nix` and edit the values to match your appimage package. for the hash, you can just change one character so it fails validation, and nix will tell you the actual hash (hacky, but easy process to remember).
so copy that to `zecwallet-lite/default.nix` and edit the values to match your appimage package. set `sha256 = lib.fakeHash`, and nix will tell you the actual hash.
then build the package, test it, and tweak it until you're happy with it. the simplest way to do that is add it to `environment.packages` and `nixos-rebuild switch`. most of the time if it builds, it'll run (and you might just need to patch up some icon/.desktop files, etc).
## Upstreaming New Packages
so the package builds, you've used it long enough to be confident in it: time to upstream it from your silo into the main nixpkgs. even if you're purely self-interested, upstreaming means less custom code for you to maintain, less burden keeping the package version up-to-date since others will be helping you with this, and less time spent rebuilding it when a dependency updates since it'll be present in the official build caches. it often takes all of ten minutes once you're familiar.
so the package builds, you've used it long enough to be confident in it: time to upstream it from your silo into the main nixpkgs. even if you're strictly self-interested, upstreaming means less custom code for you to maintain, less burden keeping the package version up-to-date since others may help with that, and less time spent rebuilding it when a dependency updates since it'll be present in the official build caches. it often takes all of ten minutes once you're familiar.
go back to your `nixpkgs` repo and check out the `master` branch. guess a reasonable folder for the package, like `pkgs/applications/misc/`, and copy your package there:
go back to your `nixpkgs` repo and check out the `master` branch. choose a reasonable folder for the package, like `pkgs/applications/misc/`, and copy your package there:
```sh
$ cp -R /etc/nixos/zecwallet-lite ~/nixpkgs/pkgs/applications/misc/
```
then take your `callPackage` invocation from your overlay and place it in `pkgs/top-level/all-packages.nix` instead. `git add` the file, and then make sure the package builds:
then take your `callPackage` invocation from your overlay and place it in `pkgs/top-level/all-packages.nix` instead. `git add` the file, and make sure the package builds:
```sh
~/nixpkgs$ nix build '.#zecwallet-lite'
~/nixpkgs$ ./result/bin/zecwallet-lite # make sure it runs
@ -95,9 +95,9 @@ commit the changes, push to a github account, and open a PR. github should pre-p
## Cleanup Work
so you could leave the duplicate package definition both in your `/etc/nixos` repo _and_ in your `nixpkgs` repo, and wait until the next release of nixos/nixpkgs before cleaning up, or you can reference commits in the nixpkgs repository _or in a PR_ from your nixos configuration directly, and consolidate this now.
you could leave the duplicate package definition both in your `/etc/nixos` repo _and_ in your `nixpkgs` repo, and wait until the next release of nixos/nixpkgs before cleaning up; or you can remove the package from your nixos repo and import it by reference to remove the duplication.
if you're using a flake for your system config, it probably looks something like this right now:
if you're using a flake for your system config, it likely looks something like this right now:
```nix
{
inputs = {
@ -141,9 +141,9 @@ instead, we use a trick shown [here](https://github.com/NixOS/nix/issues/3920#is
}
```
that is, we take the base nixpkgs repo, apply patches to it (creating a new derivation), and then import the result of that derivation so that the rest of our flake refers to this patched nixpkgs.
that is, we take the base nixpkgs repo/derivation, create a new derivation which patches the original nixpkgs, and then import the result of that derivation so that the rest of our flake refers to this patched nixpkgs.
the version above expects your patches to live in your config repo, but if the goal is to avoid duplication we'd rather point these at the the PRs live on github. we can use `fetchpatch` for this. if our PR is [180960](https://github.com/NixOS/nixpkgs/pull/180960), we can append `.diff` to the URL to get a patch. the `let` block above becomes this:
the version above expects your patches to live in the same repo as your config flake, but if the goal is to avoid duplication we'd rather point these at the the PRs live on github. you can use `fetchpatch` for this. if your PR is [180960](https://github.com/NixOS/nixpkgs/pull/180960), append `.diff` to the URL to get a patch. the `let` block above becomes this:
```nix
let
@ -161,15 +161,16 @@ let
nixosSystem = import (patchedPkgs + "/nixos/lib/eval-config.nix");
```
and we're set! as before, if you don't know the hash then set `sha256` to any value not used elsewhere and nix will tell you the real hash when you build. you can freely push to your open PR: to have the changes reflect on your system just update the hash and `nixos-rebuild switch`. once your PR is merged, the next `nix flake update` might include your patch already in which case nix will error when building `nixpkgs-patched`. at that point you can safely remove this entry from `patches`.
and we're set! as before, if you don't know the hash then set `sha256=nixpkgs.lib.fakeHash` as before. you can freely push to your open PR: to have the changes reflect on your system just reset/update the hash and `nixos-rebuild switch`. once your PR is merged, the next `nix flake update` might include your patch already, in which case nix will error when building `nixpkgs-patched`. at that point you can safely remove this entry from `patches`.
## Upstreaming Other Changes
since this approach is ultimately just applying patches onto `nixpkgs`, it's not limited to only new packages. if you want to update a package version, add a new config option, etc, you can follow effectively the same process: clone `nixpkgs`, checkout the master branch, make a change and `nix build` the result, commit & open a PR, `fetchpatch` the PR into your flake.
if you're making changes in a hot area of the codebase, the diff from your PR (which targets master) might not apply cleanly to your flake (which pulls from a release or nixos-unstable). in that case you can `git log` the files you're changing and use the same `fetchpatch` trick to cherry-pick whatever you need so that your patch applies cleanly.
if you're making changes in a hot area of the codebase, the diff from your PR (which targets master) might not apply cleanly to your flake (which pulls from a release or nixos-unstable). in that case you can `git log` the files you're changing and use the same `fetchpatch` trick to cherry-pick whatever prerequisite commits you need so that your patch applies.
## Closing Thoughts
i described here the workflow i adopted as i first began contributing. with something as configurable as nix, many people use many different workflows. some prefer maintaining their own long-running fork of nixpkgs wherein they cherry-pick all these PRs and periodically rebase against nixos-unstable, or a release branch, and have their flake point directly to their fork of nixpkgs. your workflow will surely evolve as you settle in, but hopefully this gives you enough inspiration to get started :^)
i described here the workflow i adopted as i first began contributing. with something as configurable as nix, many people use many different workflows. some prefer maintaining their own long-running fork of nixpkgs wherein they cherry-pick all these PRs and periodically rebase against nixos-unstable (or a release branch), and point their flake directly at their fork of nixpkgs. your workflow will surely evolve as you settle in, but hopefully this gives you enough inspiration to get started :^)
finally, don't be afraid to use the `nixos-unstable` branch if you're an intermediate nix user! it's really not as unstable as the name suggests. i experience maybe one package break per month across four deployments, almost always manifesting via a build failure and hence not causing any real trouble -- just a deferred update. that tiny inconvenience is worth it to have easier access to the more rapidly moving parts of nixpkgs.