blog: nixos-upstreaming: additional editing pass
This commit is contained in:
parent
ba20dbc164
commit
5987581f12
|
|
@ -3,14 +3,14 @@ title = "How To Casually Contribute to NixOS/Nixpkgs"
|
|||
description = "DRY flake-based workflow for upstreaming simple package changes"
|
||||
+++
|
||||
|
||||
as a [nix/nixOS](https://nixos.org) user, you'll inherently find yourself writing code, and some of this code might be valuable to others: be it new packages, improvements upon existing packages, or new services/options. part of what makes nix so successful (IMO) is the ease of sharing work and upstreaming, so i'm setting out here to show some easy workflows for doing so.
|
||||
as a [nix/nixOS](https://nixos.org) user, you'll inherently find yourself writing code, and some of this code might be valuable to others: be it new packages, improvements upon existing packages, or new services/options. part of what makes nix successful (IMO) is the ease of sharing work and upstreaming, so i'm setting out here to show some easy workflows for doing so.
|
||||
|
||||
the number one thing you can do -- if you're not already -- is host your nix config publicly. for example, mine is [here](https://git.uninsane.org/colin/nix-files), and even without advertising it i occasionally hear from people in my orbit that they've copied some approach. as a beginner i had some public configs i found on google or [wiby](https://wiby.me) which i routinely consulted, even though many of those authors probably never heard a word from the people like me who found use in their work.
|
||||
the number one thing you can do -- if you're not already -- is host your nix config publicly. for example, mine is [here](https://git.uninsane.org/colin/nix-files), and even without advertising it i occasionally hear from people in my orbit that they've found it useful. as a beginner i had some public configs i found on google or [wiby](https://wiby.me) which i routinely consulted, even though many of those authors probably never heard a word from the people like me who found use in their work.
|
||||
|
||||
## Authoring New Packages
|
||||
let's say you find a package you want but isn't yet packaged. first, confirm it doesn't exist by searching the package name (and likely variants) on github. remove all filters, so that you're searching both PRs and issues: maybe somebody opened a packaging request and there's already something to work off of. for [example](https://github.com/NixOS/nixpkgs/issues?q=zecwallet-lite).
|
||||
let's say you find some software you want that isn't yet packaged. first, confirm it doesn't exist by searching the package name (and probable variants) on github. remove all filters, so that you're searching both PRs and issues: maybe somebody opened a packaging request and there's already something to work off of. for [example](https://github.com/NixOS/nixpkgs/issues?q=zecwallet-lite).
|
||||
|
||||
having confirmed that, let's add a new package. anywhere in your config, add an [overlay](https://nixos.wiki/wiki/Overlays):
|
||||
having confirmed that, let's make a new package. anywhere in your config, add an [overlay](https://nixos.wiki/wiki/Overlays):
|
||||
```nix
|
||||
{ ... }:
|
||||
{
|
||||
|
|
@ -22,7 +22,7 @@ having confirmed that, let's add a new package. anywhere in your config, add an
|
|||
}
|
||||
```
|
||||
|
||||
i'm just going to show the process i followed when upstreaming that zecwallet-lite package. create the package directory so that the callPackage knows what to reference:
|
||||
i'm just going to show the process i followed when upstreaming that zecwallet-lite package. create the package directory so that `callPackage` knows what to reference:
|
||||
```sh
|
||||
$ mkdir zecwallet-lite
|
||||
$ touch zecwallet-lite/default.nix
|
||||
|
|
@ -37,7 +37,7 @@ now i locate the upstream package distribution. zecwallet-lite is distributed as
|
|||
|
||||
```sh
|
||||
$ cd ~/
|
||||
$ git clone git@github.com:uninsane/nixpkgs.git
|
||||
$ git clone https://github.com/NixOS/nixpkgs.git
|
||||
$ cd nixpkgs
|
||||
$ rg appimage pkgs/applications -l | xargs wc -l | sort -h
|
||||
19 pkgs/applications/misc/remnote/default.nix
|
||||
|
|
@ -74,18 +74,18 @@ appimageTools.wrapType2 rec {
|
|||
|
||||
so copy that to `zecwallet-lite/default.nix` and edit the values to match your appimage package. set `sha256 = lib.fakeHash`, and nix will tell you the actual hash.
|
||||
|
||||
then build the package, test it, and tweak it until you're happy with it. the simplest way to do that is add it to `environment.packages` and `nixos-rebuild switch`. most of the time if it builds, it'll run (and you might just need to patch up some icon/.desktop files, etc).
|
||||
then build the package, test it, and tweak it until you're happy. the simplest way to do that is add it to [`environment.systemPackages`](https://search.nixos.org/options?channel=unstable&show=environment.systemPackages&from=0&size=50&sort=relevance&type=packages&query=packages) and `nixos-rebuild switch`. normally, if it builds, it'll run (and you might just need to patch up some icon/.desktop files, etc).
|
||||
|
||||
## Upstreaming New Packages
|
||||
|
||||
so the package builds, you've used it long enough to be confident in it: time to upstream it from your silo into the main nixpkgs. even if you're strictly self-interested, upstreaming means less custom code for you to maintain, less burden keeping the package version up-to-date since others may help with that, and less time spent rebuilding it when a dependency updates since it'll be present in the official build caches. it often takes all of ten minutes once you're familiar.
|
||||
so the package builds, you've used it long enough to be confident in it: time to upstream it from your silo into the main nixpkgs. even if you're strictly self-interested, upstreaming means less custom code for you to maintain, less burden keeping the version up-to-date since others may help with that, and less time spent rebuilding it when a dependency updates since it'll be present in the official nixos caches. it often takes all of ten minutes once you're familiar.
|
||||
|
||||
go back to your `nixpkgs` repo and check out the `master` branch. choose a reasonable folder for the package, like `pkgs/applications/misc/`, and copy your package there:
|
||||
```sh
|
||||
$ cp -R /etc/nixos/zecwallet-lite ~/nixpkgs/pkgs/applications/misc/
|
||||
```
|
||||
|
||||
then take your `callPackage` invocation from your overlay and place it in `pkgs/top-level/all-packages.nix` instead. `git add` the file, and make sure the package builds:
|
||||
then take the `callPackage` invocation from your overlay and place it in `pkgs/top-level/all-packages.nix` instead. `git add` the file, and make sure the package builds:
|
||||
```sh
|
||||
~/nixpkgs$ nix build '.#zecwallet-lite'
|
||||
~/nixpkgs$ ./result/bin/zecwallet-lite # make sure it runs
|
||||
|
|
@ -93,9 +93,9 @@ then take your `callPackage` invocation from your overlay and place it in `pkgs/
|
|||
|
||||
commit the changes, push to a github account, and open a PR. github should pre-populate the PR description with a checklist for you to complete. near as i can tell, you don't have to check _every_ item: it's just a way for reviewers to know where to focus.
|
||||
|
||||
## Cleanup Work
|
||||
## Cleanup and DRY
|
||||
|
||||
you could leave the duplicate package definition both in your `/etc/nixos` repo _and_ in your `nixpkgs` repo, and wait until the next release of nixos/nixpkgs before cleaning up; or you can remove the package from your nixos repo and import it by reference to remove the duplication.
|
||||
you could leave the duplicate package definition both in your `/etc/nixos` repo _and_ in your `nixpkgs` repo and wait until the next release of nixos/nixpkgs before removing the former; or you can remove the package from your nixos repo now and import it by reference to avoid repetition.
|
||||
|
||||
if you're using a flake for your system config, it likely looks something like this right now:
|
||||
```nix
|
||||
|
|
@ -143,7 +143,7 @@ instead, we use a trick shown [here](https://github.com/NixOS/nix/issues/3920#is
|
|||
|
||||
that is, we take the base nixpkgs repo/derivation, create a new derivation which patches the original nixpkgs, and then import the result of that derivation so that the rest of our flake refers to this patched nixpkgs.
|
||||
|
||||
the version above expects your patches to live in the same repo as your config flake, but if the goal is to avoid duplication we'd rather point these at the the PRs live on github. you can use `fetchpatch` for this. if your PR is [180960](https://github.com/NixOS/nixpkgs/pull/180960), append `.diff` to the URL to get a patch. the `let` block above becomes this:
|
||||
the version above expects your patches to live in the same repo as your config flake, but if the goal is to avoid duplication we'd rather point these at the PRs hosted on github. you can use [`fetchpatch`](https://nixos.org/manual/nixpkgs/stable/#fetchurl) for this. if your PR is [180960](https://github.com/NixOS/nixpkgs/pull/180960), append `.diff` to the URL to get a patch. the `let` block above becomes this:
|
||||
|
||||
```nix
|
||||
let
|
||||
|
|
@ -161,16 +161,18 @@ let
|
|||
nixosSystem = import (patchedPkgs + "/nixos/lib/eval-config.nix");
|
||||
```
|
||||
|
||||
and we're set! as before, if you don't know the hash then set `sha256=nixpkgs.lib.fakeHash` as before. you can freely push to your open PR: to have the changes reflect on your system just reset/update the hash and `nixos-rebuild switch`. once your PR is merged, the next `nix flake update` might include your patch already, in which case nix will error when building `nixpkgs-patched`. at that point you can safely remove this entry from `patches`.
|
||||
and we're set! as before, if you don't know the hash just set it to `nixpkgs.lib.fakeHash`. you can freely push to your open PR: to have the changes reflect on your system just reset/update the hash and `nixos-rebuild switch`. once your PR is merged, the next `nix flake update` might include your patch already, in which case nix will error when building `nixpkgs-patched`. at that point you can safely remove this entry from `patches`.
|
||||
|
||||
## Upstreaming Other Changes
|
||||
|
||||
since this approach is ultimately just applying patches onto `nixpkgs`, it's not limited to only new packages. if you want to update a package version, add a new config option, etc, you can follow effectively the same process: clone `nixpkgs`, checkout the master branch, make a change and `nix build` the result, commit & open a PR, `fetchpatch` the PR into your flake.
|
||||
since this approach is ultimately just applying patches onto `nixpkgs`, it's not limited to only new packages. to update a package version, add a new config option, etc, you can follow effectively the same process: clone `nixpkgs`, checkout the master branch, make a change and `nix build` the result, commit & open a PR, and `fetchpatch` the PR into your flake.
|
||||
|
||||
if you're making changes in a hot area of the codebase, the diff from your PR (which targets master) might not apply cleanly to your flake (which pulls from a release or nixos-unstable). in that case you can `git log` the files you're changing and use the same `fetchpatch` trick to cherry-pick whatever prerequisite commits you need so that your patch applies.
|
||||
|
||||
## Closing Thoughts
|
||||
|
||||
i described here the workflow i adopted as i first began contributing. with something as configurable as nix, many people use many different workflows. some prefer maintaining their own long-running fork of nixpkgs wherein they cherry-pick all these PRs and periodically rebase against nixos-unstable (or a release branch), and point their flake directly at their fork of nixpkgs. your workflow will surely evolve as you settle in, but hopefully this gives you enough inspiration to get started :^)
|
||||
i described here the workflow i found easiest to get started with. with something as configurable as nix, many people use many different workflows. some prefer maintaining their own long-running fork of nixpkgs wherein they cherry-pick all these PRs and periodically rebase against nixos-unstable (or a release branch), and point their flake directly at their fork of nixpkgs. your workflow will surely evolve as you settle in, but hopefully this gives you enough inspiration to get started :-)
|
||||
|
||||
finally, don't be afraid to use the `nixos-unstable` branch if you're an intermediate nix user! it's really not as unstable as the name suggests. i experience maybe one package break per month across four deployments, almost always manifesting via a build failure and hence not causing any real trouble -- just a deferred update. that tiny inconvenience is worth it to have easier access to the more rapidly moving parts of nixpkgs.
|
||||
finally, don't be afraid to use the `nixos-unstable` branch if you're an intermediate nix user! it's really not as unstable as the name suggests. i experience maybe one package break per month across four deployments, almost always manifesting via a build failure and hence not causing any real trouble -- just a deferred update. that tiny inconvenience is worth it to have easier access to the more rapidly moving parts of nixpkgs.
|
||||
|
||||
as always, don't hesitate to [contact me](@/about.md) or reach out on the [forums/chat](https://nixos.org/community/).
|
||||
Loading…
Reference in New Issue