Since https://github.com/NixOS/nixpkgs/pull/281374, the
nixpkgs-check-by-name tooling is pinned to a specific /nix/store path to
avoid having to evaluate Nixpkgs in CI.
The same path is used for local runs, but that doesn't actually work
when you're trying to run it on a platform different from CI.
This commit makes it work by being clearer about platforms and making
local runs check out the correct Nixpkgs to evaluate the tool from.
Before this, the tool for CI would update when nixos-unstable updated,
which is kind of terrible because you don't know when it happens, and it
might break master.
In fact, the tooling _right now_ has a serious bug and shouldn't be used!
This PR addresses this by _pinning_ the tooling in Nixpkgs itself.
Updating the tooling now requires two PRs:
- The first PR to update the tooling source
- (wait for Hydra to build and publish it in nixos-unstable)
- The second PR to update the pinned tooling
In turn you know exactly when the changes are going to take effect.
This change however has additional benefits:
- It makes CI more reproducible, because it doesn't depend on the state
of nixos-unstable anymore
- Updates to the tooling can be tested with the workflow itself,
because PRs that update the pinned tool will be tested on the updated
version
- CI gets a sizable speed boost, because there's no need to download and
evaluate a channel anymore
- It makes it more realistic to move the source of the tool into a
separate repository
- It removes the brittle branch-specific logic that was previously
needed to ensure that release branches use their own version of the
tooling.
Now that we have a script to run the check locally,
there's no real need to output the information to reproduce anymore,
which allows cleaning up the CI workflow.
Furthermore, this prepares the CI workflow to be passed `--base`, as
introduced recently.
For reproducibility.
Command:
```shell
for file in .github/workflows/*.y*ml; do
npx pin-github-action --comment=' {ref}' "$file"
done
```
Then had to manually replace all the versions with accurate specifiers
(for example, "v4" → "v4.1.1" in case of `actions/checkout`).
We've had a recent PR CI mass failure event, ultimately caused by the
mergeability check GitHub API not returning a result.
But due to the `pkgs/by-name` check workflow not backing off
appropriately between retries, it pummeled the API, resulting in
exceeding the API rate limit:
https://github.com/NixOS/nixpkgs/actions/runs/7010089143/job/19069845070
This commit fixes that for the future by implementing a retry strategy limited to three
retries, with exponential backoff
Previously, even if the check also failed on the base branch, it looked
like the PR introduced the failure.
We can easily have a better error message for such cases.
Meanwhile this also paves the road for something like
https://github.com/NixOS/nixpkgs/issues/256788
This introduces the `pkgs/by-name` directory as proposed by RFC 140.
Included are:
- The implementation to add packages defined in that directory to the
top-level package scope
- Contributer documentation on how to add packages to it
- A GitHub Actions workflow to check the structure of it on all PRs
