Since https://github.com/NixOS/nixpkgs/pull/281374, the
nixpkgs-check-by-name tooling is pinned to a specific /nix/store path to
avoid having to evaluate Nixpkgs in CI.
The same path is used for local runs, but that doesn't actually work
when you're trying to run it on a platform different from CI.
This commit makes it work by being clearer about platforms and making
local runs check out the correct Nixpkgs to evaluate the tool from.
Before this, the tool for CI would update when nixos-unstable updated,
which is kind of terrible because you don't know when it happens, and it
might break master.
In fact, the tooling _right now_ has a serious bug and shouldn't be used!
This PR addresses this by _pinning_ the tooling in Nixpkgs itself.
Updating the tooling now requires two PRs:
- The first PR to update the tooling source
- (wait for Hydra to build and publish it in nixos-unstable)
- The second PR to update the pinned tooling
In turn you know exactly when the changes are going to take effect.
This change however has additional benefits:
- It makes CI more reproducible, because it doesn't depend on the state
of nixos-unstable anymore
- Updates to the tooling can be tested with the workflow itself,
because PRs that update the pinned tool will be tested on the updated
version
- CI gets a sizable speed boost, because there's no need to download and
evaluate a channel anymore
- It makes it more realistic to move the source of the tool into a
separate repository
- It removes the brittle branch-specific logic that was previously
needed to ensure that release branches use their own version of the
tooling.
nix-build failed because the tests assume to run in a CWD equal to the
project root, which is not the case in the derivation build.
This commit fixes it by not using hacky `..` references to paths,
and instead uses NIX_PATH for all implicit Nix testing path
dependencies.
Also the root of the `lib` path gets passed in from the `default.nix`
file, so all the relative path handling is done by Nix during evaluation
already, and in the Nix store when possible.
- Typo
- Rename AttributeRatchet to ToNixpkgsProblem
- Make the compare trait method into a RatchetState method
Co-Authored-By: Philip Taron <philip.taron@gmail.com>
This makes the attribute ratchet check logic more re-usable, which will
be used in a future commit.
It also renames the ratchet states to something more intuitive
Due to the check soon depending on the base branch (see `--base`),
the CI check can't reasonably share all code with a local check.
We can still make a script to run it locally, just not sharing all code.
Previously, not passing `--base` would enforce the most strict checks.
While there's currently no actual violation of these stricter checks,
this does not match the previous behavior.
This won't matter once CI passes `--base`, the code handling the
optionality can be removed then.
This implements the option for a gradual migration to stricter checks.
For now this is only done for the check against empty non-auto-called
callPackage arguments, but in the future this can be used to ensure all
new packages make use of `pkgs/by-name`.
This is implemented by adding a `--base <BASE_NIXPKGS>` flag, which then
compares the base nixpkgs against the main nixpkgs version, making sure
that there are no regressions.
The `--version` flag is removed. While it was implemented, it was never
used in CI, so this is fine.
This prepares the code base for the removal of the `--version` flag, to
be replaced with a flag that can specify a base version to compare the
main Nixpkgs against, in order to have gradual transitions to stricter
checks.
This refactoring does:
- Introduce the `version` module that can house the logic to increase
strictness, with a `version::Nixpkgs` struct that contains the
strictness conformity of a single Nixpkgs version
- Make the check return `version::Nixpkgs`
- Handle the behavior of the still-existing `--version` flag with `version::Nixpkgs`
- Introduce an intermediate `process` function to handle the top-level
logic, especially useful in the next commit
Convenience function to run another validation over a successful validation result.
This will be usable in more locations in future commits, making the code
nicer.
This makes it such that these two errors can both be thrown for a single
package:
- The attribute value not being a derivation
- The attribute not being a proper callPackage
The tests had to be adjusted to only throw the error they were testing
for
- passing it as expression gives large error messages
which are not very readable
- this commits puts the file in nix-store
and patches the final program to have access to
the path to the file as env.
- We simply pass this file to nix-instantiate
Currently the tool prints problems right as it is checking the code
without an intermediate error representation. However for various reasons
it would be beneficial to have an intermediate error type:
- It makes the code cleaner, having all errors in one place
- It allows printing the error in different ways, e.g. for a future
--json mode
This commit prepares for an incremental refactoring for an intermediate
error/problem representation. Most notable is that we want to be able to collect
multiple errors/problems and not just exit on the first one.
We introduce the type alias CheckResult and CheckError (later renamed to
NixpkgsProblem), where CheckError allows collecting multiple
CheckErrors using the utility function flatten_check_results (later
renamed to check_result::sequence)
The write_check_result function is only temporarily introduced to help
refactoring, it's removed again in later commits.
Allows detecting whether attributes are overridden in all-packages.nix.
In a future commit we'll use this to detect empty arguments being set in
all-packages.nix and complain about that.
We seem to have enough tests to run into this now:
error: creating symlink from '/private/tmp/nix-build-nixpkgs-check-by-name.drv-0/source/test-tmp/var/nix/gcroots/profiles' to '/private/tmp/nix-build-nixpkgs-check-by-name.drv-0/source/test-tmp/var/nix/profiles': File exists
On Darwin, /tmp is sometimes a symlink to /private/tmp, which couldn't
be handled before:
error: access to canonical path '/private/var/folders/xp/9_ry6h9x6l9gh_g32qspz0_40000gp/T/.tmpFbcNO0' is forbidden in restricted mode
This both fixes that and adds a test to make sure it can't happen again
This introduces the `pkgs/by-name` directory as proposed by RFC 140.
Included are:
- The implementation to add packages defined in that directory to the
top-level package scope
- Contributer documentation on how to add packages to it
- A GitHub Actions workflow to check the structure of it on all PRs
- Better filesystem case-sensitivity heuristic
We shouldn't assume that Linux is always case-sensitive.
- Don't include case-sensitive filename in tree
Was used for tests, but this broke channel updates because there's a
check to make sure there's no case-sensitive files!
https://hydra.nixos.org/build/233371356/nixlog/1
