Commit Graph

556 Commits

Author SHA1 Message Date
Silvan Mosberger
be26d74840 tests.nixpkgs-check-by-name: Add test for alternate callPackage's
Tests that alternate callPackage's don't give an error, so pkgs/by-name
is only required for pkgs.callPackage itself
2024-01-10 19:55:24 +01:00
Silvan Mosberger
d659d5f8ac tests.nixpkgs-check-by-name: Use real lib for tests
This way we can also detect regressions caused by lib
2024-01-10 19:43:17 +01:00
Silvan Mosberger
cbff02834f tests.nixpkgs-check-by-name: Sort the eval validation results
Not that important, but nice.
Also adds a nice test case show-casing the two current ratchet checks at
once.
2024-01-10 19:29:27 +01:00
Silvan Mosberger
57afdc545b tests.nixpkgs-check-by-name: Enforce pkgs/by-name for new packages 2024-01-10 19:29:26 +01:00
Silvan Mosberger
69fc71a82c tests.nixpkgs-check-by-name: Basic info for non-by-name attributes
In a future commit this will be extended
2024-01-10 19:28:19 +01:00
Silvan Mosberger
da3e72b915
Merge pull request #278805 from tweag/by-name-enforce-preparation
check-by-name: Refactor to prepare for enforcing `pkgs/by-name`, make `--base` required
2024-01-09 22:58:53 +01:00
Robert Hensing
098ffee102
Merge pull request #279487 from bryango/checkpoint-build-polish
checkpointBuildTools: mkCheckpointedBuild -> mkCheckpointBuild
2024-01-09 22:30:22 +01:00
Silvan Mosberger
a1db0cdf9b tests.nixpkgs-check-by-name: .context -> .with_context
Avoids allocation in the non-error case
2024-01-09 19:39:50 +01:00
Silvan Mosberger
4cd2e64db3 tests.nixpkgs-check-by-name: Minor improvements from feedback
- Typo
- Rename AttributeRatchet to ToNixpkgsProblem
- Make the compare trait method into a RatchetState method

Co-Authored-By: Philip Taron <philip.taron@gmail.com>
2024-01-09 19:35:11 +01:00
Silvan Mosberger
d79d3de85f cudaPackages.cuda-library-samples: Only enable on Linux, fix channel
This very weirdly broke the channel evaluation: https://hydra.nixos.org/build/245871962/nixlog/1

It appears that this attribute is only evaluated by Hydra, _not_ by
ofborg. So this wouldn't have been detected by CI anyways in the PR that
introduced the problem: https://github.com/NixOS/nixpkgs/pull/276800.

However, due to https://github.com/NixOS/nixpkgs/pull/271123#discussion_r1442134594,
the channel only broke once that was fixed with https://github.com/NixOS/nixpkgs/pull/278777

Whether the fix is good, I don't know, but the failing-on-darwin attribute
doesn't exist anymore with this commit, making the tarball build succeed
again:

    nix-build pkgs/top-level/release.nix -A tarball
2024-01-08 22:03:40 +01:00
Bryan Lai
df62c3c87f checkpointBuildTools: mkCheckpointedBuild -> mkCheckpointBuild
All other functions are in the form of `*{c,C}heckpointBuild*`, so we
deprecate the `mkCheckpointedBuild` function in favor of `mkCheckpointBuild`.

Also address some inconsistencies in the docs: some `buildOutput` should
actually be `incrementalBuildArtifacts`.
2024-01-08 19:42:16 +08:00
Silvan Mosberger
6946071175 tests.checkpoint-build: Fix evaluation with allowAliases false
Was broken since introduced, but not caught due to
https://github.com/NixOS/nixpkgs/pull/271123#discussion_r1442134594
2024-01-07 23:25:41 +01:00
Silvan Mosberger
54b05324f4 tests.nixpkgs-check-by-name: Internal strip nixpkgs prefix
Strips the Nixpkgs prefix from the callPackage paths,
makes future error messages using this path be deterministic.
2024-01-05 02:01:55 +01:00
Silvan Mosberger
27c873af99 tests.nixpkgs-check-by-name: Deterministic ordering
Makes errors for attributes deterministic so it's easier to test (also,
reproducibility is always nice)
2024-01-05 02:01:55 +01:00
Silvan Mosberger
2a8f469348 tests.nixpkgs-check-by-name: Re-usable ratchet logic
This makes the attribute ratchet check logic more re-usable, which will
be used in a future commit.

It also renames the ratchet states to something more intuitive
2024-01-05 02:01:50 +01:00
Silvan Mosberger
ba6faf428f tests.nixpkgs-check-by-name: Make --base required
CI now passes the flag, so it doesn't have to be optional anymore
2024-01-05 01:05:50 +01:00
Silvan Mosberger
66f29590c0 tests.nixpkgs-check-by-name: Set evaluation system to x86_64-linux
This was previously a checking impurity that could produce different
results when run on different systems.
2024-01-05 01:05:44 +01:00
Silvan Mosberger
5b7ae79ef0 tests.nixpkgs-check-by-name: Refactor eval code and improve comments
Does a bunch of cleanups to the eval.{rs,nix} code to make future
changes easier, no functionality is changed.
2024-01-03 19:56:54 +01:00
maralorn
5518af0a55 haskellPackages: Fix eval errors 2024-01-01 19:16:21 +01:00
maralorn
7b4156edda haskellPackages: Fix eval errors 2024-01-01 18:28:31 +01:00
Silvan Mosberger
02b3c06b1c
Merge pull request #274591 from tweag/by-name-reproducible
`pkgs/by-name`: Enable gradual migration checks and add `run-local.sh`
2023-12-22 00:17:37 +01:00
Silvan Mosberger
e130ee33a1 pkgs/test/nixpkgs-check-by-name/scripts: Various improvements
- trace function, avoids littering `echo >&2` all throughout
- Avoid `eval`, remove unneeded shellcheck

Co-Authored-By: Victor Engmark <victor@engmark.name>
2023-12-22 00:04:16 +01:00
Silvan Mosberger
1968beeabd check-by-name: Pass --base in CI and local running script
This enables the ratchet checks for pkgs/by-name, allowing gradual
migrations!
2023-12-20 17:38:06 +01:00
Silvan Mosberger
9e03178865 tests.nixpkgs-check-by-name: Add documentation for scripts 2023-12-20 17:38:05 +01:00
Silvan Mosberger
92238ac52d tests.nixpkgs-check-by-name: Create script to run locally
Due to the check soon depending on the base branch (see `--base`),
the CI check can't reasonably share all code with a local check.
We can still make a script to run it locally, just not sharing all code.
2023-12-20 17:38:03 +01:00
Silvan Mosberger
3869ba4127 workflows/check-by-name: Move tool fetching into script
This part of the CI can also be reproduced locally
2023-12-19 23:15:01 +01:00
Janne Heß
5eed5416ff
Merge pull request #167670 from messemar/incremental-builds
incremental builds: add derivation override functions
2023-12-19 11:37:15 +01:00
Silvan Mosberger
2a107bc64b
Merge pull request #272395 from tweag/by-name-migrate-empty-arg
tests.nixpkgs-check-by-name: Implement gradual empty arg check migration
2023-12-15 19:08:11 +01:00
Silvan Mosberger
fc2d26939d tests.nixpkgs-check-by-name: Improve check clarity 2023-12-15 17:27:26 +01:00
Adam Joseph
2d036511f6 pkgs/test/release/default.nix: init
This derivation verifies that all jobset attributes can be evaluated
under tryEval without producing any non-catchable errors or causing
infinite recursion.
2023-12-15 05:13:50 -08:00
Adam Joseph
80472e3754 treewide: add __attrsFailEvaluation and __recurseIntoDerivationForReleaseJobs 2023-12-15 05:13:46 -08:00
Silvan Mosberger
74e8b38dbe tests.nixpkgs-check-by-name: Move interface description into code
This would be duplicated otherwise
2023-12-15 02:28:55 +01:00
Silvan Mosberger
79618ff8cb tests.nixpkgs-check-by-name: Improve docs, introduce "ratchet" term 2023-12-15 02:14:48 +01:00
Silvan Mosberger
413dd9c03e tests.nixpkgs-check-by-name: Minor improvements from review 2023-12-15 01:02:49 +01:00
Silvan Mosberger
53b43ce0e3 tests.nixpkgs-check-by-name: Fix and document behavior without --base
Previously, not passing `--base` would enforce the most strict checks.
While there's currently no actual violation of these stricter checks,
this does not match the previous behavior.

This won't matter once CI passes `--base`, the code handling the
optionality can be removed then.
2023-12-15 01:01:14 +01:00
Silvan Mosberger
bb08bfc2d3 tests.nixpkgs-check-by-name: Test for gradual transition
This implements the ability to test gradual transitions in check
strictness, and adds one such test for the empty non-auto-called
arguments check.
2023-12-14 04:05:03 +01:00
Silvan Mosberger
d487a975cc tests.nixpkgs-check-by-name: Gradual migration from base Nixpkgs
This implements the option for a gradual migration to stricter checks.
For now this is only done for the check against empty non-auto-called
callPackage arguments, but in the future this can be used to ensure all
new packages make use of `pkgs/by-name`.

This is implemented by adding a `--base <BASE_NIXPKGS>` flag, which then
compares the base nixpkgs against the main nixpkgs version, making sure
that there are no regressions.

The `--version` flag is removed. While it was implemented, it was never
used in CI, so this is fine.
2023-12-14 03:51:10 +01:00
Silvan Mosberger
a6ba4cae31 tests.nixpkgs-check-by-name: Intermediate refactor
This prepares the code base for the removal of the `--version` flag, to
be replaced with a flag that can specify a base version to compare the
main Nixpkgs against, in order to have gradual transitions to stricter
checks.

This refactoring does:
- Introduce the `version` module that can house the logic to increase
  strictness, with a `version::Nixpkgs` struct that contains the
  strictness conformity of a single Nixpkgs version
- Make the check return `version::Nixpkgs`
- Handle the behavior of the still-existing `--version` flag with `version::Nixpkgs`
- Introduce an intermediate `process` function to handle the top-level
  logic, especially useful in the next commit
2023-12-14 03:40:49 +01:00
Silvan Mosberger
e98d22851b tests.nixpkgs-check-by-name: Introduce result_map
Convenience function to run another validation over a successful validation result.

This will be usable in more locations in future commits, making the code
nicer.
2023-12-14 03:21:41 +01:00
Silvan Mosberger
b8e4d555b4 tests.nixpkgs-check-by-name: Minor refactor, allow more simultaneous problems
This makes it such that these two errors can both be thrown for a single
package:
- The attribute value not being a derivation
- The attribute not being a proper callPackage

The tests had to be adjusted to only throw the error they were testing
for
2023-12-14 03:13:36 +01:00
Connor Baker
8e800cedaf cudaPackages: move derivations to cuda-modules & support aarch64
cudaPackages.cuda_compat: ignore missing libs provided at runtime

cudaPackages.gpus: Jetson should never build by default

cudaPackages.flags: don't build Jetson capabilities by default

cudaPackages: re-introduce filter for pre-existing CUDA redist packages in overrides

cudaPackages: only recurseIntoAttrs for the latest of each major version

cudaPackages.nvccCompatabilities: use GCC 10 through CUDA 11.5 to avoid a GLIBC incompatability

cudaPackages.cutensor: acquire libcublas through cudatoolkit prior to 11.4

cudaPackages.cuda_compat: mark as broken on aarch64-linux if not targeting Jetson

cudaPackages.cutensor_1_4: fix build

cudaPackages: adjust use of autoPatchelfIgnoreMissingDeps

cudaPackages.cuda_nvprof: remove unecessary override to add addOpenGLRunpath

cudaPackages: use getExe' to avoid patchelf warning about missing meta.mainProgram

cudaPackages: fix evaluation with Nix 2.3

cudaPackages: fix platform detection for Jetson/non-Jetson aarch64-linux

python3Packages.tensorrt: mark as broken if required packages are missing

Note: evaluating the name of the derivation will fail if tensorrt is not present,
which is why we wrap the value in `lib.optionalString`.

cudaPackages.flags.getNixSystem: add guard based on jetsonTargets

cudaPackages.cudnn: use explicit path to patchelf

cudaPackages.tensorrt: use explicit path to patchelf
2023-12-07 16:45:54 +00:00
Martin Messer
6db9612204 rename: incremental -> checkpointed builds 2023-12-07 16:56:16 +01:00
Martin Messer
fc2e3fac61 doc: add section about checkpointed build 2023-12-07 16:55:36 +01:00
Martin Messer
c8afee88bf checkpointedBuild: fix tests for checkpointedBuild functions 2023-12-07 16:38:48 +01:00
Martin Messer
17e88c2890 checkpointedBuild: add checkpointed build test based on pkgs hello 2023-12-07 16:38:48 +01:00
sternenseemann
ae29d067ff llvmPackages_7: remove at 7.1.0 2023-12-04 19:42:57 +01:00
Adam Joseph
bffbf8bb53 test.cuda: fix broken eval 2023-11-30 09:36:05 +01:00
Jan Tojnar
bc41b2db3d makeHardcodeGsettingsPatch: Support applying patches
This is useful for replacing code that cannot be easily handled by the generator,
such as the tentative settings constructor in e-d-s.
2023-11-21 08:42:47 +01:00
Silvan Mosberger
6361b22370
Merge pull request #267048 from narang99/hariom/257748/check-by-name-msg
fix long error messages in nix-instantiate
2023-11-21 01:35:11 +01:00
Hariom Narang
ca97497bfd made EXPR_PATH point to local instead of store
- we need this behavior in dev shell
- the main build still uses the store file
2023-11-21 01:11:06 +05:30