From 2093ef28d9410f274af967c44f7386a6749ae89e Mon Sep 17 00:00:00 2001 From: Doron Behar Date: Wed, 29 May 2024 15:26:56 +0300 Subject: [PATCH 1/5] doc/meta: mention how Hydra and other tools don't run passthru.tests --- doc/stdenv/meta.chapter.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/stdenv/meta.chapter.md b/doc/stdenv/meta.chapter.md index 575d3368326d..da0546a4367a 100644 --- a/doc/stdenv/meta.chapter.md +++ b/doc/stdenv/meta.chapter.md @@ -125,6 +125,8 @@ $ cd path/to/nixpkgs $ nix-build -A your-package.tests ``` +Note that Hydra and [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review) don't build these derivations by default, and that ([`@ofborg`](https://github.com/NixOS/ofborg)) only builds them when evaluating PRs for that particular package (or when manually instructed). + #### Package tests {#var-meta-tests-packages} Tests that are part of the source package are often executed in the `installCheckPhase`. From f4e7ce7719ef657ee9de9fb23e750c44e46989df Mon Sep 17 00:00:00 2001 From: Doron Behar Date: Wed, 29 May 2024 15:27:04 +0300 Subject: [PATCH 2/5] doc/meta: small link target shortenning Shorten the path from the link to the to actual content about passthru.tests - the content was moved into pkgs/README.md . --- doc/stdenv/meta.chapter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/stdenv/meta.chapter.md b/doc/stdenv/meta.chapter.md index da0546a4367a..c47c4be87d5d 100644 --- a/doc/stdenv/meta.chapter.md +++ b/doc/stdenv/meta.chapter.md @@ -137,7 +137,7 @@ Prefer `passthru.tests` for tests that are introduced in nixpkgs because: * we can run `passthru.tests` independently * `installCheckPhase` adds overhead to each build -For more on how to write and run package tests, see [](#sec-package-tests). +For more on how to write and run package tests, see [`pkgs/README.md`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests). #### NixOS tests {#var-meta-tests-nixos} From f4e6f41b726bae27f110b78c67480d9ab154dc9a Mon Sep 17 00:00:00 2001 From: Doron Behar Date: Wed, 29 May 2024 15:29:29 +0300 Subject: [PATCH 3/5] doc/meta: better explain an advantage or passthru.tests --- doc/stdenv/meta.chapter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/stdenv/meta.chapter.md b/doc/stdenv/meta.chapter.md index c47c4be87d5d..f9044fdfc9d2 100644 --- a/doc/stdenv/meta.chapter.md +++ b/doc/stdenv/meta.chapter.md @@ -134,7 +134,7 @@ Tests that are part of the source package are often executed in the `installChec Prefer `passthru.tests` for tests that are introduced in nixpkgs because: * `passthru.tests` tests the 'real' package, independently from the environment in which it was built -* we can run `passthru.tests` independently +* We can run and debug a `passthru.tests` independently, after the package was built (useful if it takes a long time). * `installCheckPhase` adds overhead to each build For more on how to write and run package tests, see [`pkgs/README.md`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests). From 444c2b6dd06e02f4bd8f153fb6a574e2350d861f Mon Sep 17 00:00:00 2001 From: Doron Behar Date: Wed, 29 May 2024 15:29:39 +0300 Subject: [PATCH 4/5] doc/meta: Mention --version as a good usecase for installCheckPhase --- doc/stdenv/meta.chapter.md | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/doc/stdenv/meta.chapter.md b/doc/stdenv/meta.chapter.md index f9044fdfc9d2..e2a5a5c31a6e 100644 --- a/doc/stdenv/meta.chapter.md +++ b/doc/stdenv/meta.chapter.md @@ -129,9 +129,29 @@ Note that Hydra and [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review) #### Package tests {#var-meta-tests-packages} -Tests that are part of the source package are often executed in the `installCheckPhase`. +Tests that are part of the source package are often executed in the `installCheckPhase`. This phase is also suitable for performing a `--version` test for packages that support such flag. Here's an example: -Prefer `passthru.tests` for tests that are introduced in nixpkgs because: +```nix +# Say the package is git +stdenv.mkDerivation(finalAttrs: { + pname = "git"; + version = "..."; + # ... + + doInstallCheck = true; + installCheckPhase = '' + runHook preInstallCheck + + echo checking if 'git --version' mentions ${finalAttrs.version} + $out/bin/git --version | grep ${finalAttrs.version} + + runHook postInstallCheck + ''; + # ... +}) +``` + +Most programs distributed by Nixpkgs support such a `--version` flag, and it can help give confidence that the package at least got compiled properly. However, tests that are slightly non trivial will better fit into `passthru.tests`, because: * `passthru.tests` tests the 'real' package, independently from the environment in which it was built * We can run and debug a `passthru.tests` independently, after the package was built (useful if it takes a long time). From 0aa9043370cc9ea101b5c5b9427c74298252ed5c Mon Sep 17 00:00:00 2001 From: Doron Behar Date: Thu, 30 May 2024 16:33:27 +0300 Subject: [PATCH 5/5] doc/meta: still mention testVersion near the installCheckPhase recommendation --- doc/stdenv/meta.chapter.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/stdenv/meta.chapter.md b/doc/stdenv/meta.chapter.md index e2a5a5c31a6e..6d9b61b0647c 100644 --- a/doc/stdenv/meta.chapter.md +++ b/doc/stdenv/meta.chapter.md @@ -157,6 +157,8 @@ Most programs distributed by Nixpkgs support such a `--version` flag, and it can * We can run and debug a `passthru.tests` independently, after the package was built (useful if it takes a long time). * `installCheckPhase` adds overhead to each build +It is also possible to still use `passthru.tests` to test the version, with [testVersion](#tester-testVersion). + For more on how to write and run package tests, see [`pkgs/README.md`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests). #### NixOS tests {#var-meta-tests-nixos}