Commit Graph

1141 Commits

Author SHA1 Message Date
github-actions[bot]
4908c48bf5
Merge master into staging-next 2024-04-15 12:01:00 +00:00
Jade Lovelace
78945a827c stdenv: make inputDerivation never fixed-output
This fixes using inputDerivation on derivations that are fixed-output.

Previously:

```
nix-repl> drv = runCommand "huh" { outputHash = "sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU="; outputHashAlgo = "sha256"; outputHashType = "flat"; } "touch $out"

nix-repl> drv.inputDerivation
«derivation /nix/store/d8mjs6cmmvsr1fv7psm6imis5pmh9bcs-huh.drv»

nix-repl> :b drv.inputDerivation
error: fixed output derivation 'huh' is not allowed to refer to other store paths.
       You may need to use the 'unsafeDiscardReferences' derivation attribute, see the manual for more details.
```

Fixes: https://github.com/NixOS/nixpkgs/issues/304209
2024-04-15 00:52:28 -07:00
github-actions[bot]
b50b9be6e4
Merge master into staging-next 2024-04-13 06:00:55 +00:00
Dan Callahan
ec6c594944 stdenv/check-meta: Fix error message for disallowed unfree packages
Nixpkgs tries to print a helpful message when it blocks unfree packages,
but the suggestion is subtly broken. The predicate only matches on the
package's name, but the suggestion includes the full name-version pair.

Fixed by formatting the message with the same function as the predicate.

This issue arises because check-meta defines its own local getName with
semantics divergent from lib.getName. The former includes the version,
the latter does not.

Example Before:

    Alternatively you can configure a predicate to allow specific packages:
      { nixpkgs.config.allowUnfreePredicate = pkg: builtins.elem (lib.getName pkg) [
          "obsidian-1.5.12"
        ];
      }

Example After:

    Alternatively you can configure a predicate to allow specific packages:
      { nixpkgs.config.allowUnfreePredicate = pkg: builtins.elem (lib.getName pkg) [
          "obsidian"
        ];
      }

Fixes #303116
2024-04-10 16:15:24 +01:00
Vladimír Čunát
3299c31f44
stdenv: avoid setuid issues
See #300635.  Maybe in time we'll have a better solution.
2024-04-10 07:52:44 +02:00
Weijia Wang
3f59355d84 Merge branch 'staging-next' into staging 2024-04-09 08:26:23 +02:00
Ryan Lahfa
36cd37b5dc
Merge pull request #297074 from lolbinarycat/tarfix
stdenv: pass --mode=+w to tar, ensuring extraction is possible
2024-04-04 13:51:39 -07:00
adisbladis
000f61a610 stdenv/check-meta: Make checkValidity only check validity
checkValidity has the responsibility to check if a derivation's attributes are valid.
Previously it also had the overloaded task of creating a subset of meta attributes:
- unfree
- broken
- unsupported
- insecure

Not only is this overloading strange, these attributes were only ever consumed by `commonMeta`.

This change makes checkValidity _only_ check for validity, and removes the creation of any meta attributes from `checkValidity` and moves them to `commonMeta`.

This is technically a breaking change but I don't expect any external nixpkgs consumers to rely on these implementation details.
2024-04-03 21:38:43 +13:00
K900
59bccda153 Revert "stdenv: add meta.repository field"
This reverts commit 7e1443abbb.
2024-03-30 17:19:21 +03:00
K900
0e8bcaa661 Revert "stdenv/check-meta: Don't create new environments when computing meta.repository"
This reverts commit f8b091d53f.
2024-03-30 17:18:47 +03:00
K900
ba79149c66 stdenv/check-meta: don't infrec on unsupported platforms 2024-03-30 14:30:20 +03:00
adisbladis
bff4c55f3a
Merge pull request #300177 from adisbladis/meta-repository-perf-fixups
stdenv/check-meta: Fix performance regressions introduced in #294347
2024-03-30 20:22:33 +13:00
adisbladis
dc49349c84 stdenv/check-meta: Inherit remaining lib access into scope
Hopefully this will result in people not adding new `lib.xxx` to check-meta.nix.
2024-03-30 19:20:59 +13:00
adisbladis
f8b091d53f stdenv/check-meta: Don't create new environments when computing meta.repository 2024-03-30 19:19:05 +13:00
adisbladis
c605bfd501
Merge pull request #297350 from adisbladis/commonmeta-output-list
stdenv/check-meta: Remove outputsToInstall list concat from common meta
2024-03-30 19:07:32 +13:00
a-n-n-a-l-e-e
2809c84cfb
Merge pull request #294347 from lolbinarycat/meta.repository
stdenv: add meta.repository field
2024-03-29 13:03:04 -07:00
binarycat
7e1443abbb stdenv: add meta.repository field 2024-03-29 14:34:09 -04:00
adisbladis
1e2f8f2a84 stdenv/check-meta: Remove outputsToInstall list concat from common meta
Normally either of "bin" or "out" will hit first so we can avoid dynamic looping altogether.
2024-03-20 16:29:17 +13:00
binarycat
f9094faf70 stdenv: pass --mode=+w to tar, ensuring extraction is possible
fixes #191137
2024-03-18 20:54:31 -04:00
Robert Hensing
05f4b7b46d make-derivation.nix: Fix checkMetaRecursively
Oddly, I can't reproduce the error, but this change will make it
more robust.
See https://github.com/NixOS/nixpkgs/pull/295378#issuecomment-2002094487
2024-03-17 19:28:47 +01:00
Robert Hensing
ba463e70e3 make-derivation.nix: Float out unsafeDerivationToUntrackedOutpath 2024-03-12 20:15:47 +01:00
Robert Hensing
0ab7b23637 make-derivation.nix: Update inline docs 2024-03-12 19:06:07 +01:00
Robert Hensing
797ad5ae87 make-derivation.nix: Float out new constant removedOrReplacedAttrNames 2024-03-12 18:43:11 +01:00
Robert Hensing
cac23248b1 make-derivation.nix: Float out knownHardeningFlags 2024-03-12 18:41:57 +01:00
Robert Hensing
464d8c4a6c make-derivation.nix: Float inward darwin-specific derivation attributes 2024-03-12 18:41:18 +01:00
Robert Hensing
2fb7255cc0 make-derivation.nix: Evaluate flip 2024-03-12 18:39:19 +01:00
Robert Hensing
90197b6833 make-derivation.nix: Apply map composition law
... after inlining chooseDevOutputs.
2024-03-12 18:37:53 +01:00
Robert Hensing
954d9ce64b make-derivation.nix: Drop duplicate functionality from previous commit
These two commits make for a cleaner commit history and git blame than
https://github.com/NixOS/nixpkgs/pull/295105, where this refactor
was developed.

See its commit messages for details and design choices, esp. up to
and including 37f76fd4c3b23b324ab7ea06ef3b747338d2080f.
2024-03-12 18:32:03 +01:00
Robert Hensing
8dad51a2e2 make-derivation.nix: Split makeDerivationArgument, mkDerivation with duplicate functionality 2024-03-12 18:16:56 +01:00
Robert Hensing
de516f6f13 make-derivation.nix: Return mkDerivation as an attribute 2024-03-11 17:06:02 +01:00
Robert Hensing
ab56d3ac71 make-derivation.nix: Move into let binding 2024-03-11 17:02:59 +01:00
Robert Scott
4a91b3e798 cc-wrapper: add trivialautovarinit hardening flag support
this equates to -ftrivial-auto-var-init=pattern

clang has removed support for -ftrivial-auto-var-init=zero and
are unlikely to re-add it, so use -ftrivial-auto-var-init=pattern
on both compilers if only to make behaviour more consistent
between the two.

add to pkgsExtraHardening's defaultHardeningFlags.
2024-02-24 12:00:09 +00:00
Vladimír Čunát
eff11adc57
Merge #286721: stdenv: fix rare tar.xz decompression issues
...into staging
2024-02-17 08:45:01 +01:00
Isidor Zeuner
11a19109b6 stdenv: disregard xz exit status in order to fix subtle decompression issues
There is a subtle bug with unpacking `tar.xz` archives which seems to happen only on some setups, and sometimes not in a reproducible manner (https://github.com/NixOS/nixpkgs/issues/278130, https://github.com/NixOS/nixpkgs/issues/20950). On the last occurrence, it could be tracked down to `xz` failing from a `SIGPIPE`, which can happen when it's connected to `tar` through a pipe and `tar` exits earlier (see e.g. https://www.linuxquestions.org/questions/slackware-14/%5Bpatch%5D-tar-issuing-a-sigpipe-in-installpkg-4175637923/ or https://bugs.gentoo.org/573642#c5).

Since `tar` should be able by itself to detect whether the archive is complete, I suggest to disregard the exit code from the `xz` invocation, done in this PR.

Fixes  https://github.com/NixOS/nixpkgs/issues/278130 (script tested here: https://github.com/NixOS/nixpkgs/pull/286579)
Probably also fixes https://github.com/NixOS/nixpkgs/issues/20950 (issue not reproduced here, feedback therefore welcome)
2024-02-13 17:00:48 +01:00
Kait Lam
355ab764b4 stdenv: refactor of --replace-{quiet,warn,fail} logic
This is a small simplification of the control flow surrounding these cases. It should make it more obvious when each case happens, and also explicitly defines the current behaviour of --replace.
2024-02-11 10:48:37 +10:00
Rick van Schijndel
2c98eaf597
Merge pull request #286526 from pbsds/fix-replace-quiet-1707148527
stdenv: fix `substituteStream --replace-quiet` deprecation warning
2024-02-07 21:35:19 +01:00
Peder Bergebakken Sundt
cb4c41f93a stdenv: fix substituteStream --replace-quiet deprecation warning 2024-02-05 17:28:02 +01:00
Robert Scott
40868719b0 cc-wrapper: add zerocallusedregs hardening flag
this uses the value `used-gpr` which seems to be a commonly
chosen value for general use
2024-01-20 13:48:33 +00:00
Peder Bergebakken Sundt
4c1b74bac7 stdenv: substituteStream: escape echoed pattern in --replace mismatch warning 2024-01-14 22:08:03 +01:00
Peder Bergebakken Sundt
e07a2fab7f stdenv: substituteStream: deprecate --replace in favor of --replace-{fail,warn,quiet} 2024-01-14 22:07:58 +01:00
sternenseemann
f734056102 stdenv: fix evaluation if !stdenv.cc.hasCC
stdenv.cc may throw, e.g. in the case of pkgsCross.ghcjs where we must
not force it for the purpose of attribute accessing (`or` doesn't
implicitly tryEval…).

Regression introduced in 1a5bd697ad.
2024-01-13 12:45:36 +01:00
K900
bff9ee8862 Merge remote-tracking branch 'origin/master' into staging-next 2024-01-09 13:03:34 +03:00
Ryan Burns
c324705cc3 treewide: simplify exec format conditionals 2024-01-07 17:43:33 -08:00
Weijia Wang
eb5334c27d Merge branch 'master' into staging-next 2024-01-03 03:22:40 +01:00
adisbladis
4a5f2bd6e8 stdenv/check-meta: Use bespoke type checking
Aka `checkMeta` goes brrr.

Using the module system type checking works OK & generates good error messages.
The performance of using it however is terrible because of the value merging it does being very allocation heavy.

By implementing a very minimal type checker we can drastically improve the performance when nixpkgs is evaluated with `checkMeta = true`.
2023-12-26 15:14:42 +13:00
Martin Weinelt
e8dd1cf066 Merge remote-tracking branch 'origin/staging-next' into staging 2023-12-12 14:13:07 +01:00
adisbladis
1e66093cca
Merge pull request #269546 from adisbladis/stdenv-meta-no-intermediate-alloc
stdenv: Avoid allocating intermediate attrset when checking meta validity
2023-12-12 10:57:44 +13:00
adisbladis
6aa414bcfd
Merge pull request #269782 from adisbladis/stdenv-meta-tolist-license-allocations
stdenv: Avoid some list allocations in check-meta when checking licenses
2023-12-12 10:55:58 +13:00
adisbladis
9a0a097a94 stdenv: Avoid allocating intermediate attrset when checking meta validity
This is a small performance optimization. It should be impercetible to most.

Benchmarks:

- Before
``` json
{
  "cpuTime": 0.2777960002422333,
  "envs": {
    "bytes": 3832648,
    "elements": 189513,
    "number": 144784
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 50229344
  },
  "list": {
    "bytes": 655304,
    "concats": 3249,
    "elements": 81913
  },
  "nrAvoided": 218962,
  "nrFunctionCalls": 127718,
  "nrLookups": 40946,
  "nrOpUpdateValuesCopied": 1563978,
  "nrOpUpdates": 8542,
  "nrPrimOpCalls": 113032,
  "nrThunks": 329605,
  "sets": {
    "bytes": 29774864,
    "elements": 1824537,
    "number": 36392
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 235909,
    "number": 24432
  },
  "values": {
    "bytes": 9691392,
    "number": 403808
  }
}
```

- After
```
{
  "cpuTime": 0.2615779936313629,
  "envs": {
    "bytes": 3833832,
    "elements": 189661,
    "number": 144784
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 50212960
  },
  "list": {
    "bytes": 655304,
    "concats": 3249,
    "elements": 81913
  },
  "nrAvoided": 218814,
  "nrFunctionCalls": 127718,
  "nrLookups": 40798,
  "nrOpUpdateValuesCopied": 1563978,
  "nrOpUpdates": 8542,
  "nrPrimOpCalls": 113032,
  "nrThunks": 329457,
  "sets": {
    "bytes": 29765392,
    "elements": 1824093,
    "number": 36244
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 235909,
    "number": 24432
  },
  "values": {
    "bytes": 9687840,
    "number": 403660
  }
}
```
2023-12-12 00:08:34 +13:00
adisbladis
3b13bd5c84 stdenv: Avoid some list allocations in check-meta when checking licenses 2023-12-12 00:03:20 +13:00