Commit Graph

3720 Commits

Author SHA1 Message Date
Silvan Mosberger
fc2b2af15e
Merge pull request #270299 from adisbladis/lib-customisation-allocs
lib.customisation: Refactor nested set access & avoid some allocations
2023-11-27 16:01:05 +01:00
Robert Hensing
51357572f2
Merge pull request #269552 from adisbladis/lib-matchattrs-list-allocs
lib.attrsets.matchAttrs: Avoid some list allocations when walking structure
2023-11-27 14:44:37 +01:00
adisbladis
8d162ec7b8 lib.customisation: Don't allocate intermediate list for missing args 2023-11-27 21:06:42 +13:00
adisbladis
7903613b0b lib.customisation: Inherit lib/builtins into scope
It makes the code more readable if we have less nested attrsets being accessed.
2023-11-27 21:06:41 +13:00
adisbladis
544a1d375b lib.attrsets.attrByPath: Don't allocate one extra list per lookup recursion
Using `tail` in a recursive loop like this needlessly allocates.
This changes the loop to look up by list index instead.
2023-11-27 11:37:57 +13:00
adisbladis
7e07b3ecd5 lib.attrsets.hasAttrByPath: Don't allocate one extra list per lookup recursion
Using `tail` in a recursive loop like this needlessly allocates.
This changes the loop to look up by list index instead.
2023-11-27 11:37:56 +13:00
Silvan Mosberger
627af21e87
Merge pull request #269637 from adisbladis/lib-meta-avoid-alloc-platformmatch
lib.meta: Avoid attrset allocation in platformMatch
2023-11-26 23:28:39 +01:00
adisbladis
013a0a1357 lib.attrsets.matchAttrs: Avoid some list allocations when walking structure
Benchmarks (`nix-instantiate ./. -A python3`):

- Before:
``` json
{
  "cpuTime": 0.29049500823020935,
  "envs": {
    "bytes": 4484216,
    "elements": 221443,
    "number": 169542
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 53086800
  },
  "list": {
    "bytes": 749424,
    "concats": 4242,
    "elements": 93678
  },
  "nrAvoided": 253991,
  "nrFunctionCalls": 149848,
  "nrLookups": 49612,
  "nrOpUpdateValuesCopied": 1587837,
  "nrOpUpdates": 10104,
  "nrPrimOpCalls": 130356,
  "nrThunks": 358981,
  "sets": {
    "bytes": 30423600,
    "elements": 1859999,
    "number": 41476
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 236145,
    "number": 24453
  },
  "values": {
    "bytes": 10502520,
    "number": 437605
  }
}
```

- After:
``` json
{
  "cpuTime": 0.2946169972419739,
  "envs": {
    "bytes": 3315224,
    "elements": 172735,
    "number": 120834
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 48718432
  },
  "list": {
    "bytes": 347568,
    "concats": 4242,
    "elements": 43446
  },
  "nrAvoided": 173252,
  "nrFunctionCalls": 101140,
  "nrLookups": 73595,
  "nrOpUpdateValuesCopied": 1587837,
  "nrOpUpdates": 10104,
  "nrPrimOpCalls": 83067,
  "nrThunks": 304216,
  "sets": {
    "bytes": 29704096,
    "elements": 1831673,
    "number": 24833
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 236145,
    "number": 24453
  },
  "values": {
    "bytes": 8961552,
    "number": 373398
  }
}
```
2023-11-27 11:20:50 +13:00
Emily Trau
1b38685b49 acc: init at 1.60 2023-11-26 16:56:53 +02:00
adisbladis
4b4d413817 lib.meta: Avoid attrset allocation in platformMatch
Benchmarks (`nix-instantiate ./. -A python3`)

- Before
``` json
{
  "cpuTime": 0.30625399947166443,
  "envs": {
    "bytes": 4484216,
    "elements": 221443,
    "number": 169542
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 53091024
  },
  "list": {
    "bytes": 749424,
    "concats": 4242,
    "elements": 93678
  },
  "nrAvoided": 253991,
  "nrFunctionCalls": 149848,
  "nrLookups": 49614,
  "nrOpUpdateValuesCopied": 1588326,
  "nrOpUpdates": 10106,
  "nrPrimOpCalls": 130356,
  "nrThunks": 359013,
  "sets": {
    "bytes": 30432320,
    "elements": 1860540,
    "number": 41480
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 236218,
    "number": 24459
  },
  "values": {
    "bytes": 10504632,
    "number": 437693
  }
}
```

- After
```
{
  "cpuTime": 0.29695799946784973,
  "envs": {
    "bytes": 3296712,
    "elements": 169055,
    "number": 121517
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 49044992
  },
  "list": {
    "bytes": 504928,
    "concats": 4242,
    "elements": 63116
  },
  "nrAvoided": 175403,
  "nrFunctionCalls": 110554,
  "nrLookups": 44907,
  "nrOpUpdateValuesCopied": 1588326,
  "nrOpUpdates": 10106,
  "nrPrimOpCalls": 82330,
  "nrThunks": 306625,
  "sets": {
    "bytes": 29943328,
    "elements": 1843076,
    "number": 28382
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 236218,
    "number": 24459
  },
  "values": {
    "bytes": 9037752,
    "number": 376573
  }
}
```
2023-11-25 11:05:23 +13:00
Alyssa Ross
62f7a6dcc1 lib.systems.elaborate: fix passing rust
Usually, attributes passed explicitly to elaborate take precedence
over the elaborated ones, but since we also elaborate the nested
"rust" attrset, we need to push that one level down, so the rest of
"rust" is still filled in if you just pass
{ rust = { config = ... } }.

I've had to drop the assertion that checked that at most one of "rust"
and "rustc" was part of the un-elaborated system, because doing this
broke passing an elaborated system in, which should be idempotent.

For the same reason, I've also had to make it possible for
rust.rustcTargetSpec to be passed in.  Otherwise, on the second call,
since platform was filled in by the first, the custom target file
would be constructed.  The only other way to avoid this would be to
compare the platform attrs to all built in Rust targets to check it
wasn't one of those, and that isn't feasible.

Fixes: e3e57b8f18 ("lib.systems: elaborate Rust metadata")
2023-11-24 12:21:30 +01:00
Silvan Mosberger
7486aee82e
Merge pull request #266362 from tweag/fileset.fileFilter-ext
`lib.fileset.fileFilter`: Predicate attribute for file extension
2023-11-24 00:15:43 +01:00
Silvan Mosberger
6816f28c96 lib.fileset.fileFilter: Predicate attribute for file extension 2023-11-23 21:05:23 +01:00
Silvan Mosberger
f37dd68fe0
Merge pull request #265964 from tweag/fileset.optional
`lib.fileset.maybeMissing`: init
2023-11-22 19:46:26 +01:00
Silvan Mosberger
827232d6dd lib.fileset: Document decision for strict existence checks
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-11-22 18:51:00 +01:00
figsoda
87cc06983c
24.05 is Uakari 2023-11-21 14:34:30 -05:00
Silvan Mosberger
1cc2c2f13d lib.fileset.maybeMissing: init 2023-11-20 19:56:45 +01:00
Silvan Mosberger
ed5e926ec4
Merge pull request #268619 from tweag/lib-descriptions
lib: Take advantage of section descriptions
2023-11-20 17:25:59 +01:00
Silvan Mosberger
055ba65fed lib: Take advantage of section descriptions
See https://github.com/nix-community/nixdoc/releases/tag/v2.6.0
2023-11-20 03:02:11 +01:00
Silvan Mosberger
4164b1c47e lib.fileset: Re-order to match reference overview 2023-11-19 23:52:13 +01:00
Silvan Mosberger
13f0af428e lib.fileset: Add overview section to reference docs 2023-11-19 23:46:23 +01:00
Silvan Mosberger
5a7118da75
Merge pull request #268520 from tweag/fileset-minor-changes
`lib.fileset`: Minor changes
2023-11-19 19:12:29 +01:00
Silvan Mosberger
b514f0433c lib.fileset: Minor changes
- Make fromSource's missing file error message more consistent with others,
  and add a test for it
- Indent some function arguments
- Fix an internal type
2023-11-19 16:06:14 +01:00
Silvan Mosberger
228df50ff9 lib.fileset: Move introduction section above the functions
Previously the introductory section and the function listings were in
different places. But now nixdoc supports having them together
with https://github.com/nix-community/nixdoc/pull/70!
2023-11-19 15:00:57 +01:00
Silvan Mosberger
ff562fa5fc
Merge pull request #266443 from amjoseph-nixpkgs/pr/lib-tests-no-aws-sdk 2023-11-19 06:17:18 +01:00
Adam Joseph
54c9a08aaf lib.tests: build nix without flaky aws-sdk-cpp
The aws-sdk-cpp tests are flaky.

Since pull requests to staging cause nix to be rebuilt, this means
that staging PRs end up getting false CI failures due to whatever is
flaky in the AWS SDK tests.  Since none of our CI needs to (or
should be able to) contact AWS S3, let's just omit it all.  Bonus:
the tests build way faster.
2023-11-18 20:19:10 -08:00
github-actions[bot]
dba5c9ef4e
Merge master into staging-next 2023-11-16 18:01:17 +00:00
Silvan Mosberger
ada680bcfa lib.fileset.gitTracked: Better error in pure eval 2023-11-16 14:42:26 +01:00
Vladimír Čunát
0fe5300699
Merge branch 'master' into staging-next 2023-11-16 09:49:24 +01:00
Silvan Mosberger
d33f1a62f5 lib.fileset.gitTrackedWith: Introduce recurseSubmodules parameter 2023-11-16 01:12:18 +01:00
Silvan Mosberger
2dfb1d36cf lib.fileset.gitTracked/gitTrackedWith: init
A configuration parameter for gitTrackedWith will be introduced in the
next commit
2023-11-16 01:12:15 +01:00
Robert Hensing
060c4ad377
Merge pull request #267384 from tweag/fileset.fileFilter-path2
`fileset.fileFilter`: Restrict second argument to paths
2023-11-16 00:34:04 +01:00
Vladimír Čunát
c46eae0f35
Merge branch 'master' into staging-next 2023-11-15 07:33:10 +01:00
Silvan Mosberger
b04b7d64f7
Merge pull request #239722 from Stunkymonkey/lib-allUnique
lib.lists.allUnique: init
2023-11-15 04:37:39 +01:00
Silvan Mosberger
1c3eb9eff1 lib.fileset.fileFilter: Restrict second argument to paths
While this change is backwards-incompatible, I think it's okay because:
- The `fileFilter` function is not yet in a stable NixOS release, it was only merged about [a month ago](https://github.com/NixOS/nixpkgs/pull/257356).
  - All public uses of the function on GitHub only pass a path
- Any `fileFilter pred fileset` can also be expressed as `intersection fileset (fileFilter pred path)` without loss of functionality.
  - This is furthermore pointed out in the new error message when a file set is passed
2023-11-15 01:20:36 +01:00
Silvan Mosberger
7e533bab6d
Merge pull request #267381 from tweag/fileset.fileFilter-path
`fileset.fileFilter`: Don't run predicate unnecessarily
2023-11-15 01:19:36 +01:00
Felix Buehler
66261e9961 lib.lists.allUnique: init 2023-11-14 19:52:32 +01:00
Martin Weinelt
3c336a1647
Merge remote-tracking branch 'origin/master' into staging-next
Conflicts:
- pkgs/development/libraries/gdcm/default.nix
2023-11-14 13:16:34 +01:00
Silvan Mosberger
e89ad83555
Merge pull request #263059 from DanielSidhion/add-doc-convention
doc: commit header convention for documentation changes
2023-11-14 08:11:41 +01:00
Silvan Mosberger
2035f8a324 lib.fileset.fileFilter: Don't run predicate unnecessarily
Before:

    nix-repl> fileset.trace (fileset.fileFilter (file: builtins.trace file.name false) ./default.nix)
    trace: README.md
    trace: benchmark.sh
    trace: default.nix
    trace: internal.nix
    trace: mock-splitRoot.nix
    trace: tests.sh

After:

    nix-repl> fileset.trace (fileset.fileFilter (file: builtins.trace file.name false) ./default.nix)
    trace: default.nix
2023-11-14 07:30:16 +01:00
Silvan Mosberger
e1d8331738 lib.fileset.fileFilter: Minor cleanups and more tests 2023-11-14 07:29:57 +01:00
github-actions[bot]
6a7c026ba7
Merge master into staging-next 2023-11-14 06:01:03 +00:00
DS
e93a3d7b58 doc: separate commit header conventions for each area, info on docs changes. 2023-11-13 20:41:08 -08:00
Silvan Mosberger
7eddbf5690
Merge pull request #265710 from ShamrockLee/lib-copy-function-args
lib.mirrorFunctionArgs: init
2023-11-14 05:33:40 +01:00
Silvan Mosberger
9ec104bb2d lib.fileset: Very minor changes 2023-11-13 23:57:52 +01:00
github-actions[bot]
9776723357
Merge master into staging-next 2023-11-11 18:00:58 +00:00
Anthony Roussel
e30f48be94
treewide: fix redirected and broken URLs
Using the script in maintainers/scripts/update-redirected-urls.sh
2023-11-11 10:49:01 +01:00
github-actions[bot]
5f5513224f
Merge staging-next into staging 2023-11-11 00:02:47 +00:00
Robert Hensing
cfd83c931f
Merge pull request #261732 from tweag/fileset.fromSource
`lib.fileset.fromSource`: init
2023-11-10 20:32:51 +01:00
github-actions[bot]
ff59322b65
Merge staging-next into staging 2023-11-10 00:02:49 +00:00