Commit Graph

96 Commits

Author SHA1 Message Date
Alyssa Ross
59fca7cb67
compressFirmwareXz: don't allow references 2024-01-28 11:57:38 +01:00
Alyssa Ross
24bb06069f
compressFirmwareXz: fail on broken symlinks
This would have caught the last two issues we had with compression.
2024-01-28 11:57:37 +01:00
Alyssa Ross
a136def4f7
compressFirmwareXz: fix symlink type check
The previous version didn't work in the case of relative symlinks in
subdirectories.  If "foo/bar" was a link to "baz", it would check for
a link to "baz" in the root, rather than under "foo".

We don't need to dereference the symlink ourselves for [ anyway, as it
dereferences its arguments itself, so all we need to do to fix this is
to pass it the link.

Fixes: 14f83d5c6f ("compressFirmwareXz: fix links to directories")
2024-01-28 11:57:30 +01:00
Franz Pletz
14f83d5c6f
compressFirmwareXz: fix links to directories
Only add the `.xz` suffix to links if link target is a regular file.
This breaks if the target is a directory.

Fixes #283005.
2024-01-23 07:16:59 +01:00
R. Ryantm
19b4bac3d0 makeInitrdNGTool: 0.1.0 -> 0.1.0 2024-01-07 10:39:28 +00:00
R. Ryantm
5ff7811393 makeInitrdNGTool: 0.1.0 -> 0.1.0 2024-01-01 20:10:05 +00:00
Atemu
d7b70652ef
Merge pull request #272125 from yaxitech/reproducible-initrd
make-initrd-ng: fix reproducibility problems
2023-12-24 11:53:33 +01:00
R. Ryantm
d2ffb24948 makeInitrdNGTool: 0.1.0 -> 0.1.0 2023-12-16 14:58:17 +00:00
Andreas Stührk
4084ee0cd5 make-initrd-ng: fix reproducibility problems
The previous find invocation didn't match the root directory, so the
root directory's access and modification time wasn't set to a
deterministic value and the build time leaked into the output.

`make-initrd` replaced `cpio` with `bsdtar` in #165892 because the
former includes the number of hardlinks in the created archive, which
depends on the filesystem (and can also be influenced by `nix-store
--optimise`). The same problem applies to `make-initrd-ng`, so this
commit replaces `cpio` with `libarchive`'s `bsdtar`.
2023-12-04 19:23:06 +01:00
Alyssa Ross
1557027e33 makeModulesClosure: handle firmware glob patterns
A handful of kernel modules use glob patterns to express their
firmware dependencies.  (`git grep 'MODULE_FIRMWARE.*\*'`)

Previously, we weren't handling these patterns.  Now, we are.
2023-11-29 16:03:16 +01:00
R. Ryantm
8c15911d79 makeInitrdNGTool: 0.1.0 -> 0.1.0 2023-11-23 13:26:47 +00:00
Will Fancher
18013cc71e systemd-stage-1: Use specific fs packages 2023-10-19 21:13:22 -04:00
R. Ryantm
3876a545ed makeInitrdNGTool: 0.1.0 -> 0.1.0 2023-10-11 02:44:30 +00:00
Adam Joseph
a97e8fc272 make-initrd-ng: use hostPlatform.ubootArch for uinitrdArch
This allows make-initrd-ng to pick up the few cases where Linux and
u-boot disagree.
2023-08-14 01:34:08 -07:00
Felix Bühler
0a2745684e
Merge pull request #239624 from Stunkymonkey/use-optionalString-then
treewide: use optionalString instead of 'then ""'
2023-07-22 13:02:47 +02:00
rnhmjoj
4124eb7bd5
compressFirmwareXz: preserve meta attributes
Among other things, this preserves the package priority, which is
important when building the `hardware.firmware` environment in NixOS.
2023-06-25 11:08:32 +02:00
Felix Buehler
f3719756b5 treewide: use optionalString instead of 'then ""' 2023-06-24 20:19:19 +02:00
Cole Helbling
3e63608c7b makeInitrdNGTool: better errors
It's extremely frustrating seeing "Error: Os { code: 13, kind:
PermissionDenied, message: "Permission denied" }" without any hint as to
where exactly that occurred.

This commit fixes that by adding context to most errors.
2023-05-25 13:26:05 -07:00
Will Fancher
762b69f2ff systemd-initrd: Fix up root directory mode 2023-04-11 15:20:47 -04:00
Lily Foster
d01bc6f9cb
make-initrd-ng: document wrapped file behavior 2023-02-20 07:02:55 -05:00
Lily Foster
4df8f9a2f8
make-initrd-ng: support wrapped executables 2023-02-20 07:02:55 -05:00
Artturin
341e6fd558 splice.nix: start deprecating nativeDrv and crossDrv 2022-11-19 00:04:54 +02:00
K900
de93795b46 make-initrd-ng: clean up a bit 2022-08-02 10:54:55 +03:00
Linus Heckemann
ee38010981 makeInitrdNG: add meta and update script 2022-08-02 10:54:55 +03:00
Linus Heckemann
6fc909a1cc makeInitrdNG: make stripping fully optional
Now the tool will only strip binaries if a strip executable is passed
via the STRIP environment variable. This is exposed via the strip
option for makeInitrdNG and the NixOS option boot.initrd.systemd.strip.
2022-08-02 10:19:48 +03:00
K900
daee67dae6 make-initrd-ng: use goblin instead of shelling out to patchelf and friends 2022-08-02 10:19:48 +03:00
K900
1356441cb1 make-initrd-ng: rustfmt 2022-08-02 10:19:48 +03:00
Alyssa Ross
76405e3077
compressFirmwareXz: fix with empty lib/firmware
Fixes: 8aa8e0ce7f ("nixos/udev: compress all firmware if supported")
2022-05-26 07:08:50 +00:00
Alyssa Ross
8aa8e0ce7f
nixos/udev: compress all firmware if supported
This should be a significant disk space saving for most NixOS
installations.  This method is a bit more complicated than doing it in
the postInstall for the firmware derivations, but this way it's
automatic, so each firmware package doesn't have to separately
implement its compression.

Currently, only xz compression is supported, but it's likely that
future versions of Linux will additionally support zstd, so I've
written the code in such a way that it would be very easy to implement
zstd compression for those kernels when they arrive, falling back to
xz for older (current) kernels.

I chose the highest possible level of compression (xz -9) because even
at this level, decompression time is negligible.  Here's how long it took
to decompress every firmware file my laptop uses:

	i915/kbl_dmc_ver1_04.bin                  	2ms
	regulatory.db                             	4ms
	regulatory.db.p7s                         	3ms
	iwlwifi-7265D-29.ucode                    	62ms
	9d71-GOOGLE-EVEMAX-0-tplg.bin             	22ms
	intel/dsp_fw_kbl.bin                      	65ms
	dsp_lib_dsm_core_spt_release.bin          	6ms
	intel/ibt-hw-37.8.10-fw-22.50.19.14.f.bseq	7ms

And since booting NixOS is a parallel process, it's unlikely (but
difficult to measure) that the time to user interaction was held up at
all by most of these.

Fixes (partially?) #148197
2022-05-13 14:36:34 +00:00
Janne Heß
e5995b2235
makeInitrdNG: Strip more and remove output
This strips all elf files as far as possible and removes a lot of
unnecessary output. Also wrap in the binaries instead of relying on
$PATH.
2022-05-01 17:06:14 +02:00
Artturi
98ff3e401c
Merge pull request #165892 from tpwrules/fix-initrd-dirlinks
make-initrd: fix reproducibility problems
2022-04-21 20:51:35 +03:00
Janne Heß
b9bf28fd70
nixos/stage-1-init: Pass all parameters to the builder
This is for compatiblity with the old builder to get compat with uboot
support and to prepend microcode updates to the initrd
2022-04-16 20:36:18 +01:00
Thomas Watson
7fd6cea253 make-initrd: fix reproducibility problems
cpio includes the number of directory hard links in archives it creates.
Some filesystems, like btrfs, do not count directory hard links the same
way as more common filesystems like ext4 or tmpfs, so archives built
when /tmp is on such a filesystem do not reproduce. This patch replaces
cpio with bsdtar, which does not have this issue. The specific
invocation is from this page:
https://reproducible-builds.org/docs/archives/
2022-04-14 19:06:30 -05:00
Will Fancher
d193ef8a57 make-initrd-ng: init 2022-03-22 07:02:22 -04:00
Thomas Watson
d8bb89b9ff make-initrd: fix reproducibility problems with hard links 2022-01-29 18:07:11 -05:00
Linus Heckemann
b0fc6e8ff9 make-initrd: fix #132059 2021-08-08 07:53:59 +02:00
Ben Siraphob
02ee14b9d5 treewide: stdenvNoCC.lib -> lib 2021-01-27 13:01:51 +07:00
John Ericson
9c213398b3 lib: Clean up how linux and gcc config is specified
Second attempt of 8929989614589ee3acd070a6409b2b9700c92d65; see that
commit for details.

This reverts commit 0bc275e634.
2021-01-23 10:01:28 -05:00
Jonathan Ringer
0bc275e634
Revert "lib: Clean up how linux and gcc config is specified"
This is a stdenv-rebuild, and should not be merged
into master

This reverts commit 8929989614.
2021-01-22 14:07:06 -08:00
John Ericson
8929989614 lib: Clean up how linux and gcc config is specified
The `platform` field is pointless nesting: it's just stuff that happens
to be defined together, and that should be an implementation detail.

This instead makes `linux-kernel` and `gcc` top level fields in platform
configs. They join `rustc` there [all are optional], which was put there
and not in `platform` in anticipation of a change like this.

`linux-kernel.arch` in particular also becomes `linuxArch`, to match the
other `*Arch`es.

The next step after is this to combine the *specific* machines from
`lib.systems.platforms` with `lib.systems.examples`, keeping just the
"multiplatform" ones for defaulting.
2021-01-21 22:44:09 -05:00
Dominik Xaver Hörl
49130f93b7 nixos/modules-closure.sh: don't fail if firmware is missing
Since fdf32154fc, we no longer allow
missing modules in the initrd. Unfortunately since before this commit,
the modules-closure script would also fail on missing firmware, which
is a very common case (e.g. xhci-pci.ko.xz lists renesas_usb_fw.mem as
dependent firmware). Fix this by only issuing a warning instead.
2021-01-11 17:26:33 +01:00
Linus Heckemann
14fbf575ec make-initrd: various improvements
- Generate a link to the initramfs file with an appropriate file
  extension, guessed based on the compressor by default
- Use correct metadata in u-boot images if generated, up to now this
  was hardcoded to gzip and would silently generate an erroneous image
  if another compressor was specified
- Document all the parameters
- Improve cross-building compatibility, by allowing passing either a
  string as before, or a function taking a package set and returning the
  path to a compressor in the "compressor" argument of the
  function.
- Support more compression algorithms
- Place compressor executable function and arguments in passthru, for
  reuse when appending initramfses

Co-Authored-By: Dominik Xaver Hörl <hoe.dom@gmx.de>
2020-12-17 11:37:04 +01:00
Linus Heckemann
3a3c9c9548 makeInitrd: include dotfiles at root 2020-12-17 11:10:37 +01:00
Linus Heckemann
85e0ae7827 makeInitrd: don't assume uImage => arm
mips for example might use uImages too
2020-12-17 11:10:37 +01:00
Linus Heckemann
a343ff7e14 makeInitrd: make uinitrd behaviour optional 2020-12-17 11:10:36 +01:00
Arthur Gautier
70bc1a3f49 makeModulesClosuse: read modules from corrent kernel version
Before this commit, the firmware information would be loaded from the
currently running kernel, not from the kernel to be loaded.

This commit ensures the correct kernel version and modules are read.
2020-08-27 10:48:18 -07:00
Arthur Gautier
ee8572d6b3 makeModulesClosure: fixup firmware extraction
After a recent upgrade of modinfo, its output is now incorrect for
builtin modules. This commit filters out the output until a fix is made
available upstream
2020-08-27 10:45:23 -07:00
CrystalGamma
b155b1dafb makeModulesClosure: handle builtin modules better
The previous code discarded entire dependency trees if the first entry in the dependency list compiled by `modprobe --show-depends` is a builtin and otherwise handled its output in a rather hackish way.
2020-07-16 20:44:07 +02:00
Markus S. Wamser
34b04a6364 modules-shrunk: preserve module priorities from build
depmod looks for files modules.order and modules.builtin which are
generated at kernel build time but were previously not passed to
the modules-shrunk derivation
2019-12-07 23:04:17 +01:00
Eelco Dolstra
328796d734
makeInitrd: Use stdenvNoCC 2019-10-11 17:12:09 +02:00