Commit Graph

61 Commits

Author SHA1 Message Date
lassulus
ed18db011f make-disk-image: add systemdMinimal to PATH
this is needed to setup /tmp inside the chroot. which is needed by
systemd-boot to sucessfully install
2022-04-05 11:57:53 +02:00
talyz
e8cc900eae
make-disk-image: Make additionalPaths work with Nix 2.4
The `nix` command is marked as experimental since 2.4, so an extra
flag is required to unlock it.
2021-11-29 21:11:15 +01:00
talyz
46f7521144 make-disk-image: Add an argument to only build a Nix store image
Add the `onlyNixStore` argument which enables building images
containing the contents of the Nix store at the root, instead of a
full system.
2021-10-28 12:55:01 +02:00
talyz
87d8eec069 make-disk-image: Add additionalPaths argument
Add an argument which makes it possible to add additional paths to the
image's nix store.
2021-10-28 12:55:01 +02:00
talyz
398a73ac98 make-disk-image: Add copyChannel argument
Add a copyChannel argument which controls whether the current source
tree will be made available as a nix channel in the image or
not. Previously, it always was. Making it available is useful for
interactive use of nix utils, but changes the hash of the image when
the sources are updated.
2021-10-28 12:55:01 +02:00
talyz
56c5efa25b make-disk-image: Reintroduce the installBootLoader argument
installBootLoader was removed in
f1708a9d7d79e2bf2961fc648625578b23b3460f; probably by mistake, since
only the logic was removed, not the argument.
2021-10-28 12:55:01 +02:00
Casey Ransom
c3f28aed94 make-disk-image: Reduce forking
The disk image calculator was using find + exec forking du for every
file in the disk image, making it very slow. Change du to accept files,
nul delimeted on stdin to speed it back up.

Before change:
nix-build nixos/tests/image-contents.nix  9.71s user 1.06s system 8% cpu 2:13.11 total

After change:
nix-build nixos/tests/image-contents.nix  9.93s user 1.23s system 21% cpu 51.601 total
2021-06-21 22:58:07 -04:00
Lassulus
ee04d772e4
Merge pull request #120489 from samueldr/fix/make-disk-image-auto-size
Fix make disk image automatic size
2021-04-26 10:34:15 +02:00
Martin Schwaighofer
f20ae954d5 make-disk-image: fix broken EFI image builds
Work around missing /dev files inside runInLinuxVM by creating a
symlink before calling nixos-enter.

This fixes https://github.com/NixOS/nixpkgs/issues/93381.
I ran into this issue when trying to create a VMware image that boots from EFI.

Thanks @colemickens for reporting this and @danielfullmer for fixing the same thing in in qemu-vm.nix (37676e77cb) and explaining what the issue was.
2021-04-26 01:12:10 +02:00
Samuel Dionne-Riel
7b8b3fab6d make-disk-image: Round image size to the next mebibyte
This ensures the following gptfdisk warning won't happen:

```
Warning: File size is not a multiple of 512 bytes! Misbehavior is likely!
```

Additionally, helps towards aligning the partition to be more optimal
for the underlying storage.

It is actually impossible to align for the actual underlying storage
optimally because we don't know what the block device will be!

But aligning on 1MiB should help.
2021-04-25 15:24:45 -04:00
Samuel Dionne-Riel
ba666011a6 make-disk-image: Account for reserved disk space
This is a bit of a thorny issue. See, the actual `diskSize` variable is
for the *total* disk size, not for the filesystem!

The automatic numbers are meant to compute the *filesystem* required
space. So we have to add any other reserved space!

We have different requirements for reserved space. E.g. there could be
none (when it's actually a filesystem image). There could also be 1MiB
for alignment for an MBR image, legacy+gpt needs 2MiB, then GPT with an
ESP ("bootSize") needs to take the boot partition and GPT size into
account too!

Though luckily(?) for this latter situation we can cheat! As noted in the
change, `bootSize` is NOT the boot partition size. It is actually the
offset where the target filesystem starts.
2021-04-24 14:49:05 -04:00
Samuel Dionne-Riel
9b18a78c73 make-disk-image: Account for the ext4 reserved space
Reserved space includes:

 - inodes space in use (2 blocks per)
 - about 5.2% of the space

The 5.2% reserved space was computed empirically when working on a
previous EXT4 image builder. It seems to stabilize around 5% even for
much larger filesystems.
2021-04-24 14:49:04 -04:00
Samuel Dionne-Riel
05c13a03e2 make-disk-image: Get proper size for automatic size
On some filesystems, `du` without `--apparent-size` will not give the
actual size for a file. Using `--apparent-size` will give us the actual
file size.

Though, this is not actually correct still. 1000 × 1 bytes is not 1000
bytes. It is 1000 × ceil(filesize/blockSize)*blockSize.

So instead of adding up the actual file sizes. We are adding up the
block sizes.

Note that this also changes the builder to work with *bytes*, rather
than with any other units. Doing maths on bytes is less likely to go
awry than doing it on other units.
2021-04-24 14:49:04 -04:00
Jörg Thalheim
e5ce05cc1e
Merge pull request #67493 from clefru/make-disk-image 2021-01-29 07:18:50 +00:00
clefru
60c8cf4e02
Update nixos/lib/make-disk-image.nix
Co-authored-by: Alyssa Ross <hi@alyssa.is>
2021-01-22 19:58:23 +01:00
Francesco Gazzetta
6f21ae7a62 nixos/lib/make-disk-image.nix: support content mode and ownership 2020-12-20 21:22:47 +01:00
Graham Christensen
bc49a0815a
utillinux: rename to util-linux 2020-11-24 12:42:06 -05:00
Graham Christensen
d78aa080f5
make-disk-image: support legacy+gpt 2020-10-30 15:50:24 -04:00
Emery Hemingway
3f922834b8 nixos/lib/make-disk-image.nix: abritary format input
Pass unrecognized format types as the output file extension to
qemu-img. The motivation is support for "vdi" output.
2020-07-31 15:33:08 +02:00
lassulus
883a6079fd make-disk-image: add hybrid and dynamic sized images 2020-07-27 19:45:55 +02:00
lassulus
8d05772134
nixos make-disk-image: fix permissions of /build
This was broken in 460c0d6 (PR #90431); now the nixos-unstable channel
should get unblocked.
vcunat modified this commit to use env-var instead of hardcoding /build
2020-07-10 10:31:26 +02:00
Emery Hemingway
b8a9e3eea1 nixos/lib/make-disk-image.nix: VDI support 2020-06-13 11:08:48 +05:30
Eelco Dolstra
39c0e489d3
Revert "make-disk-image: replace nixos-install by nix-env calls"
This reverts commit e9bf955fd6. We use
nixos-install to ensure that make-disk-image produces the same result
as a regular installation (9802da517f)
and to reduce code duplication. If there is something broken in
nixos-install, it should be fixed there.
2019-10-07 17:21:28 +02:00
Christian Kögler
e9bf955fd6 make-disk-image: replace nixos-install by nix-env calls
nixos-install can only be called from a machine
2019-09-29 19:57:19 +02:00
Clemens Fruhwirth
f664823a1a make-disk-image.nix: Handle cptofs error explicitly
Because the copy process inside the VM does not reliably
give "No space" error message leaving the user wondering what
went wrong:

unable to create directory /mnt/0000fe01///nix/store/yknzxx7w2ck9p30k81gpi5yfjlrq41lr-libsecret-0.18.7/share/locale/ro: Success
[    5.462365] reboot: Restarting system
error processing entry /build/root/nix/store/yknzxx7w2ck9p30k81gpi5yfjlrq41lr-libsecret-0.18.7/share/locale/ro, aborting
error processing entry /build/root/nix/store/yknzxx7w2ck9p30k81gpi5yfjlrq41lr-libsecret-0.18.7/share/locale, aborting
error processing entry /build/root/nix/store/yknzxx7w2ck9p30k81gpi5yfjlrq41lr-libsecret-0.18.7/share, aborting
error processing entry /build/root/nix/store/yknzxx7w2ck9p30k81gpi5yfjlrq41lr-libsecret-0.18.7, aborting
error processing entry /build/root/nix/store, aborting
error processing entry /build/root/nix, aborting
builder for '/nix/store/fsdvqxq92iai7f3w8wcsncgfwag7cj2l-libvirtd-ssh-image.drv' failed with exit code 228
2019-08-26 13:09:53 +02:00
Travis Athougies
a66ef3aa3d Add options to build disk image function (#50239)
* add options to build disk image function

* Revert suffix changes
2019-01-24 01:47:55 +02:00
Matthew Bauer
f05d8f31ec make-disk-image: use filterSource instead of cleanSource
cleanSource does not appear to work correctly in this case. The path
does not get coerced to a string, resulting in a dangling symlink
produced in channel.nix.  Not sure why, but this
seems to fix it.

Fixes #51025.

/cc @elvishjericco
2019-01-07 16:28:50 -06:00
volth
2e979e8ceb [bot] nixos/*: remove unused arguments in lambdas 2018-07-20 20:56:59 +00:00
Eelco Dolstra
cc2eeef4ab
Fix installing the Nixpkgs channel on the installation media
And test that it got installed correctly.
2018-02-27 19:58:23 +01:00
Eelco Dolstra
d12c9911df
Merge remote-tracking branch 'origin/master' into nix-2.0 2018-02-22 17:28:51 +01:00
Jan Malakhovski
2e6b796761 nixos: rename config.system.nixos* -> config.system.nixos.* 2018-02-18 12:56:30 +00:00
Eelco Dolstra
9802da517f
make-disk-image.nix: Use nixos-install again
Since nixos-install doesn't require any special privileges anymore,
this Just Works. No more need for fakeroot / nixos-prepare-root.
2018-02-07 19:39:13 +01:00
Tuomas Tynkkynen
962e79ef32 nixos/make-disk-image.nix: Support EFI images
- Add a new parameter `imageType` that can specify either "efi" or
  "legacy" (the default which should see no change in behaviour by
  this patch).

- EFI images get a GPT partition table (instead of msdos) with a
  mandatory ESP partition (so we add an assert that `partitioned`
  is true).

- Use the partx tool from util-linux to determine exact start + size
  of the root partition. This is required because GPT stores a secondary
  partition table at the end of the disk, so we can't just have
  mkfs.ext4 create the filesystem until the end of the disk.

- (Unrelated to any EFI changes) Since we're depending on the
  `-E offset=X` option to mkfs which is only supported by e2fsprogs,
  disallow any attempts of creating partitioned disk images where
  the root filesystem is not ext4.
2018-01-22 11:18:23 +02:00
Tuomas Tynkkynen
758b4c1ea4 make-disk-image.nix: Really fix write permissions in the store
I think the current one applies the -exec only to those that match
'-type d'. Let's switch it to something that humans can understand...
2017-12-20 17:28:46 +02:00
Tuomas Tynkkynen
c9f71974f8 make-disk-image.nix: Remove write permissions from /nix/store
Fakeroot seems to always give the owner write bit to any files touched
inside it (presumably to easily simulate the fact that root can still
modify such files). So do an explicit chmod to remove them.

This should finally solve #32242 after the EC2 images are regenerated
with this change.

https://hydra.nixos.org/build/66143116
2017-12-17 01:19:05 +02:00
Tuomas Tynkkynen
0d27df280f build-support/vm: Use devtmpfs, not static device nodes
In 2017, there is no reason to create a static /dev.
2017-12-12 14:31:50 +02:00
Orivej Desh
470a7752d8 make-disk-image: fix eval 2017-11-13 14:56:30 +00:00
Orivej Desh
27f5cc959b make-disk-image: embed compression switch into the format name 2017-11-12 18:39:36 +00:00
Orivej Desh
608de48fe4 make-disk-image: add compressed option to compress qcow2 images 2017-11-12 18:28:06 +00:00
Orivej Desh
a2d3ee77b8 make-disk-image: clean up 2017-11-12 18:24:26 +00:00
Orivej Desh
a84ce72fdc make-disk-image: do not use faketime
Since https://github.com/lkl/linux/pull/394 cptofs preserves the source time,
which is 1970-01-01T00:00:01Z for /nix/store and recent for other files.

This reverts commit f5b3f2c5a7.
2017-11-11 20:57:14 +00:00
Orivej Desh
f218ef647c make-disk-image: clean up 2017-11-11 20:24:26 +00:00
Dan Peebles
f5b3f2c5a7 make-disk-iamge: use faketime to get a consistent timestamp
This fixes #28768 because during an image build, Nix sees bad store
timestamps and attempts to fix them, but can't fix them on a running
system (due to being inside a builder). Since timestamps on the store
are supposed to be 1 anyway, if we fix this, that fixes image building
inside booted images made this way.

Note that this adds quite a bit of noise to the output, because running
`cptofs` under `faketime` causes a bunch of seemingly spurious error
messages and my attempts to suppress them all failed. We'll fix it when
`cptofs` gets a native timestamp preservation feature.
2017-11-10 05:15:37 +00:00
Bjørn Forsman
0ff4bb5f87 nixos: run parted with --script option
-s, --script: never prompts for user intervention

Sometimes the NixOS installer tests fail when they invoke parted, e.g.
https://hydra.nixos.org/build/62513826/nixlog/1. But instead of exiting
right there, the tests hang until the Nix builder times out (and kills
the build). With this change the tests would instead fail immediately,
which is preferred.

While at it, use "parted --script" treewide, so nobody gets build
timeout due to parted error (or misuse). (Only nixos/ use it, and only
non-interactive.)

A few instances already use the short option "-s", convert them to long
option "--short".
2017-10-14 15:29:02 +02:00
Dan Peebles
e2589b3ca2 Deduplicate some filterSource invocations
This version should have more conventional regexes that work across many
platforms and regex engines. This is an issue because up until Nix 1.11,
Nix called out to the libc regex matcher, which behaved differently on
Darwin and Linux. And in Nix 1.12, we're moving to std::regex which will
also behave differently here.

And yes, I do actually evaluate make-disk-image.nix on Darwin ;)
2017-08-29 20:27:04 -04:00
Dan Peebles
5f372ef67f amazon-image: support VPC/VHD as an output format for the builder 2017-08-10 22:57:26 +00:00
Dan Peebles
f1708a9d7d make-disk-image: change to be less VM-centric
This changes much of the make-disk-image.nix logic (and thus most NixOS
image building) to use LKL to set up the target directory structure rather
than a Linux VM. The only work we still do in a VM is less IO-heavy stuff
that while still time-consuming, is less of the overall load. The goal is
to kill more of that stuff, but that will require deeper changes to NixOS
activation scripts and switch-to-configuration.pl, and I don't want to
bite off too much at once.
2017-04-24 02:30:00 +00:00
Dan Peebles
e798f573f0 make-disk-image.nix: set last fsck time on ext4 images to enable resize-on-startup 2017-02-26 02:02:22 +00:00
Dan Peebles
49641e0de0 make-disk-image.nix: support additional filesystem contents
This makes make-disk-image.nix slightly more consistent with other image
builders we have. Unfortunately I duplicated some code in doing so, but
this is temporary duplication on the path to consolidating everything.
See https://github.com/NixOS/nixpkgs/issues/23052 for more details on that.

I'm also exposing the option in the amazon-image.nix maintainer module.
2017-02-22 23:49:49 +00:00
Domen Kožar
b6363c7bc8 make-disk-image: make store validity fix optional
This is useful for EC2 AMI generation to speedup the process.

In my case it removes 13min out of 45min when generating an image
on EC2.
2016-12-07 13:30:20 +01:00