colin/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

doc: polish the docs of checkpointBuildTools

Browse Source 
... following suggestions from @phip1611 and @infinisil.
This commit is contained in:
Bryan Lai 2024-01-07 18:57:04 +08:00
parent 5f3aad00ff
commit d7253bea6d
2 changed files with 34 additions and 30 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
doc/build-helpers/special/checkpoint-build.section.md
View File

@ -2,35 +2,38 @@
`pkgs.checkpointBuildTools` provides a way to build derivations incrementally. It consists of two functions to make checkpoint builds using Nix possible.
For hermeticity, Nix derivations do not allow any state to carry over between builds, making a transparent incremental build within a derivation impossible.
For hermeticity, Nix derivations do not allow any state to be carried over between builds, making a transparent incremental build within a derivation impossible.
However, we can tell Nix explicitly what the previous build state was, by representing that previous state as a derivation output. This allows the passed build state to be used for an incremental build.
To change a normal derivation to a checkpoint based build, these steps must be taken:
- apply `prepareCheckpointBuild` on the desired derivation
e.g.:
- apply `prepareCheckpointBuild` on the desired derivation, e.g.
```nix
checkpointArtifacts = (pkgs.checkpointBuildTools.prepareCheckpointBuild pkgs.virtualbox);
```
- change something you want in the sources of the package. (e.g. using a source override)
- change something you want in the sources of the package, e.g. use a source override:
```nix
changedVBox = pkgs.virtualbox.overrideAttrs (old: {
src = path/to/vbox/sources;
}
});
```
- use `mkCheckpointBuild changedVBox incrementalBuildArtifacts`
- use `mkCheckpointBuild changedVBox checkpointArtifacts`
- enjoy shorter build times
## Example {#sec-checkpoint-build-example}
```nix
{ pkgs ? import <nixpkgs> {} }: with (pkgs) checkpointBuildTools;
{ pkgs ? import <nixpkgs> {} }:
let
helloCheckpoint = checkpointBuildTools.prepareCheckpointBuild pkgs.hello;
inherit (pkgs.checkpointBuildTools)
prepareCheckpointBuild
mkCheckpointBuild
;
helloCheckpoint = prepareCheckpointBuild pkgs.hello;
changedHello = pkgs.hello.overrideAttrs (_: {
doCheck = false;
patchPhase = ''
sed -i 's/Hello, world!/Hello, Nix!/g' src/hello.c
'';
});
in checkpointBuildTools.mkCheckpointBuild changedHello helloCheckpoint
in mkCheckpointBuild changedHello helloCheckpoint
```

43
pkgs/build-support/checkpoint-build.nix
View File

@ -14,39 +14,40 @@ in
rec {
/* Prepare a derivation for local builds.
*
* This function prepares checkpoint builds by provinding,
* containing the build output and the sources for cross checking.
* This function prepares checkpoint builds by storing
* the build output and the sources for cross checking.
* The build output can be used later to allow checkpoint builds
* by passing the derivation output to the `mkCheckpointBuild` function.
*
* To build a project with checkpoints follow these steps:
* - run prepareIncrementalBuild on the desired derivation
* e.G `incrementalBuildArtifacts = (pkgs.checkpointBuildTools.prepareCheckpointBuild pkgs.virtualbox);`
* - change something you want in the sources of the package( e.G using source override)
* changedVBox = pkgs.virtuabox.overrideAttrs (old: {
* src = path/to/vbox/sources;
* }
* - use `mkCheckpointBuild changedVBox incrementalBuildArtifacts`
* To build a project with checkpoints, follow these steps:
* - run `prepareCheckpointBuild` on the desired derivation, e.g.
* checkpointArtifacts = prepareCheckpointBuild virtualbox;
* - change something you want in the sources of the package,
* e.g. using source override:
* changedVBox = pkgs.virtuabox.overrideAttrs (old: {
* src = path/to/vbox/sources;
* };
* - use `mkCheckpointBuild changedVBox checkpointArtifacts`
* - enjoy shorter build times
*/
prepareCheckpointBuild = drv: drv.overrideAttrs (old: {
outputs = [ "out" ];
name = drv.name + "-checkpointArtifacts";
# To determine differences between the state of the build directory
# from an earlier build and a later one we store the state of the build
# from an earlier build and a later one we store the state of the build
# directory before build, but after patch phases.
# This way, the same derivation can be used multiple times and only changes are detected.
# Additionally Removed files are handled correctly in later builds.
# Additionally, removed files are handled correctly in later builds.
preBuild = (old.preBuild or "") + ''
mkdir -p $out/sources
cp -r ./* $out/sources/
'';
# After the build the build directory is copied again
# After the build, the build directory is copied again
# to get the output files.
# We copy the complete build folder, to take care for
# Build tools, building in the source directory, instead of
# having a build root directory, e.G the Linux kernel.
# We copy the complete build folder, to take care of
# build tools that build in the source directory, instead of
# having a separate build directory such as the Linux kernel.
installPhase = ''
runHook preCheckpointInstall
mkdir -p $out/outputs
@ -56,18 +57,18 @@ rec {
});
/* Build a derivation based on the checkpoint output generated by
* the `prepareCheckpointBuild function.
* the `prepareCheckpointBuild` function.
*
* Usage:
* let
* checkpointArtifacts = prepareCheckpointBuild drv
* checkpointArtifacts = prepareCheckpointBuild drv;
* in mkCheckpointBuild drv checkpointArtifacts
*/
mkCheckpointBuild = drv: checkpointArtifacts: drv.overrideAttrs (old: {
# The actual checkpoint build phase.
# We compare the changed sources from a previous build with the current and create a patch
# Afterwards we clean the build directory to copy the previous output files (Including the sources)
# The source difference patch is applied to get the latest changes again to allow short build times.
# We compare the changed sources from a previous build with the current and create a patch.
# Afterwards we clean the build directory and copy the previous output files (including the sources).
# The source difference patch is then applied to get the latest changes again to allow short build times.
preBuild = (old.preBuild or "") + ''
set +e
sourceDifferencePatchFile=$(${mktemp}/bin/mktemp)