haskell.lib.compose.justStaticExecutables: Forbid references to GHC

This makes `justStaticExecutables` error if the produced store path
contains references to GHC. This is almost always erroneous and due to
the generated `Paths_*` module being imported. This helps prevent
`justStaticExecutables` from producing binaries with closure sizes in
the gigabytes.

See: https://github.com/NixOS/nixpkgs/issues/164630

Co-authored-by: sternenseemann <sternenseemann@systemli.org>
This commit is contained in:
Rebecca Turner 2024-04-15 12:45:45 -07:00 committed by sternenseemann
parent 29eb2c04cf
commit d2618822ab
4 changed files with 94 additions and 10 deletions

View File

@ -923,14 +923,59 @@ for this to work.
`justStaticExecutables drv`
: Only build and install the executables produced by `drv`, removing everything
that may refer to other Haskell packages' store paths (like libraries and
documentation). This dramatically reduces the closure size of the resulting
derivation. Note that the executables are only statically linked against their
Haskell dependencies, but will still link dynamically against libc, GMP and
other system library dependencies. If dependencies use their Cabal-generated
`Paths_*` module, this may not work as well if GHC's dead code elimination
is unable to remove the references to the dependency's store path that module
contains.
that may refer to other Haskell packages' store paths (like libraries and
documentation). This dramatically reduces the closure size of the resulting
derivation. Note that the executables are only statically linked against their
Haskell dependencies, but will still link dynamically against libc, GMP and
other system library dependencies.
If the library being built or its dependencies use their Cabal-generated
`Paths_*` module, this may not work as well if GHC's dead code elimination is
unable to remove the references to the dependency's store path that module
contains. (See [nixpkgs#164630][164630] for more information.)
Importing the `Paths_*` module may cause builds to fail with this message:
```
error: output '/nix/store/64k8iw0ryz76qpijsnl9v87fb26v28z8-my-haskell-package-1.0.0.0' is not allowed to refer to the following paths:
/nix/store/5q5s4a07gaz50h04zpfbda8xjs8wrnhg-ghc-9.6.3
```
If that happens, first disable the check for GHC references and rebuild the
derivation:
```nix
pkgs.haskell.lib.overrideCabal
(pkgs.haskell.lib.justStaticExecutables my-haskell-package)
(drv: {
disallowGhcReference = false;
})
```
Then use `strings` to determine which libraries are responsible:
```
$ nix-build ...
$ strings result/bin/my-haskell-binary | grep /nix/store/
...
/nix/store/n7ciwdlg8yyxdhbrgd6yc2d8ypnwpmgq-hs-opentelemetry-sdk-0.0.3.6/bin
...
```
Finally, use `remove-references-to` to delete those store paths from the produced output:
```nix
pkgs.haskell.lib.overrideCabal
(pkgs.haskell.lib.justStaticExecutables my-haskell-package)
(drv: {
postInstall = ''
${drv.postInstall or ""}
remove-references-to -t ${pkgs.haskellPackages.hs-opentelemetry-sdk}
'';
})
```
[164630]: https://github.com/NixOS/nixpkgs/issues/164630
`enableSeparateBinOutput drv`
: Install executables produced by `drv` to a separate `bin` output. This

View File

@ -23,6 +23,14 @@
before changing the package to `pkgs.stalwart-mail` in
[`services.stalwart-mail.package`](#opt-services.stalwart-mail.package).
- `haskell.lib.compose.justStaticExecutables` now disallows references to GHC in the
output by default, to alert users to closure size issues caused by
[#164630](https://github.com/NixOS/nixpkgs/issues/164630). See ["Packaging
Helpers" in the Haskell section of the Nixpkgs
manual](https://nixos.org/manual/nixpkgs/unstable/#haskell-packaging-helpers)
for information on working around `output '...' is not allowed to refer to
the following paths` errors caused by this change.
## Other Notable Changes {#sec-release-24.11-notable-changes}
- Create the first release note entry in this section!

View File

@ -110,6 +110,28 @@ in
# build products from that prior build as a starting point for accelerating
# this build
, previousIntermediates ? null
# References to these store paths are forbidden in the produced output.
, disallowedRequisites ? []
# Whether to allow the produced output to refer to `ghc`.
#
# This is used by `haskell.lib.justStaticExecutables` to help prevent static
# Haskell binaries from having erroneous dependencies on GHC.
#
# Generated `Paths_*` modules include paths for the runtime library
# directory (and similar) of the package being built. If the `Paths_*`
# module is imported, this creates a dependency from the static binary
# being built to the _library_ being built (which is dynamically linked
# and depends on the GHC used to build it).
#
# To avoid this:
# 1. Build the impacted derivation.
# 2. Run `strings` on the built binary of the impacted derivation to
# locate the store paths it depends on.
# 3. Add `remove-references-to -t ${bad-store-path-in-binary}` to the
# impacted derivation's `postInstall`.
#
# See: https://github.com/NixOS/nixpkgs/issues/164630
, disallowGhcReference ? false
, # Cabal 3.8 which is shipped by default for GHC >= 9.3 always calls
# `pkg-config --libs --static` as part of the configure step. This requires
# Requires.private dependencies of pkg-config dependencies to be present in
@ -680,9 +702,17 @@ stdenv.mkDerivation ({
runHook postInstallIntermediates
'';
disallowedRequisites =
disallowedRequisites
++ (
if disallowGhcReference
then [ghc]
else []
);
passthru = passthru // rec {
inherit pname version;
inherit pname version disallowGhcReference;
compiler = ghc;

View File

@ -290,7 +290,7 @@ rec {
/* link executables statically against haskell libs to reduce
closure size
*/
justStaticExecutables = overrideCabal (drv: {
justStaticExecutables = overrideCabal (drv: {
enableSharedExecutables = false;
enableLibraryProfiling = false;
isLibrary = false;
@ -300,6 +300,7 @@ rec {
# Remove every directory which could have links to other store paths.
rm -rf $out/lib $out/nix-support $out/share/doc
'';
disallowGhcReference = true;
});
/* Build a source distribution tarball instead of using the source files