linux_5_2, linux_5_3: fixing nondeterminism

In 5.2 kernel a new mechanism was introduced which embeds the kernel
headers in the kernel image and exposes them in procfs for simplified
use by userland tools.

It was introduced in
43d8ce9d65
and later modified a bit in
f7b101d330

The archive containing the header files had nondeterminism through the
header files metadata - specifically `mtime`, but I also decided to
normalize some other aspects just in case.

In our default setup we currently compile this as a module, so to expose
the headers to test the functionality `kheaders` needs to be loaded.

See https://lkml.org/lkml/2019/10/4/1036 and
https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=fixes&id=2cc99c9cdc8fde5e92e34f9655829449cebd3e00

I commented out the documentation part of the patch to make it cleanly apply to
5.2 and 5.3, see remark in the patch itself.
This commit is contained in:
Dima 2019-09-21 18:16:38 +02:00
parent 6be720b3a7
commit bdccffa81c
2 changed files with 89 additions and 1 deletions

View File

@ -0,0 +1,86 @@
From 2cc99c9cdc8fde5e92e34f9655829449cebd3e00 Mon Sep 17 00:00:00 2001
From: Dmitry Goldin <dgoldin+lkml@protonmail.ch>
Date: Fri, 4 Oct 2019 10:40:07 +0000
Subject: kheaders: make headers archive reproducible
In commit 43d8ce9d65a5 ("Provide in-kernel headers to make
extending kernel easier") a new mechanism was introduced, for kernels
>=5.2, which embeds the kernel headers in the kernel image or a module
and exposes them in procfs for use by userland tools.
The archive containing the header files has nondeterminism caused by
header files metadata. This patch normalizes the metadata and utilizes
KBUILD_BUILD_TIMESTAMP if provided and otherwise falls back to the
default behaviour.
In commit f7b101d33046 ("kheaders: Move from proc to sysfs") it was
modified to use sysfs and the script for generation of the archive was
renamed to what is being patched.
Signed-off-by: Dmitry Goldin <dgoldin+lkml@protonmail.ch>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
nixos note: This patch is from
https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=fixes&id=2cc99c9cdc8fde5e92e34f9655829449cebd3e00
I commented out the documentation part here, so that it easily applies
to linux 5.2 and 5.3, which does not ship with the reproducible build
documentation yet, which only was introduced recently.
---
Documentation/kbuild/reproducible-builds.rst | 13 +++++++++----
kernel/gen_kheaders.sh | 5 ++++-
2 files changed, 13 insertions(+), 5 deletions(-)
#diff --git a/Documentation/kbuild/reproducible-builds.rst b/Documentation/kbuild/reproducible-builds.rst
#index ab92e98c89c8..503393854e2e 100644
# --- a/Documentation/kbuild/reproducible-builds.rst
#+++ b/Documentation/kbuild/reproducible-builds.rst
#@@ -16,16 +16,21 @@ the kernel may be unreproducible, and how to avoid them.
# Timestamps
# ----------
#
#-The kernel embeds a timestamp in two places:
#+The kernel embeds timestamps in three places:
#
# * The version string exposed by ``uname()`` and included in
# ``/proc/version``
#
# * File timestamps in the embedded initramfs
#
#-By default the timestamp is the current time. This must be overridden
#-using the `KBUILD_BUILD_TIMESTAMP`_ variable. If you are building
#-from a git commit, you could use its commit date.
#+* If enabled via ``CONFIG_IKHEADERS``, file timestamps of kernel
#+ headers embedded in the kernel or respective module,
#+ exposed via ``/sys/kernel/kheaders.tar.xz``
#+
#+By default the timestamp is the current time and in the case of
#+``kheaders`` the various files' modification times. This must
#+be overridden using the `KBUILD_BUILD_TIMESTAMP`_ variable.
#+If you are building from a git commit, you could use its commit date.
#
# The kernel does *not* use the ``__DATE__`` and ``__TIME__`` macros,
# and enables warnings if they are used. If you incorporate external
diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
index 9ff449888d9c..aff79e461fc9 100755
--- a/kernel/gen_kheaders.sh
+++ b/kernel/gen_kheaders.sh
@@ -71,7 +71,10 @@ done | cpio --quiet -pd $cpio_dir >/dev/null 2>&1
find $cpio_dir -type f -print0 |
xargs -0 -P8 -n1 perl -pi -e 'BEGIN {undef $/;}; s/\/\*((?!SPDX).)*?\*\///smg;'
-tar -Jcf $tarfile -C $cpio_dir/ . > /dev/null
+# Create archive and try to normalize metadata for reproducibility
+tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
+ --owner=0 --group=0 --sort=name --numeric-owner \
+ -Jcf $tarfile -C $cpio_dir/ . > /dev/null
echo "$src_files_md5" > kernel/kheaders.md5
echo "$obj_files_md5" >> kernel/kheaders.md5
--
cgit 1.2-0.3.lf.el7

View File

@ -94,7 +94,9 @@ let
patches =
map (p: p.patch) kernelPatches
# Required for deterministic builds along with some postPatch magic.
++ optional (stdenv.lib.versionAtLeast version "4.13") ./randstruct-provide-seed.patch;
++ optional (stdenv.lib.versionAtLeast version "4.13") ./randstruct-provide-seed.patch
# Fixes determinism by normalizing metadata for the archive of kheaders
++ optional (stdenv.lib.versionAtLeast version "5.2") ./gen-kheaders-metadata.patch;
prePatch = ''
for mf in $(find -name Makefile -o -name Makefile.include -o -name install.sh); do