Frequently, users complain about incompatibilities between biblatex from
Tectonic's bundle releases, and Nixpkgs' biber version, compiled from
texlive's sources. The issue does not happen within Nixpkgs' texlive,
and Nixpkgs' biber, since both of them are updated at the same time, and
hence are always compatible with each other. Since Tectonic manages it's
tex packages bundle by itself, it's natural that there will be
incompatibilities between Tectonic's user's distribution `biber`, and
Tectonic's biblatex version from Tectonic's bundle. See upstream's
issue: https://github.com/tectonic-typesetting/tectonic/issues/893
For the issue not to arise, we simply hope Tectonic's bundle update
itself not long after Nixpkgs' texlive packages are updated. In anycase,
`tectonic`'s users in Nixpkgs will be better off dealing with
incompatibilities without the need to recompile `tectonic` just to
change the `biber` path in wrapper.
This fixes a bug that allowed any document compiled with LuaTeX to execute
arbitrary shell commands, even with shell escape disabled.
See https://tug.org/~mseven/luatex.html for more details.
Calling 'faketime fmtutil' fails on some non-NixOS systems, due to
fmtutil calling system, and implicitly /bin/sh, which may be
incompatible with faketime's LD_PRELOAD (e.g. using a different glibc).
This change moves faketime within fmtutil so that LD_PRELOAD is applied
only to the final binary.
This is neccessary for XeTeX to find the fonts distributed with
TeXLive.
The fontconfig file generated by makeFontsConf is almost identical to the one
from the fontconfig package and /etc/fonts/fonts.conf on NixOS -- it only adds
the specified font directories. In particular this means that all the actual
configuration files in /etc/fonts/conf.d are still included and there also is a
/usr/share/fonts font directory entry for use with non-NixOS linuxes, so
it's unlikely that this breaks someone's setup.
The 'non-relocated' packages of TeX Live expect to live in
TEXMFROOT/texmf-dist (e.g. to search for perl modules). This change
builds TEXMFROOT instead of just TEXMFDIST to accommodate those
packages, and obviate the need for patching fmtutil.
prior to this change, the build fails because the documentation cannot be
generated due to a missing latex package (probably fallout from a texlive
update):
! LaTeX Error: File `hypdoc.sty' not found.
Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)
Enter file name:
! Emergency stop.
<read *>
l.29 \begin{document}
^^M
! ==> Fatal error occurred, no output PDF file produced!
Transcript written on preview.log.
make[1]: *** [Makefile:91: preview.pdf] Error 1
make[1]: Leaving directory '/build/auctex-12.3/latex'
make: *** [Makefile:219: texmf] Error 2
This commit gets pkgsCross.*.pdfgrep a bit farther. It still fails
with the error below; if anybody recognizes it I'd appreciate a hint.
```
pdfgrep-aarch64-unknown-linux-gnu> aarch64-unknown-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I.. -I/nix/store/9kzvsy9ybsy2zvivz62ia881q9djf734-poppler-glib-aarch64-unknown-linux-gnu-23.02.0-dev/include/poppler/cpp -I/nix/store/6ib0mcvv0ascsydhpja2a1kcnwvxqnfp-pcre-aarch64-unknown-linux-gnu-8.45-dev/include -I/nix/store/yyyy71mx8j85hh1xknfy2nqgw4ihfchr-libgcrypt-aarch64-unknown-linux-gnu-1.10.1-dev/include -I/nix/store/mqx4d57dva5y5kg91a9pnlipdc6bfcxc-libgpg-error-aarch64-unknown-linux-gnu-1.46-dev/include -g -O2 -c -o pdfgrep.o pdfgrep.cc
pdfgrep-aarch64-unknown-linux-gnu> In file included from /nix/store/cha4mfjki3pv42xnfsfrgppr3w1zmh27-aarch64-unknown-linux-gnu-stage-final-gcc-12.2.0/aarch64-unknown-linux-gnu/include/c++/12.2.0/stdlib.h:36,
pdfgrep-aarch64-unknown-linux-gnu> from pdfgrep.cc:24:
pdfgrep-aarch64-unknown-linux-gnu> /nix/store/cha4mfjki3pv42xnfsfrgppr3w1zmh27-aarch64-unknown-linux-gnu-stage-final-gcc-12.2.0/aarch64-unknown-linux-gnu/include/c++/12.2.0/cstdlib:151:11: error: 'malloc' has not been declared in '::'
pdfgrep-aarch64-unknown-linux-gnu> 151 | using ::malloc;
pdfgrep-aarch64-unknown-linux-gnu> | ^~~~~~
pdfgrep-aarch64-unknown-linux-gnu> /nix/store/cha4mfjki3pv42xnfsfrgppr3w1zmh27-aarch64-unknown-linux-gnu-stage-final-gcc-12.2.0/aarch64-unknown-linux-gnu/include/c++/12.2.0/cstdlib:164:11: error: 'realloc' has not been declared in '::'
pdfgrep-aarch64-unknown-linux-gnu> 164 | using ::realloc;
pdfgrep-aarch64-unknown-linux-gnu> | ^~~~~~~
pdfgrep-aarch64-unknown-linux-gnu> /nix/store/cha4mfjki3pv42xnfsfrgppr3w1zmh27-aarch64-unknown-linux-gnu-stage-final-gcc-12.2.0/aarch64-unknown-linux-gnu/include/c++/12.2.0/stdlib.h:65:12: error: 'malloc' has not been declared in 'std'
pdfgrep-aarch64-unknown-linux-gnu> 65 | using std::malloc;
pdfgrep-aarch64-unknown-linux-gnu> | ^~~~~~
pdfgrep-aarch64-unknown-linux-gnu> /nix/store/cha4mfjki3pv42xnfsfrgppr3w1zmh27-aarch64-unknown-linux-gnu-stage-final-gcc-12.2.0/aarch64-unknown-linux-gnu/include/c++/12.2.0/stdlib.h:73:12: error: 'realloc' has not been declared in 'std'
pdfgrep-aarch64-unknown-linux-gnu> 73 | using std::realloc;
pdfgrep-aarch64-unknown-linux-gnu> | ^~~~~~~
pdfgrep-aarch64-unknown-linux-gnu> make[2]: *** [Makefile:413: pdfgrep.o] Error 1
pdfgrep-aarch64-unknown-linux-gnu> make[2]: Leaving directory '/build/pdfgrep-2.1.2/src'
pdfgrep-aarch64-unknown-linux-gnu> make[1]: *** [Makefile:388: all-recursive] Error 1
pdfgrep-aarch64-unknown-linux-gnu> make[1]: Leaving directory '/build/pdfgrep-2.1.2'
pdfgrep-aarch64-unknown-linux-gnu> make: *** [Makefile:329: all] Error 2
```
The build system generates the file "hitables.c" with the program
"./himktables" that is also compiled by the build system.
This does not work when cross-compiling since this program is built for the
wrong architecture. We already use a natively built copy of texlive.bin.core for
various other tools (like tangle, weave), so this change just adds himktables to the
this native copy and uses this program in the cross build.
When expanding path variables, kpathsea uses a trick to speed up identifying
leaf directories. The manual says
> The trick is that in every real Unix implementation (as opposed to the POSIX
> specification), a directory which contains no subdirectories will have
> exactly two links (namely, one for . and one for ..). That is to say, the
> st_nlink field in the ‘stat’ structure will be two. Thus, we don’t have to
> stat everything in the bottom-level (leaf) directories—we can just check
> st_nlink, notice it’s two, and do no more work.
>
> But if you have a directory that contains a single subdirectory and 500
> regular files, st_nlink will be 3, and Kpathsea has to stat every one of those
> 501 entries. Therein lies slowness.
>
> You can disable the trick by undefining ST_NLINK_TRICK in
> kpathsea/config.h. (It is undefined by default except under Unix.)
This does not work as expected with nixpkgs symlink trees and programs that rely
on kpathsea path expansion do not work properly.
One example is luaotfload's font database, which is populated by the font files
in the directories obtained by path-expanding the value of the `OPENTYPEFONTS`
configuration variable with kpathsea. The expanded value can be checked with
`kpsewhich --show-path="opentype fonts"`. Before this change, the expanded
value does not include the various font directories symlinked into
`/texmf/fonts/opentype/public`, since kpathsea considers this a leaf
directory (every child is a symlink, not a directory). Hence luaotfload does
not find the fonts in the texlive installation.
This patch disables this trick.
checkInputs used to be added to nativeBuildInputs. Now we have
nativeCheckInputs to do that instead. Doing this treewide change allows
to keep hashes identical to before the introduction of
nativeCheckInputs.
Co-authored-by: Robin Gloster <mail@glob.in>
stdenv: print message if structuredAttrs is enabled
stdenv: add _append
reduces the chance of a user doing it wrong
fix nix develop issue
output hooks don't work yet in nix develop though
making $outputs be the same on non-structuredAttrs and structuredAttrs
is too much trouble.
lets instead make a function that gets the output names
reading environment file '/nix/store/2x7m69a2sm2kh0r6v0q5s9z1dh41m4xf-xz-5.2.5-env-bin'
nix: src/nix/develop.cc:299: std::string Common::makeRcScript(nix::ref<nix::Store>, const BuildEnvironment&, const Path&): Assertion `outputs != buildEnvironment.vars.end()' failed.
use a function to get all output names instead of using $outputs
copy env functionality from https://github.com/NixOS/nixpkgs/pull/76732/commits
The code and typical use case suggest that ghostscript is not used
graphically. Removing this dependency reduces the build closure of
many tools that use texlive for documentation.
If this needs to be reverted, please state the graphical use case
in a comment, so we don't make this mistake again.
Due to missing `:`, wrapProgram didn't really add biber's bin dir to
PATH. Using the `makeBinaryWrapper` implementation detects such mistakes
during compilation.
This is a commandline tool, that does not advertise itself as as
consumable library, so it does not belong into the python package set.
Also fetch from git, so we can enable the tests.
Because we override ENV, we reset the environment when invoking xsltproc.
This is a good thing, since it also makes it much faster and less dependent
on the environment - but in case of `SOURCE_DATE_EPOCH` we actually want
this environment variable to be passed on, to make sure `a2x` does not leak
the build date into the generated manpages when this is not desired.
Fixes#192726
Passing `-l$NIX_BUILD_CORES` improperly limits the overall system load.
For a build machine which is configured to run `$B` builds where each
build gets `total cores / B` cores (`$C`), passing `-l $C` to make will
improperly limit the load to `$C` instead of `$B * $C`.
This effect becomes quite pronounced on machines with 80 cores, with
40 simultaneous builds and a cores limit of 2. On a machine with this
configuration, Nix will run 40 builds and make will limit the overall
system load to approximately 2. A build machine with this many cores
can happily run with a load approaching 80.
A non-solution is to oversubscribe the machine, by picking a larger
`$C`. However, there is no way to divide the number of cores in a way
which fairly subdivides the available cores when `$B` is greater than
1.
There has been exploration of passing a jobserver in to the sandbox,
or sharing a jobserver between all the builds. This is one option, but
relatively complicated and only supports make. Lots of other software
uses its own implementation of `-j` and doesn't support either `-l` or
the Make jobserver.
For the case of an interactive user machine, the user should limit
overall system load using `$B`, `$C`, and optionally systemd's
cpu/network/io limiting features.
Making this change should significantly improve the utilization of our
build farm, and improve the throughput of Hydra.
The asciidoctor-diagram extension vendors JARs and requires a JRE to
render ditaa and PlantUML diagrams.
Resolves:
$ asciidoctor --require asciidoctor-diagram - >/dev/null <<ADOC
[ditaa]
....
+---+ +---+
| A +-->| B |
+---+ +---+
....
ADOC
asciidoctor: ERROR: <stdin>: line 2: Failed to generate image: Could not find Java executable
depsBuildBuild for 'ar: command not found'
BUILDCC for failure during 'checking whether the C compiler works...'
there's still a issue but it maybe wont cause reverse dependency issues?
so overall its still an improvement
```
cd texk/texlive/tl_support && make run-texlinks
make[1]: Entering directory '/build/texlive-20210325-source/WorkDir/texk/texlive/tl_support'
if test -s /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/texlinks && test -s /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/share/texmf-dist/web2c/fmtutil.cnf; then \
echo "running texlinks in `pwd` via: /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/texlinks -v -f /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/share/texmf-dist/web2c/fmtutil.cnf -e "" /nix/stor>
PATH="/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin:/nix/store/146229z6ingpiz0g1yxs2jpq1h546ms3-gcc-wrapper-11.3.0/bin:/nix/store/sc50c3qvzlw6dmfbhxw96j9yrj47pv1v-gcc-11.3.0/bin:/nix/store/20navlfxxadg2qvkqh3vc55z27divi40-glibc-2.35-163>
else :; fi
running texlinks in /build/texlive-20210325-source/WorkDir/texk/texlive/tl_support via: /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/texlinks -v -f /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/s>
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpsewhich: line 1: syntax error: unexpected word (expecting ")")
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^?ELF^B^A^A^B<B7>^A<80>: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]A<B0>^B<B8>^B^A^B^Fp^]p^]Ap^]A^P^B^P^B^H^D^D<A8>^B<A8>^B@<A8>^B@: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]A: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 2: @@<E0><9A>@8: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 5: syntax error: unterminated quoted string
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin: no write permissions. Skipping...
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^?ELF^B^A^A^B<B7>^A<80>: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]A<B0>^B<B8>^B^A^B^Fp^]p^]Ap^]A^P^B^P^B^H^D^D<A8>^B<A8>^B@<A8>^B@: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 1: ^]A: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 2: @@<E0><9A>@8: not found
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: line 5: syntax error: unterminated quoted string
make[1]: Leaving directory '/build/texlive-20210325-source/WorkDir/texk/texlive/tl_support'
/nix/store/j00iibma3wqn88f8bi9x2c9dns3ww98v-texlinks/bin/texlinks: line 252: /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpsewhich: cannot execute binary file: Exec format error
/nix/store/j00iibma3wqn88f8bi9x2c9dns3ww98v-texlinks/bin/texlinks: line 298: /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: cannot execute binary file: Exec format error
/nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin: no write permissions. Skipping...
/nix/store/j00iibma3wqn88f8bi9x2c9dns3ww98v-texlinks/bin/texlinks: line 192: /nix/store/1hs29hn23cdh2qbyws95plk1fyhqyzpx-texlive-bin-aarch64-unknown-linux-gnu-2021/bin/kpseaccess: cannot execute binary file: Exec format error
@nix { "action": "setPhase", "phase": "fixupPhase" }
```
