colin/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1418c986b0
nixpkgs/nixos/doc/manual/default.nix

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

355 lines
12 KiB
Nix
Raw Normal View History

nixos/make-options-doc: treat missing descriptions as errors by default this partially solves the problem of "missing description" warnings of the options doc build being lost by nix build, at the cost of failing builds that previously ran. an option to disable this behaviour is provided.
2021-12-18 18:37:41 +00:00
{ pkgs
, options
, config
, version
, revision
, extraSources ? []
, baseOptionsJSON ? null
, warningsAreErrors ? true
nixos/manual: disallow docbook option docs it's been long in the making, and with 23.05 out we can finally disable docbook option docs and default to markdown instead. this brings a massive speed boost in manual and manpage builds, so much so that we may consider enabling user module documentation by default. we don't remove the docbook support code entirely yet because it's a lot all over, and probably better removed in multiple separate changes.
2023-06-11 17:29:32 +00:00
, allowDocBook ? false
nixos/make-options-doc: treat missing descriptions as errors by default this partially solves the problem of "missing description" warnings of the options doc build being lost by nix build, at the cost of failing builds that previously ran. an option to disable this behaviour is provided.
2021-12-18 18:37:41 +00:00
, prefix ? ../../..
}:
* Make the location of Nixpkgs a function argument (defaulting to ../../nixpkgs). This makes the "pkgs" symlink under the nixos checkout unnecessary. svn path=/nixos/trunk/; revision=11989
2008-06-05 15:33:17 +00:00
nixos/manual: disallow docbook option docs it's been long in the making, and with 23.05 out we can finally disable docbook option docs and default to markdown instead. this brings a massive speed boost in manual and manpage builds, so much so that we may consider enabling user module documentation by default. we don't remove the docbook support code entirely yet because it's a lot all over, and probably better removed in multiple separate changes.
2023-06-11 17:29:32 +00:00
assert ! allowDocBook;
Manual: Generate PDF Issue NixOS/nixos#237.
2014-08-09 14:30:44 +00:00
with pkgs;
Rename newOptionAttrSetToDocList -> optionAttrSetToDocList
2013-10-17 12:09:05 +00:00
strip trailing whitespace; no functional change svn path=/nixos/trunk/; revision=29285
2011-09-14 18:20:50 +00:00
let
nixos/testing: Improve option docs
2022-06-27 18:06:30 +00:00
inherit (lib) hasPrefix removePrefix;
add epub for NixOS manual (second try) (#17205)
2016-07-28 02:27:39 +00:00
lib = pkgs.lib;
* Generate valid DocBook. <emphasis> isn't allowed in a <filename>. * For manpage generation, work around a bug in the DocBook XSL stylesheet by using a <simplelist> instead of a <itemizedlist> (which is arguably better anyway). The indentation of an <itemizedlist> lasts beyond the end of that element under certain circumstances, it seems. * Disable the man.endnotes.are.numbered option to prevent endnotes to be generated for each xlink:href in the Declarations/Definitions lists (which we don't want anyway, I guess). This causes libxslt to get stuck in an infinite (?) loop. svn path=/nixos/trunk/; revision=17659
2009-10-05 23:15:06 +00:00
nixos/man: prevent duplication of options libxslt 1.1.35 fixed conflict resolution for templates to match the specification. This uncovered a bug in docbook-xsl (https://github.com/docbook/xslt10-stylesheets/issues/240), which causes option names to be duplicated into the option descriptions. Let’s resolve the conflict by patching the stylesheets. Fixes: https://github.com/NixOS/nixpkgs/issues/166304
2022-03-31 00:34:42 +00:00
docbook_xsl_ns = pkgs.docbook-xsl-ns.override {
withManOptDedupPatch = true;
};
nixos-render-docs: add options manpage converter mdoc is just too slow to render on groff, and semantic markup doesn't help us any for generated pages. this produces a lot of changes to configuration.nix.5, but only few rendering changes. most of those seem to be place losing a space where docbook emitted roff code that did not faithfully represent the input text, though a few places also gained space where docbook dropped them. notably we also don't need the compatibility code docbook-xsl emitted because that problem was fixed over a decade ago. this will handle block quotes, which the docbook stylesheets turned into a mess of roff requests that ended up showing up in the output instead of being processed.
2023-01-31 03:19:28 +00:00
manpageUrls = pkgs.path + "/doc/manpage-urls.json";
nixos manual: allow options from nix packages
2016-01-22 19:22:12 +00:00
# We need to strip references to /nix/store/* from options,
# including any `extraSources` if some modules came from elsewhere,
# or else the build will fail.
#
# E.g. if some `options` came from modules in ${pkgs.customModules}/nix,
nixos-manual: Simplify stripping prefixes Let's use a simple (unflipped) fold and break out the actual core stripPrefix function from stripAnyPrefixes (I personally love point-less^H^H^H^Hfree style but if I'd be anal I'd even go further and factor away the "fn:"). Also, let's use path as a better name for "fn" (filename), because that's what it is and also cannot be confused with "fn" meaning "function". We now toString all of the prefixes, so there shouldn't be any need to implicily toString the extraSources anymore. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-01-29 15:20:22 +00:00
# you'd need to include `extraSources = [ pkgs.customModules ]`
nixos/documentation: split options doc build most modules can be evaluated for their documentation in a very restricted environment that doesn't include all of nixpkgs. this evaluation can then be cached and reused for subsequent builds, merging only documentation that has changed into the cached set. since nixos ships with a large number of modules of which only a few are used in any given config this can save evaluation a huge percentage of nixos options available in any given config. in tests of this caching, despite having to copy most of nixos/, saves about 80% of the time needed to build the system manual, or about two second on the machine used for testing. build time for a full system config shrank from 9.4s to 7.4s, while turning documentation off entirely shortened the build to 7.1s.
2021-11-18 23:26:27 +00:00
prefixesToStrip = map (p: "${toString p}/") ([ prefix ] ++ extraSources);
treewide: fold -> foldr
2021-01-25 06:57:48 +00:00
stripAnyPrefixes = lib.flip (lib.foldr lib.removePrefix) prefixesToStrip;
Manual: Don't depend on the absolute path of modules This ensures that the manual has the same store path regardless of the directory in which Nixpkgs is stored, and thus can be fetched from the channel.
2013-10-23 17:32:19 +00:00
Extract NixOS options documentation generation to a function Motivation is to support other repositories containing nixos modules that would like to generate options documentation: - nix-darwin - private repos - arion - ??
2019-08-08 10:59:25 +00:00
optionsDoc = buildPackages.nixosOptionsDoc {
lib/options: deprecate docbook text and literalDocBook deprecate literalDocBook by adding a warning (that will not fire yet) to its uses and other docbook literal strings by adding optional warning message to mergeJSON.
2022-09-01 17:23:59 +00:00
inherit options revision baseOptionsJSON warningsAreErrors allowDocBook;
Extract NixOS options documentation generation to a function Motivation is to support other repositories containing nixos modules that would like to generate options documentation: - nix-darwin - private repos - arion - ??
2019-08-08 10:59:25 +00:00
transformOptions = opt: opt // {
# Clean up declaration sites to not refer to the NixOS source tree.
declarations = map stripAnyPrefixes opt.declarations;
};
};
Add "defined by" and "declared by" for each option. These are list of files which link to either the remote source or the local file location. svn path=/nixos/trunk/; revision=17255
2009-09-18 15:10:37 +00:00
nixos/doc: Wire up the test options reference
2022-06-27 15:52:08 +00:00
nixos-lib = import ../../lib { };
testOptionsDoc = let
eval = nixos-lib.evalTest {
# Avoid evaluating a NixOS config prototype.
config.node.type = lib.types.deferredModule;
options._module.args = lib.mkOption { internal = true; };
};
in buildPackages.nixosOptionsDoc {
inherit (eval) options;
treewide: cleanup some unused bindings
2022-12-18 22:22:17 +00:00
inherit revision;
nixos/doc: Wire up the test options reference
2022-06-27 15:52:08 +00:00
transformOptions = opt: opt // {
# Clean up declaration sites to not refer to the NixOS source tree.
declarations =
map
(decl:
if hasPrefix (toString ../../..) (toString decl)
then
let subpath = removePrefix "/" (removePrefix (toString ../../..) (toString decl));
in { url = "https://github.com/NixOS/nixpkgs/blob/master/${subpath}"; name = subpath; }
else decl)
opt.declarations;
};
documentType = "none";
variablelistId = "test-options-list";
nixos/doc: Disambiguate test option ids Changing this later on would break external links into the manual.
2022-09-29 10:41:59 +00:00
optionIdPrefix = "test-opt-";
nixos/doc: Wire up the test options reference
2022-06-27 15:52:08 +00:00
};
Manual: Reduce chunking Fixes #7607.
2015-07-22 14:09:00 +00:00
toc = builtins.toFile "toc.xml"
''
<toc role="chunk-toc">
<d:tocentry xmlns:d="http://docbook.org/ns/docbook" linkend="book-nixos-manual"><?dbhtml filename="index.html"?>
Move release notes to an appendix and a separate chunk
2015-07-22 14:17:06 +00:00
<d:tocentry linkend="ch-options"><?dbhtml filename="options.html"?></d:tocentry>
<d:tocentry linkend="ch-release-notes"><?dbhtml filename="release-notes.html"?></d:tocentry>
Manual: Reduce chunking Fixes #7607.
2015-07-22 14:09:00 +00:00
</d:tocentry>
</toc>
'';
nixos/doc: Allow refs from options to the manual My first attempt to do this was to just use a conditional <refsection/> in order to not create exact references in the manpage but create the reference in the HTML manual, as suggested by @edolstra on IRC. Later I went on to use <olink/> to reference sections of the manual, but in order to do that, we need to overhaul how we generate the manual and manpages. So, that's where we are now: There is a new derivation called "manual-olinkdb", which is the olinkdb for the HTML manual, which in turn creates the olinkdb.xml file and the manual.db. The former contains the targetdoc references and the latter the specific targetptr elements. The reason why I included the olinkdb.xml verbatim is that first of all the DTD is dependent on the Docbook XSL sources and the references within the olinkdb.xml entities are relative to the current directory. So using a store path for that would end up searching for the manual.db directly in /nix/store/manual.db. Unfortunately, the <olinks/> that end up in the output file are relative, so for example if you're clicking on one of these within the PDF, the URL is searched in the current directory. However, the sections from the olink's text are still valid, so we could use an alternative URL for that in the future. The manual doesn't contain any links, so even referencing the relative URL shouldn't do any harm. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Cc: @edolstra
2016-04-11 16:29:00 +00:00
manualXsltprocOptions = toString [
doc: Remove all section numbers Simplicity. We use anchors nowadays. Nobody should be printing this.
2022-12-12 12:04:39 +00:00
"--param chapter.autolabel 0"
"--param part.autolabel 0"
"--param preface.autolabel 0"
"--param reference.autolabel 0"
"--param section.autolabel 0"
nixos docs: syntax highlight - Rectifies diverging CSS by combining nixos/nixpkgs docs CSS - Moves our custom Highlight.js loader in to the hljs package - Switches the nixos docs to use SVG callouts too
2018-04-05 11:54:01 +00:00
"--stringparam html.stylesheet 'style.css overrides.css highlightjs/mono-blue.css'"
"--stringparam html.script './highlightjs/highlight.pack.js ./highlightjs/loader.js'"
doc: Remove all section numbers Simplicity. We use anchors nowadays. Nobody should be printing this.
2022-12-12 12:04:39 +00:00
"--param xref.with.number.and.title 0"
nixos manual: reduce toc depth to reduce the amount of information presented.
2019-10-28 10:46:10 +00:00
"--param toc.section.depth 0"
nixos/manual: make reproducible (#102234)
2020-10-31 20:18:16 +00:00
"--param generate.consistent.ids 1"
nixos/doc: Allow refs from options to the manual My first attempt to do this was to just use a conditional <refsection/> in order to not create exact references in the manpage but create the reference in the HTML manual, as suggested by @edolstra on IRC. Later I went on to use <olink/> to reference sections of the manual, but in order to do that, we need to overhaul how we generate the manual and manpages. So, that's where we are now: There is a new derivation called "manual-olinkdb", which is the olinkdb for the HTML manual, which in turn creates the olinkdb.xml file and the manual.db. The former contains the targetdoc references and the latter the specific targetptr elements. The reason why I included the olinkdb.xml verbatim is that first of all the DTD is dependent on the Docbook XSL sources and the references within the olinkdb.xml entities are relative to the current directory. So using a store path for that would end up searching for the manual.db directly in /nix/store/manual.db. Unfortunately, the <olinks/> that end up in the output file are relative, so for example if you're clicking on one of these within the PDF, the URL is searched in the current directory. However, the sections from the olink's text are still valid, so we could use an alternative URL for that in the future. The manual doesn't contain any links, so even referencing the relative URL shouldn't do any harm. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Cc: @edolstra
2016-04-11 16:29:00 +00:00
"--stringparam admon.style ''"
nixos docs: syntax highlight - Rectifies diverging CSS by combining nixos/nixpkgs docs CSS - Moves our custom Highlight.js loader in to the hljs package - Switches the nixos docs to use SVG callouts too
2018-04-05 11:54:01 +00:00
"--stringparam callout.graphics.extension .svg"
nixos/doc: Allow refs from options to the manual My first attempt to do this was to just use a conditional <refsection/> in order to not create exact references in the manpage but create the reference in the HTML manual, as suggested by @edolstra on IRC. Later I went on to use <olink/> to reference sections of the manual, but in order to do that, we need to overhaul how we generate the manual and manpages. So, that's where we are now: There is a new derivation called "manual-olinkdb", which is the olinkdb for the HTML manual, which in turn creates the olinkdb.xml file and the manual.db. The former contains the targetdoc references and the latter the specific targetptr elements. The reason why I included the olinkdb.xml verbatim is that first of all the DTD is dependent on the Docbook XSL sources and the references within the olinkdb.xml entities are relative to the current directory. So using a store path for that would end up searching for the manual.db directly in /nix/store/manual.db. Unfortunately, the <olinks/> that end up in the output file are relative, so for example if you're clicking on one of these within the PDF, the URL is searched in the current directory. However, the sections from the olink's text are still valid, so we could use an alternative URL for that in the future. The manual doesn't contain any links, so even referencing the relative URL shouldn't do any harm. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Cc: @edolstra
2016-04-11 16:29:00 +00:00
"--stringparam current.docid manual"
"--param chunk.section.depth 0"
"--param chunk.first.sections 1"
"--param use.id.as.filename 1"
"--stringparam chunk.toc ${toc}"
];
nixos/manual: split manpages-combined from manual-combined once we generate the entire manual-combined.xml with a single nixos-render-docs invocation we will no longer need any options xml files. likewise we do not need the test options xml in the manpage build. splitting manpages-combined from manual-combined also allows these two to run in parallel, slightly improving build times.
2023-02-12 08:04:07 +00:00
linterFunctions = ''
# outputs the context of an xmllint error output
# LEN lines around the failing line are printed
function context {
# length of context
local LEN=6
# lines to print before error line
local BEFORE=4
# xmllint output lines are:
# file.xml:1234: there was an error on line 1234
while IFS=':' read -r file line rest; do
echo
if [[ -n "$rest" ]]; then
echo "$file:$line:$rest"
local FROM=$(($line>$BEFORE ? $line - $BEFORE : 1))
# number lines & filter context
nl --body-numbering=a "$file" | sed -n "$FROM,+$LEN p"
else
if [[ -n "$line" ]]; then
echo "$file:$line"
else
echo "$file"
fi
fi
done
}
function lintrng {
xmllint --debug --noout --nonet \
--relaxng ${docbook5}/xml/rng/docbook/docbook.rng \
"$1" \
2>&1 | context 1>&2
# ^ redirect assumes xmllint doesn’t print to stdout
}
'';
nixos/manual: render html with nixos-render-doc if !allowDocBook this reproduces the docbook-generated html manual exactly enough to appease the compare workflows while we still support both toolchains. it's also a lot faster than the docbook toolchain, rendering the entire html manual in about two seconds on this machine (while docbook needs about 20).
2023-02-20 17:55:15 +00:00
prepareManualFromMD = ''
cp -r --no-preserve=all $inputs/* .
substituteInPlace ./manual.md \
--replace '@NIXOS_VERSION@' "${version}"
substituteInPlace ./configuration/configuration.md \
--replace \
'@MODULE_CHAPTERS@' \
${lib.escapeShellArg (lib.concatMapStringsSep "\n" (p: "${p.value}") config.meta.doc)}
substituteInPlace ./nixos-options.md \
--replace \
'@NIXOS_OPTIONS_JSON@' \
${optionsDoc.optionsJSON}/share/doc/nixos/options.json
substituteInPlace ./development/writing-nixos-tests.section.md \
--replace \
'@NIXOS_TEST_OPTIONS_JSON@' \
${testOptionsDoc.optionsJSON}/share/doc/nixos/options.json
'';
nixos manual: combine XML and validate separately
2017-06-28 17:02:33 +00:00
manual-combined = runCommand "nixos-manual-combined"
nixos/manual: clean up default.nix a bit - inline copySources into single user - remove `inherit sources` where it's not necessary - inline generatedSources. this will go away completely soon so we may as well. - inline modulesDoc into manual-combined. this too will go away soon. - inline sources into manual-combined. this too will go away soon.
2023-02-12 08:40:07 +00:00
{ inputs = lib.sourceFilesBySuffices ./. [ ".xml" ".md" ];
nativeBuildInputs = [ pkgs.nixos-render-docs pkgs.libxml2.bin pkgs.libxslt.bin ];
nixos manual: combine XML and validate separately
2017-06-28 17:02:33 +00:00
meta.description = "The NixOS manual as plain docbook XML";
NixOS: Use runCommand instead of mkDerivation in a few places
2016-09-27 13:26:37 +00:00
}
''
nixos/manual: render html with nixos-render-doc if !allowDocBook this reproduces the docbook-generated html manual exactly enough to appease the compare workflows while we still support both toolchains. it's also a lot faster than the docbook toolchain, rendering the entire html manual in about two seconds on this machine (while docbook needs about 20).
2023-02-20 17:55:15 +00:00
${prepareManualFromMD}
nixos-render-docs: add structural includes, use for manual this adds support for structural includes to nixos-render-docs. structural includes provide a way to denote the (sub)structure of the nixos manual in the markdown source files, very similar to how we used literal docbook blocks before, and are processed by nixos-render-docs without involvement of xml tooling. this will ultimately allow us to emit the nixos manual in other formats as well, e.g. html, without going through docbook at all. alternatives to this source layout were also considered: a parallel structure using e.g. toml files that describe the document tree and links to each part is possible, but much more complicated to implement than the solution chosen here and makes it harder to follow which files have what substructure. it also makes it much harder to include a substructure in the middle of a file. much the same goes for command-line arguments to the converter, only that command-lined arguments are even harder to specify correctly and cannot be reasonably pulled together from many places without involving another layer of tooling. cli arguments would also mean that the manual structure would be fixed in default.nix, which is also not ideal.
2023-02-11 07:48:06 +00:00
nixos-render-docs: use multiprocessing for options options processing is pretty slow right now, mostly because the markdown-it-py parser is pure python (and with performance pessimizations at that). options parsing *is* embarassingly parallel though, so we can just fork out all the work to worker processes and collect the results. multiprocessing probably has a greater benefit on linux than on darwin since the worker spawning method darwin uses is less efficient than fork() on linux. this hasn't been tested on darwin, only on linux, but if anything darwin will be faster with its preferred method.
2023-02-13 12:43:32 +00:00
nixos-render-docs -j $NIX_BUILD_CORES manual docbook \
nixos/manual: clean up default.nix a bit - inline copySources into single user - remove `inherit sources` where it's not necessary - inline generatedSources. this will go away completely soon so we may as well. - inline modulesDoc into manual-combined. this too will go away soon. - inline sources into manual-combined. this too will go away soon.
2023-02-12 08:40:07 +00:00
--manpage-urls ${manpageUrls} \
nixos-render-docs: add structural includes, use for manual this adds support for structural includes to nixos-render-docs. structural includes provide a way to denote the (sub)structure of the nixos manual in the markdown source files, very similar to how we used literal docbook blocks before, and are processed by nixos-render-docs without involvement of xml tooling. this will ultimately allow us to emit the nixos manual in other formats as well, e.g. html, without going through docbook at all. alternatives to this source layout were also considered: a parallel structure using e.g. toml files that describe the document tree and links to each part is possible, but much more complicated to implement than the solution chosen here and makes it harder to follow which files have what substructure. it also makes it much harder to include a substructure in the middle of a file. much the same goes for command-line arguments to the converter, only that command-lined arguments are even harder to specify correctly and cannot be reasonably pulled together from many places without involving another layer of tooling. cli arguments would also mean that the manual structure would be fixed in default.nix, which is also not ideal.
2023-02-11 07:48:06 +00:00
--revision ${lib.escapeShellArg revision} \
./manual.md \
nixos/manual: apply options preprocessing to full manual the conversion to the markdown-based workflow missed that generating the manual as docbook also generates the option docs with nixos-specific wrapper elements that require postprocessing before the document is real, valid docbook. restore this processing to the full manual. it's not the prettiest thing, done like this, but we only need it for one release so it doesn't have to be.
2023-03-13 16:58:38 +00:00
./manual-combined-pre.xml
nixos/manual: fix cross-compilation Remove absolute reference to xsltproc. This reference breaks cross-compilation: instead of host version builder tries to execute binary for target and fails.
2023-04-04 21:18:16 +00:00
xsltproc \
nixos/manual: apply options preprocessing to full manual the conversion to the markdown-based workflow missed that generating the manual as docbook also generates the option docs with nixos-specific wrapper elements that require postprocessing before the document is real, valid docbook. restore this processing to the full manual. it's not the prettiest thing, done like this, but we only need it for one release so it doesn't have to be.
2023-03-13 16:58:38 +00:00
-o manual-combined.xml ${./../../lib/make-options-doc/postprocess-option-descriptions.xsl} \
manual-combined-pre.xml
nixos/doc/manual: print context on failing xmllint validation Previously only the line numbers of a giant, internally generated XML file were printed, without any kind of debuggability. Now at least the mentioned lines are printed with a little bit of surrounding context (to have something to grep for). ``` manual-combined.xml:4863: element para: Relax-NG validity error : Did not expect element para there 4859 <chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xi="http://www.w3.org/2001/XInclude" version="5.0" xml:id="sec-writing-modules"> 4860 4861 <title>Writing NixOS Modules</title> 4862 4863 <para>NixOS has a modular system for declarative configuration. This 4864 system combines multiple <emphasis>modules</emphasis> to produce the 4865 full system configuration. One of the modules that constitute the ```
2017-11-18 23:44:17 +00:00
nixos/manual: split manpages-combined from manual-combined once we generate the entire manual-combined.xml with a single nixos-render-docs invocation we will no longer need any options xml files. likewise we do not need the test options xml in the manpage build. splitting manpages-combined from manual-combined also allows these two to run in parallel, slightly improving build times.
2023-02-12 08:04:07 +00:00
${linterFunctions}
nixos/doc/manual: print context on failing xmllint validation Previously only the line numbers of a giant, internally generated XML file were printed, without any kind of debuggability. Now at least the mentioned lines are printed with a little bit of surrounding context (to have something to grep for). ``` manual-combined.xml:4863: element para: Relax-NG validity error : Did not expect element para there 4859 <chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xi="http://www.w3.org/2001/XInclude" version="5.0" xml:id="sec-writing-modules"> 4860 4861 <title>Writing NixOS Modules</title> 4862 4863 <para>NixOS has a modular system for declarative configuration. This 4864 system combines multiple <emphasis>modules</emphasis> to produce the 4865 full system configuration. One of the modules that constitute the ```
2017-11-18 23:44:17 +00:00
nixos manual: combine XML and validate separately
2017-06-28 17:02:33 +00:00
mkdir $out
cp manual-combined.xml $out/
nixos/doc: Make nixos-manual-combined fail easy to inspect By writing the unchecked outputs before checking them, they will be written to a store path, which appears in the log, and which sticks around even if the build fails. Eventually it is GCed, but until then, you can open the file. If you run it in a terminal+editor combination like VSCode, the failure location is just one Ctrl+click away.
2022-06-27 15:18:02 +00:00
lintrng $out/manual-combined.xml
nixos/manual: split manpages-combined from manual-combined once we generate the entire manual-combined.xml with a single nixos-render-docs invocation we will no longer need any options xml files. likewise we do not need the test options xml in the manpage build. splitting manpages-combined from manual-combined also allows these two to run in parallel, slightly improving build times.
2023-02-12 08:04:07 +00:00
'';
manpages-combined = runCommand "nixos-manpages-combined.xml"
{ nativeBuildInputs = [ buildPackages.libxml2.bin buildPackages.libxslt.bin ];
meta.description = "The NixOS manpages as plain docbook XML";
}
''
mkdir generated
cp -prd ${./man-pages.xml} man-pages.xml
ln -s ${optionsDoc.optionsDocBook} generated/options-db.xml
xmllint --xinclude --noxincludenode --output $out ./man-pages.xml
${linterFunctions}
lintrng $out
nixos manual: combine XML and validate separately
2017-06-28 17:02:33 +00:00
'';
* Separate building of the NixOS manual and manpages. Usually we only need the manpages, and the manual takes much longer to build. svn path=/nixos/trunk/; revision=23112
2010-08-11 12:28:53 +00:00
in rec {
nixos/documentation: deprecate docbook option docs following the plan in https://github.com/NixOS/nixpkgs/pull/189318#discussion_r961764451 also adds an activation script to print the warning during activation instead of during build, otherwise folks using the new CLI that hides build logs by default might never see the warning.
2023-01-15 13:56:46 +00:00
inherit (optionsDoc) optionsJSON optionsNix optionsDocBook optionsUsedDocbook;
Manual: Don't depend on the JSON options
2014-11-17 12:41:18 +00:00
* Separate building of the NixOS manual and manpages. Usually we only need the manpages, and the manual takes much longer to build. svn path=/nixos/trunk/; revision=23112
2010-08-11 12:28:53 +00:00
# Generate the NixOS manual.
nixos: doc: rename `manual` to `manualHTML`, cleanup references Because when I see "config.system.build.manual.manual" after I forgot what it means I ask "Why do I need that second `.manual` there again?". Doesn't happen with `config.system.build.manual.manualHTML`.
2018-09-06 19:17:44 +00:00
manualHTML = runCommand "nixos-manual-html"
nixos/manual: render html with nixos-render-doc if !allowDocBook this reproduces the docbook-generated html manual exactly enough to appease the compare workflows while we still support both toolchains. it's also a lot faster than the docbook toolchain, rendering the entire html manual in about two seconds on this machine (while docbook needs about 20).
2023-02-20 17:55:15 +00:00
{ nativeBuildInputs =
if allowDocBook then [
buildPackages.libxml2.bin
buildPackages.libxslt.bin
] else [
buildPackages.nixos-render-docs
];
inputs = lib.optionals (! allowDocBook) (lib.sourceFilesBySuffices ./. [ ".md" ]);
NixOS: Use runCommand instead of mkDerivation in a few places
2016-09-27 13:26:37 +00:00
meta.description = "The NixOS manual in HTML format";
allowedReferences = ["out"];
}
''
* Validate the manual against the DocBook 5 RelaxNG schema. svn path=/nixos/trunk/; revision=17643
2009-10-05 13:17:45 +00:00
# Generate the HTML manual.
* Generate a manual page for configuration.nix containing all the options. svn path=/nixos/branches/modular-nixos/; revision=16358
2009-07-14 15:47:03 +00:00
dst=$out/share/doc/nixos
Another attempt to eradicate ensureDir See c556a6ea46e71e1907d78b71fab36df30297b3ad.
2014-06-30 12:56:10 +00:00
mkdir -p $dst
Update the troubleshooting section for systemd
2013-01-09 12:43:57 +00:00
nixos docs: syntax highlight - Rectifies diverging CSS by combining nixos/nixpkgs docs CSS - Moves our custom Highlight.js loader in to the hljs package - Switches the nixos docs to use SVG callouts too
2018-04-05 11:54:01 +00:00
cp ${../../../doc/style.css} $dst/style.css
cp ${../../../doc/overrides.css} $dst/overrides.css
cp -r ${pkgs.documentation-highlighter} $dst/highlightjs
strip trailing whitespace; no functional change svn path=/nixos/trunk/; revision=29285
2011-09-14 18:20:50 +00:00
nixos/manual: render html with nixos-render-doc if !allowDocBook this reproduces the docbook-generated html manual exactly enough to appease the compare workflows while we still support both toolchains. it's also a lot faster than the docbook toolchain, rendering the entire html manual in about two seconds on this machine (while docbook needs about 20).
2023-02-20 17:55:15 +00:00
${if allowDocBook then ''
xsltproc \
${manualXsltprocOptions} \
--stringparam id.warnings "1" \
--nonet --output $dst/ \
${docbook_xsl_ns}/xml/xsl/docbook/xhtml/chunktoc.xsl \
${manual-combined}/manual-combined.xml \
|& tee xsltproc.out
grep "^ID recommended on" xsltproc.out &>/dev/null && echo "error: some IDs are missing" && false
rm xsltproc.out
mkdir -p $dst/images/callouts
cp ${docbook_xsl_ns}/xml/xsl/docbook/images/callouts/*.svg $dst/images/callouts/
'' else ''
${prepareManualFromMD}
# TODO generator is set like this because the docbook/md manual compare workflow will
# trigger if it's different
nixos-render-docs -j $NIX_BUILD_CORES manual html \
--manpage-urls ${manpageUrls} \
--revision ${lib.escapeShellArg revision} \
--generator "DocBook XSL Stylesheets V${docbook_xsl_ns.version}" \
--stylesheet style.css \
--stylesheet overrides.css \
--stylesheet highlightjs/mono-blue.css \
--script ./highlightjs/highlight.pack.js \
--script ./highlightjs/loader.js \
--toc-depth 1 \
--chunk-toc-depth 1 \
./manual.md \
$dst/index.html
''}
Put the manual in the NixOS channel
2013-10-24 18:06:02 +00:00
mkdir -p $out/nix-support
echo "nix-build out $out" >> $out/nix-support/hydra-build-products
Manual: Fix build product
2014-09-10 13:03:49 +00:00
echo "doc manual $dst" >> $out/nix-support/hydra-build-products
Manual: prevent a runtime dependency on DocBook XSL
2012-07-25 15:54:24 +00:00
''; # */
Put the manual in the NixOS channel
2013-10-24 18:06:02 +00:00
nixos: doc: rename `manual` to `manualHTML`, cleanup references Because when I see "config.system.build.manual.manual" after I forgot what it means I ask "Why do I need that second `.manual` there again?". Doesn't happen with `config.system.build.manual.manualHTML`.
2018-09-06 19:17:44 +00:00
# Alias for backward compatibility. TODO(@oxij): remove eventually.
manual = manualHTML;
# Index page of the NixOS manual.
manualHTMLIndex = "${manualHTML}/share/doc/nixos/index.html";
* Separate building of the NixOS manual and manpages. Usually we only need the manpages, and the manual takes much longer to build. svn path=/nixos/trunk/; revision=23112
2010-08-11 12:28:53 +00:00
NixOS: Use runCommand instead of mkDerivation in a few places
2016-09-27 13:26:37 +00:00
manualEpub = runCommand "nixos-manual-epub"
nixos/manual: clean up default.nix a bit - inline copySources into single user - remove `inherit sources` where it's not necessary - inline generatedSources. this will go away completely soon so we may as well. - inline modulesDoc into manual-combined. this too will go away soon. - inline sources into manual-combined. this too will go away soon.
2023-02-12 08:40:07 +00:00
{ nativeBuildInputs = [ buildPackages.libxml2.bin buildPackages.libxslt.bin buildPackages.zip ];
nixos/doc: stub out epub manual epub manuals are holding back the transition away from docbook, and cursory research does not suggest that they are used very much. it's still very early in the 23.11 release cycle, so if we're going to find out just how many people do use the epub manuals it should be now. this need not be the end of epub manuals. nixos-render-docs could be extended to also export epubs, but that has not been done yet since it's going to be some effort with unknown real-world usefulness.
2023-06-11 17:26:30 +00:00
doc = ''
<book xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="book-nixos-manual">
<info>
<title>NixOS Manual</title>
<subtitle>Version ${lib.version}</subtitle>
</info>
<chapter>
<title>Temporarily unavailable</title>
<para>
The NixOS manual is currently not available in EPUB format,
please use the <link xlink:href="https://nixos.org/nixos/manual">HTML manual</link>
instead.
</para>
<para>
If you've used the EPUB manual in the past and it has been useful to you, please
<link xlink:href="https://github.com/NixOS/nixpkgs/issues/237234">let us know</link>.
</para>
</chapter>
</book>
'';
passAsFile = [ "doc" ];
NixOS: Use runCommand instead of mkDerivation in a few places
2016-09-27 13:26:37 +00:00
}
''
add epub for NixOS manual (second try) (#17205)
2016-07-28 02:27:39 +00:00
# Generate the epub manual.
dst=$out/share/doc/nixos
xsltproc \
nixos/doc: stub out epub manual epub manuals are holding back the transition away from docbook, and cursory research does not suggest that they are used very much. it's still very early in the 23.11 release cycle, so if we're going to find out just how many people do use the epub manuals it should be now. this need not be the end of epub manuals. nixos-render-docs could be extended to also export epubs, but that has not been done yet since it's going to be some effort with unknown real-world usefulness.
2023-06-11 17:26:30 +00:00
--param chapter.autolabel 0 \
add epub for NixOS manual (second try) (#17205)
2016-07-28 02:27:39 +00:00
--nonet --xinclude --output $dst/epub/ \
treewide: remove aliases in nixpkgs This makes the command ‘nix-env -qa -f. --arg config '{skipAliases = true;}'’ work in Nixpkgs. Misc... - qtikz: use libsForQt5.callPackage This ensures we get the right poppler. - rewrites: docbook5_xsl -> docbook_xsl_ns docbook_xml_xslt -> docbook_xsl diffpdf: fixup
2018-07-17 20:11:16 +00:00
${docbook_xsl_ns}/xml/xsl/docbook/epub/docbook.xsl \
nixos/doc: stub out epub manual epub manuals are holding back the transition away from docbook, and cursory research does not suggest that they are used very much. it's still very early in the 23.11 release cycle, so if we're going to find out just how many people do use the epub manuals it should be now. this need not be the end of epub manuals. nixos-render-docs could be extended to also export epubs, but that has not been done yet since it's going to be some effort with unknown real-world usefulness.
2023-06-11 17:26:30 +00:00
$docPath
add epub for NixOS manual (second try) (#17205)
2016-07-28 02:27:39 +00:00
echo "application/epub+zip" > mimetype
Fix epub generation * Hydra doesn't like spaces in filenames. * The zip file contained nix/store/.../OEBPS rather than OEBPS at top-level, causing some programs (like okular) to barf. * Remove the redundant $dst/epub directory.
2016-08-01 09:02:41 +00:00
manual="$dst/nixos-manual.epub"
zip -0Xq "$manual" mimetype
cd $dst/epub && zip -Xr9D "$manual" *
rm -rf $dst/epub
add epub for NixOS manual (second try) (#17205)
2016-07-28 02:27:39 +00:00
mkdir -p $out/nix-support
Fix epub generation * Hydra doesn't like spaces in filenames. * The zip file contained nix/store/.../OEBPS rather than OEBPS at top-level, causing some programs (like okular) to barf. * Remove the redundant $dst/epub directory.
2016-08-01 09:02:41 +00:00
echo "doc-epub manual $manual" >> $out/nix-support/hydra-build-products
add epub for NixOS manual (second try) (#17205)
2016-07-28 02:27:39 +00:00
'';
* Separate building of the NixOS manual and manpages. Usually we only need the manpages, and the manual takes much longer to build. svn path=/nixos/trunk/; revision=23112
2010-08-11 12:28:53 +00:00
NixOS: Use runCommand instead of mkDerivation in a few places
2016-09-27 13:26:37 +00:00
# Generate the NixOS manpages.
manpages = runCommand "nixos-manpages"
nixos/manual: remove sources input from manpages drv it's not used. holdover from when manpages were written in docbook.
2023-02-07 13:17:43 +00:00
{ nativeBuildInputs = [
nixos-render-docs: add options manpage converter mdoc is just too slow to render on groff, and semantic markup doesn't help us any for generated pages. this produces a lot of changes to configuration.nix.5, but only few rendering changes. most of those seem to be place losing a space where docbook emitted roff code that did not faithfully represent the input text, though a few places also gained space where docbook dropped them. notably we also don't need the compatibility code docbook-xsl emitted because that problem was fixed over a decade ago. this will handle block quotes, which the docbook stylesheets turned into a mess of roff requests that ended up showing up in the output instead of being processed.
2023-01-31 03:19:28 +00:00
buildPackages.installShellFiles
] ++ lib.optionals allowDocBook [
nixos/manual: translate manpages to mdoc since we want to move away from docbook and having docbook manpages around is going to block further progress we have to translate the current nixos manpages from docbook it *something* else. mdoc seems the most appropriate choice since markdown can't represent all the things manpages can differentiate with even more extensions. if we ever need html manpages we can render them using troff, like many of the online manpage viewers, or rewrite them again. since we haven't had html manpages for any of these in many years that seems unlikely to happen. unlike most of the recent markdown conversions this comes with a lot of minor rendering changes, mostly in spacing. docbook-xslt creates manual pages in a very different dialect than mdoc (which is used here).
2023-01-29 00:07:26 +00:00
buildPackages.libxml2.bin
buildPackages.libxslt.bin
nixos-render-docs: add options manpage converter mdoc is just too slow to render on groff, and semantic markup doesn't help us any for generated pages. this produces a lot of changes to configuration.nix.5, but only few rendering changes. most of those seem to be place losing a space where docbook emitted roff code that did not faithfully represent the input text, though a few places also gained space where docbook dropped them. notably we also don't need the compatibility code docbook-xsl emitted because that problem was fixed over a decade ago. this will handle block quotes, which the docbook stylesheets turned into a mess of roff requests that ended up showing up in the output instead of being processed.
2023-01-31 03:19:28 +00:00
] ++ lib.optionals (! allowDocBook) [
buildPackages.nixos-render-docs
nixos/manual: translate manpages to mdoc since we want to move away from docbook and having docbook manpages around is going to block further progress we have to translate the current nixos manpages from docbook it *something* else. mdoc seems the most appropriate choice since markdown can't represent all the things manpages can differentiate with even more extensions. if we ever need html manpages we can render them using troff, like many of the online manpage viewers, or rewrite them again. since we haven't had html manpages for any of these in many years that seems unlikely to happen. unlike most of the recent markdown conversions this comes with a lot of minor rendering changes, mostly in spacing. docbook-xslt creates manual pages in a very different dialect than mdoc (which is used here).
2023-01-29 00:07:26 +00:00
];
NixOS: Use runCommand instead of mkDerivation in a few places
2016-09-27 13:26:37 +00:00
allowedReferences = ["out"];
}
''
* Validate the manual against the DocBook 5 RelaxNG schema. svn path=/nixos/trunk/; revision=17643
2009-10-05 13:17:45 +00:00
# Generate manpages.
nixos/manual: translate manpages to mdoc since we want to move away from docbook and having docbook manpages around is going to block further progress we have to translate the current nixos manpages from docbook it *something* else. mdoc seems the most appropriate choice since markdown can't represent all the things manpages can differentiate with even more extensions. if we ever need html manpages we can render them using troff, like many of the online manpage viewers, or rewrite them again. since we haven't had html manpages for any of these in many years that seems unlikely to happen. unlike most of the recent markdown conversions this comes with a lot of minor rendering changes, mostly in spacing. docbook-xslt creates manual pages in a very different dialect than mdoc (which is used here).
2023-01-29 00:07:26 +00:00
mkdir -p $out/share/man/man8
installManPage ${./manpages}/*
nixos-render-docs: add options manpage converter mdoc is just too slow to render on groff, and semantic markup doesn't help us any for generated pages. this produces a lot of changes to configuration.nix.5, but only few rendering changes. most of those seem to be place losing a space where docbook emitted roff code that did not faithfully represent the input text, though a few places also gained space where docbook dropped them. notably we also don't need the compatibility code docbook-xsl emitted because that problem was fixed over a decade ago. this will handle block quotes, which the docbook stylesheets turned into a mess of roff requests that ended up showing up in the output instead of being processed.
2023-01-31 03:19:28 +00:00
${if allowDocBook
then ''
xsltproc --nonet \
--maxdepth 6000 \
--param man.output.in.separate.dir 1 \
--param man.output.base.dir "'$out/share/man/'" \
--param man.endnotes.are.numbered 0 \
--param man.break.after.slash 1 \
${docbook_xsl_ns}/xml/xsl/docbook/manpages/docbook.xsl \
nixos/manual: split manpages-combined from manual-combined once we generate the entire manual-combined.xml with a single nixos-render-docs invocation we will no longer need any options xml files. likewise we do not need the test options xml in the manpage build. splitting manpages-combined from manual-combined also allows these two to run in parallel, slightly improving build times.
2023-02-12 08:04:07 +00:00
${manpages-combined}
nixos-render-docs: add options manpage converter mdoc is just too slow to render on groff, and semantic markup doesn't help us any for generated pages. this produces a lot of changes to configuration.nix.5, but only few rendering changes. most of those seem to be place losing a space where docbook emitted roff code that did not faithfully represent the input text, though a few places also gained space where docbook dropped them. notably we also don't need the compatibility code docbook-xsl emitted because that problem was fixed over a decade ago. this will handle block quotes, which the docbook stylesheets turned into a mess of roff requests that ended up showing up in the output instead of being processed.
2023-01-31 03:19:28 +00:00
''
else ''
mkdir -p $out/share/man/man5
nixos-render-docs: use multiprocessing for options options processing is pretty slow right now, mostly because the markdown-it-py parser is pure python (and with performance pessimizations at that). options parsing *is* embarassingly parallel though, so we can just fork out all the work to worker processes and collect the results. multiprocessing probably has a greater benefit on linux than on darwin since the worker spawning method darwin uses is less efficient than fork() on linux. this hasn't been tested on darwin, only on linux, but if anything darwin will be faster with its preferred method.
2023-02-13 12:43:32 +00:00
nixos-render-docs -j $NIX_BUILD_CORES options manpage \
nixos-render-docs: add options manpage converter mdoc is just too slow to render on groff, and semantic markup doesn't help us any for generated pages. this produces a lot of changes to configuration.nix.5, but only few rendering changes. most of those seem to be place losing a space where docbook emitted roff code that did not faithfully represent the input text, though a few places also gained space where docbook dropped them. notably we also don't need the compatibility code docbook-xsl emitted because that problem was fixed over a decade ago. this will handle block quotes, which the docbook stylesheets turned into a mess of roff requests that ended up showing up in the output instead of being processed.
2023-01-31 03:19:28 +00:00
--revision ${lib.escapeShellArg revision} \
${optionsJSON}/share/doc/nixos/options.json \
$out/share/man/man5/configuration.nix.5
''}
* Build the Nix manual in a Nix expression instead of a Makefile so that it becomes easier to include it on the CD. svn path=/nixos/trunk/; revision=10070
2008-01-04 14:24:42 +00:00
'';
Prevent future store path references in the manual
2015-09-24 09:47:00 +00:00
strip trailing whitespace; no functional change svn path=/nixos/trunk/; revision=29285
2011-09-14 18:20:50 +00:00
}
Reference in New Issue Copy Permalink