Commit Graph

132 Commits

Author SHA1 Message Date
Doron Behar
307da96e17
nixos/doc: small rephrase in documentation of documentation
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-08-16 06:24:37 +00:00
Doron Behar
924a07dc22 nixos/doc: Improve documentation of documentation
- Move contents of README.md from
  nixos/modules/installer/tools/manpages/ to
  nixos/doc/manual/contributing-to-this-manual.chapter.md.
- Don't mention DocBook as its obsolete and too specific.
- Rename derivation attribute name of configuration.nix(5) manual page,
  both on the `contributing-to-this-manual.chapter.md`, and in other
  places.
2023-08-12 22:28:04 +03:00
Doron Behar
32f75a0f2a nixos/install-tools: Add manpages to packages instead of seperating them
Since each such `nixos-*` tool has it's own derivation, exposed in pkgs,
There is no point in separating the manuals from the packages. If
someone wishes to have the tools without the manuals, they can use
meta.outputsToInstall to disable the installation of the manpages of
these packages. This Fixes #244450.
2023-08-12 18:01:06 +03:00
Alejandro Sanchez Medina
a49d40a005 nixos manual: extract some build paths
Co-authored-by: Shahar "Dawn" Or <mightyiampresence@gmail.com>
2023-07-25 17:03:12 +07:00
Jacek Galowicz
846ad444c7 integration test driver: Auto-generate integration test driver's machine
method documentation of nixos docs from python doc strings
2023-07-10 08:51:52 +02:00
pennae
6fcb6eee77 nixos/doc: set meta generator for html manuals properly
we no longer have to look like docbook-xslt generates our manuals
exclusively, so let's put something useful in there instead.
2023-06-16 19:26:23 +02:00
pennae
426903d2fb nixos/manual: remove docbook intermediates
they're no longer used for anything.
2023-06-16 19:26:23 +02:00
pennae
f52f531a4e nixos/make-options-doc: deprecate docbook outputs
they're no longer necessary for us and will almost definitely start to
rot now (like commonmark and asciidoc outputs did previously). most
existing users seem to take the docbook output and run it through pandoc
to generate html, those can easily migrate to use commonmark instead.
other users will hopefully pipe up when they notice that things they rely
on are going away.

optionsUsedDocbook has only been around for one release and only exposed
to allow other places to generate warnings, so that does not deserve
such precautions.
2023-06-13 16:56:32 +02:00
pennae
20152b4269 nixos/doc: remove docbook options compatibility
no longer needed or useful, and may even produce false positives now
that markdown is the default language for option docs.
2023-06-13 16:56:31 +02:00
pennae
0997ae1903 nixos/manual: disallow docbook option docs
it's been long in the making, and with 23.05 out we can finally disable
docbook option docs and default to markdown instead. this brings a
massive speed boost in manual and manpage builds, so much so that we may
consider enabling user module documentation by default.

we don't remove the docbook support code entirely yet because it's a lot
all over, and probably better removed in multiple separate changes.
2023-06-13 16:56:30 +02:00
pennae
3edc20aa55 nixos/doc: stub out epub manual
epub manuals are holding back the transition away from docbook, and
cursory research does not suggest that they are used very much. it's
still very early in the 23.11 release cycle, so if we're going to find
out just how many people do use the epub manuals it should be now.

this need not be the end of epub manuals. nixos-render-docs could be
extended to also export epubs, but that has not been done yet since it's
going to be some effort with unknown real-world usefulness.
2023-06-12 17:50:19 +02:00
Sergey
e4f9d4f62c nixos/manual: fix cross-compilation
Remove absolute reference to xsltproc. This reference breaks cross-compilation: instead of host version builder tries to execute binary for target and fails.
2023-04-05 04:49:26 +02:00
pennae
3dc0323bae nixos/manual: apply options preprocessing to full manual
the conversion to the markdown-based workflow missed that generating the
manual as docbook also generates the option docs with nixos-specific
wrapper elements that require postprocessing before the document is
real, valid docbook. restore this processing to the full manual.

it's not the prettiest thing, done like this, but we only need it for
one release so it doesn't have to be.
2023-03-19 13:02:03 +01:00
pennae
54f4992e80 nixos/manual: render html with nixos-render-doc if !allowDocBook
this reproduces the docbook-generated html manual exactly enough to
appease the compare workflows while we still support both toolchains.
it's also a lot faster than the docbook toolchain, rendering the entire
html manual in about two seconds on this machine (while docbook needs
about 20).
2023-02-21 18:26:41 +01:00
pennae
81cf173256 nixos-render-docs: use multiprocessing for options
options processing is pretty slow right now, mostly because the
markdown-it-py parser is pure python (and with performance
pessimizations at that). options parsing *is* embarassingly parallel
though, so we can just fork out all the work to worker processes and
collect the results.

multiprocessing probably has a greater benefit on linux than on darwin
since the worker spawning method darwin uses is less efficient than
fork() on linux. this hasn't been tested on darwin, only on linux, but
if anything darwin will be faster with its preferred method.
2023-02-18 13:17:57 +01:00
pennae
1229e735ac nixos-render-docs: add structural includes, use for manual
this adds support for structural includes to nixos-render-docs.
structural includes provide a way to denote the (sub)structure of the
nixos manual in the markdown source files, very similar to how we used
literal docbook blocks before, and are processed by nixos-render-docs
without involvement of xml tooling. this will ultimately allow us to
emit the nixos manual in other formats as well, e.g. html, without going
through docbook at all.

alternatives to this source layout were also considered:

a parallel structure using e.g. toml files that describe the document
tree and links to each part is possible, but much more complicated to
implement than the solution chosen here and makes it harder to follow
which files have what substructure. it also makes it much harder to
include a substructure in the middle of a file.

much the same goes for command-line arguments to the converter, only
that command-lined arguments are even harder to specify correctly and
cannot be reasonably pulled together from many places without involving
another layer of tooling. cli arguments would also mean that the manual
structure would be fixed in default.nix, which is also not ideal.
2023-02-12 13:02:42 +01:00
pennae
8b7f0e559a nixos/manual: clean up default.nix a bit
- inline copySources into single user
- remove `inherit sources` where it's not necessary
- inline generatedSources. this will go away completely soon so we may
  as well.
- inline modulesDoc into manual-combined. this too will go away soon.
- inline sources into manual-combined. this too will go away soon.
2023-02-12 11:58:20 +01:00
pennae
ef413e3eac nixos/manual: split manpages-combined from manual-combined
once we generate the entire manual-combined.xml with a single
nixos-render-docs invocation we will no longer need any options xml
files. likewise we do not need the test options xml in the manpage
build. splitting manpages-combined from manual-combined also allows
these two to run in parallel, slightly improving build times.
2023-02-12 11:56:26 +01:00
pennae
d041641b1a nixos/manual: remove md-to-db
with manual chapters no longer needing pandoc for their conversion to
xml we can get rid of this source of confusion, and its huge cache of
xml files.
2023-02-10 06:40:15 +01:00
pennae
652a283e51 nixos-render-docs: render manual chapters during manual build
render all manual chapters to docbook from scratch every time the manual
is built. nixos-render-docs is quick enough at this to not worry about
the cost (needing only about a second), and it means we can remove
md-to-db.sh in the next commit.

no changes to the rendered html manual except for replacements and smartquotes.
2023-02-10 06:40:02 +01:00
pennae
67917ac102 nixos-render-docs: rename manual docbook converter to docbook-section
we'll soon add another docbook converter that does not emit a section as
a collection of chapters, but sections or chapters on their own. this
should clarify naming a bit before there can be any confusion.
2023-02-10 06:40:02 +01:00
pennae
10f2c3cacf nixos/manual: remove sources input from manpages drv
it's not used. holdover from when manpages were written in docbook.
2023-02-10 06:40:00 +01:00
pennae
10a4f0daca nixos-render-docs: add options manpage converter
mdoc is just too slow to render on groff, and semantic markup doesn't
help us any for generated pages.

this produces a lot of changes to configuration.nix.5, but only few
rendering changes. most of those seem to be place losing a space where
docbook emitted roff code that did not faithfully represent the input
text, though a few places also gained space where docbook dropped them.
notably we also don't need the compatibility code docbook-xsl emitted
because that problem was fixed over a decade ago.

this will handle block quotes, which the docbook stylesheets turned into
a mess of roff requests that ended up showing up in the output instead
of being processed.
2023-02-08 15:23:34 +01:00
figsoda
42d1d60a92
Merge pull request #206773 from SuperSandro2000/cleanup-unused-bindings
treewide: cleanup some unused bindings
2023-02-06 20:07:50 -05:00
Sandro Jäckel
50e0012f9d
treewide: cleanup some unused bindings 2023-02-07 01:36:15 +01:00
pennae
0c601b12bf nixos/manual: translate manpages to mdoc
since we want to move away from docbook and having docbook manpages
around is going to block further progress we have to translate the
current nixos manpages from docbook it *something* else. mdoc seems the
most appropriate choice since markdown can't represent all the things
manpages can differentiate with even more extensions. if we ever need
html manpages we can render them using troff, like many of the online
manpage viewers, or rewrite them again. since we haven't had html
manpages for any of these in many years that seems unlikely to happen.

unlike most of the recent markdown conversions this comes with a lot of
minor rendering changes, mostly in spacing. docbook-xslt creates manual
pages in a very different dialect than mdoc (which is used here).
2023-02-03 02:23:07 +01:00
pennae
0a6e6cf7e6 nixos/manual: render module chapters with nixos-render-docs
this converts meta.doc into an md pointer, not an xml pointer. since we
no longer need xml for manual chapters we can also remove support for
manual chapters from md-to-db.sh

since pandoc converts smart quotes to docbook quote elements and our
nixos-render-docs does not we lose this distinction in the rendered
output. that's probably not that bad, our stylesheet didn't make use of
this anyway (and pre-23.05 versions of the chapters didn't use quote
elements either).

also updates the nixpkgs manual to clarify that option docs support all
extensions (although it doesn't support headings at all, so heading
anchors don't work by extension).
2023-01-27 20:07:34 +01:00
pennae
617822c95b nixos/manual: remove olinkdb
as far as we can tell nixos has only ever had a total of one olink, and
currently has no olinks at all. we can't currently represent olinks in
markdown docs, and if we re-add support for cross-document links they
will take a different form (and not use docbook, which will have to be
phased out before we re-add anything).

the olinkdb is thus unused and takes 10 seconds on our machine to build,
holding up the rest of the manual for no benefit.
2023-01-26 00:32:56 +01:00
pennae
df09c21fb2 nixos/documentation: deprecate docbook option docs
following the plan in https://github.com/NixOS/nixpkgs/pull/189318#discussion_r961764451

also adds an activation script to print the warning during activation
instead of during build, otherwise folks using the new CLI that hides
build logs by default might never see the warning.
2023-01-22 17:08:40 +01:00
Robert Hensing
c23666473c doc: Remove all section numbers
Simplicity.

We use anchors nowadays. Nobody should be printing this.
2022-12-12 13:04:39 +01:00
Robert Hensing
216c5dc10d nixos/doc: Disambiguate test option ids
Changing this later on would break external links into the manual.
2022-09-29 12:41:59 +02:00
Robert Hensing
ac03757eb2 nixos/doc: Wire up the test options reference 2022-09-24 17:38:10 +01:00
Robert Hensing
6205d37747 nixos/testing: Improve option docs 2022-09-24 17:38:10 +01:00
pennae
767485a0de lib/options: deprecate docbook text and literalDocBook
deprecate literalDocBook by adding a warning (that will not fire yet) to
its uses and other docbook literal strings by adding optional warning
message to mergeJSON.
2022-09-10 18:23:13 +02:00
Robert Hensing
49237015d6 nixos/doc: Make nixos-manual-combined fail easy to inspect
By writing the unchecked outputs before checking them, they will
be written to a store path, which appears in the log, and which
sticks around even if the build fails. Eventually it is GCed, but
until then, you can open the file.

If you run it in a terminal+editor combination like VSCode, the
failure location is just one Ctrl+click away.
2022-06-27 17:41:32 +02:00
Jan Tojnar
84d6565d52 nixos/man: prevent duplication of options
libxslt 1.1.35 fixed conflict resolution for templates to match the specification.
This uncovered a bug in docbook-xsl (https://github.com/docbook/xslt10-stylesheets/issues/240),
which causes option names to be duplicated into the option descriptions.

Let’s resolve the conflict by patching the stylesheets.

Fixes: https://github.com/NixOS/nixpkgs/issues/166304
2022-04-03 21:22:47 +02:00
pennae
9d3ba92d63 nixos/documentation: fix docs cross build
a few things should've used buildPackages/nativeBuildInputs to not not require
the host architecture for building docs. tested by building aarch64-linux docs
on x86_64-linux, and the result looks good.
2022-01-08 16:03:23 +01:00
pennae
50954ad1c5 nixos/make-options-doc: treat missing descriptions as errors by default
this partially solves the problem of "missing description" warnings of the
options doc build being lost by nix build, at the cost of failing builds that
previously ran. an option to disable this behaviour is provided.
2022-01-02 19:46:13 +01:00
pennae
fc614c37c6 nixos/documentation: split options doc build
most modules can be evaluated for their documentation in a very
restricted environment that doesn't include all of nixpkgs. this
evaluation can then be cached and reused for subsequent builds, merging
only documentation that has changed into the cached set. since nixos
ships with a large number of modules of which only a few are used in any
given config this can save evaluation a huge percentage of nixos
options available in any given config.

in tests of this caching, despite having to copy most of nixos/, saves
about 80% of the time needed to build the system manual, or about two
second on the machine used for testing. build time for a full system
config shrank from 9.4s to 7.4s, while turning documentation off
entirely shortened the build to 7.1s.
2022-01-02 19:46:13 +01:00
pennae
c533b01863 nixos/doc/manual: remove non-matching optionsXML inherit 2021-12-06 16:12:32 +01:00
Ben Siraphob
1c2a2b0a08 treewide: fold -> foldr 2021-01-26 10:57:07 +07:00
zimbatm
7d834eff6c
nixos/manual: make reproducible (#102234) 2020-10-31 21:18:16 +01:00
Frederik Rietdijk
70bb2e4151 nixos manual: have a toc for each part and chapter
to be inline with the Nixpkgs manual. This makes navigating to
particular sections easier.
2019-10-30 10:25:09 +01:00
Frederik Rietdijk
227dff4e70 nixos manual: reduce toc depth
to reduce the amount of information presented.
2019-10-30 10:25:09 +01:00
Domen Kožar
5cfd034af0
Extract NixOS options documentation generation to a function
Motivation is to support other repositories containing nixos
modules that would like to generate options documentation:

- nix-darwin
- private repos
- arion
- ??
2019-08-08 16:18:09 +02:00
Graham Christensen
777e94d903
Merge pull request #55436 from layus/warn-manual-ids
Nixos manual: error out on missing IDs
2019-03-09 08:21:07 -05:00
Jan Malakhovski
ca496b194f nixos: doc: increase maxdepth for xsltproc
See https://github.com/NixOS/nixpkgs/issues/37903#issuecomment-376618117
for details. With the previous patch and some custom modules included in
`configuration.nix` the above bug is very easy to trigger.

This is a simplest workaround I have. A proper solution would look like
https://github.com/NixOS/nixpkgs/issues/37903#issuecomment-376980838.
2019-03-05 09:41:40 +00:00
Guillaume Maudoux
a65974c604 nixos/manual: error out on missing IDs 2019-02-08 10:59:46 +01:00
Guillaume Maudoux
4c0230eb56 nixos/manual: warn on missing xml:id 2019-02-07 16:35:24 +01:00
Jan Malakhovski
fe744d3fb1 nixos: doc: rename manual to manualHTML, cleanup references
Because when I see "config.system.build.manual.manual" after I forgot
what it means I ask "Why do I need that second `.manual` there again?".
Doesn't happen with `config.system.build.manual.manualHTML`.
2018-09-23 20:47:58 +00:00