nixpkgs/nixos/doc/manual/development.xml
2013-10-27 08:53:47 +02:00

599 lines
20 KiB
XML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink">
<title>Development</title>
<para>This chapter has some random notes on hacking on
NixOS.</para>
<!--===============================================================-->
<section>
<title>Hacking on NixOS</title>
<para>By default, NixOSs <command>nixos-rebuild</command> command
uses the NixOS and Nixpkgs sources provided by the
<literal>nixos-unstable</literal> channel (kept in
<filename>/nix/var/nix/profiles/per-user/root/channels/nixos</filename>).
To modify NixOS, however, you should check out the latest sources from
Git. This is done using the following command:
<screen>
$ nixos-checkout <replaceable>/my/sources</replaceable>
</screen>
or
<screen>
$ mkdir -p <replaceable>/my/sources</replaceable>
$ cd <replaceable>/my/sources</replaceable>
$ nix-env -i git
$ git clone git://github.com/NixOS/nixpkgs.git
</screen>
This will check out the latest NixOS sources to
<filename><replaceable>/my/sources</replaceable>/nixpkgs/nixos</filename> and
the Nixpkgs sources to
<filename><replaceable>/my/sources</replaceable>/nixpkgs</filename>.
If you want to rebuild your system using your (modified) sources, you
need to tell <command>nixos-rebuild</command> about them using the
<option>-I</option> flag:
<screen>
$ nixos-rebuild switch -I <replaceable>/my/sources</replaceable>/nixpkgs
</screen>
</para>
<para><command>nixos-rebuild</command> affects only the system profile.
To install packages to your user profile from expressions in
<replaceable>/my/sources</replaceable>, use
<command>nix-env -f <replaceable>/my/sources</replaceable>/nixpkgs</command>,
or change the default by replacing the symlink in
<filename>~/.nix-defexpr</filename>:
<screen>
$ rm -f ~/.nix-defexpr/channels
$ ln -s <replaceable>/my/sources</replaceable>/nixpkgs ~/.nix-defexpr/nixpkgs
</screen>
</para>
<para>You should not pass the base directory
<filename><replaceable>/my/sources</replaceable></filename>
to <command>nix-env</command>, as it will break after interpreting expressions
in <filename>nixos/</filename> as packages.</para>
</section>
<!--===============================================================-->
<section>
<title>Extending NixOS</title>
<para>NixOS is based on a modular system for declarative configuration.
This system combines multiple <emphasis>modules</emphasis> to produce one
configuration. One of the module which compose your computer
configuration is <filename>/etc/nixos/configuration.nix</filename>. Other
modules are available under NixOS <filename>modules</filename>
directory</para>
<para>A module is a file which handles one specific part of the
configuration. This part of the configuration could correspond to
hardware, a service, network settings, or preferences. A module
configuration does not have to handle everything from scratch, it can base
its configuration on other configurations provided by other modules. Thus
a module can <emphasis>define</emphasis> options to setup its
configuration, and it can also <emphasis>declare</emphasis> options to be
fed by other modules.</para>
<!-- module syntax -->
<para xml:id="para-module-syn">A module is a file which contains a Nix
expression. This expression should be either an expression which gets
evaluated into an attribute set or a function which returns an attribute
set.</para>
<para>When the expression is a function, it should expect only one argument
which is an attribute set containing an attribute
named <varname>config</varname> and another attribute
named <varname>pkgs</varname>. The <varname>config</varname> attribute
contains the result of the merge of all modules. This attribute is
evaluated lazily, such as any Nix expression. For more details on how
options are merged, see the details in <xref linkend="para-opt-decl"/>.
The <varname>pkgs</varname> attribute
contains <emphasis>nixpkgs</emphasis> attribute set of packages. This
attribute is necessary for declaring options.</para>
<example xml:id='module-syntax'><title>Usual module content</title>
<programlisting>
{ config, pkgs, ... }: <co xml:id='module-syntax-1' />
{
imports =
[ <co xml:id='module-syntax-2' />
];
options = {
<co xml:id='module-syntax-3' />
};
config = {
<co xml:id='module-syntax-4' />
};
}</programlisting>
</example>
<para><xref linkend='module-syntax' /> Illustrates
a <emphasis>module</emphasis> skeleton.
<calloutlist>
<callout arearefs='module-syntax-1'>
<para>This line makes the current Nix expression a function. This
line can be omitted if there is no reference to <varname>pkgs</varname>
and <varname>config</varname> inside the module.</para>
</callout>
<callout arearefs='module-syntax-2'>
<para>This list is used to enumerate path to other modules which are
declaring options used by the current module. In NixOS, default modules
are listed in the file <filename>modules/module-list.nix</filename>.
The default modules don't need to be added in the import list.</para>
</callout>
<callout arearefs='module-syntax-3'>
<para>This attribute set contains an attribute set of <emphasis>option
declaration</emphasis>.</para>
</callout>
<callout arearefs='module-syntax-4'>
<para>This attribute set contains an attribute set of <emphasis>option
definitions</emphasis>. If the module does not have any imported
modules or any option declarations, then this attribute set can be used
in place of its parent attribute set. This is a common case for simple
modules such
as <filename>/etc/nixos/configuration.nix</filename>.</para>
</callout>
</calloutlist>
</para>
<!-- option definitions -->
<para xml:id="para-opt-def">A module defines a configuration which would be
interpreted by other modules. To define a configuration, a module needs
to provide option definitions. An option definition is a simple
attribute assignment.</para>
<para>Option definitions are made in a declarative manner. Without
properties, options will always be defined with the same value. To
introduce more flexibility in the system, option definitions are guarded
by <emphasis>properties</emphasis>.</para>
<para>Properties are means to introduce conditional values inside option
definitions. This conditional values can be distinguished in two
categories. The condition which are local to the current configuration
and conditions which are dependent on others configurations. Local
properties are <varname>mkIf</varname>
and <varname>mkAssert</varname>. Global properties
are <varname>mkOverride</varname>, <varname>mkDefault</varname>
and <varname>mkOrder</varname>.</para>
<para><varname>mkIf</varname> is used to remove the option definitions which
are below it if the condition is evaluated to
false. <varname>mkAssert</varname> expects the condition to be evaluated
to true otherwise it raises an error message.</para>
<para><varname>mkOverride</varname> is used to mask previous definitions if
the current value has a lower mask number. The mask value is 100 (default)
for any option definition which does not use this property.
Thus, <varname>mkDefault</varname> is just a short-cut with a higher mask
(1000) than the default mask value. This means that a module can set an
option definition as a preference, and still let another module defining
it with a different value without using any property.</para>
<para><varname>mkOrder</varname> is used to sort definitions based on the
rank number. The rank number will sort all options definitions before
giving the sorted list of option definition to the merge function defined
in the option declaration. A lower rank will move the definition to the
beginning and a higher rank will move the option toward the end. The
default rank is 100.</para>
<!-- option declarations -->
<para xml:id="para-opt-decl">A module may declare options which are used by
other module to change the configuration provided by the current module.
Changes to the option definitions are made with properties which are using
values extracted from the result of the merge of all modules
(the <varname>config</varname> argument).</para>
<para>The <varname>config</varname> argument reproduce the same hierarchy of
all options declared in all modules. For each option, the result of the
option is available, it is either the default value or the merge of all
definitions of the option.</para>
<para>Options are declared with the
function <varname>pkgs.lib.mkOption</varname>. This function expects an
attribute set which at least provides a description. A default value, an
example, a type, a merge function and a post-process function can be
added.</para>
<para>Types are used to provide a merge strategy for options and to ensure
the type of each option definitions. They are defined
in <varname>pkgs.lib.types</varname>.</para>
<para>The merge function expects a list of option definitions and merge
them to obtain one result of the same type.</para>
<para>The post-process function (named <varname>apply</varname>) takes the
result of the merge or of the default value, and produce an output which
could have a different type than the type expected by the option.</para>
<!-- end -->
<example xml:id='locate-example'><title>Locate Module Example</title>
<programlisting>
{ config, pkgs, ... }:
with pkgs.lib;
let
cfg = config.services.locate;
locatedb = "/var/cache/locatedb";
logfile = "/var/log/updatedb";
cmd =''root updatedb --localuser=nobody --output=${locatedb} > ${logfile}'';
in
{
imports = [ /path/to/nixpkgs/nixos/modules/services/scheduling/cron.nix ];
options = {
services.locate = {
enable = mkOption {
default = false;
example = true;
type = with types; bool;
description = ''
If enabled, NixOS will periodically update the database of
files used by the <command>locate</command> command.
'';
};
period = mkOption {
default = "15 02 * * *";
type = with types; uniq string;
description = ''
This option defines (in the format used by cron) when the
locate database is updated.
The default is to update at 02:15 (at night) every day.
'';
};
};
};
config = mkIf cfg.enable {
services.cron = {
enable = true;
systemCronJobs = "${cfg.period} root ${cmd}";
};
};
}</programlisting>
</example>
<para><xref linkend='locate-example' /> illustrates a module which handles
the regular update of the database which index all files on the file
system. This modules has option definitions to rely on the cron service
to run the command at predefined dates. In addition, this modules
provides option declarations to enable the indexing and to use different
period of time to run the indexing. Properties are used to prevent
ambiguous definitions of option (enable locate service and disable cron
services) and to ensure that no options would be defined if the locate
service is not enabled.</para>
</section>
<!--===============================================================-->
<section>
<title>Building specific parts of NixOS</title>
<para>
<screen>
$ nix-build /path/to/nixpkgs/nixos -A <replaceable>attr</replaceable></screen>
where <replaceable>attr</replaceable> is an attribute in
<filename>/path/to/nixpkgs/nixos/default.nix</filename>. Attributes of interest include:
<variablelist>
<varlistentry>
<term><varname>config</varname></term>
<listitem><para>The computer configuration generated from
the <envar>NIXOS_CONFIG</envar> environment variable (default
is <filename>/etc/nixos/configuration.nix</filename>) with the NixOS
default set of modules.</para></listitem>
</varlistentry>
<varlistentry>
<term><varname>system</varname></term>
<listitem><para>The derivation which build your computer system. It is
built by the command <command>nixos-rebuild
build</command></para></listitem>
</varlistentry>
<varlistentry>
<term><varname>vm</varname></term>
<listitem><para>The derivation which build your computer system inside a
virtual machine. It is built by the command <command>nixos-rebuild
build-vm</command></para></listitem>
</varlistentry>
</variablelist>
</para>
<para>
Most parts of NixOS can be built through the <varname>config</varname>
attribute set. This attribute set allows you to have a view of the merged
option definitions and all its derivations. Important derivations are store
inside the option <option>system.build</option> and can be listed with the
command <command>nix-instantiate --xml --eval-only /path/to/nixpkgs/nixos -A
config.system.build</command>
</para>
</section>
<!--===============================================================-->
<section>
<title>Building your own NixOS CD</title>
<para>Building a NixOS CD is as easy as configuring your own computer. The
idea is to use another module which will replace
your <filename>configuration.nix</filename> to configure the system that
would be installed on the CD.</para>
<para>Default CD/DVD configurations are available
inside <filename>nixos/modules/installer/cd-dvd</filename>. To build them
you have to set <envar>NIXOS_CONFIG</envar> before
running <command>nix-build</command> to build the ISO.
<screen>
$ export NIXOS_CONFIG=/path/to/nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-minimal.nix
$ nix-build /path/to/nixpkgs/nixos -A config.system.build.isoImage</screen>
</para>
<para>Before burning your CD/DVD, you can check the content of the image by mounting anywhere like
suggested by the following command:
<screen>
$ mount -o loop -t iso9660 ./result/iso/cd.iso /mnt/iso</screen>
</para>
</section>
<!--===============================================================-->
<section>
<title>Testing/building the NixOS Manual</title>
<para>A quick way to see if your documentation improvements
or option descriptions look good:
<screen>
$ nix-build -A config.system.build.manual</screen>
</para>
</section>
<!--===============================================================-->
<section>
<title>Testing the installer</title>
<para>Building, burning, and
booting from an installation CD is rather
tedious, so here is a quick way to see if the installer works
properly:
<screen>
$ export NIXOS_CONFIG=/path/to/nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-minimal.nix
$ nix-build /path/to/nixpkgs/nixos -A config.system.build.nixos-install
$ dd if=/dev/zero of=diskimage seek=2G count=0 bs=1
$ yes | mke2fs -j diskimage
$ mount -o loop diskimage /mnt
$ ./result/bin/nixos-install</screen>
</para>
</section>
<!--===============================================================-->
<section>
<title>Testing the <literal>initrd</literal></title>
<para>A quick way to test whether the kernel and the initial ramdisk
boot correctly is to use QEMUs <option>-kernel</option> and
<option>-initrd</option> options:
<screen>
$ nix-build /path/to/nixpkgs/nixos -A config.system.build.initialRamdisk -o initrd
$ nix-build /path/to/nixpkgs/nixos -A config.system.build.kernel -o kernel
$ qemu-system-x86_64 -kernel ./kernel/bzImage -initrd ./initrd/initrd -hda /dev/null
</screen>
</para>
</section>
<section>
<title>Whole-system testing using virtual machines</title>
<para>
Complete NixOS GNU/Linux systems can be tested in virtual machines
(VMs). This makes it possible to test a system upgrade or
configuration change before rebooting into it, using the
<command>nixos-rebuild build-vm</command> or
<command>nixos-rebuild build-vm-with-bootloader</command> command.
</para>
<para>
<!-- The following is adapted from
http://wiki.nixos.org/wiki/NixOS_VM_tests, by Eelco Dolstra. -->
The <filename>tests/</filename> directory in the NixOS source tree
contains several <emphasis>whole-system unit tests</emphasis>.
These tests can be run<footnote><para>NixOS tests can be run both from
NixOS and from a non-NixOS GNU/Linux distribution, provided the
Nix package manager is installed.</para></footnote> from the NixOS
source tree as follows:
<screen>
$ nix-build tests/ -A nfs.test
</screen>
This performs an automated test of the NFS client and server
functionality in the Linux kernel, including file locking
semantics (e.g., whether locks are maintained across server
crashes). It will first build or download all the dependencies of
the test (e.g., all packages needed to run a NixOS VM). The test
is defined in <link
xlink:href="https://nixos.org/repos/nix/nixos/trunk/tests/nfs.nix">
<filename>tests/nfs.nix</filename></link>. If the test succeeds,
<command>nix-build</command> will place a symlink
<filename>./result</filename> in the current directory pointing at
the location in the Nix store of the test results (e.g.,
screenshots, test reports, and so on). In particular, a
pretty-printed log of the test is written to
<filename>log.html</filename>, which can be viewed using a web
browser like this:
<screen>
$ firefox result/log.html
</screen>
</para>
<para>
It is also possible to run the test environment interactively,
allowing you to experiment with the VMs. For example:
<screen>
$ nix-build tests/ -A nfs.driver
$ ./result/bin/nixos-run-vms
</screen>
The script <command>nixos-run-vms</command> starts the three
virtual machines defined in the NFS test using QEMU/KVM. The root
file system of the VMs is created on the fly and kept across VM
restarts in
<filename>./</filename><varname>hostname</varname><filename>.qcow2</filename>.
</para>
<para>
Finally, the test itself can be run interactively. This is
particularly useful when developing or debugging a test:
<screen>
$ nix-build tests/ -A nfs.driver
$ ./result/bin/nixos-test-driver
starting VDE switch for network 1
&gt;
</screen>
Perl statements can now be typed in to start or manipulate the
VMs:
<screen>
&gt; startAll;
(the VMs start booting)
&gt; $server-&gt;waitForJob("nfs-kernel-nfsd");
&gt; $client1-&gt;succeed("flock -x /data/lock -c 'sleep 100000' &amp;");
&gt; $client2-&gt;fail("flock -n -s /data/lock true");
&gt; $client1-&gt;shutdown;
(this releases client1's lock)
&gt; $client2-&gt;succeed("flock -n -s /data/lock true");
</screen>
The function <command>testScript</command> executes the entire
test script and drops you back into the test driver command line
upon its completion. This allows you to inspect the state of the
VMs after the test (e.g. to debug the test script).
</para>
<para>
This and other tests are continuously run on <link
xlink:href="http://hydra.nixos.org/jobset/nixos/trunk">the Hydra
instance at <literal>nixos.org</literal></link>, which allows
developers to be notified of any regressions introduced by a NixOS
or Nixpkgs change.
</para>
<para>
The actual Nix programming interface to VM testing is in NixOS,
under <link
xlink:href="https://nixos.org/repos/nix/nixos/trunk/lib/testing.nix">
<filename>lib/testing.nix</filename></link>. This file defines a
function which takes an attribute set containing a
<literal>nixpkgs</literal> attribute (the path to a Nixpkgs
checkout), and a <literal>system</literal> attribute (the system
type). It returns an attribute set containing several utility
functions, among which the main entry point is
<literal>makeTest</literal>.
</para>
<para>
The <literal>makeTest</literal> function takes a function similar to
that found in <link
xlink:href="https://nixos.org/repos/nix/nixos/trunk/tests/nfs.nix">
<filename>tests/nfs.nix</filename></link> (discussed above). It
returns an attribute set containing (among others):
<variablelist>
<varlistentry>
<term><varname>test</varname></term>
<listitem><para>A derivation containing the test log as an HTML file,
as seen above, suitable for presentation in the Hydra continuous
build system.</para></listitem>
</varlistentry>
<varlistentry>
<term><varname>report</varname></term>
<listitem><para>A derivation containing a code coverage report, with
meta-data suitable for Hydra.</para></listitem>
</varlistentry>
<varlistentry>
<term><varname>driver</varname></term>
<listitem><para>A derivation containing scripts to run the VM test or
interact with the VM network interactively, as seen above.</para>
</listitem>
</varlistentry>
</variablelist>
</para>
</section>
</chapter>