The absence of current-context in the right place resulted in obscure
bugs. The reason this has not been detected before can only be that
it was unused.
This fixes a class of errors seen on aarch64 during coredns import where content was seen to be missing ("digest xxx not found") or "failed to get reader from content store" depending on which coredns version was being imported.
This gets automatically disabled by docker if the docker backend is
used, but the bundled containerd also doesn't seem to support cgroupsv2,
so disable it explicitly here, too.
* nixos/k3s: simplify config expression
* nixos/k3s: add config assertions and trim unneeded bits
* nixos/k3s: add a test that k3s works; minor module improvements
This is a single-node test. Eventually we should also have a multi-node
test to verify the agent bit works, but that one's more involved.
* nixos/k3s: add option description
* nixos/k3s: add defaults for token/serveraddr
Now that the assertion enforces their presence, we dont' need to use the typesystem for it.
* nixos/k3s: remove unneeded sudo in test
* nixos/k3s: add to test list
Many options define their example to be a Nix value without using
literalExample. This sometimes gets rendered incorrectly in the manual,
causing confusion like in https://github.com/NixOS/nixpkgs/issues/25516
This fixes it by using literalExample for such options. The list of
option to fix was determined with this expression:
let
nixos = import ./nixos { configuration = {}; };
lib = import ./lib;
valid = d: {
# escapeNixIdentifier from https://github.com/NixOS/nixpkgs/pull/82461
set = lib.all (n: lib.strings.escapeNixIdentifier n == n) (lib.attrNames d) && lib.all (v: valid v) (lib.attrValues d);
list = lib.all (v: valid v) d;
}.${builtins.typeOf d} or true;
optionList = lib.optionAttrSetToDocList nixos.options;
in map (opt: {
file = lib.elemAt opt.declarations 0;
loc = lib.options.showOption opt.loc;
}) (lib.filter (opt: if opt ? example then ! valid opt.example else false) optionList)
which when evaluated will output all options that use a Nix identifier
that would need escaping as an attribute name.
A centralized list for these renames is not good because:
- It breaks disabledModules for modules that have a rename defined
- Adding/removing renames for a module means having to find them in the
central file
- Merge conflicts due to multiple people editing the central file
The kubeconfig provided to the kubernetes-control-plane-online.service
is invalid. However, the apiserver /healthz endpoint can be accessed without auth so it's
simpler to just use curl for that.
by adding targets and curl wait loops to services to ensure services
are not started before their depended services are reachable.
Extra targets cfssl-online.target and kube-apiserver-online.target
syncronize starts across machines and node-online.target ensures
docker is restarted and ready to deploy containers on after flannel
has discussed the network cidr with apiserver.
Since flannel needs to be started before addon-manager to configure
the docker interface, it has to have its own rbac bootstrap service.
The curl wait loops within the other services exists to ensure that when
starting the service it is able to do its work immediately without
clobbering the log about failing conditions.
By ensuring kubernetes.target is only reached after starting the
cluster it can be used in the tests as a wait condition.
In kube-certmgr-bootstrap mkdir is needed for it to not fail to start.
The following is the relevant part of systemctl list-dependencies
default.target
● ├─certmgr.service
● ├─cfssl.service
● ├─docker.service
● ├─etcd.service
● ├─flannel.service
● ├─kubernetes.target
● │ ├─kube-addon-manager.service
● │ ├─kube-proxy.service
● │ ├─kube-apiserver-online.target
● │ │ ├─flannel-rbac-bootstrap.service
● │ │ ├─kube-apiserver-online.service
● │ │ ├─kube-apiserver.service
● │ │ ├─kube-controller-manager.service
● │ │ └─kube-scheduler.service
● │ └─node-online.target
● │ ├─node-online.service
● │ ├─flannel.target
● │ │ ├─flannel.service
● │ │ └─mk-docker-opts.service
● │ └─kubelet.target
● │ └─kubelet.service
● ├─network-online.target
● │ └─cfssl-online.target
● │ ├─certmgr.service
● │ ├─cfssl-online.service
● │ └─kube-certmgr-bootstrap.service
to protect services from crashing and clobbering the logs when
certificates are not in place yet and make sure services are activated
when certificates are ready.
To prevent errors similar to "kube-controller-manager.path: Failed to
enter waiting state: Too many open files"
fs.inotify.max_user_instances has to be increased.
Before flannel is ready there is a brief time where docker will be
running with a default docker0 bridge. If kubernetes happens to spawn
containers before flannel is ready, docker can't be restarted when
flannel is ready because some containers are still running on the
docker0 bridge with potentially different network addresses.
Environment variables in `EnvironmentFile` override those defined via
`Environment` in the systemd service config.
Co-authored-by: Christian Albrecht <christian.albrecht@mayflower.de>
+ isolate etcd on the master node by letting it listen only on loopback
+ enabling kubelet on master and taint master with NoSchedule
The reason for the latter is that flannel requires all nodes to be "registered"
in the cluster in order to setup the cluster network. This means that the
kubelet is needed even at nodes on which we don't plan to schedule anything.
- All kubernetes components have been seperated into different files
- All TLS-enabled ports have been deprecated and disabled by default
- EasyCert option added to support automatic cluster PKI-bootstrap
- RBAC has been enforced for all cluster components by default
- NixOS kubernetes test cases make use of easyCerts to setup PKI
In fff5923686 all occurences of
users.extraUsers and users.extraGroups have been changed tree-wide to
users.users and users.group. In the meantime the hadoop modules were
introduced via #41381 (060a98e9f4).
Unfortunately those modules still use users.extraUsers, which has been
renamed a long time ago (14321ae243, about
three years from now), so let's actually rename it accordingly as well.
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @matthewbauer, @aespinosa
Kubernetes dashboard currently has cluster admin permissions,
which is not recommended.
- Renamed option "services.kubernetes.addons.dashboard.enableRBAC" to "services.kubernetes.addons.dashboard.rbac.enable"
- Added option "services.kubernetes.addons.dashboard.rbac.clusterAdmin", default = false.
- Setting recommended minimal permissions for the dashboard in accordance with https://github.com/kubernetes/dashboard/wiki/Installation
- Updated release note for 18.09.
- Added option 'cni.configDir' to allow for having CNI config outside of nix-store
Existing behavior (writing verbatim CNI conf-files to nix-store) is still available.
- Removed unused option 'apiserver.publicAddress' and changed 'apiserver.address' to 'bindAddress'
This conforms better to k8s docs and removes existing --bind-address hardcoding to 0.0.0.0
- Fixed c/p mistake in apiserver systemd unit description
- Updated 18.09 release notes to reflect changes to existing options
And fixed some typos from previous PR
- Make docker images for Kubernetes Dashboard and kube-dns configurable
As shipped with k8s 1.10.3.
Also:
- updated the definition jsons as they are distributed in k8s.
- updated the image uris as they are renamed in k8s
- added imageDigest param as per 736848723e
As shipped with k8s 1.10.3.
Also:
- updated the definition jsons as they are distributed in k8s.
- updated the image uris as they are renamed in k8s
- added imageDigest param as per 736848723e
* Fix reference CNI plugins
* The plugins were split out of the upstream cni repo around version
0.6.0
* Fix RBAC and DNS tests
* Fix broken apiVersion fields
* Change plugin linking to look in ${package}/bin rather than
${package.plugins}
* Initial work towards a working e2e test
* Test still fails, but at least the expression evaluates now
Continues @srhb's work in #37199Fixes#37199
Uses the HTTPS url for cases where the existing URL has a permanent
redirect. For each domain, at least one fixed derivation URL was
downloaded to test the domain is properly serving downloads.
Also fixes jbake source URL, which was broken.
- add flannel support
- remove deprecated authorizationRBACSuperAdmin option
- rename from deprecated poratalNet to serviceClusterIpRange
- add nodeIp option for kubelet
- kubelet, add br_netfilter to kernelModules
- enable firewall by default
- enable dns by default on node and on master
- disable iptables for docker by default on nodes
- dns, restart on failure
- update tests
and other minor changes
* lib: introduce imap0, imap1
For historical reasons, imap starts counting at 1 and it's not
consistent with the rest of the lib.
So for now we split imap into imap0 that starts counting at zero and
imap1 that starts counting at 1. And imap is marked as deprecated.
See c71e2d4235 (commitcomment-21873221)
* replace uses of lib.imap
* lib: move imap to deprecated.nix
