Fixes#241482.
Also fix test putting cluster in unhealthy `POOL_APP_NOT_ENABLED` state;
this seems to be the default state with Ceph 18.2.1 at least,
and it does not hurt to fix it now already in the way the Ceph docs say.
Also revert "nixosTests.ceph-single-node: remove dashboard check"
This reverts commit 41b27d7f4b.
the default hasn't been changed since 2009
this can improve our test performances
nixos/tests: remove explicit memorySize <1024
1024MiB is now the default
The current Ceph tests use the old method for OSDs to store data on
disks, known as Filestore. This means there are no tests for the
Bluestore functionality that run on install, which means that things
like RocksDB being broken can slip through and break the Bluestore
functionality in a subtle and difficult to debug manner.
Add a test to check that Bluestore works, at least on a single node.
