LXD should have more tools in its path

[CosmicToast]

Issue description

lxd supports multiple storage backends, but most of them are unusable.

Steps to reproduce

1. Set virtualisation.lxd.enable = true;
2. Add your user to the lxd group (or perform the rest of the steps as root).
3. Try to initialize a storage backend (e.g using lxd init) that isn't marked on the list below.

Technical details

Nix's lxd storage backend support matrix:

• Directory (see note 1)
• Btrfs (see note 1)
• Zfs (missing path)
• LVM Thin Volumes (missing path)
• Raw LVM (untested, but seems to not be present in lxd.nix)
• Ceph (untested, but seems to not be present in lxd.nix)

Note 1: LXD does somewhat work with the directory and btrfs backends, but any and all created containers insist that /var/lib/lxc/rootfs must exist.
It doesn't actually get used. This is almost certainly a bug, and is mentioned in #25280.

Potential solutions

• LVM2 and thin-provisioning tools should almost definitely be enabled unconditionally, as per the TODO.
• Zfs and Btrfs support may potentially be autodetected based on boot.supportedFilesystems.
• Ceph is sufficiently uncommon that it may be an explicitly-enabled option.

Alternatively, we can have something ala boot.supportedFilesystems under, say, virtualisation.lxd.storageBackends.

I'm going to be busy working for a bit, and am still somewhat unfamiliar with the nix expression language, but I'll take a look at making a (basic, to be expanded upon) patch once I have time.

[Shados]

FWIW, the version of Ceph in nixpkgs is 1) ancient, and 2) marked as broken (due to a DOS vulnerability), so I wouldn't worry about it too much right now. I've been working on packaging a recent (12.2.x) release, but they changed to building using cmake and some outputs are silently not being built by my current expression...

[CosmicToast]

Ceph is the one I'm worried least about (since it's quite uncommon, as far as I can tell).
LVM and Zfs, on the other hand, are very common, and currently nonfunctional.

[1pakch]

I like supported backends option more than unconditional enabling. This option is not necessarily LXD specific. For instance, current docker module also does not include thin provisioning tools automatically AFAIK.

[MrSorcus]

When it will be completed?

[CosmicToast]

Probably never, I stopped using nix since it ended up not fitting into my workflow very well, with relatively common annoying (to me) side effects (of the immutability bit).
If there was work being done it would be availabe in a linked PR ;)

[Mic92]

zfsSupport was added as a flag in 7663de1

[Mic92]

Ceph module was also revisted in the last release. It also has a test now.

[jkryl]

I had this problem with lvm storage backend and was able to work around it by putting the following line to /etc/nixos/configuration.nix:

  systemd.services.lxd.path = with pkgs; [ lvm2 thin-provisioning-tools ];

The line above includes lvm tools in the PATH of lxd daemon. Until I did that I was getting the following error from lxd init or lxc when trying to create a storage pool:

$ lxc storage create lxd lvm source=/dev/sdb1
Error: Error getting LVM version: Failed to run: lvm version:
output:''

[mt-caret]

On the topic of tools which should be included in the lxd module, I think libnvidia-container should be included as well (related issue: #27999). I really should take a stab at it someday...

[stale]

I marked this as stale due to inactivity. → More info

[Mic92]

still not addressed.

[gebner]

I believe qemu-img is missing as well:

$ lxc launch --vm images:archlinux/current
Error: Failed instance creation: Create instance from image: Failed reading image info "/var/lib/lxd/storage-pools/default/virtual-machines/sincere-eagle/root.img": Failed to run: qemu-img info --output=json /var/lib/lxd/images/ed08824e15dd8efc89afb9eba288f8417d2309c0bce7c7684ac160ece786034f.rootfs:i

But that still fails with "/usr/share not found" even if I add qemu to the path..

[stale]

I marked this as stale due to inactivity. → More info

[Mic92]

For some backends I would add additional options rather than just adding all of them. Qemu for example can be quite big (i.e. pulling in x11 and opengl dependencies).

[anpin]

following the advice of @jkryl and hitting Error: Failed instance creation: Failed creating instance from image: Error creating LVM logical volume: Error making filesystem on LVM logical volume: Failed to run: mkfs.ext4 -E nodiscard,lazy_itable_init=0,lazy_journal_init=0 /dev/vg/images_* I found here that one should also add e2fsprogs to lxd path as such

systemd.services.lxd.path = with pkgs; [ lvm2 thin-provisioning-tools e2fsprogs ];