Rootless config generates invalid mount for /etc/resolv.conf

[rneilson]

Using umoci unpack --rootless (or umoci raw config --rootless), an extra mount is generated for /etc/resolv.conf, with "type": "none" 1. This causes runc to parse the mount with the default (directory) behavior 2, trying to create the destination as a directory instead of a file 3.

TBH, not sure if this is a runtime spec problem, a runc mount parsing problem, or a umoci config generation problem, but I'm guessing it's most easily fixed here.

To reproduce:

> skopeo copy docker://busybox:latest oci:busybox:latest
> umoci raw unpack --rootless --image busybox rootfs
> umoci raw config --image=busybox --rootless --rootfs=rootfs config.json
> runc run busybox
container_linux.go:344: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/resolv.conf\\\" to rootfs \\\"/data/contained/temp/rootfs\\\" at \\\"/data/contained/temp/rootfs/etc/resolv.conf\\\" caused \\\"not a directory\\\"\""

umoci --version

umoci version 0.4.4

runc --version

runc version 1.0.0-rc6
spec: 1.0.1-dev

[cyphar]

The resolv.conf mount is usually necessary, but I think the issue might be that the rootfs doesn't already have a resolv.conf and so runc gets a bit confused? I will take a look at this...

EDIT: Yes, the issue is that resolv.conf doesn't exist inside the rootfs -- and so when runc tries to deal with the mount it does a mkdir even though the source is a file. This is probably a runc bug (to be honest) but we could fix this in umoci by not adding a mount for resolv.conf if there is no /etc/resolv.conf in the container rootfs.

[rneilson]

I tried adding an empty resolv.conf to the rootfs as an empty file before running the container, and I get another, related error instead:

container_linux.go:344: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/resolv.conf\\\" to rootfs \\\"/data/contained/unpacked/busybox/rootfs\\\" at \\\"/data/contained/unpacked/busybox/rootfs/etc/resolv.conf\\\" caused \\\"mkdir /data/contained/unpacked/busybox/rootfs/etc/resolv.conf: not a directory\\\"\""

However, if I change the mount to "type": "bind", it works fine (with or without resolv.conf in the rootfs).

EDIT: I'm fairly sure it's due to how runc is parsing mount options in createLibcontainerMount, which leads to mountToRootfs using the default case, which does mkdir, instead of case "bind". So I think it could be fixed in umoci just by setting "type": "bind" instead.

[cyphar]

Right, yeah this sounds familiar.

opencontainers/runc#1753 should have fixed this (previously we'd treated type: bind as special which was incorrect for several reasons) but that fix only makes it so that an empty type is changed to bind (which is required because internally the rootfs_linux.go code heavily relies on type: bind).

The solution from the umoci side is to not add a type to the bind-mount or erroniously add type: bind (even though it should work regardless) -- but I will also fix this in runc because the current semantic is still wrong (bind-mounts don't have their own types -- so the ability to set a type makes no sense).

[cyphar]

What's really odd is that this definitely worked for a very long time -- there must've been a runc change which regressed this.

I've submitted opencontainers/runc#2035 which will fix the upstream issue, as well as openSUSE/umoci#295 which will fix the downstream issue in umoci. Let me know if it fixes the issue for you.

(I do plan to use runc as part of the integration tests but I've historically been a bit cautious about it, because Travis has had issues with runc's CI in the past.)

[rneilson]

Thanks. Wrangled a Go build[1], and seems to work. Expectedly, but still.

[1] The makefile is, for some reason, trying to install into /usr/bin/ instead of ~/bin/