Play with containerd

87_lazy_pulling/README.md

@@ -8,6 +8,8 @@ TODO:
 * nerdctl
 * does slim ai use it?
 * Test time to pull and run from registry.
+* NOTE: This does not work on linux - even using containerd.
+* Check how to get these examples working on ubuntu with containerd
 
 ## Reason
 
@@ -52,7 +54,7 @@ export BASEIMAGE=scratch
 #export BASEIMAGE=gcr.io/distroless/nodejs16-debian11
 export IMAGE_NAME_ZSTD=ttl.sh/$(uuidgen):1h
 export IMAGE_NAME_ZSTD=${IMAGE_NAME_ZSTD:l}
-export COMPRESSION_LEVEL=15
+export COMPRESSION_LEVEL=5
 
 docker build --build-arg=baseimage=$BASEIMAGE --build-arg=NIX_FILE=ffmpeg-full.nix --build-arg=PROGRAM_FILE=ffmpeg --progress=plain -f Dockerfile.ffmpeg --target PRODUCTION --output type=image,name=$IMAGE_NAME_ZSTD,oci-mediatypes=true,compression=zstd,compression-level=$COMPRESSION_LEVEL,force-compression=true,push=true .
 

90_containerd/.gitignore

@@ -0,0 +1 @@
+*.gz

90_containerd/README.md

@@ -0,0 +1,226 @@
+# CONTAINERD
+
+Demonstrate how to use `containerd`  
+
+TODO:
+
+* firecracker can be containerd plugin
+* dive does not work with containerd.  Why is this?
+* Get Estargz compression working on linux.
+* https://www.usenix.org/system/files/conference/fast16/fast16-papers-harter.pdf 
+* https://link.medium.com/Jl8qGEQPGyb 
+*https://github.com/containerd/stargz-snapshotter/issues/258 
+* https://github.com/containerd/stargz-snapshotter/blob/main/docs/ctr-remote.md 
+* https://github.com/containerd/containerd/blob/main/docs/rootless.md
+* https://github.com/containerd/stargz-snapshotter/blob/main/docs/INSTALL.md
+* Checked vendor/modules.txt to see what version was included https://github.com/moby/moby/tree/219f21bf07502b447095649b5a2764661737f164
+* https://github.com/containerd/stargz-snapshotter/blob/main/docs/overview.md
+* https://manpages.ubuntu.com/manpages/jammy/man8/mount.fuse.8.html
+
+## Install Docker Desktop
+
+On MacOS Docker Desktop it is possible to set docker to use containerd as the default backend.  
+
+## Install Ubuntu
+
+```sh
+# install docker prereqs
+sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
+
+# check containerd is running
+sudo systemctl status containerd
+
+# sort outaa group for using non-root containerd socket
+sudo groupadd containerd
+sudo usermod -a -G containerd $(whoami)
+sudo chown :containerd /var/run/containerd/containerd.sock
+sudo chmod 660 /var/run/containerd/containerd.sock
+sudo ls -l /var/run/containerd/containerd.sock
+id
+ctr version
+
+# ctr will be installed
+sudo ctr version     
+
+ctr 
+ctr plugins ls
+
+
+# run nginx
+docker run -d -p 8080:80 nginx  
+
+# docker
+docker ps
+# runc
+runc list
+# containerd
+sudo ctr containers ls   
+sudo ctr namespaces ls   
+sudo ctr --namespace moby containers ls 
+sudo ctr --namespace moby containers info 0a5ef6d3a499e420c00692ac04cfe84fbc5f1d96ebde8bf4765c0f5f3c9088dd
+
+
+sudo /home/linuxbrew/.linuxbrew/bin/nerdctl ps 
+
+dockerd --help
+
+# it looks like it is running containerd sock
+ps -aux | grep dockerd          
+```
+
+## Upgrade to Kernel 5.19
+
+```sh
+sudo apt-get install --install-recommends linux-generic-hwe-22.04 
+```
+
+## Configuration
+
+```sh
+cat /etc/docker/daemon.json
+cat /etc/containerd/config.toml  
+cat /etc/containerd-stargz-grpc/config.toml 
+```
+
+## Plugins
+
+Instructions for installing estargz snapshotter [here](https://github.com/containerd/stargz-snapshotter/blob/main/docs/INSTALL.md)  
+https://github.com/containerd/stargz-snapshotter/blob/main/script/config/etc/containerd/config.toml
+
+```sh
+sudo nano /etc/docker/daemon.json
+sudo nano /etc/containerd/config.toml  
+
+# list plugins
+sudo ctr plugins ls  
+```
+
+Goto stargz-snapshotter [releases](https://github.com/containerd/stargz-snapshotter/releases)  
+
+```sh
+sudo -s 
+
+version=v0.14.3
+arch=amd64
+# get plugin
+curl -Lo stargz-snapshotter-${version}-linux-${arch}.tar.gz https://github.com/containerd/stargz-snapshotter/releases/download/${version}/stargz-snapshotter-${version}-linux-${arch}.tar.gz
+
+tar -C /usr/local/bin -xvf stargz-snapshotter-${version}-linux-${arch}.tar.gz containerd-stargz-grpc ctr-remote
+curl -Lo ./stargz-snapshotter.service https://raw.githubusercontent.com/containerd/stargz-snapshotter/$version/script/config/etc/systemd/system/stargz-snapshotter.service
+cp ./stargz-snapshotter.service /etc/systemd/system/stargz-snapshotter.service 
+
+systemctl enable --now stargz-snapshotter
+systemctl restart containerd
+systemctl restart docker
+```
+
+Example stargz-snapshotter.service
+
+```ini
+[Unit]
+Description=stargz snapshotter
+After=network.target
+Before=containerd.service
+
+[Service]
+Type=notify
+Environment=HOME=/root
+ExecStart=/usr/local/bin/containerd-stargz-grpc --log-level=debug --config=/etc/containerd-stargz-grpc/config.toml
+Restart=always
+RestartSec=1
+
+[Install]
+WantedBy=multi-user.target
+```
+
+```toml
+# explicitly use v2 config format
+version = 2
+
+# - Set default runtime handler to v2, which has a per-pod shim
+# - Enable to use stargz snapshotter
+[plugins."io.containerd.grpc.v1.cri".containerd]
+  default_runtime_name = "runc"
+  snapshotter = "stargz"
+  disable_snapshot_annotations = false
+[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
+  runtime_type = "io.containerd.runc.v2"
+
+# Setup a runtime with the magic name ("test-handler") used for Kubernetes
+# runtime class tests ...
+[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.test-handler]
+  runtime_type = "io.containerd.runc.v2"
+
+# Use stargz snapshotter
+[proxy_plugins]
+  [proxy_plugins.stargz]
+    type = "snapshot"
+    address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"
+```
+
+journalctl -xeu stargz-snapshotter.service   
+
+sudo /usr/local/bin/containerd-stargz-grpc
+sudo dmesg
+
+{"error":"failed to mount overlay: invalid argument","level":"fatal","msg":"snapshotter is not supported","time":"2023-04-08T19:53:06.916442927+01:00"}
+
+overlayfs: upper fs does not support RENAME_WHITEOUT.
+overlayfs: upper fs is missing required features.
+
+
+
+
+https://github.com/canonical/microk8s/issues/1378
+
+https://discuss.linuxcontainers.org/t/run-docker-on-lxd-container/11575/7
+
+https://discuss.linuxcontainers.org/t/docker-overlay2-on-btrfs-supported/16771
+
+
+
+# it seems to create this folder.
+sudo ls -la /var/lib/containerd-stargz-grpc/snapshotter    
+
+sudo strace /usr/local/bin/containerd-stargz-grpc   
+
+ZFS is sadly quite painful with Docker in Docker and similar scenarios. It might be best to avoid the problem by creating a volume in your ZFS pool, formatting that volume to ext4, and having docker use "overlay2" on top of that, instead of "zfs".
+
+zfs create -s -V 20GB rpool/docker
+mkfs.ext4 /dev/zvol/rpool/docker
+
+mkdir -p /mnt/docker      
+
+sudo mount /dev/zvol/zroot/docker /mnt/docker   
+
+
+# add the mount to /etc/fstab
+mount /dev/zvol/zroot/docker /var/lib/docker
+The zfs create -s is for sparse volumes. Analogous to thin provisioning on LVM.
+
+I just finished setting this up and it nicely solves my problems with k3s and also kind. I use these for testing and development, and there the volume should be just fine.
+
+https://github.com/k3s-io/k3s/issues/66
+
+
+
+## nerdctl
+
+```sh
+# on linux or mac
+brew install nerdctl
+```
+
+## Resources
+
+* ctr - Man Page [here](https://www.mankier.com/8/ctr)
+* nerdctl: Docker-compatible CLI for containerd [here](https://github.com/containerd/nerdctl)  
+* Getting started with containerd [here](https://github.com/containerd/containerd/blob/main/docs/getting-started.md)
+* runc is a CLI tool for spawning and running containers on Linux according to the OCI specification. [here](https://github.com/opencontainers/runc)  
+* Why and How to Use containerd From Command Line [here](https://iximiuz.com/en/posts/containerd-command-line-clients/)
+* Extending Docker’s Integration with containerd [here](https://www.docker.com/blog/extending-docker-integration-with-containerd/)
+* https://docs.docker.com/engine/reference/commandline/dockerd/
+* https://medium.com/nttlabs/startup-containers-in-lightning-speed-with-lazy-image-distribution-on-containerd-243d94522361
+* https://www.tutorialworks.com/difference-docker-containerd-runc-crio-oci/?utm_content=cmp-true
+* https://docs.docker.com/engine/install/linux-postinstall/
+* https://github.com/containerd/nerdctl/blob/main/docs/stargz.md

90_containerd/flake.nix

@@ -0,0 +1,35 @@
+# flake.nix
+{
+    description = "A flake for working with containerd";
+  inputs = {
+    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
+  };
+
+  outputs = {nixpkgs, ...}: let
+    system = "x86_64-linux";
+    #       ↑ Swap it for your system if needed
+    #       "aarch64-linux" / "x86_64-darwin" / "aarch64-darwin"
+    pkgs = nixpkgs.legacyPackages.${system};
+  in {
+    devShells.${system}.default = pkgs.mkShell {
+
+      packages = [ 
+          pkgs.containerd
+          pkgs.runc
+       ];
+
+      env = {
+        #NIXPKGS_ALLOW_UNFREE=1;
+      };
+
+      shellHook = ''
+        echo "***************************************************"
+        echo "*** Welcome to containerd"
+        echo "***************************************************"
+        echo ""
+        ctr --version
+        runc --version
+      '';       
+    };
+  };
+}

90_containerd/justfile

@@ -0,0 +1,11 @@
+set dotenv-load := true
+
+# default lists actions
+default:
+  @just -f example.justfile --list
+
+nix:
+  #!/usr/bin/env bash
+  set -eufo pipefail
+  nix develop --command zsh
+

TODO.md

@@ -1,19 +1,20 @@
 # 📝 TODO
 
-1. pipewire??
-1. cgroups and namespaces
-1. Start a cgroup manually using an unpacked container.
-1. Can I use a cgroup command inside a container?
-1. quay.io and special labels  
-1. botb - break out the box image https://github.com/brompwnie/botb
-1. Show how multistage builds work for testing
-1. Demo using cache-from.....  Use a timed layer.
-1. Use different CAPS_SYS parameters.
-1. Build a apt-mirror https://www.tecmint.com/setup-local-repositories-in-ubuntu/
-1. Speed of different file system layers.  i.e. If we have a 1000 layers how fast is it to find a file?
-1. Diffing using different tools https://docs.docker.com/engine/reference/commandline/diff/
-1. Kaniko. Caching, speed?  
-2. Docker container escape with docker copy.
+1. Use packer to create an image.
+2. cgroups and namespaces
+3. Start a cgroup manually using an unpacked container.
+4. Can I use a cgroup command inside a container?
+5. quay.io and special labels  
+6. botb - break out the box image https://github.com/brompwnie/botb
+7. Show how multistage builds work for testing
+8. Demo using cache-from.....  Use a timed layer.
+9. Use different CAPS_SYS parameters.
+10. Build a apt-mirror https://www.tecmint.com/setup-local-repositories-in-ubuntu/
+11. Speed of different file system layers.  i.e. If we have a 1000 layers how fast is it to find a file?
+12. Diffing using different tools https://docs.docker.com/engine/reference/commandline/diff/
+13. Kaniko. Caching, speed?  
+14. Calculating accurate image sizes - looks like container-diff can output this.
+15. Docker container escape with docker copy.
 https://www.andreafortuna.org/2019/11/26/cve-2019-14271-a-docker-cp-container-escape-vulnerability/
 1. Add procdump to the sidecar
 2. mounting volumes and permissions.
@@ -35,4 +36,4 @@ https://www.andreafortuna.org/2019/11/26/cve-2019-14271-a-docker-cp-container-es
 18. https://github.com/tonistiigi/xx
 19. https://pythonspeed.com/articles/podman-buildkit/
 20. https://github.com/reproducible-containers/repro-sources-list.sh
-21. https://github.com/containers/toolbox
+21. https://github.com/containers/toolbox