✨ tilt-prepare: add a flag to skip image preloading

[bnallapeta]

What this PR does / why we need it:

On Apple Silicon Macs, kind load docker-image fails for multi-arch images (like cert-manager) because Docker Desktop only stores the arm64 layers locally. This causes ctr to fail when it tries to import all architectures, resulting in a "content digest not found" error that blocks tilt up.
This PR updates tilt-prepare to skip the preloading step specifically on darwin/arm64. Instead of preloading, we rely on the Kubelet to pull the images on-demand when the pods start. This bypasses the kind/docker import issue and allows the development environment to start correctly.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #13142
Refs: kubernetes-sigs/kind#3795

/area devtools

[chrischdi]

What this PR does / why we need it:

On Apple Silicon Macs, kind load docker-image fails for multi-arch images (like cert-manager) because Docker Desktop only stores the arm64 layers locally. This causes ctr to fail when it tries to import all architectures, resulting in a "content digest not found" error that blocks tilt up. This PR updates tilt-prepare to skip the preloading step specifically on darwin/arm64. Instead of preloading, we rely on the Kubelet to pull the images on-demand when the pods start. This bypasses the kind/docker import issue and allows the development environment to start correctly.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes #13142 Refs: kubernetes-sigs/kind#3795

/area devtools

Note: this issue is also relevant when using docker community edition >= version 29 for any OS.

I think the workaround used in CI should work here too: using crane pull "$image" "image.tar" to pull the image and docker load -i "image.tar" to load it.

Afterwards preload work.

[bnallapeta]

What this PR does / why we need it:
On Apple Silicon Macs, kind load docker-image fails for multi-arch images (like cert-manager) because Docker Desktop only stores the arm64 layers locally. This causes ctr to fail when it tries to import all architectures, resulting in a "content digest not found" error that blocks tilt up. This PR updates tilt-prepare to skip the preloading step specifically on darwin/arm64. Instead of preloading, we rely on the Kubelet to pull the images on-demand when the pods start. This bypasses the kind/docker import issue and allows the development environment to start correctly.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes #13142 Refs: kubernetes-sigs/kind#3795
/area devtools

Note: this issue is also relevant when using docker community edition >= version 29 for any OS.

I think the workaround used in CI should work here too: using crane pull "$image" "image.tar" to pull the image and docker load -i "image.tar" to load it.

Afterwards preload work.

Although crane would work, I am trying to avoid any dependencies for the user. Not everybody knows/uses crane in their local.
Secondly, even if we were to automate this, right now, crane isn't a tool in the tilt up workflow unlike the CI workflow.

That's why I chose the skipping way since that's probably the simplest and most straight forward. It just relies on the standard way of kubelet pulling images.

[chrischdi]

The thing is this does not exclusively fail for runtime.GOOS == "darwin" && runtime.GOARCH == "arm64" , it can also fail for any variant of running against docker. So with that arguing we could say "we should remove it for all".

The issue here is not >how we load the images< (which is part of the tooling) it is >how we pull the images<.

I'd favor a toggle-able option to disable it if needed. Long-term this should work.

[bnallapeta]

@chrischdi Got it. I have now updated the PR so that users could provide a flag in tilt-settings.yaml. PTAL.

[fabriziopandini]

Can we not create the tasks entirely when the flag is set? (vs creating task that are no op)

[bnallapeta]

You're right. I modified the code to now conditionally execute this based on the bool. Also modified preLoadImageTask function accordingly.

[bnallapeta]

@chrischdi @fabriziopandini PTAL.

[fabriziopandini]

I personally think the approach suggested by @chrischdi (use crane) provides better results for iterative development, because you don't have to download the image every time you do kind-up.

However, considering cert-manager is not a strong requirement for CAPI, and in theory it that can be replaced or entirely removed, I'm ok to make it possible to skip cert-manager pre-load tasks.

/lgtm
/approve

[k8s-ci-robot]

LGTM label has been added.

Details

Git tree hash: b682422c12c6e0175d34c11ed15d327e9d36f012

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fabriziopandini

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [fabriziopandini]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment