Add segmentation container and publish workflow

[pradyumnae]

Summary

Adds a segmentation container and GitHub Actions workflow to build and publish it to GHCR.

What’s included

• segmentation/ directory with Dockerfile and reference notebook
• New workflow: .github/workflows/publish-segmentation-image.yaml
• Image published as: ghcr.io/als-computing/microct-segmentation

Notes

• Model weights are not included in the repo or image
• Weights should be provided via NERSC filesystem and WEIGHTS_PATH

@dabramov @taxe10 @Wiebke
PR is ready for review — thank you!

[davramov]

I am building the image on Perlmutter now, and may have more notes after that is complete.

Can you please add a README.md to the segmentation/ folder that walks through building and running this locally as well as on NERSC?

[davramov]

I tried building the image locally, but encountered an error when it tried to install detectron2:

(base)  🐍 base  david@dabramov-mac  ~/Documents/code/fork/segmentation-image/microct/segmentation   add-segmentation-container  docker compose up
Compose now can delegate build to bake for better performances
Just set COMPOSE_BAKE=true
[+] Building 761.4s (9/12)                                                                                                                                                                                                     docker:desktop-linux
 => [segmentation internal] load build definition from Dockerfile                                                                                                                                                                              0.0s
 => => transferring dockerfile: 2.37kB                                                                                                                                                                                                         0.0s
 => [segmentation internal] load metadata for docker.io/library/ubuntu:latest                                                                                                                                                                  1.6s
 => [segmentation internal] load .dockerignore                                                                                                                                                                                                 0.0s
 => => transferring context: 279B                                                                                                                                                                                                              0.0s
 => [segmentation 1/9] FROM docker.io/library/ubuntu:latest@sha256:cd1dba651b3080c3686ecf4e3c4220f026b521fb76978881737d24f200828b2b                                                                                                            6.9s
 => => resolve docker.io/library/ubuntu:latest@sha256:cd1dba651b3080c3686ecf4e3c4220f026b521fb76978881737d24f200828b2b                                                                                                                         0.0s
 => => sha256:cd1dba651b3080c3686ecf4e3c4220f026b521fb76978881737d24f200828b2b 6.69kB / 6.69kB                                                                                                                                                 0.0s
 => => sha256:a4453623f2f8319cfff65c43da9be80fe83b1a7ce689579b475867d69495b782 424B / 424B                                                                                                                                                     0.0s
 => => sha256:493218ed0f404132311952996fea8ce85e50c49f5a717f26f25c52a25fcb2e56 2.30kB / 2.30kB                                                                                                                                                 0.0s
 => => sha256:a3629ac5b9f4680dc2032439ff2354e73b06aecc2e68f0035a2d7c001c8b4114 29.73MB / 29.73MB                                                                                                                                               6.0s
 => => extracting sha256:a3629ac5b9f4680dc2032439ff2354e73b06aecc2e68f0035a2d7c001c8b4114                                                                                                                                                      0.8s
 => [segmentation 2/9] WORKDIR /opt                                                                                                                                                                                                            0.1s
 => [segmentation 3/9] RUN     apt-get update && apt-get install --yes     build-essential     gfortran     git     wget     libgl1 libegl1 libxext6 libsm6 libxrender1 &&     apt-get clean all && rm -rf /var/lib/apt/lists/*              128.2s
 => [segmentation 4/9] RUN wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh -O /tmp/miniforge.sh &&     bash /tmp/miniforge.sh -b -p /opt/miniconda &&     rm /tmp/miniforge.sh              19.6s 
 => [segmentation 5/9] RUN mamba install --yes -c conda-forge -c astra-toolbox -c simpleitk -c pytorch -c nvidia     python=3.10     jupyterlab astropy cartopy cython cfitsio "dask[distributed]" scipy scikit-learn scikit-image numba h5  600.1s 
 => ERROR [segmentation 6/9] RUN python -m pip install --no-build-isolation 'git+https://github.com/facebookresearch/detectron2.git'                                                                                                           4.7s 
------                                                                                                                                                                                                                                              
 > [segmentation 6/9] RUN python -m pip install --no-build-isolation 'git+https://github.com/facebookresearch/detectron2.git':                                                                                                                      
0.787 Collecting git+https://github.com/facebookresearch/detectron2.git                                                                                                                                                                             
0.788   Cloning https://github.com/facebookresearch/detectron2.git to /tmp/pip-req-build-gznnp_m2                                                                                                                                                   
0.826   Running command git clone --filter=blob:none --quiet https://github.com/facebookresearch/detectron2.git /tmp/pip-req-build-gznnp_m2                                                                                                         
3.087   Resolved https://github.com/facebookresearch/detectron2.git to commit fd27788985af0f4ca800bca563acdb700bb890e2                                                                                                                              
3.123   Preparing metadata (pyproject.toml): started
4.287   Preparing metadata (pyproject.toml): finished with status 'error'
4.294   error: subprocess-exited-with-error
4.294   
4.294   × Preparing metadata (pyproject.toml) did not run successfully.
4.294   │ exit code: -6
4.294   ╰─> [2 lines of output]
4.294       OMP: Error #13: Assertion failure at kmp_affinity.cpp(981).
4.294       OMP: Hint Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see https://github.com/llvm/llvm-project/issues/.
4.294       [end of output]
4.294   
4.294   note: This error originates from a subprocess, and is likely not a problem with pip.
4.627 error: metadata-generation-failed
4.627 
4.627 × Encountered error while generating package metadata.
4.627 ╰─> from git+https://github.com/facebookresearch/detectron2.git
4.627 
4.627 note: This is an issue with the package mentioned above, not pip.
4.627 hint: See above for details.
------
failed to solve: process "/bin/sh -c python -m pip install --no-build-isolation 'git+https://github.com/facebookresearch/detectron2.git'" did not complete successfully: exit code: 1

[davramov]

To get the Dockerfile to build locally on my Macbook, I had to set KMP_AFFINITY=disabled during the detectron2 install step, as well as in the notebook.

Dockerfile:

RUN KMP_AFFINITY=disabled OMP_NUM_THREADS=1 \
python -m pip install --no-build-isolation 'git+https://github.com/facebookresearch/detectron2.git'

after import os in the notebook (otherwise the kernel would crash):

os.environ["KMP_AFFINITY"] = "disabled"

I believe this should only be an issue when building on Mac, but it should be fine when building the image using GitHub actions and pulling on NERSC or another Linux machine.