Docker support (#8)

* Starting to include Docker support [skip ci]

* Docker unit test running, but permission issues to delete the results folder (created folders/files belong to root....) [skip ci]

* Updating the readme [skip ci]

* Adding Docker call parameter for read/write permissions [skip ci]

Author: dbouget

.dockerignore

@@ -0,0 +1,5 @@
+venv/
+venv*
+*unit_tests_results_dir/
+notebooks/
+tests/

Dockerfile

@@ -0,0 +1,21 @@
+FROM python:3.8-slim
+
+MAINTAINER David Bouget <david.bouget@sintef.no>
+
+ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
+
+RUN apt-get update -y
+RUN apt-get upgrade -y
+RUN apt-get -y install sudo
+RUN apt-get update && apt-get install -y git
+
+WORKDIR /workspace
+
+RUN git clone https://github.com/dbouget/validation_metrics_computation.git
+RUN pip3 install --upgrade pip
+RUN pip3 install -e validation_metrics_computation/
+
+RUN mkdir /workspace/resources
+
+# CMD ["/bin/bash"]
+ENTRYPOINT ["python3", "/workspace/validation_metrics_computation/main.py"]

README.md

@@ -192,21 +192,60 @@ An additional explanation of all parameters specified in the configuration file
 found in _/Utils/resources.py_. 
 
 </details>
-
+ 
 <details>
 <summary>
 
 ### [3. Process](https://github.com/dbouget/validation_metrics_computation#3-process)
 </summary>
-To run, you need to supply the configuration file as parameter.
 
+After filling in the configuration file specifying all runtime parameters,
+according to the pattern from [**blank_main_config.ini**](https://github.com/dbouget/validation_metrics_computation/blob/master/blank_main_config.ini),
+you should run first the __validation__ task and then the __study__ task.  
+
+
+#### [CLI](https://github.com/dbouget/validation_metrics_computation#cli)
+```
+raidionicsval -c CONFIG (-v debug)
+```
+
+CONFIG should point to a configuration file (*.ini).
+
+#### [Python module](https://github.com/dbouget/validation_metrics_computation#python-module)
+```
+from raidionicsval import compute
+compute(config_filename="/path/to/main_config.ini")
+```
+
+"/path/to/main_config.ini" should point to a valid configuration file.
+
+#### [Docker](https://github.com/dbouget/validation_metrics_computation#docker)
+When calling Docker images, the --user flag must be properly used in order for the folders and files created inside
+the container to inherit the proper read/write permissions. The user ID is retrieved on-the-fly in the following
+examples, but it can be given in a more hard-coded fashion if known by the user.
+
+```
+docker pull dbouget/raidionics-val:v1.0-py38-cpu
+```
+
+For opening the Docker image and interacting with it, run:  
+```
+docker run --entrypoint /bin/bash -v /home/<username>/<resources_path>:/workspace/resources -t -i --network=host --ipc=host --user $(id -u) dbouget/raidionics-val:v1.0-py38-cpu
+```
+
+The `/home/<username>/<resources_path>` before the column sign has to be changed to match a directory on your local 
+machine containing the data to expose to the docker image. Namely, it must contain folder(s) with data to use as input
+for the validation studies, and it will contain the destination folder where the results will be saved.
+
+For launching the Docker image as a CLI, run:  
 ```
-  python main.py -c main_config.ini (-v debug)
+docker run -v /home/<username>/<resources_path>:/workspace/resources -t -i --network=host --ipc=host --user $(id -u) dbouget/raidionics-val:v1.0-py38-cpu -c /workspace/resources/<path>/<to>/main_config.ini -v <verbose>
 ```
 
-After filling in the configuration file, you should run first the 
-__validation__ task and then the __study__ task.  
-N.B. If no study fits your need, you can create a new study file in _/Studies/_.
+The `<path>/<to>/main_config.ini` must point to a valid configuration file on your machine, as a relative path to the `/home/<username>/<resources_path>` described above.
+For example, if the file is located on my machine under `/home/myuser/Data/Validation/main_config.ini`, 
+and that `/home/myuser/Data` is the mounted resources partition mounted on the Docker image, the new relative path will be `Validation/main_config.ini`.  
+The `<verbose>` level can be selected from [debug, info, warning, error].
 
 </details>
 

tests/validation_pipeline_test.py

@@ -43,7 +43,8 @@ def validation_pipeline_test():
             raise ValueError('Resources download or extraction failed, content not available on disk.')
     except Exception as e:
         logging.error("Error during resources download with: \n {}.\n".format(traceback.format_exc()))
-        shutil.rmtree(test_dir)
+        if os.path.exists(test_dir):
+            shutil.rmtree(test_dir)
         raise ValueError("Error during resources download.\n")
 
     logging.info("Preparing configuration file.\n")
@@ -80,7 +81,8 @@ def validation_pipeline_test():
         scores_filename = os.path.join(test_dir, 'StudyResults', 'Validation', 'all_dice_scores.csv')
         if not os.path.exists(scores_filename):
             logging.error("k-fold cross-validation unit test failed, no scores were generated.\n")
-            shutil.rmtree(test_dir)
+            if os.path.exists(test_dir):
+                shutil.rmtree(test_dir)
             raise ValueError("k-fold cross-validation unit test failed, no scores were generated.\n")
 
         logging.info("k-fold cross-validation CLI unit test started.\n")
@@ -100,23 +102,27 @@ def validation_pipeline_test():
                                        '--verbose', 'debug'])
         except Exception as e:
             logging.error("Error during k-fold cross-validation CLI unit test with: \n {}.\n".format(traceback.format_exc()))
-            shutil.rmtree(test_dir)
+            if os.path.exists(test_dir):
+                shutil.rmtree(test_dir)
             raise ValueError("Error during k-fold cross-validation CLI unit test.\n")
 
         logging.info("Collecting and comparing results.\n")
         scores_filename = os.path.join(test_dir, 'StudyResults', 'Validation', 'all_dice_scores.csv')
         if not os.path.exists(scores_filename):
             logging.error("k-fold cross-validation CLI unit test failed, no scores were generated.\n")
-            shutil.rmtree(test_dir)
+            if os.path.exists(test_dir):
+                shutil.rmtree(test_dir)
             raise ValueError("k-fold cross-validation CLI unit test failed, no scores were generated.\n")
         logging.info("k-fold cross-validation CLI unit test succeeded.\n")
     except Exception as e:
         logging.error("Error during k-fold cross-validation unit test with: \n {}.\n".format(traceback.format_exc()))
-        shutil.rmtree(test_dir)
+        if os.path.exists(test_dir):
+            shutil.rmtree(test_dir)
         raise ValueError("Error during k-fold cross-validation unit test with.\n")
 
     logging.info("k-fold cross-validation unit test succeeded.\n")
-    shutil.rmtree(test_dir)
+    if os.path.exists(test_dir):
+        shutil.rmtree(test_dir)
 
 
 validation_pipeline_test()

tests/validation_pipeline_test_docker.py

@@ -0,0 +1,120 @@
+import os
+import shutil
+import configparser
+import logging
+import sys
+import subprocess
+import traceback
+import zipfile
+
+try:
+    import requests
+except:
+    subprocess.check_call([sys.executable, "-m", "pip", "install", 'requests'])
+    import requests
+
+
+def inference_test_docker():
+    """
+    Testing the CLI within a Docker container for the validation unit test, running on CPU.
+    The latest Docker image is being hosted at: dbouget/raidionics-val:v1.0-py38-cpu
+
+    Returns
+    -------
+
+    """
+    logging.basicConfig()
+    logging.getLogger().setLevel(logging.DEBUG)
+    logging.info("Running validation unit test in Docker container.\n")
+    logging.info("Downloading unit test resources.\n")
+    test_dir = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'unit_tests_results_dir')
+    if os.path.exists(test_dir):
+        shutil.rmtree(test_dir)
+    os.makedirs(test_dir)
+
+    try:
+        resources_url = 'https://github.com/raidionics/Raidionics-models/releases/download/1.2.0/Samples-RaidionicsValLib_UnitTest1.zip'
+
+        archive_dl_dest = os.path.join(test_dir, 'resources.zip')
+        headers = {}
+        response = requests.get(resources_url, headers=headers, stream=True)
+        response.raise_for_status()
+        if response.status_code == requests.codes.ok:
+            with open(archive_dl_dest, "wb") as f:
+                for chunk in response.iter_content(chunk_size=1048576):
+                    f.write(chunk)
+        with zipfile.ZipFile(archive_dl_dest, 'r') as zip_ref:
+            zip_ref.extractall(test_dir)
+
+        if not os.path.exists(os.path.join(test_dir, 'Input_dataset')):
+            raise ValueError('Resources download or extraction failed, content not available on disk.')
+    except Exception as e:
+        logging.error("Error during resources download with: \n {}.\n".format(traceback.format_exc()))
+        if os.path.exists(test_dir):
+            shutil.rmtree(test_dir)
+        raise ValueError("Error during resources download.\n")
+
+    logging.info("Preparing configuration file.\n")
+    try:
+        val_config = configparser.ConfigParser()
+        val_config.add_section('Default')
+        val_config.set('Default', 'task', "validation")
+        val_config.set('Default', 'data_root', '/workspace/resources/Input_dataset')
+        val_config.set('Default', 'number_processes', '1')
+        val_config.add_section('Validation')
+        val_config.set('Validation', 'input_folder', '/workspace/resources/StudyResults')
+        val_config.set('Validation', 'output_folder', '/workspace/resources/StudyResults')
+        val_config.set('Validation', 'gt_files_suffix', 'label_tumor.nii.gz')
+        val_config.set('Validation', 'prediction_files_suffix', 'pred_tumor.nii.gz')
+        val_config.set('Validation', 'use_index_naming_convention', 'false')
+        val_config.set('Validation', 'nb_folds', '1')
+        val_config.set('Validation', 'split_way', 'three-way')
+        val_config.set('Validation', 'detection_overlap_thresholds', '0.')
+        val_config.set('Validation', 'metrics_space', 'pixelwise, objectwise')
+        val_config.set('Validation', 'class_names', 'tumor')
+        val_config.set('Validation', 'extra_metrics', 'IOU')
+        val_config.set('Validation', 'tiny_objects_removal_threshold', '25')
+        val_config.set('Validation', 'true_positive_volume_thresholds', '0.1')
+        val_config.set('Validation', 'use_brats_data', 'false')
+        val_config_filename = os.path.join(test_dir, 'test_val_config.ini')
+        with open(val_config_filename, 'w') as outfile:
+            val_config.write(outfile)
+
+        logging.info("Running validation unit test in Docker container.\n")
+        try:
+            import platform
+            cmd_docker = ['docker', 'run', '-v', '{}:/workspace/resources'.format(test_dir),
+                          '--network=host', '--ipc=host', '--user', str(os.geteuid()),
+                          'dbouget/raidionics-val:v1.0-py38-cpu',
+                          '-c', '/workspace/resources/test_val_config.ini', '-v', 'debug']
+            logging.info("Executing the following Docker call: {}".format(cmd_docker))
+            if platform.system() == 'Windows':
+                subprocess.check_call(cmd_docker, shell=True)
+            else:
+                subprocess.check_call(cmd_docker)
+        except Exception as e:
+            logging.error("Error during validation test in Docker container with: \n {}.\n".format(traceback.format_exc()))
+            if os.path.exists(test_dir):
+                shutil.rmtree(test_dir)
+            raise ValueError("Error during validation test in Docker container.\n")
+
+        logging.info("Collecting and comparing results.\n")
+        scores_segmentation_filename = os.path.join(test_dir, 'StudyResults', 'Validation', 'all_dice_scores.csv')
+        if not os.path.exists(scores_segmentation_filename):
+            logging.error("Validation in Docker container failed, no dice scores were generated.\n")
+            if os.path.exists(test_dir):
+                shutil.rmtree(test_dir)
+            raise ValueError("Validation in Docker container failed, no dice scores were generated.\n")
+        logging.info("Validation unit test in Docker container succeeded.\n")
+    except Exception as e:
+        logging.error("Error during validation in Docker container with: \n {}.\n".format(traceback.format_exc()))
+        if os.path.exists(test_dir):
+            shutil.rmtree(test_dir)
+        raise ValueError("Error during validation in Docker container with.\n")
+
+    logging.info("Validation unit test in Docker container succeeded.\n")
+    if os.path.exists(test_dir):
+        shutil.rmtree(test_dir)
+
+
+inference_test_docker()