Skip to content

Commit fc083ce

Browse files
dbougetmathildefaanesdbouget
authored
Cleanup positives (#13)
* Initial classification metrics computation running. The average fold metrics need to be computed now. [skip ci] * fixed study computation for All, Positive and TP (#12) * Updated the Study with new Positive/TP * Major changes to objectwise metrics computation, and added a standalone version[skip ci] * Working towards the new wheel build/test/upload... [skip ci] * Finished cleaning the unit tests [skip ci] --------- Co-authored-by: mathildefaanes <122807133+mathildefaanes@users.noreply.github.com> Co-authored-by: dbouget <david.bouget@sintef.no>
1 parent b214d26 commit fc083ce

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

41 files changed

+2041
-581
lines changed

‎.github/workflows/build.yml‎

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
name: Build Universal Wheel
2+
3+
on:
4+
push:
5+
branches: [ master,update ]
6+
pull_request:
7+
branches: [ master ]
8+
workflow_dispatch:
9+
10+
jobs:
11+
build:
12+
runs-on: ${{ matrix.os }}
13+
strategy:
14+
matrix:
15+
include:
16+
- os: ubuntu-22.04
17+
TARGET: ubuntu
18+
CMD_BUILD: python -m build --wheel
19+
20+
steps:
21+
- uses: actions/checkout@v4
22+
- name: Set up Python 3.9
23+
uses: actions/setup-python@v5
24+
with:
25+
python-version: "3.9"
26+
27+
- name: Install dependencies
28+
run: |
29+
pip install --upgrade pip
30+
pip install build
31+
32+
- name: Built wheel for ${{matrix.TARGET}}
33+
run: ${{matrix.CMD_BUILD}}
34+
35+
- name: Upload Python wheel
36+
uses: actions/upload-artifact@v4
37+
with:
38+
name: Python wheel
39+
path: ${{github.workspace}}/dist/raidionicsval-*.whl
40+
if-no-files-found: error
41+
42+
setup-test-data:
43+
runs-on: ubuntu-latest
44+
steps:
45+
- uses: actions/checkout@v4
46+
47+
- name: Set up Python
48+
uses: actions/setup-python@v5
49+
with:
50+
python-version: 3.9
51+
52+
- name: Download test resources
53+
working-directory: tests
54+
run: |
55+
pip install requests
56+
python -c "from download_resources import download_resources; download_resources('../test_data')"
57+
58+
- name: Upload test resources
59+
uses: actions/upload-artifact@v4
60+
with:
61+
name: test-resources
62+
path: ./test_data
63+
test:
64+
needs: [build, setup-test-data]
65+
runs-on: ${{ matrix.os }}
66+
strategy:
67+
matrix:
68+
include:
69+
- os: ubuntu-22.04
70+
python-version: "3.8"
71+
- os: ubuntu-22.04
72+
python-version: "3.9"
73+
- os: ubuntu-22.04
74+
python-version: "3.10"
75+
- os: ubuntu-22.04
76+
python-version: "3.11"
77+
- os: ubuntu-22.04
78+
python-version: "3.12"
79+
- os: ubuntu-24.04
80+
python-version: "3.8"
81+
- os: ubuntu-24.04
82+
python-version: "3.9"
83+
- os: ubuntu-24.04
84+
python-version: "3.10"
85+
- os: ubuntu-24.04
86+
python-version: "3.11"
87+
- os: ubuntu-24.04
88+
python-version: "3.12"
89+
- os: windows-2019
90+
python-version: "3.8"
91+
- os: windows-2019
92+
python-version: "3.9"
93+
- os: windows-2019
94+
python-version: "3.10"
95+
- os: windows-2019
96+
python-version: "3.11"
97+
- os: windows-2019
98+
python-version: "3.12"
99+
- os: windows-2022
100+
python-version: "3.8"
101+
- os: windows-2022
102+
python-version: "3.9"
103+
- os: windows-2022
104+
python-version: "3.10"
105+
- os: windows-2022
106+
python-version: "3.11"
107+
- os: windows-2022
108+
python-version: "3.12"
109+
- os: macos-13
110+
python-version: "3.8"
111+
- os: macos-13
112+
python-version: "3.9"
113+
- os: macos-13
114+
python-version: "3.10"
115+
- os: macos-13
116+
python-version: "3.11"
117+
- os: macos-13
118+
python-version: "3.12"
119+
- os: macos-14
120+
python-version: "3.10"
121+
- os: macos-14
122+
python-version: "3.11"
123+
- os: macos-15
124+
python-version: "3.10"
125+
- os: macos-15
126+
python-version: "3.11"
127+
128+
steps:
129+
- name: Set up Python ${{ matrix.python-version }}
130+
uses: actions/setup-python@v5
131+
with:
132+
python-version: ${{ matrix.python-version }}
133+
134+
- name: Download artifact
135+
uses: actions/download-artifact@v4
136+
with:
137+
name: "Python wheel"
138+
139+
- name: Install wheel
140+
run: |
141+
python -m pip install --upgrade pip
142+
pip install --find-links=${{github.workspace}} --no-cache-dir --force-reinstall raidionicsval-*
143+
shell: bash
144+
145+
- name: Test CLI
146+
run: raidionicsseg --help
147+
148+
- name: Clone repo
149+
uses: actions/checkout@v4
150+
151+
- name: Download test resources
152+
uses: actions/download-artifact@v4
153+
with:
154+
name: test-resources
155+
path: ./tests/unit_tests_results_dir
156+
157+
- name: Integration tests
158+
run: |
159+
pip install pytest pytest-cov pytest-timeout requests
160+
pytest -vvv --cov=raidionicsval ${{github.workspace}}/tests/generic_tests --cov-report=xml --timeout=1000 --log-cli-level=DEBUG
161+
162+
- name: Upload coverage to Codecov
163+
if: ${{ matrix.os == 'ubuntu-22.04' && matrix.python-version == '3.9' }}
164+
uses: codecov/codecov-action@v4
165+
with:
166+
token: ${{ secrets.CODECOV_TOKEN }}
167+
slug: dbouget/validation_metrics_computation
168+
verbose: true

‎.github/workflows/build_macos.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ jobs:
7878
uses: actions/checkout@v1
7979

8080
- name: k-fold cross-validation unit test
81-
run: cd ${{github.workspace}}/tests && python validation_pipeline_test.py
81+
run: cd ${{github.workspace}}/tests && python test_validation_pipeline.py
8282

8383
- name: Segmentation study unit test
84-
run: cd ${{github.workspace}}/tests && python studies_pipeline_test.py
84+
run: cd ${{github.workspace}}/tests && python test_studies_pipeline.py

‎.github/workflows/build_macos_arm.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ jobs:
7373
uses: actions/checkout@v1
7474

7575
- name: k-fold cross-validation unit test
76-
run: cd ${{github.workspace}}/tests && python3 validation_pipeline_test.py
76+
run: cd ${{github.workspace}}/tests && python3 test_validation_pipeline.py
7777

7878
- name: Segmentation study unit test
79-
run: cd ${{github.workspace}}/tests && python3 studies_pipeline_test.py
79+
run: cd ${{github.workspace}}/tests && python3 test_studies_pipeline.py

‎.github/workflows/build_ubuntu.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ jobs:
7474
uses: actions/checkout@v1
7575

7676
- name: k-fold cross-validation unit test
77-
run: cd ${{github.workspace}}/tests && python validation_pipeline_test.py
77+
run: cd ${{github.workspace}}/tests && python test_validation_pipeline.py
7878

7979
- name: Segmentation study unit test
80-
run: cd ${{github.workspace}}/tests && python studies_pipeline_test.py
80+
run: cd ${{github.workspace}}/tests && python test_studies_pipeline.py

‎.github/workflows/build_windows.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ jobs:
7474
uses: actions/checkout@v1
7575

7676
- name: k-fold cross-validation unit test
77-
run: cd ${{github.workspace}}/tests && python validation_pipeline_test.py
77+
run: cd ${{github.workspace}}/tests && python test_validation_pipeline.py
7878

7979
- name: Segmentation study unit test
80-
run: cd ${{github.workspace}}/tests && python studies_pipeline_test.py
80+
run: cd ${{github.workspace}}/tests && python test_studies_pipeline.py

‎.github/workflows/publish.yml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: Build and Publish Universal Wheel
2+
3+
on:
4+
release:
5+
types: [published] # Automatically when you publish a GitHub Release
6+
workflow_dispatch: # Allow manual triggering too
7+
8+
jobs:
9+
build-publish:
10+
runs-on: ubuntu-latest
11+
12+
steps:
13+
- name: Checkout code
14+
uses: actions/checkout@v4
15+
16+
- name: Set up Python
17+
uses: actions/setup-python@v5
18+
with:
19+
python-version: '3.10'
20+
21+
- name: Install build tool
22+
run: pip install build
23+
24+
- name: Build wheel and source distribution
25+
run: python -m build --sdist --wheel --outdir dist
26+
27+
- name: Check the built files
28+
run: ls -lh dist
29+
30+
- name: Publish to PyPI
31+
uses: pypa/gh-action-pypi-publish@release/v1
32+
with:
33+
password: ${{ secrets.PYPI_API_TOKEN }}
34+
packages-dir: dist

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,4 @@ numpy
2424
/.ipynb_checkpoints
2525
/build
2626
*egg-info
27+
unit_tests_results_dir/

‎Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM python:3.8-slim
1+
FROM python:3.9-slim
22

33
MAINTAINER David Bouget <david.bouget@sintef.no>
44

‎LICENSE.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
BSD 2-Clause License
2+
3+
Copyright (c) 2022, SINTEF Digital, Health Research.
4+
All rights reserved.
5+
6+
Redistribution and use in source and binary forms, with or without
7+
modification, are permitted provided that the following conditions are met:
8+
9+
1. Redistributions of source code must retain the above copyright notice, this
10+
list of conditions and the following disclaimer.
11+
12+
2. Redistributions in binary form must reproduce the above copyright notice,
13+
this list of conditions and the following disclaimer in the documentation
14+
and/or other materials provided with the distribution.
15+
16+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
17+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
19+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
20+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
21+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
22+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
23+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
24+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
25+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎blank_main_config.ini‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
task= # Task to perform, to sample from [study, validation]
33
data_root= # Path to the folder containing the raw dataset, organized according to the guidelines from the README.md
44
number_processes= # Number of processes to use in parallel for computation
5+
objective= # Validation task to run, to sample from [classification, segmentation]
56

67
[Studies]
78
input_folder= # Path to the folder containing the results from the Validation step (i.e., output_folder from [Validation])
@@ -26,3 +27,14 @@ extra_metrics= # Comma-separated list of metrics to compute, to sample from [TP
2627
class_names= # Comma-separated list of strings with the names of each segmented class
2728
tiny_objects_removal_threshold= # Integer representing the minimum number of voxels an object must have to be kept as an object
2829
true_positive_volume_thresholds= # Comma-separated list of float for cut-off values to apply to each class to consider them as true positives or not
30+
use_brats_data=
31+
32+
[Standalone]
33+
groundtruth_filename=
34+
prediction_filename=
35+
class_names=
36+
metrics_space=
37+
extra_metrics=
38+
detection_overlap_thresholds=
39+
tiny_objects_removal_threshold=
40+
true_positive_volume_thresholds=

0 commit comments

Comments
 (0)