Skip to content

Commit 8590819

Browse files
v3.2.4 (#223)
* Refactor Map2ModelWarraper into Topology class (removed dependency to MapData) (#222) * sampler * fix extract_geology_contacts * fix calculate_fault_orientations and summarise_fault_data * fix dtm data type * fix get_value_from_raster import * style: style fixes by ruff and autoformatting by black * revert get_value_from_raster * move dtm and geology data parameters from sample function to constructor * Refactor contact extraction * refactor: update related method calls * refactor: simplify extract_basal_contacts method signature and logic * refactor: streamline contact extraction * refactor: update basal contacts references to use contact extractor * refactor: remove unused import of ContactExtractor * refactor: update basal contacts plotting to use contact extractor * refactor: refactor map2model wrapper to topology module * refactor: replace map2model with topology * test: add unit tests for Topology * refactor: update tests * remove self.map_data from topology class * fix variable naming * fix private attribute in topology --------- Co-authored-by: Noelle Cheng <noelle.cheng@monash.edu> Co-authored-by: noellehmcheng <143368485+noellehmcheng@users.noreply.github.com> * fix from codex/refactor-extract-methods-into-contactextractor-class * fix test_contact_extractor.py * style: style fixes by ruff and autoformatting by black * return basal contact in extract_geology_contacts functions * fix typing in ContactExtractor parameter for python 3.9 compatibility * fix conda package file extension * refactor: update basal contacts extraction * working linting_and_testing.yml from testing_v3.3.0 * Thickness Calculators - Removed Mapdata Dependency (#224) * refactor: remove MapData dependency from thickness calculators * refactor: update initialisation of thickness calculator * refactor: update tests * fix parameters pass to value_from_raster in thickness calculator * fix thickness calculator tests --------- Co-authored-by: Noelle Cheng <noelle.cheng@monash.edu> * style: style fixes by ruff and autoformatting by black * fix: use correct type hint * fix sampled_contacts in thickness tests * style: style fixes by ruff and autoformatting by black * fix test_calculate_unit_thicknesses * fix: update fault and fold allowed keys to include additional columns * fix: add regex handling * fix: thickness calculator to use DTM data and bounding box --------- Co-authored-by: Noelle Cheng <noelle.cheng@monash.edu> Co-authored-by: noellehmcheng <143368485+noellehmcheng@users.noreply.github.com>
1 parent 852ded9 commit 8590819

17 files changed

Lines changed: 534 additions & 308 deletions

.github/workflows/conda.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ jobs:
3333
conda install -c conda-forge conda-build scikit-build-core numpy anaconda-client conda-libmamba-solver -y
3434
conda config --set solver libmamba
3535
conda build --output-folder conda conda --python ${{matrix.python-version}}
36-
anaconda upload --label main conda/*/*.tar.bz2
36+
anaconda upload --label main conda/*/*.conda
3737
3838
- name: upload artifacts
3939
uses: actions/upload-artifact@v4

.github/workflows/linting_and_testing.yml

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -48,28 +48,33 @@ jobs:
4848
with:
4949
python-version: ${{ matrix.python-version }}
5050
conda-remove-defaults: "true"
51-
52-
53-
- name: Install dependencies for windows python 3.10
54-
if: ${{ matrix.os == 'windows-latest' && matrix.python-version == '3.10' }}
51+
52+
- name: Install dependencies for windows python 3.9 and 3.10
53+
if: ${{ matrix.os == 'windows-latest' && (matrix.python-version == '3.9' || matrix.python-version == '3.10') }}
5554
run: |
5655
conda run -n test conda info
5756
conda run -n test conda install -c loop3d -c conda-forge "gdal=3.4.3" python=${{ matrix.python-version }} -y
5857
conda run -n test conda install -c loop3d -c conda-forge --file dependencies.txt python=${{ matrix.python-version }} -y
5958
conda run -n test conda install pytest python=${{ matrix.python-version }} -y
6059
61-
- name: Install dependencies for other environments
62-
if: ${{ matrix.os != 'windows-latest' || matrix.python-version != '3.10' }}
60+
- name: Install dependencies for windows python 3.11
61+
if: ${{ matrix.os == 'windows-latest' && matrix.python-version == '3.11' }}
6362
run: |
6463
conda run -n test conda info
65-
conda run -n test conda install -c loop3d -c conda-forge gdal python=${{ matrix.python-version }} -y
64+
conda run -n test conda install -c loop3d -c conda-forge gdal "netcdf4=*=nompi_py311*" python=${{ matrix.python-version }} -y
6665
conda run -n test conda install -c loop3d -c conda-forge --file dependencies.txt python=${{ matrix.python-version }} -y
6766
conda run -n test conda install pytest python=${{ matrix.python-version }} -y
67+
68+
- name: Install dependencies for other environments
69+
if: ${{ !(matrix.os == 'windows-latest' && (matrix.python-version == '3.9' || matrix.python-version == '3.10' || matrix.python-version == '3.11')) }}
70+
run: |
71+
conda run -n test conda info
72+
conda run -n test conda install -c loop3d -c conda-forge python=${{ matrix.python-version }} gdal pytest --file dependencies.txt -y
6873
6974
- name: Install map2loop
7075
run: |
7176
conda run -n test python -m pip install .
7277
7378
- name: Run tests
7479
run: |
75-
conda run -n test pytest
80+
conda run -n test pytest -v

map2loop/contact_extractor.py

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
import geopandas
2+
import pandas
3+
import shapely
4+
from .logging import getLogger
5+
from typing import Optional
6+
7+
logger = getLogger(__name__)
8+
9+
class ContactExtractor:
10+
def __init__(self, geology: geopandas.GeoDataFrame, faults: Optional[geopandas.GeoDataFrame] = None):
11+
self.geology = geology
12+
self.faults = faults
13+
self.contacts = None
14+
self.basal_contacts = None
15+
self.all_basal_contacts = None
16+
17+
def extract_all_contacts(self, save_contacts: bool = True) -> geopandas.GeoDataFrame:
18+
logger.info("Extracting contacts")
19+
geology = self.geology.copy()
20+
geology = geology.dissolve(by="UNITNAME", as_index=False)
21+
geology = geology[~geology["INTRUSIVE"]]
22+
geology = geology[~geology["SILL"]]
23+
if self.faults is not None:
24+
faults = self.faults.copy()
25+
faults["geometry"] = faults.buffer(50)
26+
geology = geopandas.overlay(geology, faults, how="difference", keep_geom_type=False)
27+
units = geology["UNITNAME"].unique().tolist()
28+
column_names = ["UNITNAME_1", "UNITNAME_2", "geometry"]
29+
contacts = geopandas.GeoDataFrame(crs=geology.crs, columns=column_names, data=None)
30+
while len(units) > 1:
31+
unit1 = units[0]
32+
units = units[1:]
33+
for unit2 in units:
34+
if unit1 != unit2:
35+
join = geopandas.overlay(
36+
geology[geology["UNITNAME"] == unit1],
37+
geology[geology["UNITNAME"] == unit2],
38+
keep_geom_type=False,
39+
)[column_names]
40+
join["geometry"] = join.buffer(1)
41+
buffered = geology[geology["UNITNAME"] == unit2][["geometry"]].copy()
42+
buffered["geometry"] = buffered.boundary
43+
end = geopandas.overlay(buffered, join, keep_geom_type=False)
44+
if len(end):
45+
contacts = pandas.concat([contacts, end], ignore_index=True)
46+
contacts["length"] = [row.length for row in contacts["geometry"]]
47+
if save_contacts:
48+
self.contacts = contacts
49+
return contacts
50+
51+
def extract_basal_contacts(self,
52+
stratigraphic_column: list,
53+
save_contacts: bool = True) -> geopandas.GeoDataFrame:
54+
55+
logger.info("Extracting basal contacts")
56+
units = stratigraphic_column
57+
58+
if self.contacts is None:
59+
self.extract_all_contacts(save_contacts=True)
60+
basal_contacts = self.contacts.copy()
61+
else:
62+
basal_contacts = self.contacts.copy()
63+
if any(unit not in units for unit in basal_contacts["UNITNAME_1"].unique()):
64+
missing_units = (
65+
basal_contacts[~basal_contacts["UNITNAME_1"].isin(units)]["UNITNAME_1"]
66+
.unique()
67+
.tolist()
68+
)
69+
logger.error(
70+
"There are units in the Geology dataset, but not in the stratigraphic column: "
71+
+ ", ".join(missing_units)
72+
+ ". Please readjust the stratigraphic column if this is a user defined column."
73+
)
74+
raise ValueError(
75+
"There are units in stratigraphic column, but not in the Geology dataset: "
76+
+ ", ".join(missing_units)
77+
+ ". Please readjust the stratigraphic column if this is a user defined column."
78+
)
79+
basal_contacts["ID"] = basal_contacts.apply(
80+
lambda row: min(units.index(row["UNITNAME_1"]), units.index(row["UNITNAME_2"])), axis=1
81+
)
82+
basal_contacts["basal_unit"] = basal_contacts.apply(lambda row: units[row["ID"]], axis=1)
83+
basal_contacts["stratigraphic_distance"] = basal_contacts.apply(
84+
lambda row: abs(units.index(row["UNITNAME_1"]) - units.index(row["UNITNAME_2"])), axis=1
85+
)
86+
basal_contacts["type"] = basal_contacts.apply(
87+
lambda row: "ABNORMAL" if abs(row["stratigraphic_distance"]) > 1 else "BASAL", axis=1
88+
)
89+
basal_contacts = basal_contacts[["ID", "basal_unit", "type", "geometry"]]
90+
basal_contacts["geometry"] = [
91+
shapely.line_merge(shapely.snap(geo, geo, 1)) for geo in basal_contacts["geometry"]
92+
]
93+
if save_contacts:
94+
self.all_basal_contacts = basal_contacts
95+
self.basal_contacts = basal_contacts[basal_contacts["type"] == "BASAL"]
96+
return basal_contacts

map2loop/data_checks.py

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -471,11 +471,13 @@ def check_keys(d: dict, parent_key=""):
471471
"fault": {
472472
"structtype_column", "fault_text", "dip_null_value",
473473
"dipdir_flag", "dipdir_column", "dip_column", "orientation_type",
474-
"dipestimate_column", "dipestimate_text", "name_column",
475-
"objectid_column", "minimum_fault_length", "ignore_fault_codes",
474+
"dipestimate_column", "dipestimate_text","displacement_column",
475+
"displacement_text", "name_column","objectid_column", "featureid_column", "minimum_fault_length",
476+
"fault_length_column", "fault_length_text", "ignore_fault_codes",
476477
},
477478
"fold": {
478-
"structtype_column", "fold_text", "description_column",
479+
"structtype_column", "fold_text", "axial_plane_dipdir_column",
480+
"axial_plane_dip_column","tightness_column", "description_column",
479481
"synform_text", "foldname_column","objectid_column",
480482
},
481483
}
@@ -853,6 +855,7 @@ def validate_structtype_column(
853855
if text_keys:
854856
for text_key, config_key in text_keys.items():
855857
text_value = config.get(config_key, None)
858+
text_pattern = '|'.join(text_value) if text_value else None
856859
if text_value:
857860
if not isinstance(text_value, str):
858861
error_msg = (
@@ -862,7 +865,7 @@ def validate_structtype_column(
862865
logger.error(error_msg)
863866
return (True, error_msg)
864867

865-
if not geodata[structtype_column].str.contains(text_value, na=False).any():
868+
if not geodata[structtype_column].str.contains(text_pattern, case=False, regex=True, na=False).any():
866869
if text_key == "synform_text":
867870
warning_msg = (
868871
f"Datatype {datatype_name.upper()}: The '{text_key}' value '{text_value}' is not found in column '{structtype_column}'. "

0 commit comments

Comments
 (0)