AWI-ESM3#1479
Open
JanStreffing wants to merge 320 commits into
Open
Conversation
…s into feat/awiesm3-v3.4-co2
- Add cmip_co2_version parameter (default: 6) to select CMIP version - Add choose_cmip_co2_version block to select appropriate CO2 file - Add forcing_files entry for CO2 file linking - Update lpjg_steps.yaml.j2 template to pass cmip_co2_version to LPJ-GUESS
… cadence Regrid output quality fixes after validation runs on AWI-ESM3 CORE2/TCO95. XML changes (both xios_xml/ and xios_xml_cmip7/ domain_def): - drop interpolate_domain quantity="true" added in 4948546. quantity=true treats fields as extensive (sums area-fraction-weighted contributions); correct only for fluxes and only when source/target cells match in size. FESOM unstructured mesh has 5-20x resolution variation, and target cells in dense-mesh regions (e.g. Indonesia) contain ~12 source cells whose values summed produced impossible values (SST up to 316C). OIFS-to-regular has run without quantity for years; matching that gives intensive area-weighted average and physical per-cell values for all field types. - add interpolate_domain renormalize="true". FESOM unstructured source has no land cells, so coastal target cells were partially covered (weights summing to <1) which diluted values toward 0 across every coastline. renormalize=true divides by sum-of-weights, restoring physical ocean values up to the actual land mask. Pure-land cells retain the 0 sentinel they had before. File_def restructure (xios_xml_cmip7/file_def_fesom.xml.j2 only): - non-CMIP7 file_def back to upstream-like (no regrid blocks). - fesom_1d_reg: 13 daily 2D fields (mirrors fesom_1d) gated by fesom.regrid_surface_output. - fesom_1mo_reg: 33 node + 6 element 2D fields at 1mo (mirrors fesom_1mo 2D set, 0D scalars excluded) gated same. - fesom_1mo_reg_3d: full 3D set (cell-center + interface vertical + dMOC density coords) gated by new fesom.regrid_3d_output flag, default off — 3D regrid is expensive at production resolutions. grid_def additions (xios_xml_cmip7/grid_def_fesom.xml): - grid_3d_nod_reg, grid_3d_elem_reg (cell-center vertical) - grid_3d_nod_reg_nz1, grid_3d_elem_reg_nz1 (interface vertical) - grid_rho_nod_reg, grid_rho_elem_reg (dMOC density coord) Config flag changes (configs/components/fesom/fesom-2.7.yaml): - regrid_surface_output: description updated to reflect native-cadence mirroring and CMIP7-only scope. - regrid_3d_output: new flag, default false. - regrid_surface_daily: removed. Was only needed when the colleague's original regrid blocks output only monthly; replaced by native-cadence mirroring.
LR test runscript for regrid pipeline validation. CORE2 + TCO95 + L91, 2-day run. Enables fesom.cmip7_cmor_output (regrid blocks live in the CMIP7 file_def only) and fesom.regrid_surface_output. 3D regrid stays off via the default of fesom.regrid_3d_output.
Two HR (TCO319L137 + DARS2) 1-year variants used to characterize the
regrid path cost vs the unchanged HR baseline:
- *_1y_regrid.yaml: 2D regrid only, 1deg target (sets
fesom.regrid_surface_output: True; leaves regrid_3d_output false).
Used as Final_CMIP7_IO_Test_20.
- *_1y_regrid_3d_05deg.yaml: 2D + 3D regrid, 0.5deg target (sets
both regrid switches + xios.{fesom_,}regular_res_l{on,at} to
720/360). Used as Final_CMIP7_IO_Test_21.
Measured cost: Test_21 vs Test_20 = +0.4% wall time, +8 GB/year disk.
Test_21 vs HR baseline (no regrid) = similar. Cheap enough to make
both regrid switches default-on for CMIP7 (next commit).
Flip defaults based on HR perf characterization: - fesom.regrid_surface_output: false -> true. CMIP7 file_def emits daily and monthly 2D regridded fields alongside the native unstructured output. <1% wall, <1 GB/year at 0.5deg. - fesom.regrid_3d_output: false -> true. Adds full 3D regridded set (temp, salt, unod/vnod, w, bolus_*, N2, Kv, u/v on element grid, Av, dMOC density-coord). HR Test_21 vs Test_20 measured +0.4% wall and +8 GB/year — effectively free thanks to XIOS server async IO. - awiesm3.yaml + awicm3.yaml: bump default xios.fesom_regular_res_lon/lat from 360/180 (1deg) to 720/360 (0.5deg). Resolves the FESOM-vs-OIFS asymmetry where coastal features and surface gradients were under-resolved at 1deg despite the source mesh being much finer. 0.5deg cost is what Test_21 measured. Standalone-FESOM safe: the regrid file_def blocks only fire under the CMIP7 j2 templates (xios_xml_cmip7/), which require fesom.cmip7_cmor_output: true. Non-CMIP7 runs unaffected. To opt out of regrid per experiment, set the flags back to false in the runscript yaml.
…guard) New component versions both carry the regular-grid output work merged in this iteration. FESOM 2.7.6 (tag d42b4857, merge of PR 917): - Nadine's cell-bounds-aware XIOS domain attrs for unstructured nodes + elements (bounds_lon_1d, bounds_lat_1d with CCW sort + dateline unwrap + NMAX=10 cap). - Drop the #if defined(__oasis) gate around mesh%x_corners allocation in oce_mesh.F90 — XIOS regrid needs the corners and they aren't OASIS-specific. Fixes the standalone-FESOM CI segfault that motivated the gate removal. XIOS 2.5.3 (tag from origin/main 97ac2a88): - Cell-bounds patches in extern/remap/src for the FESOM unstructured source (Nadine). - Magnitude guard in remap update()/move(): proj(centre) was producing NaN when child centroids cancel near-antipodally on global meshes, silently dropping polygons from the kd-tree and producing no nodes_reg weight file. Now falls back to the largest-leafCount child's centre (update) / heavier-weight side (move). Plus diagnostics module with per-rank counter dump at three checkpoints. - Validated on TCO95+CORE2 and TCO319+DARS2 (HR Tests 19-21). Component yaml: new "2.5.3" arm in xios.yaml (mirrors 2.5.2 modulo branch label) + "2.7.6" arm in fesom-2.7.yaml. Bump both awiesm3_v3.4.2 and awicm3_v3.4.2 coupling specs to use the new pins.
The aEVP choose-block for DARS2 in awiesm3.yaml + awicm3.yaml only set whichevp=2 but left delta_min at the namelist.ice default (1.0e-11). That default is too stiff for aEVP on HR — every HR runscript since the AWI-ESM3-VEG-HR-CMIP7-Spinup_cont2 has been setting delta_min=2e-9 explicitly to compensate. Move the override into the choose block so anyone using version v3.4.2 or develop with DARS2 gets the correct paired (whichevp, delta_min) without needing to remember it per runscript. No effect on non-DARS2 resolutions (each keeps its per-mesh whichEVP / delta_min default from fesom-2.7.yaml).
…chevp=2" This reverts commit a0b489e.
The XIOS-side regridded surface output (enabled via
fesom.regrid_surface_output: true, present in HR Test_21+) was already
being copied to outdata/fesom by the default `*.fesom.*.nc` glob, but
left under its XIOS-default name `<var>.fesom.reg_<ystart>-<yend>.nc`.
Add a sibling renamer that normalises to `<var>.fesom.gr.<year>.nc`,
mirroring how `rename_xios_fesom.sh` normalises native XIOS output to
`<var>.fesom.<year>.nc`:
- `gr` matches the CMIP grid_label for regridded (regular lat/lon),
so the on-disk segment tells you the destination DRS grid_label
without translation.
- Visually distinct from native at `ls` time: `.fesom.` vs `.fesom.gr.`
- Disjoint glob from the native renamer: this script needs literal
".fesom.reg_" while the native one needs literal ".fesom_". Safe to
run concurrently.
- Idempotent: skips files already in target form, no-ops cleanly if
no `.fesom.reg_*` files are present (e.g. regrid_surface_output=false).
Pycmor side gets a clean parallel for the future `_gr` mirror tiers:
native: pattern: <var>\.fesom\.\d{4}\.nc
reg: pattern: <var>\.fesom\.gr\.\d{4}\.nc
which is what enables producing both gn and gr CMIP7 output from one
source experiment.
FESOM 2.7.7 (tag cdbab44f) includes 9 commits since 2.7.6 that are all production-relevant for HR: - Fix 0D CMIP diagnostic accumulation bug — diag scalars (siarean, siareas, siextentn, siextents, sivoln, sivols, volo, soga, thetaoga) were being sent to XIOS without proper accumulation, giving wrong monthly means. - Fix mesh-partitioner segfault on large meshes (#922). - Fix aEVP Wvel NaN allocation bug in MOD_ICE (alpha_evp_array wrongly allocated when whichevp=2) — relevant for our DARS2 + aEVP path. Add "2.7.7" arm in fesom-2.7.yaml and bump both awiesm3_v3.4.2 and awicm3_v3.4.2 coupling specs to the new pin.
- oifs48.cmip.yaml: add cmip7 historical scenario arm activating stratospheric volcanic aerosols (LCMIP_STRATAER_CMIP7=True, LCMIP_STRATAER_BCKGD=False). Previously only piControl set these flags, so historical runs would silently skip the strat aerosol reader. Ozone, GHG, solar, MACv2-SP, and dust already come from the always-applied cmip7 block and are year-aware via LCMIP7=True. - lpj_guess.cmip.yaml: add historical arm explicitly setting fixed_CO2/fixed_ndep/fixed_LU to -1 (transient). Removes ambiguity with awiesm3.yaml defaults vs piControl arm overrides. - runscripts/awiesm3/v3.4.2/: add 1y historical runscript starting 1850-01-01, branched from AWI-ESM3-VEG-HR-CMIP7-piControl. Restart paths are placeholders until piControl produces 1849 restarts.
Adds the runtime config + CSV fixtures + example runscripts needed to
exercise the new FESOM ship-track / mooring curtain feature
(io_tracks.F90, FESOM feature/tracks). The feature lets a coupled
FESOM run emit a (time, cell, nz) "curtain" NetCDF for an arbitrary
set of user-supplied polyline tracks, at any XIOS-supported cadence.
Files:
namelists/fesom2/tracks/context_fesom.xml
Drop-in override of the FESOM XIOS context that adds a
<variable_group id="fesom_tracks"> block with ltracks=true plus
the semicolon-separated track_files / track_vars / track_names
plus track_resolution_km and track_output_freq. io_xios_init reads
these via xios_getvar() because the XIOS-coupled path bypasses
namelist.io.
namelists/fesom2/tracks/test_track_core2_transect.csv
7 vertices, SW->NE polyline across the subpolar North Atlantic to
Fram Strait. At track_resolution_km=5 km on CORE2 densifies to
639 dense samples that resolve to 152 unique mesh nodes.
namelists/fesom2/tracks/test_track_mosaic_co{1,2,3}.csv
Real Polarstern positions per MOSAiC ice-camp window, daily-
decimated from the 1-second PANGAEA master tracks (PS122/1..5,
doi.pangaea.de/10.1594/PANGAEA.{924669, 924672, 924678, 926830,
926911}). Three separate files so the polyline densifier never
crosses an artificial geodesic between disjoint ice-camp windows.
runscripts/awiesm3/develop/awiesm3-develop-levante-TCO95L91-CORE2_tracks_{1d,1m,1y}.yaml
Example CORE2 LR runscripts at 1-day / 1-month / 1-year durations.
The 1-day variant is the smallest useful test; the 1-year version
exercises four simultaneous tracks (transect + the three MOSAiC
Central Observatories) at 3-hourly cadence and produces
(time=2920, cell=N, nz=47) NetCDFs per track.
Pre-flight: ${awiesm3.model_dir}/fesom-2.7 must be on the FESOM
feature/tracks branch and rebuilt. Verified end-to-end on CORE2 LR
test_track_11: all four expected NetCDFs written, 100% finite cells,
T_surf ranges physically right (-1.89 to +14.55 degC across the
transect, freezing-point-locked -1.88 to -1.09 degC in the MOSAiC
windows).
Contributor
Author
|
@seb-wahl, Would you have time for an hour this week to look at the merge conflicts? |
Contributor
I looked at the conflicts, and couldn't find one that is actually a problem. We just added new blocks at about the same lines in oifs.yaml, and xios.yaml. We need to accept both and make sure that "my" changes end up in the correct section so we don't break the yaml files. |
Resolved conflicts:
- configs/components/oifs/oifs.yaml: keep both new OIFS versions
(48r1v1/v2 and 48r1-foci5), keep both new choose_massfixer arms
- configs/components/xios/xios.yaml: keep both new XIOS versions
- configs/machines/levante.yaml: keep oneapi2025_openmpi5 stack and
intel2022_openmpi MPI_* aliases; use ${lib_aec}/${lib_tbbmalloc}
pattern in system_intel_libs / system_intel_impi_libs to avoid
collisions with release's new choose_libaec / choose_tbb blocks
- configs/setups/awicm3/awicm3.yaml: keep both v3.3.1 and v3.4.x
AWI-CM3 versions
Adjacent fixes pulled in alongside the merge:
- Rename ${esm_function_dir} -> ${esm_configs_dir} in
configs/components/{fesom/fesom-2.7,lpj_guess/lpj_guess,oifs/oifs}.yaml
- Drop MPI_HOME/MPI_ROOT/mpi_ROOT/MPI_DIR/mpi_DIR from system_intel_libs
(now provided by choose_compiler_mpi.intel2022_openmpi via $MPIROOT)
…/awiesm3-v3.4-co2
…vailable_versions Required by the awiesm3_v3.4.2 coupling. Each version already had a choose_version block but was missing from available_versions, so esm_master would reject the target.
Pulls in the new awiesm3-v3.4.2-levante-TCO95L91-CORE2_1d.yaml plus its sister run_files4check_runs/wam_namelist so the levante CI leg exercises v3.4.2 dispatch + config resolution alongside develop.
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With the CMIP7 PI control starting, maybe its time to think about merging?
Since we start without automatic cmorization for now, the esm_tools development is not quite over.