Skip to content

implement power spectra#2398

Open
iluise wants to merge 12 commits into
developfrom
iluise/develop/power-spectra
Open

implement power spectra#2398
iluise wants to merge 12 commits into
developfrom
iluise/develop/power-spectra

Conversation

@iluise
Copy link
Copy Markdown
Contributor

@iluise iluise commented May 21, 2026

Description

code from anemoi.models (https://github.com/ecmwf/anemoi-core/blob/main/models/src/anemoi/models/layers/)

  • implement SphericalHarmonicTransform
  • implement InverseSphericalHarmonicTransform (mostly for sanity check)
  • implement UKMet office fft transform (taken from script in evaluate/example_extras/power_spectra)

Use it as:

evaluation:
  metrics  : 
    - rmse
    - psd:
        psd_method: "set"                    # "sht" (SHT-based) or "fft" (regrid FFT)
        psd_regrid_resolution: 1.0           # degrees, only for fft method
        grid_type: "octahedral"              # "octahedral" (default), "regular", "reduced".

NB. for N320 grids we need to use reduced instead of octahedral. I tested it for both O96 and N320 grids.

The plots are in <summary_dir>/psd. The code produces one plot per forecast step.

I know it would be more appropriate to have them under results/run_id/plots/psd, but I followed the same workflow of the qq_analysis metric and I will change this in another PR that restructures this part.

image

Sanity check:

I implemented the same sanity check as what done in anemoi.models, so check that applying the SphericalTransform and its InverseSphericalTransform starting from random noise gives the identity.

Issue Number

Closes #2176

Is this PR a draft? Mark it as draft.

Checklist before asking for review

  • I have performed a self-review of my code
  • My changes comply with basic sanity checks:
    • I have fixed formatting issues with ./scripts/actions.sh lint
    • I have run unit tests with ./scripts/actions.sh unit-test
    • I have documented my code and I have updated the docstrings.
    • I have added unit tests, if relevant
  • I have tried my changes with data and code:
    • I have run the integration tests with ./scripts/actions.sh integration-test
    • (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
    • (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
  • I have informed and aligned with people impacted by my change:
    • for config changes: the MatterMost channels and/or a design doc
    • for changes of dependencies: the MatterMost software development channel

@github-actions github-actions Bot added the eval anything related to the model evaluation pipeline label May 21, 2026
- psd:
psd_method: "sht" # "sht" (SHT-based) or "fft" (regrid FFT)
psd_regrid_resolution: 1.0 # degrees, only for fft method
grid_type: "octahedral" # "octahedral" (default), "regular", "reduced".
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be determined from the dataset?

if psd_datasets is None:
continue

method_tag = psd_datasets[0].get("psd_method", "sht")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sht is only applicable for global, for the local ones we need to regard

Comment thread packages/evaluate/src/weathergen/evaluate/scores/psd.py
https://github.com/ecmwf/anemoi-core/blob/main/models/src/anemoi/models/layers/spectral_helpers.py

- **Path B – FFT PSD** (``method="fft"``):
1-D zonal FFT along the longitude dimension on a regular lat-lon grid.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about the necessary regrinding?

Comment thread packages/evaluate/src/weathergen/evaluate/scores/psd.py


# ---------------------------------------------------------------------------
# FFT PSD (absorbed from psd_calc.py)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the "absorbed ..."


data_3d = np.empty((n_samples, nlat, nlon))
for s in range(n_samples):
data_3d[s] = griddata(points, data[s], (grid_lat, grid_lon), method="nearest")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to be documented somewhere. Does this work robustly for spherical grids, e.g. for the CERRA Lambertian projection one?

Comment thread tests/test_psd.py
@@ -0,0 +1,212 @@
# (C) Copyright 2025 Anemoi contributors.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not merge this. We don't run it regularly and it's therefore not useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval anything related to the model evaluation pipeline

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

add automatic exporting to power spectra script

2 participants