added inference_only mode (skip target loading, loss computation, and metric logging)

[sbAsma]

Description

Issue Number

Closes #2256

Is this PR a draft? Mark it as draft.

Checklist before asking for review

• I have performed a self-review of my code
• My changes comply with basic sanity checks:
  • I have fixed formatting issues with ./scripts/actions.sh lint
  • I have run unit tests with ./scripts/actions.sh unit-test
  • I have documented my code and I have updated the docstrings.
  • I have added unit tests, if relevant
• I have tried my changes with data and code:
  • I have run the integration tests with ./scripts/actions.sh integration-test
  • (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
  • (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
• I have informed and aligned with people impacted by my change:
  • for config changes: the MatterMost channels and/or a design doc
  • for changes of dependencies: the MatterMost software development channel

[evenmn]

Beautiful! This is very useful for the pre-operational runs and long-term forecasts

[evenmn]

assign me

[evenmn]

This looks good to me! I tested running inference with inference_only=True, and is works as expected. We still cannot run inference outside the dataset range, but I suspect this is because the forcings are still taken from the dataset?

[sbAsma]

This looks good to me! I tested running inference with inference_only=True, and is works as expected. We still cannot run inference outside the dataset range, but I suspect this is because the forcings are still taken from the dataset?

I ran inference starting from late december 2023 to january 2024 and it worked for me.

[evenmn]

This looks good to me! I tested running inference with inference_only=True, and is works as expected. We still cannot run inference outside the dataset range, but I suspect this is because the forcings are still taken from the dataset?

I ran inference starting from late december 2023 to january 2024 and it worked for me.

My bad, I didn't change test_config.end_date, it seems to work now! I guess this works because forcings are not applied continuously for all rollout steps, but that's a separate problem. Nice implementation