Create fake CMIP7 data for recipe_ref_cre.yml

[LisaBock]

Variables:
rlut
rlutcs
rsut
rsutcs

To find on Levante: /work/bd0854/DATA/ESMValTool2/CMIP7_TEST/MIP-DRS7/CMIP7/CMIP/MPI-M/DUMMY-MODEL/historical/r1i1p1f3/glb/mon

[hb326]

Recipe is tested in the branch "test_REF_cre_recipe_with_CMIP7data", @bouweandela.

[bouweandela]

@hb326 I ran the command pixi run -e esmvalcore-dev esmvaltool run ref/recipe_enso_basicclimatology.yml --max-parallel-tasks=1 with that branch and it completed successfully. You can find the output in /home/b/b381141/esmvaltool_output_scratch/recipe_enso_basicclimatology_20260513_164355/.

[bouweandela]

Oh wait, that's probably the wrong recipe. Let me try with ref/recipe_ref_cre_cmip7.yml.

[bouweandela]

OK, I got the same error:

2026-05-13 16:49:52,416 UTC [2065938] ERROR   Failed to run preprocessor function 'save' on the data
[<iris 'Cube' of TOA Longwave Cloud Radiative Effect / (W m-2) (latitude: 180)>]
loaded from original input file(s)
[IntakeESGFDataset(name='obs4MIPs.NASA-LaRC.CERES-EBAF-4-2.mon.rlut.gn'),
 IntakeESGFDataset(name='obs4MIPs.NASA-LaRC.CERES-EBAF-4-2.mon.rlutcs.gn')]
with function argument(s)
compress = False,
compute = False,
filename = PosixPath('/home/b/b381141/esmvaltool_output_scratch/recipe_ref_cre_cmip7_20260513_164732/preproc/plot_profiles/lwcre/obs4MIPs_CERES-EBAF-4-2_lwcre_2001-2021.nc')
2026-05-13 16:49:53,445 UTC [2065938] INFO    Maximum memory used (estimate): 3.0 GB
2026-05-13 16:49:53,446 UTC [2065938] INFO    Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
2026-05-13 16:49:53,448 UTC [2065938] ERROR   Program terminated abnormally, see stack trace below for more information:
Traceback (most recent call last):
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/_main.py", line 786, in run
    fire.Fire(ESMValTool())
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/_main.py", line 620, in run
    self._run(recipe, session, cli_config_dir)
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/_main.py", line 675, in _run
    process_recipe(recipe_file=recipe, session=session)
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/_main.py", line 156, in process_recipe
    recipe.run()
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/_recipe/recipe.py", line 1336, in run
    self.tasks.run(max_parallel_tasks=self.session["max_parallel_tasks"])
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/_task.py", line 838, in run
    self._run_sequential()
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/_task.py", line 849, in _run_sequential
    task.run()
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/_task.py", line 288, in run
    input_files.extend(task.run())
                       ^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/_task.py", line 295, in run
    self.output_files = self._run(input_files)
                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 888, in _run
    delayed = product.close()
              ^^^^^^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 738, in close
    result = self.save()
             ^^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 726, in save
    return preprocess(
           ^^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 570, in preprocess
    _run_preproc_function(
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 510, in _run_preproc_function
    return function(items, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/esmvalcore/preprocessor/_io.py", line 334, in save
    return iris.save(cubes, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/iris/io/__init__.py", line 508, in save
    result = saver(source, target, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/iris/fileformats/netcdf/saver.py", line 2966, in save
    sman.write(
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/iris/fileformats/netcdf/saver.py", line 580, in write
    self.check_attribute_compliance(cube, cube.dtype)
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/iris/fileformats/netcdf/saver.py", line 713, in check_attribute_compliance
    new_val = _coerce_value(val_attr, val_attr_value, data_dtype)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/b/b381141/src/esmvalgroup/esmvaltool/.pixi/envs/esmvalcore-dev/lib/python3.12/site-packages/iris/fileformats/netcdf/saver.py", line 690, in _coerce_value
    raise ValueError(msg.format(val_attr, val_attr_value))
ValueError: "valid_min" is not of a suitable value (0.00000)

[bouweandela]

It's using the older version of ESMValCore because it has been locked to a specific commit in the pixi.lock file and rerunning pixi lock does not seem to solve it. Now trying with first deleting the lock file.

[bouweandela]

Running rm pixi.lock and then pixi lock and then pixi run -e esmvalcore-dev esmvaltool run ref/recipe_ref_cre_cmip7.yml --max-parallel-tasks=1 solved it and the results are in /home/b/b381141/esmvaltool_output_scratch/recipe_ref_cre_cmip7_20260513_170031/