Skip to content

Mh/full pipeline diffusion adjusted scores#2491

Draft
moritzhauschulz wants to merge 3 commits into
ecmwf:jk/develop/diffusion-full-pipelinefrom
moritzhauschulz:mh/full-pipeline-diffusion-adjusted-scores
Draft

Mh/full pipeline diffusion adjusted scores#2491
moritzhauschulz wants to merge 3 commits into
ecmwf:jk/develop/diffusion-full-pipelinefrom
moritzhauschulz:mh/full-pipeline-diffusion-adjusted-scores

Conversation

@moritzhauschulz

Copy link
Copy Markdown
Contributor

Description

This PR basically does two things:

  1. It fixes the SSR computation. Previously, the member RMSE was used as denominator. I (and Claude) think this should be ensemble RMSE.
  2. It fixes some dead-end ('return None') in score.py

This should probably be reviewed by someone from the eval team (maybe @iluise ?).

With this fix, and the fix in the eval config, I can now produce CRPS, SSR and spread plots and maps (pretty much out of the box).

Issue Number

Is this PR a draft? Mark it as draft.

Checklist before asking for review

  • I have performed a self-review of my code
  • [] My changes comply with basic sanity checks:
    • I have fixed formatting issues with ./scripts/actions.sh lint
    • I have run unit tests with ./scripts/actions.sh unit-test
    • I have documented my code and I have updated the docstrings.
    • I have added unit tests, if relevant
  • I have tried my changes with data and code:
    • I have run the integration tests with ./scripts/actions.sh integration-test
    • (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
    • (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
  • I have informed and aligned with people impacted by my change:
    • for config changes: the MatterMost channels and/or a design doc
    • for changes of dependencies: the MatterMost software development channel

@github-actions github-actions Bot added eval anything related to the model evaluation pipeline model Related to model training or definition (not generic infra) labels Jun 11, 2026
@clessig

clessig commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

@moritzhauschulz : can we also open this against develop please.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval anything related to the model evaluation pipeline model Related to model training or definition (not generic infra)

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants