Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix discard spinup in leaderboad and add best/worst single models #857

Merged
merged 3 commits into from
Jun 26, 2024

Conversation

Sbozzolo
Copy link
Member

@Sbozzolo Sbozzolo commented Jun 14, 2024

The routine that was discarding the spinup period had a flipped sign which resulted in everything but the spinup being discard. Now that's fixed and the first 6 months are discarded. In fixing this, I added the model that minimize/maximise errors across all the metrics (normalized to the median RMSE).

I also started decoupling the code to produce the leaderboard from the driver so that it can be run independently. Now, it still lives in run_amip.jl but if ones changes these two lines

            diagnostics_folder_path = atmos_sim.integrator.p.output_dir
            leaderboard_base_path = dir_paths.artifacts

with the correct folders, the code will produce the leaderboard for the data that is found.

This is what the leaderboard looks now (540 days run, 180 discarded)
bias_leaderboard

@Sbozzolo Sbozzolo force-pushed the gb/leaderboard branch 4 times, most recently from f463a70 to 16c84d7 Compare June 17, 2024 01:49
@Sbozzolo Sbozzolo changed the title Add best/worst single models in leaderboard Fix discard spinup in leaderboad and add best/worst single models Jun 25, 2024
@Sbozzolo Sbozzolo requested a review from szy21 June 25, 2024 23:38
Copy link
Member

@szy21 szy21 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@Sbozzolo Sbozzolo force-pushed the gb/leaderboard branch 2 times, most recently from c50a025 to 316f09d Compare June 26, 2024 16:33
@Sbozzolo Sbozzolo enabled auto-merge June 26, 2024 16:33
@Sbozzolo Sbozzolo merged commit a4046f9 into main Jun 26, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants