Fix objective.compile in Benchmarks

[YigitElma]

Some benchmarks were using obj.compile and jac_scaled_error together but obj.compile only compiles jac_scaled and compute_scaled. This cause some benchmarks to have very different min/max values, like,

With the change it is more consistent,

Misc

• Adjusts number of rounds for tests
• Adds Action Details step to tests
• Resolves Export compiled objectives for common equilibrium resolutions? #1402

[github-actions]

|             benchmark_name             |         dt(%)          |         dt(s)          |        t_new(s)        |        t_old(s)        | 
| -------------------------------------- | ---------------------- | ---------------------- | ---------------------- | ---------------------- |
 test_build_transform_fft_lowres         |     -0.05 +/- 1.68     | -2.56e-04 +/- 8.77e-03 |  5.21e-01 +/- 6.9e-03  |  5.21e-01 +/- 5.5e-03  |
 test_equilibrium_init_medres            |     -0.80 +/- 0.62     | -3.25e-02 +/- 2.51e-02 |  4.05e+00 +/- 1.4e-02  |  4.08e+00 +/- 2.1e-02  |
 test_equilibrium_init_highres           |     -0.69 +/- 0.80     | -3.70e-02 +/- 4.29e-02 |  5.30e+00 +/- 3.4e-02  |  5.34e+00 +/- 2.6e-02  |
 test_objective_compile_dshape_current   |     +0.69 +/- 3.92     | +2.78e-02 +/- 1.57e-01 |  4.04e+00 +/- 1.2e-01  |  4.02e+00 +/- 1.1e-01  |
 test_objective_compute_dshape_current   |     +0.48 +/- 1.11     | +2.46e-05 +/- 5.68e-05 |  5.12e-03 +/- 4.1e-05  |  5.10e-03 +/- 4.0e-05  |
 test_objective_jac_dshape_current       |     -0.89 +/- 7.77     | -3.74e-04 +/- 3.25e-03 |  4.15e-02 +/- 2.3e-03  |  4.18e-02 +/- 2.3e-03  |
 test_perturb_2                          |     -0.07 +/- 1.77     | -1.43e-02 +/- 3.42e-01 |  1.93e+01 +/- 2.3e-01  |  1.93e+01 +/- 2.6e-01  |
 test_proximal_freeb_jac                 |     -0.26 +/- 1.23     | -1.93e-02 +/- 9.08e-02 |  7.34e+00 +/- 4.9e-02  |  7.36e+00 +/- 7.7e-02  |
 test_solve_fixed_iter                   |     +0.28 +/- 1.76     | +8.74e-02 +/- 5.51e-01 |  3.14e+01 +/- 3.9e-01  |  3.13e+01 +/- 3.9e-01  |
 test_LinearConstraintProjection_build   |     -0.40 +/- 1.83     | -5.51e-02 +/- 2.50e-01 |  1.36e+01 +/- 1.7e-01  |  1.36e+01 +/- 1.8e-01  |
 test_build_transform_fft_midres         |     -0.64 +/- 6.05     | -4.03e-03 +/- 3.81e-02 |  6.26e-01 +/- 1.6e-02  |  6.30e-01 +/- 3.5e-02  |
 test_build_transform_fft_highres        |     -0.19 +/- 5.80     | -1.92e-03 +/- 5.77e-02 |  9.94e-01 +/- 5.2e-02  |  9.96e-01 +/- 2.5e-02  |
 test_equilibrium_init_lowres            |     -1.02 +/- 2.51     | -4.18e-02 +/- 1.02e-01 |  4.04e+00 +/- 7.7e-02  |  4.09e+00 +/- 6.7e-02  |
 test_objective_compile_atf              |     +0.24 +/- 2.06     | +2.05e-02 +/- 1.74e-01 |  8.48e+00 +/- 1.2e-01  |  8.46e+00 +/- 1.2e-01  |
 test_objective_compute_atf              |     -1.82 +/- 3.04     | -2.98e-04 +/- 4.96e-04 |  1.60e-02 +/- 2.7e-04  |  1.63e-02 +/- 4.2e-04  |
 test_objective_jac_atf                  |     -4.86 +/- 2.29     | -1.00e-01 +/- 4.72e-02 |  1.96e+00 +/- 3.5e-02  |  2.06e+00 +/- 3.1e-02  |
 test_perturb_1                          |     -1.64 +/- 1.06     | -2.53e-01 +/- 1.64e-01 |  1.52e+01 +/- 1.1e-01  |  1.54e+01 +/- 1.2e-01  |
 test_proximal_jac_atf                   |     -2.32 +/- 1.91     | -1.97e-01 +/- 1.62e-01 |  8.31e+00 +/- 6.0e-02  |  8.50e+00 +/- 1.5e-01  |
 test_proximal_freeb_compute             |     +0.88 +/- 1.43     | +1.78e-03 +/- 2.88e-03 |  2.03e-01 +/- 2.2e-03  |  2.01e-01 +/- 1.8e-03  |
 test_solve_fixed_iter_compiled          |     +0.03 +/- 2.66     | +6.77e-03 +/- 5.51e-01 |  2.07e+01 +/- 1.9e-01  |  2.07e+01 +/- 5.2e-01  |

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.64%. Comparing base (7d378c2) to head (33df48c).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1483      +/-   ##
==========================================
- Coverage   95.64%   95.64%   -0.01%     
==========================================
  Files         101      101              
  Lines       25542    25542              
==========================================
- Hits        24430    24429       -1     
- Misses       1112     1113       +1     

Files with missing lines	Coverage Δ
desc/objectives/objective_funs.py	94.74% <100.00%> (ø)

... and 1 file with indirect coverage changes

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[sinaatalay]

This PR looks good to me, although others should review it as I am looking at it with only my Python and GitHub knowledge, without DESC knowledge.

• Workflows (benchmark.yaml, notebook_tests.yaml, and regression_tests.yaml works the same way as its previous version, except a new step is added, Action Details, which moves all the debugging-related commands to a separate step. It makes sense.
• There is a change in desc.objectives.objective_funcs.ObjectiveFunction.compile method, which uses compute_scaled_error method instead of compute_scaled in lsq and all modes. This hasn't been explained in the PR or commit messages. Maybe @YigitElma should explain it, but I am sure it's okay.
• The documentation is updated and seems okay.
• Tests haven't been changed algorithmically (except changing rounds in benchmarks) but have been cleaned. It looks better, I don't see any errors.

[f0uriest]

reducing the number of rounds will make the statistics more noisy, and may lead to more false positives

[YigitElma]

I agree. For example, in the last benchmark, this PR shows speed improvement for perturb but it shouldn't. We can decide on the exact number of rounds, but my intention is to balance the time spent on tests. Previously, these equilibrium initialization tests took more time than fixed_iter_solve and perturb tests. Given that benchmark workflow started to take around 50mins, I wanted to reduce them from 50, which is a bit overkill.

[f0uriest]

why remove this?

[YigitElma]

This doesn't cause too much difference,(I can re add it) but technically only run is benchmarked, so clearing the cache here doesn't have much purpose.

[f0uriest]

moving everything to the run function is changing what its actually profiling, this now includes a bunch of other stuff besides building the linear constraints, is that what we want?

[YigitElma]

The extra stuff is just building individual constraints and objectives, right? Previously, this was effectively just benchmarking factorize_linear_constraints. Do we usually pass built constraints to the LinearConstraintProjection? Then I can revert it.