Skip to content

Commit

Permalink
Update benchmarks
Browse files Browse the repository at this point in the history
  • Loading branch information
AndrewIOM committed Mar 28, 2024
1 parent 54a48bf commit d378263
Showing 1 changed file with 6 additions and 5 deletions.
11 changes: 6 additions & 5 deletions benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,16 @@ Bristlecone Benchmarks

| Model name | Optimisation method | n runs | Target minimum | Best estimated minimum | Median estimated minimum | Sum distance from true parameters | Parameter estimates (individual) | True solutions | Milliseconds taken (median) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Bukin Sixth | random walk MCMC | 10 | 0.000 | 0.261 (0.261 -> min) | 0.459 | 3.748 | best -2.840 / median -8.403 (±6.225)<br>best 0.081 / median 0.706 (±0.656) | -10.000 / 1.000 | 955 |
| Holder Table | random walk MCMC | 10 | -19.209 | -19.208 (0.000 -> min) | -19.206 | 0.021 | best -8.058 / median -8.036 (±8.317)<br>best 9.659 / median 0.006 (±10.188) | 8.055 / 9.665<br>8.055 / -9.665<br>-8.055 / 9.665<br>-8.055 / -9.665 | 631.5 |
| Cross in tray | random walk MCMC | 10 | -2.063 | -2.063 (0.000 -> min) | -2.063 | 0.023 | best 1.346 / median 1.333 (±1.301)<br>best -1.351 / median 1.340 (±1.296) | 1.349 / -1.349<br>1.349 / 1.349<br>-1.349 / 1.349<br>-1.349 / -1.349 | 593 |
| Ackley (2D) | amoeba single | 10 | 0.000 | 0.000 (0.000 -> min) | 4.731 | 1.958 | best 0.000 / median 0.000 (±7.136)<br>best -0.000 / median 0.000 (±9.188) | 0.000 / 0.000 | 288.5 |
| Bukin Sixth | amoeba single | 10 | 0.000 | 0.002 (0.002 -> min) | 0.006 | 0.696 | best -10.214 / median -9.597 (±2.037)<br>best 1.043 / median 0.921 (±0.369) | -10.000 / 1.000 | 172 |
| Holder Table | amoeba single | 10 | -19.209 | -19.209 (0.000 -> min) | -7.619 | 6.492 | best -8.055 / median -4.913 (±6.507)<br>best -9.665 / median 1.630 (±8.518) | 8.055 / 9.665<br>8.055 / -9.665<br>-8.055 / 9.665<br>-8.055 / -9.665 | 176.5 |
| Cross in tray | amoeba single | 10 | -2.063 | -2.063 (0.000 -> min) | -2.063 | 0.001 | best 1.349 / median 1.349 (±3.721)<br>best -1.349 / median -1.349 (±2.326) | 1.349 / -1.349<br>1.349 / 1.349<br>-1.349 / 1.349<br>-1.349 / -1.349 | 192 |

## Full Model Systems

### Time-series models

| Model name | Optimisation method | success % | n runs | Distance from minimum likelihood (median) | Distance from true parameter values | Milliseconds taken (median) |
| --- | --- | --- | --- | --- | --- | --- |
| predator-prey (with gaussian noise) | random walk MCMC | 100.000% | 10 | 425.467 | [Δ] 0.027 best / median 0.0920.083)<br>[α] 0.002 best / median 0.0990.075)<br>[β] 0.006 best / median 0.097 (±0.076)<br>[γ] 0.024 best / median 0.105 (±0.075)<br>[ρ] 0.072 best / median 0.296 (±0.357)<br>[σ[x]] 0.057 best / median 0.251 (±0.176)<br>[σ[y]] 0.036 best / median 0.1700.370) | 716 |
| predator-prey | random walk MCMC | 100.000% | 10 | 0.521 | [Δ] 0.017 best / median 0.125 (±0.254)<br>[α] 0.000 best / median 0.071 (±1.785)<br>[β] 0.002 best / median 0.1021.822)<br>[γ] 0.008 best / median 0.096 (±0.218) | 240.5 |
| predator-prey (with gaussian noise) | amoeba single | 100.000% | 10 | 2718.028 | [Δ] 0.053 best / median 0.2651.094)<br>[α] 0.001 best / median 0.1701.380)<br>[β] 0.088 best / median 0.314 (±0.479)<br>[γ] 0.026 best / median 0.271 (±0.227)<br>[ρ] 0.136 best / median 0.662 (±0.342)<br>[σ[x]] 0.007 best / median 0.209 (±0.249)<br>[σ[y]] 0.014 best / median 1.6473755011282.332) | 311 |
| predator-prey | amoeba single | 100.000% | 10 | 6.543 | [Δ] 0.013 best / median 0.114 (±0.167)<br>[α] 0.005 best / median 0.071 (±0.171)<br>[β] 0.001 best / median 0.1200.097)<br>[γ] 0.027 best / median 0.171 (±0.103) | 557 |

0 comments on commit d378263

Please sign in to comment.