Skip to content

Commit

Permalink
Update pubgrub for faster range operations (#2421)
Browse files Browse the repository at this point in the history
This update pulls in pubgrub-rs/pubgrub#177,
optimizing common range operations in pubgrub. Please refer to this PR
for a more extensive description and discussion of the changes.

The changes optimize that last remaining pathological case,
`bio_embeddings[all]` on python 3.12, which has to try 100k versions,
from 12s to 3s in the cached case. It should also enable smarter
prefetching in batches (#170),
even though a naive attempt didn't show better network usage.

**before** 12s


![image](https://github.com/pubgrub-rs/pubgrub/assets/6826232/80ffdc49-1159-453d-a3ea-0dd431df6d3b)

**after** 3s


![image](https://github.com/pubgrub-rs/pubgrub/assets/6826232/69508c29-73ab-4593-a588-d8c722242513)

```
$  taskset -c 0 hyperfine --warmup 1 "../uv/target/profiling/main-uv pip compile ../uv/scripts/requirements/bio_embeddings.in"  "../uv/target/profiling/branch-uv pip compile ../uv/scripts/requirements/bio_embeddings.in"
Benchmark 1: ../uv/target/profiling/main-uv pip compile ../uv/scripts/requirements/bio_embeddings.in
  Time (mean ± σ):     12.321 s ±  0.064 s    [User: 12.014 s, System: 0.300 s]
  Range (min … max):   12.224 s … 12.406 s    10 runs

Benchmark 2: ../uv/target/profiling/branch-uv pip compile ../uv/scripts/requirements/bio_embeddings.in
  Time (mean ± σ):      3.109 s ±  0.004 s    [User: 2.782 s, System: 0.321 s]
  Range (min … max):    3.103 s …  3.116 s    10 runs

Summary
  ../uv/target/profiling/branch-uv pip compile ../uv/scripts/requirements/bio_embeddings.in ran
    3.96 ± 0.02 times faster than ../uv/target/profiling/main-uv pip compile ../uv/scripts/requirements/bio_embeddings.in
```

It also adds `bio_embeddings[all]` as a requirements test case.
  • Loading branch information
konstin authored Mar 13, 2024
1 parent 22a5239 commit 6e5a737
Show file tree
Hide file tree
Showing 3 changed files with 4 additions and 2 deletions.
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ owo-colors = { version = "4.0.0" }
pathdiff = { version = "0.2.1" }
petgraph = { version = "0.6.4" }
platform-info = { version = "2.0.2" }
pubgrub = { git = "https://github.com/zanieb/pubgrub", rev = "b5ead05c954b81690aec40255a1c36ec248e90af" }
pubgrub = { git = "https://github.com/astral-sh/pubgrub", rev = "addbaf184891d66a2dfd93d241a66d13bfe5de86" }
pyo3 = { version = "0.20.3" }
pyo3-log = { version = "0.9.0" }
pyproject-toml = { version = "0.10.0" }
Expand Down
2 changes: 2 additions & 0 deletions scripts/requirements/bio_embeddings.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Run on python 3.12 for a pathological test case
bio_embeddings[all]

0 comments on commit 6e5a737

Please sign in to comment.