Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuring accelerators with optional dependencies does not work with build system #9259

Closed
eginhard opened this issue Nov 20, 2024 · 2 comments · Fixed by #9273
Closed
Assignees
Labels
bug Something isn't working

Comments

@eginhard
Copy link

Thank you for the improvements for installing Pytorch! The example at https://docs.astral.sh/uv/guides/integration/pytorch/#configuring-accelerators-with-optional-dependencies works great. But it fails when I add a build system:

uv run --extra cpu python
  × Failed to build `project @ file:///home/.../project`
  ╰─▶ Source entry for `torch` only applies to extra `cpu`, but the `cpu` extra does not
      exist. When an extra is present on a source (e.g., `extra = "cpu"`), the relevant
      package must be included in the `project.optional-dependencies` section for that extra
      (e.g., `project.optional-dependencies = { "cpu" = ["torch"] }`).

Running on Ubuntu with uv 0.5.3. This is the pyproject.toml, with only the first 3 lines added to the example.

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12.0"
dependencies = []

[project.optional-dependencies]
cpu = [
  "torch>=2.5.1",
  "torchvision>=0.20.1",
]
cu124 = [
  "torch>=2.5.1",
  "torchvision>=0.20.1",
]

[tool.uv]
conflicts = [
  [
    { extra = "cpu" },
    { extra = "cu124" },
  ],
]

[tool.uv.sources]
torch = [
  { index = "pytorch-cpu", extra = "cpu", marker = "platform_system != 'Darwin'" },
  { index = "pytorch-cu124", extra = "cu124" },
]
torchvision = [
  { index = "pytorch-cpu", extra = "cpu", marker = "platform_system != 'Darwin'" },
  { index = "pytorch-cu124", extra = "cu124" },
]

[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true

[[tool.uv.index]]
name = "pytorch-cu124"
url = "https://download.pytorch.org/whl/cu124"
explicit = true
@charliermarsh
Copy link
Member

Oh thanks, let me take a look…

@charliermarsh charliermarsh self-assigned this Nov 20, 2024
@charliermarsh charliermarsh added the bug Something isn't working label Nov 20, 2024
@charliermarsh
Copy link
Member

Thanks, it was an oversight on my part.

charliermarsh added a commit that referenced this issue Nov 20, 2024
…9273)

## Summary

This was an oversight in the initial implementation. We shouldn't
validate sources for the `build-system.requires` field, since extras and
groups can _never_ be active.

Closes #9259.
nbovee added a commit to ali-izhar/tracr that referenced this issue Nov 20, 2024
nbovee added a commit to nbovee/tracr that referenced this issue Dec 20, 2024
* font attribution added

* font attribution added

* copy repo script reads device params from device config and fixes ssh permission issues

* code speedup and cleaning

* alexnet working

* code deduplication and speedup

* mit license added

* updated README

* updated README

* wsl setup and port forwarding

* wsl setup and port forwarding

* documentation update

* dynamic model registry based on model_name config

* registry can handle different models and datasets with pretrained weights

* registry update

* custom inference postprocessing template

* code restructuring

* data compression updated for custom error messages and dataclasses

* file manager modularized

* ssh session managment uses dataclasses

* device management fully utilizes ssh key handlers

* log manager handles devices correctly

* network client modular

* interface extended

* datasets simplified

* custom model registry updated

* inference utils modularized

* log updates

* registry updated for dynamic model loading

* alexnet imagenet visualize results functional

* visualization config update

* toml configuration

* toml configuration

* readme title update

* attribution, offset version to avoid clash with archived package

* change repo

* add optional dependency pathways (awaiting bugfix from astral-sh/uv#9259)

* add lockfile

* marker ensures cpu extra is flagged if no extra provided

* ruff format

* dependency locks

* friendly email address

* shebangs

* drop black

* lockfile dupes possible in uv0.5.4

* add yaml, remove build for now

* ruff lint & format

* bump reqs

* NotDict -> EarlyOutput for clarity

* obselete hook_style, 'pre' style is no longer used, leaving only one option

* pre-hook shouldn't ever exit

* re-named values to match current functionality

* pre-hook never needs to overwrite from banked_outputs

* simplify posthook

* clarity in variable names

* styling

* dynamic model registry tested to be working with new configs

* add fallback to run entire experiment on server

* yolosplit yaml config name change to yolov8split

* remove explicit post hook from configs

* optimize compression and reduce network overhead

* torch no grad covers both decompression and processing

* prefetch dataloader to refuce dict lookups

* cache torch softmax, vectorize detection bboxes

* list comps instead of loops, lru cache for total_time calc

* cleanup

* fallback documentation

* custom configs documentation

* run dynamic splitpoint off config

* device setup for cpu

* run at splitlayer or all layers based on config

* splitlayers added to configs

* remove redundant to_device method

* read device from config

* power monitoring for edge devices

* power analyzer for detailed consumption metrics

* device hardware utilization metrics

* bug fix: copy results over to server when experiment ends

* Update README.md

restore portion of SS's readme

---------

Co-authored-by: ali-izhar <[email protected]>
Co-authored-by: Izhar Ali <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants