Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update runners #3472

Closed
wants to merge 70 commits into from
Closed

update runners #3472

wants to merge 70 commits into from

Conversation

KevDevSha
Copy link
Member

What does this PR do?

Updating to allow listed runners

@KevDevSha KevDevSha requested a review from a team as a code owner July 15, 2024 20:57
@KevDevSha KevDevSha requested a review from mvpatel2000 July 15, 2024 20:57
Copy link
Contributor

@mvpatel2000 mvpatel2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, @KevDevSha can you resolve lint errors and then merge?

KuuCi and others added 27 commits July 21, 2024 18:32
Co-authored-by: v-chen_data <[email protected]>
* a

* a'

* a

* a

* a

* a

* a

* a

* a

* Apply suggestions from code review

Co-authored-by: Mihir Patel <[email protected]>

---------

Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Mihir Patel <[email protected]>
…3392)

Updates the requirements on [packaging](https://github.com/pypa/packaging) to permit the latest version.
- [Release notes](https://github.com/pypa/packaging/releases)
- [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst)
- [Commits](pypa/packaging@21.3...24.1)

---
updated-dependencies:
- dependency-name: packaging
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.6 to 42.0.8.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](pyca/cryptography@42.0.6...42.0.8)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Add buffering time to mlflow logger

* rename

* change default and fix comments
…ant, necessary gpu metrics logged (#3373)

* implemented min_max flag

* fixed string parsing

* refactoring compute_system_metrics for all_reduce

* keep track of rank within dict

* added compute_min_max

* added flag for both min_max and all_logging

* corrected min_max call with model_device

* removing total bytes (always going ot be constant)

* handled no gpu case in min_max flag

* removed unnecessary imports, patched unit tests

* fixed assert statement for with gpu case, world size 1

* case min_rank and max_rank as int to guarantee them working as indices

* fixed indent issue from fixing font

* made docs more concise and readable

* fixing unexpected unindent

* fixing unit test device

* modifying device to equal model_device.type

* reverting to device=model_device

* setting device in unit test = 'gpu'

* setting device = 'cuda' in unit testing

* reverting to next(state.model.parameters()).device

* removed torch as a dependecy for unit_testing

* cleaned up UI to be consistent + removed calling next to obtain device

---------

Co-authored-by: Mihir Patel <[email protected]>
Co-authored-by: Charles Tang <[email protected]>
* yo

* slam

* cuda

* cuda checks

* test

* fix_test

* gloo

* gloo

* lint

* lint

---------

Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Mihir Patel <[email protected]>
* log image fix

Signed-off-by: Jesse Chan <[email protected]>

* fixed log image tests

Signed-off-by: Jesse Chan <[email protected]>

* linter

Signed-off-by: Jesse Chan <[email protected]>

* add simd requirement

* post0?

* versioning yada yada yada

* guh

* import fix?

* update deps

* fix

* fix II

* remove other dependency

* debug statement, remove

* post1?!

* build from source

* whitespace?

* use pillow

* delete a unit test and ignore some types

* s/type/pyright

* formatting

* formatting

* ignore more stuff

* Apply suggestions from code review

* remove rest

* Update setup.py

Co-authored-by: Mihir Patel <[email protected]>

* try no ignore

* remove intenum

---------

Signed-off-by: Jesse Chan <[email protected]>
Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Milo Cress <[email protected]>
Co-authored-by: Mihir Patel <[email protected]>
* Revert "Optionally use `flash-attn`'s CE loss for metrics (#3394)"

This reverts commit 2cf9262.

revert dat boi

* remove

* slamm
* move pil dep

* remove pillow simd
…/ prefix (#3410)

* fixing os file path with /Shared/ prefix

* lstrip '/' from experiment name if not '/Shared/' or '/Users/'

Co-authored-by: Mihir Patel <[email protected]>

* doesnt modify experiment name if it has '/Shared/' as a prefix

* fix formatting

* lint

---------

Co-authored-by: Mihir Patel <[email protected]>
* Bump version v0.23.3

* update the composer version
)

* Update numpy requirement from <1.27.0,>=1.21.5 to >=1.21.5,<2.1.0

Updates the requirements on [numpy](https://github.com/numpy/numpy) to permit the latest version.
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](numpy/numpy@v1.21.5...v2.0.0)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* commit

* fix typing

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mihir Patel <[email protected]>
Co-authored-by: Saaketh Narayan <[email protected]>
mvpatel2000 and others added 19 commits July 21, 2024 18:32
* test

* test

* test

* test

* test

* test

* fix

* sleep before skip

* fix

* pull request target

* revert

* revery pr_target branches

* sleep 1

* 10 sec

* uncomment

* dist barrier

* test

* dist works!

* update 0.0.9

* mihir comment

Co-authored-by: Mihir Patel <[email protected]>

---------

Co-authored-by: v-chen_data <[email protected]>
Co-authored-by: Mihir Patel <[email protected]>
Bumps [ipykernel](https://github.com/ipython/ipykernel) from 6.29.2 to 6.29.5.
- [Release notes](https://github.com/ipython/ipykernel/releases)
- [Changelog](https://github.com/ipython/ipykernel/blob/v6.29.5/CHANGELOG.md)
- [Commits](ipython/ipykernel@v6.29.2...v6.29.5)

---
updated-dependencies:
- dependency-name: ipykernel
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Updates the requirements on [torchmetrics](https://github.com/Lightning-AI/torchmetrics) to permit the latest version.
- [Release notes](https://github.com/Lightning-AI/torchmetrics/releases)
- [Changelog](https://github.com/Lightning-AI/torchmetrics/blob/master/CHANGELOG.md)
- [Commits](Lightning-AI/torchmetrics@v0.10.0...v1.4.0.post0)

---
updated-dependencies:
- dependency-name: torchmetrics
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [databricks-sdk](https://github.com/databricks/databricks-sdk-py) from 0.28.0 to 0.29.0.
- [Release notes](https://github.com/databricks/databricks-sdk-py/releases)
- [Changelog](https://github.com/databricks/databricks-sdk-py/blob/main/CHANGELOG.md)
- [Commits](databricks/databricks-sdk-py@v0.28.0...v0.29.0)

---
updated-dependencies:
- dependency-name: databricks-sdk
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ckpoint files upload (#3376)

* a

* a

* a

* a

* a

* a

* a

* a

* fix test

* a

* a

* a

* a

* fix unit test

* a

* a

* a

* a

* a

* fix 2gpu unit test

* a

* a

* a

* a

* fix doctest

* a

* fix test and lint

* up

* a

* a

* a

* a

* a

* a

* a

* a

* address comments

* a

* a

* a

* a

* rerun test

* add logging

* remove debug comments

* comments

* a

* cleanup

* a

* linter

* lint

* Update composer/callbacks/checkpoint_saver.py

Co-authored-by: Evan Racah <[email protected]>

* commenst

* a

* fix test

* fix test

* comments

* a

---------

Co-authored-by: Evan Racah <[email protected]>
* big fix

* testing

* ignore

* ignore

* ignore

* Update test_fsdp_checkpoint.py

* Update test_fsdp_checkpoint.py

---------

Co-authored-by: Mihir Patel <[email protected]>
* a

* a

* fix lint and test

* lint

* comments

* comment
Bumps [coverage[toml]](https://github.com/nedbat/coveragepy) from 7.5.4 to 7.6.0.
- [Release notes](https://github.com/nedbat/coveragepy/releases)
- [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst)
- [Commits](nedbat/coveragepy@7.5.4...7.6.0)

---
updated-dependencies:
- dependency-name: coverage[toml]
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@KevDevSha KevDevSha force-pushed the kevin/emu-allowlisted-runners branch from d1d9c29 to 2c0eac2 Compare July 22, 2024 21:22
@KevDevSha KevDevSha closed this Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.