Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding pyright to pre-commit #477

Merged
merged 15 commits into from
Aug 2, 2023
Merged

Adding pyright to pre-commit #477

merged 15 commits into from
Aug 2, 2023

Conversation

bcui19
Copy link
Contributor

@bcui19 bcui19 commented Jul 20, 2023

This PR adds pyright to precommit and gets it to pass so that we stop making things worse. Things that should still be done in the future include (1) getting rid of as many type ignores as possible and (2) adding return types to stuff (3) fix a todo with the model gauntlet callback which currently does not actually work as a callback.

I've also removed python 3.8 from the list of python versions. The ast.unparse function used in the editing of the files for huggingface compatibility does not exist on python 3.8, and I think it is ok to require >=3.9

  • Run through quickstart and make sure nothing is broken

@dakinggg dakinggg marked this pull request as ready for review August 1, 2023 04:28
Copy link
Collaborator

@hanlint hanlint left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seriously, thank you for doing this. I only reviewed the llmfoundry folder files (stopped at pyproject.toml in this first pass). A few general notes:

  • I saw MutableMapping or Mapping used for batch. Is it possible to be consisent.. and is there a more narrow type, e.g. Dict that could be used?
  • noted a few assert that maybe shoudl be informative user errors

llmfoundry/callbacks/generate_callback.py Outdated Show resolved Hide resolved
llmfoundry/callbacks/generate_callback.py Show resolved Hide resolved
llmfoundry/data/denoising.py Show resolved Hide resolved
llmfoundry/models/hf/hf_prefix_lm.py Show resolved Hide resolved
llmfoundry/models/layers/ffn.py Show resolved Hide resolved
@dakinggg dakinggg changed the title [Draft] Adding pyright to pre-commit Adding pyright to pre-commit Aug 1, 2023
scripts/eval/eval.py Outdated Show resolved Hide resolved
Copy link
Contributor

@vchiley vchiley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bunch of ignorable comments
besides random nits, lgtm

llmfoundry/callbacks/generate_callback.py Outdated Show resolved Hide resolved
llmfoundry/callbacks/model_gauntlet_callback.py Outdated Show resolved Hide resolved
llmfoundry/data/data.py Show resolved Hide resolved
llmfoundry/callbacks/generate_callback.py Show resolved Hide resolved
llmfoundry/data/text_data.py Show resolved Hide resolved
scripts/train/benchmarking/submit_benchmarks.py Outdated Show resolved Hide resolved
scripts/data_prep/convert_dataset_hf.py Show resolved Hide resolved
llmfoundry/optim/lion.py Show resolved Hide resolved
llmfoundry/optim/adaptive_lion.py Show resolved Hide resolved
tests/test_model.py Show resolved Hide resolved
setup.py Show resolved Hide resolved
@dakinggg dakinggg merged commit 9250e84 into main Aug 2, 2023
@dakinggg dakinggg deleted the add_pyright branch October 11, 2023 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants