Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update old existing feature extractor references #24552

Merged

Conversation

amyeroberts
Copy link
Collaborator

@amyeroberts amyeroberts commented Jun 28, 2023

What does this PR do?

Updates a bunch of old references to feature extractors for vision models.

Most of the code isn't public facing in e.g. docs, but is often copied when new models are added.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Jun 28, 2023

The documentation is not available anymore as the PR was closed or merged.

@amyeroberts amyeroberts changed the title WIP: Update old existing feature extractor references Update old existing feature extractor references Jun 28, 2023
@amyeroberts amyeroberts requested a review from ydshieh June 28, 2023 16:03
Copy link
Collaborator

@ydshieh ydshieh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow. Thank you for this!

I just randomly looked the changes and got lucky to leave some comments.

The suggested changes exist in many files though, you might have to search/replace (but be careful)


if push_to_hub:
if has_lm_head:
model_name = "dit-base" if "base" in checkpoint_url else "dit-large"
else:
model_name = "dit-base-finetuned-rvlcdip" if "dit-b" in checkpoint_url else "dit-large-finetuned-rvlcdip"
feature_extractor.push_to_hub(
image_processor.push_to_hub(
repo_path_or_name=Path(pytorch_dump_folder_path, model_name),
organization="nielsr",
commit_message="Add feature extractor",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

randomly find something to improve 😆

commit_message="Add feature extractor",

to

commit_message="Add image processor",

@@ -132,11 +132,11 @@ def convert_glpn_checkpoint(checkpoint_path, pytorch_dump_folder_path, push_to_h
config = GLPNConfig(hidden_sizes=[64, 128, 320, 512], decoder_hidden_size=64, depths=[3, 8, 27, 3])

# load feature extractor (only resize + rescale)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got lucky today:

load image processor

I am going to buy loto

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:D You're more thorough than I! Can you tell I didn't search for "feature extractor" with a space?

@@ -234,7 +234,7 @@ def convert_dpt_checkpoint(checkpoint_url, pytorch_dump_folder_path, push_to_hub
print(f"Saving model to {pytorch_dump_folder_path}")
model.save_pretrained(pytorch_dump_folder_path)
print(f"Saving feature extractor to {pytorch_dump_folder_path}")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another one:

Saving image processor

@@ -366,7 +366,7 @@ def convert_segformer_checkpoint(model_name, checkpoint_path, pytorch_dump_folde
logger.info(f"Saving PyTorch model and feature extractor to {pytorch_dump_folder_path}...")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

@@ -147,12 +147,12 @@ def convert_swin_checkpoint(model_name, checkpoint_path, pytorch_dump_folder_pat
model.save_pretrained(pytorch_dump_folder_path)

print(f"Saving feature extractor to {pytorch_dump_folder_path}")
feature_extractor.save_pretrained(pytorch_dump_folder_path)
image_processor.save_pretrained(pytorch_dump_folder_path)

if push_to_hub:
print(f"Pushing model and feature extractor for {model_name} to hub")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

@amyeroberts
Copy link
Collaborator Author

@ydshieh Thanks for pointing out the extra places I missed. I've updated + other vision files needing the same update.

@amyeroberts amyeroberts merged commit ae454f4 into huggingface:main Jun 29, 2023
@amyeroberts amyeroberts deleted the remove-more-old-fe-references branch June 29, 2023 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants