Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature Improvement: Automatically download and load MetaCLIP checkpoints #29

Merged
merged 3 commits into from
Nov 27, 2023
Merged

Feature Improvement: Automatically download and load MetaCLIP checkpoints #29

merged 3 commits into from
Nov 27, 2023

Conversation

Andy-LZH
Copy link
Contributor

@Andy-LZH Andy-LZH commented Nov 8, 2023 •
edited
Loading

Feature Improvement based on Issue #26

Code modified/added

  • src/open_clip/pretrained.py
  • tests/test.py

Now loading MetaCLIP(ViT-B-32, ViT-B32-quickgelu, ViT-B-16, ViT-L-14) can be done as:

# Load MetaCLIP B32 400M
model, _, preprocess = open_clip.create_model_and_transforms('ViT-B-32-quickgelu', pretrained='metaclip400m')

# Load MetaCLIP B32 2.5B
model, _, preprocess = open_clip.create_model_and_transforms('ViT-B-32-quickgelu', pretrained='metaclip2_5b')

And thanks for reviewing this PR in advance.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 8, 2023
howardhsu
howardhsu reviewed Nov 27, 2023
View reviewed changes
src/open_clip/pretrained.py Outdated
)

_VITB32_quickgelu = dict(
openai="https://openaipublic.azureedge.net/clip/models/40d365715913c9da98579312b702a82c18be219cc2a73407c4526f58eba950af/ViT-B-32.pt",
laion400m_e31="https://github.com/mlfoundations/open_clip/releases/download/v0.2-weights/vit_b_32-quickgelu-laion400m_e31-d867053b.pt",
laion400m_e32="https://github.com/mlfoundations/open_clip/releases/download/v0.2-weights/vit_b_32-quickgelu-laion400m_e32-46683a32.pt",
metaclip400m="https://dl.fbaipublicfiles.com/MMPT/metaclip/b32_400m.pt",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for naming suggestions.

howardhsu
howardhsu reviewed Nov 27, 2023
View reviewed changes
src/open_clip/pretrained.py Outdated
@@ -46,18 +46,24 @@
laion2b_e16="https://github.com/mlfoundations/open_clip/releases/download/v0.2-weights/vit_b_32-laion2b_e16-af8dbd0c.pth",
laion400m_e31="https://github.com/mlfoundations/open_clip/releases/download/v0.2-weights/vit_b_32-quickgelu-laion400m_e31-d867053b.pt",
laion400m_e32="https://github.com/mlfoundations/open_clip/releases/download/v0.2-weights/vit_b_32-quickgelu-laion400m_e32-46683a32.pt",
metaclip400m="https://dl.fbaipublicfiles.com/MMPT/metaclip/b32_400m.pt",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all MetaCLIP are trained on quickgelu; let's NOT use nn.gelu explicitly.

@howardhsu howardhsu mentioned this pull request Nov 27, 2023
Closed
@howardhsu howardhsu merged commit 9c553d6 into facebookresearch:main Nov 27, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@howardhsu howardhsu howardhsu approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Andy-LZH @howardhsu @facebook-github-bot