Meta Llama 3

[jingli-wtbox]

Based on the competition rules, any public models can be used if they are uploaded before May 31st 2024.
But, i think transformers>=4.40.0 is required for llama 3 (current version is 4.39.3 in yml file). Is there any problem with submitting a solution if any package is upgraded?
thanks.

[dptam]

Hi, sorry for the delay. transformers>=4.40.0 or other package upgrades are fine. Just specify it in llm_merging/setup.py

[NewJerseyStyle]

Based on the competition rules, any public models can be used if they are uploaded before May 31st 2024.
But I saw update to their repository after May 31st 2024. If model got update after May 31st 2024... Do we need to write a function to verify the last modification date of the weight files in the huggingface repository?

[leloykun]

Some of these updates are significant bugfixes so it'd be counterproductive to limit the allowed commits to only those made before May 31st. However, some of these commits also update the model weights & it'd be unfair to allow them. So, I propose we follow the following rules instead:

1. We only allow the use of model weights uploaded to huggingface prior to May 31st. Any commits that include model weight updates past May 31st should be banned from use.
2. In our submissions, all parts of code that download model weights from huggingface, e.g. model = AutoModel.from_pretrained(...), must have the commit ID pinned. This is for transparency & to make sure we're not using model weights that was uploaded past May 31st.
3. We allow all non-weight updates even if they got commit past May 31st. E.g. bugfixes, minor config updates, etc.

[NewJerseyStyle]

I'd like to support your proposal.

But then we are facing an issue: How to implement the pinned commit ID? Any idea? I found this part difficult, but this technical challenge is not a part of the challenge 🤔

[leloykun]

We can do something like:

AutoModel.from_pretrained(
    MODEL_ID,
    revision=MODEL_REVISION,
    token=os.environ["HF_TOKEN"],
)

or

from huggingface_hub import snapshot_download
from transformers.utils import move_cache

os.makedirs(MODEL_DIR, exist_ok=True)

snapshot_download(
    MODEL_ID,
    revision=MODEL_REVISION,
    local_dir=MODEL_DIR,
    ignore_patterns=["*.pt", "*.bin"],  # Using safetensors
    token=os.environ["HF_TOKEN"],
)
move_cache()

[dptam]

Good suggestion - thank you! We have modified the code to allow for adding revision when loading models.