Evo2 merge 20250214 #12263
Evo2 merge 20250214 #12263
Conversation
Co-authored-by: Ali Taghibakhshi <[email protected]> Co-authored-by: Cory Ye <[email protected]> Co-authored-by: Dorota Toczydlowska <[email protected]> Co-authored-by: Guy Jacob <[email protected]> Co-authored-by: Jared Wilber <[email protected]> Co-authored-by: John St. John <[email protected]> Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
Alit/evo2 merge 20250214 See merge request ataghibakhsh/nemo-savanna!41
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
JRD971000
requested review from
marcromeyn,
hemildesai,
suiyoubi,
cuichenx and
yaoyu-33
Signed-off-by: JRD971000 <[email protected]>
Signed-off-by: artbataev <[email protected]>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
Signed-off-by: John St John <[email protected]>
| TENorm, | ||
| TERowParallelLinear, | ||
| ) | ||
| except ImportError: |
Check notice
Code scanning / CodeQL
Empty except Note
| from nemo.collections.llm.gpt.model.megatron.hyena.hyena_utils import make_upper_case | ||
|
|
||
|
|
||
| class Evo2Dataset(GPTDataset): |
There was a problem hiding this comment.
should this dataset be here on in bionemo? cc @jstjohn
There was a problem hiding this comment.
it is very bio tech specific
| from pydantic import BaseModel, model_validator | ||
|
|
||
|
|
||
| def infer_global_batch_size( |
There was a problem hiding this comment.
why this method is in this file? we used to have it in bionemo, do you need it in NeMo? if yes, then it shouldn't be under megatron/hyena
| return global_batch_size | ||
|
|
||
|
|
||
| class Evo2BlendedDatasetConfig(BaseModel): |
There was a problem hiding this comment.
also, shouldn't Evo2BlendedDatasetConfig in fact be BlendedDatasetConfig and be located somewhere here?
There was a problem hiding this comment.
it seems that in NeMo you pass mostly paths from command line
NeMo/scripts/llm/gpt_distillation.py
Line 109 in b94bff7
There was a problem hiding this comment.
Thanks @dorotat-nv , we'll have a separate following PR with the cleaups you suggested. |
Important
The
Update branchbutton must only be pressed in very rare occassions.An outdated branch is never blocking the merge of a PR.
Please reach out to the automation team before pressing that button.
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Collection: [Note which collection this PR will affect]
Changelog
Usage
# Add a code snippet demonstrating how to use thisGitHub Actions CI
The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.
The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".
Before your PR is "Ready for review"
Pre checks:
PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information