Code

Evo2#694

Code

Open

jstjohn wants to merge 113 commits intomainNVIDIA/bionemo-framework:mainfrom evo2NVIDIA/bionemo-framework:evo2Copy head branch name to clipboard

+7,630-29

Commits

Commits on Nov 16, 2024

• [cye/evo2-llm-dev] Private internal development branch for Evo2 in BioNeMo.

  cspades

  committed
  50db0ca

  Copy full SHA for 50db0ca

Commits on Dec 4, 2024

• [cye/evo2-llm-dev] Add rough draft of data preprocessing for Evo2.

  

  cspades

  authored and

  jstjohn

  committed
  737f16c

  Copy full SHA for 737f16c

• Add manual data test for evo2

  jstjohn

  committed
  a142109

  Copy full SHA for a142109

Commits on Dec 5, 2024

• Change remotes for submodules for now

  jstjohn

  committed
  0ad0bee

  Copy full SHA for 0ad0bee

• Cye/nemo2 fixes

  cspades

  committed
  82c832f

  Copy full SHA for 82c832f

Commits on Dec 10, 2024

• Write model checkpoint context and set Evo2Dataset in the pre-training.

  

  cspades

  authored and

  jstjohn

  committed
  945506f

  Copy full SHA for 945506f

Commits on Dec 11, 2024

• Fix inference script to make sense, i.e. no seq parallelism for decoding, default 0 for topk/topp.

  cspades

  committed
  4fc1d84

  Copy full SHA for 4fc1d84

Commits on Dec 16, 2024

• Cye/fix Hyena species biases

  cspades

  committed
  f5adde5

  Copy full SHA for f5adde5

Commits on Dec 19, 2024

• Hyena golden value test

  jstjohn

  committed
  b9dfd5c

  Copy full SHA for b9dfd5c

Commits on Dec 21, 2024

• [cye/blended-training] Expose blended weights for training Hyena.

  cspades

  committed
  e6278d9

  Copy full SHA for e6278d9

Commits on Dec 23, 2024

• Changes for 256 node training run

  

  jstjohn

  authored and

  cspades

  committed
  dd0aab1

  Copy full SHA for dd0aab1

Commits on Dec 24, 2024

• Integrate BioNeMo Noodles into Hyena data preprocessing.

  cspades

  committed
  0560ee4

  Copy full SHA for 0560ee4

Commits on Jan 3, 2025

• [cye/lineage-str] Clean up interface for taxonomic lineage tokens in Hyena.

  cspades

  committed
  5511fe7

  Copy full SHA for 5511fe7

• Changes made on 256 node branch

  jstjohn

  committed
  92d0352

  Copy full SHA for 92d0352

• Cye/hyena flops

  cspades

  committed
  923cbdf

  Copy full SHA for 923cbdf

• Fix broken import of blended training config.

  cspades

  committed
  6460ea3

  Copy full SHA for 6460ea3

• Cye/import fix

  cspades

  committed
  7e72f48

  Copy full SHA for 7e72f48

Commits on Jan 6, 2025

• Add improved nsys profiling support

  jstjohn

  committed
  45923c6

  Copy full SHA for 45923c6

Commits on Jan 7, 2025

• [cye/hyena-doc-update] Add data preprocessing documentation, fix tech debt in tokenizer and config, remove unused args in infer.py.

  cspades

  committed
  c805984

  Copy full SHA for c805984

Commits on Jan 8, 2025

• [cye/transcript-readme] Add main documentation snippets for Hyena, and add transcript splicing script for preprocessing.

  cspades

  committed
  f5b15f3

  Copy full SHA for f5b15f3

Commits on Jan 10, 2025

• Bump nemo version to the new context length insensitive code, and update test to use new checkpoint

  jstjohn

  committed
  9ba9e07

  Copy full SHA for 9ba9e07

Commits on Jan 13, 2025

• added flag for tflops callback

  dorotat-nv

  committed
  854951f

  Copy full SHA for 854951f

• [cye/evo2-ckpt-utils] Add Evo2 ZeRO-1/3 to NeMo checkpointing utils.

  cspades

  committed
  ada349e

  Copy full SHA for ada349e

Commits on Jan 14, 2025

• Add test for evo2 tokenizer.

  jwilber

  committed
  652dfe0

  Copy full SHA for 652dfe0

• Fix nemo-savanna repo build in CI

  dorotat-nv

  committed
  265a0be

  Copy full SHA for 265a0be

• fixing format issues on evo2-dev

  dorotat-nv

  committed
  fb09377

  Copy full SHA for fb09377

• Add tests for parallel hyena operators used in evo2

  jwilber

  committed
  9cacf1b

  Copy full SHA for 9cacf1b

• Rebase on OSS.

  cspades

  committed
  9ac11eb

  Copy full SHA for 9ac11eb

Commits on Jan 15, 2025

• [cye/tp-comm-fix] Fix TP communication overlap inconsistency.

  cspades

  committed
  5631b93

  Copy full SHA for 5631b93

Commits on Jan 16, 2025

• Add temporary fix for shard-tensor bug in Megatron-LM

  dorotat-nv

  committed
  9ae9af0

  Copy full SHA for 9ae9af0

Commits on Jan 17, 2025

• Add initial test for preprocess.py

  jwilber

  committed
  c032408

  Copy full SHA for c032408

• Bump NeMo to pick up FLOPS calculations.

  cspades

  committed
  b6d238f

  Copy full SHA for b6d238f

Commits on Jan 21, 2025

• [cye/z3-log-fix] Fix parameter count log.

  

  cspades

  authored and

  jomitchellnv

  committed
  7822c04

  Copy full SHA for 7822c04

Commits on Jan 22, 2025

• [cye/docker-patch-fix] Move Megatron patch to BioNeMo base image in Dockerfile.

  cspades

  committed
  9378223

  Copy full SHA for 9378223

Commits on Jan 24, 2025

• shipping hotfix for dockers built locally - fix from main 17c6b20

  dorotat-nv

  committed
  9b9176a

  Copy full SHA for 9b9176a

• [cye/1m-ckpt-config] Add HyenaConfig options for 1M context length dimensions.

  cspades

  committed
  329548a

  Copy full SHA for 329548a

• [cye/fix-tp-comm-overlap] Fix default tp_comm_overlap=True being used in inference, which is not appropriate.

  cspades

  committed
  2ca40b0

  Copy full SHA for 2ca40b0

Commits on Jan 27, 2025

• reducing scope of tested folders for evo2-dev

  dorotat-nv

  committed
  a494478

  Copy full SHA for a494478

Commits on Jan 28, 2025

• Adds basic inference test

  jomitchellnv

  committed
  72a311e

  Copy full SHA for 72a311e

• [cye/deactivate-infer-tpcomm] Deactivate TP communication during inference to support those checkpoints.

  cspades

  committed
  d4cd785

  Copy full SHA for d4cd785

• fix: ensure test looks in test file dir for required data

  Show description for 3ba8946
  jwilber

  committed
  3ba8946

  Copy full SHA for 3ba8946

Commits on Jan 29, 2025

• m2.5 accuracy 7b runs

  jstjohn

  committed
  34938fc

  Copy full SHA for 34938fc

Commits on Jan 30, 2025

• Fixes test_evo2.py unit test and adds enhancements to existing unit tests for Evo2 inference.

  jomitchellnv

  committed
  5fe2576

  Copy full SHA for 5fe2576

• Fix bug in wandb logger argparse.

  jstjohn

  committed
  30e71e9

  Copy full SHA for 30e71e9

Commits on Feb 3, 2025

• [cye/pad-loss-mask] Fixes TP comm overlap bug with sequence parallel and reduce the size of torch_dist checkpoints.

  cspades

  committed
  635a5df

  Copy full SHA for 635a5df

• Add longphase dataset config to repo

  jwilber

  committed
  f141830

  Copy full SHA for f141830

Commits on Feb 5, 2025

• bump Megatron-LM, nemo-savanna and rebase to main OSS

  dorotat-nv

  committed
  493d444

  Copy full SHA for 493d444

Commits on Feb 7, 2025

• CI hotfix

  dorotat-nv

  committed
  1df0176

  Copy full SHA for 1df0176

• test: Create tests for Evo2Dataset mask_phylogenetic_tags

  jwilber

  committed
  624e797

  Copy full SHA for 624e797

Commits on Feb 11, 2025

• [cye/torch_dist_fix] Remove torch_dist patch and bump Megatron, reorganize tests.

  cspades

  committed
  75205b0

  Copy full SHA for 75205b0

• Changes related to accuracy and perf with new nemo2 changes

  jstjohn

  committed
  0f6efeb

  Copy full SHA for 0f6efeb

Commits on Feb 12, 2025

• [cye/tp-comm-fp8-wgrad-fix]Require --fp8-wgrad when using TP communication overlap.

  

  Cory Ye

  authored and

  cspades

  committed
  0af5f9e

  Copy full SHA for 0af5f9e

Commits on Feb 13, 2025

• Adding evo2 to JET

  dorotat-nv

  committed
  0917616

  Copy full SHA for 0917616

• Remove sample data from evo2-dev branch

  dorotat-nv

  committed
  3d1e19e

  Copy full SHA for 3d1e19e

• [BUGFIX] evo2-dev CI

  dorotat-nv

  committed
  9811ae4

  Copy full SHA for 9811ae4

Commits on Feb 14, 2025

• Remove test_mask_phylogenetic tags (moving to nemo repo)

  jwilber

  committed
  f9133f5

  Copy full SHA for f9133f5

• attempt at merge -- nemo matches github main. Dockerfile has major bump to pytorch.

  skothenhill-nv

  committed
  e83bd26

  Copy full SHA for e83bd26

• Bump nemo to fix forward bug

  Show description for a175d5b
  jstjohn

  committed
  a175d5b

  Copy full SHA for a175d5b

• Add required changes to work with NeMo upstream

  Show description for ea70cde
  jstjohn

  committed
  ea70cde

  Copy full SHA for ea70cde

• Add back new context manager for parallel state cleanup

  Show description for dddf9a4
  jstjohn

  committed
  dddf9a4

  Copy full SHA for dddf9a4

• Move test_config into nemo where the code is

  Show description for e192982
  jstjohn

  committed
  e192982

  Copy full SHA for e192982

Commits on Feb 15, 2025

• Fix arg name mismatch

  Show description for d355729
  jstjohn

  committed
  d355729

  Copy full SHA for d355729

• add new license

  Show description for d90c10d
  jwilber

  committed
  d90c10d

  Copy full SHA for d90c10d

• remove tab from license

  Show description for a8432a2
  jwilber

  committed
  a8432a2

  Copy full SHA for a8432a2

• Bump nemo to fix bug in dataset

  jstjohn

  committed
  4f2ade5

  Copy full SHA for 4f2ade5

Commits on Feb 18, 2025

• Bump NeMo commit for perf improved loss mask

  Show description for 3965502
  jstjohn

  committed
  3965502

  Copy full SHA for 3965502

• Adding options for controlling dropout to train.py

  Show description for f09aa36
  jstjohn

  committed
  f09aa36

  Copy full SHA for f09aa36

• Bump nemo and remove nograd decorator

  Show description for 955978d
  jstjohn

  committed
  955978d

  Copy full SHA for 955978d

• Bump nemo with latest tag masking

  jstjohn

  committed
  3e14262

  Copy full SHA for 3e14262

• Cover non-DNA case due to bug in preprocessing, never have non-dna unmasked in loss other than pad/eod

  Show description for 46baa5f
  jstjohn

  committed
  46baa5f

  Copy full SHA for 46baa5f

• Try reverting some of the recent fixes related to TP

  Show description for ef3f55e
  jstjohn

  committed
  ef3f55e

  Copy full SHA for ef3f55e

• Bump nemo version with better tested

  jstjohn

  committed
  af9016e

  Copy full SHA for af9016e

• Revert loss mask updates

  jstjohn

  committed
  aafb7a3

  Copy full SHA for aafb7a3

• handle 0 token case more gracefully

  jstjohn

  committed
  a966b8b

  Copy full SHA for a966b8b

Commits on Feb 19, 2025

• bump NeMo with proper handling of control character containing sequences and handling divide by zero in loss

  jstjohn

  committed
  c4ef1f1

  Copy full SHA for c4ef1f1

• Update remote pointers to new public NeMo branches

  jstjohn

  committed
  0976fac

  Copy full SHA for 0976fac

• Remove unused Megatron torch_dist sizing patch.

  cspades

  committed
  04982ae

  Copy full SHA for 04982ae

• Remove fasta from test and replace with synthetic sequence

  Show description for 242f3fe
  jstjohn

  committed
  242f3fe

  Copy full SHA for 242f3fe

• Move fasta creation utility into testing sub-package

  Show description for 22ada77
  jstjohn

  committed
  22ada77

  Copy full SHA for 22ada77

• Add a test that verifies that the new phylo tag masking code is faster than the old code

  Show description for b5bdec8
  jstjohn

  committed
  b5bdec8

  Copy full SHA for b5bdec8

• Move phylo tag benchmark to NeMo testing

  Show description for ac1bd1f
  jstjohn

  committed
  ac1bd1f

  Copy full SHA for ac1bd1f

Commits on Feb 20, 2025

• Merge in main

  Show description for bfaebd1
  jstjohn

  committed
  bfaebd1

  Copy full SHA for bfaebd1

• Update Megatron-LM submodule to commit 62529f1d (has 1M context fix) (#697)

  Show description for 0ae0c50
  jwilber

  authored
  0ae0c50

  Copy full SHA for 0ae0c50

Commits on Feb 21, 2025

• fix config typo in test

  Show description for 2ba5da3
  jstjohn

  committed
  2ba5da3

  Copy full SHA for 2ba5da3

• bump NeMo to latest PR version

  jstjohn

  committed
  253a7f2

  Copy full SHA for 253a7f2

• Fix issue causing gh-docs-deploy failure (#698)

  Show description for 82e9c47
  jwilber

  authored
  82e9c47

  Copy full SHA for 82e9c47

• Update nemo pointer with PR updates

  Show description for fa73a00
  jstjohn

  committed
  fa73a00

  Copy full SHA for fa73a00

• Add new license to new files (failing ci) (#699)

  Show description for f466774
  jwilber

  authored
  f466774

  Copy full SHA for f466774

• Change kingdom to domain in tag description

  Show description for 39290e4
  jstjohn

  committed
  39290e4

  Copy full SHA for 39290e4

• Merge in upstream

  Show description for b688975
  jstjohn

  committed
  b688975

  Copy full SHA for b688975

• Merge branch 'main' of github.com:NVIDIA/bionemo-framework into evo2

  Show description for 94c4283
  jstjohn

  committed
  94c4283

  Copy full SHA for 94c4283

Commits on Feb 22, 2025

• Make new versions of the files available freshly converted from HF

  jstjohn

  committed
  15c7dca

  Copy full SHA for 15c7dca

• bump nemo version to fix broken import

  jstjohn

  committed
  3324bd4

  Copy full SHA for 3324bd4

• bump nemo to top of tree

  jstjohn

  committed
  ce133d2

  Copy full SHA for ce133d2

Commits on Feb 24, 2025

• Adding in the predict method and test

  Show description for 78f92b5
  jstjohn

  committed
  78f92b5

  Copy full SHA for 78f92b5

• Merge branch 'main' of github.com:NVIDIA/bionemo-framework into evo2

  Show description for bb5f5a1
  jstjohn

  committed
  bb5f5a1

  Copy full SHA for bb5f5a1

• bump NeMo commit

  Show description for d9e4952
  jstjohn

  committed
  d9e4952

  Copy full SHA for d9e4952

• Fix multipart download naming in nemo

  Show description for b148750
  jstjohn

  committed
  b148750

  Copy full SHA for b148750

• Update docs for checkpoint conversion

  Show description for ba1d9bf
  jstjohn

  committed
  ba1d9bf

  Copy full SHA for ba1d9bf

• shrink tests down to 1b case

  jstjohn

  committed
  0af3e0a

  Copy full SHA for 0af3e0a

Commits on Feb 25, 2025

• add end to end fine-tuning tutorial

  Show description for c5e42d8
  jstjohn

  committed
  c5e42d8

  Copy full SHA for c5e42d8

• ignore object hashes in precommit

  Show description for 544b7a8
  jstjohn

  committed
  544b7a8

  Copy full SHA for 544b7a8

• Bump nemo pointer to latest PR pointer

  Show description for d7a8ea7
  jstjohn

  committed
  d7a8ea7

  Copy full SHA for d7a8ea7

• Update ci/benchmarks/partial-conv/evo2_pretrain.yaml

  Show description for 07c48b8
  

  jstjohn

  and

  dorotat-nv

  authored
  07c48b8

  Copy full SHA for 07c48b8

• Update ci/benchmarks/perf/evo2_pretrain.yaml

  Show description for e779f60
  

  jstjohn

  and

  dorotat-nv

  authored
  e779f60

  Copy full SHA for e779f60

• Slightly smaller test_train.py

  Show description for a1c8048
  jstjohn

  committed
  a1c8048

  Copy full SHA for a1c8048

• Add missing main function for inference cli

  jstjohn

  committed
  46edcb6

  Copy full SHA for 46edcb6

• Add --batch-size option to predict

  Show description for e81eef3
  jstjohn

  committed
  e81eef3

  Copy full SHA for e81eef3

• Fixing the description of the 1b model

  Show description for 4e5acda
  jstjohn

  committed
  4e5acda

  Copy full SHA for 4e5acda

Commits on Feb 26, 2025

• remove hard-coded PBSS

  Show description for 5bd0e2c
  jstjohn

  committed
  5bd0e2c

  Copy full SHA for 5bd0e2c

• Remove comment block from code

  Show description for ca16c2a
  jstjohn

  committed
  ca16c2a

  Copy full SHA for ca16c2a

Commits on Feb 27, 2025

• evo2 train unit test (#704)

  Show description for 5248e5d
  dorotat-nv

  authored
  5248e5d

  Copy full SHA for 5248e5d

Commits on Feb 28, 2025

• Updates to benchmarks: evo2 (#705)

  Show description for 1e7323b
  dorotat-nv

  authored
  1e7323b

  Copy full SHA for 1e7323b