Alit/mamba #9696

JRD971000 · 2024-07-11T15:14:03Z

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Collection: [Note which collection this PR will affect]

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to # (issue)

Signed-off-by: JRD971000 <[email protected]>

…/mamba

Signed-off-by: JRD971000 <[email protected]>

…/mamba

Signed-off-by: arendu <[email protected]>

Signed-off-by: JRD971000 <[email protected]>

…/mamba

JRD971000 · 2024-07-11T15:53:42Z

examples/nlp/language_modeling/mamba_change_num_partition.py

+
+        # print("Combined tp model: {}".format(combined_tp_model.keys()))
+
+        for ii, (key, original_tensor) in enumerate(combined_tp_model.items()):


examples/nlp/language_modeling/megatron_mamba_eval.py

examples/nlp/language_modeling/mamba_change_num_partition.py

+            try:
+                layer_num = int(re.findall(r'\d+', key)[0])
+                new_key = key.replace(str(layer_num), str(layer_num), 1)
+            except:


examples/nlp/language_modeling/megatron_mamba_eval.py

scripts/checkpoint_converters/convert_mamba2_pyt_to_nemo.py

Signed-off-by: JRD971000 <[email protected]>

examples/nlp/language_modeling/mamba_change_num_partition.py

Signed-off-by: JRD971000 <[email protected]>

examples/nlp/language_modeling/mamba_change_num_partition.py

+            try:
+                layer_num = int(re.findall(r'\d+', local_key)[0])
+                new_key = local_key.replace(str(layer_num), str(layer_num), 1)
+            except:


…/mamba

* adding mamba support * fix import mixins * rm convert jamba * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * more cleanups * use GPT text gen * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * fixing gbs in TP convetor * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * add reqs * add tutorial * minor fix to tutorial * moving finetuning files Signed-off-by: arendu <[email protected]> * moving finetuning files Signed-off-by: arendu <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * add mamba dependancies * add mcore tag * modify dockerfile ci * modify dockerfile ci * fix TP>1 to TP1 * add inference, update based on latest mcore commits * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * bug fix, tutorial update --------- Signed-off-by: JRD971000 <[email protected]> Signed-off-by: arendu <[email protected]> Co-authored-by: Ali Taghibakhshi <[email protected]> Co-authored-by: JRD971000 <[email protected]> Co-authored-by: arendu <[email protected]>

* adding mamba support * fix import mixins * rm convert jamba * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * more cleanups * use GPT text gen * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * fixing gbs in TP convetor * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * add reqs * add tutorial * minor fix to tutorial * moving finetuning files Signed-off-by: arendu <[email protected]> * moving finetuning files Signed-off-by: arendu <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * add mamba dependancies * add mcore tag * modify dockerfile ci * modify dockerfile ci * fix TP>1 to TP1 * add inference, update based on latest mcore commits * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * bug fix, tutorial update --------- Signed-off-by: JRD971000 <[email protected]> Signed-off-by: arendu <[email protected]> Co-authored-by: Ali Taghibakhshi <[email protected]> Co-authored-by: JRD971000 <[email protected]> Co-authored-by: arendu <[email protected]> Signed-off-by: Tugrul Konuk <[email protected]>

* adding mamba support * fix import mixins * rm convert jamba * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * more cleanups * use GPT text gen * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * fixing gbs in TP convetor * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * add reqs * add tutorial * minor fix to tutorial * moving finetuning files Signed-off-by: arendu <[email protected]> * moving finetuning files Signed-off-by: arendu <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * add mamba dependancies * add mcore tag * modify dockerfile ci * modify dockerfile ci * fix TP>1 to TP1 * add inference, update based on latest mcore commits * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * bug fix, tutorial update --------- Signed-off-by: JRD971000 <[email protected]> Signed-off-by: arendu <[email protected]> Co-authored-by: Ali Taghibakhshi <[email protected]> Co-authored-by: JRD971000 <[email protected]> Co-authored-by: arendu <[email protected]> Signed-off-by: Malay Nagda <[email protected]>

* adding mamba support * fix import mixins * rm convert jamba * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * more cleanups * use GPT text gen * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * fixing gbs in TP convetor * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * add reqs * add tutorial * minor fix to tutorial * moving finetuning files Signed-off-by: arendu <[email protected]> * moving finetuning files Signed-off-by: arendu <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * add mamba dependancies * add mcore tag * modify dockerfile ci * modify dockerfile ci * fix TP>1 to TP1 * add inference, update based on latest mcore commits * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * bug fix, tutorial update --------- Signed-off-by: JRD971000 <[email protected]> Signed-off-by: arendu <[email protected]> Co-authored-by: Ali Taghibakhshi <[email protected]> Co-authored-by: JRD971000 <[email protected]> Co-authored-by: arendu <[email protected]> Signed-off-by: tonyjie <[email protected]>

* adding mamba support * fix import mixins * rm convert jamba * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * more cleanups * use GPT text gen * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * fixing gbs in TP convetor * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * add reqs * add tutorial * minor fix to tutorial * moving finetuning files Signed-off-by: arendu <[email protected]> * moving finetuning files Signed-off-by: arendu <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * add mamba dependancies * add mcore tag * modify dockerfile ci * modify dockerfile ci * fix TP>1 to TP1 * add inference, update based on latest mcore commits * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * bug fix, tutorial update --------- Signed-off-by: JRD971000 <[email protected]> Signed-off-by: arendu <[email protected]> Co-authored-by: Ali Taghibakhshi <[email protected]> Co-authored-by: JRD971000 <[email protected]> Co-authored-by: arendu <[email protected]>

* adding mamba support * fix import mixins * rm convert jamba * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * more cleanups * use GPT text gen * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * fixing gbs in TP convetor * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * add reqs * add tutorial * minor fix to tutorial * moving finetuning files Signed-off-by: arendu <[email protected]> * moving finetuning files Signed-off-by: arendu <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * add mamba dependancies * add mcore tag * modify dockerfile ci * modify dockerfile ci * fix TP>1 to TP1 * add inference, update based on latest mcore commits * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * bug fix, tutorial update --------- Signed-off-by: JRD971000 <[email protected]> Signed-off-by: arendu <[email protected]> Co-authored-by: Ali Taghibakhshi <[email protected]> Co-authored-by: JRD971000 <[email protected]> Co-authored-by: arendu <[email protected]> Signed-off-by: Hainan Xu <[email protected]>

* adding mamba support * fix import mixins * rm convert jamba * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * more cleanups * use GPT text gen * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * fixing gbs in TP convetor * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * add reqs * add tutorial * minor fix to tutorial * moving finetuning files Signed-off-by: arendu <[email protected]> * moving finetuning files Signed-off-by: arendu <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * address comments * add mamba dependancies * add mcore tag * modify dockerfile ci * modify dockerfile ci * fix TP>1 to TP1 * add inference, update based on latest mcore commits * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * minor fix * Apply isort and black reformatting Signed-off-by: JRD971000 <[email protected]> * bug fix, tutorial update --------- Signed-off-by: JRD971000 <[email protected]> Signed-off-by: arendu <[email protected]> Co-authored-by: Ali Taghibakhshi <[email protected]> Co-authored-by: JRD971000 <[email protected]> Co-authored-by: arendu <[email protected]>

Ali Taghibakhshi and others added 30 commits July 1, 2024 07:39

adding mamba support

8a26848

fix import mixins

73d7c4c

rm convert jamba

66886b5

Apply isort and black reformatting

f9e2066

Signed-off-by: JRD971000 <[email protected]>

more cleanups

96ab05c

Merge branch 'alit/mamba' of https://github.com/NVIDIA/NeMo into alit…

f24cd69

…/mamba

use GPT text gen

2e74b64

Apply isort and black reformatting

05c377a

Signed-off-by: JRD971000 <[email protected]>

fixing gbs in TP convetor

59f176a

resolve merge conflicts

74a30de

Apply isort and black reformatting

dfc24e2

Signed-off-by: JRD971000 <[email protected]>

add reqs

7edd5cc

Merge branch 'alit/mamba' of https://github.com/NVIDIA/NeMo into alit…

3eee1c7

…/mamba

add tutorial

c0afdc4

minor fix to tutorial

6097379

moving finetuning files

8e7aea0

Signed-off-by: arendu <[email protected]>

moving finetuning files

1db8269

Signed-off-by: arendu <[email protected]>

address comments

0f326d6

Apply isort and black reformatting

da7461a

Signed-off-by: JRD971000 <[email protected]>

address comments

022622e

Apply isort and black reformatting

7b67568

Signed-off-by: JRD971000 <[email protected]>

Merge branch 'main' into alit/mamba

7dce2bf

address comments

0d5cc37

add mamba dependancies

2cf9040

Merge branch 'main' into alit/mamba

0353eb9

add mcore tag

23a2d20

merge main

a9a24b7

Merge branch 'alit/mamba' of https://github.com/NVIDIA/NeMo into alit…

53792ab

…/mamba

Merge branch 'main' into alit/mamba

2b97d0b

modify dockerfile ci

0747052

JRD971000 added the Run CICD label Jul 11, 2024

github-advanced-security bot found potential problems Jul 11, 2024

View reviewed changes

minor fix

1294e3a

JRD971000 added Run CICD and removed Run CICD labels Jul 11, 2024

Apply isort and black reformatting

260bdfb

Signed-off-by: JRD971000 <[email protected]>

github-advanced-security bot found potential problems Jul 11, 2024

View reviewed changes

examples/nlp/language_modeling/mamba_change_num_partition.py Dismissed Show dismissed Hide dismissed

examples/nlp/language_modeling/mamba_change_num_partition.py Dismissed Show dismissed Hide dismissed

Ali Taghibakhshi and others added 4 commits July 11, 2024 09:57

minor fix

a1bb04f

resolve conflict:

73d02a4

Merge branch 'main' into alit/mamba

841b1e1

Apply isort and black reformatting

71cfb8b

Signed-off-by: JRD971000 <[email protected]>

JRD971000 added Run CICD and removed Run CICD labels Jul 11, 2024

github-advanced-security bot found potential problems Jul 11, 2024

View reviewed changes

yaoyu-33 previously approved these changes Jul 11, 2024

View reviewed changes

Ali Taghibakhshi added 2 commits July 11, 2024 10:34

bug fix, tutorial update

a583dad

Merge branch 'alit/mamba' of https://github.com/NVIDIA/NeMo into alit…

41f6229

…/mamba

JRD971000 dismissed yaoyu-33’s stale review via 41f6229 July 11, 2024 17:35

JRD971000 added Run CICD and removed Run CICD labels Jul 11, 2024

yaoyu-33 approved these changes Jul 11, 2024

View reviewed changes

JRD971000 merged commit 3482dc1 into main Jul 11, 2024
205 checks passed

JRD971000 deleted the alit/mamba branch July 11, 2024 19:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Alit/mamba #9696

Alit/mamba #9696

JRD971000 commented Jul 11, 2024

JRD971000 Jul 11, 2024


		# print("Combined tp model: {}".format(combined_tp_model.keys()))

		for ii, (key, original_tensor) in enumerate(combined_tp_model.items()):

Alit/mamba #9696

Alit/mamba #9696

Conversation

JRD971000 commented Jul 11, 2024

What does this PR do ?

Changelog

Usage

GitHub Actions CI

Before your PR is "Ready for review"

Who can review?

Additional Information

JRD971000 Jul 11, 2024

Choose a reason for hiding this comment