v0.15.0

New Features

Open Source Embedding + Contrastive Code (#1615)

LLM foundry now supports finetuning embedding models with contrastive loss. Foundry now supports various approaches to selecting negative passages for contrastive loss which can be either randomly selected or pre-defined. For more information, please view the the readme.

PyTorch 2.5.1 (#1665)

This release updates LLM Foundry to the PyTorch 2.5.1 release, bringing with it support for the new features and optimizations in PyTorch 2.5.1.

Improved error messages (#1657, #1660, #1623, #1625)

Various improved error messages, making debugging user errors more clear.

What's Changed

• Update mcli examples to use 0.14.0 by @irenedea in #1624
• Open Source Embedding + Contrastive Code by @KuuCi in #1615
• Catch delta table not found error by @milocress in #1625
• Add Mlflow 403 PL UserError by @mattyding in #1623
• Catches when data prep cluster fails to start by @milocress in #1628
• Bump mlflow max version by @dakinggg in #1629
• add another cluster connection failure wrapper by @milocress in #1630
• Add MLflow log_model option by @nancyhung in #1544
• Move loss generating token counting to the dataloader by @dakinggg in #1632
• Bump databricks-connect from 14.1.0 to 15.4.3 by @dependabot in #1636
• Fix dataset download location by @dakinggg in #1639
• Revert "Bump databricks-connect from 14.1.0 to 15.4.3" by @XiaohanZhangCMU in #1640
• Bump transformers version by @dakinggg in #1631
• Fix gpu tests test_tp_train and test_huggingface_conversion_callback_interval by @irenedea in #1642
• Update datasets requirement from <2.20,>=2.19 to >=2.20.0,<2.21 by @dependabot in #1330
• Add max shard size to transformers save_pretrained by @b-chu in #1648
• Update huggingface-hub requirement from <0.25,>=0.19.0 to >=0.19.0,<0.27 by @dependabot in #1652
• Update accelerate requirement from <0.34,>=0.25 to >=0.25,<1.2 by @dependabot in #1633
• Catch Delta Table Not Found by @KuuCi in #1653
• Add Exception for missing UC column by @milocress in #1654
• Infer step size for Embeddings by @KuuCi in #1647
• Pin FAv2 by @mvpatel2000 in #1656
• Retry catching BlockingIOError by @KuuCi in #1657
• Catch bad data prep by @milocress in #1644
• Update pytest-cov requirement from <6,>=4 to >=4,<7 by @dependabot in #1663
• Bump coverage[toml] from 7.6.1 to 7.6.4 by @dependabot in #1650
• Move transform_model_pre_registration in hf_checkpointer by @irenedea in #1664
• Catch Cluster Permissions Error by @KuuCi in #1660
• Mosaicml version bump by @j316chuck in #1661
• Changes for removing unused terms in CE loss fn by @gupta-abhay in #1643
• Update setuptools requirement from <68.0.0 to <76.0.0 by @dependabot in #1662
• Bump docker version to torch 2.5.1 by @j316chuck in #1665
• Bump ubuntu 22.04 + torch 2.5.1 by @KuuCi in #1666

New Contributors

• @mattyding made their first contribution in #1623

Full Changelog: v0.14.5...v0.15.0

v0.14.5

• Move transform_model_pre_registration in hf_checkpointer (#1664)

Full Changelog: v0.14.4...v0.14.5

v0.14.4

• Add max shard size to transformers save_pretrained by @b-chu in #1648

Full Changelog: v0.14.3...v0.14.4

v0.14.3

What's Changed

• Fix dataset download location by @dakinggg in #1639

Full Changelog: v0.14.2...v0.14.3

v0.14.2

Bug Fixes

Move loss generating token counting to the dataloader (#1632)

Fixes a throughput regression due to #1610, which was release in v0.14.0

What's Changed

• Move loss generating token counting to the dataloader by @dakinggg in #1632

Full Changelog: v0.14.1...v0.14.2

v0.14.1

New Features

Use log_model for registering models (#1544 )

Instead of calling the mlflow register API directly, we use the intended log_model API, which will both log the model to mlflow run artifacts, and register it to Unity Catalog.

What's Changed

• Catch delta table not found error by @milocress in #1625
• Add Mlflow 403 PL UserError @dakinggg in #1623
• Catches when data prep cluster fails to start by @milocress in #1628
• add another cluster connection failure wrapper by @milocress in #1630
• Use log_model API to register the model by @nancyhung @dakinggg in #1544

Full Changelog: v0.14.0...v0.14.1

v0.14.0

New Features

Load Checkpoint Callback (#1570)

We added support for Composer's LoadCheckpoint callback, which loads a checkpoint at a specified event. This enables use cases like loading model base weights with peft.

callbacks:
    load_checkpoint:
        load_path: /path/to/your/weights

Breaking Changes

Accumulate over tokens in a Batch for Training Loss (#1618,#1610,#1595)

We added a new flag accumulate_train_batch_on_tokens which specifies whether training loss is accumulated over the number of tokens in a batch, rather than the number of samples. It is true by default. This will slightly change loss curves for models trained with padding. The old behavior can be recovered by simply setting this to False explicitly.

Default Run Name (#1611)

If no run name is provided, we now will default to using composer's randomly generated run names. (Previously, we defaulted to using "llm" for the run name.)

What's Changed

• Update mcli examples to use 0.13.0 by @irenedea in #1594
• Pass accumulate_train_batch_on_tokens through to composer by @dakinggg in #1595
• Loosen MegaBlocks version pin by @mvpatel2000 in #1597
• Add configurability for hf checkpointer register timeout by @dakinggg in #1599
• Loosen MegaBlocks to <1.0 by @mvpatel2000 in #1598
• Finetuning dataloader validation tweaks by @mvpatel2000 in #1600
• Bump onnx from 1.16.2 to 1.17.0 by @dependabot in #1604
• Remove TE from dockerfile and instead add as optional dependency by @snarayan21 in #1605
• Data prep on multiple GPUs by @eitanturok in #1576
• Add env var for configuring the maximum number of processes to use for dataset processing by @irenedea in #1606
• Updated error message for cluster check by @nancyhung in #1602
• Use fun default composer run names by @irenedea in #1611
• Ensure log messages are properly formatted again by @snarayan21 in #1614
• Add UC not enabled error for delta to json conversion by @irenedea in #1613
• Use a temporary directory for downloading finetuning dataset files by @irenedea in #1608
• Bump composer version to 0.26.0 by @irenedea in #1616
• Add loss generating token counts by @dakinggg in #1610
• Change accumulate_train_batch_on_tokens default to True by @dakinggg in #1618
• Bump version to 0.15.0.dev0 by @irenedea in #1621
• Add load checkpoint callback by @irenedea in #1570

Full Changelog: v0.13.0...v0.14.0

v0.13.1

🚀 LLM Foundry v0.13.1

What's Changed

• Add configurability to HF checkpointer timeout by @dakinggg in #1599

Full Changelog: v0.13.0...v0.13.1

v0.13.0

🚀 LLM Foundry v0.13.0

🛠️ Bug Fixes & Cleanup

Pytorch 2.4 Checkpointing (#1569, #1581, #1583)

Resolved issues related to checkpointing for Curriculum Learning (CL) callbacks.

🔧 Dependency Updates

Bumped tiktoken from 0.4.0 to 0.8.0 (#1572)
Updated onnxruntime from 1.19.0 to 1.19.2 (#1590)

What's Changed

• Update mcli yamls by @dakinggg in #1552
• Use allenai/c4 instead of c4 dataset by @eitanturok in #1554
• Tensor Parallelism by @eitanturok in #1521
• Insufficient Permissions Error when trying to access table by @KuuCi in #1555
• Add NoOp optimizer by @snarayan21 in #1560
• Deterministic GCRP Errors by @KuuCi in #1559
• Simplify CL API by @b-chu in #1510
• Reapply #1389 by @dakinggg in #1561
• Add dataset swap callback by @b-chu in #1536
• Add error to catch more unknown example types by @milocress in #1562
• Add FileExtensionNotFoundError by @b-chu in #1564
• Add InvalidConversationError by @b-chu in #1565
• Release docker img by @KuuCi in #1547
• Revert FT dataloader changes from #1561, keep #1564 by @snarayan21 in #1566
• Cleanup TP by @eitanturok in #1556
• Changes for dataset swap callback by @gupta-abhay in #1569
• Do not consider run_name when auto-detecting autoresume by @irenedea in #1571
• Allow parameters with requires_grad=False in meta init by @sashaDoubov in #1567
• Bump tiktoken from 0.4.0 to 0.8.0 by @dependabot in #1572
• Add extensions to FinetuningFileNotFoundError by @b-chu in #1578
• Handle long file names in convert text to mds by @irenedea in #1579
• Set streaming log level by @mvpatel2000 in #1582
• Fix pytorch checkpointing for CL callback by @b-chu in #1581
• Fix pytorch checkpointing for CL callback by @b-chu in #1583
• Error if filtered dataset contains 0 examples by @irenedea in #1585
• Change cluster errors from NetworkError to UserError by @irenedea in #1586
• Do not autoresume if a default name is set, only on user defined ones by @irenedea in #1588
• Bump onnxruntime from 1.19.0 to 1.19.2 by @dependabot in #1590
• Make FinetuningStreamingDataset parameters more flexible by @XiaohanZhangCMU in #1580
• Add build callback tests by @irenedea in #1577
• Bump version to 0.14.0.dev0 by @irenedea in #1587
• Fix typo in eval code by using 'fsdp' instead of 'fsdp_config' by @irenedea in #1593

Full Changelog: v0.12.0...v0.13.0

v0.12.0

🚀 LLM Foundry v0.12.0

New Features

PyTorch 2.4 (#1505)

This release updates LLM Foundry to the PyTorch 2.4 release, bringing with it support for the new features and optimizations in PyTorch 2.4

Extensibility improvements (#1450, #1449, #1468, #1467, #1478, #1493, #1495, #1511, #1512, #1527)

Numerous improvements to the extensibility of the modeling and data loading code, enabling easier reuse for subclassing and extending. Please see the linked PRs for more details on each change.

Improved error messages (#1457, #1459, #1519, #1518, #1522, #1534, #1548, #1551)

Various improved error messages, making debugging user errors more clear.

Sliding window in torch attention (#1455)

We've added support for sliding window attention to the reference attention implementation, allowing easier testing and comparison against more optimized attention variants.

Bug fixes

Extra BOS token for llama 3.1 with completion data (#1476)

A bug resulted in an extra BOS token being added between prompt and response during finetuning. This is fixed so that the prompt and response supplied by the user are concatenated without any extra tokens put between them.

What's Changed

• Add test for logged_config transforms by @b-chu in #1441
• Bump version to 0.12.0.dev0. by @irenedea in #1447
• Update pytest-codeblocks requirement from <0.17,>=0.16.1 to >=0.16.1,<0.18 by @dependabot in #1445
• Bump coverage[toml] from 7.4.4 to 7.6.1 by @dependabot in #1442
• Enabled generalizing build_inner_model in ComposerHFCausalLM by @gupta-abhay in #1450
• Update llm foundry version in mcli yamls by @irenedea in #1451
• merge to main by @XiaohanZhangCMU in #865
• allow embedding resizing passed through by @jdchang1 in #1449
• Update packaging requirement from <23,>=21 to >=21,<25 by @dependabot in #1444
• Update pytest requirement from <8,>=7.2.1 to >=7.2.1,<9 by @dependabot in #1443
• Implement ruff rules enforcing PEP 585 by @snarayan21 in #1453
• Adding sliding window attn to scaled_multihead_dot_product_attention by @ShashankMosaicML in #1455
• Add user error for UnicodeDeocdeError in convert text to mds by @irenedea in #1457
• Fix log_config by @josejg in #1432
• Add EnvironmentLogger Callback by @josejg in #1350
• Update mosaicml/ci-testing to 0.1.2 by @irenedea in #1458
• Correct error message for inference wrapper by @josejg in #1459
• Update CI tests to v0.1.2 by @KuuCi in #1466
• Bump onnxruntime from 1.18.1 to 1.19.0 by @dependabot in #1461
• Update tenacity requirement from <9,>=8.2.3 to >=8.2.3,<10 by @dependabot in #1460
• Simple change to enable mapping functions for ft constructor by @gupta-abhay in #1468
• use default eval interval from composer by @milocress in #1369
• Consistent Naming EnviromentLoggingCallback by @josejg in #1470
• Register NaN Monitor Callback by @josejg in #1471
• Add train subset num batches by @mvpatel2000 in #1472
• Parent class hf models by @jdchang1 in #1467
• Remove extra bos for prompt/response data with llama3.1 by @dakinggg in #1476
• Add prepare fsdp back by @dakinggg in #1477
• Add date_string when applying tokenizer chat template by @snarayan21 in #1474
• Make sample tokenization extensible by @gupta-abhay in #1478
• Use Streaming version 0.8.1 by @snarayan21 in #1479
• Bump hf-transfer from 0.1.3 to 0.1.8 by @dependabot in #1480
• fix hf checkpointer by @milocress in #1489
• Fix device mismatch when running hf.generate by @ShashankMosaicML in #1486
• Bump composer to 0.24.1 + FSDP config device_mesh deprecation by @snarayan21 in #1487
• master_weights_dtype not supported by ComposerHFCausalLM.init() by @eldarkurtic in #1485
• Detect loss spikes and high losses during training by @joyce-chen-uni in #1473
• Enable passing in external position ids by @gupta-abhay in #1493
• Align logged attributes for errors and run metadata in kill_loss_spike_callback.py by @joyce-chen-uni in #1494
• tokenizer is never built when converting finetuning dataset by @eldarkurtic in #1496
• Removing error message for reusing kv cache with torch attn by @ShashankMosaicML in #1497
• Fix formatting of loss spike & high loss error messages by @joyce-chen-uni in #1498
• Enable cross attention layers by @gupta-abhay in #1495
• Update to ci-testing 0.2.0 by @dakinggg in #1500
• [WIP] Torch 2.4 in docker images by @snarayan21 in #1491
• [WIP] Only torch 2.4.0 compatible by @snarayan21 in #1505
• Update mlflow requirement from <2.16,>=2.14.1 to >=2.14.1,<2.17 by @dependabot in #1506
• Update ci-testing to 0.2.2 by @dakinggg in #1503
• Allow passing key_value_statest for x-attn through MPT Block by @gupta-abhay in #1511
• Fix cross attention for blocks by @gupta-abhay in #1512
• Put 2.3 image back in release examples by @dakinggg in #1513
• Sort callbacks so that CheckpointSaver goes before HuggingFaceCheckpointer by @irenedea in #1515
• Raise MisconfiguredDatasetError from original error by @irenedea in #1519
• Peft fsdp by @dakinggg in #1520
• Raise DatasetTooSmall exception if canonical nodes is less than num samples by @irenedea in #1518
• Add permissions check for delta table reading by @irenedea in #1522
• Add HuggingFaceCheckpointer option for only registering final checkpoint by @irenedea in #1516
• Replace FSDP args by @KuuCi in #1517
• enable correct padding_idx for embedding layers by @gupta-abhay in #1527
• Revert "Replace FSDP args" by @KuuCi in #1533
• Delete unneeded inner base model in PEFT HF Checkpointer by @snarayan21 in #1532
• Add deprecation warning to fsdp_config by @KuuCi in #1530
• Fix reuse kv cache for torch attention by @ShashankMosaicML in #1539
• Error on text dataset file not found by @milocress in #1534
• Make ICL tasks not required for eval by @snarayan21 in #1540
• Bumping flash attention version to 2.6.3 and adding option for softcap in attention and lm_head logits. by @ShashankMosaicML in #1374
• Register mosaic logger by @dakinggg in #1542
• Hfcheckpointer optional generation config by @KuuCi in #1543
• Bump composer version to 0.25.0 by @dakinggg in #1546
• Bump streaming version to 0.9.0 by @dakinggg in #1550
• Bump version to 0.13.0.dev0 by @dakinggg in #1549
• Add proper user error for accessing schema by @KuuCi in #1548
• Validate Cluster Access Mode by @KuuCi in #1551

New Contributors

• @jdchang1 made their first contribution in #1449
• @joyce-chen-uni made their first contribution in #1473

Full Changelog: v0.11.0...v0.12.0