- Added a hook for external program invocation after saving regular checkpoints
- Implemented support for SFT dataset packing with correct RoPE encoding and without cross-contamination
- Added support for a new data format: ALM
- Introduced support for DPO and DPO-Positive training methods
- Added optional sample buffering in the dataloader
- Added new utility scripts for data preparation and tokenizer replacement
- Fixed bugs in main training scripts and utility scripts
Full Changelog: v4.1.0...v5.0.0