Code

Mattj/requirements#212

Code

Open

revbucket wants to merge 5 commits intomainallenai/dolma:mainfrom mattj/requirementsallenai/dolma:mattj/requirementsCopy head branch name to clipboard

+1,161-702

Commits

Commits on Sep 19, 2024

• Untested commit for many DCLM-dedup, plus many more changes

  

  Matt Jordan

  committed
  cb57a09

Commits on Sep 20, 2024

• Fixed some CLI stuff -- paragraph seems to work, but still need to test dclm

  

  Matt Jordan

  committed
  69839ed

Commits on Sep 25, 2024

• Pushing local changes to s3 to verify against bff --hopefully working

  

  Matt Jordan

  committed
  e0992c4

Commits on Sep 26, 2024

• bugfixy for some edge cases

  

  Matt Jordan

  committed
  bc28fc9

• Added requirements from uv -> pip install dolma

  

  Matt Jordan

  committed
  8a778de