Code

feat(js): Vitest/Jest evals#1343

Code

Merged

jacoblee93 merged 70 commits intomainlangchain-ai/langsmith-sdk:mainfrom jacob/unitlangchain-ai/langsmith-sdk:jacob/unitCopy head branch name to clipboardJan 20, 2025

+4,018-19

Commits

Commits on Dec 18, 2024

• Initial unit test form factor exploration

  jacoblee93

  committed
  24fda8a

  Copy full SHA for 24fda8a

Commits on Dec 20, 2024

• Merge

  jacoblee93

  committed
  5dd083a

  Copy full SHA for 5dd083a

• Refactor and add matchers

  jacoblee93

  committed
  8ab9e15

  Copy full SHA for 8ab9e15

• Fix bug

  jacoblee93

  committed
  39b5ed8

  Copy full SHA for 39b5ed8

• Improve error message

  jacoblee93

  committed
  0d424e2

  Copy full SHA for 0d424e2

• Fix ALS issues

  jacoblee93

  committed
  5903486

  Copy full SHA for 5903486

• Fix lint

  jacoblee93

  committed
  b5c54af

  Copy full SHA for b5c54af

• Fix ALS issue

  jacoblee93

  committed
  c644b4d

  Copy full SHA for c644b4d

• Final fix for async local storage issue (hopefully)

  jacoblee93

  committed
  70e94f5

  Copy full SHA for 70e94f5

Commits on Dec 21, 2024

• Adds support for .each

  jacoblee93

  committed
  8a7947c

  Copy full SHA for 8a7947c

Commits on Dec 22, 2024

• Avoid default export

  jacoblee93

  committed
  e702a07

  Copy full SHA for e702a07

• Fix for Bun

  jacoblee93

  committed
  96b0c55

  Copy full SHA for 96b0c55

• Lint

  jacoblee93

  committed
  8ec7fab

  Copy full SHA for 8ec7fab

• Use passed expect

  jacoblee93

  committed
  0a91635

  Copy full SHA for 0a91635

Commits on Dec 28, 2024

• Merge branch 'main' of github.com:langchain-ai/langsmith-sdk into jacob/unit

  jacoblee93

  committed
  3fbfdd5

  Copy full SHA for 3fbfdd5

• Add n parameter, fix deduping

  jacoblee93

  committed
  4a62565

  Copy full SHA for 4a62565

Commits on Dec 31, 2024

• Allow more config for .each

  jacoblee93

  committed
  06a7d44

  Copy full SHA for 06a7d44

Commits on Jan 3, 2025

• Add test for custom matchers, typing tweaks

  jacoblee93

  committed
  bbc628f

  Copy full SHA for bbc628f

• Update signature, typing, naming

  jacoblee93

  committed
  05f17ae

  Copy full SHA for 05f17ae

• Allow passing experiment creation information into describe

  jacoblee93

  committed
  385910f

  Copy full SHA for 385910f

• Add configurable test tracking

  jacoblee93

  committed
  407470b

  Copy full SHA for 407470b

Commits on Jan 4, 2025

• Allow running test over a LangSmith datset

  jacoblee93

  committed
  cb7cb1a

  Copy full SHA for cb7cb1a

Commits on Jan 6, 2025

• Fix for bun

  jacoblee93

  committed
  6b52759

  Copy full SHA for 6b52759

• Speed up latency, make tracking opt-out instead of opt-in

  jacoblee93

  committed
  23c7466

  Copy full SHA for 23c7466

Commits on Jan 7, 2025

• Rename outputs to expected, refactor feedback logging, nicer errors

  jacoblee93

  committed
  364ffad

  Copy full SHA for 364ffad

• Fix lint

  jacoblee93

  committed
  6abd444

  Copy full SHA for 6abd444

Commits on Jan 8, 2025

• Add methods to manually log feedback/outputs

  jacoblee93

  committed
  f14b31c

  Copy full SHA for f14b31c

• Fix eval run attribution

  jacoblee93

  committed
  775cfbe

  Copy full SHA for 775cfbe

• Fix local mode

  jacoblee93

  committed
  d620051

  Copy full SHA for d620051

• Refactor, add Vitest support

  jacoblee93

  committed
  31420aa

  Copy full SHA for 31420aa

• Fix lint

  jacoblee93

  committed
  c74573e

  Copy full SHA for c74573e

• Fix type

  jacoblee93

  committed
  039e7de

  Copy full SHA for 039e7de

• Merge

  jacoblee93

  committed
  314da52

  Copy full SHA for 314da52

• Remove unused dep

  jacoblee93

  committed
  8ae1c80

  Copy full SHA for 8ae1c80

Commits on Jan 13, 2025

• Adds Jest reporter

  jacoblee93

  committed
  a77d3d5

  Copy full SHA for a77d3d5

• Merge branch 'main' of github.com:langchain-ai/langsmith-sdk into jacob/unit

  jacoblee93

  committed
  2d93811

  Copy full SHA for 2d93811

• Revert

  jacoblee93

  committed
  a810c8a

  Copy full SHA for a810c8a

• Adds Vitest reporter, refactor

  jacoblee93

  committed
  6bb23d5

  Copy full SHA for 6bb23d5

• Fix

  jacoblee93

  committed
  574dcbb

  Copy full SHA for 574dcbb

• Allow export of default entrypoints

  jacoblee93

  committed
  e1824bb

  Copy full SHA for e1824bb

• Add experiment URL, polish

  jacoblee93

  committed
  54d3825

  Copy full SHA for 54d3825

• Test name

  jacoblee93

  committed
  73712fa

  Copy full SHA for 73712fa

• Update test

  jacoblee93

  committed
  6920a90

  Copy full SHA for 6920a90

• Fix display when test fails

  jacoblee93

  committed
  0937427

  Copy full SHA for 0937427

• Naming fixes

  jacoblee93

  committed
  42a87dd

  Copy full SHA for 42a87dd

• Add feedback column collapse

  jacoblee93

  committed
  47d0b7e

  Copy full SHA for 47d0b7e

Commits on Jan 15, 2025

• Merge branch 'main' of github.com:langchain-ai/langsmith-sdk into jacob/unit

  jacoblee93

  committed
  d07f42e

  Copy full SHA for d07f42e

• Refactor for simplicity, fix experiment creation

  jacoblee93

  committed
  644aca5

  Copy full SHA for 644aca5

• Fix typo

  jacoblee93

  committed
  2a45e00

  Copy full SHA for 2a45e00

• Adds docstrings

  jacoblee93

  committed
  539e5f2

  Copy full SHA for 539e5f2

• Add docs link in thrown errors

  jacoblee93

  committed
  49e1580

  Copy full SHA for 49e1580

• Merge branch 'main' of github.com:langchain-ai/langsmith-sdk into jacob/unit

  jacoblee93

  committed
  c0323ab

  Copy full SHA for c0323ab

• Fix typo

  jacoblee93

  committed
  2485055

  Copy full SHA for 2485055

• Merge

  jacoblee93

  committed
  c781c9e

  Copy full SHA for c781c9e

Commits on Jan 16, 2025

• Merge branch 'main' of github.com:langchain-ai/langsmith-sdk into jacob/unit

  jacoblee93

  committed
  1056a67

  Copy full SHA for 1056a67

• Adds tracking

  jacoblee93

  committed
  c3c6b6d

  Copy full SHA for c3c6b6d

• Allow other parameterization of test inputs

  jacoblee93

  committed
  b41d129

  Copy full SHA for b41d129

Commits on Jan 17, 2025

• Rename logOutput to logOutputs

  jacoblee93

  committed
  f6a1c28

  Copy full SHA for f6a1c28

• Fix concurrency issue

  jacoblee93

  committed
  b615035

  Copy full SHA for b615035

• Rename expected -> referenceOutputs and actual -> outputs

  jacoblee93

  committed
  ff412f6

  Copy full SHA for ff412f6

• Docstring

  jacoblee93

  committed
  922d20c

  Copy full SHA for 922d20c

• Fix lint

  jacoblee93

  committed
  503f128

  Copy full SHA for 503f128

Commits on Jan 18, 2025

• Bump default test timeout

  jacoblee93

  committed
  89313dd

  Copy full SHA for 89313dd

• Add git commit tagging

  jacoblee93

  committed
  d12e1ce

  Copy full SHA for d12e1ce

• Fix tests

  jacoblee93

  committed
  d22f759

  Copy full SHA for d22f759

• Update

  jacoblee93

  committed
  d02dc31

  Copy full SHA for d02dc31

Commits on Jan 20, 2025

• Remove log

  jacoblee93

  committed
  bd8ecd5

  Copy full SHA for bd8ecd5

• Adds CI

  jacoblee93

  committed
  7a982bd

  Copy full SHA for 7a982bd

• Bump version

  jacoblee93

  committed
  0046863

  Copy full SHA for 0046863

• Fix CI

  jacoblee93

  committed
  98feca9

  Copy full SHA for 98feca9