Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate Text Generation Metrics to valor-lite #802

Merged
merged 60 commits into from
Oct 31, 2024
Merged

Migrate Text Generation Metrics to valor-lite #802

merged 60 commits into from
Oct 31, 2024

Conversation

bnativi
Copy link
Contributor

@bnativi bnativi commented Oct 16, 2024 •
edited
Loading

Move the text generation core code to lite. Adapted the code to follow a similar organization to the other task types. Changes include:

  • Renaming files and moving code around (reorganized to be more similar to the other task types)
  • Remove metadata.
  • Update tests. Improve code coverage.
  • Remove evaluation objects. Now the main text generation evaluation function returns just a list of dictionaries, with one dictionary per computed metric.

b.nativi and others added 29 commits October 16, 2024 20:34
move text gen files from core to lite
1695b5e
remove non text stuff from text gen in lite
0c2bd14
add coverage for text gen in lite, add lite dependencies (except pand…
3b7f6de 
…as) to pyproject.toml
add pandas to lite pyproject
a7ba917
move lite text gen enums into metric
8e38375
remove exceptions from lite.text_generation
f53071a
remove utilities.py from valor_lite.text_generation
6ace451
rename managers.py to manager.py in lite.text_generation
b88e1b8
file renames in lite.text_generation
3712de1
separate lite text gen schemas into annotation and evaluation
cec71b5
remove metadata from valor_lite.text_generation
db9c0b0
file renames
b818e09
Merge branch 'main' into core_to_lite_text_gen
9b2b57c
reintroduce utilities.py and exceptions.py for lite text gen
7ec0570
coverage for lite text gen metric.py
09c1eeb
update to lite workflow for text gen
05d79b2
reorganize text gen metrics
cb1638c
validation for rouge params in lite
9bf3ebb
remove evaluation objections from lite.text_generation, improved vali…
5cac0e7 
…dation for evaluation parameters
code coverage for utilities
b941e4f
code coverage for valor_lite.text_generation
2714650
fix for LLM client mocks in tests
81be4c6
improve code coverage
9e2035c
@czaloom
wip - converting to lite formatting
c473248
@czaloom
wip
6876844
@czaloom
wip - adding loose ends
c05e9b7
@czaloom
refactoring tests
6fb6ec6
@czaloom
reorganizing so testing can be more granular
99320bb
@czaloom
renamed functions
8c1b407
@czaloom czaloom self-assigned this Oct 28, 2024
@czaloom czaloom changed the title move text gen files from core to lite Migrate Text Generation Metrics to valor-lite Oct 29, 2024
@czaloom czaloom marked this pull request as ready for review October 31, 2024 17:53
@czaloom czaloom merged commit ce8d261 into main Oct 31, 2024
13 checks passed
@czaloom czaloom deleted the core_to_lite_text_gen branch October 31, 2024 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ekorman ekorman ekorman approved these changes

@czaloom czaloom Awaiting requested review from czaloom czaloom is a code owner

@jyono jyono Awaiting requested review from jyono jyono is a code owner

@rsbowman-striveworks rsbowman-striveworks Awaiting requested review from rsbowman-striveworks rsbowman-striveworks is a code owner

Assignees

@czaloom czaloom

Labels
feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@bnativi @ekorman @czaloom