Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add chat api format with standard open ai chat format #1314

Merged
merged 29 commits into from
Nov 12, 2024
Merged

Add chat api format with standard open ai chat format #1314

merged 29 commits into from
Nov 12, 2024

Conversation

elronbandel
Copy link
Member

No description provided.

@elronbandel
Upload artifact
e15dc6c 
Signed-off-by: elronbandel <[email protected]>
OfirArviv
OfirArviv reviewed Oct 30, 2024
View reviewed changes
src/unitxt/formats.py Outdated Show resolved Hide resolved
@elronbandel
Add docstring and a small fix
71d87c3 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Fix
e613de1 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
SImplify and remove copy code
163aaaa 
Signed-off-by: elronbandel <[email protected]>
@yoavkatz
Copy link
Member

yoavkatz commented Oct 30, 2024 •
edited
Loading

@elronbandel - I think we should make this self contained

  1. Add a HF inference engine (that uses the chat API)
  2. Add en example code that calls this API both as text and with image.

@OfirArviv
Copy link
Collaborator

OfirArviv commented Oct 30, 2024 •
edited
Loading

  • Add a HF inference engine (that uses the chat API)
  • Add en example code that calls this API both as text and with image.

We should wait for Pawel pr for this. The changes to the inference engine are minimal. No need to duplicate it. Right now no one is waiting for this. Im the one who asked for it, so no pressure I think

@elronbandel
Merge remote-tracking branch 'origin/main' into add-chat-api-format
c357364 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Add lite llm
fef3cee 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Merge branch 'main' into add-chat-api-format
8a0d9ea
@elronbandel
Evaluate BlueBench with LiteLLM
817b125 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Fix token
fe6c3cf 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Update requirements for inference tests
844e8dc 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Update
c94b800 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Update
817bab3 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Update
d2a144a 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Update
3e1a20a 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Update
9912de1 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Fix
c768822 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Fix prepration
7a6404a 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Bring vision back to order
ac7c8ae 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Fix cards
3d0a871 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Fix consistency
32f0e6e 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Fix example tests
fbd5913 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Some more fixes
8441fdd 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Update
1adcc46 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Fix example testing
944f885 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Organize examples tests
b4a0d9f 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Update
5f72658 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Update examples
473803e 
Signed-off-by: elronbandel <[email protected]>
@elronbandel
Update Inference Engines
f6112ac 
Signed-off-by: elronbandel <[email protected]>
@elronbandel elronbandel merged commit 2d46594 into main Nov 12, 2024
11 checks passed
@elronbandel elronbandel deleted the add-chat-api-format branch November 12, 2024 08:42
perlitz pushed a commit that referenced this pull request Nov 12, 2024
@elronbandel @perlitz
Add chat api format with standard open ai chat format (#1314)
4d1b823 
* Add chat api format with standard open ai chat format

Signed-off-by: elronbandel <[email protected]>

* Upload artifact

Signed-off-by: elronbandel <[email protected]>

* Add docstring and a small fix

Signed-off-by: elronbandel <[email protected]>

* Fix

Signed-off-by: elronbandel <[email protected]>

* SImplify and remove copy code

Signed-off-by: elronbandel <[email protected]>

* Add lite llm

Signed-off-by: elronbandel <[email protected]>

* Evaluate BlueBench with LiteLLM

Signed-off-by: elronbandel <[email protected]>

* Fix token

Signed-off-by: elronbandel <[email protected]>

* Update requirements for inference tests

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Fix

Signed-off-by: elronbandel <[email protected]>

* Fix prepration

Signed-off-by: elronbandel <[email protected]>

* Bring vision back to order

Signed-off-by: elronbandel <[email protected]>

* Fix cards

Signed-off-by: elronbandel <[email protected]>

* Fix consistency

Signed-off-by: elronbandel <[email protected]>

* Fix example tests

Signed-off-by: elronbandel <[email protected]>

* Some more fixes

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Fix example testing

Signed-off-by: elronbandel <[email protected]>

* Organize examples tests

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Update examples

Signed-off-by: elronbandel <[email protected]>

* Update Inference Engines

Signed-off-by: elronbandel <[email protected]>

---------

Signed-off-by: elronbandel <[email protected]>
elronbandel added a commit that referenced this pull request Nov 12, 2024
@perlitz @elronbandel
Bluebench Update (#1342)
86d8533 
* move judge to generic in arenahard

Signed-off-by: Yotam-Perlitz <[email protected]>

* remain with 1 demo in finqa

Signed-off-by: Yotam-Perlitz <[email protected]>

* fix names in subsets

Signed-off-by: Yotam-Perlitz <[email protected]>

* fix names in subsets

Signed-off-by: Yotam-Perlitz <[email protected]>

* make n_examples a sum of subsets

Signed-off-by: Yotam-Perlitz <[email protected]>

* push to catalog

Signed-off-by: Yotam-Perlitz <[email protected]>

* Add chat api format with standard open ai chat format (#1314)

* Add chat api format with standard open ai chat format

Signed-off-by: elronbandel <[email protected]>

* Upload artifact

Signed-off-by: elronbandel <[email protected]>

* Add docstring and a small fix

Signed-off-by: elronbandel <[email protected]>

* Fix

Signed-off-by: elronbandel <[email protected]>

* SImplify and remove copy code

Signed-off-by: elronbandel <[email protected]>

* Add lite llm

Signed-off-by: elronbandel <[email protected]>

* Evaluate BlueBench with LiteLLM

Signed-off-by: elronbandel <[email protected]>

* Fix token

Signed-off-by: elronbandel <[email protected]>

* Update requirements for inference tests

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Fix

Signed-off-by: elronbandel <[email protected]>

* Fix prepration

Signed-off-by: elronbandel <[email protected]>

* Bring vision back to order

Signed-off-by: elronbandel <[email protected]>

* Fix cards

Signed-off-by: elronbandel <[email protected]>

* Fix consistency

Signed-off-by: elronbandel <[email protected]>

* Fix example tests

Signed-off-by: elronbandel <[email protected]>

* Some more fixes

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Fix example testing

Signed-off-by: elronbandel <[email protected]>

* Organize examples tests

Signed-off-by: elronbandel <[email protected]>

* Update

Signed-off-by: elronbandel <[email protected]>

* Update examples

Signed-off-by: elronbandel <[email protected]>

* Update Inference Engines

Signed-off-by: elronbandel <[email protected]>

---------

Signed-off-by: elronbandel <[email protected]>

* revert generic inference engine change

Signed-off-by: Yotam-Perlitz <[email protected]>

* fix type

Signed-off-by: Yotam-Perlitz <[email protected]>

---------

Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: elronbandel <[email protected]>
Co-authored-by: Elron Bandel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@OfirArviv OfirArviv OfirArviv left review comments

@yoavkatz yoavkatz Awaiting requested review from yoavkatz

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@elronbandel @yoavkatz @OfirArviv