Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GMC: response timeout with some specific models #338

Closed
KfreeZ opened this issue Aug 22, 2024 · 2 comments
Closed

GMC: response timeout with some specific models #338

KfreeZ opened this issue Aug 22, 2024 · 2 comments
Labels
duplicate This issue or pull request already exists gmc

Comments

@KfreeZ
Copy link
Collaborator

KfreeZ commented Aug 22, 2024

In GMC e2e tests, there are some failed case caused by response timeout,

tgi2.2.0 + meta-llama/CodeLlama-7b-hf on xeon in codegen test

tgi2.2.0 + meta-llama/CodeLlama-7b-hf on guadi is ok

switch to another model "HuggingFaceH4/mistral-7b-grok" also ok

due to the manifest test (without GMC) cannot detect this issue, we need to find out whether it's GMC's specific.

@lianhao
Copy link
Collaborator

lianhao commented Aug 22, 2024

on gaudi, we use tgi-gaudi 2.0.1, not tgi2.2.0

@lianhao lianhao added the gmc label Aug 22, 2024
@KfreeZ
Copy link
Collaborator Author

KfreeZ commented Aug 27, 2024

similar issue to #258, close this one as duplication

@KfreeZ KfreeZ added the duplicate This issue or pull request already exists label Aug 27, 2024
@KfreeZ KfreeZ closed this as completed Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists gmc
Projects
None yet
Development

No branches or pull requests

2 participants