New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce T5 model size and enhance perfomances #6

Open

GabrielePicco opened this issue Oct 3, 2022 · 0 comments

Labels

Contributor

GabrielePicco commented Oct 3, 2022 •

edited

Loading

Scenario summary

Current inference with t5 models is slow

Proposed solution

Investigate and implement solution to reduce model size and speed-up inference, some of the ideas to consider:

Reduce T5 model size by 3X and increase the inference speed up to 5X: https://github.com/Ki6an/fastT5
How to adapt a multilingual T5 model for a single language: https://towardsdatascience.com/how-to-adapt-a-multilingual-t5-model-for-a-single-language-b9f94f3d9c90
https://huggingface.co/blog/optimum-inference

GabrielePicco added the enhancement label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment