Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: instruction_template not applying from user_settings #4957

Closed
wants to merge 1 commit into from

Conversation

hronoas
Copy link
Contributor

@hronoas hronoas commented Dec 17, 2023

Checklist:

@oobabooga
Copy link
Owner

Instruction templates are not set by name anymore. They are set by value in settings.yaml

The idea is to have the templates in the model metadata, and in models/config.yaml otherwise.

@FartyPants
Copy link
Contributor

FartyPants commented Dec 17, 2023

We could add button that would write the jinja script to the metadata if it is not already there. Once people start including the template in the metadata, all the problems would be solved.
Or I could prob. make extension that does it.

@hronoas
Copy link
Contributor Author

hronoas commented Dec 18, 2023

How to use and save custom template for gguf in this case?
Settings.yaml - used globally, I would like to use an individual custom template for each model.
There are models in which a start prompt is prescribed. I prefer not to change the template manually every time.

@oobabooga
Copy link
Owner

If the template is included in the GGUF metadata, it should be automatically set after #4975

Otherwise, the best option is to add a regex pattern to models/config.yaml or models/config-user.yaml (look at models/config.yaml for examples).

@hronoas
Copy link
Contributor Author

hronoas commented Dec 18, 2023

Otherwise, the best option is to add a regex pattern to models/config.yaml or models/config-user.yaml (look at models/config.yaml for examples).

The fact of the matter is that when you specify instruction template in models/config-user.yaml, the template is not loaded. This commit is precisely about the fact that config-user.yaml is loaded later than instruction_template_str is created

@hronoas
Copy link
Contributor Author

hronoas commented Dec 18, 2023

изображение

@oobabooga oobabooga closed this Dec 18, 2023
@oobabooga
Copy link
Owner

instruction template is model metadata, not an user setting.

@hronoas hronoas deleted the patch-2 branch April 21, 2024 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants