Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add pile calib, rename quant_block_list to to_quant_block_names #322

Merged
merged 9 commits into from
Nov 15, 2024

Conversation

WeiweiZhang1
Copy link
Collaborator

No description provided.

@wenhuach21 wenhuach21 self-requested a review November 14, 2024 13:07
Copy link
Contributor

@wenhuach21 wenhuach21 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't carefully read the code, if possible, please test more scenarios

@wenhuach21 wenhuach21 self-requested a review November 14, 2024 13:08
@wenhuach21
Copy link
Contributor

could fp layer also support fuzzy matching

@wenhuach21
Copy link
Contributor

fp_layers = args.fp_layers.split(",")
if bool(fp_layers):
    for n, m in model.named_modules():
        if isinstance(m, torch.nn.Linear) or isinstance(m, transformers.modeling_utils.Conv1D):
            name = n.split('.')[-1]
            if n in fp_layers or name in fp_layers:
                layer_config[n] = {"bits": 16}
                logger.info(
                    f"{n} will not be quantized.")

why coding like this , name = n.split('.')[-1]? how to exclude a exact layer

@wenhuach21 wenhuach21 merged commit d493572 into main Nov 15, 2024
12 of 14 checks passed
@wenhuach21 wenhuach21 deleted the rename_quant_block_list branch November 15, 2024 01:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants