account for the new merged/unmerged weight to perform the quantization again

[pacman100]

What does this PR do?

BNB has CI failures with 2 tests failing due to the following:
In merge method, the following things happens wherein the lora delta weights are added to the dequantized base layer weights. This is followed by setting the weight of the base layer to a new instance of Params4bit with the new merged weights:

w_data = bnb.functional.dequantize_4bit(weight.data, weight.quant_state) + lora_data
self.get_base_layer().weight = bnb.nn.Params4bit(w_data.to("cpu"), requires_grad=False,**kwargs).to(weight.device)            

Now, kwargs copies the ones currently there which has bnb_quantized=True and the old quant_state so the new merged weights aren't quantized and the old quant state remains. After this, during Linear4bit forward call, the old quant state is used while the weights are unquantized. This happens due to changes of the PR bitsandbytes-foundation/bitsandbytes#970.

This PR fixes the failures by setting bnb_quantized=False when merging/unmerging 4-bit layers.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Titus-von-Koeller]

Thanks @pacman100 for the quick reaction and fix! Really appreciated.

Seems like it's ready to merge :)

[younesbelkada]

Thanks for the quick fix!