Support for microsoft/Florence-2-large-ft vision model

[AshD]

microsoft/Florence-2-large-ft works really well and it is much smaller than Phi-3 vision.

Would it be possible to convert this model in Onnx format?

Thanks,
Ash

[tarzain]

+1! Would love to have this in ONNX

[balachandarsv]

+1 It would be great to have this in ONNX.

[aciddelgado]

Thanks all for the suggestion; we'll add this model to our roadmap and we'll discuss the priority internally. Thanks!

[AshD]

Xenova created an Onnx version of the Florence model
https://huggingface.co/onnx-community/Florence-2-base-ft/tree/main/onnx

[nogifeet]

Is it possible to get access to scripts/branch used to convert to onnx ?

I want to convert some of my fine-tuned checkpoints to onnx

[Youho99]

I believe you can create your own onnx now

https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/awq-quantized-model.md

[nogifeet]

@Youho99 Looks like florence2 is still not supported using AWQ.

Traceback (most recent call last):
File "/content/awq-quantized-model.py", line 140, in
main()
File "/content/awq-quantized-model.py", line 117, in main
quantize_model(args)
File "/content/awq-quantized-model.py", line 53, in quantize_model
model = AutoAWQForCausalLM.from_pretrained(
File "/content/AutoAWQ/awq/models/auto.py", line 70, in from_pretrained
model_type = check_and_get_model_type(
File "/content/AutoAWQ/awq/models/auto.py", line 48, in check_and_get_model_type
raise TypeError(f"{config.model_type} isn't supported yet.")
TypeError: florence2 isn't supported yet.

[timothycarambat]

@AshD Been looking for that, thank you!

[trincadev]

Xenova created an Onnx version of the Florence model https://huggingface.co/onnx-community/Florence-2-base-ft/tree/main/onnx

Thank you. Any news for support on python? Xenova's onnx florence-2 model requires transformers.js

[natke]

Thank you for your interest in Florence-2 V. We are looking into it. Moving this issue into Discussions

[Youho99]

+1

[callmehanyu]

+1

[qingfengcss]

Could the author write a document about how to convert florence model to onnx format?

[Youho99]

Add same support request for new phi-3.5-vision
Discussion here : #827

[happyme531]

I made an onnxruntime inference script for Florence2:
https://huggingface.co/happyme531/Florence-2-base-ft-ONNX-RKNN2

[AshD]

Do you have a C# version of this script?