OpenAI's CLIP & Torchvision

[TheNaila]

I am working on a python project and in order to make it faster I've stumbled upon Pythran and it seems like a worthwhile solution. However, the python code that I would like to speed up requires importing OpenAI's CLIP model as well as torchvision. I have been battling for the past day on how to get it all to work and will continue to. I did come across the documentation for adding supported modules, but I'm not familiar with how to do those things. I wanted to be able to bring this to your attention because I doubt that I am the only person who would appreciate these functionalities. I hope to hear from you soon about this!

[serge-sans-paille]

Hi and thanks for trying out pythran. Pythran can only optimize functions that uses builtin Python constructs and/or reference supported modules, which doesn't include torchvision or OpenAI CLIP.

That being said, it is possible that a particular computation-intensive function might be extracted from your project and built by pythran, then imported in your project. Hard to tell without the code :-)

[TheNaila]

Hi Serge, thank you for getting back to me! The computation for my code takes 34 mins which is why I was looking for a faster alternative. Below is the function that I would ideally be able to export, but it requires functions from the clip and torch modules. For example, pythran would give the error message something along the lines of there being no identifier for the function which is why I added the import commands but since those modules are not supported, it didn't work

a) is there a way to use ctypes to bypass this need? I was reading the documentation and got to the portion about ctypes, but was a bit confused because I've never used it before

b) might there be a way to bypass this need by using cython,ctypes in conjunction with pythran?

[TheNaila]

if it were possible to add just the torch/torchvision module as a supported module, I think that would help also actually

[Dapid]

I doubt Pythran could do anything better than Pytorch. What Pythran does is compile Python code to C++, while PyTorch's backend is already compiled, and even optimised for GPUs.

Depending on your GPU, you might be able to speed it up by processing several images in parallel.