libtorch location

[rhobro]

Hi there,

In the documentation, it says that libtorch must be on the system for the crate to work.

I am creating a program. Does this mean that libtorch must be on every system that I run the program on or will it be packaged with the binary?

Thanks.

[LaurentMazare]

Right, libtorch is loaded as a shared library so the library binaries should be deployed on the boxes where you want to run this and in a path so that they could be discovered (i.e. either a system path or you could adjust the LD_LIBRARY_PATH environment variable).

[danieldk]

If you want to distribute libtorch with your binary, you could also put the libtorch libraries in the same directory and use patchelf to patch the libraries and your binary to load libraries from the binary's directory. See e.g.:

https://github.com/tensordot/syntaxdot/blob/519b96491b08c847d91739faff95ea344ed4e870/.github/workflows/release.yml#L39

[rhobro]

@danieldk so patchelf would add all of the libtorch libraries into the binary? However, in that example, they seem to be .so files but when I downloaded libtorch, it seemed to be in .dylib files?

[danieldk]

@danieldk so patchelf would add all of the libtorch libraries into the binary?

No, the libraries are dynamically loaded. If you want to link everything into a single binary, you need a static build of libtorch (I am not sure if libtorch static builds are currently functional, the stuff included in the libtorch archive is incomplete). You probably need to patch tch-rs as well to work with static libtorch libraries.

patchelf makes it possible to hardcode the path of the dynamic libraries in the binary (e.g. in the same directory as the binary), so that you don't need LD_LIBRARY_PATH or a system-wide install.

However, in that example, they seem to be .so files but when I downloaded libtorch, it seemed to be in .dylib files?

Right, this is for Linux, which uses .so as the shared library extension. .dylib files are macOS shared libraries, but since macOS uses Mach-O format, you cannot use patchelf. For macOS binaries, you can use install_name_tool to similar effect.

Another option on macOS is to add the libtorch library path to the DYLD_LIBRARY_PATH variable on macOS.

[rhobro]

@danieldk thank you. This is proving to be a harder task.

When programs use the c++ API of libtorch, is it packaged into the binary or is it loaded in a similar fashion?

Would it be possible to take libtorch's code and interact with it using swig or something so that it's in the same binary?

[danieldk]

When programs use the c++ API of libtorch, is it packaged into the binary or is it loaded in a similar fashion?

As long as you build libtorch as a dynamic library, it doesn't matter if you use libtorch directly through the C++ API or tch-rs (which also uses the C++ API). You need to provide the Torch libraries in addition to you program. It's just a property of dynamic libraries and has very little to do with Torch itself. As I said in the earlier comment, you could try to do a static build of libtorch, which would allow you to link libtorch into a binary.

If distributing libtorch is too heavy, you should look at many of the other deployment options, like ONNX.

Though this is probably getting a bit off-topic...

[rhobro]

Ok. I'll try a static build.

Haha yes. I thought about onnx but couldn't find any way to train models with it in rust. Hence why I'm thinking about tch-rs.

[mrvollger]

I have a rust binary that uses tch and I'd also like to be able to statically link libtorch, and I am trying to follow this thread so I can get started.

From what I understand I would start by getting the static libtorch: e.g.:

 wget https://download.pytorch.org/libtorch/cu116/libtorch-cxx11-abi-static-with-deps-1.12.0%2Bcu116.zip

And then the build.rs script won't work because it looks for .so files, so I would need to create a patch that allows it to search for libtorch.a in addition to libtorch.so?

Thanks in advance!

[LaurentMazare]

@mrvollger Static linking should be supported in the latest version, some details in the readme.