Skip to content

Publishing Compiled Code

Jump to bottom Edit New page
Tim Caswell edited this page Jul 30, 2015 · 16 revisions

While the luvi base provides a great amount of native system APIs, sometimes it's necessary to use or create libraries written in C.

There are two basic ways to consume C libraries in the LuaJIT runtime, both involve compiling shared libraries (so/dll/dynlib) for each platform and machine architecture you want to support. Also you need to watch out for system libraries that vary between machines. In general avoid dependencies and link against an older version of libc on unixes.

Lua C API Bindings

The first way to interact with LuaJIT is through the normal Lua C API. Essentially this involves writing some C code to glue between your Lua code and C library or system you wish to access.

There are many tutorials on the web for learning this technique in general, but there are a few things particluar to this environment.

  • Use CMake for your build system. We use it for everything in luvi and our addons and it will make things easier if you use the same. It makes cross-platform building much easier.
  • Export a single C function named luaopen_foobar where foobar is the name of your library. Then name the final shared library as foobar.so or foobar.dll depending on your platform. This function needs to create your exports value on the stack and return it as a single return value.
  • The standard require module luvit/require automatically will look for these in the search path and load them with package.loadlib.

LuaJIT FFI Bindings

In addition to the normal Lua way of binding, we also have access to LuaJIT's FFI extension that's built into the jit engine itself. This is very fast and doesn't involve any C compiling beyond creating the raw library you wish to consume. If it's an already existing system library, then you don't have to build anything.

The FFI interface is much more flexible, there is nothing special about it's usage in the lit ecosystem.

Directory Structure

As of lit v0.11.4, there are two magic variables in the files filter that help with including the correct binary blobs into your application or library.

As an example, consider a library that builds for Linux, Windows, and OSX, all 64-bit that has the following directory structure:

├── init.lua
├── package.lua
├── glfw.h
├── wrapper.h
├── wrapper.c
├── README.md
├── Linux-x64
│   ├── libglfw.so.3.1
│   └── libwglfw.so
├── OSX-x64
│   ├── libglfw.3.1.dylib
│   └── libwglfw.dylib
└── Windows-x64
    ├── glfw3.dll
    └── wglfw.dll

Notice that the library names themselves vary depending on the platform. Here is the contents of our package.lua file describing which files to include when publishing and installing the library.

return {
  name = "creationix/glfw",
  version = "3.1.3",
  homepage = "https://github.com/creationix/lit-glfw",
  files = {
    "*.lua",
    "*.h",
    "$OS-$ARCH/*",
  }
}

When publishing, lit will match broadly the $OS and $ARCH variables and try to match all the possible values for ffi.os and ffi.arch in LuaJIT. This way your git repository can contain pre-built binaries for all the platforms you support in one place. When it's published to lit, it will contain all the variants.

But when someone consumes your library, lit will match them exactly as the values of ffi.os and ffi.arch so that your app only contains the versions needed for the machine it's running on. This affects the lit make and lit install commands.

Loading Bundled FFI Libraries

The luvit/require package that is used by almost all luvi/lit based runtimes has a couple really useful methods on the per-file module table.

Sometimes you'll want to bundle the dll/so/dylib with your app instead of assuming it's pre-installed on the host system. In this case, you'll know the path to the shared library relative to your script. Using module:action you can get an absolute path to this dll for ffi.load to use. Also if you're running in a luvi app as a bundled zip, this will extract the library to a temp folder, give you the path, and clean up once the function returns.

So for example:

local lib = module:action("./libs/mylib.so", ffi.load)

Also you'll usually want to include the headers (often modified or customized for luajit ffi) with your app. In this case, you just want the contents of the file as a string for ffi.cdef to use. Use the module:load method for this.

ffi.cdef(module:load("./include/someheader.h"))

A full example can be seen in the creationix/gamepad package.

local names = {
  ["Windows-x64"] = "gamepad.dll",
  ["Linux-arm"] = "libgamepad.so",
  ["Linux-x64"] = "libgamepad.so",
  ["OSX-x64"] = "libgamepad.dylib",
}

local ffi = require('ffi')
ffi.cdef(module:load("gamepad.h"))

local arch = ffi.os .. "-" .. ffi.arch
return module:action(arch .. "/" .. names[arch], ffi.load)

Bundling Lua C API Libraries

Bundling of Lua bindings is more or less the same, except you want to integrate with the require system. The easiest way to do this is you place the libraries at libs/$OS-$ARCH/*.

Suppose you had the following layout:

├── init.lua
├── libs
│   ├── Linux-x64
│   │   └── foobar.so
│   ├── OSX-x64
│   │   └── foobar.so
│   └── Windows-x64
│       └── foobar.dll
└── package.lua

Loading the foobar binding would be as simple as:

local ffi = require('ffi')
local foobar = require(ffi.os .. "-" .. ffi.arch .. "/foobar")

The package.lua for this would look something like:

return {
  name = "foobar",
  version = "0.1.2",
  files = {
    "*.lua",
    "libs/$OS-$ARCH/*"
  }
}

Example Makefiles

Automation will help in creating these binaries on various machines. I like to maintain some scripts in the form of Makefile and make.bat for unix and windows.

LUAJIT_OS=$(shell luajit -e "print(require('ffi').os)")
LUAJIT_ARCH=$(shell luajit -e "print(require('ffi').arch)")
TARGET_DIR=$(LUAJIT_OS)-$(LUAJIT_ARCH)

ifeq ($(LUAJIT_OS), OSX)
BASE_LIB=libglfw.3.1.dylib
WRAPPER_LIB=libwglfw.dylib
endif
ifeq ($(LUAJIT_OS), Linux)
BASE_LIB=libglfw.so.3.1
WRAPPER_LIB=libwglfw.so
endif

all: build
	cmake --build build --config Release
	cp build/glfw/src/$(BASE_LIB) $(TARGET_DIR)
	cp build/$(WRAPPER_LIB) $(TARGET_DIR)

build:
	cmake -Bbuild -H. -GNinja

clean:
	rm -rf build

This has a target to configure the build using cmake and another target to build the binaries and copy them to the correct place in the git tree for publishing.

On Windows I make an equally functional file:

@SET LUAJIT_OS=Windows
@SET LUAJIT_ARCH=x64
@SET TARGET_DIR=%LUAJIT_OS%-%LUAJIT_ARCH%

@SET BASE_LIB=glfw3.dll
@SET WRAPPER_LIB=wglfw.dll

@if not "x%1" == "x" GOTO :%1

:compile
@IF NOT EXIST build CALL make.bat configure
cmake --build build --config Release
COPY build\glfw\src\Release\%BASE_LIB% %TARGET_DIR%\
COPY build\Release\%WRAPPER_LIB% %TARGET_DIR%\
@GOTO :end

:configure
cmake -Bbuild -H. -G"Visual Studio 12 Win64"
@GOTO :end

:clean
rmdir /S /Q build
@GOTO :end

:end
Add a custom footer
Add a custom sidebar
Clone this wiki locally