Adding Luau to a CMake C project

[arrays-start-at-zero]

Hello, I do not have a lot of experience with CMake. How do I add Luau to a project that uses C instead of C++?

My current CMakeLists.txt looks like this:

cmake_minimum_required(VERSION 3.10)

project(MyProject C CXX)

option(LUAU_BUILD_CLI "" OFF)
option(LUAU_BUILD_TESTS "" OFF)
option(LUAU_EXTERN_C "" ON)
add_subdirectory(extern/luau)

set(CMAKE_C_STANDARD 23)
set(CMAKE_C_STANDARD_REQUIRED ON)
set(CMAKE_C_COMPILER /usr/bin/clang-20)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

add_executable(main src/main.c)
target_link_libraries(main PUBLIC Luau.Compiler Luau.VM)

But this gives me the following error:

main.c:

extern/luau/VM/include/lualib.h:

[biseson]

You're going to want to update your CMAKELists.txt and main.c to align with the compatibility issues. Just putting my judgement based off a similar repo I saw for the Lua -> CPP embed try this?:

CMakeLists.txt

cmake_minimum_required(VERSION 3.10)

project(MyProject C)
set(CMAKE_C_STANDARD 23)
set(CMAKE_C_STANDARD_REQUIRED ON)
set(CMAKE_C_COMPILER /usr/bin/clang-20)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

option(LUAU_BUILD_CLI "" OFF)
option(LUAU_BUILD_TESTS "" OFF)
option(LUAU_EXTERN_C "" ON)

add_subdirectory(extern/luau)
add_executable(main src/main.c)

target_link_libraries(main PUBLIC Luau.Compiler Luau.VM)
target_compile_definitions(main PRIVATE LUA_COMPAT_MODULE LUA_USE_LINUX)

main.c

#include "lua.h"
#include "lualib.h"
#include "lauxlib.h"

int main(void) {
    lua_State *lua = luaL_newstate();
    luaL_openlibs(lua);
    lua_close(lua);
    return 0;
}

[arrays-start-at-zero]

Thanks for the reply, unfortunately this does not work. This creates an endless build loop.

My main problem is that if I want to build for C I have to enable the LUAU_EXTERN_C option because of C++'s name mangling rules. The problem with this is that C does not support extern "C" and since both my project and Luau depend on certain headers like lualib.h and luacode.h, either I enable this option and the C compiler complains about extern "C", or I leave this option disabled and the linker complains it can't find functions like luaL_newState because their names are mangled.

I know that normally this can be fixed by wrapping the code with:

#ifdef __cplusplus
extern "C" {
#endif
// ..
#ifdef __cplusplus
}
#endif

But I don't think that's the right way here. So my question would be if there's an easier way. Like I said I don't have a lot of experience with CMake so I don't know some of the concepts but maybe there's a way I can enable LUAU_EXTERN_C only for Luau and leave it disabled for my main project?

[biseson]

You're correct in thinking that LUAU_EXTERN_C must be enabled for the Luau library to be compatible with your C. Only issue, enabling it globally causes problems because of the extern "C" block; C does not support that iirc.

I suggest trying to wrap the LUAU_EXTERN_C flag around only the Luau target, leaving your project unaffected by the extern "C" declaration.

This should validate the fact that the LUAU_EXTERN_C flag only applies when building Luau; leaving the rest of the project compiler unaffected:

CMakeLists.txt

cmake_minimum_required(VERSION 3.10)

project(MyProject C)

set(LUAU_BUILD_CLI OFF)
set(LUAU_BUILD_TESTS OFF)

add_subdirectory(extern/luau)

target_compile_definitions(Luau.Compiler PUBLIC LUAU_EXTERN_C=1)
target_compile_definitions(Luau.VM PUBLIC LUAU_EXTERN_C=1)

set(CMAKE_C_STANDARD 23)
set(CMAKE_C_STANDARD_REQUIRED ON)
set(CMAKE_C_COMPILER /usr/bin/clang-20)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

add_executable(main src/main.c)

target_link_libraries(main PUBLIC Luau.Compiler Luau.VM)

target_compile_definitions should ensure that LUAU_EXTERN_C is applied only to the Luau targets (Luau.Compiler/Luau.VM). Your C project will not see extern "C", but Luau's symbols will still be unmangled and compatible with your project.

Let me know if this works for your setup.