Build failure with custom LuaJIT (OpenResty fork) #242

carlocab · 2021-01-20T12:10:06Z

I'm trying to build luvi with a custom LuaJIT, which is the latest release build from the OpenResty fork. This produces the following build error:

luajit: unknown luaJIT command or jit.* modules not installed
luajitluajit: unknown luaJIT command or jit.* modules not installed
: unknown luaJIT command or jit.* modules not installed
make[2]: *** [jitted_tmp/src/lua/luvibundle.lua_luvi_generated.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [jitted_tmp/deps/lpeg/re.lua_luvi_generated.o] Error 1
make[2]: *** [jitted_tmp/src/lua/init.lua_luvi_generated.o] Error 1
luajit: unknown luaJIT command or jit.* modules not installed
make[2]: *** [jitted_tmp/src/lua/luvipath.lua_luvi_generated.o] Error 1
make[1]: *** [CMakeFiles/luvi.dir/all] Error 2
make: *** [all] Error 2

LuaJIT was built using the standard make amalg, so I'm not sure why I'm getting this error. Setting LUA_PATH to LuaJIT's jit directory as explained in a9dc8da didn't help. Assistance would be appreciated.

Complete build logs: https://gist.github.com/carlocab/2a6c25344be31127f61635d8978b3720

These were generated while attempting a build in Homebrew, but I can reproduce this without Homebrew using just a make tiny with the appropriate environment variables for set to enable building with a shared libluv. (I'm actually doing this to assist with Homebrew packaging of luvit. The latest luvi release doesn't build on Apple Silicon.)

The text was updated successfully, but these errors were encountered:

squeek502 · 2021-01-20T23:03:50Z

Looks like the errors are coming from here:

luvi/cmake/Modules/LuaJITAddExecutable.cmake

Lines 29 to 37 in 1c7cb3d

    
                 add_custom_command( 
        
                   OUTPUT ${generated_file} 
        
                   MAIN_DEPENDENCY ${source_file} 
        
                   COMMAND "LUA_PATH=${LUA_PATH}" luajit 
        
                   ARGS -b ${LJ_BYTECODE_OPTS} 
        
                     ${source_file} 
        
                     ${generated_file} 
        
                   COMMENT "Building Luajitted ${source_file}: ${generated_file}" 
        
                 )

where the bundled lua files are compiled using luajit -b. Does your custom LuaJIT support the -b option?

carlocab · 2021-01-21T01:11:26Z

What is the best way to check that it does? It recognises the -b option by itself, but I'm not sure if that's enough:

❯ ./luajit -b
Save LuaJIT bytecode: luajit -b[options] input output
  -l        Only list bytecode.
  -L        Only list bytecode with lineinfo.
  -s        Strip debug info (default).
  -g        Keep debug info.
  -n name   Set module name (default: auto-detect from input name).
  -t type   Set output file type (default: auto-detect from output name).
  -a arch   Override architecture for object files (default: native).
  -o os     Override OS for object files (default: native).
  -e chunk  Use chunk string as input.
  --        Stop handling options.
  -         Use stdin as input and/or stdout as output.

carlocab · 2021-01-21T01:20:15Z

Seems like it does. I ran the command manually:

❯ pwd
/tmp/luvi/src/lua
❯ /bin/ls
init.lua        luvibundle.lua  luvipath.lua
❯ /usr/local/opt/luajit-openresty/bin/luajit -b init.lua init.lua_luvi_generated.o
❯ /bin/ls
init.lua                        init.lua_luvi_generated.o       luvibundle.lua                  luvipath.lua

squeek502 · 2021-01-21T01:29:39Z

What's LUA_PATH set to when luajit -b works? Seems like the error comes from LuaJIT not being able to load things like bcsave.lua (similar problem here: openresty/opm#9).

The message in a9dc8da does seem to be relevant in that case.

carlocab · 2021-01-21T01:35:58Z

It's not set at all. echo $LUA_PATH returns nothing. However, when I set it to the jit directory:

❯ echo $LUA_PATH
/usr/local/opt/luajit-openresty/share/luajit-2.1.0-beta3/jit/?.lua
❯ /usr/local/opt/luajit-openresty/bin/luajit -b init.lua init.lua_luvi_generated.o
/usr/local/opt/luajit-openresty/bin/luajit: unknown luaJIT command or jit.* modules not installed

In fact, it seems like setting LUA_PATH to an empty variable causes problems:

❯ LUA_PATH= /usr/local/opt/luajit-openresty/bin/luajit -b init.lua init.lua_luvi_generated.o
/usr/local/opt/luajit-openresty/bin/luajit: unknown luaJIT command or jit.* modules not installed

carlocab · 2021-01-21T01:38:05Z

For reference, these are the contents of the jit directory:

❯ fd '.*\.lua'
share/luajit-2.1.0-beta3/jit/bc.lua
share/luajit-2.1.0-beta3/jit/bcsave.lua
share/luajit-2.1.0-beta3/jit/dis_arm.lua
share/luajit-2.1.0-beta3/jit/dis_arm64.lua
share/luajit-2.1.0-beta3/jit/dis_arm64be.lua
share/luajit-2.1.0-beta3/jit/dis_mips.lua
share/luajit-2.1.0-beta3/jit/dis_mips64.lua
share/luajit-2.1.0-beta3/jit/dis_mips64el.lua
share/luajit-2.1.0-beta3/jit/dis_mipsel.lua
share/luajit-2.1.0-beta3/jit/dis_ppc.lua
share/luajit-2.1.0-beta3/jit/dis_x64.lua
share/luajit-2.1.0-beta3/jit/dis_x86.lua
share/luajit-2.1.0-beta3/jit/dump.lua
share/luajit-2.1.0-beta3/jit/p.lua
share/luajit-2.1.0-beta3/jit/v.lua
share/luajit-2.1.0-beta3/jit/vmdef.lua
share/luajit-2.1.0-beta3/jit/zone.lua

squeek502 · 2021-01-21T01:38:56Z

Can you try changing this line:

luvi/cmake/Modules/LuaJITAddExecutable.cmake

Line 32 in 1c7cb3d

COMMAND "LUA_PATH=${LUA_PATH}" luajit

to:

        COMMAND luajit

and see if that fixes it for you?

carlocab · 2021-01-21T01:47:04Z

It works! ~~Tests are running atm. Let's see if that broke anything...~~ Tests pass.

squeek502 · 2021-01-21T02:00:07Z

Nice!

I'm not totally sure why that line has the "LUA_PATH=${LUA_PATH}" in it currently. Will need to investigate this more to see if it's necessary at all.

carlocab · 2021-01-21T02:05:26Z

In case it's helpful: I can also reproduce this with a release build of (classic, non-forked) LuaJIT. Of course, that's a bit older than the one pinned in your deps submodule.

I'm only using the OpenResty fork because that's the one with a tagged stable release that builds on Apple Silicon.

carlocab · 2021-01-21T02:32:41Z

I should also actually mention that using a shared libluv also causes errors with how LPEGLIB_DIR is set. Without manual intervention, I get this build error:

CMake Error in CMakeLists.txt:
  Cannot find source file:

    /tmp/luvit--luvi-20210121-94985-vucl7x/tmp/luvit--luvi-20210121-94985-vucl7x/deps/lpeg/re.lua

The correct path to re.lua is tmp/luvit--luvi-20210121-94985-vucl7x/deps/lpeg/re.lua. Not sure why the prefix is getting doubled there. Setting

export LPEGLIB_DIR=deps/lpeg

fixes the problem.

joerg-krause · 2021-11-07T11:26:30Z

Nice!

I'm not totally sure why that line has the "LUA_PATH=${LUA_PATH}" in it currently. Will need to investigate this more to see if it's necessary at all.

"LUA_PATH=${LUA_PATH}" was added by me in commit 0be7acc. This allows to use LuaJIT in build environments, where it is not installed in a default location, e.g. when cross-compiling, or when LuaJIT is installed in a custom directory.

carlocab · 2021-11-07T11:29:41Z

I have LuaJIT installed in a non-default location above, but the LUA_PATH setting seems to break my build (and getting rid of it fixes it).

joerg-krause · 2021-11-07T12:27:05Z

In case of a non-default location, you should pass LUA_PATH=/usr/local/opt/luajit-openresty/share/luajit-2.1.0-beta3/jit/?.lua to the build environment, when compiling luvi.

carlocab · 2021-12-22T15:32:10Z

In case of a non-default location, you should pass LUA_PATH=/usr/local/opt/luajit-openresty/share/luajit-2.1.0-beta3/jit/?.lua to the build environment, when compiling luvi.

But my non-default location works when luajit is invoked without setting LUA_PATH. It seems to me that we should only be doing LUA_PATH=${LUA_PATH} when it's already set in the environment, or else it'll break things even for default locations. See neovim/neovim#16714.

Is there any reason why we don't want to just set LUA_PATH to the output of luajit -e 'print(package.path)'?

…ng set Should fix luvit#242

…ng set (#258) Should fix #242

squeek502 added bug build labels Jan 21, 2021

carlocab mentioned this issue Jan 21, 2021

Build failure on Apple Silicon #243

Closed

squeek502 added a commit to squeek502/luvi that referenced this issue Jan 6, 2022

CMake: Make usage of LUA_PATH in LuaJITAddExecutable depend on it bei…

7060a5d

…ng set Should fix luvit#242

squeek502 mentioned this issue Jan 6, 2022

CMake: Make usage of LUA_PATH in LuaJITAddExecutable depend on it being set #258

Merged

squeek502 closed this as completed in #258 Jan 11, 2022

squeek502 added a commit that referenced this issue Jan 11, 2022

CMake: Make usage of LUA_PATH in LuaJITAddExecutable depend on it bei…

06a5c66

…ng set (#258) Should fix #242

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build failure with custom LuaJIT (OpenResty fork) #242

Build failure with custom LuaJIT (OpenResty fork) #242

carlocab commented Jan 20, 2021 •

edited

Loading

squeek502 commented Jan 20, 2021

carlocab commented Jan 21, 2021

carlocab commented Jan 21, 2021

squeek502 commented Jan 21, 2021 •

edited

Loading

carlocab commented Jan 21, 2021 •

edited

Loading

carlocab commented Jan 21, 2021

squeek502 commented Jan 21, 2021

carlocab commented Jan 21, 2021 •

edited

Loading

squeek502 commented Jan 21, 2021 •

edited

Loading

carlocab commented Jan 21, 2021 •

edited

Loading

carlocab commented Jan 21, 2021

joerg-krause commented Nov 7, 2021

carlocab commented Nov 7, 2021

joerg-krause commented Nov 7, 2021

carlocab commented Dec 22, 2021 •

edited

Loading

Build failure with custom LuaJIT (OpenResty fork) #242

Build failure with custom LuaJIT (OpenResty fork) #242

Comments

carlocab commented Jan 20, 2021 • edited Loading

squeek502 commented Jan 20, 2021

carlocab commented Jan 21, 2021

carlocab commented Jan 21, 2021

squeek502 commented Jan 21, 2021 • edited Loading

carlocab commented Jan 21, 2021 • edited Loading

carlocab commented Jan 21, 2021

squeek502 commented Jan 21, 2021

carlocab commented Jan 21, 2021 • edited Loading

squeek502 commented Jan 21, 2021 • edited Loading

carlocab commented Jan 21, 2021 • edited Loading

carlocab commented Jan 21, 2021

joerg-krause commented Nov 7, 2021

carlocab commented Nov 7, 2021

joerg-krause commented Nov 7, 2021

carlocab commented Dec 22, 2021 • edited Loading

carlocab commented Jan 20, 2021 •

edited

Loading

squeek502 commented Jan 21, 2021 •

edited

Loading

carlocab commented Jan 21, 2021 •

edited

Loading

carlocab commented Jan 21, 2021 •

edited

Loading

squeek502 commented Jan 21, 2021 •

edited

Loading

carlocab commented Jan 21, 2021 •

edited

Loading

carlocab commented Dec 22, 2021 •

edited

Loading