OS X and Win10: Exported apps can't find SimpleOpenNI libs

[n1ckfg]

Issue Summary

Works fine in the Processing IDE, but exported apps look at root of the boot drive for the SimpleOpenNI folder, instead of in the correct location. This is an issue shared by all versions of SimpleOpenNI 1.96, including the original Max Rheiner version.

Executed Command (if any)

File > Export Application

Processing Output (if any)

Exported app crashes with gray screen.

Type of Issue

• Execution error

Your System Configuration

Operating system: OS X 10.12, Win10.

[totovr]

Hello I will try to get one Windows PC this week to taste on Windows, currently I just have a Mac to support the library.

[n1ckfg]

I wouldn't worry about Windows support yet--this is a bug that's been in every version of SimpleOpenNI since the beginning. If you can fix the Mac side, I can test on other OSes. Could you add the source of the SimpleOpenNI library itself? Like:
https://github.com/jaegonlee/SimpleOpenNI/tree/master/src/SimpleOpenNI

[ttks01]

I try this library for Processing 3.3.7 on Windows10. It goes well. However, export problem does not fix yet. Would you please fix nativLibPath in simpleOpenNI.java. It is explained in wexstorm/simple-openni#73 .

[n1ckfg]

OK, I think I have a solution--it needed a few changes from the wexstorm one. Sent a PR for 3.4.
https://github.com/n1ckfg/SimpleOpenNI/blob/Processing_3.4/SimpleOpenNI/src/SimpleOpenNI/SimpleOpenNI.java#L32

[n1ckfg]

Unfortunately the fix worked for depth but not for NITE--seems like NITE looks in the user's home directory (for some reason) when exported as a library regardless of the path set in the main jar.

[ffd8]

Eek – just ran into this issue... it's not quite clear if it's been fixed or what one needs to do?
(my setup is Processing 3.4, MacOS 10.9.5, hoping to run app on MacOS 10.13.6 high sierra)

[n1ckfg]

Oh hey, big fan of XYscope. So I've researched this quite a bit, even experimented with recompiling the jar, and my conclusion is it's a path issue with NITE, which we don't have the source for, so it can't be completely fixed. (Hope I'm wrong though.) The good news is, there's a workaround: place the SimpleOpenNI library folder at the root of your boot drive, and all exported apps can find it. This makes it really hard for distribution to civilians, but otherwise it's not even that bad a solution, because you don't need the same 200MB folder for every app. Further good news is it still works up to Win10 and High Sierra. (Haven't tried Mojave.)

[ffd8]

Thaanks + super thanks for quick reply. Ahaa, I was wondering about this when seeing what was changed in your various commits.. yup yup worked great (indeed not sooo user-friendly) but better than a sad gray-screen of nothingness.

Is there a flag or snippet to prevent it from including the 200mb library one each export, or simply have to inspect package contents and trash it?

If using skeleton tracking, does one still need to install OpenNi2:

brew tap brewsci/science
brew install openni2

My setup already has it.. so hard to test if app runs on fresh machine..

[n1ckfg]

I just trash it...don't believe it's necessary to install OpenNI separately but need to test. Fwiw I always install the SenseCast version of OpenNI 1 on a new machine as a good-luck ritual.

[ffd8]

Aha, nice tip (will check if OpenNI 1 suites my needs)– thanks again.

[TroikaTronix]

I don't know if it helps here, but I think NiTE's decision about where to load things from has to do with the current working directoy when you load it. I've recently developed a plugin for Isadora that uses NiTE. To get that to work I had to determine the path the the dylib and then use the chdir() command to set the working directory. My code for this is below.

`

#import <dlfcn.h>
#import <mach-o/dyld.h>
#import <mach-o/nlist.h>
#import <stdio.h>
#import <string.h>

#ifdef __LP64__
typedef struct mach_header_64 mach_header_t;
typedef struct segment_command_64 segment_command_t;
typedef struct nlist_64 nlist_t;
#else
typedef struct mach_header mach_header_t;
typedef struct segment_command segment_command_t;
typedef struct nlist nlist_t;
#endif

static const char * first_external_symbol_for_image(const mach_header_t *header)
{
	Dl_info info;
	if (dladdr(header, &info) == 0)
		return NULL;

	segment_command_t *seg_linkedit = NULL;
	segment_command_t *seg_text = NULL;
	struct symtab_command *symtab = NULL;

	struct load_command *cmd = (struct load_command *)((intptr_t)header + sizeof(mach_header_t));
	for (uint32_t i = 0; i < header->ncmds; i++, cmd = (struct load_command *)((intptr_t)cmd + cmd->cmdsize))
	{
		switch(cmd->cmd)
		{
			case LC_SEGMENT:
			case LC_SEGMENT_64:
				if (!strcmp(((segment_command_t *)cmd)->segname, SEG_TEXT))
					seg_text = (segment_command_t *)cmd;
				else if (!strcmp(((segment_command_t *)cmd)->segname, SEG_LINKEDIT))
					seg_linkedit = (segment_command_t *)cmd;
				break;

			case LC_SYMTAB:
				symtab = (struct symtab_command *)cmd;
				break;
		}
	}

	if ((seg_text == NULL) || (seg_linkedit == NULL) || (symtab == NULL))
		return NULL;

	intptr_t file_slide = ((intptr_t)seg_linkedit->vmaddr - (intptr_t)seg_text->vmaddr) - seg_linkedit->fileoff;
	intptr_t strings = (intptr_t)header + (symtab->stroff + file_slide);
	nlist_t *sym = (nlist_t *)((intptr_t)header + (symtab->symoff + file_slide));

	for (uint32_t i = 0; i < symtab->nsyms; i++, sym++)
	{
		if ((sym->n_type & N_EXT) != N_EXT || !sym->n_value)
			continue;

		return (const char *)strings + sym->n_un.n_strx;
	}

	return NULL;
}

const char * pathname_for_handle(void *handle)
{
	for (int32_t i = _dyld_image_count(); i >= 0 ; i--)
	{
		const char *first_symbol = first_external_symbol_for_image((const mach_header_t *)_dyld_get_image_header(i));
		if (first_symbol && strlen(first_symbol) > 1)
		{
			handle = (void *)((intptr_t)handle | 1); // in order to trigger findExportedSymbol instead of findExportedSymbolInImageOrDependentImages. See `dlsym` implementation at http://opensource.apple.com/source/dyld/dyld-239.3/src/dyldAPIs.cpp
			first_symbol++; // in order to remove the leading underscore
			void *address = dlsym(handle, first_symbol);
			Dl_info info;
			if (dladdr(address, &info))
				return info.dli_fname;
		}
	}
	return NULL;
}


void
SetWorkingDirectoryToNiteLibraryPath()
{
	void* lib_handle = dlopen("libNite2.dylib", RTLD_LOCAL| RTLD_LAZY);
	if (lib_handle != NULL) {
		std::string path = pathname_for_handle(lib_handle);
		if (path.length() > 1) {
			size_t pos = path.rfind("/");
			if (pos >= 1) {
				path = path.substr(0, pos);
				int result = chdir(path.c_str());
				int debug_breakpopint = result;
			}
		}
	}
}

`

Hopefully that might help you address the problem.

Also, I had to muck about with @loader_path to so that everytthing would work properly, but maybe that doesn't apply to you.

Best Wishses,
Mark

[n1ckfg]

Whoa, this is the first solid lead I think we've had in the several years this issue has been hanging around. Thanks!

[huangy10]

I've done some research about this problem today. Below are what I have found out:

the error info is:

Can't load SimpleOpenNI library (libSimpleOpenNI.jnilib) : java.lang.UnsatisfiedLinkError: Can't load library: /SimpleOpenNI/library/libSimpleOpenNI.jnilib
Verify if you installed SimpleOpenNI correctly.
http://code.google.com/p/simple-openni/wiki/Installation

java.lang.UnsatisfiedLinkError: SimpleOpenNI.SimpleOpenNIJNI.swig_module_init()V
	at SimpleOpenNI.SimpleOpenNIJNI.swig_module_init(Native Method)
	at SimpleOpenNI.SimpleOpenNIJNI.<clinit>(SimpleOpenNIJNI.java:290)
	at SimpleOpenNI.ContextWrapper.<init>(ContextWrapper.java:54)
	at SimpleOpenNI.SimpleOpenNI.<init>(SimpleOpenNI.java:253)
	at Sketch.settings(Sketch.java:28)
	at processing.core.PApplet.handleSettings(PApplet.java:954)
	at processing.core.PApplet.runSketch(PApplet.java:10786)
	at processing.core.PApplet.main(PApplet.java:10511)
	at Main.main(Main.java:7)

Why this happend?

Open SimpleOpenNI.jar through IntelliJ, and view SimpleOpenNI.class, below are how libSimpleOpenNI.jnilib is loaded:

static {
        String var0 = System.getProperty("os.name").toLowerCase();
        String var1 = "SimpleOpenNI";
        String var2 = System.getProperty("os.arch").toLowerCase();
        if (var0.indexOf("win") >= 0) {
            // ...
        } else if (var0.indexOf("nix") < 0 && var0.indexOf("linux") < 0) {
            if (var0.indexOf("mac") >= 0) {
                var1 = "lib" + var1 + ".jnilib";
                nativLibPath = getLibraryPathLinux() + "/SimpleOpenNI/library/";
                nativDepLibPath = nativLibPath + "osx/";
            }
        } else {
            nativLibPath = "/SimpleOpenNI/library/linux";
            if (var2.indexOf("86") >= 0) {
                var1 = var1 + "32";
            } else if (var2.indexOf("64") >= 0) {
                var1 = "lib" + var1 + "64.so";
                nativLibPath = getLibraryPathLinux() + "/SimpleOpenNI/library/";
                nativDepLibPath = nativLibPath + "linux64/";
            }
        }

        try {
            System.load(nativLibPath + var1);
        } catch (UnsatisfiedLinkError var5) {
            System.out.println("Can't load SimpleOpenNI library (" + var1 + ") : " + var5);
            System.out.println("Verify if you installed SimpleOpenNI correctly.\nhttp://code.google.com/p/simple-openni/wiki/Installation");
        }

        _initFlag = false;
    }

notice that, the result of getLibraryPathLinux() should be added before /SimpleOpenNI/library/libSimpleOpenNI.jnilib, the string appeared in the error information. Let's look at the function:

public static String getLibraryPathLinux() {
        URL var0 = SimpleOpenNI.class.getResource("SimpleOpenNI.class");
        if (var0 != null) {
            String var1 = var0.toString().replace("%20", " ");
            int var2 = var1.indexOf(47);
            boolean var3 = true;
            int var4 = var1.indexOf("/SimpleOpenNI/library");
            return -1 < var2 && -1 < var4 ? var1.substring(var2, var4) : "";
        } else {
            return "";
        }
    }
···

Notice this line

```java
int var4 = var1.indexOf("/SimpleOpenNI/library");

SimpleOpenNI tries to find substring /SimpleOpenNI/library from the full path of SimpleOpenNI.class file. If the substring does not show up, this function return an empty string. This is why processing fails to load SimpleOpenNI.jnilib file in exported application: when running from processing ide, the SimpleOpenNI.jar in ~/Documents/Processing/libraries/SimpleOpenNI/library directory is used, the full path of which contains /SimpleOpenNI/library. On the other hand, when running from exported application, the SimpleOpenNI.jar embedded in the application is used, which does not contain the required substring.

How to solve it.

A new implementation of getLibraryPathLinux is required to fix this problem thoroughly. A temporary solution is given below:

1. cd into your exported application bundle, go to Contents/MacOS, there should be a executable file with the same name of your exported application, Sketch, for example. Remove this file, and create a new empty file with the same name, edit the file and add the following contents:

#!/bin/bash
cd "$(dirname ${BASH_SOURCE})"
cd ../..
APP_ROOT=$(pwd)
cd Contents/Java

JAR_LIBS=$(ls *.jar | tr "\n" ":")
JAR_LIBS=${JAR_LIBS}./SimpleOpenNI/library/SimpleOpenNI.jar

APP_NAME=$(basename "${BASH_SOURCE}")

# Caution: if your embedded java has a different version, replace jdk1.8.0_181.jdk with the correct name
# if you didn't choose to embedded java in your exported application, set JAVA_BIN=java to use the global java
JAVA_BIN=${APP_ROOT}/Contents/PlugIns/jdk1.8.0_181.jdk/Contents/Home/jre/bin/java

${JAVA_BIN} \
-Djna.nosys=true \
-Djava.ext.dirs=$APP_ROOT/Contents/PlugIns/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext \
-Xdock:icon=$APP_ROOT/Contents/Resources/sketch.icns \
-Djava.library.path=$APP_ROOT/Contents/Java \
-Dapple.laf.useScreenMenuBar=true \
-Dcom.apple.macos.use-file-dialog-packages=true \
-Dcom.apple.macos.useScreenMenuBar=true \
-Dcom.apple.mrj.application.apple.menu.about.name=${APP_NAME} \
-classpath ${JAR_LIBS} ${APP_NAME}

make this file executable

chmod +x ./Sketch

2. Copy the entire folder ~/Documents/Processing/libraries/SimpleOpenNI to Contents/Java.

[n1ckfg]

Amazing! I actually saved the jar source with my fork: https://github.com/n1ckfg/SimpleOpenNI/tree/Processing_3.4_test/SimpleOpenNI/src ...tried a couple path fixes and recompiled but no luck. Do you have an idea of how you would implement that directly in the source?

[spacorum]

Amazing! I actually saved the jar source with my fork: https://github.com/n1ckfg/SimpleOpenNI/tree/Processing_3.4_test/SimpleOpenNI/src ...tried a couple path fixes and recompiled but no luck. Do you have an idea of how you would implement that directly in the source?

FINALLY! Thanks to your fork I was able to apply a definitive fix for the infamous path bug in .exe files. With your build.bat file and small changes to the solution posted in this link, you can easily generate a brand new .jar file which works perfectly with standalone .exe files. Here´s how (requires Processing 3.4 + SimpleOpenNI-Processing_3.4):

1. Edit @n1ckfg ´s fork "SimpleOpenNI-Processing_3.4_test\SimpleOpenNI\src\SimpleOpenNI\SimpleOpenNI.java" file with any text editor.

2. Insert this code before line 32:
   static String workingDir = System.getProperty("user.dir");

3. Replace lines 50 and 56 with:
   nativLibPath = getLibraryPathWin() + workingDir + "/SimpleOpenNI/library/";

4. (optional) Edit @n1ckfg ´s .bat file and replace the "core.jar" location with the one at your current Processing library path if it´s not the same (as in my case). Save file changes if required.

5. Launch build.bat file, which generates the new SimpleOpenNI.jar

6. Overwrite the current file at export folder "\application.windows64\lib" with the new one.

7. Manually copy the folder SimpleOpenNI to the same folder where your .exe is located. (the one that contains subfolders: "documentation", "examples" and "library").

8. Launch the .exe file and this time it will load the SimpleOpenNI library and run the Kinect. Et voilà!

This should do the trick. Thanks again!

[n1ckfg]

Amazing!

[n1ckfg]

@totovr If we put this solution in a pull request, which branch would you like us to work in?

[n1ckfg]

Both OS X and Windows solutions tested and working.