Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libsoftokn3.so issue #35

Open
probonopd opened this issue Oct 22, 2016 · 36 comments
Open

libsoftokn3.so issue #35

probonopd opened this issue Oct 22, 2016 · 36 comments

Comments

@probonopd
Copy link
Owner

probonopd commented Oct 22, 2016

When I try to use -bundle-non-qt-libs on a working Otter AppDir or try to convert it to an AppImage, Otter fails to run with

[1022/144633:ERROR:nss_util.cc(740)] Error initializing NSS with a persistent database (sql:/home/me/.pki/nssdb): libsoftokn3.so: cannot open shared object file: No such file or directory
[1022/144633:ERROR:nss_util.cc(211)] Error initializing NSS without a persistent database: NSS error code: -5925
[1022/144633:FATAL:nss_util.cc(213)] nss_error=-5925, os_error=0

The reason for this is to be investigated.

When I delete libnss3.so from the AppDir, then it starts working again.

@probonopd
Copy link
Owner Author

Possibly fixed in 2a19bc3, please confirm

@g-maxime
Copy link

g-maxime commented Oct 26, 2016

At last libnssutils3.so, libsmime3.so and libssl3.so depends on it, causing :

/usr/lib64/libnss3.so: version `NSS_3.21' not found (required by ./lib64/libssl3.so)

@probonopd
Copy link
Owner Author

Now we have an interesting situation here; recently we ran into an application (Firefox) that refused to run without libnss3.so.

So I am not entirely sure what is the best path of action here.

Reopening the issue. Deleting libnss3.so appears to be the wrong course of action.

@probonopd probonopd reopened this Oct 26, 2016
@probonopd
Copy link
Owner Author

probonopd commented Mar 19, 2017

We should check which libraries upstream Firefox bundles, and do the same. (Be sure to carry this over to linuxdeployqt too.)

@darealshinji
Copy link

At least on Ubuntu the following libraries are all part of the same libnss3 binary package:

/usr/lib/x86_64-linux-gnu/libnss3.so
/usr/lib/x86_64-linux-gnu/libnssutil3.so
/usr/lib/x86_64-linux-gnu/libsmime3.so
/usr/lib/x86_64-linux-gnu/libssl3.so
/usr/lib/x86_64-linux-gnu/nss/libfreebl3.so
/usr/lib/x86_64-linux-gnu/nss/libfreeblpriv3.so
/usr/lib/x86_64-linux-gnu/nss/libnssckbi.so
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.so
/usr/lib/x86_64-linux-gnu/nss/libnsssysinit.so
/usr/lib/x86_64-linux-gnu/nss/libsoftokn3.so

I hope that helps to figure out what stuff needs to be bundled or excluded.

@probonopd
Copy link
Owner Author

Very good observation @darealshinji - this probably suggests that we should treat all of those together; especially either bundle both libnss3 and libssl3 or neither. We should try both approaches with the Otter AppImage.

@probonopd
Copy link
Owner Author

Potentially we could even build some logic/database that would always treat libraries that are part of the same package together...

@tamlok
Copy link

tamlok commented Apr 7, 2018

@probonopd

Hi,

I also have this problem with VNote. After deleting the libnss3.so and libnssutil3.so in the extracted AppImage, it could run successfully. Then I change the script in travis CI like this:

# Get linuxdeployqt tool
wget -c "https://github.com/probonopd/linuxdeployqt/releases/download/continuous/linuxdeployqt-continuous-x86_64.AppImage"
chmod a+x linuxdeployqt*.AppImage
unset QTDIR; unset QT_PLUGIN_PATH ; unset LD_LIBRARY_PATH
./linuxdeployqt*.AppImage ./dist/usr/share/applications/*.desktop -bundle-non-qt-libs

# Copy translations
cp /opt/qt59/translations/*_zh_CN.qm ./dist/usr/translations/

# Package it for the second time.
./linuxdeployqt*.AppImage ./dist/usr/share/applications/*.desktop -appimage

tree dist/

ls -l *.AppImage

wget "https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage"
chmod a+x appimagetool-x86_64.AppImage

mv VNote-*.AppImage VNote-x86_64.AppImage
chmod a+x VNote-x86_64.AppImage
./VNote-x86_64.AppImage --appimage-extract
rm VNote-x86_64.AppImage
# Delete libnss3.so libnssutil3.so.
rm ./squashfs-root/usr/lib/libnss3.so ./squashfs-root/usr/lib/libnssutil3.so

./appimagetool-x86_64.AppImage squashfs-root

mv VNote-*.AppImage VNote_x86_64_${version}.AppImage

However, the re-packaged AppImage lack the libFcitxQt5DBusAddons.so.1, which seems to exist in the squashfs-root. Append the travis log here.

Any idea to remove the libnss3.so in the AppImage?

Thanks!

@probonopd
Copy link
Owner Author

However, the re-packaged AppImage lack the libFcitxQt5DBusAddons.so.1, which seems to exist in the squashfs-root

Are you saying it magically dissapeared when re-packing the AppImage? What did you use to repack? Please use appimagetool to repack an extracted AppImage.

@tamlok
Copy link

tamlok commented Apr 8, 2018

Yes, the lib disappeared after re-pack. I used appimagetool to pack it. Finally I try the new -exclude-libs option and it works now. Thanks!

@forelabs
Copy link

forelabs commented Apr 18, 2018

Currently I have the same issue, using Qt 5.9.4 and latest linuxdeploymentqt.
But working with -exclude-libs argument seems not to be an option for me,
because then my little WebEngine (WebView) component renders also the css in front of the html as plain text, which should not happen.

https://launchpad.net/~beineri/+archive/ubuntu/opt-qt-5.4.9-xenial is missing the libQt5WebView.so, which gets installed by using Qt 5.9.4 from the qt installer.

BTW: When using Qt 5.10.1 it fails with a different error. Currently trying to get around this problem by using an older qt version. Maybe I will open a new issue according the problem in Qt 5.10.1, because using this version without the lib seems not to be a working solution :/

@Silex
Copy link

Silex commented Apr 19, 2018

I have the same problem and what is weird is that it appeared recently without touching anything else: builds from january were working fine, and now I have this NSS error.

We are building on 14.04 for historical reasons, maybe libnss was recently upgraded?

My app does not want to start if I delete libnss3.so from the lib directory.

@probonopd
Copy link
Owner Author

Check with strace what is actually going on - is it loading /usr/lib/x86_64-linux-gnu/nss/libsoftokn3.so correctly from the system (in case it is a Debian/Ubuntu based system)?

@Silex
Copy link

Silex commented Apr 20, 2018

@probonopd: actually it was not!

Using this works:

LD_LIBRARY_PATH="/usr/lib/x86_64-linux-gnu/nss" mybinary

So basically, it'd just be a matter of packaging that lib in the appimage. I'll play around and report.

@Silex
Copy link

Silex commented Apr 20, 2018

Ok I just checked based on my latest successful build, and found out that 46a1e7f is able to build & run my Appimage.

I also confirm that using -exclude-libs="libnss3.so,libnssutil3.so" with the latest version fixes the problem.

@Silex
Copy link

Silex commented Apr 20, 2018

So after verifications, basically in previous versions it did not package libnss* but it was loaded from the system dir, because it was already installed by 3rd party packages (e.g firefox & friends).

In the latest version, it tries to bundle this dependency but is not complete enough so the loading fails?

Here is the difference between the appimages (I sorted it a bit so it's easier to visualize):

root@P080027f4c1bf ~ # diff -rl squashfs-root-46a1e7f/ squashfs-root-latest/
Binary files squashfs-root-46a1e7f/AppRun and squashfs-root-latest/AppRun differ

Only in squashfs-root-46a1e7f/doc: libatk-bridge2.0-0
Only in squashfs-root-46a1e7f/doc: libatk1.0-0
Only in squashfs-root-46a1e7f/doc: libatspi2.0-0
Only in squashfs-root-46a1e7f/doc: libcairo-gobject2
Only in squashfs-root-46a1e7f/doc: libcairo2
Only in squashfs-root-46a1e7f/doc: libdatrie1
Only in squashfs-root-46a1e7f/doc: libfreetype6
Only in squashfs-root-46a1e7f/doc: libgraphite2-3
Only in squashfs-root-46a1e7f/doc: libgtk-3-0
Only in squashfs-root-46a1e7f/doc: libharfbuzz0b
Only in squashfs-root-46a1e7f/doc: libpixman-1-0
Only in squashfs-root-46a1e7f/doc: libselinux1
Only in squashfs-root-46a1e7f/doc: libthai0
Only in squashfs-root-46a1e7f/doc: libwayland-client0
Only in squashfs-root-46a1e7f/doc: libwayland-cursor0
Only in squashfs-root-46a1e7f/doc: libxinerama1
Only in squashfs-root-46a1e7f/doc: libxkbcommon0
Only in squashfs-root-46a1e7f/doc: libxrandr2
Only in squashfs-root-latest/doc: libk5crypto3

Only in squashfs-root-46a1e7f/lib: libXinerama.so.1
Only in squashfs-root-46a1e7f/lib: libXrandr.so.2
Only in squashfs-root-46a1e7f/lib: libatk-1.0.so.0
Only in squashfs-root-46a1e7f/lib: libatk-bridge-2.0.so.0
Only in squashfs-root-46a1e7f/lib: libatspi.so.0
Only in squashfs-root-46a1e7f/lib: libcairo-gobject.so.2
Only in squashfs-root-46a1e7f/lib: libcairo.so.2
Only in squashfs-root-46a1e7f/lib: libdatrie.so.1
Only in squashfs-root-46a1e7f/lib: libfreetype.so.6
Only in squashfs-root-46a1e7f/lib: libgdk-3.so.0
Only in squashfs-root-46a1e7f/lib: libgraphite2.so.3
Only in squashfs-root-46a1e7f/lib: libgtk-3.so.0
Only in squashfs-root-46a1e7f/lib: libharfbuzz.so.0
Only in squashfs-root-46a1e7f/lib: libpixman-1.so.0
Only in squashfs-root-46a1e7f/lib: libselinux.so.1
Only in squashfs-root-46a1e7f/lib: libthai.so.0
Only in squashfs-root-46a1e7f/lib: libwayland-client.so.0
Only in squashfs-root-46a1e7f/lib: libwayland-cursor.so.0
Only in squashfs-root-46a1e7f/lib: libxkbcommon.so.0
Only in squashfs-root-latest/lib: libk5crypto.so.3
Only in squashfs-root-latest/lib: libnss3.so
Only in squashfs-root-latest/lib: libnssutil3.so

Only in squashfs-root-46a1e7f/plugins: platformthemes

Binary files squashfs-root-46a1e7f/provision_browser and squashfs-root-latest/provision_browser differ

@Silex
Copy link

Silex commented Apr 20, 2018

Ok, another fix is doing cp /usr/lib/x86_64-linux-gnu/nss/* lib before building the appimage.

Also it's opened by the app:

stvs@P080027f4c1bf:~/squashfs-root$ strace -f ./AppRun 2>&1 | grep libsoftokn3
[pid  4720] open("/home/stvs/squashfs-root/lib/nss/libsoftokn3.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[pid  4720] open("/home/stvs/squashfs-root/lib/libsoftokn3.so", O_RDONLY|O_CLOEXEC) = 36

From that I infer that it'd be maybe better to do cp -r /usr/lib/x86_64-linux-gnu/nss lib/ instead.

@probonopd
Copy link
Owner Author

probonopd commented Apr 20, 2018

Ok, another fix is doing cp /usr/lib/x86_64-linux-gnu/nss/* lib before building the appimage.

Correct, that is the way to do it. linuxdeployqt will only deploy libraries ldd can find on the system.

Unless upstream Qt fixes the principal issue, since it seems their binary distribution does not work properly on Debian/Ubuntu-based system without the above workaround. Did you open a ticket there?

@DoctorJohn
Copy link

DoctorJohn commented Oct 13, 2018

What kind of patch could that be?

Looks like those .chk files are always called the same as the .so files they belong to.
linuxdeployqt determines what .so files the program depends on using ldd right?
So we could check whether there are matching .chk files next to them(?)

Any opinions on that?

(Sorry, posted the same thing earlier using a wrong github account)

@probonopd
Copy link
Owner Author

Yes, exactly as you describe @DoctorJohn

@probonopd
Copy link
Owner Author

libsoftokn3.so is seemingly still a pain:

me@host:~$ Downloads/REDasm_x86_64_20181231.AppImage Downloads/AppImageUpdate-x86_64.AppImage 
QBuffer::setData: Buffer is open
[9708:9734:0102/034434.903619:ERROR:nss_util.cc(712)] Error initializing NSS with a persistent database (sql:/home/me/.pki/nssdb): libsoftokn3.so: cannot open shared object file: No such file or directory
[9708:9734:0102/034434.903712:ERROR:nss_util.cc(201)] Error initializing NSS without a persistent database: NSS error code: -5925
[9708:9734:0102/034434.903732:FATAL:nss_util.cc(203)] nss_error=-5925, os_error=0
#0 0x7f35b3c0f5de <unknown>
#1 0x7f35b3c21d1e <unknown>
....

@shoogle
Copy link

shoogle commented Jan 14, 2020

For MuseScore we are bundling all the files that are part of the libnss3 package, namely:

/usr/lib/x86_64-linux-gnu/libnss3.so
/usr/lib/x86_64-linux-gnu/libnssutil3.so
/usr/lib/x86_64-linux-gnu/libsmime3.so
/usr/lib/x86_64-linux-gnu/libssl3.so
/usr/lib/x86_64-linux-gnu/nss/libfreebl3.chk
/usr/lib/x86_64-linux-gnu/nss/libfreebl3.so
/usr/lib/x86_64-linux-gnu/nss/libnssckbi.so
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.chk
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.so
/usr/lib/x86_64-linux-gnu/nss/libnsssysinit.so
/usr/lib/x86_64-linux-gnu/nss/libsoftokn3.chk
/usr/lib/x86_64-linux-gnu/nss/libsoftokn3.so

However, we don't load them by default. Instead we put them in a separate directory within the AppImage that only gets added to ${LD_LIBRARY_PATH} at runtime if libnss3.so is not found on the system.

altairwei added a commit to altairwei/WizNotePlus that referenced this issue Feb 10, 2020
@andrewpriddis
Copy link

libsoftokn3.so is seemingly still a pain:

me@host:~$ Downloads/REDasm_x86_64_20181231.AppImage Downloads/AppImageUpdate-x86_64.AppImage 
QBuffer::setData: Buffer is open
[9708:9734:0102/034434.903619:ERROR:nss_util.cc(712)] Error initializing NSS with a persistent database (sql:/home/me/.pki/nssdb): libsoftokn3.so: cannot open shared object file: No such file or directory
[9708:9734:0102/034434.903712:ERROR:nss_util.cc(201)] Error initializing NSS without a persistent database: NSS error code: -5925
[9708:9734:0102/034434.903732:FATAL:nss_util.cc(203)] nss_error=-5925, os_error=0
#0 0x7f35b3c0f5de <unknown>
#1 0x7f35b3c21d1e <unknown>
....

I'm seeing this same problem @probonopd mentioned on Jan 1. I don't know if this matters, but if I build my Qt project it runs properly. When I guild the AppImage, I see the error above. The interesting part is that if I go try to run the executable that I previously ran from the standard build it no longer works and gives the same error.

@martinrotter
Copy link

So guys, is there any "recommended" (dirty?) solution to get things working?

timangus added a commit to graphia-app/graphia that referenced this issue Dec 17, 2020
timangus added a commit to graphia-app/graphia that referenced this issue Dec 17, 2020
@chapati
Copy link

chapati commented Jun 14, 2021

For MuseScore we are bundling all the files that are part of the libnss3 package, namely:

/usr/lib/x86_64-linux-gnu/libnss3.so
/usr/lib/x86_64-linux-gnu/libnssutil3.so
/usr/lib/x86_64-linux-gnu/libsmime3.so
/usr/lib/x86_64-linux-gnu/libssl3.so
/usr/lib/x86_64-linux-gnu/nss/libfreebl3.chk
/usr/lib/x86_64-linux-gnu/nss/libfreebl3.so
/usr/lib/x86_64-linux-gnu/nss/libnssckbi.so
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.chk
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.so
/usr/lib/x86_64-linux-gnu/nss/libnsssysinit.so
/usr/lib/x86_64-linux-gnu/nss/libsoftokn3.chk
/usr/lib/x86_64-linux-gnu/nss/libsoftokn3.so

However, we don't load them by default. Instead we put them in a separate directory within the AppImage that only gets added to ${LD_LIBRARY_PATH} at runtime if libnss3.so is not found on the system.

We've run into the same issue with libsoftokn3.so on Ubuntu 18 & 20 LTS while building on Ubuntu 16 LTS and went with a temporary exclude libs fix. @probonopd would you please add the workaround proposed by @shoogle above to the linuxdeployqt. May be it would be even better to add some general mechanism to define some list of libraries that would be loaded from inside appimage only if they are not present on a target system. Thanks.

@chapati
Copy link

chapati commented Jun 14, 2021

So guys, is there any "recommended" (dirty?) solution to get things working?

There are two fixes that work at the moment:

  1. Short and dirty and not for all systems (but seems to work on majority) libsoftokn3.so issue #35 (comment)
  2. Long and better but significantly more painful to implement libsoftokn3.so issue #35 (comment)

tim-janik added a commit to tim-janik/anklang that referenced this issue Jun 24, 2022
There seem to exist some incompatibilities with an appimage-packed libnss3.so
loading it's corresponding libsoftokn3.so from the host system libraries.
As a workaround, we avoid packaging libnss3.so alltogether which is present
on at least Ubuntu 20.04, 22.04, Debian 11 and Arch in 2022.

For a detailed discussion, see:
probonopd/linuxdeployqt#35 (comment)

Signed-off-by: Tim Janik <[email protected]>
tim-janik added a commit to tim-janik/anklang that referenced this issue Jun 24, 2022
* electron-fixes:
  Makefile.mk: fix ase/Makefile.mk missing devices/Makefile.mk definitions
  MISC: package.json.in: downgrade to electron-18.3.5 due to a CORS bug
	Some requests fail sporadically with a "blocked by CORS policy" error.
	The details are described here:
	  Issue 1339128: Local CORS requests fail sproradically
	  https://bugs.chromium.org/p/chromium/issues/detail?id=1339128
  ELECTRON: main.js: support --dev option for devtools
  MISC: Makefile.mk: appimage: exclude libnss*3.so
	There seem to exist some incompatibilities with an appimage-packed libnss3.so
	loading it's corresponding libsoftokn3.so from the host system libraries.
	As a workaround, we avoid packaging libnss3.so alltogether which is present
	on at least Ubuntu 20.04, 22.04, Debian 11 and Arch in 2022.
	For a detailed discussion, see:
	probonopd/linuxdeployqt#35 (comment)

Signed-off-by: Tim Janik <[email protected]>
@paoletto
Copy link

stumbled upon this one when building on 22.04 (because the appimage generated on 18.04 does not run on 22.04 sadly).
So even on the same 22.04 system, when running the app from the appimage triggers this one :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests