Meson AppImage releases #7518
Meson AppImage releases #7518
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
mesonbuild/environment.py
Outdated
| return (ninja, vers) | ||
| return None | ||
|
|
||
| env_ninja = os.environ.get('NINJA', None) |
There was a problem hiding this comment.
If more NINJA wrapper are coming, here would be more and more complicated
mesonbuild/mdist.py
Outdated
| @@ -184,15 +184,15 @@ def run_dist_steps(meson_command, unpacked_src_dir, builddir, installdir, ninja_ | |||
| if subprocess.call(meson_command + ['--backend=ninja', unpacked_src_dir, builddir]) != 0: | |||
| print('Running Meson on distribution package failed') | |||
| return 1 | |||
| if subprocess.call([ninja_bin], cwd=builddir) != 0: | |||
| if subprocess.call(ninja_bin, cwd=builddir) != 0: | |||
There was a problem hiding this comment.
Not a bin anymore, it's a list
|
The main conceptual question is whether these Appimages are going to be "best effort" kinda things or something we intend to officially support. The latter adds to the maintenance burden. |
|
What's the advantage of this over some python freeze program (e.g. pyinstaller) to create a meson executable, then expect the user to use the ninja standalone binaries available from the ninja project, and the cmake binaries from the cmake project? Especially cmake is probably better off distributed by kitware, and would pointlessly bloat the binary release for users who don't want or need cmake. |
|
To get things rolling, how about the following:
Than later if there is real world usage and requirements we can expand on this? |
Codecov Report
@@ Coverage Diff @@
## master #7518 +/- ##
===========================================
+ Coverage 39.66% 63.29% +23.62%
===========================================
Files 188 188
Lines 42048 42063 +15
Branches 8742 8747 +5
===========================================
+ Hits 16679 26624 +9945
+ Misses 23633 13058 -10575
- Partials 1736 2381 +645 Continue to review full report at Codecov.
|
|
This pull request introduces 1 alert when merging 2d84e5a into 5381ec8 - view on LGTM.com new alerts:
|
|
I have updated it to no longer include CMake and there are now also some major changes to the mechanism itself:
You can try the current x86_64 AppImage / Meson runtime here: runtime.tar.gz (zipped, since GitHub won't let me upload ist otherwise) |
The main point of providing AppImages / Meson runtimes is that it is convenient. You can run the script yourself, but then you have to install all the |
|
The requirements for this are pretty much:
If all three are met then sure, we can provide these. |
|
1. Basically everyone who complains when we bump our min Python version requirements
2. This is already the case. Just run the shell script next to the build.py. It will take a bit though, since we are building python from source
3. That would be me.
Another big advantage we get out of this is that we might be able to switch to newer python versions sooner because we now have a convienient, standalone alternative.
On 6 August 2021 00:08:50 CEST, Jussi Pakkanen ***@***.***> wrote:
The requirements for this are pretty much:
- There is an actual need for these binaries
- Building one on release is a matter of running at most one command
- Someone volunteers to take on the ongoing maintenance burden
If all three are met then sure, we can provide these.
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
#7518 (comment)
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
|
mensinda
force-pushed
the
AppImage
branch
2 times, most recently
from
13b54fc
to
3a6db4b
Compare
|
This pull request introduces 1 alert when merging 3a6db4b into 943596a - view on LGTM.com new alerts:
|
I have added a GitHub actions release script, so it now is just a case of waiting 15min. See https://github.com/mensinda/meson/releases/tag/0.0.13-runtime and https://github.com/mensinda/meson/runs/3269603623?check_suite_focus=true |
|
This pull request introduces 1 alert when merging 8c8829a into 943596a - view on LGTM.com new alerts:
|
|
Please rebase so CI is rerun. |
|
This pull request introduces 1 alert when merging 49a2c63 into f0b16e4 - view on LGTM.com new alerts:
|
|
The LGTM alert is not really helpful in this case... (Maybe we should add a UUID module to meson :D) |
I still think this could and should be solved the same way we do it for Windows... run the PyInstaller command to create an executable that unpacks its own copy of a) the python interpreter, b) the packaged module (mesonbuild). What does AppImage bring as an advantage, other than bundling its own copy of ninja, pkg-config, and cmake? None of which have to do with python, and:
This AppImage is trying to do too much, and the actual stated advantages of it are only advantages over "just use your OS python", not over "pyinstaller binaries". Using pyinstaller would have the advantage of it being architecturally identical to a I would love to see a PyInstaller Linux build of meson, but I really think AppImage is too complicated by design. |
It seems completely broken with --wipe, which deletes the entire directory and then the AppImage runtime tries to load itself from there anyway and falls over. Then once there is nothing there at all, it still crashes, with the same traceback but without mentioning "Using exsisting meson-runtime in builddir//meson-runtime" (typo intentional), but I am not sure why it wants to use something nonexistent... Trying to manage the lifecycle of a temporary extracted runtime strikes me as a bad move. If we want to have an extracted version of the standalone program, this is a long-ago solved problem: ship both python and mesonbuild/ in a tarball and ask users to extract it into |
|
The main advantage of the AppImage is that it contains everything Meson needs to run and that it is runnable pretty much everywhere (including ancient distros) because the runtime is a statically linked binary and it bundles libc (so literally everything needed to run meson). pyinstaller binaries still depend on the host libc. Additionally, we need a stable way to invoke the python from ninja, since we guarantee that it exists (and by also with the min required python version from Meson). I currently don't know how to achieve this with pyinstaller (though I also did not look into it in too much detail).
Yeah, required by AppImage... I am not denying that AppImages (and specifically this solution) is more complicated, but it is also more user-friendly. |
Except that you now need root privileges to install meson. Thanks for pointing out the bug with |
|
As an alternative: We could drop the runtime and AppImage part and just ship the AppDir (without the special *.desktop stuff). Then the users just needs to add You can currently easily get the same result for testing if you run |
Isn't this just
The traditional Linux way to be super compatible with everything is to build on the oldest libc you intend to support, or to make it an easy choice, build on CentOS 7. Another option should be, since IIRC pyinstaller bundles up the python interpreter you ran it with -- run pyinstaller from a custom built python which is statically linked to glibc, in Modules/Setup it appears you can tell python to include all modules in the interpreter directly rather than lib-dynload. |
You can optionally choose to install it to ~/opt/meson instead, and not need root. I don't know why people (this is not the first time I've seen that objection) persist in somehow thinking that a standalone binary which for convention and convenience gets installed to /opt (and symlinked to /usr/bin) so that it's convenient for all users on a system to access, must therefore require root as a mandatory part of the installation process, without which the standalone binary will crash. |
Then we have to teach meson to only generate rules that use
That's also how AppImages are usually built, but I wanted to support as many systems as possible (even those with a completely different libc (musl for instance)).
That would be great. Do you know the configure flag because I couldn't find a way to do this... |
edit the file Add to the top: Start uncommenting lines describing individual modules, e.g. diff --git a/Modules/Setup b/Modules/Setup
index 02cfb67518..b851ab5cce 100644
--- a/Modules/Setup
+++ b/Modules/Setup
@@ -1,4 +1,5 @@
# -*- makefile -*-
+*static*
# The file Setup is used by the makesetup script to construct the files
# Makefile and config.c, from Makefile.pre and config.c.in,
# respectively. Note that Makefile.pre is created from Makefile.pre.in
@@ -179,13 +180,13 @@ _symtable symtablemodule.c
#_testinternalcapi _testinternalcapi.c -I$(srcdir)/Include/internal -DPy_BUILD_CORE_MODULE # Python internal C API test module
#_random _randommodule.c -DPy_BUILD_CORE_MODULE # Random number generator
#_elementtree -I$(srcdir)/Modules/expat -DHAVE_EXPAT_CONFIG_H -DUSE_PYEXPAT_CAPI _elementtree.c # elementtree accelerator
-#_pickle _pickle.c # pickle accelerator
+_pickle _pickle.c # pickle accelerator
#_datetime _datetimemodule.c # datetime accelerator
#_zoneinfo _zoneinfo.c # zoneinfo accelerator
#_bisect _bisectmodule.c # Bisection algorithms
#_heapq _heapqmodule.c # Heap queue algorithm
#_asyncio _asynciomodule.c # Fast asyncio Future
-#_json -I$(srcdir)/Include/internal -DPy_BUILD_CORE_BUILTIN _json.c # _json speedups
+_json -I$(srcdir)/Include/internal -DPy_BUILD_CORE_BUILTIN _json.c # _json speedups
#_statistics _statisticsmodule.c # statistics accelerator
#unicodedata unicodedata.c # static Unicode character database
@@ -207,14 +208,14 @@ _symtable symtablemodule.c
#_csv _csv.c
# Socket module helper for socket(2)
-#_socket socketmodule.c
+_socket socketmodule.c
# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
-#SSL=/usr/local/ssl
-#_ssl _ssl.c \
-# -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-# -L$(SSL)/lib -lssl -lcrypto
+SSL=/usr/local/ssl
+_ssl _ssl.c \
+ -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
+ -L$(SSL)/lib -lssl -lcrypto
# The crypt module is now disabled by default because it breaks builds
# on many systems (where -lcrypt is needed), e.g. Linux (I believe).run |
From bf547ae1af86f17adda054eb3267f4e7b95fee00 Mon Sep 17 00:00:00 2001
From: Eli Schwartz <[email protected]>
Date: Sun, 15 Aug 2021 20:00:57 -0400
Subject: [PATCH] first version of python-static which successfully ran meson
built using
./configure --prefix /does/not/exist --disable-shared
make python LIBS=
./python interpreter loads no modules, only .py scripts, ld-linux.so
links to libc.so and libm.so (on versions of glibc < 2.34 where this is
a separate shared lib) but that can be configured away once this is
serious
LIBS= will prevent overlinking to libcrypt.so, which I don't have a
static version of, but again hold off until serious
---
Modules/Setup | 39 ++++++++++++++++++++-------------------
1 file changed, 20 insertions(+), 19 deletions(-)
diff --git a/Modules/Setup b/Modules/Setup
index 02cfb67518..f25987d3be 100644
--- a/Modules/Setup
+++ b/Modules/Setup
@@ -1,4 +1,5 @@
# -*- makefile -*-
+*static*
# The file Setup is used by the makesetup script to construct the files
# Makefile and config.c, from Makefile.pre and config.c.in,
# respectively. Note that Makefile.pre is created from Makefile.pre.in
@@ -169,36 +170,36 @@ _symtable symtablemodule.c
# Modules that should always be present (non UNIX dependent):
-#array arraymodule.c # array objects
+array arraymodule.c # array objects
#cmath cmathmodule.c _math.c -DPy_BUILD_CORE_MODULE # -lm # complex math library functions
-#math mathmodule.c _math.c -DPy_BUILD_CORE_MODULE # -lm # math library functions, e.g. sin()
-#_contextvars _contextvarsmodule.c # Context Variables
-#_struct _struct.c # binary structure packing/unpacking
+math mathmodule.c _math.c -DPy_BUILD_CORE_MODULE # -lm # math library functions, e.g. sin()
+_contextvars _contextvarsmodule.c # Context Variables
+_struct _struct.c # binary structure packing/unpacking
#_weakref _weakref.c # basic weak reference support
#_testcapi _testcapimodule.c # Python C API test module
#_testinternalcapi _testinternalcapi.c -I$(srcdir)/Include/internal -DPy_BUILD_CORE_MODULE # Python internal C API test module
-#_random _randommodule.c -DPy_BUILD_CORE_MODULE # Random number generator
+_random _randommodule.c -DPy_BUILD_CORE_MODULE # Random number generator
#_elementtree -I$(srcdir)/Modules/expat -DHAVE_EXPAT_CONFIG_H -DUSE_PYEXPAT_CAPI _elementtree.c # elementtree accelerator
-#_pickle _pickle.c # pickle accelerator
+_pickle _pickle.c # pickle accelerator
#_datetime _datetimemodule.c # datetime accelerator
#_zoneinfo _zoneinfo.c # zoneinfo accelerator
#_bisect _bisectmodule.c # Bisection algorithms
#_heapq _heapqmodule.c # Heap queue algorithm
#_asyncio _asynciomodule.c # Fast asyncio Future
-#_json -I$(srcdir)/Include/internal -DPy_BUILD_CORE_BUILTIN _json.c # _json speedups
+_json -I$(srcdir)/Include/internal -DPy_BUILD_CORE_BUILTIN _json.c # _json speedups
#_statistics _statisticsmodule.c # statistics accelerator
-#unicodedata unicodedata.c # static Unicode character database
+unicodedata unicodedata.c # static Unicode character database
# Modules with some UNIX dependencies -- on by default:
# (If you have a really backward UNIX, select and socket may not be
# supported...)
-#fcntl fcntlmodule.c # fcntl(2) and ioctl(2)
+fcntl fcntlmodule.c # fcntl(2) and ioctl(2)
#spwd spwdmodule.c # spwd(3)
#grp grpmodule.c # grp(3)
-#select selectmodule.c # select(2); not on ancient System V
+select selectmodule.c # select(2); not on ancient System V
# Memory-mapped files (also works on Win32).
#mmap mmapmodule.c
@@ -207,14 +208,14 @@ _symtable symtablemodule.c
#_csv _csv.c
# Socket module helper for socket(2)
-#_socket socketmodule.c
+_socket socketmodule.c
# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
-#SSL=/usr/local/ssl
-#_ssl _ssl.c \
-# -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-# -L$(SSL)/lib -lssl -lcrypto
+SSL=/usr/local/ssl
+_ssl _ssl.c \
+ -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
+ -L$(SSL)/lib -l:libssl.a -l:libcrypto.a
# The crypt module is now disabled by default because it breaks builds
# on many systems (where -lcrypt is needed), e.g. Linux (I believe).
@@ -229,7 +230,7 @@ _symtable symtablemodule.c
#termios termios.c # Steen Lumholt's termios module
#resource resource.c # Jeremy Hylton's rlimit interface
-#_posixsubprocess _posixsubprocess.c # POSIX subprocess module helper
+_posixsubprocess _posixsubprocess.c # POSIX subprocess module helper
# Multimedia modules -- off by default.
# These don't work for 64-bit platforms!!!
@@ -256,7 +257,7 @@ _symtable symtablemodule.c
#_sha3 _sha3/sha3module.c
# _blake module
-#_blake2 _blake2/blake2module.c _blake2/blake2b_impl.c _blake2/blake2s_impl.c
+_blake2 _blake2/blake2module.c _blake2/blake2b_impl.c _blake2/blake2s_impl.c
# The _tkinter module.
#
@@ -329,7 +330,7 @@ _symtable symtablemodule.c
# Helper module for various ascii-encoders
-#binascii binascii.c
+binascii binascii.c
# Fred Drake's interface to the Python parser
#parser parsermodule.c
@@ -338,7 +339,7 @@ _symtable symtablemodule.c
# Andrew Kuchling's zlib module.
# This require zlib 1.1.3 (or later).
# See http://www.gzip.org/zlib/
-#zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz
+zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -l:libz.a
# Interface to the Expat XML parser
# More information on Expat can be found at www.libexpat.org.
--
2.32.0With this patch plus a static copy of openssl and perhapsCC=musl-gcc there is a serious thought to meson shipping with a single-file executable version of python, plus the standard library of pure python. |
|
I tried building the image but it did not seem to work. First it gave this: Then eventually it failed with this: |
This PR adds an automatic AppImage builder for meson. Releasing with AppImages would allow us to support ancient operating systems (Ubuntu 12.04 for instance) far longer, while still working with newer dependencies (mostly python). It could also help with bootstrapping and broken python installations.
The resulting AppImage should work on almost all systems.
Included dependencies:
The AppImage produced by this PR can be downloaded here for testing.
Stuff to address in followup PRs:
Note: The required
appimagetoolisn't automatically downloaded and has to be manually passed as a command-line flag.