Skip to content
This repository has been archived by the owner on Mar 29, 2024. It is now read-only.

tulip-python: Remove tulipgui packaging and deployment to PyPI #48

Closed
wants to merge 6 commits into from
Closed

tulip-python: Remove tulipgui packaging and deployment to PyPI #48

wants to merge 6 commits into from

Conversation

anlambert
Copy link
Contributor

@anlambert anlambert commented Sep 27, 2018 •
edited
Loading

Hello,

I saw that there was some move lately in the repository regarding the generation of Python wheels to upload on PyPI. As I am the one who put this in place at the time, I thought that some advices from
me could be useful in order to ease the maintenance of that part.

First, as I said in the comments of commit 96f11e4#commitcomment-30659658, this process will not work to generate binary wheels on Linux compatible with all available Python versions. I fixed that.

Secondly, as also explained in 96f11e4#commitcomment-30659994, I think that the distribution of the tulipgui-python module on PyPI should be purely and simply abandonned.

Thinking it back, it was a silly idea from me to distribute that module on PyPI. I truly doubt that a lot of people use it as it is quite redundant with the main Tulip software capabilities while being less intuitive to use. Moreover it is really painful to build the wheel (especially on Linux) as there is a lot of fancy dependencies (Qt, GLEW, freetype, ...) that need to be bundled in it, and this is not really a good practice to upload such a wheel on PyPI. Based on my experience, the maintenance of that thing is not pleasant and too much time consuming for something that nobody use.

So I think that users should now be redirected to use the main Tulip software for visualization purposes instead of using the tulipgui module from PyPI (they can still use the one bundled with the Tulip installation if needed).

In that PR, the PyPI packaging for tulipgui-python is dropped while of course keeping the one for tulip-python. As there is only one wheel left, I renamed the following make targets:

  • wheels -> wheel
  • wheels-upload -> wheel-upload
  • wheels-upload-test -> wheel-upload-test

I have also added a CI jobs to generate and test the wheels on Linux, MacOS and Windows.

@tulipdev
Copy link
Contributor

tulipdev commented Sep 28, 2018 via email

@anlambert
Copy link
Contributor Author

Hi,

Below are some answers to your comments.

Thanks a lot again for all the great support!

You're welcome. Even if I am not involved in Tulip development anymore, I think helping
on the maintenance of the parts I was involved in and fix my mistakes is the least that I
can do.

If the only choice is to redirect them to the Tulip app, it may have a negative impact on Tulip.

I do not agree on that. tulipgui-python is basically a wrapper to display Tulip views from Python
so that's exactly the same thing you got when using the Tulip software.

Tulip app still suffers from a lot of unstability (let be honest, from users' perspective, the crashes are more or less random, very much like Meshlab, which is still a powerful and popular software). Only power users get the reflex (saving saving saving) and good practices to overcome these unstabilities and make the best out of Tulip. Meaning that a lot of people may turn to other frameworks if we force them to switch to the Tulip app

I understand that the user experience can be painful due to the frequent crashes. But if nobody fills a bug report or indicates how to reproduce the encountered issue, Tulip maintainers won't know it and so won't fix it. It's a shame because Tulip outputs a stack trace when it crashes and this is the kind of information of interest to the maintainers. An interesting feature would be to save that trace to a file and then upload it to your servers when Tulip is executed again, could help a lot tracking common bugs imho. Also, maybe adding a section in the official documentation regarding how to help debugging and making Tulip more stable would be a good idea ?

Another problematic issue is that the Github repository has never been officially advertised (see #46 ),
there is no link to the repository on the main page of Tulip website !!!. People will usually find
it through a google search but indicating clearly where to report bugs would be a good idea in my opinion.

To that mean, the Tulip-gui bindings are heaven, since they are more stable

From my point of view, simply loading a graph file produced with tulip-python in the main Tulip
software will not lead to a direct crash. If you do not apply a lot of complex operations on
you graph and simply visualize it as you would do with the tulipgui-python module, there is
no reason that a crash appears. That's why I said that the module is redundant with the
main software.

but the fact that they make a separated package is non-trivial.

Trust me, maintaining and compiling those binary wheels for tulipgui-python is a nightmare.
I can't count the time I worked on making the wheels work on all platforms at the time.
Currently, this works because Qt4 can still be used. But once Qt4 support will be dropped (based
on what I read in the 5.2 changelog) and Qt5 will be the only supported Qt version, things will get really complicated. The main issue is for the Linux platform as Qt5 plus other dependencies need to be compiled manually on a dedicated docker image based on Centos 5.11, and this is not a piece of cake. During last weeks, I tried to see if using Qt5 to build the tulipgui-python on Linux was feasible. I painfully found a way but at the cost of some hacks. This led me to the conclusion that the distribution of tulipgui-python
on PyPI should be abandonned because it is too time consuming and main development efforts should be done on the software itself and not on that complex binary packaging stuff.

Moreover, those kind of wheels including a lot of exotic binaries dependencies should not be uploaded on PyPI. It was fun to do at the time but thinking it back, that was not a good idea and it is just a giant hack on the capability of Python to import modules written in C/C++. For tulip-python, it is OK as it has a real solid API and it only relies on libstdc++ and zlib which are lightweight dependencies.

So I think it is ok not to distribute the Tulip-gui package, but we should for sure propose them for compilation (and advertise this as a great feature of Tulip).

Yes the feature will still be available if you compile Tulip yourself or use an official installation.

However, in the default tulip-python package, for the sake of Tulip's popularity and usability, I think we should propose at least a basic visualization of the graphs. People want to effortless check their graph at some point (not an export/import in a different software kind of thing). Maybe we can forget a second about interaction, and propose a rendering that would fit into pyplot?

From my point of view, that's out of scope. tulip-python is only dedicated to the creation and manipulation of graphs hierarchy and the application of algorithms. We gave the user the possibility
to draw any imported graph using Tulip layout algorithms and it's up to him to visualize it afterwards
with the technology of its choice (Tulip being the most adequate of course). Moreover tulip-python should not depend on other non standard Python modules and it should also remain usable in a server environment (without display available), let's keep it simple.

Anyway, based on that commit (b0b9321) I see that you will still try to build and upload the tulipgui wheel.
Based on my experience on the subject, you will need to do a lot of tests before effectively deploy
it to the PyPI server. My advices are the following:

  • create an account on https://test.pypi.org/ and use the wheels-upload-test make targets to upload the produced wheels on the PyPI test server (more details here: https://packaging.python.org/guides/using-testpypi/)
  • prior building the wheel, set the CMake variable named PythonWheelVersionSuffix to ".devN", start with N=1 then increment it after each upload (PyPI does not allow to upload twice a wheel with the same version number).
  • once a wheel has been uploaded, install it with pip in a clean environment different from the one you used to compile it. For linux, I recommend to use VirtualBox and test against various commonly used distributions (Debian stable, Fedora, ArchLinux, Ubuntu, CentOS).
  • once you are sure that the produced modules can be successfully imported in the wild, proceed to the upload on the real PyPI server through the wheels-upload make target.

In the mean time, I will continue to work on that PR (rebasing and pushing force the branch). You would be wise to merge it once it is ready.

Cheers!

@tulipdev
Copy link
Contributor

tulipdev commented Sep 29, 2018 via email

@anlambert
travis.yml: Fix some MacOS builds
4cbd8d7 
Use Sphinx 1.7 as Sphinx 1.8 seems buggy on that platform.
@anlambert
AppVeyor: Fix and improve MSYS2 builds
2bd39b6 
  - store the Inetc NSIS plugin in the repository instead of downloading it

  - use CMake provided by AppVeyor instead of the MSYS2 one

  - force install dependencies

  - remove Sphinx/Doxygen installation as it is not needed anymore
@anlambert
tulip-python: Remove tulipgui packaging and deployment on PyPI
774921f 
Distributing tulipgui-python on PyPI was a silly idea from my part.
Thinking it back, only tulip-python should be provided on PyPI.

The reasons of this removal are the following:

  - the module is not really used

  - bundling needed binaries in the wheel (notably the Qt ones) is painful
    and hard to maintain (especially on Linux)

  - the module is quite redundant with the main Tulip software, while being
    less intuituve to use

Previous users should now be redirected to use the main Tulip software for
visualization purposes or use the modules located in an official Tulip
installation instead.
@anlambert
travis.yml: Add job to build tulip-python wheel on manylinux1
296ff72
@anlambert
travis.yml: Add tulip-python wheels build for MacOS
dc9d77c
@anlambert
AppVeyor: Add tulip-python wheels build jobs
d62cde8
@anlambert anlambert force-pushed the tulip-python-maintenance-2 branch from 0df1b6a to d62cde8 Compare October 2, 2018 18:00
@anlambert anlambert changed the title tulip-python: wheel generation maintenance and miscellaneous fixes tulip-python: Remove tulipgui packaging and deployment to PyPI Oct 2, 2018
@anlambert
Copy link
Contributor Author

anlambert commented Oct 2, 2018 •
edited
Loading

I have updated the PR by fixing some noticed issues and by adding CI jobs to build tulip-python wheels on all platforms. There is still the documentation to update.

Ping me when you think it is time to merge it in master and I will rebase it to facilitate the operation.

@anlambert
Copy link
Contributor Author

anlambert commented Dec 12, 2018 •
edited
Loading

I saw my prayers have been heard (85f572a) so closing that PR (you could have pinged me instead of doing all the work on your own, a simple git rebase operation from my side and you won a couple days of work, anyway ...)

@anlambert anlambert closed this Dec 12, 2018
@anlambert
Copy link
Contributor Author

MacOS (https://travis-ci.org/Tulip-Dev/tulip/jobs/466597865) and Windows (https://ci.appveyor.com/project/tulip-dev/tulip/build/job/6wrn42xjbtr770k0) builds are broken ... I can help on the subject but you should ping me next time ...

This was referenced Apr 18, 2019
Closed
Closed
This was referenced Mar 12, 2020
Open
Open
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@anlambert @tulipdev