Change build to use pip

[ageorgou]

Currently, the Maven build uses easy_install to download and install the dependencies, but this is proving problematic in some cases (#302, #303). Ideally we can change this to use pip instead, and make the process more straightforward. The mavenjython plugin will address that in version 2.0, when we fix the setuptools import problem and publish it on the Nexus maven repo.

[raquelalegre]

Progress so far on the maven jython plugin to use pip in version 2.0 is here:

https://sourceforge.net/p/mavenjython/code/ci/pip/tree/

[ageorgou]

Somewhat related: using pip would potentially be easier if we move to Gradle instead of Maven, as suggested in #112, through the Juthon Gradle plugin.

[sgrieve]

The plugin is now working, and is able to pip install all of the dependencies. We have submitted the new version on nexus and are waiting for it to be published online so that we can build jar files without a local development version of the plugin.

We expect version 2.0 of the plugin to appear here: https://repo.maven.apache.org/maven2/net/sf/mavenjython/jython-compile-maven-plugin/

And the successfully submitted version 2.0 is listed here: https://oss.sonatype.org/#nexus-search;gav~net.sf.mavenjython~jython-compile-maven-plugin~~~~kw,versionexpand

We followed the instructions in #35 to submit the new version, the only thing we had to change was the server we submitted my public key to, as the mit server was down when we tried. I submitted my key to http://keyserver.ubuntu.com:11371 using the online form instead.

I generated my armoured public key using the command gpg --export -a "username"

[raquelalegre]

Somehow this works perfectly on Stuart's machine, but doesn't seem to work on Travis or mine one :( It complains about pyyaml and mako, and when commenting those out, it complains about pyoracc's mako dependency. These are errors we had before and were gone when we last tested in Stuart's machine with version 2.0 of the plugin.

Collecting pyyaml
/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:115: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  warnings.warn(
  Downloading https://files.pythonhosted.org/packages/4a/85/db5a2df477072b2902b0eb892feb37d88ac635d36245a72a6a69b23b383a/PyYAML-3.12.tar.gz (253kB)
    Error Cannot run program "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/jython" (in directory "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/build/pyyaml"): error=13, Permission denied while executing command python setup.py egg_info
Exception:
Traceback (most recent call last):
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/req/req_set.py", line 376, in prepare_files
    discovered_reqs.extend(self._prepare_file(
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/req/req_set.py", line 376, in prepare_files
    discovered_reqs.extend(self._prepare_file(
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/req/req_set.py", line 634, in _prepare_file
    abstract_dist.prep_for_dist()
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/req/req_set.py", line 129, in prep_for_dist
    self.req_to_install.run_egg_info()
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/req/req_install.py", line 435, in run_egg_info
    call_subprocess(
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/utils/__init__.py", line 665, in call_subprocess
    proc = subprocess.Popen(
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/subprocess.py", line 830, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/subprocess.py", line 1352, in _execute_child
    raise OSError(e.getMessage() or e)
OSError: Cannot run program "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/jython" (in directory "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/build/pyyaml"): error=13, Permission denied

After commenting pyyaml out in the pom file:

Collecting mako (from pyoracc)
  Using cached https://files.pythonhosted.org/packages/eb/f3/67579bb486517c0d49547f9697e36582cd19dafb5df9e687ed8e22de57fa/Mako-1.0.7.tar.gz
    Error Cannot run program "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/jython" (in directory "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/build/mako"): error=13, Permission denied while executing command python setup.py egg_info
Exception:
Traceback (most recent call last):
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/req/req_set.py", line 376, in prepare_files
    discovered_reqs.extend(self._prepare_file(
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/req/req_set.py", line 376, in prepare_files
    discovered_reqs.extend(self._prepare_file(
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/req/req_set.py", line 634, in _prepare_file
    abstract_dist.prep_for_dist()
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/req/req_set.py", line 129, in prep_for_dist
    self.req_to_install.run_egg_info()
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/req/req_install.py", line 435, in run_egg_info
    call_subprocess(
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/site-packages/pip/utils/__init__.py", line 665, in call_subprocess
    proc = subprocess.Popen(
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/subprocess.py", line 830, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/Lib/subprocess.py", line 1352, in _execute_child
    raise OSError(e.getMessage() or e)
OSError: Cannot run program "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/jython" (in directory "/Users/raquelalegre/workspace/ORACC/nammu/target/jython-plugins-tmp/build/mako"): error=13, Permission denied

I can't replicate the plugin working well, so I'll have a look again with Stuart on Wednesday when he's back.

[gulyx]

hi @raquel-ucl , not sure if this is the right place to post my question as I guess it could be also ralated with #302 or #304

Btw, I am trying to use the mavenjython plugin in a Java project that should integrate python libraries. This is the pom of the specific module using mavenjython. The resulting python environment should include/use the xxhash library which has been included according to the documentation of the plugin.

Running mvn package, it results with the error:

OSError: Cannot run program "/home/travis/build/gulyx/groucho/target/jython-plugins-tmp/jython" (in directory "/home/travis/build/gulyx/groucho/target/jython-plugins-tmp/build/xxhash"): error=13, Permission denied

These error is raised both on my local set-up and on travis-ci.
Also, as already discussed in #302, I noticed that the jython executable in the target directory created by maven is actually 0B.

Have you any suggestion on how to address the problem?

[ageorgou]

Hi @gulyx. Your question has to do with the mavenjython plugin itself rather than Nammu. I'm afraid that neither @raquel-ucl or anyone involved with Nammu actively maintain the mavenjython plugin at the moment, however, so it may be hard to get an answer. Perhaps the fix from #304 or another point in the discussion in #302 will help. Sorry we can't give more support with this!