Failure to build apk due to incorrect invocation of "ant" by "sh.ant"...

[bidafiwoc]

[This issue report represents a condensation of the exchange with Alexander Taylor on the google group for p4a. If interested, please reference original at https://groups.google.com/forum/#!topic/python-android/2fLGqp6Bcec .]

I'm using the new (master) branch of python-for-android (at commit 93728aa), on top of a clean, up-to-date install of Debian jessie; all package dependencies are installed as listed in the p4a readme docs. I'm finding the error shown below.

I see the same error (below) whether I call p4a directly, or if I use buildozer to call it (via "android_new"). The old-toolchain of python-for-android works fine on this system to produce APKs. The error below seems strange to me, as the command being called by p4a:

/opt/android-sdk-linux/tools/ant debug

doesn't seem right to me (more below).

Here's my call to p4a:

p4a apk --sdk_dir /opt/android-sdk-linux --ndk_dir /opt/android-ndk-r10e
--android_api 15 --ndk_ver r10e-rc4 --private /root/HelloWorld
--package=org.hw.helloworld --name="HelloWorld"
--version=0.5 --bootstrap=pygame --requirements=sdl,python2
--dist_name=helloworld

Here's some initial logging output from p4a showing things going well:

INFO: This python-for-android revamp is an experimental alpha release!
INFO: It should work (mostly), but you may experience missing features or bugs.
INFO: Will compile for the following archs: armeabi
INFO: Getting Android API version from user argument
INFO: Available Android APIs are (15)
INFO: Requested API target 15 is available, continuing.
INFO: Getting NDK dir from from user argument
INFO: Got NDK version from from user argument
INFO: Using Google NDK r10e-rc4
INFO: Found virtualenv at /usr/local/bin/virtualenv
INFO: Found the following toolchain versions: '4.8', '4.9', 'clang3.5', 'clang3.6': Picking the latest gcc toolchain, here 4.9
INFO: Of the existing distributions, the following meet the given requirements:
INFO: helloworld: includes recipes (hostpython2, pygame_bootstrap_components, python2, sdl, six, pygame, pyjnius, android, kivy), built for archs (armeabi)
INFO: helloworld has compatible recipes, using this one
INFO: Of the existing distributions, the following meet the given requirements:
INFO: helloworld: includes recipes (hostpython2, pygame_bootstrap_components, python2, sdl, six, pygame, pyjnius, android, kivy), built for archs (armeabi)
INFO: helloworld has compatible recipes, using this one
INFO: -> directory context /root/.local/share/python-for-android/dists/helloworld

Here's the relevant snippet of log output showing the error:

INFO: -> running ant debug

   Traceback (most recent call last):
File "/usr/local/bin/p4a", line 9, in <module>
  load_entry_point('python-for-android==0.3', 'console_scripts', 'p4a')()
File "/usr/local/lib/python2.7/dist-packages/pythonforandroid/toolchain.py", line 708, in main
  ToolchainCL()
File "/usr/local/lib/python2.7/dist-packages/pythonforandroid/toolchain.py", line 323, in __init__
  getattr(self, args.command)(unknown)
File "/usr/local/lib/python2.7/dist-packages/pythonforandroid/toolchain.py", line 106, in wrapper_func
  func(self, args)
File "/usr/local/lib/python2.7/dist-packages/pythonforandroid/toolchain.py", line 556, in apk
  shprint(sh.ant, 'debug', _tail=20, _critical=True)
File "/usr/local/lib/python2.7/dist-packages/pythonforandroid/logger.py", line 164, in shprint
  output = command(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/sh.py", line 769, in __call__
  return RunningCommand(cmd, call_args, stdin, stdout, stderr)
File "/usr/lib/python2.7/dist-packages/sh.py", line 327, in __init__
  self.call_args, pipe=pipe)
File "/usr/lib/python2.7/dist-packages/sh.py", line 891, in __init__
  os.execv(cmd[0], cmd)

OSError: [Errno 13] Permission denied

cd /root/.local/share/python-for-android/dists/helloworld && /opt/android-sdk-linux/tools/ant debug

[WARNING]: ERROR: /opt/android-sdk-linux/tools/ant failed!

But, the command being executed ("/opt/android-sdk-linux/tools/ant") is referencing what is the ant DIRECTORY, not any ant executable (so the permission denied error message is unsurprising). At least that's the internal structure of the Android SDK (r24.4.1) I downloaded and extracted earlier today:

root@laptop:~# ls -d /opt/android-sdk-linux/tools/ant/*
/opt/android-sdk-linux/tools/ant/build.xml
/opt/android-sdk-linux/tools/ant/NOTICE
/opt/android-sdk-linux/tools/ant/uibuild.xml

Python-for-android is definitely trying to call the ant executable above (using its "sh.ant" functionality) --- what strikes me as strange, though, are two aspects of this call:

(1) the call to ant uses a fully-canonicalized path (shown above, in log from my run as: "/opt/android-sdk-linux/tools/ant"). So, irrespective of my particular $PATH setting, or location of ant on my system, it's not going to search any further locations than that one. Incidentally, though not mentioned above (I should have), my version of ant was installed via standard Debian (Jessie) package, so the primary ant executable lives at normal location of "/usr/bin/ant". (If the solution were as simple as more careful construction of my $PATH env variable, I probably wouldn't have even posted here.)

(2) the fully-canonicalized path that python-for-android calls ("/opt/android-sdk-linux/tools/ant") just happens to correspond to that of a directory for ant in the android SDK as extracted (r24.4.1), so there is indeed an "ant" object at that location, it's just that it's an ant directory, not the expected ant binary executable. Weird to see this coincidence (?), and not sure what to make of it.

In examining why "sh.ant" canonicalizes ant to "/opt/android-sdk-linux/tools/ant" as above, let's consider the output on my system from a few key commands:

root@laptop:# whereis ant
ant: /usr/bin/ant /usr/share/ant /usr/share/man/man1/ant.1.gz
root@laptop:# which ant
/usr/bin/ant
root@laptop:~# ls -l /usr/bin/ant /usr/share/ant
lrwxrwxrwx 1 root root 20 Oct 7 2014 /usr/bin/ant -> ../share/ant/bin/ant

/usr/share/ant:
total 12
drwxr-xr-x 2 root root 4096 Mar 7 21:02 bin
drwxr-xr-x 3 root root 4096 Mar 7 21:02 etc
drwxr-xr-x 2 root root 4096 Mar 7 21:02 lib
root@laptop:# ls -l /usr/bin/ant
lrwxrwxrwx 1 root root 20 Oct 7 2014 /usr/bin/ant -> ../share/ant/bin/ant
root@laptop:# whereis -l
bin: /usr/bin
bin: /usr/sbin
bin: /usr/lib
bin: /bin
bin: /sbin
bin: /etc
bin: /lib
bin: /lib64
bin: /usr/games
bin: /usr/local/bin
bin: /usr/local/sbin
bin: /usr/local/etc
bin: /usr/local/lib
bin: /usr/local/games
bin: /usr/include
bin: /usr/local
bin: /usr/share
man: /usr/share/man/fr
man: /usr/share/man/man7
man: /usr/share/man/es
man: /usr/share/man/sr
man: /usr/share/man/pl
man: /usr/share/man/sl
man: /usr/share/man/cs
man: /usr/share/man/lt
man: /usr/share/man/gl
man: /usr/share/man/man2
man: /usr/share/man/man5
man: /usr/share/man/da
man: /usr/share/man/man6
man: /usr/share/man/ar
man: /usr/share/man/man3
man: /usr/share/man/uk
man: /usr/share/man/ug
man: /usr/share/man/id
man: /usr/share/man/hu
man: /usr/share/man/sk
man: /usr/share/man/zh_TW
man: /usr/share/man/ru
man: /usr/share/man/ko
man: /usr/share/man/el
man: /usr/share/man/de
man: /usr/share/man/man4
man: /usr/share/man/ja
man: /usr/share/man/pt
man: /usr/share/man/sv
man: /usr/share/man/fi
man: /usr/share/man/tr
man: /usr/share/man/zh_CN
man: /usr/share/man/nl
man: /usr/share/man/it
man: /usr/share/man/ca
man: /usr/share/man/man8
man: /usr/share/man/man1
man: /usr/share/man/pt_BR
root@laptop:~#

It's interesting to note that "whereis ant" does not actually find the "/opt/android-sdk-linux/tools/ant" command invocation, as identified by "sh.ant" in p4a. Instead, it correctly identifies "/usr/bin/ant" (among others). Further, it's apparent that whereis does not even look in the "/opt" directory (see "whereis -l" above), so "sh.ant" must be getting that from somewhere else. One possibility to consider is that you'll note (from my previous message here) that I use the "--sdk" command parameter to pass the location of the Android SDK to p4a, rather than using an environmental variable; according to p4a docs, that's fully supported, though.

Incidentally, I've tried this as both root, and as a normal user, just to see if it mattered (it did not, as I'd suspected).

Thanks for your thoughts on a bug fix.

[kived]

This is an old bug in the sh package. You need to upgrade that package.

amoffat/sh@d42573c

[kived]

Specifically version 1.10 or greater. setup.py now requires this version.

5ed0ee8

[bidafiwoc]

Thanks so much --- this fixes that particular problem. I had assumed (erroneously, of course) there was no specific version dependency on python-sh, as I noted, conversely, the colorama version dependency as ">0.3" in "install_requires" in "setup.py". I see that your recent commit (5ed0ee8) adds the ">1.10" dependency on "sh" --- thanks.