Google requiring API Target 26 in Aug/Nov 2018

[gorgonaut04]

Just wanted to put it on the devs’ radar. We’ve been limited to 19 for the last couple years, afaik, but kivy will need to support targeting 26 for new apps starting in August, and for app updates starting in November: https://android-developers.googleblog.com/2017/12/improving-app-security-and-performance.html?m=1

If there’s currently a straightforward way to target 21 or higher then I’ll stand corrected and be very interested in hearing how.

[Zen-CODE]

Looking into this now, when setting target API as as 26, it tries to download:

    Downloading http://dl.google.com/android/android-sdk_r26-linux.tgz

which fails as this file does not exist. The one below does, so that's why it works with sdk 23.

   Downloading http://dl.google.com/android/android-sdk_r23-linux.tgz

So google have altered the path of the downloads? It also appears they have changed the location of some files and deprecated the "./android" command which now says you should use the "bin/sdkmanager" tool instead to install sdk's. But I can't find any mention of where they have moved the sdks for downloading. It seems "buildozer/target/android.py " file would need to be changed to use these need tools?

[jb1123]

Google post mentioned at the beginning of this discussion reads:

In order to provide users with the best Android experience possible, the Google Play Console will require that apps target a recent API level:

• August 2018: New apps required to target API level 26 (Android 8.0) or higher.

• November 2018: Updates to existing apps required to target API level 26 or higher.
  

• 2019 onwards: Each year the targetSdkVersion requirement will advance. Within one year following each Android dessert release, new apps and app updates will need to target the corresponding API level or higher.
  

Existing apps that are not receiving updates are unaffected. Developers remain free to use a minSdkVersion of their choice, so there is no change to your ability to build apps for older Android versions. We encourage developers to provide backwards compatibility as far as reasonably possible. Future Android versions will also restrict apps that don't target a recent API level and adversely impact performance or security.

Current p4a documentation says ndk r7, SDK 14 - this is years behind current versions. I've tried to build app with SDK=26 and NDK 15c but it has failed. API 27 requires NDK 16, since its platform is not supported in 15c. I haven't tried to use it but I presume it will fail. I've notices p4a uses gcc, gcc was deprecated in NDK r11 and replaced by Clang. Android development team retired ant form SDK a year ago (see https://stackoverflow.com/questions/42912824/the-ant-folder-is-suddenly-missing-from-android-sdk-did-google-remove-it). P4a adopted gradle only last summer. It looks there is plenty to be done and not much time left to catch up. Fortunately Google made revamp easier: it only makes sense to use latest SDK/NdK while building and Android app...

[gorgonaut04]

This seems to be the biggest issue facing Kivy (at least for Android devs). Is there some hope that there will be a re-vamp to support SDK 26+? In August, just 3 months away, no one will be able to create a new Kivy app for Android, and in November, any existing apps will be stuck as-is until/unless this is addressed. I need to figure out what I'm doing for current and in-development apps asap. Do we need a bounty?

[inclement]

Unless someone steps in to do the work (which would be welcome, and I can give some guidance), this is waiting on me having some serious time to work on p4a. I hope to get to it soon enough, but I'm not going to make any guarantees right now.

Edit: The hard part as far as I'm concerned is moving to clang. This probably shouldn't actually be a big issue, but I think it's the most unknown quantity involved.

[Zen-CODE]

@inclement. I would be happy to help out here, but it's not an area I am familiar with. I was going to take the following approach:

a. Get a VM building against API 27 on an 18.04 Mate Ubuntu OS using AndroidStudio for the SDK installation.
b. Once that is working, clean up, snapshot that and make it publically available for those who need it.
c. Dig into the the python-for-android and buildozer parts that need to automate this process

I think it's urgent to get a working build VM so the deadline is not a do-or-die. The full solution can then follow in time?

edit: I should probably try as use the commandline tools to install the sdk's, as we will probably need to use those via buildozer for the final solution?

[gorgonaut04]

Sounds great to me!

…

On Fri, May 11, 2018 at 5:31 AM Richard Larkin ***@***.***> wrote: @inclement <https://github.com/inclement>. I would be happy to help out here, but it's not an area I am familiar with. I was going to take the following approach: a. Get a VM building against API 27 on an 18.04 Mate Ubuntu OS using AndroidStudio for the SDK installation. b. Once that is working, clean up, snapshot that and make it publically available for those who need it. c. Dig into the the python-for-android and buildozer parts that need to automate this process I think it's urgent to get a working build VM so the deadline is not a do-or-die. The full solution can then follow in time? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1219 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKTLxybbp6nWO0_nTmyvmsk1o0HArVaLks5txYScgaJpZM4RZ5U5> .

[inclement]

From my point of view, this issue is not with creating a VM but with changing p4a's code to support the latest NDK and SDK. The important steps (which can be done semi-independently) are:

1. Make sure p4a works with API 27, using an up to date Android SDK. I haven't tried this at all, but it shouldn't be too hard, even if it requires a few small changes.

2. Make p4a work with the current NDK. This means making it use clang instead of gcc, as gcc is no longer available. I think this is already working with jni recipes (like sdl2), but the other recipe types use gcc explicitly. In principle this shouldn't be too hard, the process for switching to clang is fairly clear (I've tried it briefly before), but messing with the compiler toolchain inevitably brings up issues to fix and I can't predict how much work will be needed. It could also be possible that using clang would have bigger problems to solve but I hopefully not.

[Zen-CODE]

Yes, the VM is my thing for a few reasons. I wanted to (a) check building works on 18.04, (b) be able to experiment with safe snapshots and (c) be able to distribute a working build VM.

But that need not concern you. I will post here when I hit issues and need your assistance. If you need to make changes to p4a, you could perhaps do that on a branch that I can pull and test on. I can also push there with any changes I need to make. I will try with sdk = 27 in the buildozer.spec. You mention a different NDK? What should that be?

Lastly,with regards to gcc vs. clang, don't know much but am willing to learn if you can point me to some docs on changes that will need to be made, which version I need to install etc.

I may need a bit of baby sitting getting started, but will document my learning so this whole process is a bit less "black box" like. I think many people would like to help, but it's not at all obvious how to on a project this size. And it's a bit daunting getting started when no docs exists. I know you do tons, and would like to help relieve you, but how to do that is the question...:-)

[inclement]

You mention a different NDK? What should that be?

I just mean the latest NDK release from https://developer.android.com/ndk/downloads/ , currently r17. The main change is likely to be the need to move to clang, I don't expect anything else in particular to break.

Lastly,with regards to gcc vs. clang, don't know much but am willing to learn if you can point me to some docs on changes that will need to be made, which version I need to install etc.

On p4a's side, it currently chooses a gcc toolchain to use, and uses the path to that later in the build process. It does this here. I think this needs to be changed to look for the clang toolchain instead.

Beyond that, I would basically proceed by trying to change it and seeing what breaks. I don't have a deep idea about what will need doing. There is some clang documentation at https://developer.android.com/ndk/guides/standalone_toolchain#working_with_clang , and I've found before that since many projects have had to make the same change there are plenty of questions already asked online about the problems that come up.

Just to note, while this does need doing, NDK r16 should actually work currently using gcc. I don't think this is going to break in the near future, unlike targeting older apis, although moving to clang is obviously still important.

[Zen-CODE]

Okay, so I've got the API-27 SDK installed and unzipped the latest NDK. Working from buildozer master.

# Android SDK found at /home/kivy/Android/Sdk
# Android NDK found at /home/kivy/Android/android-ndk-r16b

When I run 'buidozer android debug', I get.
https://gist.github.com/Zen-CODE/9f982e93f437276b9968e1160061a26f

I am using the default-jre/default-jdk with ubuntu 18.04, which seems to be Java 11. Should I be using another java version? Or how to fix?

[inclement]

This error results from using a recent SDK with older versions of python-for-android, but it should be okay if you use p4a master. Did you set the p4a.branch = master option in buildozer.spec? (I think that's the right option name, it should already be there but commented out and set to the default value of stable).

[Zen-CODE]

Okay, I have done a git clone of https://github.xom/kivy/python-for-android into /home/kivy/Repos. So the clone lies in /home/kivy/Repos/python-for-android.
In the buildozer.spec file, I have set

p4a.source_dir = /home/kivy/Repos/python-for-android

and I get

# Apache ANT found at /home/kivy/.buildozer/android/platform/apache-ant-1.9.4
# Android SDK found at /home/kivy/Android/Sdk
# Android NDK found at /home/kivy/Android/android-ndk-r16b
# Run '/home/kivy/Android/Sdk/tools/android list sdk -u -e'
# Cwd /home/kivy/.buildozer/android/platform
*************************************************************************
The "android" command is deprecated.
For manual SDK, AVD, and project management, please use Android Studio.
For command-line tools, use tools/bin/sdkmanager and tools/bin/avdmanager
*************************************************************************
"android" SDK commands can be translated to sdkmanager commands on a best-effort basis.
Continue? (This prompt can be suppressed with the --use-sdk-wrapper command-line argument
or by setting the USE_SDK_WRAPPER environment variable) [y/N]: y
Running /home/kivy/Android/Sdk/tools/bin/sdkmanager --list --verbose

Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema
	at com.android.repository.api.SchemaModule$SchemaModuleVersion.<init>(SchemaModule.java:156)
	at com.android.repository.api.SchemaModule.<init>(SchemaModule.java:75)
	at com.android.sdklib.repository.AndroidSdkHandler.<clinit>(AndroidSdkHandler.java:81)
	at com.android.sdklib.tool.sdkmanager.SdkManagerCli.main(SdkManagerCli.java:73)
	at com.android.sdklib.tool.sdkmanager.SdkManagerCli.main(SdkManagerCli.java:48)
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.annotation.XmlSchema
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499)
	... 5 more
# Command failed: /home/kivy/Android/Sdk/tools/android list sdk -u -e
# 
# Buildozer failed to execute the last command
# The error might be hidden in the log above this error
# Please read the full log, and search for it before
# raising an issue with buildozer itself.
# In case of a bug report, please add a full log with log_level = 2
kivy@kivy-VirtualBox:~/Repos/kivy/examples/demo/touchtracer$ 

edit: master is definitely checked out in the python-for-android clone.

[inclement]

Ah, my mistake, the actual issue is that buildozer itself is not compatible with newer SDKs. It's still good that you're set up with p4a master, your clone method is what I'd recommend for working on this, but it isn't the immediate problem

The issue is that, as outlined by the message, newer SDKs replaced the android binary with sdkmanager. That means buildozer's attempt to query the sdk list here needs to be changed to use sdkmanager, or ideally to use sdkmanager or android depending on what's available.

The correct command is something like sdkmanager --list, as has automatically been guessed in your log, but in that case it looks like some java error has occurred. Does running /home/kivy/Android/Sdk/tools/bin/sdkmanager --list --verbose work if you do it directly?

[Zen-CODE]

Running "./android list sdk" gives the same error as does the sdkmanager. So I downgraded to openjdk-8 and it sorted that issue. Now I get a venv issue?

https://gist.github.com/Zen-CODE/d2210ec72984eebd48f286d9fc96b315

[inclement]

Do you have virtualenv installed?

[Zen-CODE]

kivy@kivy-VirtualBox:~/Repos/kivy/examples/demo/touchtracer$ pip install virtualenv
Collecting virtualenv
  Using cached https://files.pythonhosted.org/packages/b6/30/96a02b2287098b23b875bc8c2f58071c35d2efe84f747b64d523721dc2b5/virtualenv-16.0.0-py2.py3-none-any.whl
Installing collected packages: virtualenv
Successfully installed virtualenv-16.0.0

But still get same error.

[inclement]

Is the virtualenv executable in your $PATH, so you get some answer from which virtualenv? That's the method p4a uses to find it, I'm not sure what would cause it to not work here.

[Zen-CODE]

Okay, progress. I now get

Error compiling Cython file:
------------------------------------------------------------
...
    cdef:
        bcm.DISPMANX_UPDATE_HANDLE_T hdl
        bcm.UpdateHandle U
    hdl = bcm.vc_dispmanx_update_start( priority )
    if hdl == 0:
        raise bcm.BCMDisplayException("Couldn't open handle to update-start")
                ^
------------------------------------------------------------

kivy/lib/vidcore_lite/egl.pyx:619:17: cimported module has no attribute 'BCMDisplayException'


  STDERR:

# Command failed: /usr/bin/python -m pythonforandroid.toolchain create --dist_name=myapp --bootstrap=sdl2 --requirements=kivy --arch armeabi-v7a --copy-libs --color=always --storage-dir=/home/kivy/Repos/kivy/examples/demo/touchtracer/.buildozer/android/platform/build
# 
# Buildozer failed to execute the last command
# The error might be hidden in the log above this error
# Please read the full log, and search for it before
# raising an issue with buildozer itself.
# In case of a bug report, please add a full log with log_level = 2

Which I think is a cython version issue. Will downgrade cython and try again....

[inclement]

Rather than downgrading cython, could you make sure you're using kivy master? You can set kivy==master in the requirements, as normal.

[Zen-CODE]

Still get the same error. It did pull master now. cython version is 28.3. Perhaps a clean or distclean?

[inclement]

Might be worth it. I do recognise this error as a cython problem that was fixed at some point, so some cython version should definitely work.

[Zen-CODE]

clean fixed that. Now getting to java errors?
https://gist.github.com/Zen-CODE/36db7280eba0d98207241b10c18f10d4

[inclement]

Ah, this is good, it looks like the build is almost completed at the point it fails.

This error is new to me, though. It looks like some kind of ssl problem when downloading gradle. Do you get the same message (without the p4a stuff) if you navigate to home/kivy/Repos/kivy/examples/demo/touchtracer/.buildozer/android/platform/build/dists/myapp/ and run ./gradlew assembleDebug?

[Zen-CODE]

Yes. https://gist.github.com/Zen-CODE/e9624b4f8eb27635e9472a5525bade3a

[Zen-CODE]

Downloading gradle now. Fixed via the 18.04 tip here: https://stackoverflow.com/questions/6784463/error-trustanchors-parameter-must-be-non-empty

[inclement]

Oh, nice find, I've never seen anything like it before.

[Zen-CODE]

Built!!! :-)

# Android packaging done!
# APK myapp-0.1-debug.apk available in the bin directory

Will test shortly

[gorgonaut04]

Thanks for working on this guys! Hopefully there's a recap or updated build process down the line, but I'm really looking forward to upgrading my apps to the latest sdks. I haven't had to deal with runtime permissions requests on Android yet, but I'm guessing we'll need to update (for instance) the plyer gps for android to handle this permission request like it already does on iOS :-P

…

On Fri, May 18, 2018 at 3:07 PM Richard Larkin ***@***.***> wrote: Built!!! :-) # Android packaging done! # APK myapp-0.1-debug.apk available in the bin directory Will test shortly — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1219 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKTLx5AJSEuhWzSt0Dm_ZjMVaDHrn1RQks5tz0YXgaJpZM4RZ5U5> .

[Zen-CODE]

The APK installs but the app crashes. Will launch with buildozer and post the logs....
@gorgonaut04. My pleasure. I see my submission to the iStore today also warned that we need to build using the iOS 11 SDK. It all keeps moving! :-)

We identified one or more issues with a recent delivery for your app, "CAMI-Apps". Your delivery was successful, but you may wish to correct the following issues in your next delivery:
SDK Version Issue - In July 2018, iOS app updates submitted to the App Store will need to be built with the iOS '11.0' SDK or later, included in Xcode [9.0] or later. Make sure to update Xcode for future app deliveries.

[clevermindgames]

@OptimusGREEN getting this folder through jnius is slow, but you only need to do it once. Just store the results in some StringProperty and use it everywhere.
Google labeled those permissions dangerous for a reason, and unless it is really required for the app it should be avoided. I used it to be able to share saved data between paid and free version of my App, now learning to live without it.
There is also a benefit in using private storage for the app: when user backs up his phone's data, his SDcard content is not saved in a backup.

[Zen-CODE]

Okay, I can confirm the build APK per StaceyMAI's procedure runs without the ftello error AND is accepted by google. Yay. Thanks all.

@inclement. You mentioned a config file that incorrectly stores the previous config. Which one? If you give me some pointers, I can try to prepare a PR to fix that?

[inclement]

project.properties is the file.

[tkuam]

@clevermindgames I modified build.tmpl.gradle as you suggested and that seems to solve the need to find play-services-ads as my log shows it's been downloaded during the build, but now I've got a bunch of these errors for what seems like all of the various android widgets:

/home/tkuam/Desktop/wcpy2/.buildozer/android/platform/build/dists/workoutapp/build/intermediates/res/merged/debug/values-v26/values-v26.xml:14: error: Error: No resource found that matches the given name: attr 'android:touchscreenBlocksFocus'.

Not sure if it's related to the kivmob/admob changes or not, but I don't get these errors and successfully build when I don't make those changes.

Edit: Been poking around a bit, when I go to the directory and open up the .xml file(s) indicated in the error messages I'm getting, I'm finding the line that the error is referring to: <item name="android:touchscreenBlocksFocus">true</item> and I see that everything has to do with AppCompat (as in gradle_dependencies = 'com.android.support:appcompat-v7:26.1.0' from the buildozer.spec file perhaps?). Not sure entirely what I'm looking at in these files in my limited programming experience but maybe that gradle dependency is not working as it should?

[clevermindgames]

@tkuam this error basically means your build tools are incompatible with com.android.support:appcompat version. I will write up a more detailed guide how to get it to work with all the moving pieces put together in the next couple of days

[tkuam]

@clevermindgames okay yeah that's the conclusion I came upon after doing some googling. Thanks for working on a detailed guide. In the meantime I'll see if I can't mess around with the build tools version and get it working.

[clevermindgames]

@tkuam Here is the guide: http://quadropoly.com.au/kivy-and-admob-for-android-api-27/
Please let me know if it works for you. Let's discuss it there in the comments to stop hijacking this conversation :)
Update: I have simplified the integration and added support for Admob Rewarded Videos if anyone is interested: https://quadropoly.com.au/kivy-and-admob-for-android-guide-part2/

[Petar-Luketina]

Hello, @StacyMAI! I was reading through these comments and I noticed you mention "aeabi_memclr8"

When I try to build my app, it compiles the debug APK but crashes at start. I read the logcat file and it came back with: cannot locate symbol "__aeabi_memclr8"

Would you, or anyone else, know how to fix this issue? Thank you!

[inclement]

@Petar-Luketina I think this issue is likely to be caused by compiling against an api level higher than your device supports. There isn't currently a good way around this, it's a bug in p4a, although there are workarounds (I think discussed earlier in this thread).

[Petar-Luketina]

That's better news! Thanks @inclement . Would the app work if I uploaded it on to the app store, then downloaded it on the same phone from there?

UPDATE:
@inclement Hey, so I tried to upload it on the app store and there are many errors that came back from the testers. I also tried to upload it on a newer phone, and the app still crashes.

Here is the error logs from Google:

Issue: java.lang.UnsatisfiedLinkError: No implementation found for void org.libsdl.app.SDLActivity.nativeSetEnv(java.lang.String, java.lang.String) (tried Java_org_libsdl_app_SDLActivity_nativeSetEnv and Java_org_libsdl_app_SDLActivity_nativeSetEnv__Ljava_lang_String_2Ljava_lang_String_2)

Any insight?

[Malakanorth]

@Petar-Luketina have you tried this ?
In first build with API 19, when it's good remove your dist from .buildozer/android/platforms/build/dists and re-build with API 27.

[Petar-Luketina]

Ah, I skimmed over the part that said to make the 19 API then delete the dists and re-build to API 27, my mistake.

It works fine on debug mode, but it's crashing when I download it from the app store. I'm going to keep working at the problem. Maybe I'm missing something else.

@StacyMAI , if I may suggest an edit though. Your above post says to change 1.5 to 1.7:

<property name="java.target" value="7" />
<property name="java.source" value="1.7" />

But I had to change both to 1.8 because that's my version of Java (that's what that number is, correct?). Perhaps suggesting either one of those two may lead the next person to less confusion.

[noembryo]

@Zen-CODE Any news from that VM plan of yours? :o)
I'm in widows and I rely on the Kivy's android VM (v2.0) to build apks for my apps.
Is the building process still more complicated that the old one, so it's not possible with a VM?

[Zen-CODE]

@noembryon. There is not much to more to add to it at this point. It builds for API 27 with the "API 19, clear dists, build for 27" process. I'm working on fixing that but blocked by my python-for-android service dying on imports. That's also really import to get working for us, so I'm trying to sort that out before releasing a new VM...

[noembryo]

@Zen-CODE Yes, I understand.
Thank you for your (and the other guys) work on this.
It is greatly appreciated..

[atnccetti]

I created a solution that at the moment makes inclusion possible, it is extremely beta, but it already works for me.

https://github.com/atnccetti/launcherKivyAndroid

[Zen-CODE]

@atnccetti. Thanks. There are a few issues around python services and pyjnius that I want to resolve before redoing the VM. When I get there, I'll definitely look at this. Thanks :-)

[AndreMiras]

Should we close that one since #1412 was merged?

[noembryo]

Does that mean that there will be a new Virtual Image for us poor windows users? ;o)

[Zen-CODE]

For my 2c, there are more issue around this than the python3 recipe. Python2? And until there is a buildozer.spec file that can build a python2/python3 APK from scratch, I think it best to keep it open....

[RezVIKONG]

Hi guys :)
in the last month i built up from scratch an Ubuntu 18.10 x64 VM image with VirtualBox 5.2.22 especially for Python-for-Android builds ;)
@inclement i want this project to succeed and i will try my best to play as a Beta-Tester (I'm not a pro developer, I'm just a 1st-2nd level Desktop Support & Service Desk IT technician guy...)

So here below the VM specs (full details in attachments):

• Ubuntu 18.10 x64 on VirtualBox 5.2.22 with GuestAddons 5.2.22 ready (FULLY updated as of 7 Dec)
• GCC 8.2 & Clang 7
• Python-for-Android Master Branch (updated as of 7 Dec)
• Android SDK CLT 26.1.1 & Android NDK r16b + r18b
• OpenJDK 8

As you can see from the r16b build log, i was able to make the APK (API 26, ARM64, Python3) but unfortunately my Oreo phone doesn't even let install the package (error during package analysis).
I have to check more deeply the build logs...but for now i think that it's a step in the right direction :)
BuildLog_NDKr16b_API26_ARM64.log
BuildLog_NDKr18b_API26_ARM64.log
SdkManager_list.txt
PIP&PIP3_List.txt
Ubuntu_APT_List.txt

[Zen-CODE]

@RezVIKONG. The build succeeds it seems, so the logs you've posted I don't really help us (as far as I can tell anyway). If deployment to the phone fails, we need to see why. That can (hopefully) be done by examining the output of

buildozer android debug deploy run logcat

whilst your device is connected. The output of the logcat command should indicate why the deployment fails.

Also, I would suggest continuing this discussion by posting on the kivy user group. This ticket is long and ready for closing, as support for API 26+ has been achieved.

[alensiljak]

Any references to "support for API 26+ has been achieved", please?

I am currently unable to build a webview bootstrap with python3 and flask, using target api 27.
Also the suggestion to "build for 19" does not work since python3 recipe requires api 21+.

Thanks for all the work so far.

[inclement]

@mistery p4a now properly supports different build versions in a way it didn't before, but only with the sdl2 bootstrap. The work @Jonast has been doing to unify the bootstraps has moved towards fixing this in the other ones as well (especially webview), but I don't think this particular issue is covered yet. There are a few other outstanding build target problems that I hope to look at together over Christmas.

Also the suggestion to "build for 19" does not work since python3 recipe requires api 21+.

You can just substitute 21 for 19 and it should work the same way. However, the webview bootstrap might have issues right now with the master branch so you may need to make sure your p4a is not quite up to date.

[alensiljak]

@inclement, thanks a lot for the tips. I'm creating a Discord account for further discussion. I'm really hoping to get at least something going and would be looking to help as much as I can.

[inclement]

@mistery For what it's worth, it shouldn't be very hard to make it work.