MatPlotLib not building with Buildozer (but it did 2 months ago)

[edwit1971]

I believe this is a BUG. Below is the output when I tried to build an APK with Buildozer. It was working fine just 2 months ago. I disabled MatPlotLib for the last few weeks and when I added it back suddenly I get an Error.

[INFO]: Prebuilding matplotlib for armeabi-v7a
[INFO]: Downloading jquery-ui for matplatlib web backend
[INFO]: Will download into /home/edwit3/Dev/WeightLoss/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/matplotlib/armeabi-v7a__ndk_target_21/matplotlib/p4a_files
[INFO]: Downloading matplotlib from https://jqueryui.com/resources/download/jquery-ui-1.12.1.zip
Traceback (most recent call last):
File "/home/edwit3/anaconda3/envs/myenv/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/edwit3/anaconda3/envs/myenv/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/edwit3/Dev/WeightLoss/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 1266, in
main()
File "/home/edwit3/Dev/WeightLoss/.buildozer/android/platform/python-for-android/pythonforandroid/entrypoints.py", line 18, in main
ToolchainCL()
File "/home/edwit3/Dev/WeightLoss/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 715, in init
getattr(self, command)(args)
File "/home/edwit3/Dev/WeightLoss/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 153, in wrapper_func
build_dist_from_args(ctx, dist, args)
File "/home/edwit3/Dev/WeightLoss/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 215, in build_dist_from_args
args, "ignore_setup_py", False
File "/home/edwit3/Dev/WeightLoss/.buildozer/android/platform/python-for-android/pythonforandroid/build.py", line 571, in build_recipes
recipe.prebuild_arch(arch)
File "/home/edwit3/Dev/WeightLoss/.buildozer/android/platform/python-for-android/pythonforandroid/recipes/matplotlib/init.py", line 104, in prebuild_arch
self.download_web_backend_dependencies(arch)
File "/home/edwit3/Dev/WeightLoss/.buildozer/android/platform/python-for-android/pythonforandroid/recipes/matplotlib/init.py", line 93, in download_web_backend_dependencies
self.download_file(url, target_file)
File "/home/edwit3/Dev/WeightLoss/.buildozer/android/platform/python-for-android/pythonforandroid/recipe.py", line 196, in download_file
urlretrieve(url, target, report_hook)
File "/home/edwit3/anaconda3/envs/myenv/lib/python3.7/urllib/request.py", line 247, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File "/home/edwit3/anaconda3/envs/myenv/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/home/edwit3/anaconda3/envs/myenv/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/home/edwit3/anaconda3/envs/myenv/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/home/edwit3/anaconda3/envs/myenv/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/home/edwit3/anaconda3/envs/myenv/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/home/edwit3/anaconda3/envs/myenv/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
Download failed retrying in a second...Download failed retrying in a second...Download failed retrying in a second...Download failed retrying in a second...# Command failed: /home/edwit3/anaconda3/envs/myenv/bin/python -m pythonforandroid.toolchain create --dist_name=weightloss --bootstrap=sdl2 --requirements=python3==3.7.6,hostpython3==3.7.6,sqlite3,matplotlib,android,kivy==1.11.1,git+https://github.com/HeaTTheatR/KivyMD.git,pygments --arch armeabi-v7a --copy-libs --color=always --storage-dir="/home/edwit3/Dev/WeightLoss/.buildozer/android/platform/build-armeabi-v7a" --ndk-api=21

Checklist

• the issue is indeed a bug and not a support request
• issue doesn't already exist: https://github.com/kivy/python-for-android/issues
• I have a short, runnable example that reproduces the issue
• I reproduced the problem with the latest development version (p4a.branch = develop)
• I used the grave accent (aka backticks) to format code or logs when appropriated

Versions

• Python:
• OS:
• Kivy:
• Cython:
• OpenJDK:

Description

I'm using buildozer to make an APK
buildozer -v android debug

and am getting an ERROR because I added MatPlotLib to the Spec file.

buildozer.spec

Command:

// REPLACE ME: buildozer command ran? e.g. buildozer android debug
// Keep the triple grave accent (aka backquote/backtick) to have the code formatted

Spec file:

[app]

# (str) Title of your application
title = WeightLoss

# (str) Package name
package.name = weightloss

# (str) Package domain (needed for android/ios packaging)
package.domain = com.handgems

# (str) Source code where the main.py live
source.dir = .

# (list) Source files to include (let empty to include all the files)
source.include_exts = py, png, atlas, db, ttf, md, json

# (list) List of inclusions using pattern matching
source.include_patterns = assets/*,images/*.png

# (list) Source files to exclude (let empty to not exclude anything)
source.exclude_exts = spec

# (list) List of directory to exclude (let empty to not exclude anything)
source.exclude_dirs = bin, __pycache__

# (list) List of exclusions using pattern matching
#source.exclude_patterns = license,images/*/*.jpg

# (str) Application versioning (method 1)
version = 1.0

# (str) Application versioning (method 2)
# version.regex = __version__ = ['"](.*)['"]
# version.filename = %(source.dir)s/main.py

# (list) Application requirements
# comma separated e.g. requirements = sqlite3,kivy
#requirements = python3==3.7.6, hostpython3==3.7.6, sqlite3, android, kivy==1.11.1, kivymd
requirements = python3==3.7.6, hostpython3==3.7.6, sqlite3, matplotlib, android, kivy==1.11.1, git+https://github.com/HeaTTheatR/KivyMD.git, pygments

# (str) Custom source folders for requirements
# Sets custom source for any requirements with recipes
requirements.source.kivymd = ../../kivymd

# (list) Garden requirements
#garden_requirements = matplotlib

# (str) Presplash of the application
presplash.filename = %(source.dir)s/images/presplash.png

# (str) Icon of the application
icon.filename = %(source.dir)s/images/icon.png

# (str) Supported orientation (one of landscape, sensorLandscape, portrait or all)
orientation = all

# (list) List of service to declare
#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY

#
# OSX Specific
#

#
# author = © Copyright Info

# change the major version of python used by the app
osx.python_version = 3

# Kivy version to use
osx.kivy_version = 1.9.1

#
# Android specific
#

# (bool) Indicate if the application should be fullscreen or not
fullscreen = True

# (string) Presplash background color (for new android toolchain)
# Supported formats are: #RRGGBB #AARRGGBB or one of the following names:
# red, blue, green, black, white, gray, cyan, magenta, yellow, lightgray,
# darkgray, grey, lightgrey, darkgrey, aqua, fuchsia, lime, maroon, navy,
# olive, purple, silver, teal.
#android.presplash_color = #FFFFFF

# (list) Permissions
android.permissions = WRITE_EXTERNAL_STORAGE

# (int) Target Android API, should be as high as possible.
android.api = 29

# (int) Minimum API your APK will support. 21 = Lollipop v5.0
android.minapi = 21

# (str) Android NDK version to use
android.ndk = 19c

# (bool) Use --private data storage (True) or --dir public storage (False)
#android.private_storage = False

# (str) Android NDK directory (if empty, it will be automatically downloaded.)
#android.ndk_path =

# (str) Android SDK directory (if empty, it will be automatically downloaded.)
#android.sdk_path =

# (str) ANT directory (if empty, it will be automatically downloaded.)
#android.ant_path =

# (bool) If True, then skip trying to update the Android sdk
# This can be useful to avoid excess Internet downloads or save time
# when an update is due and you just want to test/build your package
# android.skip_update = False

# (bool) If True, then automatically accept SDK license
# agreements. This is intended for automation only. If set to False,
# the default, you will be shown the license when first running
# buildozer.
# android.accept_sdk_license = False

# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity

# (str) Android app theme, default is ok for Kivy-based app
# android.apptheme = "@android:style/Theme.NoTitleBar"

# (list) Pattern to whitelist for the whole project
#android.whitelist =

# (str) Path to a custom whitelist file
#android.whitelist_src =

# (str) Path to a custom blacklist file
#android.blacklist_src =

# (list) List of Java .jar files to add to the libs so that pyjnius can access
# their classes. Don't add jars that you do not need, since extra jars can slow
# down the build process. Allows wildcards matching, for example:
# OUYA-ODK/libs/*.jar
#android.add_jars = foo.jar,bar.jar,path/to/more/*.jar

# (list) List of Java files to add to the android project (can be java or a
# directory containing the files)
#android.add_src =

# (list) Android AAR archives to add (currently works only with sdl2_gradle
# bootstrap)
#android.add_aars =

# (list) Gradle dependencies to add (currently works only with sdl2_gradle
# bootstrap)
#android.gradle_dependencies =

# (list) add java compile options
# this can for example be necessary when importing certain java libraries using the 'android.gradle_dependencies' option
# see https://developer.android.com/studio/write/java8-support for further information
# android.add_compile_options = "sourceCompatibility = 1.8", "targetCompatibility = 1.8"

# (list) Gradle repositories to add {can be necessary for some android.gradle_dependencies}
# please enclose in double quotes 
# e.g. android.gradle_repositories = "maven { url 'https://kotlin.bintray.com/ktor' }"
#android.add_gradle_repositories =

# (list) packaging options to add 
# see https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.PackagingOptions.html
# can be necessary to solve conflicts in gradle_dependencies
# please enclose in double quotes 
# e.g. android.add_packaging_options = "exclude 'META-INF/common.kotlin_module'", "exclude 'META-INF/*.kotlin_module'"
#android.add_gradle_repositories =

# (list) Java classes to add as activities to the manifest.
#android.add_activites = com.example.ExampleActivity

# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME

# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =

# (str) launchMode to set for the main activity
#android.manifest.launch_mode = standard

# (list) Android additional libraries to copy into libs/armeabi
#android.add_libs_armeabi = libs/android/*.so
#android.add_libs_armeabi_v7a = libs/android-v7/*.so
#android.add_libs_arm64_v8a = libs/android-v8/*.so
#android.add_libs_x86 = libs/android-x86/*.so
#android.add_libs_mips = libs/android-mips/*.so

# (bool) Indicate whether the screen should stay on
# Don't forget to add the WAKE_LOCK permission if you set this to True
#android.wakelock = False

# (list) Android application meta-data to set (key=value format)
#android.meta_data =

# (list) Android library project to add (will be added in the
# project.properties automatically.)
#android.library_references =

# (list) Android shared libraries which will be added to AndroidManifest.xml using <uses-library> tag
#android.uses_library =

# (str) Android logcat filters to use
android.logcat_filters = *:S python:D

# (bool) Copy library instead of making a libpymodules.so
#android.copy_libs = 1

# (str) The Android arch to build for, choices: armeabi-v7a, arm64-v8a, x86, x86_64
android.arch = armeabi-v7a

#
# Python for android (p4a) specific
#

# (str) python-for-android fork to use, defaults to upstream (kivy)
p4a.fork = edwit1971

# (str) python-for-android branch to use, defaults to master
p4a.branch = develop

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#p4a.source_dir =

# (str) The directory in which python-for-android should look for your own build recipes (if any)
#p4a.local_recipes = %(source.dir)s/Recipes

# (str) Filename to the hook for p4a
#p4a.hook =

# (str) Bootstrap to use for android builds
# p4a.bootstrap = sdl2

# (int) port number to specify an explicit --port= p4a argument (eg for bootstrap flask)
#p4a.port =


#
# iOS specific
#

# (str) Path to a custom kivy-ios folder
#ios.kivy_ios_dir = ../kivy-ios
# Alternately, specify the URL and branch of a git checkout:
ios.kivy_ios_url = https://github.com/kivy/kivy-ios
ios.kivy_ios_branch = master

# Another platform dependency: ios-deploy
# Uncomment to use a custom checkout
#ios.ios_deploy_dir = ../ios_deploy
# Or specify URL and branch
ios.ios_deploy_url = https://github.com/phonegap/ios-deploy
ios.ios_deploy_branch = 1.7.0

# (str) Name of the certificate to use for signing the debug version
# Get a list of available identities: buildozer ios list_identities
#ios.codesign.debug = "iPhone Developer: <lastname> <firstname> (<hexstring>)"

# (str) Name of the certificate to use for signing the release version
#ios.codesign.release = %(ios.codesign.debug)s


[buildozer]

# (int) Log level (0 = error only, 1 = info, 2 = debug (with command output))
log_level = 2

# (int) Display warning if buildozer is run as root (0 = False, 1 = True)
warn_on_root = 1

# (str) Path to build artifact storage, absolute or relative to spec file
# build_dir = ./.buildozer

# (str) Path to build output (i.e. .apk, .ipa) storage
# bin_dir = ./bin

#    -----------------------------------------------------------------------------
#    List as sections
#
#    You can define all the "list" as [section:key].
#    Each line will be considered as a option to the list.
#    Let's take [app] / source.exclude_patterns.
#    Instead of doing:
#
#[app]
#source.exclude_patterns = license,data/audio/*.wav,data/images/original/*
#
#    This can be translated into:
#
#[app:source.exclude_patterns]
#license
#data/audio/*.wav
#data/images/original/*
#


#    -----------------------------------------------------------------------------
#    Profiles
#
#    You can extend section / key with a profile
#    For example, you want to deploy a demo version of your application without
#    HD content. You could first change the title to add "(demo)" in the name
#    and extend the excluded directories to remove the HD content.
#
#[app@demo]
#title = My Application (demo)
#
#[app:source.exclude_patterns@demo]
#images/hd/*
#
#    Then, invoke the command line with the "demo" profile:
#
#buildozer --profile demo android debug

Logs

// REPLACE ME: Paste the build output containing the error
// Keep the triple grave accent (a.k.a. backquote/backtick) to have the code formatted

[RobertFlatt]

Could this be an example of #2178 ?
try sudo apt install libssl-dev
Now documented https://buildozer.readthedocs.io/en/latest/installation.html#android-on-ubuntu-20-04-64bit

[edwit1971]

I tried sudo pacman -S libssl-dev
but it said target not found: libssl-dev
I googled it and for Manjaro it told me to try openSSL so I did.
I typed sudo pacman -S openssl
and tried buildozer again but got the same error.

[tshirtman]

you might need to clean and rebuild
rm -rf .buildozer
buildozer android debug

[edwit1971]

Whenever I get stubborn Buildozer errors the first thing i always do is DELETE the hidden .buildozer folder and run buildozer again as a clean rebuild. I've done 2 clean rebuilds and this error still occurs.
Are other people able to build APKs with buildozer putting matplotlib in the requirements line of the Buildozer.spec file?
If i remove matplotlib from the Requirements line, it builds the APK without any errors but of course matplotlib code doesn't work on the android device.

[RobertFlatt]

matplotlib builds here (Ubuntu), so I suggest the issue is somthing about the configuration there.

A quick search shows
https://jqueryui.com/resources/download/jquery-ui-1.12.1.zip
is a dependancy of matplotlib.

Other sites presumably download ok, so the question is why does the system there get a "urllib.error.HTTPError: HTTP Error 403: Forbidden" from this one site ? To narrow the issue does your web browser download this file?

The ssl stuff changed so that was a possibility, even though the message was not explicitly ssl. I have no other suggestions.

[edwit1971]

Yes my web-browser downloaded the file fine, no problem... (I wonder if I can use the file I downloaded to build the APK)

https://jqueryui.com/resources/download/jquery-ui-1.12.1.zip

[edwit1971]

no solution...

I opened a better issue about this bug with code that is public and can be debugged by anyone with the ability to step through python-4-android or buildozer code to find why MatPlotLib is throwing a fatal error and not building.