Recipe for argon2-cffi #2398

Arjun-Somvanshi · 2021-01-03T13:42:16Z

What have I done so far?

In my code I have been using the pycryptodome lib which already has a recipe in p4a, since it seemed to have the same dependencies as argon2-cffi, that being: cffi and setuptools, I decided to use it's code for argon2's recipe. After I did the fork I added the dependency in my spec file. It is visible that it got downloaded (but probably is not getting built properly) but always fails when I run the app. This is the error I got on a logcat after the build was done:

      `working: Traceback (most recent call last):                                             �[0m
       working:   File "setup.py", line 13, in <module>                                        �[0m
       working:     from setuptools import find_packages, setup                                �[0m
       working: ModuleNotFoundError: No module named 'setuptools'                              �[0m

Command failed: /usr/bin/python3 -m pythonforandroid.toolchain create --dist_name=reminiscor --bootstrap=sdl2 --requirements=python3,kivy,pycryptodome,argon2-cffi --arch armeabi-v7a --copy-libs --color=always --storage-dir="/home/kivy/Reminiscor/.buildozer/android/platform/build-armeabi-v7a" --ndk-api=21

inclement · 2021-01-03T13:46:00Z

Try using a CompiledComponentsPythonRecipe - you'll need to do something like this to get the argon C components to build.

Arjun-Somvanshi · 2021-01-03T14:04:18Z

I tried to look into what is a CompiledComponentsPythonRecipe but this is what it says in the documentation:

Using a CompiledComponentsPythonRecipe

This is similar to a CythonRecipe but is intended for modules like numpy which include compiled but non-cython components. It uses a similar mechanism to compile with the right environment.

This isn’t documented yet because it will probably be changed so that CythonRecipe inherits from it (to avoid code duplication).

Arjun-Somvanshi · 2021-01-03T14:15:23Z

Is this link relevant to what you think I should look for : (https://askpythonquestions.com/2020/12/13/kivy-python-for-android-how-to-create-custom-recipes/)

obfusk · 2021-01-03T14:15:42Z

You may (also) need to set call_hostpython_via_targetpython = False; e.g.

from pythonforandroid.recipe import CompiledComponentsPythonRecipe


class Argon2Recipe(CompiledComponentsPythonRecipe):
    version = '20.1.0'
    url = 'https://github.com/hynek/argon2-cffi/archive/master.zip'
    depends = ['setuptools', 'cffi']
    call_hostpython_via_targetpython = False


recipe = Argon2Recipe()

obfusk · 2021-01-03T14:37:55Z

pythonforandroid/recipes/argon2-cffi/__init__.py

+    url = 'https://github.com/hynek/argon2-cffi/archive/master.zip'
+    depends = ['setuptools', 'cffi']
+    call_hostpython_via_targetpython = False
+


flake8 requires an extra blank line here.

it doesn't like the fact that the "blank line" contains whitespace.

Arjun-Somvanshi · 2021-01-03T15:36:28Z

From what I understand it's still not able to compile the c?

Arjun-Somvanshi · 2021-01-03T15:42:18Z

From what I understand it's still not able to compile the c?

It can't find the argon.h file do I need to include the c code somehow?

obfusk · 2021-01-03T15:46:06Z

It can't find the argon.h file do I need to include the c code somehow?

Looks like it uses a git submodule for the extras/libargon2 directory, which is not included in the .zip.

obfusk · 2021-01-03T15:50:44Z

Try using url = 'git+https://github.com/hynek/argon2-cffi@{version}'. I think that will include submodules.

Arjun-Somvanshi · 2021-01-03T15:50:47Z

It can't find the argon.h file do I need to include the c code somehow?

Looks like it uses a git submodule for the extras/libargon2 directory, which is not included in the .zip.

shall I make my own zip with the other repo included inside this one from: https://github.com/p-h-c/phc-winner-argon2/tree/62358ba2123abd17fccf2a108a301d4b52c01a7c

Arjun-Somvanshi · 2021-01-03T15:51:33Z

Try using url = 'git+https://github.com/hynek/argon2-cffi@{version}'.

okay, sorry I typed my thing before seeing your comment

obfusk · 2021-01-03T15:54:23Z

Try using url = 'git+https://github.com/hynek/argon2-cffi@{version}'. I think that will include submodules.

Actually, I think that should just be url = 'git+https://github.com/hynek/argon2-cffi'.

Normally, you should include {version} in the url, but the @{version} syntax for git urls is for pip, whereas p4a seems to use the recipe's .version instead (and check out that tag/branch).

obfusk · 2021-01-03T15:56:10Z

pythonforandroid/recipes/argon2-cffi/__init__.py

@@ -3,7 +3,7 @@

 class Argon2Recipe(CompiledComponentsPythonRecipe):
    version = '20.1.0'
-    url = 'https://github.com/hynek/argon2-cffi/archive/master.zip'
+    url = 'url = 'git+https://github.com/hynek/argon2-cffi@{version}'


that doesn't look quite right :)

this is my first pr ever, sorry if it's a bit shabby.

np. though you may want to do a rebase and squash all those intermediate commits when it's working.

yeah okay I will do that

obfusk · 2021-01-03T16:07:15Z

pythonforandroid/recipes/argon2-cffi/__init__.py

+
+class Argon2Recipe(CompiledComponentsPythonRecipe):
+    version = '20.1.0'
+    url = 'git+https://github.com/hynek/argon2-cffi@{version}'


I think you need to remove the @{version} here. Sorry.

np. I didn't see you edited that message.

Arjun-Somvanshi · 2021-01-03T17:26:17Z

Okay, I don't quite understand what the issue is now? It seemed like it almost passed it this time though

obfusk · 2021-01-03T19:03:50Z

I think the problem is that p4a calls setup_ext but not setup_clib.
Try adding build_cmd = 'build' to the recipe. That should call both.

Arjun-Somvanshi · 2021-01-04T13:58:53Z

You may (also) want to try building the recipe on your own computer instead.
With buildozer, you can use e.g.
p4a.branch = develop
p4a.local_recipes = ./my-p4a-recipes

I don't understand how the p4a.local_recipes works, this is what I have so far:

# (str) python-for-android fork to use, defaults to upstream (kivy)
p4a.fork = Arjun-Somvanshi

# (str) python-for-android branch to use, defaults to master
p4a.branch = develop

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#p4a.source_dir =

# (str) The directory in which python-for-android should look for your own build recipes (if any)
#p4a.local_recipes =

If I am using my own fork, then why does it need the local_recipes thing?

Arjun-Somvanshi · 2021-01-04T13:59:52Z

Incase you are wondering if I have done a build with those settings above in me .spec file, here is the output
build_output.txt

obfusk · 2021-01-04T14:09:40Z

If I am using my own fork, then why does it need the local_recipes thing?

In that case it doesn't :)
I'm using it so that I don't need to use a fork (and keep it up to date) just to have an extra recipe.

Arjun-Somvanshi · 2021-01-04T14:14:33Z

Incase you are wondering if I have done a build with those settings above in me .spec file, here is the output
build_output.txt

Oh okay, so if I use a local recipe do I still need to specify the package in the requirement field? Also what do you think of that build output I uploaded, is that some mistake on my part?

obfusk · 2021-01-04T20:42:08Z

This recipe seems to work:

from pythonforandroid.recipe import CompiledComponentsPythonRecipe


class Argon2Recipe(CompiledComponentsPythonRecipe):
    version = '20.1.0'
    url = 'git+https://github.com/hynek/argon2-cffi'
    depends = ['setuptools', 'cffi']
    call_hostpython_via_targetpython = False
    build_cmd = 'build'

    def get_recipe_env(self, arch):
        env = super().get_recipe_env(arch)
        env['ARGON2_CFFI_USE_SSE2'] = '0'
        return env


recipe = Argon2Recipe()

obfusk · 2021-01-04T21:49:22Z

You may run into #2400 when testing that locally.

AndreMiras

Thanks for the PR and the follow up changes with the help of @inclement and @obfusk
Looking good, will merge after the CI builds green

pythonforandroid/recipes/argon2-cffi/__init__.py

Arjun-Somvanshi · 2021-01-05T04:15:37Z

i'll do a rebase and squash the commits

Arjun-Somvanshi · 2021-01-05T06:03:17Z

I just added the change in the url

Arjun-Somvanshi · 2021-01-05T06:36:01Z

yeah, so as I thought, the libargon2 is missing hence we used that url from before, so this time I am gonna make the changes, squash the commits, and force push.

Arjun-Somvanshi · 2021-01-05T08:24:20Z

I am getting a lot of merge conflicts even though I followed the steps to a git rebase properly

AndreMiras · 2021-01-05T08:39:10Z

Thanks for giving it a try 👍
Weird for the rebase. I can't think of what would have made conflicts, but that's fine we'll try to deal without.
FYI what I would have tried is:

git fetch --all --prune
# squashing
git reset --soft HEAD~17
git commit --amend
# rebasing
git rebase upstream/develop # given the remote is called upstream

I'll try to at least squash from the GitHub interface at least because it allows to

Arjun-Somvanshi · 2021-01-05T08:58:38Z

Thanks for giving it a try 👍
Weird for the rebase. I can't think of what would have made conflicts, but that's fine we'll try to deal without.
FYI what I would have tried is:
git fetch --all --prune
# squashing
git reset --soft HEAD~17
git commit --amend
# rebasing
git rebase upstream/develop # given the remote is called upstream
I'll try to at least squash from the GitHub interface at least because it allows to
I'm sorry, I was trying to post the comment and closed the pr by mistake, the issue I think is that I mixed up the commits, I did a few directly from github instead of the command line

Arjun-Somvanshi · 2021-01-05T09:46:55Z

okay tests passed, no more commits now. Just gotta squash the commits from before.

Arjun-Somvanshi · 2021-01-05T09:56:20Z

@AndreMiras I tried the git commands you suggested the git log on my local repo gives the 17 commits even after running those commands

AndreMiras · 2021-01-05T11:44:03Z

No problem, thanks again for trying 👍

AndreMiras

All good thanks!

obfusk · 2021-01-05T15:37:40Z

git fetch --all --prune
# squashing
git reset --soft HEAD~17
git commit --amend
# rebasing
git rebase upstream/develop # given the remote is called upstream

Using an interactive rebase to squash always works for me:

git fetch --all
git rebase -i upstream/develop

obfusk · 2021-01-05T16:20:42Z

yeah, so as I thought, the libargon2 is missing hence we used that url from before, so this time I am gonna make the changes, squash the commits, and force push.

An alternative solution would be to add a separate recipe for libargon2 and depend on that.
This might be a good idea anyway, since that can then be updated separately (and shared by other recipes).

Arjun-Somvanshi · 2021-01-05T16:31:40Z

@obfusk just give me a few days, I am in the middle of my semester exams, I will get back to this and clean it up as well as try and do this the right way. :) Thanks a lot for your help.

obfusk · 2021-01-05T16:52:45Z

just give me a few days, I am in the middle of my semester exams

np. There's no hurry :) Good luck!

I will get back to this and clean it up

The current version is squashed and merged. So it's already fine :)

as well as try and do this the right way.

There's nothing "wrong" now. Using a separate libargon2 recipe would just be a nice improvement IMHO. And should be a separate PR.

* testing argon2-cffi * Using CompiledComponentsPythonRecipe * argon2-cffi recipe

Arjun-Somvanshi and others added 4 commits December 29, 2020 11:14

testing argon2-cffi

d5d8944

retrying argon2-cffi test

1f6c5f4

Update __init__.py

d3a2710

Update __init__.py

50de293

Arjun-Somvanshi added 2 commits January 3, 2021 19:51

Using CompiledComponentsPythonRecipe

06720ee

Using CompiledComponentsPythonRecipe

10e6bca

obfusk reviewed Jan 3, 2021

View reviewed changes

Arjun-Somvanshi added 4 commits January 3, 2021 20:09

Update __init__.py

e23be18

Update __init__.py

bc05d83

adding the extra lines

8fbb4cd

removing whitespace from line 10 (jeez)

a396f06

url change cause the extras of argon2 were excluded

1c2fb73

obfusk reviewed Jan 3, 2021

View reviewed changes

Update __init__.py

023dc26

obfusk reviewed Jan 3, 2021

View reviewed changes

remove @Version

981c01b

added build_cmd

ff3f43f

Update __init__.py

f5146fd

AndreMiras previously approved these changes Jan 5, 2021

View reviewed changes

pythonforandroid/recipes/argon2-cffi/__init__.py Show resolved Hide resolved

Recipe for argon2-cffi

5b37ba7

Arjun-Somvanshi dismissed AndreMiras’s stale review via 5b37ba7 January 5, 2021 06:02

Arjun-Somvanshi added 2 commits January 5, 2021 13:42

argon2-cffi recipe

bc4381b

argon2-cffi recipe final changes

9e0f0b9

Arjun-Somvanshi closed this Jan 5, 2021

Arjun-Somvanshi reopened this Jan 5, 2021

AndreMiras approved these changes Jan 5, 2021

View reviewed changes

AndreMiras merged commit aadcfc4 into kivy:develop Jan 5, 2021

vesellov pushed a commit to vesellov/python-for-android that referenced this pull request Feb 10, 2021

Recipe for argon2-cffi (kivy#2398)

1089fac

* testing argon2-cffi * Using CompiledComponentsPythonRecipe * argon2-cffi recipe

Recipe for argon2-cffi #2398

Recipe for argon2-cffi #2398

Conversation

Arjun-Somvanshi commented Jan 3, 2021

What have I done so far?

inclement commented Jan 3, 2021

Arjun-Somvanshi commented Jan 3, 2021

Arjun-Somvanshi commented Jan 3, 2021

obfusk commented Jan 3, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Arjun-Somvanshi commented Jan 3, 2021

Arjun-Somvanshi commented Jan 3, 2021

obfusk commented Jan 3, 2021

obfusk commented Jan 3, 2021 • edited Loading

Arjun-Somvanshi commented Jan 3, 2021

Arjun-Somvanshi commented Jan 3, 2021

obfusk commented Jan 3, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Arjun-Somvanshi Jan 3, 2021 • edited Loading

Choose a reason for hiding this comment

Arjun-Somvanshi commented Jan 3, 2021 • edited Loading

obfusk commented Jan 3, 2021

Arjun-Somvanshi commented Jan 4, 2021

Arjun-Somvanshi commented Jan 4, 2021 • edited Loading

obfusk commented Jan 4, 2021

Arjun-Somvanshi commented Jan 4, 2021 • edited Loading

obfusk commented Jan 4, 2021

obfusk commented Jan 4, 2021

AndreMiras left a comment

Choose a reason for hiding this comment

Arjun-Somvanshi commented Jan 5, 2021

Arjun-Somvanshi commented Jan 5, 2021

Arjun-Somvanshi commented Jan 5, 2021

Arjun-Somvanshi commented Jan 5, 2021

AndreMiras commented Jan 5, 2021

Arjun-Somvanshi commented Jan 5, 2021 • edited Loading

Arjun-Somvanshi commented Jan 5, 2021 • edited Loading

Arjun-Somvanshi commented Jan 5, 2021

AndreMiras commented Jan 5, 2021

AndreMiras left a comment

Choose a reason for hiding this comment

obfusk commented Jan 5, 2021

obfusk commented Jan 5, 2021

Arjun-Somvanshi commented Jan 5, 2021

obfusk commented Jan 5, 2021

obfusk commented Jan 3, 2021 •

edited

Loading

obfusk commented Jan 3, 2021 •

edited

Loading

obfusk commented Jan 3, 2021 •

edited

Loading

Arjun-Somvanshi Jan 3, 2021 •

edited

Loading

Arjun-Somvanshi commented Jan 3, 2021 •

edited

Loading

Arjun-Somvanshi commented Jan 4, 2021 •

edited

Loading

Arjun-Somvanshi commented Jan 4, 2021 •

edited

Loading

Arjun-Somvanshi commented Jan 5, 2021 •

edited

Loading

Arjun-Somvanshi commented Jan 5, 2021 •

edited

Loading