"to_python" function not working in class CredentialsField with python3

[chanpete]

I am a newbie with python.
Recently I was trying to build a small web app with Google API in Django and Python 3.4.

And I use the storage utilities from oauth2client.django_orm

from oauth2client.django_orm import Storage
...
storage = Storage(CredentialsModel, 'id', user, 'credential')
...
storage.put(credential)
...
credential = storage.get()
...

The storage.put() function work fine but when I use the storage.get() function, it just give me a string but not a credentials object.
After testing and reading the source code of oauth2client.django_orm.
I found the problem was the "to_python" function doesn't work and just return the Base64 data back.

Finally I found the cause from the internet: "In python 3 the module-global __ metaclass__ variable is no longer supported."
And it has to change like this to work in Python 3

class CredentialsField(models.Field, metaclass=models.SubfieldBase):
...

[csssaz]

I just had the same issue, but when applying your solution I now get "Incorrect padding" when calling len(entities) inside the function locked_get inside oauth2client.django_orm.

Did you had the same problem?

[chanpete]

I am sorry, I don't have the same problem.
I am not sure what's happened to you. But I found that the "Incorrect padding" may be came from base64.
Others on the internet have the "Incorrect padding" error when base64 decoding.
"The length of any properly encoded base64 string should be divisible by 4."
Maybe you can try to find out the length of the saved data can be divided by 4 or not.

[willdady]

Just encountered this under Python 3.4.3 on Django 1.8.2. Is there a fix in the works for this?

[csssaz]

I couldn't make it work.

[vooft]

Here's the explanation of issue: https://docs.djangoproject.com/en/1.7/howto/custom-model-fields/#the-subfieldbase-metaclass

If you want your code to work on Python 2 & 3, you can use six.with_metaclass():

from django.utils.six import with_metaclass
class HandField(with_metaclass(models.SubfieldBase, models.Field)):
  ...

To fix this you should modify django_orm.py file according this explanation.

[Arlus]

I confirm vooft's fix works. However, you have to modify the oauth2client/django_orm.py source.

[tibbiyelininja]

@Arlus would you share your django_orm.py source please? I modified like this and now I have incorrect padding error.

from django.utils.six import with_metaclass
class CredentialsField(with_metaclass(models.SubfieldBase, models.Field)):

  def __init__(self, *args, **kwargs):

[tidushue]

My solution for python3, django 1.8 with postgres database:

There is a missing step right before saving the byte data to database, and after retrieving the data back from database: The byte data need to be converted to/from string first. You can convert byte to string and vice versa with decode("utf-8") and encode("utf-8").

You also do not need the __metaclass__, but need to have get_prep_value() and from_db_value() functions.
The entire class CredentialsField should be rewritten like so:

class CredentialsField(models.Field):
    def __init__(self, *args, **kwargs):
        if 'null' not in kwargs:
            kwargs['null'] = True
        super().__init__(*args, **kwargs)

    def get_internal_type(self):
        return "TextField"

    def to_python(self, value):
        if value is None:
            return None
        if isinstance(value, oauth2client.client.Credentials):
            return value

        value = value.encode("utf-8")  # string to byte

        return pickle.loads(base64.b64decode(value))

    def from_db_value(self, value, expression, connection, context):
        return self.to_python(value)

    def get_db_prep_value(self, value, connection, prepared=False):
        if value is None:
            return None

        byte_repr = base64.b64encode(pickle.dumps(value))

        return byte_repr.decode("utf-8")  # byte to string

    def get_prep_value(self, value):
        return self.get_db_prep_value(value)

[dhermes]

Fixed in #316. Please re-open if this is still broken.