[Python 3] oauth exception (the JSON object must be str, not 'bytes') when using gcloud.storage

[changzhuo]

In https://github.com/google/oauth2client/blob/0aad0a09f50312e0fcc3a1234f0e7fb525010bfd/oauth2client/gce.py#L86, the code uses d = json.loads(content) to load content as JSON. The code is broken in Python 3 because content is binary, while json.loads expects string.

[dhermes]

Thanks for the report. Do you have some code you used to test this? I'm planning on doing:

>>> from oauth2client.gce import AppAssertionCredentials
>>> scope = u'https://www.googleapis.com/auth/userinfo.email'
>>> credentials = AppAssertionCredentials([scope])
>>> import httplib2
>>> http = httplib2.Http()
>>> credentials._refresh(http.request)
Traceback (most recent call last):
  File "/opt/python3.4/lib/python3.4/site-packages/oauth2client/gce.py", line 86, in _refresh
    d = json.loads(content)
  File "/opt/python3.4/lib/python3.4/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/python3.4/lib/python3.4/site-packages/oauth2client/gce.py", line 88, in _refresh
    raise AccessTokenRefreshError(str(e))
oauth2client.client.AccessTokenRefreshError: the JSON object must be str, not 'bytes'
>>> credentials.access_token

once I get an easy Python 3 set-up script in place for a new instance.

[dhermes]

To set up the instance:

$ sudo apt-get update
$ sudo apt-get -y upgrade
$ sudo apt-get install -y build-essential cmake
$ # http://binarynature.blogspot.com/2014/12/gns3-with-google-compute-engine.html
$ sudo apt-get install -y libncurses5-dev libncursesw5-dev libreadline6-dev
$ sudo apt-get install -y libdb5.1-dev libgdbm-dev libsqlite3-dev libssl-dev
$ sudo apt-get install -y libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev
$ cd /tmp
$ curl -L https://www.python.org/ftp/python/3.4.2/Python-3.4.2.tgz | tar -xz
$ cd Python-3.4.2/
$ ./configure --prefix=/opt/python3.4
$ make
$ sudo make install
$ # Now for installing our stuff
$ sudo /opt/python3.4/bin/pip3 install --upgrade pip
$ sudo /opt/python3.4/bin/pip3 install --upgrade oauth2client
$ /opt/python3.4/bin/python3.4

[dhermes]

If someone has time to send a fix (and test it with both bytes and unicode) it's as simple as:

$ git diff
diff --git a/oauth2client/gce.py b/oauth2client/gce.py
index fc3bd77..9a10d94 100644
--- a/oauth2client/gce.py
+++ b/oauth2client/gce.py
@@ -21,6 +21,7 @@ __author__ = '[email protected] (Joe Gregorio)'

 import json
 import logging
+import six
 from six.moves import urllib

 from oauth2client import util
@@ -81,6 +82,8 @@ class AppAssertionCredentials(AssertionCredentials):
     query = '?scope=%s' % urllib.parse.quote(self.scope, '')
     uri = META.replace('{?scope}', query)
     response, content = http_request(uri)
+    if not isinstance(content, six.text_type):
+      content = content.decode('utf-8')
     if response.status == 200:
       try:
         d = json.loads(content)

[czchen]

Looks good in my environment. Please send a PR and release a bug fixed version.

My environment are:

• Google Compute Engine (ubuntu 14.04)
• virtualenv with Python 3.4

with the following test code

import gcloud.storage

client = gcloud.storage.Client('this-is-project-name')
client.get_bucket('this-is-bucket-name')

The original oauth2client causes the following exception

oauth2client.client.AccessTokenRefreshError: the JSON object must be str, not 'bytes'

And the patched code does not.

[dhermes]

This was fixed in #152 but has never been merged because no tests