Workers no longer start, error in pubsub

[kardamom]

As of this morning, when I gcloud app deploy, the workers no longer start.

I'm using Google AppEngine, Python Flexible Environment

[2017-08-24 19:41:31 +0000] [9] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/env/lib/python3.5/site-packages/gunicorn/arbiter.py", line 578, in spawn_worker
worker.init_process()
File "/env/lib/python3.5/site-packages/gunicorn/workers/base.py", line 126, in init_process
self.load_wsgi()
File "/env/lib/python3.5/site-packages/gunicorn/workers/base.py", line 135, in load_wsgi
self.wsgi = self.app.wsgi()
File "/env/lib/python3.5/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/env/lib/python3.5/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
return self.load_wsgiapp()
File "/env/lib/python3.5/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
return util.import_app(self.app_uri)
File "/env/lib/python3.5/site-packages/gunicorn/util.py", line 352, in import_app
import(module)
File "/home/vmagent/app/main.py", line 2, in
import episteme.factory
File "/home/vmagent/app/episteme/factory.py", line 25, in
import episteme.model
File "/home/vmagent/app/episteme/model/init.py", line 662, in
from . import user
File "/home/vmagent/app/episteme/model/user.py", line 563, in
from episteme.mail import EpistemeMailManager
File "/home/vmagent/app/episteme/mail/init.py", line 16, in
from google.cloud import pubsub
File "/env/lib/python3.5/site-packages/google/cloud/pubsub.py", line 17, in
from google.cloud.pubsub_v1 import PublisherClient
File "/env/lib/python3.5/site-packages/google/cloud/pubsub_v1/init.py", line 18, in
from google.cloud.pubsub_v1.publisher import Client as PublisherClient
File "/env/lib/python3.5/site-packages/google/cloud/pubsub_v1/publisher/init.py", line 17, in
from google.cloud.pubsub_v1.publisher.client import Client
File "/env/lib/python3.5/site-packages/google/cloud/pubsub_v1/publisher/client.py", line 27, in
from google.cloud.pubsub_v1.publisher.batch import thread
File "/env/lib/python3.5/site-packages/google/cloud/pubsub_v1/publisher/batch/thread.py", line 22, in
from google.cloud.pubsub_v1.publisher import exceptions
File "/env/lib/python3.5/site-packages/google/cloud/pubsub_v1/publisher/exceptions.py", line 19, in
from google.api.core.exceptions import GoogleAPICallError
ImportError: No module named 'google.api.core'

[lukesneeringer]

Confirmed. Investigating.

[lukesneeringer]

Hi @kardamom,
Thanks for reporting. This was a stupid packaging error on my part.
An 0.28.1 that fixes this is releasing right now, and should be on PyPI within a couple minutes.

[kardamom]

Hi @lukesneeringer Thanks for picking that up so quickly.

However, requiring google-cloud-core 0.27 breaks google-cloud-logging, which needs 0.26

make test (env: epweb)
pytest -s --color=yes ./tests
Traceback (most recent call last):
File "/Users/dave/.virtualenvs/epweb/lib/python3.6/site-packages/_pytest/config.py", line 342, in _getconftestmodules
return self._path2confmods[path]
KeyError: local('/Users/dave/Projects/episteme/web/tests')

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dave/.virtualenvs/epweb/lib/python3.6/site-packages/_pytest/config.py", line 373, in _importconftest
return self._conftestpath2mod[conftestpath]
KeyError: local('/Users/dave/Projects/episteme/web/tests/conftest.py')

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dave/.virtualenvs/epweb/lib/python3.6/site-packages/_pytest/config.py", line 379, in _importconftest
mod = conftestpath.pyimport()
File "/Users/dave/.virtualenvs/epweb/lib/python3.6/site-packages/py/_path/local.py", line 662, in pyimport
import(modname)
File "/Users/dave/.virtualenvs/epweb/lib/python3.6/site-packages/pytest/assertion/rewrite.py", line 212, in load_module
py.builtin.exec(co, mod.dict)
File "/Users/dave/Projects/episteme/web/tests/conftest.py", line 7, in
import episteme.factory
File "/Users/dave/Projects/episteme/web/episteme/factory.py", line 15, in
from google.cloud import logging as google_logging
File "/Users/dave/.virtualenvs/epweb/lib/python3.6/site-packages/google/cloud/logging/init.py", line 21, in
from google.cloud.logging.client import Client
File "/Users/dave/.virtualenvs/epweb/lib/python3.6/site-packages/google/cloud/logging/client.py", line 34, in
from google.cloud.logging._http import Connection
File "/Users/dave/.virtualenvs/epweb/lib/python3.6/site-packages/google/cloud/logging/_http.py", line 20, in
from google.cloud.iterator import HTTPIterator
ModuleNotFoundError: No module named 'google.cloud.iterator'
ERROR: could not load /Users/dave/Projects/episteme/web/tests/conftest.py

If I uninstall google-cloud-core then upgrade google-cloud-logging, I'm left with google-cloud-core 0.26, and back to the error I initially reported.

[dhermes]

@kardamom google-cloud-logging==1.3.0 was just pushed to PyPI

[kardamom]

Thanks @dhermes. picked up 1.3.0 and everything imports ok.

Alas, the API seems ahead of the docs: #3875

[lukesneeringer]

Thanks for picking that up so quickly.

Thanks for reporting it. That was a boneheaded (but easy to make) mistake on my part and I am glad it was not in the wild for long.

[ensonic]

0.27.0 is that last release with the old api. Maybe update the release notes and mention that it is broken or publish a 0.27.1 if easy. Also 0.26.0 seems to suffer from the import error. So which is the last release that works with the old api?
EDIT: different import, files #4122 - sorry for the noise.