Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chained async tasks result in error in celery worker #447

Open
Xarthisius opened this issue Feb 8, 2021 · 0 comments
Open

Chained async tasks result in error in celery worker #447

Xarthisius opened this issue Feb 8, 2021 · 0 comments

Comments

@Xarthisius
Copy link
Collaborator

Steps to reproduce

  1. Try to launch any instance

Expected result

Everything runs

Actual results

From celery_worker logs:

[2021-02-08 21:48:11,942: ERROR/ForkPoolWorker-2] Task gwvolman.tasks.build_tale_image[f1787146-44a3-472b-8c52-468dc565e189] raised unexpected: AttributeError("'_IDProxy' object has no attribute 'apply_async'")
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/kombu/utils/objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
KeyError: '_apply_async'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/celery/app/trace.py", line 474, in trace_task
    _chsig.apply_async(
  File "/usr/local/lib/python3.8/dist-packages/celery/canvas.py", line 224, in apply_async
    _apply = self._apply_async
  File "/usr/local/lib/python3.8/dist-packages/kombu/utils/objects.py", line 44, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "/usr/local/lib/python3.8/dist-packages/celery/canvas.py", line 516, in _apply_async
    return self.type.apply_async
AttributeError: '_IDProxy' object has no attribute 'apply_async'

Notes

It's caused by a backward incompatible release of jsonpickle-1.5.1 that we happen to hit during wholetale/girder image build. Both jsonpickle-1.5.0 and jsonpickle-1.5.2 work just fine. (See jsonpickle/jsonpickle#351)

For now, let's pin it to 1.5.2.

TODO

Create a reduced test case and figure out where does it belong upstream.

Xarthisius added a commit that referenced this issue Feb 8, 2021
Xarthisius added a commit that referenced this issue Feb 9, 2021
Xarthisius added a commit to whole-tale/gwvolman that referenced this issue Mar 22, 2021
Xarthisius added a commit to whole-tale/gwvolman that referenced this issue Mar 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant