Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: returnvalue not JSON.stringified in python job #2196

Closed
1 task done
klaemo opened this issue Sep 25, 2023 · 1 comment · Fixed by #2198
Closed
1 task done

[Bug]: returnvalue not JSON.stringified in python job #2196

klaemo opened this issue Sep 25, 2023 · 1 comment · Fixed by #2198
Labels
bug Something isn't working

Comments

@klaemo
Copy link
Contributor

klaemo commented Sep 25, 2023

Version

1.13.2

Platform

Python

What happened?

Writing a processor function that returns a plain string

async def process(job):
    return 'foo'

and listening to the queue events in JavaScript will throw an error when JavaScript tries to JSON.parse the return value:

SyntaxError: Unexpected token 'o', "foo" is not valid JSON
    at JSON.parse (<anonymous>)
    at QueueEvents.consumeEvents (/Users/clemens/Projects/bullmq-example/producer/node_modules/bullmq/dist/cjs/classes/queue-events.js:92:53)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async QueueEvents.run (/Users/clemens/Projects/bullmq-example/producer/node_modules/bullmq/dist/cjs/classes/queue-events.js:60:17)

Similarily, returning an array will also fail

async def process(job):
    return ['foo']
Error processing job Invalid input of type: 'list'. Convert to a bytes, string, int or float first.

I think the python code should probably use json.dumps(result) here

job.returnvalue = result

How to reproduce.

from bullmq import Worker

async def process(job):
    return 'foo'

async def main():
    # Feel free to remove the connection parameter, if your redis runs on localhost
    worker = Worker("myQueue", process, {"connection": "localhost"})

    # This while loop is just for the sake of this example
    # you won't need it in practice.
    while True: # Add some breaking conditions here
        await asyncio.sleep(1)

    # When no need to process more jobs we should close the worker
    await worker.close()

if __name__ == "__main__":
    asyncio.run(main())
const events = new QueueEvents("myQueue");

events.on("completed", (evt) => {
  console.log(evt);
});

Relevant log output

SyntaxError: Unexpected token 'o', "foo" is not valid JSON
    at JSON.parse (<anonymous>)
    at QueueEvents.consumeEvents (/Users/clemens/Projects/bullmq-example/producer/node_modules/bullmq/dist/cjs/classes/queue-events.js:92:53)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async QueueEvents.run (/Users/clemens/Projects/bullmq-example/producer/node_modules/bullmq/dist/cjs/classes/queue-events.js:60:17)

Code of Conduct

  • I agree to follow this project's Code of Conduct
@klaemo
Copy link
Contributor Author

klaemo commented Sep 26, 2023

Wow, thank you so much for the quick fix! 🤩 Can confirm that it works as intended now.

github-actions bot pushed a commit that referenced this issue Sep 29, 2023
# [4.12.0](v4.11.4...v4.12.0) (2023-09-29)

### Bug Fixes

* **move-to-finished:** stringify any return value [python] ([#2198](#2198)) fixes [#2196](#2196) ([07f1335](07f1335))

### Features

* expose addJobLog and updateJobProgress to the Queue instance ([#2202](#2202)) ([2056939](2056939))
* **queue:** add clean method [python] ([#2194](#2194)) ([3b67193](3b67193))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant