Crashed when using celery to execute Weasyprint tasks

[guojiaqi1027]

Hi:

I'm using Weasyprint to generate PDFs in celery tasks. And it crashed with trace logs below:

[2018-09-11 15:14:24,550: ERROR/MainProcess] Process 'ForkPoolWorker-2' pid:11224 exited with 'signal 6 (SIGABRT)'
[2018-09-11 15:14:24,567: ERROR/MainProcess] Task handler raised error: WorkerLostError('Worker exited prematurely: signal 6 (SIGABRT).',)
Traceback (most recent call last):
File "/Users/guo/Documents/dev/soda_admin/erp_server/env/lib/python3.6/site-packages/billiard/pool.py", line 1223, in mark_as_worker_lost
human_status(exitcode)),
billiard.exceptions.WorkerLostError: Worker exited prematurely: signal 6 (SIGABRT).

After investigation, I found the issue:

In my task code, I have imported a module which import WeasyPrint at the top of code, and when celery execute the task, it crashed because Celery execute WeasyPrint in another process, which cause the multi-processing or thread issues. So I imported WeasyPrint lib inside the method executed instead of the whole module.

Is this a thread-safety design for you, or it can be fixed since async execute for PDF generation is really common-used?

Thanks,

[liZe]

Thanks for reporting the issue.

It's probably the same problem as #167, and I honestly don't know how to fix it. Is it possible for you to get a stack trace? I suppose that the crash happens in C code (probably with Pango or Cairo).

Is this a thread-safety design for you, or it can be fixed since async execute for PDF generation is really common-used?

It's not designed to crash with multiple threads, but it's not designed to be thread-safe neither. Cairo is supposed to be thread-safe (see https://gitlab.freedesktop.org/cairo/cairo/issues/266) and I've been using WeasyPrint on multiprocess environments with uWSGI for years, so I suppose it somehow works.

File "/Users/guo/Documents/dev/soda_admin/erp_server/env/lib/python3.6/site-packages/billiard/pool.py"

Just as in #167, you're on macOS. Maybe there's something wrong with Cairo or Pango on macOS…

[guojiaqi1027]

Thanks for reporting the issue.

It's probably the same problem as #167, and I honestly don't know how to fix it. Is it possible for you to get a stack trace? I suppose that the crash happens in C code (probably with Pango or Cairo).

Is this a thread-safety design for you, or it can be fixed since async execute for PDF generation is really common-used?

It's not designed to crash with multiple threads, but it's not designed to be thread-safe neither. Cairo is supposed to be thread-safe (see https://gitlab.freedesktop.org/cairo/cairo/issues/266) and I've been using WeasyPrint on multiprocess environments with uWSGI for years, so I suppose it somehow works.

File "/Users/guo/Documents/dev/soda_admin/erp_server/env/lib/python3.6/site-packages/billiard/pool.py"

Just as in #167, you're on macOS. Maybe there's something wrong with Cairo or Pango on macOS…

Thank you for reply.

And I finally solve this problem by using Eventlet concurrency in Celery and it works fine.

I think it crashed because of the multi-process issues in which the default Celery concurrency used. And Mac has a strict protection for multi-process resource sharing.

When I using Eventlet, it use green threads instead of prefork pool and the OS protection for processes is ignored. And the Celery tasks work fine.

Thank you for your help!

[liZe]

Thank you for your help!

No problem.

We can assume that we can't do anything about this … unless some macOS guru comes and explains what is going on. Until then, I think that we can safely close this issue.

[JSv4]

I'm using the latest version of celery (5+) and weasyprint as of September 2021, and I'm seeing a similar issue. I'm getting segmentation fault errors if I try to call new instances of weasyprint in separate celery tasks. I tried every which way to refactor how my tasks were structured to no avail, so I thought to switch from a prefork worker pool to a gevent pool in my celery worker, which has helped me get around threading issues in the past with celery. The segmentation fault went away and I'm now calling weasyprint successfully from my tasks. I'll be honest, like @liZe, I don't really understand why this error is occurring with a prefork pool, but I don't really have the time or appetite to debug it and switching to a gevent pool solved my problem for now. Perhaps this will help someone else.